-next
-- anonymous indexes (via Column(unique=True) etc) use column._label for naming
-to avoid collisions
-0.1.7
-- some fixes to topological sort algorithm
-- added DISTINCT ON support to Postgres (just supply distinct=[col1,col2..])
-- added __mod__ (% operator) to sql expressions
-- "order_by" mapper property inherited from inheriting mapper
-- fix to column type used when mapper UPDATES/DELETEs
-- with convert_unicode=True, reflection was failing, has been fixed
-- types types types! still werent working....have to use TypeDecorator again :(
-- mysql binary type converts array output to buffer, fixes PickleType
-- fixed the attributes.py memory leak once and for all
-- unittests are qualified based on the databases that support each one
-- fixed bug where column defaults would clobber VALUES clause of insert objects
-- fixed bug where table def w/ schema name would force engine connection
-- fix for parenthesis to work correctly with subqueries in INSERT/UPDATE
-- HistoryArraySet gets extend() method
-- fixed lazyload support for other comparison operators besides =
-- lazyload fix where two comparisons in the join condition point to the
-samem column
-- added "construct_new" flag to mapper, will use __new__ to create instances
-instead of __init__ (standard in 0.2)
-- added selectresults.py to SVN, missed it last time
-- tweak to allow a many-to-many relationship from a table to itself via
-an association table
-- small fix to "translate_row" function used by polymorphic example
-- create_engine uses cgi.parse_qsl to read query string (out the window in 0.2)
-- tweaks to CAST operator
-- fixed function names LOCAL_TIME/LOCAL_TIMESTAMP -> LOCALTIME/LOCALTIMESTAMP
-- fixed order of ORDER BY/HAVING in compile
+0.2
+- overhaul to Engine system so that what was formerly the SQLEngine
+is now a ComposedSQLEngine which consists of a variety of components,
+including a Dialect, ConnectionProvider, etc. This impacted all the
+db modules as well as Session and Mapper.
+- create_engine now takes only RFC-1738-style strings:
+driver://user:password@host:port/database
+- total rewrite of connection-scoping methodology, Connection objects
+can now execute clause elements directly, added explicit "close" as
+well as support throughout Engine/ORM to handle closing properly,
+no longer relying upon __del__ internally to return connections
+to the pool [ticket:152].
+- overhaul to Session interface and scoping. uses hibernate-style
+methods, including query(class), save(), save_or_update(), etc.
+no threadlocal scope is installed by default. Provides a binding
+interface to specific Engines and/or Connections so that underlying
+Schema objects do not need to be bound to an Engine. Added a basic
+SessionTransaction object that can simplistically aggregate transactions
+across multiple engines.
+- overhaul to mapper's dependency and "cascade" behavior; dependency logic
+factored out of properties.py into a separate module "dependency.py".
+"cascade" behavior is now explicitly controllable, proper implementation
+of "delete", "delete-orphan", etc. dependency system can now determine at
+flush time if a child object has a parent or not so that it makes better
+decisions on how that child should be updated in the DB with regards to deletes.
+- overhaul to Schema to build upon MetaData object instead of an Engine.
+Entire SQL/Schema system can be used with no Engines whatsoever, executed
+solely by an explicit Connection object. the "bound" methodlogy exists via the
+BoundMetaData for schema objects. ProxyEngine is generally not needed
+anymore and is replaced by DynamicMetaData.
+- true polymorphic behavior implemented, fixes [ticket:167]
+- "oid" system has been totally moved into compile-time behavior;
+if they are used in an order_by where they are not available, the order_by
+doesnt get compiled, fixes [ticket:147]
+- overhaul to packaging; "mapping" is now "orm", "objectstore" is now
+"session", the old "objectstore" namespace gets loaded in via the
+"threadlocal" mod if used
+- mods now called in via "import <modname>". extensions favored over
+mods as mods are globally-monkeypatching
+- fix to add_property so that it propigates properties to inheriting
+mappers [ticket:154]
+- backrefs create themselves against primary mapper of its originating
+property, priamry/secondary join arguments can be specified to override.
+helps their usage with polymorphic mappers
+- "table exists" function has been implemented [ticket:31]
+- "create_all/drop_all" added to MetaData object [ticket:98]
+- improvements and fixes to topological sort algorithm, as well as more
+unit tests
+- tutorial page added to docs which also can be run with a custom doctest
+runner to insure its properly working. docs generally overhauled to
+deal with new code patterns
0.1.6
- support for MS-SQL added courtesy Rick Morrison, Runar Petursson
--- /dev/null
+<html>
+<head>
+ <link href="style.css" rel="stylesheet" type="text/css"></link>
+ <link href="docs.css" rel="stylesheet" type="text/css"></link>
+ <script src="scripts.js"></script>
+ <title>SQLAlchemy Documentation</title>
+</head>
+<body>
+ <h3>What is an Alpha API Feature?</h3>
+<p><b>Alpha API</b> indicates that the best way for a particular feature to be presented hasn't been firmly settled on as of yet, and the current way is being introduced on a trial basis. Its spirit is not as much a warning that "this API might change", its more an invitation to the users saying, "heres a new idea I had. I'm not sure if this is the best way to do it. Do you like it ? Should we do this differently? Or is it good the way it is ?". Alpha API features are always small in scope and are presented in releases so that the greatest number of users get some hands-on experience with it; large-scoped API or architectural changes will always be discussed on the mailing list/Wiki first.</p>
+
+<p>Reasons why a feature might want to change include:
+ <ul>
+ <li>The API for the feature is too difficult to use for the typical task, and needs to be more "convenient"</li>
+ <li>The feature only implements a subsection of what it really should be doing</li>
+ <li>The feature's interface is inconsistent with that of other features which operate at a similar level</li>
+ <li>The feature is confusing and is often misunderstood, and would be better replaced by a more manual feature that makes the task clearer</li>
+ <li>The feature overlaps with another feature and effectively provides too many ways to do the same thing</li>
+ <li>The feature made some assumptions about the total field of use cases which is not really true, and it breaks in other scenarios</li>
+ </ul>
+
+</p>
+<p>A good example of what was essentially an "alpha feature" is the <code>private=True</code> flag. This flag on a <code>relation()</code> indicates that child objects should be deleted along with the parent. After this flag experienced some usage by the SA userbase, some users remarked that a more generic and configurable way was Hibernates <code>cascade="all, delete-orphan"</code>, and also that the term <code>cascade</code> was clearer in purpose than the more ambiguous <code>private</code> keyword, which could be construed as a "private variable".</p>
+
+<center><input type="button" value="close window" onclick="window.close()"></center>
+</body>
+</html>
\ No newline at end of file
--- /dev/null
+<html>
+<head>
+ <link href="style.css" rel="stylesheet" type="text/css"></link>
+ <link href="docs.css" rel="stylesheet" type="text/css"></link>
+ <script src="scripts.js"></script>
+ <title>SQLAlchemy Documentation</title>
+</head>
+<body>
+ <h3>What is an Alpha Implementation Feature?</h3>
+<p><b>Alpha Implementation</b> indicates a feature where developer confidence in its functionality has not yet been firmly established. This typically includes brand new features for which adequate unit tests have not been completed, and/or features whose scope is broad enough that its not clear what additional unit tests might be needed.</p>
+
+<p>Alpha implementation is not meant to discourage the usage of a feature, it is only meant to indicate that some difficulties in getting full functionality from the feature may occur, and to encourage the reporting of these difficulties either via the mailing list or through <a href="http://www.sqlalchemy.org/trac/newticket" target="_blank">submitting a ticket</a>.</p>
+
+<center><input type="button" value="close window" onclick="window.close()"></center>
+</body>
+</html>
\ No newline at end of file
import sqlalchemy.schema as schema
import sqlalchemy.engine as engine
+import sqlalchemy.engine.strategies as strategies
import sqlalchemy.sql as sql
import sqlalchemy.pool as pool
-import sqlalchemy.mapping as mapping
+import sqlalchemy.orm as orm
import sqlalchemy.exceptions as exceptions
import sqlalchemy.ext.proxy as proxy
+import sqlalchemy.ext.sessioncontext as sessioncontext
+import sqlalchemy.mods.threadlocal as threadlocal
objects = []
def make_doc(obj, classes=None, functions=None):
objects.append(docstring.ObjectDoc(obj, classes=classes, functions=functions))
+make_doc(obj=sql, classes=[sql.Engine, sql.AbstractDialect, sql.ClauseParameters, sql.Compiled, sql.ClauseElement, sql.TableClause, sql.ColumnClause])
make_doc(obj=schema)
-make_doc(obj=engine, classes=[engine.SQLSession, engine.SQLEngine, engine.ResultProxy, engine.RowProxy])
-make_doc(obj=sql, classes=[sql.ClauseParameters, sql.Compiled, sql.ClauseElement, sql.TableClause, sql.ColumnClause])
-make_doc(obj=pool, classes=[pool.DBProxy, pool.Pool, pool.QueuePool, pool.SingletonThreadPool])
-make_doc(obj=mapping, classes=[mapping.Mapper, mapping.MapperExtension])
-make_doc(obj=mapping.query, classes=[mapping.query.Query])
-make_doc(obj=mapping.objectstore, classes=[mapping.objectstore.Session, mapping.objectstore.Session.SessionTrans])
+make_doc(obj=engine, classes=[engine.ComposedSQLEngine, engine.Connection, engine.Transaction, engine.Dialect, engine.ConnectionProvider, engine.ExecutionContext, engine.ResultProxy, engine.RowProxy])
+make_doc(obj=strategies)
+make_doc(obj=orm, classes=[orm.Mapper, orm.MapperExtension])
+make_doc(obj=orm.query, classes=[orm.query.Query])
+make_doc(obj=orm.session, classes=[orm.session.Session, orm.session.SessionTransaction])
+make_doc(obj=sessioncontext)
+make_doc(obj=threadlocal)
make_doc(obj=exceptions)
+make_doc(obj=pool, classes=[pool.DBProxy, pool.Pool, pool.QueuePool, pool.SingletonThreadPool])
make_doc(obj=proxy)
+
output = os.path.join(os.getcwd(), 'content', "compiled_docstrings.pickle")
-pickle.dump(objects, file(output, 'w'))
\ No newline at end of file
+pickle.dump(objects, file(output, 'w'))
title = None
syntaxtype = 'python'
html_escape = False
+ use_sliders = False
</%args>
<%init>
return "<pre>" + highlight.highlight(fix_indent(match.group(1)), html_escape = html_escape, syntaxtype = syntaxtype) + "</pre>"
content = p.sub(hlight, "<pre>" + m.content() + "</pre>")
</%init>
-<div class="code">
+<div class="<% use_sliders and "sliding_code" or "code" %>">
% if title is not None:
<div class="codetitle"><% title %></div>
%
+++ /dev/null
-<%flags>inherit='document_base.myt'</%flags>
-<%attr>title='Advanced Data Mapping'</%attr>
-<&|doclib.myt:item, name="adv_datamapping", description="Advanced Data Mapping" &>
-<p>This section details all the options available to Mappers, as well as advanced patterns.</p>
-
-<p>To start, heres the tables we will work with again:</p>
- <&|formatting.myt:code&>
- from sqlalchemy import *
- db = create_engine('sqlite://filename=mydb', echo=True)
-
- # a table to store users
- users = Table('users', db,
- Column('user_id', Integer, primary_key = True),
- Column('user_name', String(40)),
- Column('password', String(80))
- )
-
- # a table that stores mailing addresses associated with a specific user
- addresses = Table('addresses', db,
- Column('address_id', Integer, primary_key = True),
- Column('user_id', Integer, ForeignKey("users.user_id")),
- Column('street', String(100)),
- Column('city', String(80)),
- Column('state', String(2)),
- Column('zip', String(10))
- )
-
- # a table that stores keywords
- keywords = Table('keywords', db,
- Column('keyword_id', Integer, primary_key = True),
- Column('name', VARCHAR(50))
- )
-
- # a table that associates keywords with users
- userkeywords = Table('userkeywords', db,
- Column('user_id', INT, ForeignKey("users")),
- Column('keyword_id', INT, ForeignKey("keywords"))
- )
-
- </&>
-
-<&|doclib.myt:item, name="relations", description="More On Relations" &>
- <&|doclib.myt:item, name="customjoin", description="Custom Join Conditions" &>
- <p>When creating relations on a mapper, most examples so far have illustrated the mapper and relationship joining up based on the foreign keys of the tables they represent. in fact, this "automatic" inspection can be completely circumvented using the <span class="codeline">primaryjoin</span> and <span class="codeline">secondaryjoin</span> arguments to <span class="codeline">relation</span>, as in this example which creates a User object which has a relationship to all of its Addresses which are in Boston:
- <&|formatting.myt:code&>
- class User(object):
- pass
- class Address(object):
- pass
- Address.mapper = mapper(Address, addresses)
- User.mapper = mapper(User, users, properties={
- 'boston_addreses' : relation(Address.mapper, primaryjoin=
- and_(users.c.user_id==Address.c.user_id,
- Addresses.c.city=='Boston'))
- })
- </&>
- <P>Many to many relationships can be customized by one or both of <span class="codeline">primaryjoin</span> and <span class="codeline">secondaryjoin</span>, shown below with just the default many-to-many relationship explicitly set:</p>
- <&|formatting.myt:code&>
- class User(object):
- pass
- class Keyword(object):
- pass
- Keyword.mapper = mapper(Keyword, keywords)
- User.mapper = mapper(User, users, properties={
- 'keywords':relation(Keyword.mapper,
- primaryjoin=users.c.user_id==userkeywords.c.user_id,
- secondaryjoin=userkeywords.c.keyword_id==keywords.c.keyword_id
- )
- })
- </&>
- </&>
- <&|doclib.myt:item, name="multiplejoin", description="Lazy/Eager Joins Multiple Times to One Table" &>
-
- <p>The previous example leads in to the idea of joining against the same table multiple times. Below is a User object that has lists of its Boston and New York addresses, both lazily loaded when they are first accessed:</p>
- <&|formatting.myt:code&>
- User.mapper = mapper(User, users, properties={
- 'boston_addreses' : relation(Address.mapper, primaryjoin=
- and_(users.c.user_id==Address.c.user_id,
- Addresses.c.city=='Boston')),
- 'newyork_addresses' : relation(Address.mapper, primaryjoin=
- and_(users.c.user_id==Address.c.user_id,
- Addresses.c.city=='New York')),
- })
- </&>
- <p>A complication arises with the above pattern if you want the relations to be eager loaded. Since there will be two separate joins to the addresses table during an eager load, an alias needs to be used to separate them. You can create an alias of the addresses table to separate them, but then you are in effect creating a brand new mapper for each property, unrelated to the main Address mapper, which can create problems with commit operations. So an additional argument <span class="codeline">use_alias</span> can be used with an eager relationship to specify the alias to be used just within the eager query:</p>
- <&|formatting.myt:code&>
- User.mapper = mapper(User, users, properties={
- 'boston_addreses' : relation(Address.mapper, primaryjoin=
- and_(User.c.user_id==Address.c.user_id,
- Addresses.c.city=='Boston'), lazy=False, use_alias=True),
- 'newyork_addresses' : relation(Address.mapper, primaryjoin=
- and_(User.c.user_id==Address.c.user_id,
- Addresses.c.city=='New York'), lazy=False, use_alias=True),
- })
-
- <&formatting.myt:poplink&>u = User.mapper.select()
-
- <&|formatting.myt:codepopper, link="sql" &>
- SELECT users.user_id AS users_user_id, users.user_name AS users_user_name,
- users.password AS users_password,
- addresses_EF45.address_id AS addresses_EF45_address_id, addresses_EF45.user_id AS addresses_EF45_user_id,
- addresses_EF45.street AS addresses_EF45_street, addresses_EF45.city AS addresses_EF45_city,
- addresses_EF45.state AS addresses_EF45_state, addresses_EF45.zip AS addresses_EF45_zip,
- addresses_63C5.address_id AS addresses_63C5_address_id, addresses_63C5.user_id AS addresses_63C5_user_id,
- addresses_63C5.street AS addresses_63C5_street, addresses_63C5.city AS addresses_63C5_city,
- addresses_63C5.state AS addresses_63C5_state, addresses_63C5.zip AS addresses_63C5_zip
- FROM users
- LEFT OUTER JOIN addresses AS addresses_EF45 ON users.user_id = addresses_EF45.user_id
- AND addresses_EF45.city = :addresses_city
- LEFT OUTER JOIN addresses AS addresses_63C5 ON users.user_id = addresses_63C5.user_id
- AND addresses_63C5.city = :addresses_city_1
- ORDER BY users.oid, addresses_EF45.oid, addresses_63C5.oid
- {'addresses_city_1': 'New York', 'addresses_city': 'Boston'}
- </&>
- </&>
- </&>
-
- <&|doclib.myt:item, name="relationoptions", description="Relation Options" &>
- Keyword options to the <span class="codeline">relation</span> function include:
- <ul>
- <li>lazy=(True|False|None) - specifies how the related items should be loaded. a value of True indicates they should be loaded when the property is first accessed. A value of False indicates they should be loaded by joining against the parent object query, so parent and child are loaded in one round trip. A value of None indicates the related items are not loaded by the mapper in any case; the application will manually insert items into the list in some other way. A relationship with lazy=None is still important; items added to the list or removed will cause the appropriate updates and deletes upon commit().</li>
- <li>primaryjoin - a ClauseElement that will be used as the primary join of this child object against the parent object, or in a many-to-many relationship the join of the primary object to the association table. By default, this value is computed based on the foreign key relationships of the parent and child tables (or association table).</li>
- <li>secondaryjoin - a ClauseElement that will be used as the join of an association table to the child object. By default, this value is computed based on the foreign key relationships of the association and child tables.</li>
- <li>foreignkey - specifies which column in this relationship is "foreign", i.e. which column refers to the parent object. This value is automatically determined in all cases, based on the primary and secondary join conditions, except in the case of a self-referential mapper, where it is needed to indicate the child object's reference back to it's parent.</li>
- <li>uselist - a boolean that indicates if this property should be loaded as a list or a scalar. In most cases, this value is determined based on the type and direction of the relationship - one to many forms a list, one to one forms a scalar, many to many is a list. If a scalar is desired where normally a list would be present, set uselist to False.</li>
- <li>private - indicates if these child objects are "private" to the parent; removed items will also be deleted, and if the parent item is deleted, all child objects are deleted as well. See the example in <&formatting.myt:link, path="datamapping_relations_private"&>.</li>
- <li>backreference - indicates the name of a property to be placed on the related mapper's class that will handle this relationship in the other direction, including synchronizing the object attributes on both sides of the relation. See the example in <&formatting.myt:link, path="datamapping_relations_backreferences"&>.</li>
- <li>order_by - indicates the ordering that should be applied when loading these items. See the section <&formatting.myt:link, path="adv_datamapping_orderby" &> for details.</li>
- <li>association - When specifying a many to many relationship with an association object, this keyword should reference the mapper of the target object of the association. See the example in <&formatting.myt:link, path="datamapping_association"&>.</li>
- <li>post_update - this indicates that the relationship should be handled by a second UPDATE statement after an INSERT, or before a DELETE. using this flag essentially means the relationship will not incur any "dependency" between parent and child item, as the particular foreign key relationship between them is handled by a second statement. use this flag when a particular mapping arrangement will incur two rows that are dependent on each other, such as a table that has a one-to-many relationship to a set of child rows, and also has a column that references a single child row within that list (i.e. both tables contain a foreign key to each other). If a commit() operation returns an error that a "cyclical dependency" was detected, this is a cue that you might want to use post_update.</li>
- </ul>
- </&>
-
-</&>
-<&|doclib.myt:item, name="orderby", description="Controlling Ordering" &>
-<p>By default, mappers will not supply any ORDER BY clause when selecting rows. This can be modified in several ways.</p>
-
-<p>A "default ordering" can be supplied by all mappers, by enabling the "default_ordering" flag to the engine, which indicates that table primary keys or object IDs should be used as the default ordering:</p>
-<&|formatting.myt:code&>
- db = create_engine('postgres://username=scott&password=tiger', default_ordering=True)
-</&>
-<p>The "order_by" parameter can be sent to a mapper, overriding the per-engine ordering if any. A value of None means that the mapper should not use any ordering, even if the engine's default_ordering property is True. A non-None value, which can be a column, an <span class="codeline">asc</span> or <span class="codeline">desc</span> clause, or an array of either one, indicates the ORDER BY clause that should be added to all select queries:</p>
-<&|formatting.myt:code&>
- # disable all ordering
- mapper = mapper(User, users, order_by=None)
-
- # order by a column
- mapper = mapper(User, users, order_by=users.c.user_id)
-
- # order by multiple items
- mapper = mapper(User, users, order_by=[users.c.user_id, desc(users.c.user_name)])
-</&>
-<p>"order_by" can also be specified to an individual <span class="codeline">select</span> method, overriding all other per-engine/per-mapper orderings:
-<&|formatting.myt:code&>
- # order by a column
- l = mapper.select(users.c.user_name=='fred', order_by=users.c.user_id)
-
- # order by multiple criterion
- l = mapper.select(users.c.user_name=='fred', order_by=[users.c.user_id, desc(users.c.user_name)])
-</&>
-<p>For relations, the "order_by" property can also be specified to all forms of relation:</p>
-<&|formatting.myt:code&>
- # order address objects by address id
- mapper = mapper(User, users, properties = {
- 'addresses' : relation(mapper(Address, addresses), order_by=addresses.c.address_id)
- })
-
- # eager load with ordering - the ORDER BY clauses of parent/child will be organized properly
- mapper = mapper(User, users, properties = {
- 'addresses' : relation(mapper(Address, addresses), order_by=desc(addresses.c.email_address), eager=True)
- }, order_by=users.c.user_id)
-
-</&>
-</&>
-<&|doclib.myt:item, name="limits", description="Limiting Rows" &>
-<p>You can limit rows in a regular SQL query by specifying <span class="codeline">limit</span> and <span class="codeline">offset</span>. A Mapper can handle the same concepts:</p>
-<&|formatting.myt:code&>
- class User(object):
- pass
-
- m = mapper(User, users)
-<&formatting.myt:poplink&>r = m.select(limit=20, offset=10)
-<&|formatting.myt:codepopper, link="sql" &>SELECT users.user_id AS users_user_id,
-users.user_name AS users_user_name, users.password AS users_password
-FROM users ORDER BY users.oid
- LIMIT 20 OFFSET 10
-{}
-</&>
-</&>
-However, things get tricky when dealing with eager relationships, since a straight LIMIT of rows does not represent the count of items when joining against other tables to load related items as well. So here is what SQLAlchemy will do when you use limit or offset with an eager relationship:
- <&|formatting.myt:code&>
- class User(object):
- pass
- class Address(object):
- pass
- m = mapper(User, users, properties={
- 'addresses' : relation(mapper(Address, addresses), lazy=False)
- })
- r = m.select(User.c.user_name.like('F%'), limit=20, offset=10)
-<&|formatting.myt:poppedcode, link="sql" &>
-SELECT users.user_id AS users_user_id, users.user_name AS users_user_name,
-users.password AS users_password, addresses.address_id AS addresses_address_id,
-addresses.user_id AS addresses_user_id, addresses.street AS addresses_street,
-addresses.city AS addresses_city, addresses.state AS addresses_state,
-addresses.zip AS addresses_zip
-FROM
-(SELECT users.user_id FROM users WHERE users.user_name LIKE %(users_user_name)s
-ORDER BY users.oid LIMIT 20 OFFSET 10) AS rowcount,
- users LEFT OUTER JOIN addresses ON users.user_id = addresses.user_id
-WHERE rowcount.user_id = users.user_id ORDER BY users.oid, addresses.oid
-{'users_user_name': 'F%'}
-
- </&>
- </&>
- <p>The main WHERE clause as well as the limiting clauses are coerced into a subquery; this subquery represents the desired result of objects. A containing query, which handles the eager relationships, is joined against the subquery to produce the result.</p>
-</&>
-<&|doclib.myt:item, name="colname", description="Overriding Column Names" &>
-<p>When mappers are constructed, by default the column names in the Table metadata are used as the names of attributes on the mapped class. This can be customzed within the properties by stating the key/column combinations explicitly:</p>
-<&|formatting.myt:code&>
- user_mapper = mapper(User, users, properties={
- 'id' : users.c.user_id,
- 'name' : users.c.user_name,
- })
-</&>
-<p>In the situation when column names overlap in a mapper against multiple tables, columns may be referenced together with a list:
-<&|formatting.myt:code&>
- # join users and addresses
- usersaddresses = sql.join(users, addresses, users.c.user_id == addresses.c.user_id)
- m = mapper(User, usersaddresses,
- properties = {
- 'id' : [users.c.user_id, addresses.c.user_id],
- }
- )
-</&>
-</&>
-<&|doclib.myt:item, name="deferred", description="Deferred Column Loading" &>
-<p>This feature allows particular columns of a table to not be loaded by default, instead being loaded later on when first referenced. It is essentailly "column-level lazy loading". This feature is useful when one wants to avoid loading a large text or binary field into memory when its not needed. Individual columns can be lazy loaded by themselves or placed into groups that lazy-load together.</p>
-<&|formatting.myt:code&>
- book_excerpts = Table('books', db,
- Column('book_id', Integer, primary_key=True),
- Column('title', String(200), nullable=False),
- Column('summary', String(2000)),
- Column('excerpt', String),
- Column('photo', Binary)
- )
-
- class Book(object):
- pass
-
- # define a mapper that will load each of 'excerpt' and 'photo' in
- # separate, individual-row SELECT statements when each attribute
- # is first referenced on the individual object instance
- book_mapper = mapper(Book, book_excerpts, properties = {
- 'excerpt' : deferred(book_excerpts.c.excerpt),
- 'photo' : deferred(book_excerpts.c.photo)
- })
-</&>
-<p>Deferred columns can be placed into groups so that they load together:</p>
-<&|formatting.myt:code&>
- book_excerpts = Table('books', db,
- Column('book_id', Integer, primary_key=True),
- Column('title', String(200), nullable=False),
- Column('summary', String(2000)),
- Column('excerpt', String),
- Column('photo1', Binary),
- Column('photo2', Binary),
- Column('photo3', Binary)
- )
-
- class Book(object):
- pass
-
- # define a mapper with a 'photos' deferred group. when one photo is referenced,
- # all three photos will be loaded in one SELECT statement. The 'excerpt' will
- # be loaded separately when it is first referenced.
- book_mapper = mapper(Book, book_excerpts, properties = {
- 'excerpt' : deferred(book_excerpts.c.excerpt),
- 'photo1' : deferred(book_excerpts.c.photo1, group='photos'),
- 'photo2' : deferred(book_excerpts.c.photo2, group='photos'),
- 'photo3' : deferred(book_excerpts.c.photo3, group='photos')
- })
-</&>
-</&>
-<&|doclib.myt:item, name="options", description="More on Mapper Options" &>
- <p>The <span class="codeline">options</span> method of mapper, first introduced in <&formatting.myt:link, path="datamapping_relations_options" &>, supports the copying of a mapper into a new one, with any number of its relations replaced by new ones. The method takes a variable number of <span class="codeline">MapperOption</span> objects which know how to change specific things about the mapper. The five available options are <span class="codeline">eagerload</span>, <span class="codeline">lazyload</span>, <span class="codeline">noload</span>, <span class="codeline">deferred</span> and <span class="codeline">extension</span>.</p>
- <P>An example of a mapper with a lazy load relationship, upgraded to an eager load relationship:
- <&|formatting.myt:code&>
- class User(object):
- pass
- class Address(object):
- pass
-
- # a 'lazy' relationship
- User.mapper = mapper(User, users, properties = {
- 'addreses':relation(mapper(Address, addresses), lazy=True)
- })
-
- # copy the mapper and convert 'addresses' to be eager
- eagermapper = User.mapper.options(eagerload('addresses'))
- </&>
-
- <p>The load options also can take keyword arguments that apply to the new relationship. To take the "double" address lazy relationship from the previous section and upgrade it to eager, adding the "selectalias" keywords as well:</p>
- <&|formatting.myt:code&>
- m = User.mapper.options(
- eagerload('boston_addresses', selectalias='boston_ad'),
- eagerload('newyork_addresses', selectalias='newyork_ad')
- )
- </&>
- <p>The <span class="codeline">defer</span> and <span class="codeline">undefer</span> options can control the deferred loading of attributes:</p>
- <&|formatting.myt:code&>
- # set the 'excerpt' deferred attribute to load normally
- m = book_mapper.options(undefer('excerpt'))
-
- # set the referenced mapper 'photos' to defer its loading of the column 'imagedata'
- m = book_mapper.options(defer('photos.imagedata'))
- </&>
- <p>Options can also take a limited set of keyword arguments which will be applied to a new mapper. For example, to create a mapper that refreshes all objects loaded each time:</p>
- <&|formatting.myt:code&>
- m2 = mapper.options(always_refresh=True)
- </&>
- <p>Or, a mapper with different ordering:</p>
- <&|formatting.myt:code&>
- m2 = mapper.options(order_by=[newcol])
- </&>
-
-</&>
-
-
-<&|doclib.myt:item, name="inheritance", description="Mapping a Class with Table Inheritance" &>
-
- <p>Table Inheritance indicates the pattern where two tables, in a parent-child relationship, are mapped to an inheritance chain of classes. If a table "employees" contains additional information about managers in the table "managers", a corresponding object inheritance pattern would have an Employee class and a Manager class. Loading a Manager object means you are joining managers to employees. For SQLAlchemy, this pattern is just a special case of a mapper that maps against a joined relationship, and is provided via the <span class="codeline">inherits</span> keyword.
- <&|formatting.myt:code&>
- class User(object):
- """a user object."""
- pass
- User.mapper = mapper(User, users)
-
- class AddressUser(User):
- """a user object that also has the users mailing address."""
- pass
-
- # define a mapper for AddressUser that inherits the User.mapper, and joins on the user_id column
- AddressUser.mapper = mapper(
- AddressUser,
- addresses, inherits=User.mapper
- )
-
- items = AddressUser.mapper.select()
- </&>
-<P>Above, the join condition is determined via the foreign keys between the users and the addresses table. To specify the join condition explicitly, use <span class="codeline">inherit_condition</span>:
-<&|formatting.myt:code&>
- AddressUser.mapper = mapper(
- AddressUser,
- addresses, inherits=User.mapper,
- inherit_condition=users.c.user_id==addresses.c.user_id
- )
-</&>
-</&>
-
-<&|doclib.myt:item, name="joins", description="Mapping a Class against Multiple Tables" &>
- <P>The more general case of the pattern described in "table inheritance" is a mapper that maps against more than one table. The <span class="codeline">join</span> keyword from the SQL package creates a neat selectable unit comprised of multiple tables, complete with its own composite primary key, which can be passed in to a mapper as the table.</p>
- <&|formatting.myt:code&>
- # a class
- class AddressUser(object):
- pass
-
- # define a Join
- j = join(users, addresses)
-
- # map to it - the identity of an AddressUser object will be
- # based on (user_id, address_id) since those are the primary keys involved
- m = mapper(AddressUser, j)
- </&>
-
- A second example:
- <&|formatting.myt:code&>
- # many-to-many join on an association table
- j = join(users, userkeywords,
- users.c.user_id==userkeywords.c.user_id).join(keywords,
- userkeywords.c.keyword_id==keywords.c.keyword_id)
-
- # a class
- class KeywordUser(object):
- pass
-
- # map to it - the identity of a KeywordUser object will be
- # (user_id, keyword_id) since those are the primary keys involved
- m = mapper(KeywordUser, j)
- </&>
-</&>
-<&|doclib.myt:item, name="selects", description="Mapping a Class against Arbitary Selects" &>
-<p>Similar to mapping against a join, a plain select() object can be used with a mapper as well. Below, an example select which contains two aggregate functions and a group_by is mapped to a class:</p>
- <&|formatting.myt:code&>
- s = select([customers,
- func.count(orders).label('order_count'),
- func.max(orders.price).label('highest_order')],
- customers.c.customer_id==orders.c.customer_id,
- group_by=[c for c in customers.c]
- )
- class Customer(object):
- pass
-
- mapper = mapper(Customer, s)
- </&>
-<p>Above, the "customers" table is joined against the "orders" table to produce a full row for each customer row, the total count of related rows in the "orders" table, and the highest price in the "orders" table, grouped against the full set of columns in the "customers" table. That query is then mapped against the Customer class. New instances of Customer will contain attributes for each column in the "customers" table as well as an "order_count" and "highest_order" attribute. Updates to the Customer object will only be reflected in the "customers" table and not the "orders" table. This is because the primary keys of the "orders" table are not represented in this mapper and therefore the table is not affected by save or delete operations.</p>
-</&>
-<&|doclib.myt:item, name="multiple", description="Multiple Mappers for One Class" &>
- <p>By now it should be apparent that the mapper defined for a class is in no way the only mapper that exists for that class. Other mappers can be created at any time; either explicitly or via the <span class="codeline">options</span> method, to provide different loading behavior.</p>
-
- <p>However, its not as simple as that. The mapper serves a dual purpose; one is to generate select statements and load objects from executing those statements; the other is to keep track of the defined dependencies of that object when save and delete operations occur, and to extend the attributes of the object so that they store information about their history and communicate with the unit of work system. For this reason, it is a good idea to be aware of the behavior of multiple mappers. When creating dependency relationships between objects, one should insure that only the primary mappers are used in those relationships, else deep object traversal operations will fail to load in the expected properties, and update operations will not take all the dependencies into account. </p>
-
- <p>Generally its as simple as, the <i>first</i> mapper that is defined for a particular class is the one that gets to define that classes' relationships to other mapped classes, and also decorates its attributes and constructors with special behavior. Any subsequent mappers created for that class will be able to load new instances, but object manipulation operations will still function via the original mapper. The special keyword <span class="codeline">is_primary</span> will override this behavior, and make any mapper the new "primary" mapper.
- </p>
- <&|formatting.myt:code&>
- class User(object):
- pass
-
- # mapper one - mark it as "primary", meaning this mapper will handle
- # saving and class-level properties
- m1 = mapper(User, users, is_primary=True)
-
- # mapper two - this one will also eager-load address objects in
- m2 = mapper(User, users, properties={
- 'addresses' : relation(mapper(Address, addresses), lazy=False)
- })
-
- # get a user. this user will not have an 'addreses' property
- u1 = m1.select(User.c.user_id==10)
-
- # get another user. this user will have an 'addreses' property.
- u2 = m2.select(User.c.user_id==27)
-
- # make some modifications, including adding an Address object.
- u1.user_name = 'jack'
- u2.user_name = 'jane'
- u2.addresses.append(Address('123 green street'))
-
- # upon commit, the User objects will be saved.
- # the Address object will not, since the primary mapper for User
- # does not have an 'addresses' relationship defined
- objectstore.commit()
- </&>
-</&>
-<&|doclib.myt:item, name="circular", description="Circular Mapping" &>
-<p>Oftentimes it is necessary for two mappers to be related to each other. With a datamodel that consists of Users that store Addresses, you might have an Address object and want to access the "user" attribute on it, or have a User object and want to get the list of Address objects. The easiest way to do this is via the <span class="codeline">backreference</span> keyword described in <&formatting.myt:link, path="datamapping_relations_backreferences"&>. Although even when backreferences are used, it is sometimes necessary to explicitly specify the relations on both mappers pointing to each other.</p>
-<p>To achieve this involves creating the first mapper by itself, then creating the second mapper referencing the first, then adding references to the first mapper to reference the second:</p>
-<&|formatting.myt:code&>
- class User(object):
- pass
- class Address(object):
- pass
- User.mapper = mapper(User, users)
- Address.mapper = mapper(Address, addresses, properties={
- 'user':relation(User.mapper)
- })
- User.mapper.add_property('addresses', relation(Address.mapper))
-</&>
-<p>Note that with a circular relationship as above, you cannot declare both relationships as "eager" relationships, since that produces a circular query situation which will generate a recursion exception. So what if you want to load an Address and its User eagerly? Just make a second mapper using options:
-<&|formatting.myt:code&>
- eagermapper = Address.mapper.options(eagerload('user'))
- s = eagermapper.select(Address.c.address_id==12)
-</&>
-</&>
-<&|doclib.myt:item, name="recursive", description="Self Referential Mappers" &>
-<p>A self-referential mapper is a mapper that is designed to operate with an <b>adjacency list</b> table. This is a table that contains one or more foreign keys back to itself, and is usually used to create hierarchical tree structures. SQLAlchemy's default model of saving items based on table dependencies is not sufficient in this case, as an adjacency list table introduces dependencies between individual rows. Fortunately, SQLAlchemy will automatically detect a self-referential mapper and do the extra lifting to make it work. </p>
- <&|formatting.myt:code&>
- # define a self-referential table
- trees = Table('treenodes', engine,
- Column('node_id', Integer, primary_key=True),
- Column('parent_node_id', Integer, ForeignKey('treenodes.node_id'), nullable=True),
- Column('node_name', String(50), nullable=False),
- )
-
- # treenode class
- class TreeNode(object):
- pass
-
- # mapper defines "children" property, pointing back to TreeNode class,
- # with the mapper unspecified. it will point back to the primary
- # mapper on the TreeNode class.
- TreeNode.mapper = mapper(TreeNode, trees, properties={
- 'children' : relation(
- TreeNode,
- private=True
- ),
- }
- )
-
- # or, specify the circular relationship after establishing the original mapper:
- mymapper = mapper(TreeNode, trees)
-
- mymapper.add_property('children', relation(
- mymapper,
- private=True
- ))
-
- </&>
- <p>This kind of mapper goes through a lot of extra effort when saving and deleting items, to determine the correct dependency graph of nodes within the tree.</p>
-
- <p>A self-referential mapper where there is more than one relationship on the table requires that all join conditions be explicitly spelled out. Below is a self-referring table that contains a "parent_node_id" column to reference parent/child relationships, and a "root_node_id" column which points child nodes back to the ultimate root node:</p>
- <&|formatting.myt:code&>
- # define a self-referential table with several relations
- trees = Table('treenodes', engine,
- Column('node_id', Integer, primary_key=True),
- Column('parent_node_id', Integer, ForeignKey('treenodes.node_id'), nullable=True),
- Column('root_node_id', Integer, ForeignKey('treenodes.node_id'), nullable=True),
- Column('node_name', String(50), nullable=False),
- )
-
- # treenode class
- class TreeNode(object):
- pass
-
- # define the "children" property as well as the "root" property
- TreeNode.mapper = mapper(TreeNode, trees, properties={
- 'children' : relation(
- TreeNode,
- primaryjoin=trees.c.parent_node_id==trees.c.node_id
- private=True
- ),
- 'root' : relation(
- TreeNode,
- primaryjoin=trees.c.root_node_id=trees.c.node_id,
- foreignkey=trees.c.node_id,
- uselist=False
- )
- }
- )
- </&>
-<p>The "root" property on a TreeNode is a many-to-one relationship. By default, a self-referential mapper declares relationships as one-to-many, so the extra parameter <span class="codeline">foreignkey</span>, pointing to the "many" side of a relationship, is needed to indicate a "many-to-one" self-referring relationship.</p>
-<p>Both TreeNode examples above are available in functional form in the <span class="codeline">examples/adjacencytree</span> directory of the distribution.</p>
-</&>
-<&|doclib.myt:item, name="resultset", description="Result-Set Mapping" &>
- <p>Take any result set and feed it into a mapper to produce objects. Multiple mappers can be combined to retrieve unrelated objects from the same row in one step. The <span class="codeline">instances</span> method on mapper takes a ResultProxy object, which is the result type generated from SQLEngine, and delivers object instances.</p>
- <&|formatting.myt:code, title="single object"&>
- class User(object):
- pass
-
- User.mapper = mapper(User, users)
-
- # select users
- c = users.select().execute()
-
- # get objects
- userlist = User.mapper.instances(c)
- </&>
-
- <&|formatting.myt:code, title="multiple objects"&>
- # define a second class/mapper
- class Address(object):
- pass
-
- Address.mapper = mapper(Address, addresses)
-
- # select users and addresses in one query
- s = select([users, addresses], users.c.user_id==addresses.c.user_id)
-
- # execute it, and process the results with the User mapper, chained to the Address mapper
- r = User.mapper.instances(s.execute(), Address.mapper)
-
- # result rows are an array of objects, one for each mapper used
- for entry in r:
- user = r[0]
- address = r[1]
- </&>
-</&>
-<&|doclib.myt:item, name="arguments", description="Mapper Arguments" &>
-<p>Other arguments not covered above include:</p>
-<ul>
- <li>version_id_col=None - an integer-holding Column object that will be assigned an incrementing
- counter, which is added to the WHERE clause used by UPDATE and DELETE statements. The matching row
- count returned by the database is compared to the expected row count, and an exception is raised if they dont match. This is a basic "optimistic concurrency" check. Without the version id column, SQLAlchemy still compares the updated rowcount.</li>
- <li>always_refresh=False - this option will cause the mapper to refresh all the attributes of all objects loaded by select/get statements, regardless of if they already exist in the current session. this includes all lazy- and eager-loaded relationship attributes, and will also overwrite any changes made to attributes on the column.</li>
- <li>entity_name=None - this is an optional "entity name" that will be appended to the key used to associate classes to this mapper. What this basically means is, several primary mappers can be made against the same class by using different entity names; object instances will have the entity name tagged to them, so that all operations will occur on them relative to that mapper. When instantiating new objects, use <code>_sa_entity='name'</code> to tag them to the appropriate mapper.</li>
-</ul>
-</&>
-<&|doclib.myt:item, name="extending", description="Extending Mapper" &>
-<p>Mappers can have functionality augmented or replaced at many points in its execution via the usage of the MapperExtension class. This class is just a series of "hooks" where various functionality takes place. An application can make its own MapperExtension objects, overriding only the methods it needs.
- <&|formatting.myt:code&>
- class MapperExtension(object):
- def create_instance(self, mapper, row, imap, class_):
- """called when a new object instance is about to be created from a row.
- the method can choose to create the instance itself, or it can return
- None to indicate normal object creation should take place.
-
- mapper - the mapper doing the operation
- row - the result row from the database
- imap - a dictionary that is storing the running set of objects collected from the
- current result set
- class_ - the class we are mapping.
- """
- def append_result(self, mapper, row, imap, result, instance, isnew, populate_existing=False):
- """called when an object instance is being appended to a result list.
-
- If it returns True, it is assumed that this method handled the appending itself.
-
- mapper - the mapper doing the operation
- row - the result row from the database
- imap - a dictionary that is storing the running set of objects collected from the
- current result set
- result - an instance of util.HistoryArraySet(), which may be an attribute on an
- object if this is a related object load (lazy or eager). use result.append_nohistory(value)
- to append objects to this list.
- instance - the object instance to be appended to the result
- isnew - indicates if this is the first time we have seen this object instance in the current result
- set. if you are selecting from a join, such as an eager load, you might see the same object instance
- many times in the same result set.
- populate_existing - usually False, indicates if object instances that were already in the main
- identity map, i.e. were loaded by a previous select(), get their attributes overwritten
- """
- def before_insert(self, mapper, instance):
- """called before an object instance is INSERTed into its table.
-
- this is a good place to set up primary key values and such that arent handled otherwise."""
- def after_insert(self, mapper, instance):
- """called after an object instance has been INSERTed"""
- def before_delete(self, mapper, instance):
- """called before an object instance is DELETEed"""
-
- </&>
- <p>To use MapperExtension, make your own subclass of it and just send it off to a mapper:</p>
- <&|formatting.myt:code&>
- mapper = mapper(User, users, extension=MyExtension())
- </&>
- <p>An existing mapper can create a copy of itself using an extension via the <span class="codeline">extension</span> option:
- <&|formatting.myt:code&>
- extended_mapper = mapper.options(extension(MyExtension()))
- </&>
-
-</&>
-<&|doclib.myt:item, name="class", description="How Mapper Modifies Mapped Classes" &>
-<p>This section is a quick summary of what's going on when you send a class to the <span class="codeline">mapper()</span> function. This material, not required to be able to use SQLAlchemy, is a little more dense and should be approached patiently!</p>
-
-<p>The primary changes to a class that is mapped involve attaching property objects to it which represent table columns. These property objects essentially track changes. In addition, the __init__ method of the object is decorated to track object creates.</p>
-<p>Here is a quick rundown of all the changes in code form:
- <&|formatting.myt:code&>
- # step 1 - override __init__ to 'register_new' with the Unit of Work
- oldinit = myclass.__init__
- def init(self, *args, **kwargs):
- nohist = kwargs.pop('_mapper_nohistory', False)
- oldinit(self, *args, **kwargs)
- if not nohist:
- # register_new with Unit Of Work
- objectstore.uow().register_new(self)
- myclass.__init__ = init
-
- # step 2 - set a string identifier that will
- # locate the classes' primary mapper
- myclass._mapper = mapper.hashkey
-
- # step 3 - add column accessor
- myclass.c = mapper.columns
-
- # step 4 - attribute decorating.
- # this happens mostly within the package sqlalchemy.attributes
-
- # this dictionary will store a series of callables
- # that generate "history" containers for
- # individual object attributes
- myclass._class_managed_attributes = {}
-
- # create individual properties for each column -
- # these objects know how to talk
- # to the attribute package to create appropriate behavior.
- # the next example examines the attributes package more closely.
- myclass.column1 = SmartProperty().property('column1', uselist=False)
- myclass.column2 = SmartProperty().property('column2', uselist=True)
- </&>
-<p>The attribute package is used when save operations occur to get a handle on modified values. In the example below,
-a full round-trip attribute tracking operation is illustrated:</p>
-<&|formatting.myt:code&>
- import sqlalchemy.attributes as attributes
-
- # create an attribute manager.
- # the sqlalchemy.mapping package keeps one of these around as
- # 'objectstore.global_attributes'
- manager = attributes.AttributeManager()
-
- # regular old new-style class
- class MyClass(object):
- pass
-
- # register a scalar and a list attribute
- manager.register_attribute(MyClass, 'column1', uselist=False)
- manager.register_attribute(MyClass, 'column2', uselist=True)
-
- # create/modify an object
- obj = MyClass()
- obj.column1 = 'this is a new value'
- obj.column2.append('value 1')
- obj.column2.append('value 2')
-
- # get history objects
- col1_history = manager.get_history(obj, 'column1')
- col2_history = manager.get_history(obj, 'column2')
-
- # whats new ?
- >>> col1_history.added_items()
- ['this is a new value']
-
- >>> col2_history.added_items()
- ['value1', 'value2']
-
- # commit changes
- manager.commit(obj)
-
- # the new values become the "unchanged" values
- >>> col1_history.added_items()
- []
-
- >>> col1_history.unchanged_items()
- ['this is a new value']
-
- >>> col2_history.added_items()
- []
-
- >>> col2_history.unchanged_items()
- ['value1', 'value2']
-</&>
-<p>The above AttributeManager also includes a method <span class="codeline">value_changed</span> which is triggered whenever change events occur on the managed object attributes. The Unit of Work (objectstore) package overrides this method in order to receive change events; its essentially this:</p>
-<&|formatting.myt:code&>
- import sqlalchemy.attributes as attributes
- class UOWAttributeManager(attributes.AttributeManager):
- def value_changed(self, obj, key, value):
- if hasattr(obj, '_instance_key'):
- uow().register_dirty(obj)
- else:
- uow().register_new(obj)
-
- global_attributes = UOWAttributeManager()
-</&>
-<p>Objects that contain the attribute "_instance_key" are already registered with the Identity Map, and are assumed to have come from the database. They therefore get marked as "dirty" when changes happen. Objects without an "_instance_key" are not from the database, and get marked as "new" when changes happen, although usually this will already have occured via the object's __init__ method.</p>
-</&>
-</&>
--- /dev/null
+[alpha_api]: javascript:alphaApi()
+[alpha_implementation]: javascript:alphaImplementation()
+
+Advanced Data Mapping {@name=advdatamapping}
+======================
+
+This section details all the options available to Mappers, as well as advanced patterns.
+
+To start, heres the tables we will work with again:
+
+ {python}
+ from sqlalchemy import *
+
+ metadata = MetaData()
+
+ # a table to store users
+ users_table = Table('users', metadata,
+ Column('user_id', Integer, primary_key = True),
+ Column('user_name', String(40)),
+ Column('password', String(80))
+ )
+
+ # a table that stores mailing addresses associated with a specific user
+ addresses_table = Table('addresses', metadata,
+ Column('address_id', Integer, primary_key = True),
+ Column('user_id', Integer, ForeignKey("users.user_id")),
+ Column('street', String(100)),
+ Column('city', String(80)),
+ Column('state', String(2)),
+ Column('zip', String(10))
+ )
+
+ # a table that stores keywords
+ keywords_table = Table('keywords', metadata,
+ Column('keyword_id', Integer, primary_key = True),
+ Column('name', VARCHAR(50))
+ )
+
+ # a table that associates keywords with users
+ userkeywords_table = Table('userkeywords', metadata,
+ Column('user_id', INT, ForeignKey("users")),
+ Column('keyword_id', INT, ForeignKey("keywords"))
+ )
+
+
+### More On Mapper Properties {@name=properties}
+
+#### Overriding Column Names {@name=colname}
+
+When mappers are constructed, by default the column names in the Table metadata are used as the names of attributes on the mapped class. This can be customzed within the properties by stating the key/column combinations explicitly:
+
+ {python}
+ user_mapper = mapper(User, users_table, properties={
+ 'id' : users_table.c.user_id,
+ 'name' : users_table.c.user_name,
+ })
+
+In the situation when column names overlap in a mapper against multiple tables, columns may be referenced together with a list:
+
+ {python}
+ # join users and addresses
+ usersaddresses = sql.join(users_table, addresses_table, users_table.c.user_id == addresses_table.c.user_id)
+ m = mapper(User, usersaddresses,
+ properties = {
+ 'id' : [users_table.c.user_id, addresses_table.c.user_id],
+ }
+ )
+
+#### Overriding Properties {@name=overriding}
+
+A common request is the ability to create custom class properties that override the behavior of setting/getting an attribute. Currently, the easiest way to do this in SQLAlchemy is how it would be done in any Python program; define your attribute with a different name, such as "_attribute", and use a property to get/set its value. The mapper just needs to be told of the special name:
+
+ {python}
+ class MyClass(object):
+ def _set_email(self, email):
+ self._email = email
+ def _get_email(self, email):
+ return self._email
+ email = property(_get_email, _set_email)
+
+ mapper(MyClass, mytable, properties = {
+ # map the '_email' attribute to the "email" column
+ # on the table
+ '_email': mytable.c.email
+ })
+
+In a later release, SQLAlchemy will also allow `_get_email` and `_set_email` to be attached directly to the "email" property created by the mapper, and
+will also allow this association to occur via decorators.
+
+
+#### Custom List Classes {@name=customlist}
+
+Feature Status: [Alpha API][alpha_api]
+
+A one-to-many or many-to-many relationship results in a list-holding element being attached to all instances of a class. Currently, this list is an instance of `sqlalchemy.util.HistoryArraySet`, is a `UserDict` instance that *decorates* an underlying list object. The implementation of this list can be controlled, and can in fact be any object that implements a `list`-style `append` and `__iter__` method. A common need is for a list-based relationship to actually be a dictionary. This can be achieved by subclassing `dict` to have `list`-like behavior.
+
+In this example, a class `MyClass` is defined, which is associated with a parent object `MyParent`. The collection of `MyClass` objects on each `MyParent` object will be a dictionary, storing each `MyClass` instance keyed to its `name` attribute.
+
+ {python}
+ # a class to be stored in the list
+ class MyClass(object):
+ def __init__(self, name):
+ self.name = name
+
+ # create a dictionary that will act like a list, and store
+ # instances of MyClass
+ class MyDict(dict):
+ def append(self, item):
+ self[item.name] = item
+ def __iter__(self):
+ return self.values()
+
+ # parent class
+ class MyParent(object):
+ # this class-level attribute provides the class to be
+ # used by the 'myclasses' attribute
+ myclasses = MyDict
+
+ # mappers, constructed normally
+ mapper(MyClass, myclass_table)
+ mapper(MyParent, myparent_table, properties={
+ 'myclasses' : relation(MyClass)
+ })
+
+ # elements on 'myclasses' can be accessed via string keyname
+ myparent = MyParent()
+ myparent.myclasses.append(MyClass('this is myclass'))
+ myclass = myparent.myclasses['this is myclass']
+
+
+#### Custom Join Conditions {@name=customjoin}
+
+When creating relations on a mapper, most examples so far have illustrated the mapper and relationship joining up based on the foreign keys of the tables they represent. in fact, this "automatic" inspection can be completely circumvented using the `primaryjoin` and `secondaryjoin` arguments to `relation`, as in this example which creates a User object which has a relationship to all of its Addresses which are in Boston:
+
+ {python}
+ class User(object):
+ pass
+ class Address(object):
+ pass
+
+ mapper(Address, addresses_table)
+ mapper(User, users_table, properties={
+ 'boston_addreses' : relation(Address, primaryjoin=
+ and_(users_table.c.user_id==Address.c.user_id,
+ Addresses.c.city=='Boston'))
+ })
+
+Many to many relationships can be customized by one or both of `primaryjoin` and `secondaryjoin`, shown below with just the default many-to-many relationship explicitly set:
+
+ {python}
+ class User(object):
+ pass
+ class Keyword(object):
+ pass
+ mapper(Keyword, keywords_table)
+ mapper(User, users_table, properties={
+ 'keywords':relation(Keyword, secondary=userkeywords_table
+ primaryjoin=users_table.c.user_id==userkeywords_table.c.user_id,
+ secondaryjoin=userkeywords_table.c.keyword_id==keywords_table.c.keyword_id
+ )
+ })
+
+#### Lazy/Eager Joins Multiple Times to One Table {@name=multiplejoin}
+
+The previous example leads in to the idea of joining against the same table multiple times. Below is a User object that has lists of its Boston and New York addresses:
+
+ {python}
+ mapper(User, users_table, properties={
+ 'boston_addreses' : relation(Address, primaryjoin=
+ and_(users_table.c.user_id==Address.c.user_id,
+ Addresses.c.city=='Boston')),
+ 'newyork_addresses' : relation(Address, primaryjoin=
+ and_(users_table.c.user_id==Address.c.user_id,
+ Addresses.c.city=='New York')),
+ })
+
+Both lazy and eager loading support multiple joins equally well.
+
+#### Deferred Column Loading {@name=deferred}
+
+This feature allows particular columns of a table to not be loaded by default, instead being loaded later on when first referenced. It is essentailly "column-level lazy loading". This feature is useful when one wants to avoid loading a large text or binary field into memory when its not needed. Individual columns can be lazy loaded by themselves or placed into groups that lazy-load together.
+
+ {python}
+ book_excerpts = Table('books', db,
+ Column('book_id', Integer, primary_key=True),
+ Column('title', String(200), nullable=False),
+ Column('summary', String(2000)),
+ Column('excerpt', String),
+ Column('photo', Binary)
+ )
+
+ class Book(object):
+ pass
+
+ # define a mapper that will load each of 'excerpt' and 'photo' in
+ # separate, individual-row SELECT statements when each attribute
+ # is first referenced on the individual object instance
+ mapper(Book, book_excerpts, properties = {
+ 'excerpt' : deferred(book_excerpts.c.excerpt),
+ 'photo' : deferred(book_excerpts.c.photo)
+ })
+
+Deferred columns can be placed into groups so that they load together:
+
+ {python}
+ book_excerpts = Table('books', db,
+ Column('book_id', Integer, primary_key=True),
+ Column('title', String(200), nullable=False),
+ Column('summary', String(2000)),
+ Column('excerpt', String),
+ Column('photo1', Binary),
+ Column('photo2', Binary),
+ Column('photo3', Binary)
+ )
+
+ class Book(object):
+ pass
+
+ # define a mapper with a 'photos' deferred group. when one photo is referenced,
+ # all three photos will be loaded in one SELECT statement. The 'excerpt' will
+ # be loaded separately when it is first referenced.
+ mapper(Book, book_excerpts, properties = {
+ 'excerpt' : deferred(book_excerpts.c.excerpt),
+ 'photo1' : deferred(book_excerpts.c.photo1, group='photos'),
+ 'photo2' : deferred(book_excerpts.c.photo2, group='photos'),
+ 'photo3' : deferred(book_excerpts.c.photo3, group='photos')
+ })
+
+#### Relation Options {@name=relationoptions}
+
+Keyword options to the `relation` function include:
+
+* lazy=(True|False|None) - specifies how the related items should be loaded. a value of True indicates they should be loaded when the property is first accessed. A value of False indicates they should be loaded by joining against the parent object query, so parent and child are loaded in one round trip. A value of None indicates the related items are not loaded by the mapper in any case; the application will manually insert items into the list in some other way. A relationship with lazy=None is still important; items added to the list or removed will cause the appropriate updates and deletes upon flush(). Future capabilities for lazy might also include "lazy='extra'", which would allow lazy loading of child elements one at a time, for very large collections.
+* cascade - a string list of **cascade rules** which determines how persistence operations should be "cascaded" from parent to child. For a description of cascade rules, see [datamapping_relations_cycle](rel:datamapping_relations_lifecycle) and [unitofwork_cascade](rel:unitofwork_cascade).
+* secondary - for a many-to-many relationship, specifies the intermediary table.
+* primaryjoin - a ClauseElement that will be used as the primary join of this child object against the parent object, or in a many-to-many relationship the join of the primary object to the association table. By default, this value is computed based on the foreign key relationships of the parent and child tables (or association table).
+* secondaryjoin - a ClauseElement that will be used as the join of an association table to the child object. By default, this value is computed based on the foreign key relationships of the association and child tables.
+* foreignkey - specifies which column in this relationship is "foreign", i.e. which column refers to the parent object. This value is automatically determined in most cases based on the primary and secondary join conditions, except in the case of a self-referential mapper, where it is needed to indicate the child object's reference back to it's parent, or in the case where the join conditions do not represent any primary key columns to properly represent the direction of the relationship.
+* uselist - a boolean that indicates if this property should be loaded as a list or a scalar. In most cases, this value is determined based on the type and direction of the relationship - one to many forms a list, many to one forms a scalar, many to many is a list. If a scalar is desired where normally a list would be present, such as a bi-directional one-to-one relationship, set uselist to False.
+* private - setting `private=True` is the equivalent of setting `cascade="all, delete-orphan"`, and indicates the lifecycle of child objects should be contained within that of the parent. See the example in [datamapping_relations_cycle](rel:datamapping_relations_lifecycle).
+* backref - indicates the name of a property to be placed on the related mapper's class that will handle this relationship in the other direction, including synchronizing the object attributes on both sides of the relation. Can also point to a `backref()` construct for more configurability. See [datamapping_relations_backreferences](rel:datamapping_relations_backreferences).
+* order_by - indicates the ordering that should be applied when loading these items. See the section [advdatamapping_orderby](rel:advdatamapping_orderby) for details.
+* association - When specifying a many to many relationship with an association object, this keyword should reference the mapper or class of the target object of the association. See the example in [datamapping_association](rel:datamapping_association).
+* post_update - this indicates that the relationship should be handled by a second UPDATE statement after an INSERT, or before a DELETE. using this flag essentially means the relationship will not incur any "dependency" between parent and child item, as the particular foreign key relationship between them is handled by a second statement. use this flag when a particular mapping arrangement will incur two rows that are dependent on each other, such as a table that has a one-to-many relationship to a set of child rows, and also has a column that references a single child row within that list (i.e. both tables contain a foreign key to each other). If a flush() operation returns an error that a "cyclical dependency" was detected, this is a cue that you might want to use post_update.
+
+### Controlling Ordering {@name=orderby}
+
+By default, mappers will attempt to ORDER BY the "oid" column of a table, or the primary key column, when selecting rows. This can be modified in several ways.
+
+The "order_by" parameter can be sent to a mapper, overriding the per-engine ordering if any. A value of None means that the mapper should not use any ordering. A non-None value, which can be a column, an `asc` or `desc` clause, or an array of either one, indicates the ORDER BY clause that should be added to all select queries:
+
+ {python}
+ # disable all ordering
+ mapper = mapper(User, users_table, order_by=None)
+
+ # order by a column
+ mapper = mapper(User, users_table, order_by=users_tableusers_table.c.user_id)
+
+ # order by multiple items
+ mapper = mapper(User, users_table, order_by=[users_table.c.user_id, desc(users_table.c.user_name)])
+
+"order_by" can also be specified to an individual `select` method, overriding all other per-engine/per-mapper orderings:
+
+ {python}
+ # order by a column
+ l = mapper.select(users_table.c.user_name=='fred', order_by=users_table.c.user_id)
+
+ # order by multiple criterion
+ l = mapper.select(users_table.c.user_name=='fred', order_by=[users_table.c.user_id, desc(users_table.c.user_name)])
+
+For relations, the "order_by" property can also be specified to all forms of relation:
+
+ {python}
+ # order address objects by address id
+ mapper = mapper(User, users_table, properties = {
+ 'addresses' : relation(mapper(Address, addresses_table), order_by=addresses_table.c.address_id)
+ })
+
+ # eager load with ordering - the ORDER BY clauses of parent/child will be organized properly
+ mapper = mapper(User, users_table, properties = {
+ 'addresses' : relation(mapper(Address, addresses_table), order_by=desc(addresses_table.c.email_address), eager=True)
+ }, order_by=users_table.c.user_id)
+
+### Limiting Rows {@name=limits}
+
+You can limit rows in a regular SQL query by specifying `limit` and `offset`. A Mapper can handle the same concepts:
+
+ {python}
+ class User(object):
+ pass
+
+ mapper(User, users_table)
+ {sql}r = session.query(User).select(limit=20, offset=10)
+ SELECT users.user_id AS users_user_id,
+ users.user_name AS users_user_name, users.password AS users_password
+ FROM users ORDER BY users.oid
+ LIMIT 20 OFFSET 10
+ {}
+
+However, things get tricky when dealing with eager relationships, since a straight LIMIT of rows does not represent the count of items when joining against other tables to load related items as well. So here is what SQLAlchemy will do when you use limit or offset with an eager relationship:
+
+ {python}
+ class User(object):
+ pass
+ class Address(object):
+ pass
+ mapper(User, users_table, properties={
+ 'addresses' : relation(mapper(Address, addresses_table), lazy=False)
+ })
+ r = session.query(User).select(User.c.user_name.like('F%'), limit=20, offset=10)
+ {opensql}SELECT users.user_id AS users_user_id, users.user_name AS users_user_name,
+ users.password AS users_password, addresses.address_id AS addresses_address_id,
+ addresses.user_id AS addresses_user_id, addresses.street AS addresses_street,
+ addresses.city AS addresses_city, addresses.state AS addresses_state,
+ addresses.zip AS addresses_zip
+ FROM
+ (SELECT users.user_id FROM users WHERE users.user_name LIKE %(users_user_name)s
+ ORDER BY users.oid LIMIT 20 OFFSET 10) AS rowcount,
+ users LEFT OUTER JOIN addresses ON users.user_id = addresses.user_id
+ WHERE rowcount.user_id = users.user_id ORDER BY users.oid, addresses.oid
+ {'users_user_name': 'F%'}
+
+The main WHERE clause as well as the limiting clauses are coerced into a subquery; this subquery represents the desired result of objects. A containing query, which handles the eager relationships, is joined against the subquery to produce the result.
+
+### More on Mapper Options {@name=options}
+
+The `options` method on the `Query` object, first introduced in [datamapping_relations_options](rel:datamapping_relations_options), produces a new `Query` object by creating a copy of the underlying `Mapper` and placing modified properties on it. The `options` method is also directly available off the `Mapper` object itself, so that the newly copied `Mapper` can be dealt with directly. The `options` method takes a variable number of `MapperOption` objects which know how to change specific things about the mapper. The five available options are `eagerload`, `lazyload`, `noload`, `deferred` and `extension`.
+
+An example of a mapper with a lazy load relationship, upgraded to an eager load relationship:
+
+ {python}
+ class User(object):
+ pass
+ class Address(object):
+ pass
+
+ # a 'lazy' relationship
+ mapper(User, users_table, properties = {
+ 'addreses':relation(mapper(Address, addresses_table), lazy=True)
+ })
+
+ # copy the mapper and convert 'addresses' to be eager
+ eagermapper = class_mapper(User).options(eagerload('addresses'))
+
+The `defer` and `undefer` options can control the deferred loading of attributes:
+
+ {python}
+ # set the 'excerpt' deferred attribute to load normally
+ m = book_mapper.options(undefer('excerpt'))
+
+ # set the referenced mapper 'photos' to defer its loading of the column 'imagedata'
+ m = book_mapper.options(defer('photos.imagedata'))
+
+### Mapping a Class with Table Inheritance {@name=inheritance}
+
+Feature Status: [Alpha Implementation][alpha_implementation]
+
+Inheritance in databases comes in three forms: *single table inheritance*, where several types of classes are stored in one table, *concrete table inheritance*, where each type of class is stored in its own table, and *multiple table inheritance*, where the parent/child classes are stored in their own tables that are joined together in a select.
+
+There is also a concept of `polymorphic` loading, which indicates if multiple kinds of classes can be loaded in one pass.
+
+SQLAlchemy supports all three kinds of inheritance. Additionally, true `polymorphic` loading is supported in a straightfoward way for single table inheritance, and has some more manually-configured features that can make it happen for concrete and multiple table inheritance.
+
+Working examples of polymorphic inheritance come with the distribution in the directory `examples/polymorphic`.
+
+Here are the classes we will use to represent an inheritance relationship:
+
+ {python}
+ class Employee(object):
+ def __init__(self, name):
+ self.name = name
+ def __repr__(self):
+ return self.__class__.__name__ + " " + self.name
+
+ class Manager(Employee):
+ def __init__(self, name, manager_data):
+ self.name = name
+ self.manager_data = manager_data
+ def __repr__(self):
+ return self.__class__.__name__ + " " + self.name + " " + self.manager_data
+
+ class Engineer(Employee):
+ def __init__(self, name, engineer_info):
+ self.name = name
+ self.engineer_info = engineer_info
+ def __repr__(self):
+ return self.__class__.__name__ + " " + self.name + " " + self.engineer_info
+
+Each class supports a common `name` attribute, while the `Manager` class has its own attribute `manager_data` and the `Engineer` class has its own attribute `engineer_info`.
+
+#### Single Table Inheritance
+
+This will support polymorphic loading via the `Employee` mapper.
+
+ {python}
+ employees_table = Table('employees', metadata,
+ Column('employee_id', Integer, primary_key=True),
+ Column('name', String(50)),
+ Column('manager_data', String(50)),
+ Column('engineer_info', String(50)),
+ Column('type', String(20))
+ )
+
+ employee_mapper = mapper(Employee, employees_table, polymorphic_on=employees_table.c.type)
+ manager_mapper = mapper(Manager, inherits=employee_mapper, polymorphic_identity='manager')
+ engineer_mapper = mapper(Engineer, inherits=employee_mapper, polymorphic_identity='engineer')
+
+#### Concrete Table Inheritance
+
+Without polymorphic loading, you just define a separate mapper for each class.
+
+ {python title="Concrete Inheritance, Non-polymorphic"}
+ managers_table = Table('managers', metadata,
+ Column('employee_id', Integer, primary_key=True),
+ Column('name', String(50)),
+ Column('manager_data', String(50)),
+ )
+
+ engineers_table = Table('engineers', metadata,
+ Column('employee_id', Integer, primary_key=True),
+ Column('name', String(50)),
+ Column('engineer_info', String(50)),
+ )
+
+ manager_mapper = mapper(Manager, managers_table)
+ engineer_mapper = mapper(Engineer, engineers_table)
+
+With polymorphic loading, the SQL query to do the actual polymorphic load must be constructed, usually as a UNION. There is a helper function to create these UNIONS called `polymorphic_union`.
+
+ {python title="Concrete Inheritance, Polymorphic"}
+ pjoin = polymorphic_union({
+ 'manager':managers_table,
+ 'engineer':engineers_table
+ }, 'type', 'pjoin')
+
+ employee_mapper = mapper(Employee, pjoin, polymorphic_on=pjoin.c.type)
+ manager_mapper = mapper(Manager, managers_table, inherits=employee_mapper, concrete=True, polymorphic_identity='manager')
+ engineer_mapper = mapper(Engineer, engineers_table, inherits=employee_mapper, concrete=True, polymorphic_identity='engineer')
+
+A future release of SQLALchemy might better merge the generated UNION into the mapper construction phase.
+
+#### Multiple Table Inheritance
+
+Like concrete table inheritance, this can be done non-polymorphically, or with a little more complexity, polymorphically:
+
+ {python title="Multiple Table Inheritance, Non-polymorphic"}
+ people = Table('people', metadata,
+ Column('person_id', Integer, primary_key=True),
+ Column('name', String(50)),
+ Column('type', String(30)))
+
+ engineers = Table('engineers', metadata,
+ Column('person_id', Integer, ForeignKey('people.person_id'), primary_key=True),
+ Column('engineer_info', String(50)),
+ )
+
+ managers = Table('managers', metadata,
+ Column('person_id', Integer, ForeignKey('people.person_id'), primary_key=True),
+ Column('manager_data', String(50)),
+ )
+
+ person_mapper = mapper(Person, people)
+ mapper(Engineer, engineers, inherits=person_mapper)
+ mapper(Manager, managers, inherits=person_mapper)
+
+Polymorphic:
+
+ {python title="Multiple Table Inheritance, Polymorphic"}
+ person_join = polymorphic_union(
+ {
+ 'engineer':people.join(engineers),
+ 'manager':people.join(managers),
+ 'person':people.select(people.c.type=='person'),
+ }, None, 'pjoin')
+
+ person_mapper = mapper(Person, people, select_table=person_join, polymorphic_on=person_join.c.type, polymorphic_identity='person')
+ mapper(Engineer, engineers, inherits=person_mapper, polymorphic_identity='engineer')
+ mapper(Manager, managers, inherits=person_mapper, polymorphic_identity='manager')
+
+The join condition in a multiple table inheritance relationship can be specified explicitly, using `inherit_condition`:
+
+ {python}
+ AddressUser.mapper = mapper(
+ AddressUser,
+ addresses_table, inherits=User.mapper,
+ inherit_condition=users_table.c.user_id==addresses_table.c.user_id
+ )
+
+### Mapping a Class against Multiple Tables {@name=joins}
+
+Mappers can be constructed against arbitrary relational units (called `Selectables`) as well as plain `Tables`. For example, The `join` keyword from the SQL package creates a neat selectable unit comprised of multiple tables, complete with its own composite primary key, which can be passed in to a mapper as the table.
+
+ {python}
+ # a class
+ class AddressUser(object):
+ pass
+
+ # define a Join
+ j = join(users_table, addresses_table)
+
+ # map to it - the identity of an AddressUser object will be
+ # based on (user_id, address_id) since those are the primary keys involved
+ m = mapper(AddressUser, j)
+
+ A second example:
+ {python}
+ # many-to-many join on an association table
+ j = join(users_table, userkeywords,
+ users_table.c.user_id==userkeywords.c.user_id).join(keywords,
+ userkeywords.c.keyword_id==keywords.c.keyword_id)
+
+ # a class
+ class KeywordUser(object):
+ pass
+
+ # map to it - the identity of a KeywordUser object will be
+ # (user_id, keyword_id) since those are the primary keys involved
+ m = mapper(KeywordUser, j)
+
+### Mapping a Class against Arbitary Selects {@name=selects}
+
+Similar to mapping against a join, a plain select() object can be used with a mapper as well. Below, an example select which contains two aggregate functions and a group_by is mapped to a class:
+
+ {python}
+ s = select([customers,
+ func.count(orders).label('order_count'),
+ func.max(orders.price).label('highest_order')],
+ customers.c.customer_id==orders.c.customer_id,
+ group_by=[c for c in customers.c]
+ )
+ class Customer(object):
+ pass
+
+ m = mapper(Customer, s)
+
+Above, the "customers" table is joined against the "orders" table to produce a full row for each customer row, the total count of related rows in the "orders" table, and the highest price in the "orders" table, grouped against the full set of columns in the "customers" table. That query is then mapped against the Customer class. New instances of Customer will contain attributes for each column in the "customers" table as well as an "order_count" and "highest_order" attribute. Updates to the Customer object will only be reflected in the "customers" table and not the "orders" table. This is because the primary keys of the "orders" table are not represented in this mapper and therefore the table is not affected by save or delete operations.
+
+### Multiple Mappers for One Class {@name=multiple}
+
+The first mapper created for a certain class is known as that class's "primary mapper." Other mappers can be created as well, these come in two varieties.
+
+* **secondary mapper** - this is a mapper that must be constructed with the keyword argument `non_primary=True`, and represents a load-only mapper. Objects that are loaded with a secondary mapper will have their save operation processed by the primary mapper. It is also invalid to add new `relation()`s to a non-primary mapper. To use this mapper with the Session, specify it to the `query` method:
+
+example:
+
+ {python}
+ # primary mapper
+ mapper(User, users_table)
+
+ # make a secondary mapper to load User against a join
+ othermapper = mapper(User, users_table.join(someothertable), non_primary=True)
+
+ # select
+ result = session.query(othermapper).select()
+
+* **entity name mapper** - this is a mapper that is a fully functioning primary mapper for a class, which is distinguished from the regular primary mapper by an `entity_name` parameter. Instances loaded with this mapper will be totally managed by this new mapper and have no connection to the original one. Most methods on `Session` include an optional `entity_name` parameter in order to specify this condition.
+
+example:
+
+ {python}
+ # primary mapper
+ mapper(User, users_table)
+
+ # make an entity name mapper that stores User objects in another table
+ mapper(User, alternate_users_table, entity_name='alt')
+
+ # make two User objects
+ user1 = User()
+ user2 = User()
+
+ # save one in in the "users" table
+ session.save(user1)
+
+ # save the other in the "alternate_users_table"
+ session.save(user2, entity_name='alt')
+
+ session.flush()
+
+ # select from the alternate mapper
+ session.query(User, entity_name='alt').select()
+
+### Circular Mapping {@name=circular}
+
+Oftentimes it is necessary for two mappers to be related to each other. With a datamodel that consists of Users that store Addresses, you might have an Address object and want to access the "user" attribute on it, or have a User object and want to get the list of Address objects. The easiest way to do this is via the `backref` keyword described in [datamapping_relations_backreferences](rel:datamapping_relations_backreferences). Although even when backreferences are used, it is sometimes necessary to explicitly specify the relations on both mappers pointing to each other.
+To achieve this involves creating the first mapper by itself, then creating the second mapper referencing the first, then adding references to the first mapper to reference the second:
+
+ {python}
+ usermapper = mapper(User, users)
+ mapper(Address, addresses_table, properties={
+ 'user':relation(User)
+ })
+
+ usermapper.add_property('addresses', relation(Address))
+
+Note that with a circular relationship as above, you cannot declare both relationships as "eager" relationships, since that produces a circular query situation which will generate a recursion exception. So what if you want to load an Address and its User eagerly? Just use eager options:
+
+ {python}
+ eagerquery = session.query(Address).options(eagerload('user'))
+ s = eagerquery.select(Address.c.address_id==12)
+
+### Self Referential Mappers {@name=recursive}
+
+A self-referential mapper is a mapper that is designed to operate with an <b>adjacency list</b> table. This is a table that contains one or more foreign keys back to itself, and is usually used to create hierarchical tree structures. SQLAlchemy's default model of saving items based on table dependencies is not sufficient in this case, as an adjacency list table introduces dependencies between individual rows. Fortunately, SQLAlchemy will automatically detect a self-referential mapper and do the extra lifting to make it work.
+
+ {python}
+ # define a self-referential table
+ trees = Table('treenodes', engine,
+ Column('node_id', Integer, primary_key=True),
+ Column('parent_node_id', Integer, ForeignKey('treenodes.node_id'), nullable=True),
+ Column('node_name', String(50), nullable=False),
+ )
+
+ # treenode class
+ class TreeNode(object):
+ pass
+
+ # mapper defines "children" property, pointing back to TreeNode class,
+ # with the mapper unspecified. it will point back to the primary
+ # mapper on the TreeNode class.
+ TreeNode.mapper = mapper(TreeNode, trees, properties={
+ 'children' : relation(
+ TreeNode,
+ cascade="all, delete-orphan"
+ ),
+ }
+ )
+
+ # or, specify the circular relationship after establishing the original mapper:
+ mymapper = mapper(TreeNode, trees)
+
+ mymapper.add_property('children', relation(
+ mymapper,
+ cascade="all, delete-orphan"
+ ))
+
+This kind of mapper goes through a lot of extra effort when saving and deleting items, to determine the correct dependency graph of nodes within the tree.
+
+A self-referential mapper where there is more than one relationship on the table requires that all join conditions be explicitly spelled out. Below is a self-referring table that contains a "parent_node_id" column to reference parent/child relationships, and a "root_node_id" column which points child nodes back to the ultimate root node:
+
+ {python}
+ # define a self-referential table with several relations
+ trees = Table('treenodes', engine,
+ Column('node_id', Integer, primary_key=True),
+ Column('parent_node_id', Integer, ForeignKey('treenodes.node_id'), nullable=True),
+ Column('root_node_id', Integer, ForeignKey('treenodes.node_id'), nullable=True),
+ Column('node_name', String(50), nullable=False),
+ )
+
+ # treenode class
+ class TreeNode(object):
+ pass
+
+ # define the "children" property as well as the "root" property
+ TreeNode.mapper = mapper(TreeNode, trees, properties={
+ 'children' : relation(
+ TreeNode,
+ primaryjoin=trees.c.parent_node_id==trees.c.node_id
+ cascade="all, delete-orphan"
+ ),
+ 'root' : relation(
+ TreeNode,
+ primaryjoin=trees.c.root_node_id=trees.c.node_id,
+ foreignkey=trees.c.node_id,
+ uselist=False
+ )
+ }
+ )
+
+The "root" property on a TreeNode is a many-to-one relationship. By default, a self-referential mapper declares relationships as one-to-many, so the extra parameter `foreignkey`, pointing to the remote side of a relationship, is needed to indicate a "many-to-one" self-referring relationship.
+Both TreeNode examples above are available in functional form in the `examples/adjacencytree` directory of the distribution.
+
+### Result-Set Mapping {@name=resultset}
+
+Take any result set and feed it into a mapper to produce objects. Multiple mappers can be combined to retrieve unrelated objects from the same row in one step. The `instances` method on mapper takes a ResultProxy object, which is the result type generated from SQLEngine, and delivers object instances.
+
+ {python}
+ class User(object):
+ pass
+
+ User.mapper = mapper(User, users_table)
+
+ # select users
+ c = users_table.select().execute()
+
+ # get objects
+ userlist = User.mapper.instances(c)
+
+ {python}
+ # define a second class/mapper
+ class Address(object):
+ pass
+
+ Address.mapper = mapper(Address, addresses_table)
+
+ # select users and addresses in one query
+ s = select([users_table, addresses_table], users_table.c.user_id==addresses_table.c.user_id)
+
+ # execute it, and process the results with the User mapper, chained to the Address mapper
+ r = User.mapper.instances(s.execute(), Address.mapper)
+
+ # result rows are an array of objects, one for each mapper used
+ for entry in r:
+ user = r[0]
+ address = r[1]
+
+### Mapper Arguments {@name=arguments}
+
+Other arguments not covered above include:
+
+* select\_table=None - often used with polymorphic mappers, this is a `Selectable` which will take the place of the `Mapper`'s main table argument when performing queries.
+* version\_id\_col=None - an integer-holding Column object that will be assigned an incrementing
+counter, which is added to the WHERE clause used by UPDATE and DELETE statements. The matching row
+count returned by the database is compared to the expected row count, and an exception is raised if they dont match. This is a basic "optimistic concurrency" check. Without the version id column, SQLAlchemy still compares the updated rowcount.
+* always\_refresh=False - this option will cause the mapper to refresh all the attributes of all objects loaded by select/get statements, regardless of if they already exist in the current session. this includes all lazy- and eager-loaded relationship attributes, and will also overwrite any changes made to attributes on the column.
+
+### Extending Mapper {@name=extending}
+
+Mappers can have functionality augmented or replaced at many points in its execution via the usage of the MapperExtension class. This class is just a series of "hooks" where various functionality takes place. An application can make its own MapperExtension objects, overriding only the methods it needs. Methods that are not overridden return the special value `sqlalchemy.orm.mapper.EXT_PASS`, which indicates the operation should proceed as normally.
+
+ {python}
+ class MapperExtension(object):
+ def select_by(self, query, *args, **kwargs):
+ """overrides the select_by method of the Query object"""
+ def select(self, query, *args, **kwargs):
+ """overrides the select method of the Query object"""
+ def create_instance(self, mapper, session, row, imap, class_):
+ """called when a new object instance is about to be created from a row.
+ the method can choose to create the instance itself, or it can return
+ None to indicate normal object creation should take place.
+
+ mapper - the mapper doing the operation
+
+ row - the result row from the database
+
+ imap - a dictionary that is storing the running set of objects collected from the
+ current result set
+
+ class_ - the class we are mapping.
+ """
+ def append_result(self, mapper, session, row, imap, result, instance, isnew, populate_existing=False):
+ """called when an object instance is being appended to a result list.
+
+ If this method returns True, it is assumed that the mapper should do the appending, else
+ if this method returns False, it is assumed that the append was handled by this method.
+
+ mapper - the mapper doing the operation
+
+ row - the result row from the database
+
+ imap - a dictionary that is storing the running set of objects collected from the
+ current result set
+
+ result - an instance of util.HistoryArraySet(), which may be an attribute on an
+ object if this is a related object load (lazy or eager). use result.append_nohistory(value)
+ to append objects to this list.
+
+ instance - the object instance to be appended to the result
+
+ isnew - indicates if this is the first time we have seen this object instance in the current result
+ set. if you are selecting from a join, such as an eager load, you might see the same object instance
+ many times in the same result set.
+
+ populate_existing - usually False, indicates if object instances that were already in the main
+ identity map, i.e. were loaded by a previous select(), get their attributes overwritten
+ """
+ def populate_instance(self, mapper, session, instance, row, identitykey, imap, isnew):
+ """called right before the mapper, after creating an instance from a row, passes the row
+ to its MapperProperty objects which are responsible for populating the object's attributes.
+ If this method returns True, it is assumed that the mapper should do the appending, else
+ if this method returns False, it is assumed that the append was handled by this method.
+
+ Essentially, this method is used to have a different mapper populate the object:
+
+ def populate_instance(self, mapper, session, instance, row, identitykey, imap, isnew):
+ othermapper.populate_instance(session, instance, row, identitykey, imap, isnew, frommapper=mapper)
+ return True
+ """
+ def before_insert(self, mapper, connection, instance):
+ """called before an object instance is INSERTed into its table.
+
+ this is a good place to set up primary key values and such that arent handled otherwise."""
+ def before_update(self, mapper, connection, instance):
+ """called before an object instnace is UPDATED"""
+ def after_update(self, mapper, connection, instance):
+ """called after an object instnace is UPDATED"""
+ def after_insert(self, mapper, connection, instance):
+ """called after an object instance has been INSERTed"""
+ def before_delete(self, mapper, connection, instance):
+ """called before an object instance is DELETEed"""
+ def after_delete(self, mapper, connection, instance):
+ """called after an object instance is DELETEed"""
+
+To use MapperExtension, make your own subclass of it and just send it off to a mapper:
+
+ {python}
+ m = mapper(User, users_table, extension=MyExtension())
+
+Multiple extensions will be chained together and processed in order; they are specified as a list:
+
+ {python}
+ m = mapper(User, users_table, extension=[ext1, ext2, ext3])
+
-Data Mapping
+[alpha_api]: javascript:alphaApi()
+[alpha_implementation]: javascript:alphaImplementation()
+
+Data Mapping {@name=datamapping}
============
### Basic Data Mapping {@name=datamapping}
Data mapping describes the process of defining *Mapper* objects, which associate table metadata with user-defined classes.
-The Mapper's role is to perform SQL operations upon the database, associating individual table rows with instances of those classes, and individual database columns with properties upon those instances, to transparently associate in-memory objects with a persistent database representation.
+The `Mapper`'s role is to perform SQL operations upon the database, associating individual table rows with instances of those classes, and individual database columns with properties upon those instances, to transparently associate in-memory objects with a persistent database representation.
+
+When a `Mapper` is created to associate a `Table` object with a class, all of the columns defined in the `Table` object are associated with the class via property accessors, which add overriding functionality to the normal process of setting and getting object attributes. These property accessors keep track of changes to object attributes; these changes will be stored to the database when the application "flushes" the current state of objects (known as a *Unit of Work*).
+
+Two objects provide the primary interface for interacting with Mappers and the "unit of work" in SA 0.2, which are the `Query` object and the `Session` object. `Query` deals with selecting objects from the database, whereas `Session` provides a context for loaded objects and the ability to communicate changes on those objects back to the database.
-When a Mapper is created to associate a Table object with a class, all of the columns defined in the Table object are associated with the class via property accessors, which add overriding functionality to the normal process of setting and getting object attributes. These property accessors keep track of changes to object attributes; these changes will be stored to the database when the application "commits" the current transactional context (known as a *Unit of Work*). The `__init__()` method of the object is also decorated to communicate changes when new instances of the object are created.
+The primary method on `Query` for loading objects is its `select()` method, which has similar arguments to a `sqlalchemy.sql.Select` object. But this select method executes automatically and returns results, instead of awaiting an execute() call. Instead of returning a cursor-like object, it returns an array of objects.
-The Mapper also provides the interface by which instances of the object are loaded from the database. The primary method for this is its `select()` method, which has similar arguments to a `sqlalchemy.sql.Select` object. But this select method executes automatically and returns results, instead of awaiting an execute() call. Instead of returning a cursor-like object, it returns an array of objects.
+The three configurational elements to be defined, i.e. the `Table` metadata, the user-defined class, and the `Mapper`, are typically defined as module-level variables, and may be defined in any fashion suitable to the application, with the only requirement being that the class and table metadata are described before the mapper. For the sake of example, we will be defining these elements close together, but this should not be construed as a requirement; since SQLAlchemy is not a framework, those decisions are left to the developer or an external framework.
-The three elements to be defined, i.e. the Table metadata, the user-defined class, and the Mapper, are typically defined as module-level variables, and may be defined in any fashion suitable to the application, with the only requirement being that the class and table metadata are described before the mapper. For the sake of example, we will be defining these elements close together, but this should not be construed as a requirement; since SQLAlchemy is not a framework, those decisions are left to the developer or an external framework.
+Also, keep in mind that the examples in this section deal with explicit `Session` objects mapped directly to `Engine` objects, which represents the most explicit style of using the ORM. Options exist for how this is configured, including binding `Table` objects directly to `Engines` (described in [metadata_tables_binding](rel:metadata_tables_binding)), as well as using the "Threadlocal" plugin which provides various code shortcuts by using an implicit Session associated to the current thread (described in [plugins_threadlocal](rel:plugins_threadlocal)).
### Synopsis {@name=synopsis}
-This is the simplest form of a full "round trip" of creating table meta data, creating a class, mapping the class to the table, getting some results, and saving changes. For each concept, the following sections will dig in deeper to the available capabilities.
+First, the metadata/mapper configuration code:
{python}
from sqlalchemy import *
- # engine
- engine = create_engine("sqlite://mydb.db")
-
- # table metadata
- users = Table('users', engine,
+ # metadata
+ meta = MetaData()
+
+ # table object
+ users_table = Table('users', meta,
Column('user_id', Integer, primary_key=True),
Column('user_name', String(16)),
Column('password', String(20))
class User(object):
pass
- # create a mapper
- usermapper = mapper(User, users)
+ # create a mapper and associate it with the User class.
+ # technically we dont really need the 'usermapper' variable.
+ usermapper = mapper(User, users_table)
+
+Note that no database definitions are required. Next we will define an `Engine` and connect a `Session` to it, and perform a simple select:
+
+ {python}
+ # engine
+ engine = create_engine("sqlite://mydb.db")
+
+ # session
+ session = create_session(bind_to=engine)
# select
- {sql}user = usermapper.select_by(user_name='fred')[0]
+ {sql}user = session.query(User).select_by(user_name='fred')[0]
SELECT users.user_id AS users_user_id, users.user_name AS users_user_name,
users.password AS users_password
FROM users
# modify
user.user_name = 'fred jones'
- # commit - saves everything that changed
- {sql}objectstore.commit()
+ # flush - saves everything that changed
+ {sql}session.flush()
UPDATE users SET user_name=:user_name
WHERE users.user_id = :user_id
[{'user_name': 'fred jones', 'user_id': 1}]
-
-
-#### Attaching Mappers to their Class {@name=attaching}
-For convenience's sake, the Mapper can be attached as an attribute on the class itself as well:
+### The Query Object {@name=query}
- {python}
- User.mapper = mapper(User, users)
-
- userlist = User.mapper.select_by(user_id=12)
-
-There is also a full-blown "monkeypatch" function that creates a primary mapper, attaches the above mapper class property, and also the methods `get, get_by, select, select_by, selectone, selectfirst, commit, expire, refresh, expunge` and `delete`:
+The method `session.query(class_or_mapper)` returns a `Query` object. Below is a synopsis of things you can do with `Query`:
{python}
- # "assign" a mapper to the User class/users table
- assign_mapper(User, users)
-
- # methods are attached to the class for selecting
- userlist = User.select_by(user_id=12)
+ # get a query from a Session based on class:
+ query = session.query(User)
- myuser = User.get(1)
+ # get a query from a Session given a Mapper:
+ query = session.query(usermapper)
- # mark an object as deleted for the next commit
- myuser.delete()
-
- # commit the changes on a specific object
- myotheruser.commit()
-
-Other methods of associating mappers and finder methods with their corresponding classes, such as via common base classes or mixins, can be devised as well. SQLAlchemy does not aim to dictate application architecture and will always allow the broadest variety of architectural patterns, but may include more helper objects and suggested architectures in the future.
-
-#### Overriding Properties {@name=overriding}
-
-A common request is the ability to create custom class properties that override the behavior of setting/getting an attribute. Currently, the easiest way to do this in SQLAlchemy is how it would be done in any Python program; define your attribute with a different name, such as "_attribute", and use a property to get/set its value. The mapper just needs to be told of the special name:
-
- {python}
- class MyClass(object):
- def _set_email(self, email):
- self._email = email
- def _get_email(self, email):
- return self._email
- email = property(_get_email, _set_email)
-
- m = mapper(MyClass, mytable, properties = {
- # map the '_email' attribute to the "email" column
- # on the table
- '_email': mytable.c.email
- })
-
-In a later release, SQLAlchemy will also allow _get_email and _set_email to be attached directly to the "email" property created by the mapper, and will also allow this association to occur via decorators.
+ # select_by, which takes keyword arguments. the
+ # keyword arguments represent property names and the values
+ # represent values which will be compared via the = operator.
+ # the comparisons are joined together via "AND".
+ result = query.select_by(name='john', street='123 green street')
-### Selecting from a Mapper {@name=selecting}
-
-There are a variety of ways to select from a mapper. These range from minimalist to explicit. Below is a synopsis of the these methods:
-
- {python}
- # select_by, using property names or column names as keys
- # the keys are grouped together by an AND operator
- result = mapper.select_by(name='john', street='123 green street')
-
- # select_by can also combine SQL criterion with key/value properties
- result = mapper.select_by(users.c.user_name=='john',
- addresses.c.zip_code=='12345', street='123 green street')
+ # select_by can also combine ClauseElements with key/value properties.
+ # all ClauseElements and keyword-based criterion are combined together
+ # via "AND".
+ result = query.select_by(users_table.c.user_name=='john',
+ addresses_table.c.zip_code=='12345', street='123 green street')
# get_by, which takes the same arguments as select_by
# returns a single scalar result or None if no results
- user = mapper.get_by(id=12)
+ user = query.get_by(id=12)
# "dynamic" versions of select_by and get_by - everything past the
# "select_by_" or "get_by_" is used as the key, and the function argument
# as the value
- result = mapper.select_by_name('fred')
- u = mapper.get_by_name('fred')
+ result = query.select_by_name('fred')
+ u = query.get_by_name('fred')
# get an object directly from its primary key. this will bypass the SQL
# call if the object has already been loaded
- u = mapper.get(15)
+ u = query.get(15)
# get an object that has a composite primary key of three columns.
# the order of the arguments matches that of the table meta data.
- myobj = mapper.get(27, 3, 'receipts')
+ myobj = query.get((27, 3, 'receipts'))
# using a WHERE criterion
- result = mapper.select(or_(users.c.user_name == 'john', users.c.user_name=='fred'))
+ result = query.select(or_(users_table.c.user_name == 'john', users_table.c.user_name=='fred'))
# using a WHERE criterion to get a scalar
- u = mapper.selectfirst(users.c.user_name=='john')
-
+ u = query.selectfirst(users_table.c.user_name=='john')
+
# selectone() is a stricter version of selectfirst() which
# will raise an exception if there is not exactly one row
- u = mapper.selectone(users.c.user_name=='john')
+ u = query.selectone(users_table.c.user_name=='john')
# using a full select object
- result = mapper.select(users.select(users.c.user_name=='john'))
-
- # using straight text
- result = mapper.select_text("select * from users where user_name='fred'")
+ result = query.select(users_table.select(users_table.c.user_name=='john'))
- # or using a "text" object
- result = mapper.select(text("select * from users where user_name='fred'", engine=engine))
-
Some of the above examples above illustrate the usage of the mapper's Table object to provide the columns for a WHERE Clause. These columns are also accessible off of the mapped class directly. When a mapper is assigned to a class, it also attaches a special property accessor `c` to the class itself, which can be used just like the table metadata to access the columns of the table:
{python}
- User.mapper = mapper(User, users)
-
- userlist = User.mapper.select(User.c.user_id==12)
-
+ userlist = session.query(User).select(User.c.user_id==12)
### Saving Objects {@name=saving}
-When objects corresponding to mapped classes are created or manipulated, all changes are logged by a package called `sqlalchemy.mapping.objectstore`. The changes are then written to the database when an application calls `objectstore.commit()`. This pattern is known as a *Unit of Work*, and has many advantages over saving individual objects or attributes on those objects with individual method invocations. Domain models can be built with far greater complexity with no concern over the order of saves and deletes, excessive database round-trips and write operations, or deadlocking issues. The commit() operation uses a transaction as well, and will also perform "concurrency checking" to insure the proper number of rows were in fact affected (not supported with the current MySQL drivers). Transactional resources are used effectively in all cases; the unit of work handles all the details.
+When objects corresponding to mapped classes are created or manipulated, all changes are logged by the `Session` object. The changes are then written to the database when an application calls `flush()`. This pattern is known as a *Unit of Work*, and has many advantages over saving individual objects or attributes on those objects with individual method invocations. Domain models can be built with far greater complexity with no concern over the order of saves and deletes, excessive database round-trips and write operations, or deadlocking issues. The `flush()` operation batches its SQL statements into a transaction, and can also perform optimistic concurrency checks (using a version id column) to insure the proper number of rows were in fact affected (not supported with the current MySQL drivers).
-The Unit of Work is a powerful tool, and has some important concepts that must be understood in order to use it effectively. While this section illustrates rudimentary Unit of Work usage, it is strongly encouraged to consult the [unitofwork](rel:unitofwork) section for a full description on all its operations, including session control, deletion, and developmental guidelines.
+The Unit of Work is a powerful tool, and has some important concepts that should be understood in order to use it effectively. See the [unitofwork](rel:unitofwork) section for a full description on all its operations.
-When a mapper is created, the target class has its mapped properties decorated by specialized property accessors that track changes, and its `__init__()` method is also decorated to mark new objects as "new".
+When a mapper is created, the target class has its mapped properties decorated by specialized property accessors that track changes. New objects by default must be explicitly added to the `Session`, however this can be made automatic by using [plugins_threadlocal](rel:plugins_threadlocal) or [plugins_sessioncontext](rel:plugins_sessioncontext).
{python}
- User.mapper = mapper(User, users)
-
+ mapper(User, users_table)
+
# create a new User
myuser = User()
myuser.user_name = 'jane'
myuser.password = 'hello123'
-
+
# create another new User
myuser2 = User()
myuser2.user_name = 'ed'
myuser2.password = 'lalalala'
-
+
+ # create a Session and save them
+ sess = create_session()
+ sess.save(myuser)
+ sess.save(myuser2)
+
# load a third User from the database
- {sql}myuser3 = User.mapper.select(User.c.user_name=='fred')[0]
+ {sql}myuser3 = sess.query(User).select(User.c.user_name=='fred')[0]
SELECT users.user_id AS users_user_id,
users.user_name AS users_user_name, users.password AS users_password
FROM users WHERE users.user_name = :users_user_name
{'users_user_name': 'fred'}
-
+
myuser3.user_name = 'fredjones'
-
+
# save all changes
- {sql}objectstore.commit()
+ {sql}session.flush()
UPDATE users SET user_name=:user_name
WHERE users.user_id =:users_user_id
[{'users_user_id': 1, 'user_name': 'fredjones'}]
INSERT INTO users (user_name, password) VALUES (:user_name, :password)
{'password': 'lalalala', 'user_name': 'ed'}
-In the examples above, we defined a User class with basically no properties or methods. Theres no particular reason it has to be this way, the class can explicitly set up whatever properties it wants, whether or not they will be managed by the mapper. It can also specify a constructor, with the restriction that the constructor is able to function with no arguments being passed to it (this restriction can be lifted with some extra parameters to the mapper; more on that later):
+The mapped class can also specify whatever methods and/or constructor it wants:
{python}
class User(object):
- def __init__(self, user_name = None, password = None):
+ def __init__(self, user_name, password):
self.user_id = None
self.user_name = user_name
self.password = password
def __repr__(self):
return "User id %s name %s password %s" % (repr(self.user_id),
repr(self.user_name), repr(self.password))
- User.mapper = mapper(User, users)
+ mapper(User, users_table)
+ sess = create_session()
u = User('john', 'foo')
- {sql}objectstore.commit()
+ sess.save(u)
+ {sql}session.flush()
INSERT INTO users (user_name, password) VALUES (:user_name, :password)
{'password': 'foo', 'user_name': 'john'}
>>> u
User id 1 name 'john' password 'foo'
-Recent versions of SQLAlchemy will only put modified object attributes columns into the UPDATE statements generated upon commit. This is to conserve database traffic and also to successfully interact with a "deferred" attribute, which is a mapped object attribute against the mapper's primary table that isnt loaded until referenced by the application.
+SQLAlchemy will only put modified object attributes columns into the UPDATE statements generated upon flush. This is to conserve database traffic and also to successfully interact with a "deferred" attribute, which is a mapped object attribute against the mapper's primary table that isnt loaded until referenced by the application.
### Defining and Using Relationships {@name=relations}
-So that covers how to map the columns in a table to an object, how to load objects, create new ones, and save changes. The next step is how to define an object's relationships to other database-persisted objects. This is done via the `relation` function provided by the mapper module. So with our User class, lets also define the User has having one or more mailing addresses. First, the table metadata:
+So that covers how to map the columns in a table to an object, how to load objects, create new ones, and save changes. The next step is how to define an object's relationships to other database-persisted objects. This is done via the `relation` function provided by the `orm` module.
+
+#### One to Many {@name=onetomany}
+
+So with our User class, lets also define the User has having one or more mailing addresses. First, the table metadata:
{python}
from sqlalchemy import *
- engine = create_engine('sqlite://filename=mydb')
+
+ metadata = MetaData()
# define user table
- users = Table('users', engine,
+ users_table = Table('users', metadata,
Column('user_id', Integer, primary_key=True),
Column('user_name', String(16)),
Column('password', String(20))
)
# define user address table
- addresses = Table('addresses', engine,
+ addresses_table = Table('addresses', metadata,
Column('address_id', Integer, primary_key=True),
Column('user_id', Integer, ForeignKey("users.user_id")),
Column('street', String(100)),
Column('zip', String(10))
)
-Of importance here is the addresses table's definition of a *foreign key* relationship to the users table, relating the user_id column into a parent-child relationship. When a Mapper wants to indicate a relation of one object to another, this ForeignKey object is the default method by which the relationship is determined (although if you didn't define ForeignKeys, or you want to specify explicit relationship columns, that is available as well).
+Of importance here is the addresses table's definition of a *foreign key* relationship to the users table, relating the user_id column into a parent-child relationship. When a `Mapper` wants to indicate a relation of one object to another, the `ForeignKey` relationships are the default method by which the relationship is determined (options also exist to describe the relationships explicitly).
-So then lets define two classes, the familiar User class, as well as an Address class:
+So then lets define two classes, the familiar `User` class, as well as an `Address` class:
{python}
class User(object):
- def __init__(self, user_name = None, password = None):
+ def __init__(self, user_name, password):
self.user_name = user_name
self.password = password
class Address(object):
- def __init__(self, street=None, city=None, state=None, zip=None):
+ def __init__(self, street, city, state, zip):
self.street = street
self.city = city
self.state = state
self.zip = zip
-And then a Mapper that will define a relationship of the User and the Address classes to each other as well as their table metadata. We will add an additional mapper keyword argument `properties` which is a dictionary relating the name of an object property to a database relationship, in this case a `relation` object against a newly defined mapper for the Address class:
+And then a `Mapper` that will define a relationship of the `User` and the `Address` classes to each other as well as their table metadata. We will add an additional mapper keyword argument `properties` which is a dictionary relating the names of class attributes to database relationships, in this case a `relation` object against a newly defined mapper for the Address class:
{python}
- User.mapper = mapper(User, users, properties = {
- 'addresses' : relation(mapper(Address, addresses))
- }
- )
+ mapper(Address, addresses_table)
+ mapper(User, users_table, properties = {
+ 'addresses' : relation(Address)
+ }
+ )
Lets do some operations with these classes and see what happens:
{python}
+ engine = create_engine('sqlite:///mydb.db')
+ metadata.create_all(engine)
+
+ session = create_session(bind_to=engine)
+
u = User('jane', 'hihilala')
u.addresses.append(Address('123 anywhere street', 'big city', 'UT', '76543'))
u.addresses.append(Address('1 Park Place', 'some other city', 'OK', '83923'))
- objectstore.commit()
+ session.save(u)
+ session.flush()
{opensql}INSERT INTO users (user_name, password) VALUES (:user_name, :password)
{'password': 'hihilala', 'user_name': 'jane'}
INSERT INTO addresses (user_id, street, city, state, zip) VALUES (:user_id, :street, :city, :state, :zip)
INSERT INTO addresses (user_id, street, city, state, zip) VALUES (:user_id, :street, :city, :state, :zip)
{'city': 'some other city', 'state': 'OK', 'street': '1 Park Place', 'user_id':1, 'zip': '83923'}
-A lot just happened there! The Mapper object figured out how to relate rows in the addresses table to the users table, and also upon commit had to determine the proper order in which to insert rows. After the insert, all the User and Address objects have all their new primary and foreign keys populated.
+A lot just happened there! The `Mapper` figured out how to relate rows in the addresses table to the users table, and also upon flush had to determine the proper order in which to insert rows. After the insert, all the `User` and `Address` objects have their new primary and foreign key attributes populated.
-Also notice that when we created a Mapper on the User class which defined an 'addresses' relation, the newly created User instance magically had an "addresses" attribute which behaved like a list. This list is in reality a property accessor function, which returns an instance of `sqlalchemy.util.HistoryArraySet`, which fulfills the full set of Python list accessors, but maintains a *unique* set of objects (based on their in-memory identity), and also tracks additions and deletions to the list:
+Also notice that when we created a `Mapper` on the `User` class which defined an `addresses` relation, the newly created `User` instance magically had an "addresses" attribute which behaved like a list. This list is in reality a property function which returns an instance of `sqlalchemy.util.HistoryArraySet`. This object fulfills the full set of Python list accessors, but maintains a *unique* set of objects (based on their in-memory identity), and also tracks additions and deletions to the list:
{python}
del u.addresses[1]
u.addresses.append(Address('27 New Place', 'Houston', 'TX', '34839'))
- objectstore.commit()
+ session.flush()
{opensql}UPDATE addresses SET user_id=:user_id
WHERE addresses.address_id = :addresses_address_id
INSERT INTO addresses (user_id, street, city, state, zip)
VALUES (:user_id, :street, :city, :state, :zip)
{'city': 'Houston', 'state': 'TX', 'street': '27 New Place', 'user_id': 1, 'zip': '34839'}
-
-#### Useful Feature: Private Relations {@name=private}
-So our one address that was removed from the list, was updated to have a user_id of `None`, and a new address object was inserted to correspond to the new Address added to the User. But now, theres a mailing address with no user_id floating around in the database of no use to anyone. How can we avoid this ? This is acheived by using the `private=True` parameter of `relation`:
+Note that when creating a relation with the `relation()` function, the target can either be a class, in which case the primary mapper for that class is used as the target, or a `Mapper` instance itself, as returned by the `mapper()` function.
+
+#### Lifecycle Relations {@name=lifecycle}
+
+In the previous example, a single address was removed from the `addresses` attribute of a `User` object, resulting in the corresponding database row being updated to have a user_id of `None`. But now, theres a mailing address with no user_id floating around in the database of no use to anyone. How can we avoid this ? This is acheived by using the `cascade` parameter of `relation`:
{python}
- User.mapper = mapper(User, users, properties = {
- 'addresses' : relation(mapper(Address, addresses), private=True)
- }
- )
+ clear_mappers() # clear mappers from the previous example
+ mapper(Address, addresses_table)
+ mapper(User, users_table, properties = {
+ 'addresses' : relation(Address, cascade="all, delete-orphan")
+ }
+ )
+
del u.addresses[1]
u.addresses.append(Address('27 New Place', 'Houston', 'TX', '34839'))
- objectstore.commit()
+ session.flush()
{opensql}INSERT INTO addresses (user_id, street, city, state, zip)
VALUES (:user_id, :street, :city, :state, :zip)
{'city': 'Houston', 'state': 'TX', 'street': '27 New Place', 'user_id': 1, 'zip': '34839'}
DELETE FROM addresses WHERE addresses.address_id = :address_id
[{'address_id': 2}]
-In this case, with the private flag set, the element that was removed from the addresses list was also removed from the database. By specifying the `private` flag on a relation, it is indicated to the Mapper that these related objects exist only as children of the parent object, otherwise should be deleted.
+In this case, with the `delete-orphan` **cascade rule** set, the element that was removed from the addresses list was also removed from the database. Specifying `cascade="all, delete-orphan"` means that every persistence operation performed on the parent object will be *cascaded* to the child object or objects handled by the relation, and additionally that each child object cannot exist without being attached to a parent. Such a relationship indicates that the **lifecycle** of the `Address` objects are bounded by that of their parent `User` object.
+
+Cascading is described fully in [unitofwork_cascade](rel:unitofwork_cascade).
-#### Useful Feature: Backreferences {@name=backreferences}
+#### Backreferences {@name=backreferences}
-By creating relations with the `backref` keyword, a bi-directional relationship can be created which will keep both ends of the relationship updated automatically, even without any database queries being executed. Below, the User mapper is created with an "addresses" property, and the corresponding Address mapper receives a "backreference" to the User object via the property name "user":
+By creating relations with the `backref` keyword, a bi-directional relationship can be created which will keep both ends of the relationship updated automatically, independently of database operations. Below, the `User` mapper is created with an `addresses` property, and the corresponding `Address` mapper receives a "backreference" to the `User` object via the property name `user`:
{python}
- Address.mapper = mapper(Address, addresses)
- User.mapper = mapper(User, users, properties = {
- 'addresses' : relation(Address.mapper, backref='user')
+ Address = mapper(Address, addresses_table)
+ User = mapper(User, users_table, properties = {
+ 'addresses' : relation(Address, backref='user')
}
)
>>> a1.user is user and a2.user is user
True
-The backreference feature also works with many-to-many relationships, which are described later. When creating a backreference, a corresponding property is placed on the child mapper. The default arguments to this property can be overridden using the `backref()` function:
+The backreference feature also works with many-to-many relationships, which are described later. When creating a backreference, a corresponding property (i.e. a second `relation()`) is placed on the child mapper. The default arguments to this property can be overridden using the `backref()` function:
{python}
- Address.mapper = mapper(Address, addresses)
-
- User.mapper = mapper(User, users, properties = {
- 'addresses' : relation(Address.mapper,
- backref=backref('user', lazy=False, private=True))
- }
- )
+ mapper(User, users_table)
+ mapper(Address, addresses_table, properties={
+ 'user':relation(User, backref=backref('addresses', cascade="all, delete-orphan"))
+ })
-#### Creating Relationships Automatically with cascade_mappers {@name=cascade}
-The mapper package has a helper function `cascade_mappers()` which can simplify the task of linking several mappers together. Given a list of classes and/or mappers, it identifies the foreign key relationships between the given mappers or corresponding class mappers, and creates relation() objects representing those relationships, including a backreference. Attempts to find the "secondary" table in a many-to-many relationship as well. The names of the relations are a lowercase version of the related class. In the case of one-to-many or many-to-many, the name is "pluralized", which currently is based on the English language (i.e. an 's' or 'es' added to it):
+The `backref()` function is often used to set up a bi-directional one-to-one relationship. This is because the `relation()` function by default creates a "one-to-many" relationship when presented with a primary key/foreign key relationship, but the `backref()` function can redefine the `uselist` property to make it a scalar:
{python}
- # create two mappers. the 'users' and 'addresses' tables have a foreign key
- # relationship
- mapper1 = mapper(User, users)
- mapper2 = mapper(Address, addresses)
-
- # cascade the two mappers together (can also specify User, Address as the arguments)
- cascade_mappers(mapper1, mapper2)
-
- # two new object instances
- u = User('user1')
- a = Address('test')
+ mapper(User, users_table)
+ mapper(Address, addresses_table, properties={
+ 'user' : relation(User, backref=backref('address', uselist=False))
+ })
- # "addresses" and "user" property are automatically added
- u.addresses.append(a)
- print a.user
-#### Selecting from Relationships: Lazy Load {@name=lazyload}
+### Selecting from Relationships {@name=selectrelations}
-We've seen how the `relation` specifier affects the saving of an object and its child items, how does it affect selecting them? By default, the relation keyword indicates that the related property should be attached a *Lazy Loader* when instances of the parent object are loaded from the database; this is just a callable function that when accessed will invoke a second SQL query to load the child objects of the parent.
+We've seen how the `relation` specifier affects the saving of an object and its child items, how does it affect selecting them? By default, the relation keyword indicates that the related property should be attached a *lazy loader* when instances of the parent object are loaded from the database; this is just a callable function that when accessed will invoke a second SQL query to load the child objects of the parent.
{python}
# define a mapper
- User.mapper = mapper(User, users, properties = {
- 'addresses' : relation(mapper(Address, addresses), private=True)
- })
+ mapper(User, users_table, properties = {
+ 'addresses' : relation(mapper(Address, addresses_table))
+ })
# select users where username is 'jane', get the first element of the list
# this will incur a load operation for the parent table
- {sql}user = User.mapper.select(user_name='jane')[0]
+ {sql}user = session.query(User).select(User.c.user_name=='jane')[0]
SELECT users.user_id AS users_user_id,
users.user_name AS users_user_name, users.password AS users_password
FROM users WHERE users.user_name = :users_user_name ORDER BY users.oid
addresses.city AS addresses_city, addresses.state AS addresses_state,
addresses.zip AS addresses_zip FROM addresses
WHERE addresses.user_id = :users_user_id ORDER BY addresses.oid
- {'users_user_id': 1}
- print repr(a)
+ {'users_user_id': 1}
+
+ print repr(a)
-##### Useful Feature: Creating Joins via select_by {@name=relselectby}
+#### Creating Joins Across Relations {@name=relselectby}
-In mappers that have relationships, the `select_by` method and its cousins include special functionality that can be used to create joins. Just specify a key in the argument list which is not present in the primary mapper's list of properties or columns, but *is* present in the property list of one of its relationships:
+For mappers that have relationships, the `select_by` method of the `Query` object can create queries that include automatically created joins. Just specify a key in the argument list which is not present in the primary mapper's list of properties or columns, but *is* present in the property list of one of its relationships:
{python}
- {sql}l = User.mapper.select_by(street='123 Green Street')
+ {sql}l = session.query(User).select_by(street='123 Green Street')
SELECT users.user_id AS users_user_id,
users.user_name AS users_user_name, users.password AS users_password
FROM users, addresses
The above example is shorthand for:
{python}
- l = User.mapper.select(and_(
+ l = session.query(User).select(and_(
Address.c.user_id==User.c.user_id,
Address.c.street=='123 Green Street')
)
-
-##### How to Refresh the List? {@name=refreshing}
-Once the child list of Address objects is loaded, it is done loading for the lifetime of the object instance. Changes to the list will not be interfered with by subsequent loads, and upon commit those changes will be saved. Similarly, if a new User object is created and child Address objects added, a subsequent select operation which happens to touch upon that User instance, will also not affect the child list, since it is already loaded.
-
-The issue of when the mapper actually gets brand new objects from the database versus when it assumes the in-memory version is fine the way it is, is a subject of *transactional scope*. Described in more detail in the Unit of Work section, for now it should be noted that the total storage of all newly created and selected objects, *within the scope of the current thread*, can be reset via releasing or otherwise disregarding all current object instances, and calling:
+All keyword arguments sent to `select_by` are used to create query criterion. This means that familiar `select` keyword options like `order_by` and `limit` are not directly available. To enable these options with `select_by`, you can try the [plugins_selectresults](rel:plugins_selectresults) extension which offers methods off the result of a `select` or `select_by` such as `order_by()` and array slicing functions that generate new queries.
+
+Also, `select_by` will *not* create joins derived from `Column`-based expressions (i.e. `ClauseElement` objects); the reason is that a `Column`-based expression may include many columns, and `select_by` has no way to know which columns in the expression correspond to properties and which don't (it also prefers not to dig into column expressions which may be very complex). The next section describes some ways to combine `Column` expressions with `select_by`'s auto-joining capabilities.
+
+#### More Granular Join Control Using join\_to, join\_via {@name=jointo}
+
+Feature Status: [Alpha API][alpha_api]
+
+The `join_to` method of `Query` is a component of the `select_by` operation, and is given a keyname in order to return a "join path" from the Query's mapper to the mapper which is referenced by a `relation()` of the given name:
{python}
- objectstore.clear()
-
-This operation will clear out all currently mapped object instances, and subsequent select statements will load fresh copies from the databse.
-
-To operate upon a single object, just use the `remove` function:
+ >>> q = session.query(User)
+ >>> j = q.join_to('addresses')
+ >>> print j
+ users.user_id=addresses.user_id
+
+`join_to` can also be given the name of a column-based property, in which case it will locate a path to the nearest mapper which has that property as a column:
{python}
- # (this function coming soon)
- objectstore.remove(myobject)
+ >>> q = session.query(User)
+ >>> j = q.join_to('street')
+ >>> print j
+ users.user_id=addresses.user_id
+Also available is the `join_via` function, which is similar to `join_to`, except instead of traversing through all properties to find a path to the given key, its given an explicit path to the target property:
-#### Selecting from Relationships: Eager Load {@name=eagerload}
+ {python}
+ >>> q = session.query(User)
+ >>> j = q.join_via(['orders', 'items'])
+ >>> print j
+ users.c.user_id==orders.c.user_id AND orders.c.item_id==items.c.item_id
-With just a single parameter "lazy=False" specified to the relation object, the parent and child SQL queries can be joined together.
+Expressions produced by `join_to` and `join_via` can be used with `select` to create more complicated query criterion across multiple relations:
{python}
- Address.mapper = mapper(Address, addresses)
- User.mapper = mapper(User, users, properties = {
- 'addresses' : relation(Address.mapper, lazy=False)
- }
- )
-
- {sql}user = User.mapper.get_by(user_name='jane')
- SELECT users.user_id AS users_user_id, users.user_name AS users_user_name,
- users.password AS users_password,
- addresses.address_id AS addresses_address_id, addresses.user_id AS addresses_user_id,
- addresses.street AS addresses_street, addresses.city AS addresses_city,
- addresses.state AS addresses_state, addresses.zip AS addresses_zip
- FROM users LEFT OUTER JOIN addresses ON users.user_id = addresses.user_id
- WHERE users.user_name = :users_user_name ORDER BY users.oid, addresses.oid
+ >>> l = q.select(
+ (addresses_table.c.street=='some address') &
+ (items_table.c.item_name=='item #4') &
+ q.join_to('addresses') &
+ q.join_via(['orders', 'items'])
+ )
+
+
+#### Eager Loading {@name=eagerload}
+
+With just a single parameter `lazy=False` specified to the relation object, the parent and child SQL queries can be joined together.
+
+ {python}
+ mapper(Address, addresses_table)
+ mapper(User, users_table, properties = {
+ 'addresses' : relation(Address, lazy=False)
+ }
+ )
+
+ {sql}users = session.query(User).select(User.c.user_name=='Jane')
+ SELECT users.user_name AS users_user_name, users.password AS users_password,
+ users.user_id AS users_user_id, addresses_4fb8.city AS addresses_4fb8_city,
+ addresses_4fb8.address_id AS addresses_4fb8_address_id, addresses_4fb8.user_id AS addresses_4fb8_user_id,
+ addresses_4fb8.zip AS addresses_4fb8_zip, addresses_4fb8.state AS addresses_4fb8_state,
+ addresses_4fb8.street AS addresses_4fb8_street
+ FROM users LEFT OUTER JOIN addresses AS addresses_4fb8 ON users.user_id = addresses_4fb8.user_id
+ WHERE users.user_name = :users_user_name ORDER BY users.oid, addresses_4fb8.oid
{'users_user_name': 'jane'}
- for a in user.addresses:
- print repr(a)
-
+ for u in users:
+ print repr(u)
+ for a in u.addresses:
+ print repr(a)
Above, a pretty ambitious query is generated just by specifying that the User should be loaded with its child Addresses in one query. When the mapper processes the results, it uses an *Identity Map* to keep track of objects that were already loaded, based on their primary key identity. Through this method, the redundant rows produced by the join are organized into the distinct object instances they represent.
The generation of this query is also immune to the effects of additional joins being specified in the original query. To use our select_by example above, joining against the "addresses" table to locate users with a certain street results in this behavior:
{python}
- {sql}users = User.mapper.select_by(street='123 Green Street')
- SELECT users.user_id AS users_user_id,
- users.user_name AS users_user_name, users.password AS users_password,
- addresses.address_id AS addresses_address_id,
- addresses.user_id AS addresses_user_id, addresses.street AS addresses_street,
- addresses.city AS addresses_city, addresses.state AS addresses_state,
- addresses.zip AS addresses_zip
- FROM addresses AS addresses_417c,
- users LEFT OUTER JOIN addresses ON users.user_id = addresses.user_id
- WHERE addresses_417c.street = :addresses_street
- AND users.user_id = addresses_417c.user_id
- ORDER BY users.oid, addresses.oid
+ {sql}users = session.query(User).select_by(street='123 Green Street')
+
+ SELECT users.user_name AS users_user_name,
+ users.password AS users_password, users.user_id AS users_user_id,
+ addresses_6ca7.city AS addresses_6ca7_city,
+ addresses_6ca7.address_id AS addresses_6ca7_address_id,
+ addresses_6ca7.user_id AS addresses_6ca7_user_id,
+ addresses_6ca7.zip AS addresses_6ca7_zip, addresses_6ca7.state AS addresses_6ca7_state,
+ addresses_6ca7.street AS addresses_6ca7_street
+ FROM addresses, users LEFT OUTER JOIN addresses AS addresses_6ca7 ON users.user_id = addresses_6ca7.user_id
+ WHERE addresses.street = :addresses_street AND users.user_id = addresses.user_id ORDER BY users.oid, addresses_6ca7.oid
{'addresses_street': '123 Green Street'}
-The join implied by passing the "street" parameter is converted into an "aliasized" clause by the eager loader, so that it does not conflict with the join used to eager load the child address objects.
+The join implied by passing the "street" parameter is stated as an *additional* join between the `addresses` and `users` tables. Also, since the eager join is "aliasized", no name conflict occurs.
-#### Switching Lazy/Eager, No Load {@name=options}
+#### Using Options to Change the Loading Strategy {@name=options}
-The `options` method of mapper provides an easy way to get alternate forms of a mapper from an original one. The most common use of this feature is to change the "eager/lazy" loading behavior of a particular mapper, via the functions `eagerload()`, `lazyload()` and `noload()`:
+The `options` method on the `Query` object provides an easy way to get alternate forms of a mapper query from an original one. The most common use of this feature is to change the "eager/lazy" loading behavior of a particular mapper, via the functions `eagerload()`, `lazyload()` and `noload()`:
{python}
# user mapper with lazy addresses
- User.mapper = mapper(User, users, properties = {
- 'addresses' : relation(mapper(Address, addresses))
+ mapper(User, users_table, properties = {
+ 'addresses' : relation(mapper(Address, addresses_table))
}
)
- # make an eager loader
- eagermapper = User.mapper.options(eagerload('addresses'))
- u = eagermapper.select()
+ # query object
+ query = session.query(User)
- # make another mapper that wont load the addresses at all
- plainmapper = User.mapper.options(noload('addresses'))
+ # make an eager loading query
+ eagerquery = query.options(eagerload('addresses'))
+ u = eagerquery.select()
+
+ # make another query that wont load the addresses at all
+ plainquery = query.options(noload('addresses'))
# multiple options can be specified
- mymapper = oldmapper.options(lazyload('tracker'), noload('streets'), eagerload('members'))
+ myquery = oldquery.options(lazyload('tracker'), noload('streets'), eagerload('members'))
# to specify a relation on a relation, separate the property names by a "."
- mymapper = oldmapper.options(eagerload('orders.items'))
+ myquery = oldquery.options(eagerload('orders.items'))
### One to One/Many to One {@name=onetoone}
The above examples focused on the "one-to-many" relationship. To do other forms of relationship is easy, as the `relation` function can usually figure out what you want:
{python}
+ metadata = MetaData()
+
# a table to store a user's preferences for a site
- prefs = Table('user_prefs', engine,
+ prefs_table = Table('user_prefs', metadata,
Column('pref_id', Integer, primary_key = True),
Column('stylename', String(20)),
Column('save_password', Boolean, nullable = False),
Column('timezone', CHAR(3), nullable = False)
)
- # user table gets 'preference_id' column added
- users = Table('users', engine,
+ # user table with a 'preference_id' column
+ users_table = Table('users', metadata,
Column('user_id', Integer, primary_key = True),
Column('user_name', String(16), nullable = False),
Column('password', String(20), nullable = False),
- Column('preference_id', Integer, ForeignKey("prefs.pref_id"))
+ Column('preference_id', Integer, ForeignKey("user_prefs.pref_id"))
)
- # class definition for preferences
+ # engine and some test data
+ engine = create_engine('sqlite:///', echo=True)
+ metadata.create_all(engine)
+ engine.execute(prefs_table.insert(), dict(pref_id=1, stylename='green', save_password=1, timezone='EST'))
+ engine.execute(users_table.insert(), dict(user_name = 'fred', password='45nfss', preference_id=1))
+
+ # classes
+ class User(object):
+ def __init__(self, user_name, password):
+ self.user_name = user_name
+ self.password = password
+
class UserPrefs(object):
pass
- UserPrefs.mapper = mapper(UserPrefs, prefs)
-
- # address mapper
- Address.mapper = mapper(Address, addresses)
+
+ mapper(UserPrefs, prefs_table)
- # make a new mapper referencing everything.
- m = mapper(User, users, properties = dict(
- addresses = relation(Address.mapper, lazy=True, private=True),
- preferences = relation(UserPrefs.mapper, lazy=False, private=True),
+ mapper(User, users_table, properties = dict(
+ preferences = relation(UserPrefs, lazy=False, cascade="all, delete-orphan"),
))
# select
- {sql}user = m.get_by(user_name='fred')
- SELECT users.user_id AS users_user_id, users.user_name AS users_user_name,
- users.password AS users_password, users.preference_id AS users_preference_id,
- user_prefs.pref_id AS user_prefs_pref_id, user_prefs.stylename AS user_prefs_stylename,
- user_prefs.save_password AS user_prefs_save_password, user_prefs.timezone AS user_prefs_timezone
- FROM users LEFT OUTER JOIN user_prefs ON user_prefs.pref_id = users.preference_id
- WHERE users.user_name = :users_user_name ORDER BY users.oid, user_prefs.oid
+ session = create_session(bind_to=engine)
+ {sql}user = session.query(User).get_by(user_name='fred')
+ SELECT users.preference_id AS users_preference_id, users.user_name AS users_user_name,
+ users.password AS users_password, users.user_id AS users_user_id,
+ user_prefs_4eb2.timezone AS user_prefs_4eb2_timezone, user_prefs_4eb2.stylename AS user_prefs_4eb2_stylename,
+ user_prefs_4eb2.save_password AS user_prefs_4eb2_save_password, user_prefs_4eb2.pref_id AS user_prefs_4eb2_pref_id
+ FROM (SELECT users.user_id AS users_user_id FROM users WHERE users.user_name = :users_user_name ORDER BY users.oid
+ LIMIT 1 OFFSET 0) AS rowcount,
+ users LEFT OUTER JOIN user_prefs AS user_prefs_4eb2 ON user_prefs_4eb2.pref_id = users.preference_id
+ WHERE rowcount.users_user_id = users.user_id ORDER BY users.oid, user_prefs_4eb2.oid
{'users_user_name': 'fred'}
-
+
save_password = user.preferences.save_password
-
+
# modify
user.preferences.stylename = 'bluesteel'
- {sql}user.addresses.append(Address('freddy@hi.org'))
- SELECT email_addresses.address_id AS email_addresses_address_id,
- email_addresses.user_id AS email_addresses_user_id,
- email_addresses.email_address AS email_addresses_email_address
- FROM email_addresses
- WHERE email_addresses.user_id = :users_user_id
- ORDER BY email_addresses.oid, email_addresses.oid
- {'users_user_id': 1}
-
- # commit
- {sql}objectstore.commit()
+
+ # flush
+ {sql}session.flush()
UPDATE user_prefs SET stylename=:stylename
WHERE user_prefs.pref_id = :pref_id
[{'stylename': 'bluesteel', 'pref_id': 1}]
- INSERT INTO email_addresses (address_id, user_id, email_address)
- VALUES (:address_id, :user_id, :email_address)
- {'email_address': 'freddy@hi.org', 'address_id': None, 'user_id': 1}
### Many to Many {@name=manytomany}
The `relation` function handles a basic many-to-many relationship when you specify the association table:
{python}
- articles = Table('articles', engine,
+ metadata = MetaData()
+
+ articles_table = Table('articles', metadata,
Column('article_id', Integer, primary_key = True),
Column('headline', String(150), key='headline'),
Column('body', TEXT, key='body'),
)
- keywords = Table('keywords', engine,
+ keywords_table = Table('keywords', metadata,
Column('keyword_id', Integer, primary_key = True),
Column('keyword_name', String(50))
)
- itemkeywords = Table('article_keywords', engine,
+ itemkeywords_table = Table('article_keywords', metadata,
Column('article_id', Integer, ForeignKey("articles.article_id")),
Column('keyword_id', Integer, ForeignKey("keywords.keyword_id"))
)
+ engine = create_engine('sqlite:///')
+ metadata.create_all(engine)
+
# class definitions
class Keyword(object):
- def __init__(self, name = None):
+ def __init__(self, name):
self.keyword_name = name
class Article(object):
pass
+
+ mapper(Keyword, keywords_table)
# define a mapper that does many-to-many on the 'itemkeywords' association
# table
- Article.mapper = mapper(Article, articles, properties = dict(
- keywords = relation(mapper(Keyword, keywords), itemkeywords, lazy=False)
+ mapper(Article, articles_table, properties = dict(
+ keywords = relation(Keyword, secondary=itemkeywords_table, lazy=False)
)
)
+ session = create_session(bind_to=engine)
+
article = Article()
article.headline = 'a headline'
article.body = 'this is the body'
article.keywords.append(Keyword('politics'))
article.keywords.append(Keyword('entertainment'))
- {sql}objectstore.commit()
+ session.save(article)
+
+ {sql}session.flush()
INSERT INTO keywords (name) VALUES (:name)
{'name': 'politics'}
INSERT INTO keywords (name) VALUES (:name)
[{'keyword_id': 1, 'article_id': 1}, {'keyword_id': 2, 'article_id': 1}]
# select articles based on a keyword. select_by will handle the extra joins.
- {sql}articles = Article.mapper.select_by(keyword_name='politics')
- SELECT articles.article_id AS articles_article_id,
- articles.article_headline AS articles_article_headline,
- articles.article_body AS articles_article_body,
- keywords.keyword_id AS keywords_keyword_id,
- keywords.keyword_name AS keywords_keyword_name
- FROM keywords AS keywords_f008,
- article_keywords AS article_keywords_dbf0,
- articles LEFT OUTER JOIN article_keywords ON
- articles.article_id = article_keywords.article_id
- LEFT OUTER JOIN keywords ON
- keywords.keyword_id = article_keywords.keyword_id
- WHERE (keywords_f008.keyword_name = :keywords_keyword_name
- AND articles.article_id = article_keywords_dbf0.article_id)
- AND keywords_f008.keyword_id = article_keywords_dbf0.keyword_id
- ORDER BY articles.oid, article_keywords.oid
+ {sql}articles = session.query(Article).select_by(keyword_name='politics')
+ SELECT keywords_e2f2.keyword_id AS keywords_e2f2_keyword_id, keywords_e2f2.keyword_name AS keywords_e2f2_keyword_name,
+ articles.headline AS articles_headline, articles.body AS articles_body, articles.article_id AS articles_article_id
+ FROM keywords, article_keywords, articles
+ LEFT OUTER JOIN article_keywords AS article_keyword_3da2 ON articles.article_id = article_keyword_3da2.article_id
+ LEFT OUTER JOIN keywords AS keywords_e2f2 ON keywords_e2f2.keyword_id = article_keyword_3da2.keyword_id
+ WHERE (keywords.keyword_name = :keywords_keywords_name AND articles.article_id = article_keywords.article_id)
+ AND keywords.keyword_id = article_keywords.keyword_id ORDER BY articles.oid, article_keyword_3da2.oid
{'keywords_keyword_name': 'politics'}
- # modify
a = articles[0]
- del a.keywords[:]
+
+ # clear out keywords with a new list
+ a.keywords = []
a.keywords.append(Keyword('topstories'))
a.keywords.append(Keyword('government'))
- # commit. individual INSERT/DELETE operations will take place only for the list
- # elements that changed.
- {sql}objectstore.commit()
+ # flush
+ {sql}session.flush()
INSERT INTO keywords (name) VALUES (:name)
{'name': 'topstories'}
INSERT INTO keywords (name) VALUES (:name)
Many to Many can also be done with an association object, that adds additional information about how two items are related. This association object is set up in basically the same way as any other mapped object. However, since an association table typically has no primary key columns, you have to tell the mapper what columns will compose its "primary key", which are the two (or more) columns involved in the association. Also, the relation function needs an additional hint as to the fact that this mapped object is an association object, via the "association" argument which points to the class or mapper representing the other side of the association.
{python}
+ from sqlalchemy import *
+ metadata = MetaData()
+
+ users_table = Table('users', metadata,
+ Column('user_id', Integer, primary_key = True),
+ Column('user_name', String(16), nullable = False),
+ )
+
+ articles_table = Table('articles', metadata,
+ Column('article_id', Integer, primary_key = True),
+ Column('headline', String(150), key='headline'),
+ Column('body', TEXT, key='body'),
+ )
+
+ keywords_table = Table('keywords', metadata,
+ Column('keyword_id', Integer, primary_key = True),
+ Column('keyword_name', String(50))
+ )
+
# add "attached_by" column which will reference the user who attached this keyword
- itemkeywords = Table('article_keywords', engine,
+ itemkeywords_table = Table('article_keywords', metadata,
Column('article_id', Integer, ForeignKey("articles.article_id")),
Column('keyword_id', Integer, ForeignKey("keywords.keyword_id")),
Column('attached_by', Integer, ForeignKey("users.user_id"))
)
- # define an association class
+ engine = create_engine('sqlite:///', echo=True)
+ metadata.create_all(engine)
+
+ # class definitions
+ class User(object):
+ pass
+ class Keyword(object):
+ def __init__(self, name):
+ self.keyword_name = name
+ class Article(object):
+ pass
class KeywordAssociation(object):
pass
+ mapper(User, users_table)
+ mapper(Keyword, keywords_table)
+
# mapper for KeywordAssociation
# specify "primary key" columns manually
- KeywordAssociation.mapper = mapper(KeywordAssociation, itemkeywords,
- primary_key = [itemkeywords.c.article_id, itemkeywords.c.keyword_id],
+ mapper(KeywordAssociation, itemkeywords_table,
+ primary_key = [itemkeywords_table.c.article_id, itemkeywords_table.c.keyword_id],
properties={
- 'keyword' : relation(Keyword, lazy = False), # uses primary Keyword mapper
- 'user' : relation(User, lazy = True) # uses primary User mapper
+ 'keyword' : relation(Keyword, lazy = False),
+ 'user' : relation(User, lazy = False)
}
)
- # mappers for Users, Keywords
- User.mapper = mapper(User, users)
- Keyword.mapper = mapper(Keyword, keywords)
-
- # define the mapper.
- m = mapper(Article, articles, properties={
- 'keywords':relation(KeywordAssociation.mapper, lazy=False, association=Keyword)
+ # Article mapper, relates to Keyword via KeywordAssociation
+ mapper(Article, articles_table, properties={
+ 'keywords':relation(KeywordAssociation, lazy=False, association=Keyword)
}
)
- # bonus step - well, we do want to load the users in one shot,
- # so modify the mapper via an option.
- # this returns a new mapper with the option switched on.
- m2 = mapper.options(eagerload('keywords.user'))
-
- # select by keyword again
- {sql}alist = m2.select_by(keyword_name='jacks_stories')
- SELECT articles.article_id AS articles_article_id,
- articles.article_headline AS articles_article_headline,
- articles.article_body AS articles_article_body,
- article_keywords.article_id AS article_keywords_article_id,
- article_keywords.keyword_id AS article_keywords_keyword_id,
- article_keywords.attached_by AS article_keywords_attached_by,
- users.user_id AS users_user_id, users.user_name AS users_user_name,
- users.password AS users_password, users.preference_id AS users_preference_id,
- keywords.keyword_id AS keywords_keyword_id, keywords.name AS keywords_name
- FROM article_keywords article_keywords_3a64, keywords keywords_11b7,
- articles LEFT OUTER JOIN article_keywords ON articles.article_id = article_keywords.article_id
- LEFT OUTER JOIN users ON users.user_id = article_keywords.attached_by
- LEFT OUTER JOIN keywords ON keywords.keyword_id = article_keywords.keyword_id
- WHERE keywords_11b7.keyword_id = article_keywords_3a64.keyword_id
- AND article_keywords_3a64.article_id = articles.article_id
- AND keywords_11b7.name = :keywords_name
- ORDER BY articles.oid, article_keywords.oid, users.oid, keywords.oid
- {'keywords_name': 'jacks_stories'}
+ session = create_session(bind_to=engine)
+ # select by keyword
+ {sql}alist = session.query(Article).select_by(keyword_name='jacks_stories')
+ SELECT article_keyword_f9af.keyword_id AS article_keyword_f9af_key_b3e1,
+ article_keyword_f9af.attached_by AS article_keyword_f9af_att_95d4,
+ article_keyword_f9af.article_id AS article_keyword_f9af_art_fd49,
+ users_9c30.user_name AS users_9c30_user_name, users_9c30.user_id AS users_9c30_user_id,
+ keywords_dc54.keyword_id AS keywords_dc54_keyword_id, keywords_dc54.keyword_name AS keywords_dc54_keyword_name,
+ articles.headline AS articles_headline, articles.body AS articles_body, articles.article_id AS articles_article_id
+ FROM keywords, article_keywords, articles
+ LEFT OUTER JOIN article_keywords AS article_keyword_f9af ON articles.article_id = article_keyword_f9af.article_id
+ LEFT OUTER JOIN users AS users_9c30 ON users_9c30.user_id = article_keyword_f9af.attached_by
+ LEFT OUTER JOIN keywords AS keywords_dc54 ON keywords_dc54.keyword_id = article_keyword_f9af.keyword_id
+ WHERE (keywords.keyword_name = :keywords_keywords_name AND keywords.keyword_id = article_keywords.keyword_id)
+ AND articles.article_id = article_keywords.article_id
+ ORDER BY articles.oid, article_keyword_f9af.oid, users_9c30.oid, keywords_dc54.oid
+ {'keywords_keywords_name': 'jacks_stories'}
# user is available
for a in alist:
Database Engines {@name=dbengine}
-================
+============================
-A database engine is a subclass of `sqlalchemy.engine.SQLEngine`, and is the starting point for where SQLAlchemy provides a layer of abstraction on top of the various DBAPI2 database modules. It serves as an abstract factory for database-specific implementation objects as well as a layer of abstraction over the most essential tasks of a database connection, including connecting, executing queries, returning result sets, and managing transactions.
-
-The average developer doesn't need to know anything about the interface or workings of a SQLEngine in order to use it. Simply creating one, and then specifying it when constructing tables and other SQL objects is all that's needed.
-
-A SQLEngine is also a layer of abstraction on top of the connection pooling described in the previous section. While a DBAPI connection pool can be used explicitly alongside a SQLEngine, its not really necessary. Once you have a SQLEngine, you can retrieve pooled connections directly from its underlying connection pool via its own `connection()` method. However, if you're exclusively using SQLALchemy's SQL construction objects and/or object-relational mappers, all the details of connecting are handled by those libraries automatically.
-
+A database engine is a subclass of `sqlalchemy.sql.Engine`, and is the starting point for where SQLAlchemy provides a layer of abstraction on top of the various DBAPI2 database modules. For all databases supported by SA, there is a specific "implementation" module, found in the `sqlalchemy.databases` package, that provides all the objects an `Engine` needs in order to perform its job. A typical user of SQLAlchemy never needs to deal with these modules directly. For many purposes, the only knowledge that's needed is how to create an Engine for a particular connection URL. When dealing with direct execution of SQL statements, one would also be aware of Result, Connection, and Transaction objects. The primary public facing objects are:
-### Establishing a Database Engine {@name=establishing}
-
-Engines exist for SQLite, Postgres, MySQL, MS-SQL, and Oracle, using the Pysqlite, Psycopg (1 or 2), MySQLDB, adodbapi or pymssql, and cx_Oracle modules (there is also experimental support for Firebird). Each engine imports its corresponding module which is required to be installed. For Postgres and Oracle, an alternate module may be specified at construction time as well.
-
-The string based argument names for connecting are translated to the appropriate names when the connection is made; argument names include "host" or "hostname" for database host, "database", "db", or "dbname" for the database name (also is dsn for Oracle), "user" or "username" for the user, and "password", "pw", or "passwd" for the password. SQLite expects "filename" or "file" for the filename, or if None it defaults to "":memory:".
+* **URL** - represents the identifier for a particular database. URL objects are usually created automatically based on a given connect string passed to the `create_engine()` function.
+* **Engine** - Combines a connection-providing resource with implementation-provided objects that know how to generate, execute, and gather information about SQL statements. It also provides the primary interface by which Connections are obtained, as well as a context for constructed SQL objects and schema constructs to "implicitly execute" themselves, which is an optional feature of SA 0.2. The Engine object that is normally dealt with is an instance of `sqlalchemy.engine.base.ComposedSQLEngine`.
+* **Connection** - represents a connection to the database. The underlying connection object returned by a DBAPI's connect() method is referenced internally by the Connection object. Connection provides methods that handle the execution of SQLAlchemy's own SQL constructs, as well as literal string-based statements.
+* **Transaction** - represents a transaction on a single Connection. Includes `begin()`, `commit()` and `rollback()` methods that support basic "nestable" behavior, meaning an outermost transaction is maintained against multiple nested calls to begin/commit.
+* **ResultProxy** - Represents the results of an execution, and is most analgous to the cursor object in DBAPI. It primarily allows iteration over result sets, but also provides an interface to information about inserts/updates/deletes, such as the count of rows affected, last inserted IDs, etc.
+* **RowProxy** - Represents a single row returned by the fetchone() method on ResultProxy.
-The connection arguments can be specified as a string + dictionary pair, or a single URL-encoded string, as follows:
-
- {python}from sqlalchemy import *
+Underneath the public-facing API of `ComposedSQLEngine`, several components are provided by database implementations to provide the full behavior, including:
- # sqlite in memory
- sqlite_engine = create_engine('sqlite', {'filename':':memory:'}, **opts)
+* **Dialect** - this object is provided by database implementations to describe the behavior of a particular database. It acts as a repository for metadata about a database's characteristics, and provides factory methods for other objects that deal with generating SQL strings and objects that handle some of the details of statement execution.
+* **ConnectionProvider** - this object knows how to return a DBAPI connection object. It typically talks to a connection pool which maintains one or more connections in memory for quick re-use.
+* **ExecutionContext** - this object is created for each execution of a single SQL statement, and tracks information about its execution such as primary keys inserted, the total count of rows affected, etc. It also may implement any special logic that various DBAPI implementations may require before or after a statement execution.
+* **Compiler** - receives SQL expression objects and assembles them into strings that are suitable for direct execution, as well as collecting bind parameters into a dictionary or list to be sent along with the statement.
+* **SchemaGenerator** - receives collections of Schema objects and knows how to generate the appropriate SQL for `CREATE` and `DROP` statements.
- # via URL
- sqlite_engine = create_engine('sqlite://', **opts)
-
- # sqlite using a file
- sqlite_engine = create_engine('sqlite', {'filename':'querytest.db'}, **opts)
-
- # via URL
- sqlite_engine = create_engine('sqlite://filename=querytest.db', **opts)
-
- # postgres
- postgres_engine = create_engine('postgres',
- {'database':'test',
- 'host':'127.0.0.1',
- 'user':'scott',
- 'password':'tiger'}, **opts)
-
- # via URL
- postgres_engine = create_engine('postgres://database=test&host=127.0.0.1&user=scott&password=tiger')
-
- # mysql
- mysql_engine = create_engine('mysql',
- {
- 'db':'mydb',
- 'user':'scott',
- 'passwd':'tiger',
- 'host':'127.0.0.1'
- }
- **opts)
- # oracle
- oracle_engine = create_engine('oracle',
- {'dsn':'mydsn',
- 'user':'scott',
- 'password':'tiger'}, **opts)
-
-
-
-Note that the general form of connecting to an engine is:
+### Supported Databases {@name=supported}
- {python}# separate arguments
- engine = create_engine(
- <enginename>,
- {<named DBAPI arguments>},
- <sqlalchemy options>
- )
+Engines exist for SQLite, Postgres, MySQL, and Oracle, using the Pysqlite, Psycopg (1 or 2), MySQLDB, and cx_Oracle modules. There is also preliminary support for MS-SQL using adodbapi or pymssql, as well as Firebird. For each engine, a distinct Python module exists in the `sqlalchemy.databases` package, which provides implementations of some of the objects mentioned in the previous section.
- # url
- engine = create_engine('<enginename>://<named DBAPI arguments>', <sqlalchemy options>)
+### Establishing a Database Engine {@name=establishing}
-### Database Engine Methods {@name=methods}
+SQLAlchemy 0.2 indicates the source of an Engine strictly via [RFC-1738](http://rfc.net/rfc1738.html) style URLs, combined with optional keyword arguments to specify options for the Engine. The form of the URL is:
-A few useful methods off the SQLEngine are described here:
+ $ driver://username:password@host:port/database
- {python}engine = create_engine('postgres://hostname=localhost&user=scott&password=tiger&database=test')
+Available drivernames are `sqlite`, `mysql`, `postgres`, `oracle`, `mssql`, and `firebird`. For sqlite, the database name is the filename to connect to, or the special name ":memory:" which indicates an in-memory database. The URL is typically sent as a string to the `create_engine()` function:
- # get a pooled DBAPI connection
- conn = engine.connection()
+ {python}
+ pg_db = create_engine('postgres://scott:tiger@localhost:5432/mydatabase')
+ sqlite_db = create_engine('sqlite:///mydb.txt')
+ mysql_db = create_engine('mysql://localhost/foo')
+ oracle_db = create_engine('oracle://scott:tiger@dsn')
- # create/drop tables based on table metadata objects
- # (see the next section, Table Metadata, for info on table metadata)
- engine.create(mytable)
- engine.drop(mytable)
+The `Engine` will create its first connection to the database when a SQL statement is executed. As concurrent statements are executed, the underlying connection pool will grow to a default size of five connections, and will allow a default "overflow" of ten. Since the `Engine` is essentially "home base" for the connection pool, it follows that you should keep a single `Engine` per database established within an application, rather than creating a new one for each connection.
- # get the DBAPI module being used
- dbapi = engine.dbapi()
+### Database Engine Options {@name=options}
- # get the default schema name
- name = engine.get_default_schema_name()
+Keyword options can also be specified to `create_engine()`, following the string URL as follows:
- # execute some SQL directly, returns a ResultProxy (see the SQL Construction section for details)
- result = engine.execute("select * from table where col1=:col1", {'col1':'foo'})
+ {python}
+ db = create_engine('postgres://...', encoding='latin1', echo=True, module=psycopg1)
- # log a message to the engine's log stream
- engine.log('this is a message')
-
-### Database Engine Options {@name=options}
+Options that can be specified include the following:
-The remaining arguments to `create_engine` are keyword arguments that are passed to the specific subclass of `sqlalchemy.engine.SQLEngine` being used, as well as the underlying `sqlalchemy.pool.Pool` instance. All of the options described in the previous section [pooling_configuration](rel:pooling_configuration) can be specified, as well as engine-specific options:
-
-* pool=None : an instance of `sqlalchemy.pool.Pool` to be used as the underlying source for connections, overriding the engine's connect arguments (pooling is described in the previous section). If None, a default Pool (QueuePool or SingletonThreadPool as appropriate) will be created using the engine's connect arguments.
+* strategy='plain' : the Strategy describes the general configuration used to create this Engine. The two available values are `plain`, which is the default, and `threadlocal`, which applies a "thread-local context" to implicit executions performed by the Engine. This context is further described in [dbengine_connections_context](rel:dbengine_connections_context).
+* pool=None : an instance of `sqlalchemy.pool.Pool` to be used as the underlying source for connections, overriding the engine's connect arguments (pooling is described in [pooling](rel:pooling)). If None, a default `Pool` (usually `QueuePool`, or `SingletonThreadPool` in the case of SQLite) will be created using the engine's connect arguments.
Example:
- {python}from sqlalchemy import *
+ {python}
+ from sqlalchemy import *
import sqlalchemy.pool as pool
import MySQLdb
engine = create_engine('mysql', pool=pool.QueuePool(getconn, pool_size=20, max_overflow=40))
-* echo=False : if True, the SQLEngine will log all statements as well as a repr() of their parameter lists to the engines logger, which defaults to sys.stdout. A SQLEngine instances' "echo" data member can be modified at any time to turn logging on and off. If set to the string 'debug', result rows will be printed to the standard output as well.
-* logger=None : a file-like object where logging output can be sent, if echo is set to True. This defaults to sys.stdout.
-* module=None : used by Oracle and Postgres, this is a reference to a DBAPI2 module to be used instead of the engine's default module. For Postgres, the default is psycopg2, or psycopg1 if 2 cannot be found. For Oracle, its cx_Oracle.
-* default_ordering=False : if True, table objects and associated joins and aliases will generate information used for ordering by primary keys (or OIDs, if the database supports OIDs). This information is used by the Mapper system to when it constructs select queries to supply a default ordering to mapped objects.
-* use_ansi=True : used only by Oracle; when False, the Oracle driver attempts to support a particular "quirk" of some Oracle databases, that the LEFT OUTER JOIN SQL syntax is not supported, and the "Oracle join" syntax of using <column1>(+)=<column2> must be used in order to achieve a LEFT OUTER JOIN. Its advised that the Oracle database be configured to have full ANSI support instead of using this feature.
-* use_oids=False : used only by Postgres, will enable the column name "oid" as the object ID column. Postgres as of 8.1 has object IDs disabled by default.
-* convert_unicode=False : if set to True, all String/character based types will convert Unicode values to raw byte values going into the database, and all raw byte values to Python Unicode coming out in result sets. This is an engine-wide method to provide unicode across the board. For unicode conversion on a column-by-column level, use the Unicode column type instead.
-* encoding='utf-8' : the encoding to use for Unicode translations - passed to all encode/decode methods.
-* echo_uow=False : when True, logs unit of work commit plans to the standard output.
+* pool_size=5 : the number of connections to keep open inside the connection pool. This is only used with `QueuePool`.
+* max_overflow=10 : the number of connections to allow in "overflow", that is connections that can be opened above and beyond the initial five. this is only used with `QueuePool`.
+* pool_timeout=30 : number of seconds to wait before giving up on getting a connection from the pool. This is only used with `QueuePool`.
+* echo=False : if True, the Engine will log all statements as well as a repr() of their parameter lists to the engines logger, which defaults to sys.stdout. The `echo` attribute of `ComposedSQLEngine` can be modified at any time to turn logging on and off. If set to the string `"debug"`, result rows will be printed to the standard output as well.
+* logger=None : a file-like object where logging output can be sent, if echo is set to True. Newlines will not be sent with log messages. This defaults to an internal logging object which references `sys.stdout`.
+* module=None : used by database implementations which support multiple DBAPI modules, this is a reference to a DBAPI2 module to be used instead of the engine's default module. For Postgres, the default is psycopg2, or psycopg1 if 2 cannot be found. For Oracle, its cx_Oracle.
+* use_ansi=True : used only by Oracle; when False, the Oracle driver attempts to support a particular "quirk" of Oracle versions 8 and previous, that the LEFT OUTER JOIN SQL syntax is not supported, and the "Oracle join" syntax of using `<column1>(+)=<column2>` must be used in order to achieve a LEFT OUTER JOIN.
+* threaded=True : used by cx_Oracle; sets the `threaded` parameter of the connection indicating thread-safe usage. cx_Oracle docs indicate setting this flag to `False` will speed performance by 10-15%. While this defaults to `False` in cx_Oracle, SQLAlchemy defaults it to `True`, preferring stability over early optimization.
+* use_oids=False : used only by Postgres, will enable the column name "oid" as the object ID column, which is also used for the default sort order of tables. Postgres as of 8.1 has object IDs disabled by default.
+* convert_unicode=False : if set to True, all String/character based types will convert Unicode values to raw byte values going into the database, and all raw byte values to Python Unicode coming out in result sets. This is an engine-wide method to provide unicode across the board. For unicode conversion on a column-by-column level, use the `Unicode` column type instead.
+* encoding='utf-8' : the encoding to use for all Unicode translations, both by engine-wide unicode conversion as well as the `Unicode` type object.
+
+### Using Connections {@name=connections}
+
+In this section we describe the SQL execution interface available from an `Engine` instance. Note that when using the Object Relational Mapper (ORM) as well as when dealing with with "bound" metadata objects (described later), SQLAlchemy deals with the Engine for you and you generally don't need to know much about it; in those cases, you can skip this section and go to [metadata](rel:metadata).
+
+The Engine provides a `connect()` method which returns a `Connection` object. This object provides methods by which literal SQL text as well as SQL clause constructs can be compiled and executed.
+
+ {python}
+ engine = create_engine('sqlite:///:memory:')
+ connection = engine.connect()
+ result = connection.execute("select * from mytable where col1=:col1", col1=5)
+ for row in result:
+ print row['col1'], row['col2']
+ connection.close()
+
+The `close` method on `Connection` does not actually remove the underlying connection to the database, but rather indicates that the underlying resources can be returned to the connection pool. When using the `connect()` method, the DBAPI connection referenced by the `Connection` object is not referenced anywhere else.
+
+
+In both execution styles above, the `Connection` object will also automatically return its resources to the connection pool when the object is garbage collected, i.e. its `__del__()` method is called. When using the standard C implementation of Python, this method is usually called immediately as soon as the object is dereferenced. With other Python implementations such as Jython, this is not so guaranteed.
-### Using the Proxy Engine {@name=proxy}
-
-The ProxyEngine is useful for applications that need to swap engines
-at runtime, or to create their tables and mappers before they know
-what engine they will use. One use case is an application meant to be
-pluggable into a mix of other applications, such as a WSGI
-application. Well-behaved WSGI applications should be relocatable; and
-since that means that two versions of the same application may be
-running in the same process (or in the same thread at different
-times), WSGI applications ought not to depend on module-level or
-global configuration. Using the ProxyEngine allows a WSGI application
-to define tables and mappers in a module, but keep the specific
-database connection uri as an application instance or thread-local
-value.
-
-The ProxyEngine is used in the same way as any other engine, with one
-additional method:
+The execute method on `Engine` and `Connection` can also receive SQL clause constructs as well, which are described in [sql](rel:sql):
+
+ {python}
+ connection = engine.connect()
+ result = connection.execute(select([table1], table1.c.col1==5))
+ for row in result:
+ print row['col1'], row['col2']
+ connection.close()
+
+Both `Connection` and `Engine` fulfill an interface known as `Connectable` which specifies common functionality between the two objects, such as getting a `Connection` and executing queries. Therefore, most SQLAlchemy functions which take an `Engine` as a parameter with which to execute SQL will also accept a `Connection`:
+
+ {python title="Specify Engine or Connection"}
+ engine = create_engine('sqlite:///:memory:')
- {python}# define the tables and mappers
- from sqlalchemy import *
- from sqlalchemy.ext.proxy import ProxyEngine
+ # specify some Table metadata
+ metadata = MetaData()
+ table = Table('sometable', metadata, Column('col1', Integer))
- engine = ProxyEngine()
+ # create the table with the Engine
+ table.create(engine=engine)
- users = Table('users', engine, ... )
+ # drop the table with a Connection off the Engine
+ connection = engine.connect()
+ table.drop(engine=connection)
+
+#### Implicit Connection Contexts {@name=context}
+
+An **implicit connection** refers to connections that are allocated by the `Engine` internally. There are two general cases when this occurs: when using the various `execute()` methods that are available off the `Engine` object itself, and when calling the `execute()` method on constructed SQL objects, which are described in [sqlconstruction](rel:sqlconstruction).
+
+ {python title="Implicit Connection"}
+ engine = create_engine('sqlite:///:memory:')
+ result = engine.execute("select * from mytable where col1=:col1", col1=5)
+ for row in result:
+ print row['col1'], row['col2']
+ result.close()
+
+When using implicit connections, the returned `ResultProxy` has a `close()` method which will return the resources used by the underlying `Connection`.
+
+The `strategy` keyword argument to `create_engine()` affects the algorithm used to retreive the underlying DBAPI connection used by implicit executions. When set to `plain`, each implicit execution requests a unique connection from the connection pool, which is returned to the pool when the resulting `ResultProxy` falls out of scope (i.e. `__del__()` is called) or its `close()` method is called. If a second implicit execution occurs while the `ResultProxy` from the previous execution is still open, then a second connection is pulled from the pool.
+
+When `strategy` is set to `threadlocal`, the `Engine` still checks out a connection which is closeable in the same manner via the `ResultProxy`, except the connection it checks out will be the **same** connection as one which is already checked out, assuming the operation is in the same thread. When all `ResultProxy` objects are closed, the connection is returned to the pool normally.
+
+It is crucial to note that the `plain` and `threadlocal` contexts **do not impact the connect() method on the Engine.** `connect()` always returns a unique connection. Implicit connections use a different method off of `Engine` for their operations called `contextual_connect()`.
+
+The `plain` strategy is better suited to an application that insures the explicit releasing of the resources used by each execution. This is because each execution uses its own distinct connection resource, and as those resources remain open, multiple connections can be checked out from the pool quickly. Since the connection pool will block further requests when too many connections have been checked out, not keeping track of this can impact an application's stability.
+
+ {python title="Plain Strategy"}
+ db = create_engine('mysql://localhost/test', strategy='plain')
- class Users(object):
- pass
-
- assign_mapper(Users, users)
+ # execute one statement and receive results. r1 now references a DBAPI connection resource.
+ r1 = db.execute("select * from table1")
- def app(environ, start_response):
- # later, connect the proxy engine to a real engine via the connect() method
- engine.connect(environ['db_uri'])
- # now you have a real db connection and can select, insert, etc.
+ # execute a second statement and receive results. r2 now references a *second* DBAPI connection resource.
+ r2 = db.execute("select * from table2")
+ for row in r1:
+ ...
+ for row in r2:
+ ...
+ # release connection 1
+ r1.close()
+ # release connection 2
+ r2.close()
-#### Using the Global Proxy {@name=defaultproxy}
-
-There is an instance of ProxyEngine available within the schema package as `default_engine`. You can construct Table objects and not specify the engine parameter, and they will connect to this engine by default. To connect the default_engine, use the `global_connect` function.
+Advantages to `plain` include that connection resources are immediately returned to the connection pool, without any reliance upon the `__del__()` method; there is no chance of resources being left around by a Python implementation that doesn't necessarily call `__del__()` immediately.
- {python}# define the tables and mappers
- from sqlalchemy import *
+The `threadlocal` strategy is better suited to a programming style which relies upon the `__del__()` method of Connection objects in order to return them to the connection pool, rather than explicitly issuing a `close()` statement upon the `ResultProxy` object. This is because all of the executions within a single thread will share the same connection, if one has already been checked out in the current thread. Using this style, an application will use only one connection per thread at most within the scope of all implicit executions.
+
+ {python title="Threadlocal Strategy"}
+ db = create_engine('mysql://localhost/test', strategy='threadlocal')
- # specify a table with no explicit engine
- users = Table('users',
- Column('user_id', Integer, primary_key=True),
- Column('user_name', String)
- )
+ # execute one statement and receive results. r1 now references a DBAPI connection resource.
+ r1 = db.execute("select * from table1")
- # connect the global proxy engine
- global_connect('sqlite://filename=foo.db')
+ # execute a second statement and receive results. r2 now references the *same* resource as r1
+ r2 = db.execute("select * from table2")
- # create the table in the selected database
- users.create()
+ for row in r1:
+ ...
+ for row in r2:
+ ...
+ # dereference r1. the connection is still held by r2.
+ r1 = None
+ # dereference r2. with no more references to the underlying connection resources, they
+ # are returned to the pool.
+ r2 = None
-### Transactions {@name=transactions}
+While the `close()` method is still available with the "threadlocal" strategy, it should be used carefully. Above, if we issued a `close()` call on `r1`, and then tried to further work with results from `r2`, `r2` would be in an invalid state since its connection was already returned to the pool. By relying on `__del__()` to automatically clean up resources, this condition will never occur.
-A SQLEngine also provides an interface to the transactional capabilities of the underlying DBAPI connection object, as well as the connection object itself. Note that when using the object-relational-mapping package, described in a later section, basic transactional operation is handled for you automatically by its "Unit of Work" system; the methods described here will usually apply just to literal SQL update/delete/insert operations or those performed via the SQL construction library.
-
-Typically, a connection is opened with `autocommit=False`. So to perform SQL operations and just commit as you go, you can simply pull out a connection from the connection pool, keep it in the local scope, and call commit() on it as needed. As long as the connection remains referenced, all other SQL operations within the same thread will use this same connection, including those used by the SQL construction system as well as the object-relational mapper, both described in later sections:
+Advantages to `threadlocal` include that resources can be left to clean up after themselves, application code can be more minimal, its guaranteed that only one connection is used per thread, and there is no chance of a "connection pool block", which is when an execution hangs because the current thread has already checked out all remaining resources.
- {python}conn = engine.connection()
-
- # execute SQL via the engine
- engine.execute("insert into mytable values ('foo', 'bar')")
- conn.commit()
-
- # execute SQL via the SQL construction library
- mytable.insert().execute(col1='bat', col2='lala')
- conn.commit()
-
-There is a more automated way to do transactions, and that is to use the engine's begin()/commit() functionality. When the begin() method is called off the engine, a connection is checked out from the pool and stored in a thread-local context. That way, all subsequent SQL operations within the same thread will use that same connection. Subsequent commit() or rollback() operations are performed against that same connection. In effect, its a more automated way to perform the "commit as you go" example above.
+To get at the actual `Connection` object which is used by implicit executions, call the `contextual_connection()` method on `Engine`:
+
+ {python title="Contextual Connection"}
+ # threadlocal strategy
+ db = create_engine('mysql://localhost/test', strategy='threadlocal')
- {python}engine.begin()
- engine.execute("insert into mytable values ('foo', 'bar')")
- mytable.insert().execute(col1='foo', col2='bar')
- engine.commit()
-
+ conn1 = db.contextual_connection()
+ conn2 = db.contextual_connection()
-A traditional "rollback on exception" pattern looks like this:
+ >>> assert conn1 is conn2
+ True
- {python}engine.begin()
+When the `plain` strategy is used, the `contextual_connection()` method is synonymous with the `connect()` method; both return a distinct connection from the pool.
+
+### Transactions {@name=transactions}
+
+The `Connection` object provides a `begin()` method which returns a `Transaction` object. This object is usually used within a try/except clause so that it is guaranteed to `rollback()` or `commit()`:
+
+ {python}
+ trans = connection.begin()
try:
- engine.execute("insert into mytable values ('foo', 'bar')")
- mytable.insert().execute(col1='foo', col2='bar')
+ r1 = connection.execute(table1.select())
+ connection.execute(table1.insert(), col1=7, col2='this is some data')
+ trans.commit()
except:
- engine.rollback()
+ trans.rollback()
raise
- engine.commit()
-
-
-An shortcut which is equivalent to the above is provided by the `transaction` method:
- {python}def do_stuff():
- engine.execute("insert into mytable values ('foo', 'bar')")
- mytable.insert().execute(col1='foo', col2='bar')
+The `Transaction` object also handles "nested" behavior by keeping track of the outermost begin/commit pair. In this example, two functions both issue a transaction on a Connection, but only the outermost Transaction object actually takes effect when it is committed.
- engine.transaction(do_stuff)
-
-An added bonus to the engine's transaction methods is "reentrant" functionality; once you call begin(), subsequent calls to begin() will increment a counter that must be decremented corresponding to each commit() statement before an actual commit can happen. This way, any number of methods that want to insure a transaction can call begin/commit, and be nested arbitrarily:
-
- {python}# method_a starts a transaction and calls method_b
- def method_a():
- engine.begin()
+ {python}
+ # method_a starts a transaction and calls method_b
+ def method_a(connection):
+ trans = connection.begin() # open a transaction
try:
- method_b()
+ method_b(connection)
+ trans.commit() # transaction is committed here
except:
- engine.rollback()
+ trans.rollback() # this rolls back the transaction unconditionally
raise
- engine.commit()
- # method_b starts a transaction, or joins the one already in progress,
- # and does some SQL
- def method_b():
- engine.begin()
+ # method_b also starts a transaction
+ def method_b(connection):
+ trans = connection.begin() # open a transaction - this runs in the context of method_a's transaction
try:
- engine.execute("insert into mytable values ('bat', 'lala')")
- mytable.insert().execute(col1='bat', col2='lala')
+ connection.execute("insert into mytable values ('bat', 'lala')")
+ connection.execute(mytable.insert(), col1='bat', col2='lala')
+ trans.commit() # transaction is not committed yet
except:
- engine.rollback()
+ trans.rollback() # this rolls back the transaction unconditionally
raise
- engine.commit()
- # call method_a
- method_a()
+ # open a Connection and call method_a
+ conn = engine.connect()
+ method_a(conn)
+ conn.close()
-Above, `method_a` is called first, which calls `engine.begin()`. Then it calls `method_b`. When `method_b` calls `engine.begin()`, it just increments a counter that is decremented when it calls `commit()`. If either `method_a` or `method_b` calls `rollback()`, the whole transaction is rolled back. The transaction is not committed until `method_a` calls the `commit()` method.
+Above, `method_a` is called first, which calls `connection.begin()`. Then it calls `method_b`. When `method_b` calls `connection.begin()`, it just increments a counter that is decremented when it calls `commit()`. If either `method_a` or `method_b` calls `rollback()`, the whole transaction is rolled back. The transaction is not committed until `method_a` calls the `commit()` method.
-The object-relational-mapper capability of SQLAlchemy includes its own `commit()` method that gathers SQL statements into a batch and runs them within one transaction. That transaction is also invokved within the scope of the "reentrant" methodology above; so multiple objectstore.commit() operations can also be bundled into a larger database transaction via the above methodology.
-
+Note that SQLAlchemy's Object Relational Mapper also provides a way to control transaction scope at a higher level; this is described in [unitofwork_transaction](rel:unitofwork_transaction).
<%python scope="global">
files = [
- #'tutorial',
- 'trailmap',
- 'pooling',
+ 'tutorial',
'dbengine',
'metadata',
'sqlconstruction',
'unitofwork',
'adv_datamapping',
'types',
+ 'pooling',
+ 'plugins',
'docstrings',
]
wrapper='section_wrapper.myt'
onepage='documentation'
index='index'
- title='SQLAlchemy 0.1 Documentation'
- version = '0.1.7'
+ title='SQLAlchemy 0.2 Documentation'
+ version = '0.2.0'
</%attr>
<%method title>
-
Database Meta Data {@name=metadata}
==================
-### Describing Tables with MetaData {@name=tables}
+### Describing Databases with MetaData {@name=tables}
-The core of SQLAlchemy's query and object mapping operations is table metadata, which are Python objects that describe tables. Metadata objects can be created by explicitly naming the table and all its properties, using the Table, Column, ForeignKey, and Sequence objects imported from `sqlalchemy.schema`, and a database engine constructed as described in the previous section, or they can be automatically pulled from an existing database schema. First, the explicit version:
+The core of SQLAlchemy's query and object mapping operations is database metadata, which are Python objects that describe tables and other schema-level objects. Metadata objects can be created by explicitly naming the various components and their properties, using the Table, Column, ForeignKey, Index, and Sequence objects imported from `sqlalchemy.schema`. There is also support for *reflection*, which means you only specify the *name* of the entities and they are recreated from the database automatically.
+
+A collection of metadata entities is stored in an object aptly named `MetaData`. This object takes an optional `name` parameter:
{python}
from sqlalchemy import *
- engine = create_engine('sqlite', {'filename':':memory:'}, **opts)
- users = Table('users', engine,
+ metadata = MetaData(name='my metadata')
+
+Then to construct a Table, use the `Table` class:
+
+ {python}
+ users = Table('users', metadata,
Column('user_id', Integer, primary_key = True),
Column('user_name', String(16), nullable = False),
Column('email_address', String(60), key='email'),
Column('password', String(20), nullable = False)
)
- user_prefs = Table('user_prefs', engine,
+ user_prefs = Table('user_prefs', metadata,
Column('pref_id', Integer, primary_key=True),
Column('user_id', Integer, ForeignKey("users.user_id"), nullable=False),
Column('pref_name', String(40), nullable=False),
Column('pref_value', String(100))
)
+
+The specific datatypes for each Column, such as Integer, String, etc. are described in [types](rel:types), and exist within the module `sqlalchemy.types` as well as the global `sqlalchemy` namespace.
+
+The `MetaData` object supports some handy methods, such as getting a list of Tables in the order (or reverse) of their dependency:
+
+ {python}
+ >>> for t in metadata.table_iterator(reverse=False):
+ ... print t.name
+ users
+ user_prefs
-The specific datatypes, such as Integer, String, etc. are defined in [types](rel:types) and are automatically pulled in when you import * from `sqlalchemy`. Note that for Column objects, an altername name can be specified via the "key" parameter; if this parameter is given, then all programmatic references to this Column object will be based on its key, instead of its actual column name.
-
-Once constructed, the Table object provides a clean interface to the table's properties as well as that of its columns:
+And `Table` provides an interface to the table's properties as well as that of its columns:
{python}
- employees = Table('employees', engine,
+ employees = Table('employees', metadata,
Column('employee_id', Integer, primary_key=True),
Column('employee_name', String(60), nullable=False, key='name'),
Column('employee_dept', Integer, ForeignKey("departments.department_id"))
for fkey in employees.foreign_keys:
# ...
- # access the table's SQLEngine object:
+ # access the table's MetaData:
+ employees.metadata
+
+ # access the table's Engine, if its MetaData is bound:
employees.engine
# access a column's name, type, nullable, primary key, foreign key
# get the table related by a foreign key
fcolumn = employees.c.employee_dept.foreign_key.column.table
-
-Metadata objects can also be <b>reflected</b> from tables that already exist in the database. Reflection means based on a table name, the names, datatypes, and attributes of all columns, including foreign keys, will be loaded automatically. This feature is supported by all database engines:
+
+#### Binding MetaData to an Engine {@name=binding}
+
+A MetaData object can be associated with one or more Engine instances. This allows the MetaData and the elements within it to perform operations automatically, using the connection resources of that Engine. This includes being able to "reflect" the columns of tables, as well as to perform create and drop operations without needing to pass an `Engine` or `Connection` around. It also allows SQL constructs to be created which know how to execute themselves (called "implicit execution").
+
+To bind `MetaData` to a single `Engine`, use `BoundMetaData`:
+
+ {python}
+ engine = create_engine('sqlite://', **kwargs)
+
+ # create BoundMetaData from an Engine
+ meta = BoundMetaData(engine)
+
+ # create the Engine and MetaData in one step
+ meta = BoundMetaData('postgres://db/', **kwargs)
+
+Another form of `MetaData` exists which allows connecting to any number of engines, within the context of the current thread. This is `DynamicMetaData`:
+
+ {python}
+ meta = DynamicMetaData()
+
+ meta.connect(engine) # connect to an existing Engine
+
+ meta.connect('mysql://user@host/dsn') # create a new Engine and connect
+
+`DynamicMetaData` is ideal for applications that need to use the same set of `Tables` for many different database connections in the same process, such as a CherryPy web application which handles multiple application instances in one process.
+
+#### Reflecting Tables
+
+Once you have a `BoundMetaData` or a connected `DynamicMetaData`, you can create `Table` objects without specifying their columns, just their names, using `autoload=True`:
{python}
- >>> messages = Table('messages', engine, autoload = True)
+ >>> messages = Table('messages', meta, autoload = True)
>>> [c.name for c in messages.columns]
['message_id', 'message_name', 'date']
-
+
+At the moment the Table is constructed, it will query the database for the columns and constraints of the `messages` table.
+
Note that if a reflected table has a foreign key referencing another table, then the metadata for the related table will be loaded as well, even if it has not been defined by the application:
{python}
- >>> shopping_cart_items = Table('shopping_cart_items', engine, autoload = True)
+ >>> shopping_cart_items = Table('shopping_cart_items', meta, autoload = True)
>>> print shopping_cart_items.c.cart_id.table.name
shopping_carts
To get direct access to 'shopping_carts', simply instantiate it via the Table constructor. You'll get the same instance of the shopping cart Table as the one that is attached to shopping_cart_items:
{python}
- >>> shopping_carts = Table('shopping_carts', engine)
+ >>> shopping_carts = Table('shopping_carts', meta)
>>> shopping_carts is shopping_cart_items.c.cart_id.table.name
True
-This works because when the Table constructor is called for a particular name and database engine, if the table has already been created then the instance returned will be the same as the original. This is a <b>singleton</b> constructor:
+This works because when the Table constructor is called for a particular name and `MetaData` object, if the table has already been created then the instance returned will be the same as the original. This is a <b>singleton</b> constructor:
{python}
- >>> news_articles = Table('news', engine,
+ >>> news_articles = Table('news', meta,
... Column('article_id', Integer, primary_key = True),
... Column('url', String(250), nullable = False)
... )
- >>> othertable = Table('news', engine)
+ >>> othertable = Table('news', meta)
>>> othertable is news_articles
True
-
+
+#### Specifying the Schema Name {@name=schema}
+
+Some databases support the concept of multiple schemas. A `Table` can reference this by specifying the `schema` keyword argument:
+
+ {python}
+ financial_info = Table('financial_info', meta,
+ Column('id', Integer, primary_key=True),
+ Column('value', String(100), nullable=False),
+ schema='remote_banks'
+ )
+
+Within the `MetaData` collection, this table will be identified by the combination of `financial_info` and `remote_banks`. If another table called `financial_info` is referenced without the `remote_banks` schema, it will refer to a different `Table`. `ForeignKey` objects can reference columns in this table using the form `remote_banks.financial_info.id`.
+
+#### Other Options {@name=options}
+
+`Tables` may support database-specific options, such as MySQL's `engine` option that can specify "MyISAM", "InnoDB", and other backends for the table:
+
+ {python}
+ addresses = Table('engine_email_addresses', meta,
+ Column('address_id', Integer, primary_key = True),
+ Column('remote_user_id', Integer, ForeignKey(users.c.user_id)),
+ Column('email_address', String(20)),
+ mysql_engine='InnoDB'
+ )
+
### Creating and Dropping Database Tables {@name=creating}
-Creating and dropping is easy, just use the `create()` and `drop()` methods:
+Creating and dropping individual tables can be done via the `create()` and `drop()` methods of `Table`; these methods take an optional `engine` parameter which references an `Engine` or a `Connection`. If not supplied, the `Engine` bound to the `MetaData` will be used, else an error is raised:
{python}
- employees = Table('employees', engine,
+ meta = BoundMetaData('sqlite:///:memory:')
+ employees = Table('employees', meta,
Column('employee_id', Integer, primary_key=True),
Column('employee_name', String(60), nullable=False, key='name'),
Column('employee_dept', Integer, ForeignKey("departments.department_id"))
employee_dept INTEGER REFERENCES departments(department_id)
)
{}
+
+`drop()` method:
- {sql}employees.drop()
+ {python}
+ {sql}employees.drop(engine=e)
DROP TABLE employees
{}
+
+Entire groups of Tables can be created and dropped directly from the `MetaData` object with `create_all()` and `drop_all()`, each of which take an optional `engine` keyword argument which can reference an `Engine` or a `Connection`, else the underlying bound `Engine` is used:
+
+ {python}
+ engine = create_engine('sqlite:///:memory:')
+
+ metadata = MetaData()
+
+ users = Table('users', metadata,
+ Column('user_id', Integer, primary_key = True),
+ Column('user_name', String(16), nullable = False),
+ Column('email_address', String(60), key='email'),
+ Column('password', String(20), nullable = False)
+ )
+ user_prefs = Table('user_prefs', metadata,
+ Column('pref_id', Integer, primary_key=True),
+ Column('user_id', Integer, ForeignKey("users.user_id"), nullable=False),
+ Column('pref_name', String(40), nullable=False),
+ Column('pref_value', String(100))
+ )
+
+ {sql}metadata.create_all(engine=engine)
+ PRAGMA table_info(users){}
+ CREATE TABLE users(
+ user_id INTEGER NOT NULL PRIMARY KEY,
+ user_name VARCHAR(16) NOT NULL,
+ email_address VARCHAR(60),
+ password VARCHAR(20) NOT NULL
+ )
+ PRAGMA table_info(user_prefs){}
+ CREATE TABLE user_prefs(
+ pref_id INTEGER NOT NULL PRIMARY KEY,
+ user_id INTEGER NOT NULL REFERENCES users(user_id),
+ pref_name VARCHAR(40) NOT NULL,
+ pref_value VARCHAR(100)
+ )
+
### Column Defaults and OnUpdates {@name=defaults}
SQLAlchemy includes flexible constructs in which to create default values for columns upon the insertion of rows, as well as upon update. These defaults can take several forms: a constant, a Python callable to be pre-executed before the SQL is executed, a SQL expression or function to be pre-executed before the SQL is executed, a pre-executed Sequence (for databases that support sequences), or a "passive" default, which is a default function triggered by the database itself upon insert, the value of which can then be post-fetched by the engine, provided the row provides a primary key in which to call upon.
i += 1
return i
- t = Table("mytable", db,
+ t = Table("mytable", meta,
# function-based default
Column('id', Integer, primary_key=True, default=mydefault),
The "default" keyword can also take SQL expressions, including select statements or direct function calls:
{python}
- t = Table("mytable", db,
+ t = Table("mytable", meta,
Column('id', Integer, primary_key=True),
# define 'create_date' to default to now()
Similar to an on-insert default is an on-update default, which is most easily specified by the "onupdate" keyword to Column, which also can be a constant, plain Python function or SQL expression:
{python}
- t = Table("mytable", db,
+ t = Table("mytable", meta,
Column('id', Integer, primary_key=True),
# define 'last_updated' to be populated with current_timestamp (the ANSI-SQL version of now())
A PassiveDefault indicates a column default or on-update value that is executed automatically by the database. This construct is used to specify a SQL function that will be specified as "DEFAULT" when creating tables, and also to indicate the presence of new data that is available to be "post-fetched" after an insert or update execution.
{python}
- t = Table('test', e,
+ t = Table('test', meta,
Column('mycolumn', DateTime, PassiveDefault("sysdate"))
)
mycolumn datetime default sysdate
)
-PassiveDefaults also send a message to the SQLEngine that data is available after update or insert. The object-relational mapper system uses this information to post-fetch rows after insert or update, so that instances can be refreshed with the new data. Below is a simplified version:
+PassiveDefaults also send a message to the `Engine` that data is available after update or insert. The object-relational mapper system uses this information to post-fetch rows after insert or update, so that instances can be refreshed with the new data. Below is a simplified version:
{python}
# table with passive defaults
Column('data2', Integer, PassiveDefault("d2_func", for_update=True))
)
# insert a row
- mytable.insert().execute(name='fred')
+ r = mytable.insert().execute(name='fred')
- # ask the engine: were there defaults fired off on that row ?
- if table.engine.lastrow_has_defaults():
+ # check the result: were there defaults fired off on that row ?
+ if r.lastrow_has_defaults():
# postfetch the row based on primary key.
# this only works for a table with primary key columns defined
- primary_key = table.engine.last_inserted_ids()
+ primary_key = r.last_inserted_ids()
row = table.select(table.c.id == primary_key[0])
-When Tables are reflected from the database using <code>autoload=True</code>, any DEFAULT values set on the columns will be reflected in the Table object as PassiveDefault instances.
+When Tables are reflected from the database using `autoload=True`, any DEFAULT values set on the columns will be reflected in the Table object as PassiveDefault instances.
##### The Catch: Postgres Primary Key Defaults always Pre-Execute {@name=postgres}
Current Postgres support does not rely upon OID's to determine the identity of a row. This is because the usage of OIDs has been deprecated with Postgres and they are disabled by default for table creates as of PG version 8. Pyscopg2's "cursor.lastrowid" function only returns OIDs. Therefore, when inserting a new row which has passive defaults set on the primary key columns, the default function is <b>still pre-executed</b> since SQLAlchemy would otherwise have no way of retrieving the row just inserted.
-
-
+
#### Defining Sequences {@name=sequences}
A table with a sequence looks like:
{python}
- table = Table("cartitems", db,
+ table = Table("cartitems", meta,
Column("cart_id", Integer, Sequence('cart_id_seq'), primary_key=True),
Column("description", String(40)),
Column("createdate", DateTime())
)
-The Sequence is used with Postgres or Oracle to indicate the name of a Sequence that will be used to create default values for a column. When a table with a Sequence on a column is created by SQLAlchemy, the Sequence object is also created. Similarly, the Sequence is dropped when the table is dropped. Sequences are typically used with primary key columns. When using Postgres, if an integer primary key column defines no explicit Sequence or other default method, SQLAlchemy will create the column with the SERIAL keyword, and will pre-execute a sequence named "tablename_columnname_seq" in order to retrieve new primary key values. Oracle, which has no "auto-increment" keyword, requires that a Sequence be created for a table if automatic primary key generation is desired. Note that for all databases, primary key values can always be explicitly stated within the bind parameters for any insert statement as well, removing the need for any kind of default generation function.
+The Sequence is used with Postgres or Oracle to indicate the name of a database sequence that will be used to create default values for a column. When a table with a Sequence on a column is created in the database by SQLAlchemy, the database sequence object is also created. Similarly, the database sequence is dropped when the table is dropped. Sequences are typically used with primary key columns. When using Postgres, if an integer primary key column defines no explicit Sequence or other default method, SQLAlchemy will create the column with the SERIAL keyword, and will pre-execute a sequence named "tablename_columnname_seq" in order to retrieve new primary key values, if they were not otherwise explicitly stated. Oracle, which has no "auto-increment" keyword, requires that a Sequence be created for a table if automatic primary key generation is desired.
-A Sequence object can be defined on a Table that is then used for a non-sequence-supporting database. In that case, the Sequence object is simply ignored. Note that a Sequence object is <b>entirely optional for all databases except Oracle</b>, as other databases offer options for auto-creating primary key values, such as AUTOINCREMENT, SERIAL, etc. SQLAlchemy will use these default methods for creating primary key values if no Sequence is present on the table metadata.
+A Sequence object can be defined on a Table that is then used for a non-sequence-supporting database. In that case, the Sequence object is simply ignored. Note that a Sequence object is **entirely optional for all databases except Oracle**, as other databases offer options for auto-creating primary key values, such as AUTOINCREMENT, SERIAL, etc. SQLAlchemy will use these default methods for creating primary key values if no Sequence is present on the table metadata.
A sequence can also be specified with `optional=True` which indicates the Sequence should only be used on a database that requires an explicit sequence, and not those that supply some other method of providing integer values. At the moment, it essentially means "use this sequence only with Oracle and not Postgres".
Indexes can be defined on table columns, including named indexes, non-unique or unique, multiple column. Indexes are included along with table create and drop statements. They are not used for any kind of run-time constraint checking; SQLAlchemy leaves that job to the expert on constraint checking, the database itself.
{python}
- mytable = Table('mytable', engine,
+ boundmeta = BoundMetaData('postgres:///scott:tiger@localhost/test')
+ mytable = Table('mytable', boundmeta,
# define a unique index
Column('col1', Integer, unique=True),
# which can then be created separately (will also get created with table creates)
i.create()
-### Adapting Tables to Alternate Engines {@name=adapting}
+### Adapting Tables to Alternate Metadata {@name=adapting}
-A Table object created against a specific engine can be re-created against a new engine using the `toengine` method:
+A `Table` object created against a specific `MetaData` object can be re-created against a new MetaData using the `tometadata` method:
{python}
- # create two engines
- sqlite_engine = create_engine('sqlite', {'filename':'querytest.db'})
- postgres_engine = create_engine('postgres',
- {'database':'test',
- 'host':'127.0.0.1', 'user':'scott', 'password':'tiger'})
+ # create two metadata
+ meta1 = BoundMetaData('sqlite:///querytest.db')
+ meta2 = MetaData()
# load 'users' from the sqlite engine
- users = Table('users', sqlite_engine, autoload=True)
+ users_table = Table('users', meta1, autoload=True)
- # create the same Table object for the other engine
- pg_users = users.toengine(postgres_engine)
+ # create the same Table object for the plain metadata
+ users_table_2 = users_table.tometadata(meta2)
-Also available is the "database neutral" ansisql engine:
-
- {python}
- import sqlalchemy.ansisql as ansisql
- generic_engine = ansisql.engine()
-
- users = Table('users', generic_engine,
- Column('user_id', Integer),
- Column('user_name', String(50))
- )
-
-Flexible "multi-engined" tables can also be achieved via the proxy engine, described in the section [dbengine_proxy](rel:dbengine_proxy).
-
-#### Non-engine primitives: TableClause/ColumnClause {@name=primitives}
-
-TableClause and ColumnClause are "primitive" versions of the Table and Column objects which dont use engines at all; applications that just want to generate SQL strings but not directly communicate with a database can use TableClause and ColumnClause objects (accessed via 'table' and 'column'), which are non-singleton and serve as the "lexical" base class of Table and Column:
-
- {python}
- tab1 = table('table1',
- column('id'),
- column('name'))
-
- tab2 = table('table2',
- column('id'),
- column('email'))
-
- tab1.select(tab1.c.name == 'foo')
-
-TableClause and ColumnClause are strictly lexical. This means they are fully supported within the full range of SQL statement generation, but they don't support schema concepts like creates, drops, primary keys, defaults, nullable status, indexes, or foreign keys.
--- /dev/null
+Plugins {@name=plugins}
+======================
+
+SQLAlchemy has a variety of extensions and "mods" available which provide extra functionality to SA, either via explicit usage or by augmenting the core behavior.
+
+### threadlocal
+
+**Author:** Mike Bayer and Daniel Miller
+
+Establishes `threadlocal` as the default strategy for new `ComposedSQLEngine` objects, installs a threadlocal `SessionContext` that is attached to all Mappers via a global `MapperExtension`, and establishes the global `SessionContext` under the name `sqlalchemy.objectstore`. Usually this is used in combination with `Tables` that are associated with `BoundMetaData` or `DynamicMetaData`, so that the `Session` does not need to be bound to any `Engine` explicitly.
+
+ {python}
+ import sqlalchemy.mods.threadlocal
+ from sqlalchemy import *
+
+ metadata = BoundMetaData('sqlite:///')
+ user_table = Table('users', metadata,
+ Column('user_id', Integer, primary_key=True),
+ Column('user_name', String(50), nullable=False)
+ )
+
+ class User(object):
+ pass
+ mapper(User, user_table)
+
+ # thread local session
+ session = objectstore.get_session()
+
+ # "user" object is added to the session automatically
+ user = User()
+
+ session.flush()
+
+#### get_session() Implemented on All Mappers
+
+All `Mapper` objects constructed after the `threadlocal` import will receive a default `MapperExtension` which implements the `get_session()` method, returning the `Session` that is associated with the current thread by the global `SessionContext`. All newly constructed objects will automatically be attached to the `Session` corresponding to the current thread, i.e. they will skip the "transient" state and go right to "pending".
+
+This occurs because when a `Mapper` is first constructed for a class, it decorates the classes' `__init__()` method in a manner like the following:
+
+ {python}
+ oldinit = class_.__init__ # the previous init method
+ def __init__(self):
+ session = ext.get_session() # get Session from this Mapper's MapperExtension
+ if session is EXT_PASS:
+ session = None
+ if session is not None:
+ session.save(self) # attach to the current session
+ oldinit(self) # call previous init method
+
+An instance can be redirected at construction time to a different `Session` by specifying the keyword parameter `_sa_session`:
+
+ {python}
+ session = create_session() # create a new session distinct from the thread-local session
+ myuser = User(_sa_session=session) # make a new User that is saved to this session
+
+Similarly, the **entity_name** parameter, which specifies an alternate `Mapper` to be used when attaching this instance to the `Session`, can be specified via `_sa_entity_name`:
+
+ {python}
+ myuser = User(_sa_session=session, _sa_entity_name='altentity')
+
+#### Default Query Objects
+
+The `MapperExtension` object's `get_session()` method is also used by the `Query` object to locate a `Session` with which to store newly loaded instances, if the `Query` is not already associated with a specific `Session`. As a result, the `Query` can be constructed standalone from a mapper or class:
+
+ {python}
+ # create a Query from a class
+ query = Query(User)
+
+ # specify entity name
+ query = Query(User, entity_name='foo')
+
+ # create a Query from a mapper
+ query = Query(mapper)
+
+#### objectstore Namespace {@name=objectstore}
+
+The `objectstore` is an instance of `SessionContext`, available in the `sqlalchemy` namespace which provides a proxy to the underlying `Session` bound to the current thread. `objectstore` can be treated just like the `Session` itself:
+
+ {python}
+ objectstore.save(instance)
+ objectstore.flush()
+
+ objectstore.clear()
+
+#### Attaching Mappers to their Class {@name=attaching}
+
+With `get_session()` handling the details of providing a `Session` in all cases, the `assign_mapper` function provides some of the functionality of `Query` and `Session` directly off the mapped instances themselves. This is a "monkeypatch" function that creates a primary mapper, attaches the mapper to the class, and also the methods `get, get_by, select, select_by, selectone, selectfirst, commit, expire, refresh, expunge` and `delete`:
+
+ {python}
+ # "assign" a mapper to the User class/users table
+ assign_mapper(User, users)
+
+ # methods are attached to the class for selecting
+ userlist = User.select_by(user_id=12)
+
+ myuser = User.get(1)
+
+ # mark an object as deleted for the next commit
+ myuser.delete()
+
+ # flush the changes on a specific object
+ myotheruser.flush()
+
+#### Engine Strategy Set to threadlocal By Default {@name=engine}
+
+The `threadlocal` mod also establishes `threadlocal` as the default *strategy* when calling the `create_engine()` function. This strategy is specified by the `strategy` keyword argument to `create_engine()` and can still be overridden to be "`plain`" or "`threadlocal`" explicitly.
+
+An `Engine` created with the `threadlocal` strategy will use a thread-locally managed connection object for all **implicit** statement executions and schema operations. Recall from [dbengine](rel:dbengine) that an implicit execution is an execution where the `Connection` object is opened and closed internally, and the `connect()` method on `Engine` is not used; such as:
+
+ {python}
+ result = table.select().execute()
+
+Above, the `result` variable holds onto a `ResultProxy` which is still referencing a connection returned by the connection pool. `threadlocal` strategy means that a second `execute()` statement in the same thread will use the same connection as the one referenced by `result`, assuming `result` is still referenced in memory.
+
+The `Mapper`, `Session`, and `Query` implementations work equally well with either the `default` or `threadlocal` engine strategies. However, using the `threadlocal` strategy means that `Session` operations will use the same underlying connection as that of straight `execute()` calls with constructed SQL objects:
+
+ {python}
+ # assume "threadlocal" strategy is enabled, and there is no transaction in progress
+
+ result = table.select().execute() # 'result' references a DBAPI connection, bound to the current thread
+
+ object = session.select() # the 'select' operation also uses the current thread's connection,
+ # i.e. the same connection referenced by 'result'
+
+ result.close() # return the connection to the pool. now there is no connection
+ # associated with the current thread. the next execution will re-check out a
+ # connection and re-attach to the current thread.
+
+### SessionContext
+
+**Author:** Daniel Miller
+
+This plugin is a generalized version of the `objectstore` object provided by the `threadlocal` plugin:
+
+ {python}
+ import sqlalchemy
+ from sqlalchemy.ext.sessioncontext import SessionContext
+
+ ctx = SessionContext(sqlalchemy.create_session)
+
+ class User(object):
+ pass
+
+ mapper(User, users_table, extension=ctx.mapperextension)
+
+ # 'u' is automatically added to the current session of 'ctx'
+ u = User()
+
+ # get the current session and flush
+ ctx.current.flush()
+
+The construction of each `Session` instance can be customized by providing a "creation function" which returns a new `Session`. The "scope" to which the session is associated, which by default is the current thread, can be customized by providing a "scope callable" which returns a hashable key that represents the current scope:
+
+ {python}
+ import sqlalchemy
+ from sqlalchemy.ext.sessioncontext import SessionContext
+
+ # create an engine
+ someengine = sqlalchemy.create_engine('sqlite:///')
+
+ # a function to return a Session bound to our engine
+ def make_session():
+ return sqlalchemy.create_session(bind_to=someengine)
+
+ # global declaration of "scope"
+ scope = "scope1"
+
+ # a function to return the current "session scope"
+ def global_scope_func():
+ return scope
+
+ # create SessionContext with our two functions
+ ctx = SessionContext(make_session, scopefunc=global_scope_func)
+
+ # get the session corresponding to "scope1", bound to engine "someengine":
+ session = ctx.current
+
+ # switch the "scope"
+ scope = "scope2"
+
+ # get the session corresponding to "scope2", bound to engine "someengine":
+ session = ctx.current
+
+
+### ActiveMapper
+
+**Author:** Jonathan LaCour
+
+ActiveMapper is a so-called "declarative layer" which allows the construction of a class, a `Table`, and a `Mapper` all in one step:
+
+ {python}
+ class Person(ActiveMapper):
+ class mapping:
+ id = column(Integer, primary_key=True)
+ full_name = column(String)
+ first_name = column(String)
+ middle_name = column(String)
+ last_name = column(String)
+ birth_date = column(DateTime)
+ ssn = column(String)
+ gender = column(String)
+ home_phone = column(String)
+ cell_phone = column(String)
+ work_phone = column(String)
+ prefs_id = column(Integer, foreign_key=ForeignKey('preferences.id'))
+ addresses = one_to_many('Address', colname='person_id', backref='person')
+ preferences = one_to_one('Preferences', colname='pref_id', backref='person')
+
+ def __str__(self):
+ s = '%s\n' % self.full_name
+ s += ' * birthdate: %s\n' % (self.birth_date or 'not provided')
+ s += ' * fave color: %s\n' % (self.preferences.favorite_color or 'Unknown')
+ s += ' * personality: %s\n' % (self.preferences.personality_type or 'Unknown')
+
+ for address in self.addresses:
+ s += ' * address: %s\n' % address.address_1
+ s += ' %s, %s %s\n' % (address.city, address.state, address.postal_code)
+
+ return s
+
+
+ class Preferences(ActiveMapper):
+ class mapping:
+ __table__ = 'preferences'
+ id = column(Integer, primary_key=True)
+ favorite_color = column(String)
+ personality_type = column(String)
+
+
+ class Address(ActiveMapper):
+ class mapping:
+ id = column(Integer, primary_key=True)
+ type = column(String)
+ address_1 = column(String)
+ city = column(String)
+ state = column(String)
+ postal_code = column(String)
+ person_id = column(Integer, foreign_key=ForeignKey('person.id'))
+
+More discussion on ActiveMapper can be found at [Jonathan LaCour's Blog](http://cleverdevil.org/computing/35/declarative-mapping-with-sqlalchemy) as well as the [SQLAlchemy Wiki](http://www.sqlalchemy.org/trac/wiki/ActiveMapper).
+
+### SqlSoup
+
+**Author:** Jonathan Ellis
+
+SqlSoup creates mapped classes on the fly from tables. It is essentially a nicer version of the "row data gateway" pattern.
+
+ {python}
+ >>> from sqlalchemy.ext.sqlsoup import SqlSoup
+ >>> soup = SqlSoup('sqlite://filename=:memory:')
+
+ >>> users = soup.users.select()
+ >>> users.sort()
+ >>> users
+ [Class_Users(name='Bhargan Basepair',email='basepair@example.edu',password='basepair',classname=None,admin=1),
+ Class_Users(name='Joe Student',email='student@example.edu',password='student',classname=None,admin=0)]
+
+Read more about SqlSoup on [Jonathan Ellis' Blog](http://spyced.blogspot.com/2006/04/introducing-sqlsoup.html).
+
+### ProxyEngine
+
+**Author:** Jason Pellerin
+
+The `ProxyEngine` is used to "wrap" an `Engine`, and via subclassing `ProxyEngine` one can instrument the functionality of an arbitrary `Engine` instance through the decorator pattern. It also provides a `connect()` method which will send all `Engine` requests to different underlying engines. Its functionality in that regard is largely superceded now by `DynamicMetaData` which is a better solution.
+
+ {python}
+ from sqlalchemy.ext.proxy import ProxyEngine
+ proxy = ProxyEngine()
+
+ proxy.connect('postgres://user:pw@host/db')
+
+### SelectResults
+
+**Author:** Jonas Borgström
+
+SelectResults gives generator-like behavior to the results returned from the `select` and `select_by` method of `Query`. It supports three modes of operation; per-query, per-mapper, and per-application.
+
+ {python title="SelectResults with a Query Object"}
+ from sqlalchemy.ext.selectresults import SelectResults
+
+ query = session.query(MyClass)
+ res = SelectResults(query, table.c.column == "something")
+ res = res.order_by([table.c.column]) #add an order clause
+
+ for x in res[:10]: # Fetch and print the top ten instances
+ print x.column2
+
+ x = list(res) # execute the query
+
+ # Count how many instances that have column2 > 42
+ # and column == "something"
+ print res.filter(table.c.column2 > 42).count()
+
+
+Per mapper:
+
+ {python title="SelectResults with a Mapper Object"}
+ from sqlalchemy.ext.selectresults import SelectResultsExt
+ mapper(MyClass, mytable, extension=SelectResultsExt())
+ session.query(MyClass).select(mytable.c.column=="something").order_by([mytable.c.column])[2:7]
+
+Or across an application via the `selectresults` mod:
+
+ {python title="SelectResults via mod"}
+ import sqlalchemy.mods.selectresults
+
+ mapper(MyClass, mytable)
+ session.query(MyClass).select(mytable.c.column=="something").order_by([mytable.c.column])[2:7]
+
+++ /dev/null
-<%flags>inherit='document_base.myt'</%flags>
-<%attr>title='Connection Pooling'</%attr>
-<&|doclib.myt:item, name="pooling", description="Connection Pooling" &>
- <P><b>Note:</b>This section describes the connection pool module of SQLAlchemy, which is the smallest component of the library that can be used on its own. If you are interested in using SQLAlchemy for query construction or Object Relational Mapping, this module is automatically managed behind the scenes; you can skip ahead to <&formatting.myt:link,path="dbengine"&> in that case.</p>
- <p>At the base of any database helper library is a system of efficiently acquiring connections to the database. Since the establishment of a database connection is typically a somewhat expensive operation, an application needs a way to get at database connections repeatedly without incurring the full overhead each time. Particularly for server-side web applications, a connection pool is the standard way to maintain a "pool" of database connections which are used over and over again among many requests. Connection pools typically are configured to maintain a certain "size", which represents how many connections can be used simultaneously without resorting to creating more newly-established connections.
- </p>
- <p>SQLAlchemy includes a pooling module that can be used completely independently of the rest of the toolset. This section describes how it can be used on its own, as well as the available options. If SQLAlchemy is being used more fully, the connection pooling described below occurs automatically. The options are still available, though, so this core feature is a good place to start.
- </p>
- <&|doclib.myt:item, name="establishing", description="Establishing a Transparent Connection Pool" &>
- Any DBAPI module can be "proxied" through the connection pool using the following technique (note that the usage of 'psycopg2' is <b>just an example</b>; substitute whatever DBAPI module you'd like):
-
- <&|formatting.myt:code&>
- import sqlalchemy.pool as pool
- import psycopg2 as psycopg
- psycopg = pool.manage(psycopg)
-
- # then connect normally
- connection = psycopg.connect(database='test', username='scott', password='tiger')
- </&>
- <p>This produces a <span class="codeline">sqlalchemy.pool.DBProxy</span> object which supports the same <span class="codeline">connect()</span> function as the original DBAPI module. Upon connection, a thread-local connection proxy object is returned, which delegates its calls to a real DBAPI connection object. This connection object is stored persistently within a connection pool (an instance of <span class="codeline">sqlalchemy.pool.Pool</span>) that corresponds to the exact connection arguments sent to the <span class="codeline">connect()</span> function. The connection proxy also returns a proxied cursor object upon calling <span class="codeline">connection.cursor()</span>. When all cursors as well as the connection proxy are de-referenced, the connection is automatically made available again by the owning pool object.</p>
-
- <p>Basically, the <span class="codeline">connect()</span> function is used in its usual way, and the pool module transparently returns thread-local pooled connections. Each distinct set of connect arguments corresponds to a brand new connection pool created; in this way, an application can maintain connections to multiple schemas and/or databases, and each unique connect argument set will be managed by a different pool object.</p>
- </&>
-
- <&|doclib.myt:item, name="configuration", description="Connection Pool Configuration" &>
- <p>When proxying a DBAPI module through the <span class="codeline">pool</span> module, options exist for how the connections should be pooled:
- </p>
- <ul>
- <li>echo=False : if set to True, connections being pulled and retrieved from/to the pool will be logged to the standard output, as well as pool sizing information.</li>
- <li>use_threadlocal=True : if set to True, repeated calls to connect() within the same application thread will be guaranteed to return the <b>same</b> connection object, if one has already been retrieved from the pool and has not been returned yet. This allows code to retrieve a connection from the pool, and then while still holding on to that connection, to call other functions which also ask the pool for a connection of the same arguments; those functions will act upon the same connection that the calling method is using. Note that once the connection is returned to the pool, it then may be used by another thread. To guarantee a single unique connection per thread that <b>never</b> changes, use the option <span class="codeline">poolclass=SingletonThreadPool</span>, in which case the use_threadlocal parameter is automatically set to False.</li>
- <li>poolclass=QueuePool : the Pool class used by the pool module to provide pooling. QueuePool uses the Python <span class="codeline">Queue.Queue</span> class to maintain a list of available connections. A developer can supply his or her own Pool class to supply a different pooling algorithm. Also included is the ThreadSingletonPool, which provides a single distinct connection per thread and is required with SQLite.</li>
- <li>pool_size=5 : used by QueuePool - the size of the pool to be maintained. This is the largest number of connections that will be kept persistently in the pool. Note that the pool begins with no connections; once this number of connections is requested, that number of connections will remain.</li>
- <li>max_overflow=10 : used by QueuePool - the maximum overflow size of the pool. When the number of checked-out connections reaches the size set in pool_size, additional connections will be returned up to this limit. When those additional connections are returned to the pool, they are disconnected and discarded. It follows then that the total number of simultaneous connections the pool will allow is pool_size + max_overflow, and the total number of "sleeping" connections the pool will allow is pool_size. max_overflow can be set to -1 to indicate no overflow limit; no limit will be placed on the total number of concurrent connections.</li>
- </ul>
- </&>
-
- <&|doclib.myt:item, name="custom", description="Custom Pool Construction" &>
- <p>One level below using a DBProxy to make transparent pools is creating the pool yourself. The pool module comes with two implementations of connection pools: <span class="codeline">QueuePool</span> and <span class="codeline">SingletonThreadPool</span>. While QueuePool uses Queue.Queue to provide connections, SingletonThreadPool provides a single per-thread connection which SQLite requires.</p>
-
- <p>Constructing your own pool involves passing a callable used to create a connection. Through this method, custom connection schemes can be made, such as a connection that automatically executes some initialization commands to start. The options from the previous section can be used as they apply to QueuePool or SingletonThreadPool.</p>
- <&|formatting.myt:code, title="Plain QueuePool"&>
- import sqlalchemy.pool as pool
- import psycopg2
-
- def getconn():
- c = psycopg2.connect(username='ed', host='127.0.0.1', dbname='test')
- # execute an initialization function on the connection before returning
- c.cursor.execute("setup_encodings()")
- return c
-
- p = pool.QueuePool(getconn, max_overflow=10, pool_size=5, use_threadlocal=True)
- </&>
-
- <&|formatting.myt:code, title="SingletonThreadPool"&>
- import sqlalchemy.pool as pool
- import sqlite
-
- def getconn():
- return sqlite.connect(filename='myfile.db')
-
- # SQLite connections require the SingletonThreadPool
- p = pool.SingletonThreadPool(getconn)
- </&>
-
- </&>
-</&>
\ No newline at end of file
--- /dev/null
+Connection Pooling {@name=pooling}
+======================
+
+This section describes the connection pool module of SQLAlchemy. The `Pool` object it provides is normally embedded within an `Engine` instance. For most cases, explicit access to the pool module is not required. However, the `Pool` object can be used on its own, without the rest of SA, to manage DBAPI connections; this section describes that usage. Also, this section will describe in more detail how to customize the pooling strategy used by an `Engine`.
+
+At the base of any database helper library is a system of efficiently acquiring connections to the database. Since the establishment of a database connection is typically a somewhat expensive operation, an application needs a way to get at database connections repeatedly without incurring the full overhead each time. Particularly for server-side web applications, a connection pool is the standard way to maintain a "pool" of database connections which are used over and over again among many requests. Connection pools typically are configured to maintain a certain "size", which represents how many connections can be used simultaneously without resorting to creating more newly-established connections.
+
+### Establishing a Transparent Connection Pool {@name=establishing}
+
+Any DBAPI module can be "proxied" through the connection pool using the following technique (note that the usage of 'psycopg2' is **just an example**; substitute whatever DBAPI module you'd like):
+
+ {python}
+ import sqlalchemy.pool as pool
+ import psycopg2 as psycopg
+ psycopg = pool.manage(psycopg)
+
+ # then connect normally
+ connection = psycopg.connect(database='test', username='scott', password='tiger')
+
+This produces a `sqlalchemy.pool.DBProxy` object which supports the same `connect()` function as the original DBAPI module. Upon connection, a thread-local connection proxy object is returned, which delegates its calls to a real DBAPI connection object. This connection object is stored persistently within a connection pool (an instance of `sqlalchemy.pool.Pool`) that corresponds to the exact connection arguments sent to the `connect()` function. The connection proxy also returns a proxied cursor object upon calling `connection.cursor()`. When all cursors as well as the connection proxy are de-referenced, the connection is automatically made available again by the owning pool object.
+
+Basically, the `connect()` function is used in its usual way, and the pool module transparently returns thread-local pooled connections. Each distinct set of connect arguments corresponds to a brand new connection pool created; in this way, an application can maintain connections to multiple schemas and/or databases, and each unique connect argument set will be managed by a different pool object.
+
+### Connection Pool Configuration {@name=configuration}
+
+When proxying a DBAPI module through the `pool` module, options exist for how the connections should be pooled:
+
+* echo=False : if set to True, connections being pulled and retrieved from/to the pool will be logged to the standard output, as well as pool sizing information.
+* use\_threadlocal=True : if set to True, repeated calls to connect() within the same application thread will be guaranteed to return the **same** connection object, if one has already been retrieved from the pool and has not been returned yet. This allows code to retrieve a connection from the pool, and then while still holding on to that connection, to call other functions which also ask the pool for a connection of the same arguments; those functions will act upon the same connection that the calling method is using. Note that once the connection is returned to the pool, it then may be used by another thread. To guarantee a single unique connection per thread that **never** changes, use the option `poolclass=SingletonThreadPool`, in which case the use_threadlocal parameter is automatically set to False.
+* poolclass=QueuePool : the Pool class used by the pool module to provide pooling. QueuePool uses the Python `Queue.Queue` class to maintain a list of available connections. A developer can supply his or her own Pool class to supply a different pooling algorithm. Also included is the `SingletonThreadPool`, which provides a single distinct connection per thread and is required with SQLite.
+* pool\_size=5 : used by `QueuePool` - the size of the pool to be maintained. This is the largest number of connections that will be kept persistently in the pool. Note that the pool begins with no connections; once this number of connections is requested, that number of connections will remain.
+* max\_overflow=10 : used by `QueuePool` - the maximum overflow size of the pool. When the number of checked-out connections reaches the size set in pool_size, additional connections will be returned up to this limit. When those additional connections are returned to the pool, they are disconnected and discarded. It follows then that the total number of simultaneous connections the pool will allow is `pool_size` + `max_overflow`, and the total number of "sleeping" connections the pool will allow is `pool_size`. `max_overflow` can be set to -1 to indicate no overflow limit; no limit will be placed on the total number of concurrent connections.
+* timeout=30 : used by `QueuePool` - the timeout before giving up on returning a connection, if none are available and the `max_overflow` has been reached.
+
+
+### Custom Pool Construction {@name=custom}
+
+One level below using a DBProxy to make transparent pools is creating the pool yourself. The pool module comes with two implementations of connection pools: `QueuePool` and `SingletonThreadPool`. While `QueuePool` uses `Queue.Queue` to provide connections, `SingletonThreadPool` provides a single per-thread connection which SQLite requires.
+
+Constructing your own pool involves passing a callable used to create a connection. Through this method, custom connection schemes can be made, such as a connection that automatically executes some initialization commands to start. The options from the previous section can be used as they apply to `QueuePool` or `SingletonThreadPool`.
+
+ {python title="Plain QueuePool"}
+ import sqlalchemy.pool as pool
+ import psycopg2
+
+ def getconn():
+ c = psycopg2.connect(username='ed', host='127.0.0.1', dbname='test')
+ # execute an initialization function on the connection before returning
+ c.cursor.execute("setup_encodings()")
+ return c
+
+ p = pool.QueuePool(getconn, max_overflow=10, pool_size=5, use_threadlocal=True)
+
+Or with SingletonThreadPool:
+
+ {python title="SingletonThreadPool"}
+ import sqlalchemy.pool as pool
+ import sqlite
+
+ def getconn():
+ return sqlite.connect(filename='myfile.db')
+
+ # SQLite connections require the SingletonThreadPool
+ p = pool.SingletonThreadPool(getconn)
+
-Constructing SQL Queries via Python Expressions
+Constructing SQL Queries via Python Expressions {@name=sql}
===============================================
-*Note:* This section describes how to use SQLAlchemy to construct SQL queries and receive result sets. It does *not* cover the object relational mapping capabilities of SQLAlchemy; that is covered later on in [datamapping](rel:datamapping). However, both areas of functionality work similarly in how selection criterion is constructed, so if you are interested just in ORM, you should probably skim through basic [sql_select_whereclause](rel:sql_select_whereclause) construction before moving on.
+*Note:* This section describes how to use SQLAlchemy to construct SQL queries and receive result sets. It does *not* cover the object relational mapping capabilities of SQLAlchemy; that is covered later on in [datamapping](rel:datamapping). However, both areas of functionality work similarly in how selection criterion is constructed, so if you are interested just in ORM, you should probably skim through basic [sql_whereclause](rel:sql_whereclause) construction before moving on.
-Once you have used the `sqlalchemy.schema` module to construct your tables and/or reflect them from the database, performing SQL queries using those table meta data objects is done via the `sqlalchemy.sql` package. This package defines a large set of classes, each of which represents a particular kind of lexical construct within a SQL query; all are descendants of the common base class `sqlalchemy.sql.ClauseElement`. A full query is represented via a structure of ClauseElements. A set of reasonably intuitive creation functions is provided by the `sqlalchemy.sql` package to create these structures; these functions are described in the rest of this section.
+Once you have used the `sqlalchemy.schema` module to construct your tables and/or reflect them from the database, performing SQL queries using those table meta data objects is done via the `sqlalchemy.sql` package. This package defines a large set of classes, each of which represents a particular kind of lexical construct within a SQL query; all are descendants of the common base class `sqlalchemy.sql.ClauseElement`. A full query is represented via a structure of `ClauseElement`s. A set of reasonably intuitive creation functions is provided by the `sqlalchemy.sql` package to create these structures; these functions are described in the rest of this section.
-To execute a query, you create its structure, then call the resulting structure's `execute()` method, which returns a cursor-like object (more on that later). The same clause structure can be used repeatedly. A ClauseElement is compiled into a string representation by an underlying SQLEngine object, which is located by searching through the clause's child items for a Table object, which provides a reference to its SQLEngine.
-
+Executing a `ClauseElement` structure can be performed in two general ways. You can use an `Engine` or a `Connection` object's `execute()` method to which you pass the query structure; this is known as **explicit style**. Or, if the `ClauseElement` structure is built upon Table metadata which is bound to an `Engine` directly, you can simply call `execute()` on the structure itself, known as **implicit style**. In both cases, the execution returns a cursor-like object (more on that later). The same clause structure can be executed repeatedly. The `ClauseElement` is compiled into a string representation by an underlying `Compiler` object which is associated with the `Engine` via its `Dialect`.
-The examples below all include a dump of the generated SQL corresponding to the query object, as well as a dump of the statement's bind parameters. In all cases, bind parameters are named parameters using the colon format (i.e. ':name'). A named parameter scheme, either ':name' or '%(name)s', is used with all databases, including those that use positional schemes. For those, the named-parameter statement and its bind values are converted to the proper list-based format right before execution. Therefore a SQLAlchemy application that uses ClauseElements can standardize on named parameters for all databases.
+The examples below all include a dump of the generated SQL corresponding to the query object, as well as a dump of the statement's bind parameters. In all cases, bind parameters are shown as named parameters using the colon format (i.e. ':name'). When the statement is compiled into a database-specific version, the named-parameter statement and its bind values are converted to the proper paramstyle for that database automatically.
-For this section, we will assume the following tables:
+For this section, we will mostly use the implcit style of execution, meaning the `Table` objects are associated with an instance of `BoundMetaData`, and constructed `ClauseElement` objects support self-execution. Assume the following configuration:
- {python}from sqlalchemy import *
- db = create_engine('sqlite://filename=mydb', echo=True)
+ {python}
+ from sqlalchemy import *
+ metadata = BoundMetaData('sqlite:///mydb.db', strategy='threadlocal', echo=True)
# a table to store users
- users = Table('users', db,
+ users = Table('users', metadata,
Column('user_id', Integer, primary_key = True),
Column('user_name', String(40)),
Column('password', String(80))
)
# a table that stores mailing addresses associated with a specific user
- addresses = Table('addresses', db,
+ addresses = Table('addresses', metadata,
Column('address_id', Integer, primary_key = True),
Column('user_id', Integer, ForeignKey("users.user_id")),
Column('street', String(100)),
)
# a table that stores keywords
- keywords = Table('keywords', db,
+ keywords = Table('keywords', metadata,
Column('keyword_id', Integer, primary_key = True),
Column('name', VARCHAR(50))
)
# a table that associates keywords with users
- userkeywords = Table('userkeywords', db,
+ userkeywords = Table('userkeywords', metadata,
Column('user_id', INT, ForeignKey("users")),
Column('keyword_id', INT, ForeignKey("keywords"))
)
A select is done by constructing a `Select` object with the proper arguments, adding any extra arguments if desired, then calling its `execute()` method.
- {python}from sqlalchemy import *
+ {python title="Basic Select"}
+ from sqlalchemy import *
# use the select() function defined in the sql package
s = select([users])
s = users.select()
# then, call execute on the Select object:
- {sql}c = s.execute()
+ {sql}result = s.execute()
SELECT users.user_id, users.user_name, users.password FROM users
{}
>>> str(s)
SELECT users.user_id, users.user_name, users.password FROM users
-The object returned by the execute call is a `sqlalchemy.engine.ResultProxy` object, which acts much like a DBAPI `cursor` object in the context of a result set, except that the rows returned can address their columns by ordinal position, column name, or even column object:
+#### Explicit Execution {@name=explicit}
+
+As mentioned above, `ClauseElement` structures can also be executed with a `Connection` object explicitly:
+
+ {python}
+ engine = create_engine('sqlite:///myfile.db')
+ conn = engine.connect()
+
+ s = users.select()
+ {sql}result = conn.execute(s)
+ SELECT users.user_id, users.user_name, users.password FROM users
+ {}
+
+ conn.close()
+
+#### Binding ClauseElements to Engines {@name=binding}
+
+For queries that don't contain any tables, `ClauseElement`s that represent a fully executeable statement support an `engine` keyword parameter which can bind the object to an `Engine`, thereby allowing implicit execution:
+
+ {python}
+ # select a literal
+ {sql}select(["current_time"], engine=myengine).execute()
+ SELECT current_time
+ {}
+
+ # select a function
+ {sql}select([func.now()], engine=db).execute()
+ SELECT now()
+ {}
+
+#### Getting Results {@name=resultproxy}
+
+The object returned by `execute()` is a `sqlalchemy.engine.ResultProxy` object, which acts much like a DBAPI `cursor` object in the context of a result set, except that the rows returned can address their columns by ordinal position, column name, or even column object:
- {python}# select rows, get resulting ResultProxy object
- {sql}c = users.select().execute()
+ {python title="Using the ResultProxy"}
+ # select rows, get resulting ResultProxy object
+ {sql}result = users.select().execute()
SELECT users.user_id, users.user_name, users.password FROM users
{}
# get one row
- row = c.fetchone()
+ row = result.fetchone()
# get the 'user_id' column via integer index:
user_id = row[0]
password = row.password
# ResultProxy object also supports fetchall()
- rows = c.fetchall()
+ rows = result.fetchall()
# or get the underlying DBAPI cursor object
- cursor = c.cursor
+ cursor = result.cursor
+
+ # close the result. If the statement was implicitly executed (i.e. without an explicit Connection), this will
+ # return the underlying connection resources back to the connection pool. de-referencing the result
+ # will also have the same effect.
+ # if an explicit Connection was used, then close() does nothing.
+ result.close()
#### Using Column Labels {@name=labels}
A common need when writing statements that reference multiple tables is to create labels for columns, thereby separating columns from different tables with the same name. The Select construct supports automatic generation of column labels via the `use_labels=True` parameter:
- {python}{sql}c = select([users, addresses],
+ {python title="use_labels Flag"}
+ {sql}c = select([users, addresses],
users.c.user_id==addresses.c.address_id,
use_labels=True).execute()
SELECT users.user_id AS users_user_id, users.user_name AS users_user_name,
The table name part of the label is affected if you use a construct such as a table alias:
- {python}person = users.alias('person')
+ {python title="use_labels with an Alias"}
+ person = users.alias('person')
{sql}c = select([person, addresses],
person.c.user_id==addresses.c.address_id,
use_labels=True).execute()
addresses.zip AS addresses_zip FROM users AS person, addresses
WHERE person.user_id = addresses.address_id
+Labels are also generated in such a way as to never go beyond 30 characters. Most databases support a limit on the length of symbols, such as Postgres, and particularly Oracle which has a rather short limit of 30:
+
+ {python title="use_labels Generates Abbreviated Labels"}
+ long_named_table = users.alias('this_is_the_person_table')
+ {sql}c = select([person], use_labels=True).execute()
+ SELECT this_is_the_person_table.user_id AS this_is_the_person_table_b36c,
+ this_is_the_person_table.user_name AS this_is_the_person_table_f76a,
+ this_is_the_person_table.password AS this_is_the_person_table_1e7c
+ FROM users AS this_is_the_person_table
+ {}
+
You can also specify custom labels on a per-column basis using the `label()` function:
- {python}{sql}c = select([users.c.user_id.label('id'), users.c.user_name.label('name')]).execute()
+ {python title="label() Function on Column"}
+ {sql}c = select([users.c.user_id.label('id'), users.c.user_name.label('name')]).execute()
SELECT users.user_id AS id, users.user_name AS name
FROM users
{}
But in addition to selecting all the columns off a single table, any set of columns can be specified, as well as full tables, and any combination of the two:
- {python}# individual columns
+ {python title="Specify Columns to Select"}
+ # individual columns
{sql}c = select([users.c.user_id, users.c.user_name]).execute()
SELECT users.user_id, users.user_name FROM users
{}
addresses.zip FROM users, addresses
{}
-#### WHERE Clause {@name=whereclause}
+### WHERE Clause {@name=whereclause}
The WHERE condition is the named keyword argument `whereclause`, or the second positional argument to the `select()` constructor and the first positional argument to the `select()` method of `Table`.
WHERE conditions are constructed using column objects, literal values, and functions defined in the `sqlalchemy.sql` module. Column objects override the standard Python operators to provide clause compositional objects, which compile down to SQL operations:
- {python}{sql}c = users.select(users.c.user_id == 7).execute()
+ {python title="Basic WHERE Clause"}
+ {sql}c = users.select(users.c.user_id == 7).execute()
SELECT users.user_id, users.user_name, users.password,
FROM users WHERE users.user_id = :users_user_id
{'users_user_id': 7}
More where clauses:
- {python}# another comparison operator
+ {python}
+ # another comparison operator
{sql}c = select([users], users.c.user_id>7).execute()
SELECT users.user_id, users.user_name, users.password,
FROM users WHERE users.user_id > :users_user_id
Select statements can also generate a WHERE clause based on the parameters you give it. If a given parameter, which matches the name of a column or its "label" (the combined tablename + "_" + column name), and does not already correspond to a bind parameter in the select object, it will be added as a comparison against that column. This is a shortcut to creating a full WHERE clause:
- {python}# specify a match for the "user_name" column
+ {python}
+ # specify a match for the "user_name" column
{sql}c = users.select().execute(user_name='ed')
SELECT users.user_id, users.user_name, users.password
FROM users WHERE users.user_name = :users_user_name
FROM users WHERE users.user_name = :users_user_name AND users.user_id = :users_user_id
{'users_user_name': 'ed', 'users_user_id': 10}
-##### Operators {@name=operators}
+#### Operators {@name=operators}
Supported column operators so far are all the numerical comparison operators, i.e. '==', '>', '>=', etc., as well as like(), startswith(), endswith(), between(), and in(). Boolean operators include not_(), and_() and or_(), which also can be used inline via '~', '&', and '|'. Math operators are '+', '-', '*', '/'. Any custom operator can be specified via the op() function shown below.
- {python}# "like" operator
+ {python}
+ # "like" operator
users.select(users.c.user_name.like('%ter'))
# equality operator
SELECT users.user_id, users.user_name, users.password
FROM users
WHERE users.user_name IS NULL
-
-#### Specifying the Engine {@name=engine}
-
-For queries that don't contain any tables, the SQLEngine can be specified to any constructed statement via the `engine` keyword parameter:
-
- {python}# select a literal
- select(["hi"], engine=myengine)
-
- # select a function
- select([func.now()], engine=db)
-
+
#### Functions {@name=functions}
Functions can be specified using the `func` keyword:
- {python}{sql}select([func.count(users.c.user_id)]).execute()
+ {python}
+ {sql}select([func.count(users.c.user_id)]).execute()
SELECT count(users.user_id) FROM users
{sql}users.select(func.substr(users.c.user_name, 1) == 'J').execute()
Functions also are callable as standalone values:
- {python}# call the "now()" function
+ {python}
+ # call the "now()" function
time = func.now(engine=myengine).scalar()
# call myfunc(1,2,3)
You can drop in a literal value anywhere there isnt a column to attach to via the `literal` keyword:
- {python}{sql}select([literal('foo') + literal('bar'), users.c.user_name]).execute()
+ {python}
+ {sql}select([literal('foo') + literal('bar'), users.c.user_name]).execute()
SELECT :literal + :literal_1, users.user_name
FROM users
{'literal_1': 'bar', 'literal': 'foo'}
Literals also take an optional `type` parameter to give literals a type. This can sometimes be significant, for example when using the "+" operator with SQLite, the String type is detected and the operator is converted to "||":
- {python}{sql}select([literal('foo', type=String) + 'bar'], engine=e).execute()
+ {python}
+ {sql}select([literal('foo', type=String) + 'bar'], engine=e).execute()
SELECT ? || ?
['foo', 'bar']
The ORDER BY clause of a select statement can be specified as individual columns to order by within an array specified via the `order_by` parameter, and optional usage of the asc() and desc() functions:
- {python}# straight order by
+ {python}
+ # straight order by
{sql}c = users.select(order_by=[users.c.user_name]).execute()
SELECT users.user_id, users.user_name, users.password
FROM users ORDER BY users.user_name
These are specified as keyword arguments:
- {python}{sql}c = select([users.c.user_name], distinct=True).execute()
+ {python}
+ {sql}c = select([users.c.user_name], distinct=True).execute()
SELECT DISTINCT users.user_name FROM users
{sql}c = users.select(limit=10, offset=20).execute()
As some of the examples indicated above, a regular inner join can be implicitly stated, just like in a SQL expression, by just specifying the tables to be joined as well as their join conditions:
- {python}{sql}addresses.select(addresses.c.user_id==users.c.user_id).execute()
+ {python}
+ {sql}addresses.select(addresses.c.user_id==users.c.user_id).execute()
SELECT addresses.address_id, addresses.user_id, addresses.street,
addresses.city, addresses.state, addresses.zip FROM addresses, users
WHERE addresses.user_id = users.user_id
There is also an explicit join constructor, which can be embedded into a select query via the `from_obj` parameter of the select statement:
- {python}{sql}addresses.select(from_obj=[
+ {python}
+ {sql}addresses.select(from_obj=[
addresses.join(users, addresses.c.user_id==users.c.user_id)
]).execute()
SELECT addresses.address_id, addresses.user_id, addresses.street, addresses.city,
The join constructor can also be used by itself:
- {python}{sql}join(users, addresses, users.c.user_id==addresses.c.user_id).select().execute()
+ {python}
+ {sql}join(users, addresses, users.c.user_id==addresses.c.user_id).select().execute()
SELECT users.user_id, users.user_name, users.password,
addresses.address_id, addresses.user_id, addresses.street, addresses.city,
addresses.state, addresses.zip
The join criterion in a join() call is optional. If not specified, the condition will be derived from the foreign key relationships of the two tables. If no criterion can be constructed, an exception will be raised.
- {python}{sql}join(users, addresses).select().execute()
+ {python}
+ {sql}join(users, addresses).select().execute()
SELECT users.user_id, users.user_name, users.password,
addresses.address_id, addresses.user_id, addresses.street, addresses.city,
addresses.state, addresses.zip
A join can be created on its own using the `join` or `outerjoin` functions, or can be created off of an existing Table or other selectable unit via the `join` or `outerjoin` methods:
- {python}{sql}outerjoin(users, addresses, users.c.user_id==addresses.c.address_id).select().execute()
+ {python}
+ {sql}outerjoin(users, addresses, users.c.user_id==addresses.c.address_id).select().execute()
SELECT users.user_id, users.user_name, users.password, addresses.address_id,
addresses.user_id, addresses.street, addresses.city, addresses.state, addresses.zip
FROM users LEFT OUTER JOIN addresses ON users.user_id = addresses.address_id
Aliases are used primarily when you want to use the same table more than once as a FROM expression in a statement:
- {python}address_b = addresses.alias('addressb')
+ {python}
+ address_b = addresses.alias('addressb')
{sql}# select users who have an address on Green street as well as Orange street
users.select(and_(
users.c.user_id==addresses.c.user_id,
SQLAlchemy allows the creation of select statements from not just Table objects, but from a whole class of objects that implement the `Selectable` interface. This includes Tables, Aliases, Joins and Selects. Therefore, if you have a Select, you can select from the Select:
+ {python}
>>> s = users.select()
>>> str(s)
SELECT users.user_id, users.user_name, users.password FROM users
+ {python}
>>> s = s.select()
>>> str(s)
SELECT user_id, user_name, password
Any Select, Join, or Alias object supports the same column accessors as a Table:
+ {python}
>>> s = users.select()
>>> [c.key for c in s.columns]
['user_id', 'user_name', 'password']
When you use `use_labels=True` in a Select object, the label version of the column names become the keys of the accessible columns. In effect you can create your own "view objects":
- {python}s = select([users, addresses], users.c.user_id==addresses.c.user_id, use_labels=True)
+ {python}
+ s = select([users, addresses], users.c.user_id==addresses.c.user_id, use_labels=True)
{sql}select([
s.c.users_user_name, s.c.addresses_street, s.c.addresses_zip
], s.c.addresses_city=='San Francisco').execute()
To specify a SELECT statement as one of the selectable units in a FROM clause, it usually should be given an alias.
- {python}{sql}s = users.select().alias('u')
+ {python}
+ {sql}s = users.select().alias('u')
select([addresses, s]).execute()
SELECT addresses.address_id, addresses.user_id, addresses.street, addresses.city,
addresses.state, addresses.zip, u.user_id, u.user_name, u.password
Select objects can be used in a WHERE condition, in operators such as IN:
- {python}# select user ids for all users whos name starts with a "p"
+ {python}
+ # select user ids for all users whos name starts with a "p"
s = select([users.c.user_id], users.c.user_name.like('p%'))
# now select all addresses for those users
Subqueries can be used in the column clause of a select statement by specifying the `scalar=True` flag:
- {python}{sql}select([table2.c.col1, table2.c.col2, select([table1.c.col1], table1.c.col2==7, scalar=True)])
+ {python}
+ {sql}select([table2.c.col1, table2.c.col2, select([table1.c.col1], table1.c.col2==7, scalar=True)])
SELECT table2.col1, table2.col2,
(SELECT table1.col1 AS col1 FROM table1 WHERE col2=:table1_col2)
FROM table2
When a select object is embedded inside of another select object, and both objects reference the same table, SQLAlchemy makes the assumption that the table should be correlated from the child query to the parent query. To disable this behavior, specify the flag `correlate=False` to the Select statement.
- {python}# make an alias of a regular select.
+ {python}
+ # make an alias of a regular select.
s = select([addresses.c.street], addresses.c.user_id==users.c.user_id).alias('s')
>>> str(s)
SELECT addresses.street FROM addresses, users
An EXISTS clause can function as a higher-scaling version of an IN clause, and is usually used in a correlated fashion:
- {python}# find all users who have an address on Green street:
+ {python}
+ # find all users who have an address on Green street:
{sql}users.select(
exists(
[addresses.c.address_id],
Unions come in two flavors, UNION and UNION ALL, which are available via module level functions or methods off a Selectable:
- {python}{sql}union(
+ {python}
+ {sql}union(
addresses.select(addresses.c.street=='123 Green Street'),
addresses.select(addresses.c.street=='44 Park Ave.'),
addresses.select(addresses.c.street=='3 Mill Road'),
### Custom Bind Parameters {@name=bindparams}
-Throughout all these examples, SQLAlchemy is busy creating bind parameters wherever literal expressions occur. You can also specify your own bind parameters with your own names, and use the same statement repeatedly. As mentioned at the top of this section, named bind parameters are always used regardless of the type of DBAPI being used; for DBAPI's that expect positional arguments, bind parameters are converted to lists right before execution, and Pyformat strings in statements, i.e. '%(name)s', are converted to the appropriate positional style.
+Throughout all these examples, SQLAlchemy is busy creating bind parameters wherever literal expressions occur. You can also specify your own bind parameters with your own names, and use the same statement repeatedly. The bind parameters, shown here in the "named" format, will be converted to the appropriate named or positional style according to the database implementation being used.
- {python}s = users.select(users.c.user_name==bindparam('username'))
+ {python title="Custom Bind Params"}
+ s = users.select(users.c.user_name==bindparam('username'))
+
+ # execute implicitly
{sql}s.execute(username='fred')
SELECT users.user_id, users.user_name, users.password
FROM users WHERE users.user_name = :username
{'username': 'fred'}
- {sql}s.execute(username='jane')
+ # execute explicitly
+ conn = engine.connect()
+ {sql}conn.execute(s, username='fred')
SELECT users.user_id, users.user_name, users.password
FROM users WHERE users.user_name = :username
- {'username': 'jane'}
+ {'username': 'fred'}
- {sql}s.execute(username='mary')
- SELECT users.user_id, users.user_name, users.password
- FROM users WHERE users.user_name = :username
- {'username': 'mary'}
-`executemany()` is also available, but that applies more to INSERT/UPDATE/DELETE, described later.
-
-The generation of bind parameters is performed specific to the engine being used. The examples in this document all show "named" parameters like those used in sqlite and oracle. Depending on the parameter type specified by the DBAPI module, the correct bind parameter scheme will be used.
+`executemany()` is also available by supplying multiple dictionary arguments instead of keyword arguments to the `execute()` method of `ClauseElement` or `Connection`. Examples can be found later in the sections on INSERT/UPDATE/DELETE.
#### Precompiling a Query {@name=precompiling}
The sql package tries to allow free textual placement in as many ways as possible. In the examples below, note that the from_obj parameter is used only when no other information exists within the select object with which to determine table metadata. Also note that in a query where there isnt even table metadata used, the SQLEngine to be used for the query has to be explicitly specified:
- {python}# strings as column clauses
+ {python}
+ # strings as column clauses
{sql}select(["user_id", "user_name"], from_obj=[users]).execute()
SELECT user_id, user_name FROM users
{}
# a full query
{sql}text("select user_name from users", engine=db).execute()
- select user_name from users
+ SELECT user_name FROM users
{}
- # or call text() off of the engine
- engine.text("select user_name from users").execute()
-
- # execute off the engine directly - you must use the engine's native bind parameter
- # style (i.e. named, pyformat, positional, etc.)
- {sql}db.execute(
- "select user_name from users where user_id=:user_id",
- {'user_id':7}).execute()
- select user_name from users where user_id=:user_id
- {'user_id':7}
#### Using Bind Parameters in Text Blocks {@name=textual_binds}
One of the primary motivations for a programmatic SQL library is to allow the piecemeal construction of a SQL statement based on program variables. All the above examples typically show Select objects being created all at once. The Select object also includes "builder" methods to allow building up an object. The below example is a "user search" function, where users can be selected based on primary key, user name, street address, keywords, or any combination:
- {python}def find_users(id=None, name=None, street=None, keywords=None):
- statement = users.select()
- if id is not None:
- statement.append_whereclause(users.c.user_id==id)
- if name is not None:
- statement.append_whereclause(users.c.user_name==name)
- if street is not None:
- # append_whereclause joins "WHERE" conditions together with AND
- statement.append_whereclause(users.c.user_id==addresses.c.user_id)
- statement.append_whereclause(addresses.c.street==street)
- if keywords is not None:
- statement.append_from(
- users.join(userkeywords, users.c.user_id==userkeywords.c.user_id).join(
- keywords, userkeywords.c.keyword_id==keywords.c.keyword_id))
- statement.append_whereclause(keywords.c.name.in_(keywords))
- # to avoid multiple repeats, set query to be DISTINCT:
- statement.distinct=True
- return statement.execute()
+ {python}
+ def find_users(id=None, name=None, street=None, keywords=None):
+ statement = users.select()
+ if id is not None:
+ statement.append_whereclause(users.c.user_id==id)
+ if name is not None:
+ statement.append_whereclause(users.c.user_name==name)
+ if street is not None:
+ # append_whereclause joins "WHERE" conditions together with AND
+ statement.append_whereclause(users.c.user_id==addresses.c.user_id)
+ statement.append_whereclause(addresses.c.street==street)
+ if keywords is not None:
+ statement.append_from(
+ users.join(userkeywords, users.c.user_id==userkeywords.c.user_id).join(
+ keywords, userkeywords.c.keyword_id==keywords.c.keyword_id))
+ statement.append_whereclause(keywords.c.name.in_(keywords))
+ # to avoid multiple repeats, set query to be DISTINCT:
+ statement.distinct=True
+ return statement.execute()
{sql}find_users(id=7)
SELECT users.user_id, users.user_name, users.password
The values to be populated for an INSERT or an UPDATE can be specified to the insert()/update() functions as the `values` named argument, or the query will be compiled based on the values of the parameters sent to the execute() method.
- {python}# basic insert
+ {python title="Using insert()"}
+ # basic insert
{sql}users.insert().execute(user_id=1, user_name='jack', password='asdfdaf')
INSERT INTO users (user_id, user_name, password)
VALUES (:user_id, :user_name, :password)
Updates work a lot like INSERTS, except there is an additional WHERE clause that can be specified.
- {python}# change 'jack' to 'ed'
+ {python title="Using update()"}
+ # change 'jack' to 'ed'
{sql}users.update(users.c.user_name=='jack').execute(user_name='ed')
UPDATE users SET user_name=:user_name WHERE users.user_name = :users_user_name
{'users_user_name': 'jack', 'user_name': 'ed'}
--- /dev/null
+The threadlocal mod {@name=threadlocal}
+============
+++ /dev/null
-<%flags>inherit='document_base.myt'</%flags>
-<%attr>title='How to Read this Manual'</%attr>
-<&|doclib.myt:item, name="howtoread", description="How to Read this Manual" &>
-
-<p>SQLAlchemy features a lot of tools and patterns to help in every area of writing applications that talk to relational databases. To achieve this, it has a lot of areas of functionality which work together to provide a cohesive package. Ultimately, just a little bit of familiarity with each concept is all that's needed to get off the ground.</p>
-
-<p>That said, here's two quick links that summarize the two most prominent features of SQLAlchemy:
-<ul>
- <li><&formatting.myt:link, path="datamapping", class_="trailbold"&> - a synopsis of how to map objects to database tables (Object Relational Mapping)</li>
- <li><&formatting.myt:link, path="sql", class_="trailbold"&> - SQLAlchemy's own domain-oriented approach to constructing and executing SQL statements.</li>
-</ul>
-</p>
-
-<&|doclib.myt:item, name="trailmap", description="Trail Map" &>
-<p>For a comprehensive tour through all of SQLAlchemy's components, below is a "Trail Map" of the knowledge dependencies between these components indicating the order in which concepts may be learned. Concepts marked in bold indicate features that are useful on their own.
-</p>
-<pre>
-Start
- |
- |
- |--- <&formatting.myt:link, class_="trailbold", path="pooling" &>
- | |
- | |
- | |------ <&formatting.myt:link, path="pooling_configuration" &>
- | |
- | |
- +--- <&formatting.myt:link, path="dbengine_establishing" &> |
- | |
- | |
- |--------- <&formatting.myt:link, path="dbengine_options" &>
- |
- |
- +---- <&formatting.myt:link, path="metadata_tables" &>
- |
- |
- |---- <&formatting.myt:link, path="metadata_creating" &>
- |
- |
- |---- <&formatting.myt:link, path="sql", class_="trailbold" &>
- | |
- | |
- +---- <&formatting.myt:link, path="datamapping", class_="trailbold"&> |
- | | |
- | | |
- | <&formatting.myt:link, path="unitofwork"&> |
- | | |
- | | |
- | +----------- <&formatting.myt:link, path="adv_datamapping"&>
- |
- +----- <&formatting.myt:link, path="types"&>
-</pre>
-</&>
-</&>
-Tutorial\r
-========\r
-This tutorial provides a relatively simple walking tour through the basic concepts of SQLAlchemy. You may wish to skip it and dive into the [main manual][manual] which is more reference-oriented.\r
-\r
-[manual]: rel:howtoread\r
-\r
-Installation\r
-------------\r
-\r
-### Installing SQLAlchemy {@name=sqlalchemy}\r
-\r
-Installing SQLAlchemy from scratch is most easily achieved with [setuptools][]. ([setuptools installation][install setuptools]). Just run this from the command-line:\r
- \r
- $ easy_install SQLAlchemy\r
-\r
-This command will download the latest version of SQLAlchemy from the [Python Cheese Shop][cheese] and install it to your system.\r
-\r
-[setuptools]: http://peak.telecommunity.com/DevCenter/setuptools\r
-[install setuptools]: http://peak.telecommunity.com/DevCenter/EasyInstall#installation-instructions\r
-[cheese]: http://cheeseshop.python.org/pypi\r
-\r
-### Installing a Database API {@name=dbms}\r
-\r
-SQLAlchemy is designed to operate with a [DBAPI][DBAPI] implementation built for a particular database, and includes support for the most popular databases. If you have one of the [supported DBAPI implementations][supported dbms], you can proceed to the following section. Otherwise [SQLite][] is an easy-to-use database to get started with, which works with plain files or in-memory databases.\r
-\r
-[DBAPI]: http://www.python.org/doc/peps/pep-0249/\r
-\r
-To work with SQLite, you'll need:\r
-\r
- * [pysqlite][] - Python interface for SQLite\r
- * SQLite library \r
-\r
-Note that the SQLite library download is not required with Windows, as the Windows Pysqlite library already includes it linked in. Pysqlite and SQLite can also be installed on Linux or FreeBSD via pre-made [packages][pysqlite packages] or [from sources][pysqlite].\r
-\r
-[supported dbms]: rel:dbengine_establishing\r
-[sqlite]: http://sqlite.org/\r
-[pysqlite]: http://pysqlite.org/\r
-[pysqlite packages]: http://initd.org/tracker/pysqlite/wiki/PysqlitePackages\r
-\r
-Getting Started {@name=gettingstarted}\r
---------------------------\r
-\r
-### Connecting to the Database\r
-\r
-The first thing needed is a handle to the desired database, represented by a `SQLEngine` object. This object handles the business of managing connections and dealing with the specifics of a particular database. Below, we will make a SQLite connection to a file-based database called "tutorial.db".\r
-\r
- >>> from sqlalchemy import *\r
- >>> db = create_engine('sqlite://filename=tutorial.db')\r
-\r
-For full information on creating database engines, including those for SQLite and others, see [dbengine](rel:dbengine).\r
-\r
-### Creating a Table {@name=table}\r
-\r
-A core philosophy of SQLAlchemy is that tables and domain classes are different beasts. For this reason, SQLAlchemy provides constructs that represent tables by themselves (known as *table metadata*). So we will begin by constructing table metadata objects and performing SQL operations with them directly, keeping in mind that there is also an Object Relational Mapper (ORM) which does the same thing except via domain models. Let's construct an object that represents a table:\r
-\r
- >>> users = Table('users', db,\r
- ... Column('user_id', Integer, primary_key = True),\r
- ... Column('user_name', String(40)),\r
- ... Column('password', String(80))\r
- ... )\r
-\r
-As you might have guessed, we have just defined a table named `users` which has three columns: `user_id` (which is a primary key column), `user_name` and `password`. Currently it is just an object that may not correspond to an existing table in your database. To actually create the table, we use the `create()` method. To make it interesting we will have SQLAlchemy to echo the SQL statements it sends to the database:\r
-\r
- >>> db.echo = True\r
- >>> users.create() # doctest:+ELLIPSIS,+NORMALIZE_WHITESPACE\r
- CREATE TABLE users(\r
- user_id INTEGER NOT NULL PRIMARY KEY,\r
- user_name VARCHAR(40),\r
- password VARCHAR(80)\r
- )\r
- ...\r
- >>> db.echo = False # you can skip this if you want to keep logging SQL statements\r
-\r
-Alternatively, the `users` table might already exist (such as, if you're running examples from this tutorial for the second time), in which case you can just skip the `create()` method call. You can even skip defining the individual columns in the `users` table and ask SQLAlchemy to load its definition from the database:\r
-\r
- >>> users = Table('users', db, autoload = True)\r
- >>> list(users.columns)[0].name\r
- 'user_id'\r
-\r
-Documentation on table metadata is available in [metadata](rel:metadata).\r
-\r
-### Inserting Rows\r
-\r
-Inserting is achieved via the `insert()` method, which defines a *clause object* representing an INSERT statement:\r
-\r
- >>> i = users.insert()\r
- >>> i # doctest:+ELLIPSIS\r
- <sqlalchemy.sql.Insert object at 0x...>\r
- >>> print i\r
- INSERT INTO users (user_id, user_name, password) VALUES (?, ?, ?)\r
-\r
-The `execute()` method of the clause object executes the statement at the database level:\r
-\r
- >>> for name in ['Tom', 'Dick', 'Harry']: # doctest:+ELLIPSIS\r
- ... i.execute(user_name = name)\r
- <sqlalchemy.engine.ResultProxy instance at 0x...>\r
- ...\r
- >>> i.execute(user_name = 'Mary', password = 'secure') # doctest:+ELLIPSIS\r
- <sqlalchemy.engine.ResultProxy instance at 0x...>\r
-\r
-When constructing clause objects, SQLAlchemy will bind all literal values into bind parameters, according to the paramstyle of the underlying DBAPI. This allows for better performance, as the database may cache a compiled representation of the statement and reuse it for new executions, substituting the new values. Also, when using bound values, you need not worry about [SQL injection][] attacks.\r
-\r
-[SQL injection]: http://en.wikipedia.org/wiki/SQL_injection\r
-\r
-Documentation on inserting: [sql_insert](rel:sql_insert).\r
-\r
-### Constructing Queries\r
-\r
-Let's check that the data we have put into `users` table is actually there. The procedure is analogous to the insert example above, except you now call the `select()` method off the `users` table:\r
-\r
- >>> s = users.select()\r
- >>> print s\r
- SELECT users.user_id, users.user_name, users.password \r
- FROM users\r
- >>> r = s.execute()\r
-\r
-This time, we won't ignore the return value of `execute()`:\r
-\r
- >>> r # doctest:+ELLIPSIS\r
- <sqlalchemy.engine.ResultProxy instance at 0x...>\r
- >>> r.keys\r
- ['user_id', 'user_name', 'password']\r
- >>> row = r.fetchone()\r
- >>> row['user_name']\r
- u'Tom'\r
- >>> r.fetchall()\r
- [(2, u'Dick', None), (3, u'Harry', None), (4, u'Mary', u'secure')]\r
-\r
-Documentation on selecting: [sql_select](rel:sql_select).\r
-\r
-### Related Table\r
-\r
-Main documentation: [sql](rel:sql).\r
-\r
-### Fancier Querying {@name=fancyquery}\r
-\r
-Main documentation: [sql](rel:sql).\r
-\r
-### Data Mapping {@name=mapping}\r
-\r
-Main documentation: [datamapping](rel:datamapping), [adv_datamapping](rel:adv_datamapping).\r
-\r
-### Transactions\r
-\r
-Main documentation: [unitofwork](rel:unitofwork), [dbengine_transactions](rel:dbengine_transactions).\r
-\r
-Conclusion\r
-----------\r
+Tutorial
+========
+This tutorial provides a relatively simple walking tour through the basic concepts of SQLAlchemy. You may wish to skip it and dive into the [main manual][manual] which is more reference-oriented. The examples in this tutorial comprise a fully working interactive Python session, and are guaranteed to be functioning courtesy of [doctest][].
+
+[doctest]: http://www.python.org/doc/lib/module-doctest.html
+[manual]: rel:howtoread
+
+Installation
+------------
+
+### Installing SQLAlchemy {@name=sqlalchemy}
+
+Installing SQLAlchemy from scratch is most easily achieved with [setuptools][]. ([setuptools installation][install setuptools]). Just run this from the command-line:
+
+ $ easy_install SQLAlchemy
+
+This command will download the latest version of SQLAlchemy from the [Python Cheese Shop][cheese] and install it to your system.
+
+[setuptools]: http://peak.telecommunity.com/DevCenter/setuptools
+[install setuptools]: http://peak.telecommunity.com/DevCenter/EasyInstall#installation-instructions
+[cheese]: http://cheeseshop.python.org/pypi
+
+Otherwise, you can install from the distribution using the `setup.py` script:
+
+ $ python setup.py install
+
+### Installing a Database API {@name=dbms}
+
+SQLAlchemy is designed to operate with a [DBAPI][DBAPI] implementation built for a particular database, and includes support for the most popular databases. If you have one of the [supported DBAPI implementations][supported dbms], you can proceed to the following section. Otherwise [SQLite][] is an easy-to-use database to get started with, which works with plain files or in-memory databases.
+
+[DBAPI]: http://www.python.org/doc/peps/pep-0249/
+
+To work with SQLite, you'll need:
+
+ * [pysqlite][] - Python interface for SQLite
+ * SQLite library
+
+Note that the SQLite library download is not required with Windows, as the Windows Pysqlite library already includes it linked in. Pysqlite and SQLite can also be installed on Linux or FreeBSD via pre-made [packages][pysqlite packages] or [from sources][pysqlite].
+
+[supported dbms]: rel:dbengine_establishing
+[sqlite]: http://sqlite.org/
+[pysqlite]: http://pysqlite.org/
+[pysqlite packages]: http://initd.org/tracker/pysqlite/wiki/PysqlitePackages
+
+Getting Started {@name=gettingstarted}
+--------------------------
+
+### Imports
+
+SQLAlchemy provides the entire namespace of everything you'll need under the module name `sqlalchemy`. For the purposes of this tutorial, we will import its full list of symbols into our own local namespace.
+
+ {python}
+ >>> from sqlalchemy import *
+
+### Connecting to the Database
+
+After our imports, the next thing we need is a handle to the desired database, represented by an `Engine` object. This object handles the business of managing connections and dealing with the specifics of a particular database. Below, we will make a SQLite connection to a file-based database called "tutorial.db".
+
+ {python}
+ >>> db = create_engine('sqlite:///tutorial.db')
+
+
+For full information on creating database engines, including those for SQLite and others, see [dbengine](rel:dbengine).
+
+Working with Database Objects {@name=schemasql}
+-----------------------------------------------
+
+A core philosophy of SQLAlchemy is that tables and domain classes are different beasts. For this reason, SQLAlchemy provides constructs that represent tables by themselves (known as *table metadata*). So we will begin by constructing table metadata objects and performing SQL operations with them directly. Later, we will look into SQLAlchemy's Object Relational Mapper (ORM), which provides an additional layer of abstraction onto table metadata, allowing us to load and save objects of any arbitrary Python class.
+
+### Defining Metadata, Binding to Engines {@name=metadata}
+
+Firstly, your Tables have to belong to a collection called `MetaData`. We will create a handy form of `MetaData` that automatically connects to our `Engine` (connecting a schema object to an Engine is called *binding*):
+
+ {python}
+ >>> metadata = BoundMetaData(db)
+
+An equivalent operation is to create the `BoundMetaData` object directly with an Engine URL, which calls the `create_engine` call for us:
+
+ {python}
+ >>> metadata = BoundMetaData('sqlite:///tutorial.db')
+
+Now, when we tell "metadata" about the tables in our database, we can issue CREATE statements for those tables, as well as create and execute SQL statements derived from them, without needing to open or close any connections; that will be all done automatically. Note that this feature is **entirely optional**. SQLAlchemy includes full support for explicit Connections used with schema and SQL constructs that are entirely unbound to any Engine.
+
+For the purposes of this tutorial, we will stick with "bound" objects, as it makes the code simpler and easier to read.
+
+### Creating a Table {@name=table}
+
+With `metadata` as our established home for tables, lets make a Table for it:
+
+ {python}
+ >>> users_table = Table('users', metadata,
+ ... Column('user_id', Integer, primary_key=True),
+ ... Column('user_name', String(40)),
+ ... Column('password', String(10))
+ ... )
+
+As you might have guessed, we have just defined a table named `users` which has three columns: `user_id` (which is a primary key column), `user_name` and `password`. Currently it is just an object that doesn't necessarily correspond to an existing table in our database. To actually create the table, we use the `create()` method. To make it interesting, we will have SQLAlchemy echo the SQL statements it sends to the database, by setting the `echo` flag on the `Engine` associated with our `BoundMetaData`:
+
+ {python}
+ >>> metadata.engine.echo = True
+ >>> users_table.create() # doctest:+ELLIPSIS,+NORMALIZE_WHITESPACE
+ CREATE TABLE users(
+ user_id INTEGER NOT NULL PRIMARY KEY,
+ user_name VARCHAR(40),
+ password VARCHAR(10)
+ )
+ ...
+
+Alternatively, the `users` table might already exist (such as, if you're running examples from this tutorial for the second time), in which case you can just skip the `create()` method call. You can even skip defining the individual columns in the `users` table and ask SQLAlchemy to load its definition from the database:
+
+ {python}
+ >>> users_table = Table('users', metadata, autoload=True)
+ >>> list(users_table.columns)[0].name
+ 'user_id'
+
+Documentation on table metadata is available in [metadata](rel:metadata).
+
+### Inserting Rows
+
+Inserting is achieved via the `insert()` method, which defines a *clause object* (known as a `ClauseElement`) representing an INSERT statement:
+
+ {python}
+ >>> i = users_table.insert()
+ >>> i # doctest:+ELLIPSIS
+ <sqlalchemy.sql.Insert object at 0x...>
+ >>> print i
+ INSERT INTO users (user_id, user_name, password) VALUES (?, ?, ?)
+
+Since we created this insert statement object from the `users` table which is bound to our `Engine`, the statement itself is also bound to the `Engine`, and supports executing itself. The `execute()` method of the clause object will *compile* the object into a string according to the underlying *dialect* of the Engine to which the statement is bound, and will then execute the resulting statement.
+
+ {python}
+ >>> i.execute(user_name='Mary', password='secure') # doctest:+ELLIPSIS
+ INSERT INTO users (user_name, password) VALUES (?, ?)
+ ['Mary', 'secure']
+ COMMIT
+ <sqlalchemy.engine.base.ResultProxy instance at 0x...>
+
+ >>> i.execute({'user_name':'Tom'}, {'user_name':'Fred'}, {'user_name':'Harry'}) # doctest:+ELLIPSIS,+NORMALIZE_WHITESPACE
+ INSERT INTO users (user_name) VALUES (?)
+ [['Tom'], ['Fred'], ['Harry']]
+ COMMIT
+ <sqlalchemy.engine.base.ResultProxy instance at 0x...>
+
+
+Note that the `VALUES` clause of each `INSERT` statement was automatically adjusted to correspond to the parameters sent to the `execute()` method. This is because the compilation step of a `ClauseElement` takes into account not just the constructed SQL object and the specifics of the type of database being used, but the execution parameters sent along as well.
+
+When constructing clause objects, SQLAlchemy will bind all literal values into bind parameters. On the construction side, bind parameters are always treated as named parameters. At compilation time, SQLAlchemy will convert them into their proper format, based on the paramstyle of the underlying DBAPI. This works equally well for all named and positional bind parameter formats described in the DBAPI specification.
+
+Documentation on inserting: [sql_insert](rel:sql_insert).
+
+### Selecting
+
+Let's check that the data we have put into `users` table is actually there. The procedure is analogous to the insert example above, except you now call the `select()` method off the `users` table:
+
+ {python}
+ >>> s = users_table.select()
+ >>> print s
+ SELECT users.user_id, users.user_name, users.password
+ FROM users
+ >>> r = s.execute()
+ SELECT users.user_id, users.user_name, users.password
+ FROM users
+ []
+
+This time, we won't ignore the return value of `execute()`. Its an instance of `ResultProxy`, which is a result-holding object that behaves very similarly to the `cursor` object one deals with directly with a database API:
+
+ {python}
+ >>> r # doctest:+ELLIPSIS
+ <sqlalchemy.engine.base.ResultProxy instance at 0x...>
+ >>> r.fetchone()
+ (1, u'Mary', u'secure')
+ >>> r.fetchall()
+ [(2, u'Tom', None), (3, u'Fred', None), (4, u'Harry', None)]
+
+Query criterion for the select is specified using Python expressions, using the `Column` objects in the `Table` as a base. All expressions constructed from `Column` objects are themselves instances of `ClauseElements`, just like the `Select`, `Insert`, and `Table` objects themselves.
+
+ {python}
+ >>> r = users_table.select(users_table.c.user_name=='Harry').execute()
+ SELECT users.user_id, users.user_name, users.password
+ FROM users
+ WHERE users.user_name = ?
+ ['Harry']
+ >>> row = r.fetchone()
+ >>> print row
+ (4, u'Harry', None)
+
+Pretty much the full range of standard SQL operations are supported as constructed Python expressions, including joins, ordering, grouping, functions, correlated subqueries, unions, etc. Documentation on selecting: [sql_select](rel:sql_select).
+
+### Working with Rows
+
+You can see that when we print out the rows returned by an execution result, it prints the rows as tuples. These rows in fact support both the list and dictionary interfaces. The dictionary interface allows the addressing of columns by string column name, or even the original `Column` object:
+
+ {python}
+ >>> row.keys()
+ ['user_id', 'user_name', 'password']
+ >>> row['user_id'], row[1], row[users_table.c.password]
+ (4, u'Harry', None)
+
+Addressing the columns in a row based on the original `Column` object is especially handy, as it eliminates the need to work with literal column names altogether.
+
+Result sets also support iteration. We'll show this with a slightly different form of `select` that allows you to specify the specific columns to be selected:
+
+ {python}
+ >>> for row in select([users_table.c.user_id, users_table.c.user_name]).execute(): # doctest:+NORMALIZE_WHITESPACE
+ ... print row
+ SELECT users.user_id, users.user_name
+ FROM users
+ []
+ (1, u'Mary')
+ (2, u'Tom')
+ (3, u'Fred')
+ (4, u'Harry')
+
+### Table Relationships
+
+Lets create a second table, `email_addresses`, which references the `users` table. To define the relationship between the two tables, we will use the `ForeignKey` construct. We will also issue the `CREATE` statement for the table in one step:
+
+ {python}
+ >>> email_addresses_table = Table('email_addresses', metadata,
+ ... Column('address_id', Integer, primary_key=True),
+ ... Column('email_address', String(100), nullable=False),
+ ... Column('user_id', Integer, ForeignKey('users.user_id'))).create() # doctest:+ELLIPSIS,+NORMALIZE_WHITESPACE
+ CREATE TABLE email_addresses(
+ address_id INTEGER NOT NULL PRIMARY KEY,
+ email_address VARCHAR(100) NOT NULL,
+ user_id INTEGER REFERENCES users(user_id)
+ )
+ ...
+
+Above, the `email_addresses` table is related to the `users` table via the `ForeignKey('users.user_id')`. The `ForeignKey` constructor can take a `Column` object or a string representing the table and column name. When using the string argument, the referenced table must exist within the same `MetaData` object; thats where it looks for the other table!
+
+Next, lets put a few rows in:
+
+ {python}
+ >>> email_addresses_table.insert().execute(
+ ... {'email_address':'tom@tom.com', 'user_id':2},
+ ... {'email_address':'mary@mary.com', 'user_id':1}) #doctest:+ELLIPSIS
+ INSERT INTO email_addresses (email_address, user_id) VALUES (?, ?)
+ [['tom@tom.com', 2], ['mary@mary.com', 1]]
+ COMMIT
+ <sqlalchemy.engine.base.ResultProxy instance at 0x...>
+
+With two related tables, we can now construct a join amongst them using the `join` method:
+
+ {python}
+ >>> r = users_table.join(email_addresses_table).select().execute()
+ SELECT users.user_id, users.user_name, users.password, email_addresses.address_id, email_addresses.email_address, email_addresses.user_id
+ FROM users JOIN email_addresses ON users.user_id = email_addresses.user_id
+ []
+ >>> print [row for row in r]
+ [(1, u'Mary', u'secure', 2, u'mary@mary.com', 1), (2, u'Tom', None, 1, u'tom@tom.com', 2)]
+
+The `join` method is also a standalone function in the `sqlalchemy` namespace. The join condition is figured out from the foreign keys of the Table objects given. The condition (also called the "ON clause") can be specified explicitly, such as in this example where we locate all users that used their email address as their password:
+
+ {python}
+ >>> print join(users_table, email_addresses_table,
+ ... and_(users_table.c.user_id==email_addresses_table.c.user_id,
+ ... users_table.c.password==email_addresses_table.c.email_address)
+ ... )
+ users JOIN email_addresses ON users.user_id = email_addresses.user_id AND users.password = email_addresses.email_address
+
+Working with Object Mappers {@name=orm}
+-----------------------------------------------
+
+Now that we have a little bit of Table and SQL operations covered, lets look into SQLAlchemy's ORM (object relational mapper). With the ORM, you associate Tables (and other *Selectable* units, like queries and table aliases) with Python classes, into units called *Mappers*. Then you can execute queries that return lists of object instances, instead of result sets. The object instances themselves are associated with an object called a *Session*, which automatically tracks changes on each object and supports a "save all at once" operation called a *flush*.
+
+### Creating a Mapper {@name=mapper}
+
+A Mapper is usually created once per Python class, and at its core primarily means to say, "objects of this class are to be stored as rows in this table". Lets create a class called `User`, which will represent a user object that is stored in our `users` table:
+
+ {python}
+ >>> class User(object):
+ ... def __repr__(self):
+ ... return "(User %s,password:%s)" % (self.user_name, self.password)
+
+The class is a new style class (i.e. it extends `object`) and does not require a constructor (although one may be provided if desired). We just have one `__repr__` method on it which will display basic information about the User. Note that the `__repr__` method references the instance variables `user_name` and `password` which otherwise aren't defined. While we are free to explicitly define these attributes and treat them normally, this is optional; as SQLAlchemy's `Mapper` construct will manage them for us, since their names correspond to the names of columns in the `users` table. Lets create a mapper, and observe that these attributes are now defined:
+
+ {python}
+ >>> usermapper = mapper(User, users_table)
+ >>> u1 = User()
+ >>> print u1.user_name
+ None
+ >>> print u1.password
+ None
+
+The `mapper` function returns a new instance of `Mapper`. As it is the first Mapper we have created for the `User` class, it is known as the classes' *primary mapper*. We generally don't need to hold onto the `usermapper` instance variable; SA's ORM can automatically locate this Mapper when it deals with the class, or instances of that class.
+
+### Obtaining a Session {@name=session}
+
+After you create a Mapper, all operations with that Mapper require the usage of an important object called a `Session`. All objects loaded or saved by the Mapper must be *attached* to a `Session` object, which represents a kind of "workspace" of objects that are loaded into memory. A particular object instance can only be attached to one `Session` at a time.
+
+By default, you have to create a `Session` object explicitly before you can load or save objects. Theres several ways to manage sessions, but the most straightforward is to just create one, which we will do by saying, `create_session()`:
+
+ {python}
+ >>> session = create_session()
+ >>> session # doctest:+ELLIPSIS
+ <sqlalchemy.orm.session.Session object at 0x...>
+
+### The Query Object {@name=query}
+
+The Session has all kinds of methods on it to retrieve and store objects, and also to view their current status. The Session also provides an easy interface which can be used to query the database, by giving you an instance to a `Query` object corresponding to a particular Python class:
+
+ {python}
+ >>> query = session.query(User)
+ >>> print query.select_by(user_name='Harry')
+ SELECT users.user_name AS users_user_name, users.password AS users_password, users.user_id AS users_user_id
+ FROM users
+ WHERE users.user_name = ? ORDER BY users.oid
+ ['Harry']
+ [(User Harry,password:None)]
+
+All querying for objects is performed via an instance of `Query`. The various `select` methods on an instance of `Mapper` also use an underlying `Query` object to perform the operation. A `Query` is always bound to a specific `Session`.
+
+Lets turn off the database echoing for a moment, and try out a few methods on `Query`. Methods that end with the suffix `_by` primarily take keyword arguments which correspond to properties on the object. Other methods take `ClauseElement` objects, which are constructed by using `Column` objects inside of Python expressions, in the same way as we did with our SQL select example in the previous section of this tutorial. Using `ClauseElement` structures to query objects is more verbose but more flexible:
+
+ {python}
+ >>> metadata.engine.echo = False
+ >>> print query.select(User.c.user_id==3)
+ [(User Fred,password:None)]
+ >>> print query.get(2)
+ (User Tom,password:None)
+ >>> print query.get_by(user_name='Mary')
+ (User Mary,password:secure)
+ >>> print query.selectfirst(User.c.password==None)
+ (User Tom,password:None)
+ >>> print query.count()
+ 4
+
+Notice that our `User` class has a special attribute `c` attached to it. This 'c' represents the columns on the User's mapper's Table object. Saying `User.c.user_name` is synonymous with saying `users_table.c.user_name`, recalling that `User` is the Python class and `users` is our `Table` object.
+
+### Making Changes {@name=changes}
+
+With a little experience in loading objects, lets see what its like to make changes. First, lets create a new user "Ed". We do this by just constructing the new object. Then, we just add it to the session:
+
+ {python}
+ >>> ed = User()
+ >>> ed.user_name = 'Ed'
+ >>> ed.password = 'edspassword'
+ >>> session.save(ed)
+ >>> ed in session
+ True
+
+Lets also make a few changes on some of the objects in the database. We will load them with our `Query` object, and then change some things.
+
+ {python}
+ >>> mary = query.get_by(user_name='Mary')
+ >>> harry = query.get_by(user_name='Harry')
+ >>> mary.password = 'marysnewpassword'
+ >>> harry.password = 'harrysnewpassword'
+
+At the moment, nothing has been saved to the database; all of our changes are in memory only. What happens if some other part of the application also tries to load 'Mary' from the database and make some changes before we had a chance to save it ? Assuming that the same `Session` is used, loading 'Mary' from the database a second time will issue a second query in order locate the primary key of 'Mary', but will *return the same object instance as the one already loaded*. This behavior is due to an important property of the `Session` known as the **identity map**:
+
+ {python}
+ >>> mary2 = query.get_by(user_name='Mary')
+ >>> mary is mary2
+ True
+
+With the identity map, a single `Session` can be relied upon to keep all loaded instances straight.
+
+As far as the issue of the same object being modified in two different Sessions, that's an issue of concurrency detection; SQLAlchemy does some basic concurrency checks when saving objects, with the option for a stronger check using version ids. See [adv_datamapping](rel:adv_datamapping) for more details.
+
+### Saving {@name=saving}
+
+With a new user "ed" and some changes made on "Mary" and "Harry", lets also mark "Fred" as deleted:
+
+ {python}
+ >>> fred = query.get_by(user_name='Fred')
+ >>> session.delete(fred)
+
+Then to send all of our changes to the database, we `flush()` the Session. Lets turn echo back on to see this happen!:
+
+ {python}
+ >>> metadata.engine.echo = True
+ >>> session.flush()
+ BEGIN
+ UPDATE users SET password=? WHERE users.user_id = ?
+ ['marysnewpassword', 1]
+ UPDATE users SET password=? WHERE users.user_id = ?
+ ['harrysnewpassword', 4]
+ INSERT INTO users (user_name, password) VALUES (?, ?)
+ ['Ed', 'edspassword']
+ DELETE FROM users WHERE users.user_id = ?
+ [3]
+ COMMIT
+
+### Relationships
+
+When our User object contains relationships to other kinds of information, such as a list of email addresses, we can indicate this by using a function when creating the `Mapper` called `relation()`. While there is a lot you can do with relations, we'll cover a simple one here. First, recall that our `users` table has a foreign key relationship to another table called `email_addresses`. A single row in `email_addresses` has a column `user_id` that references a row in the `users` table; since many rows in the `email_addresses` table can reference a single row in `users`, this is called a *one to many* relationship.
+
+First, deal with the `email_addresses` table by itself. We will create a new class `Address` which represents a single row in the `email_addresses` table, and a corresponding `Mapper` which will associate the `Address` class with the `email_addresses` table:
+
+ {python}
+ >>> class Address(object):
+ ... def __init__(self, email_address):
+ ... self.email_address = email_address
+ ... def __repr__(self):
+ ... return "(Address %s)" % (self.email_address)
+ >>> mapper(Address, email_addresses_table) # doctest: +ELLIPSIS
+ <sqlalchemy.orm.mapper.Mapper object at 0x...>
+
+Next, we associate the `User` and `Address` classes together by creating a relation using `relation()`, and then adding that relation to the `User` mapper, using the `add_property` function:
+
+ {python}
+ >>> usermapper.add_property('addresses', relation(Address))
+
+The `relation()` function takes either a class or a Mapper as its first argument, and has many options to further control its behavior. The 'User' mapper has now placed additional property on each `User` instance called `addresses`. SQLAlchemy will automatically determine that this relationship is a one-to-many relationship, and will subsequently create `addresses` as a list. When a new `User` is created, this list will begin as empty.
+
+Lets see what we get for the email addresses already in the database. Since we have made a change to the mapper's configuration, its best that we clear out our `Session`, which is currently holding onto every `User` object we have already loaded:
+
+ {python}
+ >>> session.clear()
+
+We can then treat the `addresses` attribute on each `User` object like a regular list:
+
+ {python}
+ >>> mary = query.get_by(user_name='Mary') # doctest: +NORMALIZE_WHITESPACE
+ SELECT users.user_name AS users_user_name, users.password AS users_password, users.user_id AS users_user_id
+ FROM users
+ WHERE users.user_name = ? ORDER BY users.oid
+ LIMIT 1 OFFSET 0
+ ['Mary']
+ >>> print [a for a in mary.addresses]
+ SELECT email_addresses.user_id AS email_addresses_user_id, email_addresses.address_id AS email_addresses_address_id, email_addresses.email_address AS email_addresses_email_address
+ FROM email_addresses
+ WHERE ? = email_addresses.user_id ORDER BY email_addresses.oid
+ [1]
+ [(Address mary@mary.com)]
+
+Adding to the list is just as easy. New `Address` objects will be detected and saved when we `flush` the Session:
+
+ {python}
+ >>> mary.addresses.append(Address('mary2@gmail.com'))
+ >>> session.flush() # doctest: +NORMALIZE_WHITESPACE
+ BEGIN
+ INSERT INTO email_addresses (email_address, user_id) VALUES (?, ?)
+ ['mary2@gmail.com', 1]
+ COMMIT
+
+Main documentation for using mappers: [datamapping](rel:datamapping)
+
+### Transactions
+
+You may have noticed from the example above that when we say `session.flush()`, SQLAlchemy indicates the names `BEGIN` and `COMMIT` to indicate a transaction with the database. The `flush()` method, since it may execute many statements in a row, will automatically use a transaction in order to execute these instructions. But what if we want to use `flush()` inside of a larger transaction? This is performed via the `SessionTransaction` object, which we can establish using `session.create_transaction()`. Below, we will perform a more complicated `SELECT` statement, make several changes to our collection of users and email addresess, and then create a new user with two email addresses, within the context of a transaction. We will perform a `flush()` in the middle of it to write the changes we have so far, and then allow the remaining changes to be written when we finally `commit()` the transaction. We enclose our operations within a `try/except` block to insure that resources are properly freed:
+
+ {python}
+ >>> transaction = session.create_transaction()
+ >>> try: # doctest: +NORMALIZE_WHITESPACE
+ ... (ed, harry, mary) = session.query(User).select(
+ ... User.c.user_name.in_('Ed', 'Harry', 'Mary'), order_by=User.c.user_name
+ ... )
+ ... del mary.addresses[1]
+ ... harry.addresses.append(Address('harry2@gmail.com'))
+ ... session.flush()
+ ... print "***flushed the session***"
+ ... fred = User()
+ ... fred.user_name = 'fred_again'
+ ... fred.addresses.append(Address('fred@fred.com'))
+ ... fred.addresses.append(Address('fredsnewemail@fred.com'))
+ ... session.save(fred)
+ ... transaction.commit()
+ ... except:
+ ... transaction.rollback()
+ ... raise
+ BEGIN
+ SELECT users.user_name AS users_user_name, users.password AS users_password, users.user_id AS users_user_id
+ FROM users
+ WHERE users.user_name IN (?, ?, ?) ORDER BY users.user_name
+ ['Ed', 'Harry', 'Mary']
+ SELECT email_addresses.user_id AS email_addresses_user_id, email_addresses.address_id AS email_addresses_address_id, email_addresses.email_address AS email_addresses_email_address
+ FROM email_addresses
+ WHERE ? = email_addresses.user_id ORDER BY email_addresses.oid
+ [4]
+ UPDATE email_addresses SET user_id=? WHERE email_addresses.address_id = ?
+ [None, 3]
+ INSERT INTO email_addresses (email_address, user_id) VALUES (?, ?)
+ ['harry2@gmail.com', 4]
+ ***flushed the session***
+ INSERT INTO users (user_name, password) VALUES (?, ?)
+ ['fred_again', None]
+ INSERT INTO email_addresses (email_address, user_id) VALUES (?, ?)
+ ['fred@fred.com', 6]
+ INSERT INTO email_addresses (email_address, user_id) VALUES (?, ?)
+ ['fredsnewemail@fred.com', 6]
+ COMMIT
+
+Main documentation: [unitofwork](rel:unitofwork)
+
+Next Steps
+----------
+
+That covers a quick tour through the basic idea of SQLAlchemy, in its simplest form. Beyond that, one should familiarize oneself with the basics of Sessions, the various patterns that can be used to define different kinds of Mappers and relations among them, the rudimentary SQL types that are available when constructing Tables, and the basics of Engines, SQL statements, and database Connections.
+
class Float(Numeric):
def __init__(self, precision=10)
- # DateTime, Date, and Time work with Python datetime objects
+ # DateTime, Date and Time types deal with datetime objects from the Python datetime module
class DateTime(TypeEngine)
class Date(TypeEngine)
# as bind params, raw bytes to unicode as
# rowset values, using the unicode encoding
# setting on the engine (defaults to 'utf-8')
- class Unicode(TypeDecorator)
+ class Unicode(TypeDecorator):
+ impl = String
# uses the pickle protocol to serialize data
# in/out of Binary columns
- class PickleType(TypeDecorator)
+ class PickleType(TypeDecorator):
+ impl = Binary
More specific subclasses of these types are available, which various database engines may choose to implement specifically, allowing finer grained control over types:
User-defined types can be created, to support either database-specific types, or customized pre-processing of query parameters as well as post-processing of result set data. You can make your own classes to perform these operations. To augment the behavior of a `TypeEngine` type, such as `String`, the `TypeDecorator` class is used:
- {python title="Basic Example"}
+ {python}
import sqlalchemy.types as types
class MyType(types.TypeDecorator):
-Unit of Work
+[alpha_api]: javascript:alphaApi()
+[alpha_implementation]: javascript:alphaImplementation()
+
+Session / Unit of Work {@name=unitofwork}
============
### Overview {@name=overview}
-The concept behind Unit of Work is to track modifications to a field of objects, and then be able to commit those changes to the database in a single operation. Theres a lot of advantages to this, including that your application doesn't need to worry about individual save operations on objects, nor about the required order for those operations, nor about excessive repeated calls to save operations that would be more efficiently aggregated into one step. It also simplifies database transactions, providing a neat package with which to insert into the traditional database begin/commit phase.
+The concept behind Unit of Work is to track modifications to a field of objects, and then be able to flush those changes to the database in a single operation. Theres a lot of advantages to this, including that your application doesn't need to worry about individual save operations on objects, nor about the required order for those operations, nor about excessive repeated calls to save operations that would be more efficiently aggregated into one step. It also simplifies database transactions, providing a neat package with which to insert into the traditional database begin/commit phase.
SQLAlchemy's unit of work includes these functions:
* The ability to monitor scalar and list attributes on object instances, as well as object creates. This is handled via the attributes package.
* The ability to maintain and process a list of modified objects, and based on the relationships set up by the mappers for those objects as well as the foreign key relationships of the underlying tables, figure out the proper order of operations so that referential integrity is maintained, and also so that on-the-fly values such as newly created primary keys can be propigated to dependent objects that need them before they are saved. The central algorithm for this is the *topological sort*.
-* The ability to define custom functionality that occurs within the unit-of-work commit phase, such as "before insert", "after insert", etc. This is accomplished via MapperExtension.
+* The ability to define custom functionality that occurs within the unit-of-work flush phase, such as "before insert", "after insert", etc. This is accomplished via MapperExtension.
* an Identity Map, which is a dictionary storing the one and only instance of an object for a particular table/primary key combination. This allows many parts of an application to get a handle to a particular object without any chance of modifications going to two different places.
-* Thread-local operation. the Identity map as well as its enclosing Unit of Work are normally instantiated and accessed in a manner that is local to the current thread, within an object called a Session. Another concurrently executing thread will therefore have its own Session, so unless an application explicitly shares objects between threads, the operation of the object relational mapping is automatically threadsafe. Session objects can also be constructed manually to allow any user-defined scoping.
+* The sole interface to the unit of work is provided via the `Session` object. Transactional capability, which rides on top of the transactions provided by `Engine` objects, is provided by the `SessionTransaction` object.
+* Thread-locally scoped Session behavior is available as an option, which allows new objects to be automatically added to the Session corresponding to by the *default Session context*. Without a default Session context, an application must explicitly create a Session manually as well as add new objects to it. The default Session context, disabled by default, can also be plugged in with other user-defined schemes, which may also take into account the specific class being dealt with for a particular operation.
+* The Session object in SQLAlchemy 0.2 borrows conceptually from that of [Hibernate](http://www.hibernate.org), a leading ORM for Java that is largely based on [JSR-220](http://jcp.org/aboutJava/communityprocess/pfd/jsr220/index.html). SQLAlchemy, under no obligation to conform to EJB specifications, is in general very different from Hibernate, providing a different paradigm for producing queries, a SQL API that is useable independently of the ORM, and of course Pythonic configuration as opposed to XML; however, JSR-220/Hibernate makes some pretty good suggestions with regards to the mechanisms of persistence.
+
+### Object States {@name=states}
+
+When dealing with mapped instances with regards to Sessions, an instance may be *attached* or *unattached* to a particular Session. An instance also may or may not correspond to an actual row in the database. The product of these two binary conditions yields us four general states a particular instance can have within the perspective of the Session:
+
+* *Transient* - a transient instance exists within memory only and is not associated with any Session. It also has no database identity and does not have a corresponding record in the database. When a new instance of a class is constructed, and no default session context exists with which to automatically attach the new instance, it is a transient instance. The instance can then be saved to a particular session in which case it becomes a *pending* instance. If a default session context exists, new instances are added to that Session by default and therefore become *pending* instances immediately.
-### The Session Interface {@name=session}
+* *Pending* - a pending instance is a Session-attached object that has not yet been assigned a database identity. When the Session is flushed (i.e. changes are persisted to the database), a pending instance becomes persistent.
-The current unit of work is accessed via a Session object. The Session is available in a thread-local context from the objectstore module as follows:
+* *Persistent* - a persistent instance has a database identity and a corresponding record in the database, and is also associated with a particular Session. By "database identity" we mean the object is associated with a table or relational concept in the database combined with a particular primary key in that table. Objects that are loaded by SQLAlchemy in the context of a particular session are automatically considered persistent, as are formerly pending instances which have been subject to a session `flush()`.
+
+* *Detached* - a detached instance is an instance which has a database identity and corresponding row in the database, but is not attached to any Session. This occurs when an instance has been removed from a Session, either because the session itself was cleared or closed, or the instance was explicitly removed from the Session. The object can be re-attached with a session again in which case it becomes Persistent again. Detached instances are useful when an application needs to represent a long-running operation across multiple Sessions, needs to store an object in a serialized state and then restore it later (such as within an HTTP "session" object), or in some cases where code needs to load instances locally which will later be associated with some other Session.
+
+### Acquiring a Session {@name=getting}
+
+A new Session object is constructed via the `create_session()` function:
{python}
- # get the current thread's session
- session = objectstore.get_session()
-
-The Session object acts as a proxy to an underlying UnitOfWork object. Common methods include commit(), begin(), clear(), and delete(). Most of these methods are available at the module level in the objectstore module, which operate upon the Session returned by the get_session() function:
+ session = create_session()
+
+A common option used with `create_session()` is to specify a specific `Engine` or `Connection` to be used for all operations performed by this Session:
+
+ {python}
+ # create an engine
+ e = create_engine('postgres://some/url')
+
+ # create a Session that will use this engine for all operations.
+ # it will open and close Connections as needed.
+ session = create_session(bind_to=e)
+
+ # open a Connection
+ conn = e.connect()
+
+ # create a Session that will use this specific Connection for all operations
+ session = create_session(bind_to=conn)
- {python}# this...
- objectstore.get_session().commit()
- # is the same as this:
- objectstore.commit()
+The session to which an object is attached can be acquired via the `object_session()` function, which returns the appropriate `Session` if the object is pending or persistent, or `None` if the object is transient or detached:
-A description of the most important methods and concepts follows.
+ {python}
+ session = object_session(obj)
-#### Identity Map {@name=identitymap}
+It is possible to install a default "threadlocal" session context by importing a *mod* called `sqlalchemy.mods.threadlocal`. This mod creates a familiar SA 0.1 keyword `objectstore` in the `sqlalchemy` namespace. The `objectstore` may be used directly like a session; all session actions performed on `sqlalchemy.objectstore` will be *proxied* to the thread-local Session:
-The first concept to understand about the Unit of Work is that it is keeping track of all mapped objects which have been loaded from the database, as well as all mapped objects which have been saved to the database in the current session. This means that everytime you issue a `select` call to a mapper which returns results, all of those objects are now installed within the current Session, mapped to their identity.
+ {python}
+ # install 'threadlocal' mod (only need to call this once per application)
+ import sqlalchemy.mods.threadlocal
+
+ # then 'objectstore' is available within the 'sqlalchemy' namespace
+ from sqlalchemy import objectstore
+
+ # flush the current thread-local session using the objectstore directly
+ objectstore.flush()
-In particular, it is insuring that only *one* instance of a particular object, corresponding to a particular database identity, exists within the Session at one time. By "database identity" we mean a table or relational concept in the database combined with a particular primary key in that table. The session accomplishes this task using a dictionary known as an *Identity Map*. When `select` or `get` calls on mappers issue queries to the database, they will in nearly all cases go out to the database on each call to fetch results. However, when the mapper *instantiates* objects corresponding to the result set rows it receives, it will *check the current identity map first* before instantating a new object, and return *the same instance* already present in the identiy map if it already exists.
+ # which is the same as this (assuming we are still on the same thread):
+ session = objectstore.get_session()
+ session.flush()
+
+We will now cover some of the key concepts used by Sessions and its underlying Unit of Work.
+
+### Introduction to the Identity Map {@name=identitymap}
+
+A primary concept of the Session's underlying Unit of Work is that it is keeping track of all persistent instances; recall that a persistent instance has a database identity and is attached to a Session. In particular, the Unit of Work must insure that only *one* copy of a particular persistent instance exists within the Session at any given time. The UOW accomplishes this task using a dictionary known as an *Identity Map*. When a `Query` is used to issue `select` or `get` requests to the database, it will in nearly all cases result in an actual SQL execution to the database, and a corresponding traversal of rows received from that execution. However, when the underlying mapper *instantiates* objects corresponding to the result set rows it receives, it will check the session's identity map first before instantating a new object, and return the same instance already present in the identity map if it already exists, essentially *ignoring* the object state represented by that row. There are several ways to override this behavior and truly refresh an already-loaded instance which are described later, but the main idea is that once your instance is loaded into a particular Session, it will *never change* its state without your explicit approval, regardless of what the database says about it.
-Example:
+For example; below, two separate calls to load an instance with database identity "15" are issued, and the results assigned to two separate variables. However, since the same `Session` was used, the two instances are the same instance:
- {python}mymapper = mapper(MyClass, mytable)
+ {python}
+ mymapper = mapper(MyClass, mytable)
- obj1 = mymapper.selectfirst(mytable.c.id==15)
- obj2 = mymapper.selectfirst(mytable.c.id==15)
+ session = create_session()
+ obj1 = session.query(MyClass).selectfirst(mytable.c.id==15)
+ obj2 = session.query(MyClass).selectfirst(mytable.c.id==15)
>>> obj1 is obj2
True
-The Identity Map is an instance of `weakref.WeakValueDictionary`, so that when an in-memory object falls out of scope, it will be removed automatically. However, this may not be instant if there are circular references upon the object. The current SA attributes implementation places some circular refs upon objects, although this may change in the future. There are other ways to remove object instances from the current session, as well as to clear the current session entirely, which are described later in this section.
+The Identity Map is an instance of `weakref.WeakValueDictionary`, so that when an in-memory object falls out of scope, it will be removed automatically. However, this may not be instant if there are circular references upon the object. To guarantee that an instance is removed from the identity map before removing references to it, use the `expunge()` method, described later, to remove it.
-To view the Session's identity map, it is accessible via the `identity_map` accessor, and is an instance of `weakref.WeakValueDictionary`:
+The Session supports an iterator interface in order to see all objects in the identity map:
{python}
- >>> objectstore.get_session().identity_map.values()
+ for obj in session:
+ print obj
+
+As well as `__contains__()`:
+
+ {python}
+ if obj in session:
+ print "Object is present"
+
+The identity map itself is accessible via the `identity_map` accessor:
+
+ {python}
+ >>> session.identity_map.values()
[<__main__.User object at 0x712630>, <__main__.Address object at 0x712a70>]
-The identity of each object instance is available via the _instance_key property attached to each object instance, and is a tuple consisting of the object's class and an additional tuple of primary key values, in the order that they appear within the table definition:
+The identity of each object instance is available via the `_instance_key` property attached to each object instance, and is a tuple consisting of the object's class and an additional tuple of primary key values, in the order that they appear within the table definition:
{python}
>>> obj._instance_key
(<class 'test.tables.User'>, (7,))
-At the moment that an object is assigned this key, it is also added to the current thread's unit-of-work's identity map.
+At the moment that an object is assigned this key within a `flush()` operation, it is also added to the session's identity map.
-The get() method on a mapper, which retrieves an object based on primary key identity, also checks in the current identity map first to save a database round-trip if possible. In the case of an object lazy-loading a single child object, the get() method is used as well, so scalar-based lazy loads may in some cases not query the database; this is particularly important for backreference relationships as it can save a lot of queries.
+The `get()` method on `Query`, which retrieves an object based on primary key identity, also checks in the Session's identity map first to save a database round-trip if possible. In the case of an object lazy-loading a single child object, the `get()` method is used as well, so scalar-based lazy loads may in some cases not query the database; this is particularly important for backreference relationships as it can save a lot of queries.
+
+### Whats Changed ? {@name=changed}
+
+The next concept is that in addition to the `Session` storing a record of all objects loaded or saved, it also stores lists of all *newly created* (i.e. pending) objects, lists of all persistent objects whose attributes have been *modified*, and lists of all persistent objects that have been marked as *deleted*. These lists are used when a `flush()` call is issued to save all changes. After the flush occurs, these lists are all cleared out.
-Methods on mappers and the objectstore module, which are relevant to identity include the following:
+These records are all tracked by a collection of `Set` objects (which are a SQLAlchemy-specific instance called a `HashSet`) that are also viewable off the `Session`:
{python}
- # assume 'm' is a mapper
- m = mapper(User, users)
+ # pending objects recently added to the Session
+ session.new
+
+ # persistent objects with modifications
+ session.dirty
+
+ # persistent objects that have been marked as deleted via session.delete(obj)
+ session.deleted
+
+Unlike the identity map, the `new`, `dirty`, and `deleted` lists are *not weak referencing.* This means if you abandon all references to new or modified objects within a session, *they are still present* and will be saved on the next flush operation, unless they are removed from the Session explicitly (more on that later). The `new` list may change in a future release to be weak-referencing, however for the `deleted` list, one can see that its quite natural for a an object marked as deleted to have no references in the application, yet a DELETE operation is still required.
- # get the identity key corresponding to a primary key
- key = m.identity_key(7)
+### The Session API {@name=api}
- # for composite key, list out the values in the order they
- # appear in the table
- key = m.identity_key(12, 'rev2')
+#### query() {@name=query}
- # get the identity key given a primary key
- # value as a tuple and a class
- key = objectstore.get_id_key((12, 'rev2'), User)
+The `query()` function takes a class or `Mapper` as an argument, along with an optional `entity_name` parameter, and returns a new `Query` object which will issue mapper queries within the context of this Session. If a Mapper is passed, then the Query uses that mapper. Otherwise, if a class is sent, it will locate the primary mapper for that class which is used to construct the Query.
- # get the identity key for an object, whether or not it actually
- # has one attached to it (m is the mapper for obj's class)
- key = m.instance_key(obj)
+ {python}
+ # query from a class
+ session.query(User).select_by(name='ed')
+
+ # query from a mapper
+ query = session.query(usermapper)
+ x = query.get(1)
- # is this key in the current identity map?
- session.has_key(key)
+ # query from a class mapped with entity name 'alt_users'
+ q = session.query(User, entity_name='alt_users')
+ y = q.options(eagerload('orders')).select()
+
+`entity_name` is an optional keyword argument sent with a class object, in order to further qualify which primary mapper to be used; this only applies if there was a `Mapper` created with that particular class/entity name combination, else an exception is raised. All of the methods on Session which take a class or mapper argument also take the `entity_name` argument, so that a given class can be properly matched to the desired primary mapper.
- # is this object in the current identity map?
- session.has_instance(obj)
+All instances retrieved by the returned `Query` object will be stored as persistent instances within the originating `Session`.
- # get this object from the current identity map based on
- # singular/composite primary key, or if not go
- # and load from the database
- obj = m.get(12, 'rev2')
+#### get() {@name=get}
-#### Whats Changed ? {@name=changed}
+Given a class or mapper, a scalar or tuple-based identity, and an optional `entity_name` keyword argument, creates a `Query` corresponding to the given mapper or class/entity_name combination, and calls the `get()` method with the given identity value. If the object already exists within this Session, it is simply returned, else it is queried from the database. If the instance is not found, the method returns `None`.
-The next concept is that in addition to the Session storing a record of all objects loaded or saved, it also stores records of all *newly created* objects, records of all objects whose attributes have been *modified*, records of all objects that have been marked as *deleted*, and records of all *modified list-based attributes* where additions or deletions have occurred. These lists are used when a `commit()` call is issued to save all changes. After the commit occurs, these lists are all cleared out.
+ {python}
+ # get Employer primary key 5
+ employer = session.get(Employer, 5)
-These records are all tracked by a collection of `Set` objects (which are a SQLAlchemy-specific instance called a `HashSet`) that are also viewable off the Session:
+ # get Report composite primary key 7,12, using mapper 'report_mapper_b'
+ report = session.get(Report, (7,12), entity_name='report_mapper_b')
+
+
+#### load() {@name=load}
+
+load() is similar to get() except it will raise an exception if the instance does not exist in the database. It will also load the object's data from the database in all cases, and **overwrite** all changes on the object if it already exists in the session with the latest data from the database.
{python}
- # new objects that were just constructed
- session.new
+ # load Employer primary key 5
+ employer = session.load(Employer, 5)
- # objects that exist in the database, that were modified
- session.dirty
+ # load Report composite primary key 7,12, using mapper 'report_mapper_b'
+ report = session.load(Report, (7,12), entity_name='report_mapper_b')
- # objects that have been marked as deleted via session.delete(obj)
- session.deleted
+#### save() {@name=save}
- # list-based attributes thave been appended
- session.modified_lists
-
-Heres an interactive example, assuming the `User` and `Address` mapper setup first outlined in [datamapping_relations](rel:datamapping_relations):
+save() is called with a single transient (unsaved, unattached) instance as an argument, which is then added to the Session and becomes pending. When the session is next `flush`ed, the instance will be saved to the database uponwhich it becomes persistent (saved, attached). If the given instance is not transient, meaning it is either attached to an existing Session or it has a database identity, an exception is raised.
{python}
- >>> # get the current thread's session
- >>> session = objectstore.get_session()
-
- >>> # create a new object, with a list-based attribute
- >>> # containing two more new objects
- >>> u = User(user_name='Fred')
- >>> u.addresses.append(Address(city='New York'))
- >>> u.addresses.append(Address(city='Boston'))
+ user1 = User(name='user1')
+ user2 = User(name='user2')
+ session.save(user1)
+ session.save(user2)
- >>> # objects are in the "new" list
- >>> session.new
- [<__main__.User object at 0x713630>,
- <__main__.Address object at 0x713a70>,
- <__main__.Address object at 0x713b30>]
-
- >>> # view the "modified lists" member,
- >>> # reveals our two Address objects as well, inside of a list
- >>> session.modified_lists
- [[<__main__.Address object at 0x713a70>, <__main__.Address object at 0x713b30>]]
+ session.flush() # write changes to the database
- >>> # lets view what the class/ID is for the list object
- >>> ["%s %s" % (l.__class__, id(l)) for l in session.modified_lists]
- ['sqlalchemy.mapping.unitofwork.UOWListElement 7391872']
-
- >>> # now commit
- >>> session.commit()
-
- >>> # the "new" list is now empty
- >>> session.new
- []
-
- >>> # the "modified lists" list is now empty
- >>> session.modified_lists
- []
-
- >>> # now lets modify an object
- >>> u.user_name='Ed'
-
- >>> # it gets placed in the "dirty" list
- >>> session.dirty
- [<__main__.User object at 0x713630>]
-
- >>> # delete one of the addresses
- >>> session.delete(u.addresses[0])
-
- >>> # and also delete it off the User object, note that
- >>> # this is *not automatic* when using session.delete()
- >>> del u.addresses[0]
- >>> session.deleted
- [<__main__.Address object at 0x713a70>]
-
- >>> # commit
- >>> session.commit()
-
- >>> # all lists are cleared out
- >>> session.new, session.dirty, session.modified_lists, session.deleted
- ([], [], [], [])
-
- >>> # identity map has the User and the one remaining Address
- >>> session.identity_map.values()
- [<__main__.Address object at 0x713b30>, <__main__.User object at 0x713630>]
-
-Unlike the identity map, the `new`, `dirty`, `modified_lists`, and `deleted` lists are *not weak referencing.* This means if you abandon all references to new or modified objects within a session, *they are still present* and will be saved on the next commit operation, unless they are removed from the Session explicitly (more on that later). The `new` list may change in a future release to be weak-referencing, however for the `deleted` list, one can see that its quite natural for a an object marked as deleted to have no references in the application, yet a DELETE operation is still required.
+save() is called automatically for new instances by the classes' associated mapper, if a default Session context is in effect (such as a thread-local session), which means that newly created instances automatically become pending. If there is no default session available, then the instance remains transient (unattached) until it is explicitly added to a Session via the save() method.
+
+A transient instance also can be automatically `save`ed if it is associated with a parent object which specifies `save-update` within its `cascade` rules, and that parent is already attached or becomes attached to a Session. For more information on `cascade`, see the next section.
+
+The `save_or_update()` method, covered later, is a convenience method which will call the `save()` or `update()` methods appropriately dependening on whether or not the instance has a database identity (but the instance still must be unattached).
-#### Commit {@name=commit}
+#### flush() {@name=flush}
-This is the main gateway to what the Unit of Work does best, which is save everything ! It should be clear by now that a commit looks like:
+This is the main gateway to what the Unit of Work does best, which is save everything ! It should be clear by now what a flush looks like:
{python}
- objectstore.get_session().commit()
+ session.flush()
-It also can be called with a list of objects; in this form, the commit operation will be limited only to the objects specified in the list, as well as any child objects within `private` relationships for a delete operation:
+It also can be called with a list of objects; in this form, the flush operation will be limited only to the objects specified in the list, as well as any child objects within `private` relationships for a delete operation:
{python}
# saves only user1 and address2. all other modified
# objects remain present in the session.
- objectstore.get_session().commit(user1, address2)
+ session.flush(user1, address2)
-This second form of commit should be used more carefully as it will not necessarily locate other dependent objects within the session, whose database representation may have foreign constraint relationships with the objects being operated upon.
+This second form of flush should be used carefully as it will not necessarily locate other dependent objects within the session, whose database representation may have foreign constraint relationships with the objects being operated upon.
-##### What Commit is, and Isn't {@name=whatis}
+##### Notes on Flush {@name=whatis}
-The purpose of the Commit operation, as defined by the `objectstore` package, is to instruct the Unit of Work to analyze its lists of modified objects, assemble them into a dependency graph, fire off the appopriate INSERT, UPDATE, and DELETE statements via the mappers related to those objects, and to synchronize column-based object attributes that correspond directly to updated/inserted database columns.
+A common misconception about the `flush()` operation is that once performed, the newly persisted instances will automatically have related objects attached to them, based on the values of primary key identities that have been assigned to the instances before they were persisted. An example would be, you create a new `Address` object, set `address.user_id` to 5, and then `flush()` the session. The erroneous assumption would be that there is now a `User` object of identity "5" attached to the `Address` object, but in fact this is not the case. If you were to `refresh()` the `Address`, invalidating its current state and re-loading, *then* it would have the appropriate `User` object present.
-Its important to note that the *objectstore.get_session().commit() operation is not the same as the commit() operation on SQLEngine.* A `SQLEngine`, described in [database](rel:database), has its own `begin` and `commit` statements which deal directly with transactions opened on DBAPI connections. While the `session.commit()` makes use of these calls in order to issue its own SQL within a database transaction, it is only dealing with "committing" its own in-memory changes and only has an indirect relationship with database connection objects.
-
-The `session.commit()` operation also does not affect any `relation`-based object attributes, that is attributes that reference other objects or lists of other objects, in any way. A brief list of what will *not* happen includes:
-
-* It will not append or delete any object instances to/from any list-based object attributes. Any objects that have been created or marked as deleted will be updated as such in the database, but if a newly deleted object instance is still attached to a parent object's list, the object itself will remain in that list.
-* It will not set or remove any scalar references to other objects, even if the corresponding database identifier columns have been committed.
+This misunderstanding is related to the observed behavior of backreferences ([datamapping_relations_backreferences](rel:datamapping_relations_backreferences)), which automatically associates an instance "A" with another instance "B", in response to the manual association of instance "B" to instance "A" by the user. The backreference operation occurs completely externally to the `flush()` operation, and is pretty much the only example of a SQLAlchemy feature that manipulates the relationships of persistent objects.
-This means, if you set `address.user_id` to 5, that integer attribute will be saved, but it will not place an `Address` object in the `addresses` attribute of the corresponding `User` object. In some cases there may be a lazy-loader still attached to an object attribute which when first accesed performs a fresh load from the database and creates the appearance of this behavior, but this behavior should not be relied upon as it is specific to lazy loading and also may disappear in a future release. Similarly, if the `Address` object is marked as deleted and a commit is issued, the correct DELETE statements will be issued, but if the object instance itself is still attached to the `User`, it will remain.
-
-So the primary guideline for dealing with commit() is, *the developer is responsible for maintaining in-memory objects and their relationships to each other, the unit of work is responsible for maintaining the database representation of the in-memory objects.* The typical pattern is that the manipulation of objects *is* the way that changes get communicated to the unit of work, so that when the commit occurs, the objects are already in their correct in-memory representation and problems dont arise. The manipulation of identifier attributes like integer key values as well as deletes in particular are a frequent source of confusion.
+The primary guideline for dealing with `flush()` is, the developer is responsible for maintaining in-memory objects and their relationships to each other, the unit of work is responsible for maintaining the database representation of the in-memory objects. The typical pattern is that the manipulation of objects *is* the way that changes get communicated to the unit of work, so that when the flush occurs, the objects are already in their correct in-memory representation and problems dont arise. The manipulation of identifier attributes like integer key values as well as deletes in particular are a frequent source of confusion.
-A terrific feature of SQLAlchemy which is also a supreme source of confusion is the backreference feature, described in [datamapping_relations_backreferences](rel:datamapping_relations_backreferences). This feature allows two types of objects to maintain attributes that reference each other, typically one object maintaining a list of elements of the other side, which contains a scalar reference to the list-holding object. When you append an element to the list, the element gets a "backreference" back to the object which has the list. When you attach the list-holding element to the child element, the child element gets attached to the list. *This feature has nothing to do whatsoever with the Unit of Work.*`*` It is strictly a small convenience feature intended to support the developer's manual manipulation of in-memory objects, and the backreference operation happens at the moment objects are attached or removed to/from each other, independent of any kind of database operation. It does not change the golden rule, that the developer is reponsible for maintaining in-memory object relationships.
+#### close() {@name=close}
-`*` there is an internal relationship between two `relations` that have a backreference, which state that a change operation is only logged once to the unit of work instead of two separate changes since the two changes are "equivalent", so a backreference does affect the information that is sent to the Unit of Work. But the Unit of Work itself has no knowledge of this arrangement and has no ability to affect it.
+This method first calls `clear()`, removing all objects from this `Session`, and then insures that any transactional resources are closed.
-#### Delete {@name=delete}
+#### delete() {@name=delete}
-The delete call places an object or objects into the Unit of Work's list of objects to be marked as deleted:
+The `delete` method places an instance into the Unit of Work's list of objects to be marked as deleted:
{python}
- # mark three objects to be deleted
- objectstore.get_session().delete(obj1, obj2, obj3)
+ # mark two objects to be deleted
+ session.delete(obj1)
+ session.delete(obj2)
- # commit
- objectstore.get_session().commit()
-
-When objects which contain references to other objects are deleted, the mappers for those related objects will issue UPDATE statements for those objects that should no longer contain references to the deleted object, setting foreign key identifiers to NULL. Similarly, when a mapper contains relations with the `private=True` option, DELETE statements will be issued for objects within that relationship in addition to that of the primary deleted object; this is called a *cascading delete*.
+ # flush
+ session.flush()
-As stated before, the purpose of delete is strictly to issue DELETE statements to the database. It does not affect the in-memory structure of objects, other than changing the identifying attributes on objects, such as setting foreign key identifiers on updated rows to None. It has no effect on the status of references between object instances, nor any effect on the Python garbage-collection status of objects.
+The delete operation will have an effect on instances that are attached to the deleted instance according to the `cascade` style of the relationship; cascade rules are described further in the following section. By default, associated instances may need to be updated in the database to reflect that they no longer are associated with the parent object, before the parent is deleted. If the relationship specifies `cascade="delete"`, then the associated instance will also be deleted upon flush, assuming it is still attached to the parent. If the relationship additionally includes the `delete-orphan` cascade style, the associated instance will be deleted if it is still attached to the parent, or is unattached to any other parent.
-#### Clear {@name=clear}
+The `delete()` operation has no relationship to the in-memory status of the instance, including usage of the `del` Python statement. An instance marked as deleted and flushed will still exist within memory until references to it are freed; similarly, removing an instance from memory via the `del` statement will have no effect, since the persistent instance will still be referenced by its Session. Obviously, if the instance is removed from the Session and then totally dereferenced, it will no longer exist in memory, but also won't exist in any Session and is therefore not deleted from the database.
-To clear out the current thread's UnitOfWork, which has the effect of discarding the Identity Map and the lists of all objects that have been modified, just issue a clear:
+#### clear() {@name=clear}
+
+This method detaches all instances from the Session, sending them to the detached or transient state as applicable, and replaces the underlying UnitOfWork with a new one.
{python}
- # via module
- objectstore.clear()
-
- # or via Session
- objectstore.get_session().clear()
+ session.clear()
-This is the easiest way to "start fresh", as in a web application that wants to have a newly loaded graph of objects on each request. Any object instances created before the clear operation should either be discarded or at least not used with any Mapper or Unit Of Work operations (with the exception of `import_instance()`), as they no longer have any relationship to the current Unit of Work, and their behavior with regards to the current session is undefined.
+The `clear()` method is particularly useful with a "default context" session such as a thread-local session, which can stay attached to the current thread to handle a new field of objects without having to re-attach a new Session.
-#### Refresh / Expire {@name=refreshexpire}
+#### refresh() / expire() {@name=refreshexpire}
To assist with the Unit of Work's "sticky" behavior, individual objects can have all of their attributes immediately re-loaded from the database, or marked as "expired" which will cause a re-load to occur upon the next access of any of the object's mapped attributes. This includes all relationships, so lazy-loaders will be re-initialized, eager relationships will be repopulated. Any changes marked on the object are discarded:
{python}
# immediately re-load attributes on obj1, obj2
- session.refresh(obj1, obj2)
+ session.refresh(obj1)
+ session.refresh(obj2)
# expire objects obj1, obj2, attributes will be reloaded
# on the next access:
- session.expire(obj1, obj2, obj3)
+ session.expire(obj1)
+ session.expire(obj2)
-#### Expunge {@name=expunge}
+#### expunge() {@name=expunge}
-Expunge simply removes all record of an object from the current Session. This includes the identity map, and all history-tracking lists:
+Expunge removes an object from the Session, sending persistent instances to the detached state, and pending instances to the transient state:
{python}
session.expunge(obj1)
-Use `expunge` when youd like to remove an object altogether from memory, such as before calling `del` on it, which will prevent any "ghost" operations occuring when the session is committed.
+Use `expunge` when youd like to remove an object altogether from memory, such as before calling `del` on it, which will prevent any "ghost" operations occuring when the session is flushed.
-#### Import Instance {@name=import}
+#### bind\_mapper() / bind\_table() {@name=bind}
-The _instance_key attribute placed on object instances is designed to work with objects that are serialized into strings and brought back again. As it contains no references to internal structures or database connections, applications that use caches or session storage which require serialization (i.e. pickling) can store SQLAlchemy-loaded objects. However, as mentioned earlier, an object with a particular database identity is only allowed to exist uniquely within the current unit-of-work scope. So, upon deserializing such an object, it has to "check in" with the current Session. This is achieved via the `import_instance()` method:
+Both of these methods receive two arguments; in the case of `bind_mapper()`, it is a `Mapper` and an `Engine` or `Connection` instance; in the case of `bind_table()`, it is a `Table` instance or other `Selectable` (such as an `Alias`, `Select`, etc.), and an `Engine` or `Connection` instance.
+
+ {python}
+ engine1 = create_engine('sqlite:///file1.db')
+ engine2 = create_engine('mysql://localhost')
+
+ sqlite_conneciton = engine1.connect()
+
+ sess = create_session()
+
+ sess.bind_mapper(mymapper, sqlite_connection) # bind mymapper operations to a single SQLite connection
+ sess.bind_table(email_addresses_table, engine2) # bind operations with the email_addresses_table to mysql
+
+Normally, when a `Session` is created via `create_session()` with no arguments, the Session has no awareness of individual `Engines`, and when mappers use the `Session` to retrieve connections, the underlying `MetaData` each `Table` is associated with is expected to be "bound" to an `Engine`, else no engine can be located and an exception is raised. A second form of `create_session()` takes the argument `bind_to=engine_or_connection`, where all SQL operations performed by this `Session` use the single `Engine` or `Connection` (collectively known as a `Connectable`) passed to the constructor. With `bind_mapper()` and `bind_table()`, the operations of individual mapper and/or tables are bound to distinct engines or connections, thereby overriding not only the engine which may be "bound" to the underlying `MetaData`, but also the `Engine` or `Connection` which may have been passed to the `create_session()` function. Configurations which interact with multiple explicit database connections at one time must use either or both of these methods in order to associate `Session` operations with the appropriate connection resource.
+
+Binding a `Mapper` to a resource takes precedence over a `Table` bind, meaning if mapper A is associated with table B, and the Session binds mapper A to connection X and table B to connection Y, an operation with mapper A will use connection X, not connection Y.
+
+#### update() {@name=update}
+
+The update() method is used *only* with detached instances. A detached instance only exists if its `Session` was cleared or closed, or the instance was `expunge()`d from its session. `update()` will re-attach the detached instance with this Session, bringing it back to the persistent state, and allowing any changes on the instance to be saved when the `Session` is next `flush`ed. If the instance is already attached to an existing `Session`, an exception is raised.
+
+A detached instance also can be automatically `update`ed if it is associated with a parent object which specifies `save-update` within its `cascade` rules, and that parent is already attached or becomes attached to a Session. For more information on `cascade`, see the next section.
+
+The `save_or_update()` method is a convenience method which will call the `save()` or `update()` methods appropriately dependening on whether or not the instance has a database identity (but the instance still must be unattached).
+
+#### save\_or\_update() {@name=saveorupdate}
+
+This method is a combination of the `save()` and `update()` methods, which will examine the given instance for a database identity (i.e. if it is transient or detached), and will call the implementation of `save()` or `update()` as appropriate. Use `save_or_update()` to add unattached instances to a session when you're not sure if they were newly created or not. Like `save()` and `update()`, `save_or_update()` cascades along the `save-update` cascade indicator, described in the `cascade` section below.
+
+#### merge() {@name=merge}
+
+Feature Status: [Alpha Implementation][alpha_implementation]
+
+`merge()` is used to return the persistent version of an instance that is not attached to this Session. When passed an instance, if an instance with its database identity already exists within this Session, it is returned. If the instance does not exist in this Session, it is loaded from the database and then returned.
+
+A future version of `merge()` will also update the Session's instance with the state of the given instance (hence the name "merge").
+
+This method is useful for bringing in objects which may have been restored from a serialization, such as those stored in an HTTP session:
{python}
# deserialize an object
myobj = pickle.loads(mystring)
- # "import" it. if the objectstore already had this object in the
+ # "merge" it. if the session already had this object in the
# identity map, then you get back the one from the current session.
- myobj = session.import_instance(myobj)
+ myobj = session.merge(myobj)
+
+Note that `merge()` *does not* associate the given instance with the Session; it remains detached (or attached to whatever Session it was already attached to).
-Note that the import_instance() function will either mark the deserialized object as the official copy in the current identity map, which includes updating its _instance_key with the current application's class instance, or it will discard it and return the corresponding object that was already present. Thats why its important to receive the return results from the method and use the result as the official object instance.
+### Cascade rules {@name=cascade}
+
+Feature Status: [Alpha Implementation][alpha_implementation]
-#### Begin {@name=begin}
+Mappers support the concept of configurable *cascade* behavior on `relation()`s. This behavior controls how the Session should treat the instances that have a parent-child relationship with another instance that is operated upon by the Session. Cascade is indicated as a comma-separated list of string keywords, with the possible values `all`, `delete`, `save-update`, `refresh-expire`, `merge`, `expunge`, and `delete-orphan`.
-The "scope" of the unit of work commit can be controlled further by issuing a begin(). A begin operation constructs a new UnitOfWork object and sets it as the currently used UOW. It maintains a reference to the original UnitOfWork as its "parent", and shares the same identity map of objects that have been loaded from the database within the scope of the parent UnitOfWork. However, the "new", "dirty", and "deleted" lists are empty. This has the effect that only changes that take place after the begin() operation get logged to the current UnitOfWork, and therefore those are the only changes that get commit()ted. When the commit is complete, the "begun" UnitOfWork removes itself and places the parent UnitOfWork as the current one again.
-The begin() method returns a transactional object, upon which you can call commit() or rollback(). *Only this transactional object controls the transaction* - commit() upon the Session will do nothing until commit() or rollback() is called upon the transactional object.
+Cascading is configured by setting the `cascade` keyword argument on a `relation()`:
{python}
- # modify an object
- myobj1.foo = "something new"
-
- # begin
- trans = session.begin()
-
- # modify another object
- myobj2.lala = "something new"
-
- # only 'myobj2' is saved
- trans.commit()
-
-begin/commit supports the same "nesting" behavior as the SQLEngine (note this behavior is not the original "nested" behavior), meaning that many begin() calls can be made, but only the outermost transactional object will actually perform a commit(). Similarly, calls to the commit() method on the Session, which might occur in function calls within the transaction, will not do anything; this allows an external function caller to control the scope of transactions used within the functions.
-
-### Advanced UnitOfWork Management {@name=advscope}
+ mapper(Order, order_table, properties={
+ 'items' : relation(Item, items_table, cascade="all, delete-orphan"),
+ 'customer' : relation(User, users_table, user_orders_table, cascade="save-update"),
+ })
-#### Nesting UnitOfWork in a Database Transaction {@name=transactionnesting}
+The above mapper specifies two relations, `items` and `customer`. The `items` relationship specifies "all, delete-orphan" as its `cascade` value, indicating that all `save`, `update`, `merge`, `expunge`, `refresh` `delete` and `expire` operations performed on a parent `Order` instance should also be performed on the child `Item` instances attached to it (`save` and `update` are cascaded using the `save_or_update()` method, so that the database identity of the instance doesn't matter). The `delete-orphan` cascade value additionally indicates that if an `Item` instance is no longer associated with an `Order`, it should also be deleted. The "all, delete-orphan" cascade argument allows a so-called *lifecycle* relationship between an `Order` and an `Item` object.
-The UOW commit operation places its INSERT/UPDATE/DELETE operations within the scope of a database transaction controlled by a SQLEngine:
+The `customer` relationship specifies only the "save-update" cascade value, indicating most operations will not be cascaded from a parent `Order` instance to a child `User` instance, except for if the `Order` is attached with a particular session, either via the `save()`, `update()`, or `save-update()` method.
- {python}
- engine.begin()
- try:
- # run objectstore update operations
- except:
- engine.rollback()
- raise
- engine.commit()
-
-If you recall from the [dbengine_transactions](rel:dbengine_transactions) section, the engine's begin()/commit() methods support reentrant behavior. This means you can nest begin and commits and only have the outermost begin/commit pair actually take effect (rollbacks however, abort the whole operation at any stage). From this it follows that the UnitOfWork commit operation can be nested within a transaction as well:
+Additionally, when a child item is attached to a parent item that specifies the "save-update" cascade value on the relationship, the child is automatically passed to `save_or_update()` (and the operation is further cascaded to the child item).
+
+Note that cascading doesn't do anything that isn't possible by manually calling Session methods on individual instances within a hierarchy, it merely automates common operations on a group of associated instances.
+
+The default value for `cascade` on `relation()`s is `save-update`, and the `private=True` keyword argument is a synonym for `cascade="all, delete-orphan"`.
+
+### SessionTransaction {@name=transaction}
+
+SessionTransaction is a multi-engine transaction manager, which aggregates one or more Engine/Connection pairs and keeps track of a Transaction object for each one. As the Session receives requests to execute SQL statements, it uses the Connection that is referenced by the SessionTransaction. At commit time, the underyling Session is flushed, and each Transaction is the committed.
+
+Example usage is as follows:
{python}
- engine.begin()
+ sess = create_session()
+ trans = sess.create_transaction()
try:
- # perform custom SQL operations
- objectstore.commit()
- # perform custom SQL operations
+ item1 = sess.query(Item).get(1)
+ item2 = sess.query(Item).get(2)
+ item1.foo = 'bar'
+ item2.bar = 'foo'
+ trans.commit()
except:
- engine.rollback()
+ trans.rollback()
raise
- engine.commit()
-#### Per-Object Sessions {@name=object}
+The `create_transaction()` method creates a new SessionTransaction object but does not declare any connection/transaction resources. At the point of the first `get()` call, a connection resource is opened off the engine that corresponds to the Item classes' mapper and is stored within the `SessionTransaction` with an open `Transaction`. When `trans.commit()` is called, the `flush()` method is called on the `Session` and the corresponding update statements are issued to the database within the scope of the transaction already opened; afterwards, the underying Transaction is committed, and connection resources are freed.
-Sessions can be created on an ad-hoc basis and used for individual groups of objects and operations. This has the effect of bypassing the normal thread-local Session and explicitly using a particular Session:
+`SessionTransaction`, like the `Transaction` off of `Connection` also supports "nested" behavior, and is safe to pass to other functions which then issue their own `begin()`/`commit()` pair; only the outermost `begin()`/`commit()` pair actually affects the transaction, and any call to `rollback()` within a particular call stack will issue a rollback.
- {python}
- # make a new Session with a global UnitOfWork
- s = objectstore.Session()
-
- # make objects bound to this Session
- x = MyObj(_sa_session=s)
-
- # perform mapper operations bound to this Session
- # (this function coming soon)
- r = MyObj.mapper.using(s).select_by(id=12)
-
- # get the session that corresponds to an instance
- s = objectstore.get_session(x)
-
- # commit
- s.commit()
+Note that while SessionTransaction is capable of tracking multiple transactions across multiple databases, it currently is in no way a fully functioning two-phase commit engine; generally, when dealing with multiple databases simultaneously, there is the distinct possibility that a transaction can succeed on the first database and fail on the second, which for some applications may be an invalid state. If this is an issue, its best to either refrain from spanning transactions across databases, or to look into some of the available technologies in this area, such as [Zope](http://www.zope.org) which offers a two-phase commit engine; some users have already created their own SQLAlchemy/Zope hybrid implementations to deal with scenarios like these.
- # perform a block of operations with this session set within the current scope
- objectstore.push_session(s)
- try:
- r = mapper.select_by(id=12)
- x = new MyObj()
- objectstore.commit()
- finally:
- objectstore.pop_session()
+#### Using SQL with SessionTransaction {@name=sql}
-##### Nested Transaction Sessions {@name=nested}
+The SessionTransaction can interact with direct SQL queries in two general ways. Either specific `Connection` objects can be associated with the `SessionTransaction`, which are then useable both for direct SQL as well as within `flush()` operations performed by the `SessionTransaction`, or via accessing the `Connection` object automatically referenced within the `SessionTransaction`.
-Sessions also now support a "nested transaction" feature whereby a second Session can use a different database connection. This can be used inside of a larger database transaction to issue commits to the database that will be committed independently of the larger transaction's status:
+To associate a specific `Connection` with the `SessionTransaction`, use the `add()` method:
- {python}
- engine.begin()
+ {python title="Associate a Connection with the SessionTransaction"}
+ connection = engine.connect()
+ trans = session.create_transaction()
try:
- a = MyObj()
- b = MyObj()
-
- sess = Session(nest_on=engine)
- objectstore.push_session(sess)
- try:
- c = MyObj()
- objectstore.commit() # will commit "c" to the database,
- # even if the external transaction rolls back
- finally:
- objectstore.pop_session()
-
- objectstore.commit() # commit "a" and "b" to the database
- engine.commit()
+ trans.add(connection)
+ connection.execute(mytable.update(), {'col1':4, 'col2':17})
+ session.flush() # flush() operation will use the same connection
+ trans.commit()
except:
- engine.rollback()
+ trans.rollback()
raise
-#### Custom Session Objects/Custom Scopes {@name=scope}
+The `add()` method will key the `Connection`'s underlying `Engine` to this `SessionTransaction`. When mapper operations are performed against this `Engine`, the `Connection` explicitly added will be used. This **overrides** any other `Connection` objects that the underlying Session was associated with, corresponding to the underlying `Engine` of that `Connection`. However, if the `SessionTransaction` itself is already associated with a `Connection`, then an exception is thrown.
-For users who want to make their own Session subclass, or replace the algorithm used to return scoped Session objects (i.e. the objectstore.get_session() method):
+The other way is just to use the `Connection` referenced by the `SessionTransaction`. This is performed via the `connection()` method, and requires passing in a class or `Mapper` which indicates which underlying `Connection` should be returned (recall that different `Mappers` may use different underlying `Engines`). If the `class_or_mapper` argument is `None`, then the `Session` must be globally bound to a specific `Engine` when it was constructed, else the method returns `None`.
- {python title="Create a Session"}
- # make a new Session
- s = objectstore.Session()
+ {python title="Get a Connection from the SessionTransaction"}
+ trans = session.create_transaction()
+ try:
+ connection = trans.connection(UserClass) # get the Connection used by the UserClass' Mapper
+ connection.execute(mytable.update(), {'col1':4, 'col2':17})
+ trans.commit()
+ except:
+ trans.rollback()
+ raise
+
+The `connection()` method also exists on the `Session` object itself, and can be called regardless of whether or not a `SessionTransaction` is in progress. If a `SessionTransaction` is in progress, it will return the connection referenced by the transaction. If an `Engine` is being used with `threadlocal` strategy, the `Connection` returned will correspond to the connection resources that are bound to the current thread, if any (i.e. it is obtained by calling `contextual_connection()`).
- # set it as the current thread-local session
- objectstore.session_registry.set(s)
+#### Using Engine-level Transactions with Sessions
- {python title="Create a custom Registry Algorithm"}
- # set the objectstore's session registry to a different algorithm
+The transactions issued by `SessionTransaction` as well as internally by the `Session`'s `flush()` operation use the same `Transaction` object off of `Connection` that is publically available. Recall that this object supports "nestable" behavior, meaning any number of actors can call `begin()` off a particular `Connection` object, and they will all be managed within the scope of a single transaction. Therefore, the `flush()` operation can similarly take place within the scope of a regular `Transaction`:
- def create_session():
- """creates new sessions"""
- return objectstore.Session()
- def mykey():
- """creates contextual keys to store scoped sessions"""
- return "mykey"
-
- objectstore.session_registry = sqlalchemy.util.ScopedRegistry(createfunc=create_session, scopefunc=mykey)
+ {python title="Transactions with Sessions"}
+ connection = engine.connect() # Connection
+ session = create_session(bind_to=connection) # Session bound to the Connection
+ trans = connection.begin() # start transaction
+ try:
+ stuff = session.query(MyClass).select() # Session operation uses connection
+ stuff[2].foo = 'bar'
+ connection.execute(mytable.insert(), dict(id=12, value="bar")) # use connection explicitly
+ session.flush() # Session flushes with "connection", using transaction "trans"
+ trans.commit() # commit
+ except:
+ trans.rollback() # or rollback
+ raise
-#### Analyzing Object Commits {@name=logging}
+### Analyzing Object Flushes {@name=logging}
-The objectstore module can log an extensive display of its "commit plans", which is a graph of its internal representation of objects before they are committed to the database. To turn this logging on:
+The session module can log an extensive display of its "flush plans", which is a graph of its internal representation of objects before they are written to the database. To turn this logging on:
{python}
- # make an engine with echo_uow
- engine = create_engine('myengine...', echo_uow=True)
+ # make an Session with echo_uow
+ session = create_session(echo_uow=True)
-Commits will then dump to the standard output displays like the following:
+The `flush()` operation will then dump to the standard output displays like the following:
{code}
Task dump:
|----
The above graph can be read straight downwards to determine the order of operations. It indicates "save User 6016624, process each element in the 'addresses' list on User 6016624, save Address 6034384, Address 6034256".
+
+Of course, one can also get a good idea of the order of operations just by logging the actual SQL statements executed.
+
+The format of the above display is definitely a work in progress and amazingly, is far simpler to read than it was in earlier releases. It will hopefully be further refined in future releases to be more intuitive (while not losing any information).
+import sys\r
+sys.path = ['../../lib', './lib/'] + sys.path\r
+\r
import os\r
import re\r
import doctest\r
+import sqlalchemy.util as util\r
+\r
+# monkeypatch a plain logger\r
+class Logger(object):\r
+ def __init__(self, *args, **kwargs):\r
+ pass\r
+ def write(self, msg):\r
+ print msg\r
+util.Logger = Logger\r
\r
def teststring(s, name, globs=None, verbose=None, report=True, \r
optionflags=0, extraglobs=None, raise_on_error=False, \r
\r
return runner.failures, runner.tries\r
\r
-def replace_file(s, oldfile, newfile):\r
- engine = r"(^\s*>>>\s*[a-zA-Z_]\w*\s*=\s*create_engine\('sqlite',\s*\{'filename':\s*')" + oldfile+ "('\}\)$)"\r
+def replace_file(s, newfile):\r
+ engine = r"'(sqlite|postgres|mysql):///.*'"\r
engine = re.compile(engine, re.MULTILINE)\r
- s, n = re.subn(engine, r'\1' + newfile + r'\2', s, 1)\r
+ s, n = re.subn(engine, "'sqlite:///" + newfile + "'", s)\r
if not n:\r
raise ValueError("Couldn't find suitable create_engine call to replace '%s' in it" % oldfile)\r
return s\r
\r
filename = 'content/tutorial.txt'\r
s = open(filename).read()\r
-s = replace_file(s, 'tutorial.db', ':memory:')\r
+s = replace_file(s, ':memory:')\r
teststring(s, filename)\r
\r
# consumed as Myghty comments.\r
text = re.compile(r'^(?!<&)', re.M).sub(' ', text)\r
\r
- sqlre = re.compile(r'{sql}(.*?)((?:SELECT|INSERT|DELETE|UPDATE|CREATE|DROP).*?)\n\s*(\n|$)', re.S)\r
+ sqlre = re.compile(r'{sql}(.*?)((?:SELECT|INSERT|DELETE|UPDATE|CREATE|DROP|PRAGMA|DESCRIBE).*?)\n\s*(\n|$)', re.S)\r
+ if sqlre.search(text) is not None:\r
+ use_sliders = False\r
+ else:\r
+ use_sliders = True\r
+ \r
text = sqlre.sub(r"<&formatting.myt:poplink&>\1\n<&|formatting.myt:codepopper, link='sql'&>\2</&>\n\n", text)\r
\r
sqlre2 = re.compile(r'{opensql}(.*?)((?:SELECT|INSERT|DELETE|UPDATE|CREATE|DROP).*?)\n\s*(\n|$)', re.S)\r
text = sqlre2.sub(r"<&|formatting.myt:poppedcode &>\1\n\2</&>\n\n", text)\r
\r
pre_parent = parent[pre]\r
+ opts = {}\r
if type == 'python':\r
- syntype = 'python'\r
+ opts['syntaxtype'] = 'python'\r
else:\r
- syntype = None\r
+ opts['syntaxtype'] = None\r
+\r
if title is not None:\r
- tag = MyghtyTag(CODE_BLOCK, {'title':title, 'syntaxtype':syntype})\r
- else:\r
- tag = MyghtyTag(CODE_BLOCK, {'syntaxtype':syntype})\r
+ opts['title'] = title\r
+ \r
+ if use_sliders:\r
+ opts['use_sliders'] = True\r
+ \r
+ tag = MyghtyTag(CODE_BLOCK, opts)\r
tag.text = text\r
tag.tail = pre.tail\r
pre_parent[index(pre_parent, pre)] = tag\r
print inname, '->', outname\r
input = file(inname).read()\r
html = markdown.markdown(input)\r
- file(inname[:-3] + "html", 'w').write(html)\r
+ #file(inname[:-3] + "html", 'w').write(html)\r
myt = html2myghtydoc(html)\r
file(outname, 'w').write(myt)\r
font-size: 12px;
font-weight: bold;
text-decoration:underline;
- padding:5px;
+ padding:5px 5px 5px 0px;
}
code {
line-height:1.2em;
}
+.sliding_code {
+ font-family:courier, serif;
+ font-size:12px;
+ background-color: #E2E2EB;
+ padding:2px 2px 2px 10px;
+ margin: 5px 5px 5px 5px;
+ line-height:1.2em;
+ overflow:auto;
+}
+
.codepop {
font-weight:bold;
font-family: verdana, sans-serif;
}
}
+function alphaApi() {
+ window.open("alphaapi.html", "_blank", "width=600,height=400, scrollbars=yes,resizable=yes,toolbar=no");
+}
+
+function alphaImplementation() {
+ window.open("alphaimplementation.html", "_blank", "width=600,height=400, scrollbars=yes,resizable=yes,toolbar=no");
+}
\ No newline at end of file
"""a basic Adjacency List model tree."""
-engine = create_engine('sqlite://', echo = True)
-#engine = sqlalchemy.engine.create_engine('mysql', {'db':'test', 'host':'127.0.0.1', 'user':'scott'}, echo=True)
-#engine = sqlalchemy.engine.create_engine('postgres', {'database':'test', 'host':'127.0.0.1', 'user':'scott', 'password':'tiger'}, echo=True)
-#engine = sqlalchemy.engine.create_engine('oracle', {'dsn':os.environ['DSN'], 'user':os.environ['USER'], 'password':os.environ['PASSWORD']}, echo=True)
+metadata = BoundMetaData('sqlite:///', echo=True)
-
-"""create the treenodes table. This is ia basic adjacency list model table."""
-
-trees = Table('treenodes', engine,
+trees = Table('treenodes', metadata,
Column('node_id', Integer, Sequence('treenode_id_seq',optional=False), primary_key=True),
Column('parent_node_id', Integer, ForeignKey('treenodes.node_id'), nullable=True),
Column('node_name', String(50), nullable=False),
)
-
class NodeList(util.OrderedDict):
- """extends an Ordered Dictionary, which is just a dictionary that returns its keys and values
- in order upon iteration. Adds functionality to automatically associate
- the parent of a TreeNode with itself, upon append to the parent's list of child nodes."""
- def __init__(self, parent):
- util.OrderedDict.__init__(self)
- self.parent = parent
+ """subclasses OrderedDict to allow usage as a list-based property."""
def append(self, node):
- node.parent = self.parent
self[node.name] = node
def __iter__(self):
return iter(self.values())
class TreeNode(object):
"""a rich Tree class which includes path-based operations"""
- def __init__(self, name=None):
- self.children = NodeList(self)
+ children = NodeList
+ def __init__(self, name):
+ self.children = NodeList()
self.name = name
self.parent = None
self.id = None
self.parent_id = None
def append(self, node):
if isinstance(node, str):
- self.children.append(TreeNode(node))
- else:
- self.children.append(node)
+ node = TreeNode(node)
+ node.parent = self
+ self.children.append(node)
def __repr__(self):
return self._getstring(0, False)
def __str__(self):
def print_nodes(self):
return self._getstring(0, True)
-# define the mapper. we will make "convenient" property
-# names vs. the more verbose names in the table definition
-
-assign_mapper(TreeNode, trees, properties=dict(
+mapper(TreeNode, trees, properties=dict(
id=trees.c.node_id,
name=trees.c.node_name,
parent_id=trees.c.parent_node_id,
- children=relation(TreeNode, private=True),
+ children=relation(TreeNode, private=True, backref=backref("parent", foreignkey=trees.c.node_id)),
))
print "\n\n\n----------------------------"
print node.print_nodes()
print "\n\n\n----------------------------"
-print "Committing:"
+print "Flushing:"
print "----------------------------"
-objectstore.commit()
+session = create_session()
+session.save(node)
+session.flush()
print "\n\n\n----------------------------"
print "Tree After Save:"
print node.print_nodes()
print "\n\n\n----------------------------"
-print "Committing:"
+print "Flushing:"
print "----------------------------"
-objectstore.commit()
+session.flush()
print "\n\n\n----------------------------"
print "Tree After Save:"
nodeid = node.id
print "\n\n\n----------------------------"
-print "Clearing objectstore, selecting "
+print "Clearing session, selecting "
print "tree new where node_id=%d:" % nodeid
print "----------------------------"
-objectstore.clear()
-t = TreeNode.mapper.select(TreeNode.c.node_id==nodeid)[0]
+session.clear()
+t = session.query(TreeNode).select(TreeNode.c.id==nodeid)[0]
print "\n\n\n----------------------------"
print "Full Tree:"
print "\n\n\n----------------------------"
print "Marking root node as deleted"
-print "and committing:"
+print "and flushing:"
print "----------------------------"
-objectstore.delete(t)
-objectstore.commit()
+session.delete(t)
+session.flush()
import sqlalchemy.util as util
import string, sys, time
-"""a more advanced example of basic_tree.py. illustrates MapperExtension objects which
-add application-specific functionality to a Mapper object."""
+"""a more advanced example of basic_tree.py. treenodes can now reference their "root" node, and
+introduces a new selection method which selects an entire tree of nodes at once, taking
+advantage of a custom MapperExtension to assemble incoming nodes into their correct structure."""
-engine = create_engine('sqlite://', echo = True)
-#engine = sqlalchemy.engine.create_engine('mysql', {'db':'test', 'host':'127.0.0.1', 'user':'scott'}, echo=True)
-#engine = sqlalchemy.engine.create_engine('postgres', {'database':'test', 'host':'127.0.0.1', 'user':'scott', 'password':'tiger'}, echo=True)
-#engine = sqlalchemy.engine.create_engine('oracle', {'dsn':os.environ['DSN'], 'user':os.environ['USER'], 'password':os.environ['PASSWORD']}, echo=True)
+engine = create_engine('sqlite:///:memory:', echo=True)
+
+metadata = BoundMetaData(engine)
"""create the treenodes table. This is ia basic adjacency list model table.
One additional column, "root_node_id", references a "root node" row and is used
in the 'byroot_tree' example."""
-trees = Table('treenodes', engine,
+trees = Table('treenodes', metadata,
Column('node_id', Integer, Sequence('treenode_id_seq',optional=False), primary_key=True),
Column('parent_node_id', Integer, ForeignKey('treenodes.node_id'), nullable=True),
Column('root_node_id', Integer, ForeignKey('treenodes.node_id'), nullable=True),
)
treedata = Table(
- "treedata", engine,
+ "treedata", metadata,
Column('data_id', Integer, primary_key=True),
Column('value', String(100), nullable=False)
)
-
class NodeList(util.OrderedDict):
- """extends an Ordered Dictionary, which is just a dictionary that iterates its keys and values
- in the order they were inserted. Adds an "append" method, which appends a node to the
- dictionary as though it were a list, and also within append automatically associates
- the parent of a TreeNode with itself."""
- def __init__(self, parent):
- util.OrderedDict.__init__(self)
- self.parent = parent
+ """subclasses OrderedDict to allow usage as a list-based property."""
def append(self, node):
- node.parent = self.parent
- node._set_root(self.parent.root)
self[node.name] = node
def __iter__(self):
return iter(self.values())
identifiable root. Any node can return its root node and therefore the "tree" that it
belongs to, and entire trees can be selected from the database in one query, by
identifying their common root ID."""
+ children = NodeList
def __init__(self, name):
"""for data integrity, a TreeNode requires its name to be passed as a parameter
to its constructor, so there is no chance of a TreeNode that doesnt have a name."""
self.name = name
- self.children = NodeList(self)
+ self.children = NodeList()
self.root = self
self.parent = None
self.id = None
c._set_root(root)
def append(self, node):
if isinstance(node, str):
- self.children.append(TreeNode(node))
- else:
- self.children.append(node)
+ node = TreeNode(node)
+ node.parent = self
+ node._set_root(self.root)
+ self.children.append(node)
def __repr__(self):
return self._getstring(0, False)
def __str__(self):
class TreeLoader(MapperExtension):
"""an extension that will plug-in additional functionality to the Mapper."""
- def create_instance(self, mapper, row, imap, class_):
- """creates an instance of a TreeNode. since the TreeNode constructor requires
- the 'name' argument, this method pulls the data from the database row directly."""
- return TreeNode(row[mapper.c.name], _mapper_nohistory=True)
- def after_insert(self, mapper, instance):
+ def after_insert(self, mapper, connection, instance):
"""runs after the insert of a new TreeNode row. The primary key of the row is not determined
until the insert is complete, since most DB's use autoincrementing columns. If this node is
the root node, we will take the new primary key and update it as the value of the node's
"root ID" as well, since its root node is itself."""
if instance.root is instance:
- mapper.primarytable.update(TreeNode.c.id==instance.id, values=dict(root_node_id=instance.id)).execute()
+ connection.execute(mapper.mapped_table.update(TreeNode.c.id==instance.id, values=dict(root_node_id=instance.id)))
instance.root_id = instance.id
- def append_result(self, mapper, row, imap, result, instance, isnew, populate_existing=False):
+ def append_result(self, mapper, session, row, imap, result, instance, isnew, populate_existing=False):
"""runs as results from a SELECT statement are processed, and newly created or already-existing
instances that correspond to each row are appended to result lists. This method will only
append root nodes to the result list, and will attach child nodes to their appropriate parent
print "Creating Tree Table:"
print "----------------------------"
-treedata.create()
-trees.create()
+metadata.create_all()
-
-assign_mapper(TreeNode, trees, properties=dict(
+# the mapper is created with properties that specify "lazy=None" - this is because we are going
+# to handle our own "eager load" of nodes based on root id
+mapper(TreeNode, trees, properties=dict(
id=trees.c.node_id,
name=trees.c.node_name,
parent_id=trees.c.parent_node_id,
root_id=trees.c.root_node_id,
root=relation(TreeNode, primaryjoin=trees.c.root_node_id==trees.c.node_id, foreignkey=trees.c.node_id, lazy=None, uselist=False),
- children=relation(TreeNode, primaryjoin=trees.c.parent_node_id==trees.c.node_id, lazy=None, uselist=True, private=True),
- data=relation(mapper(TreeData, treedata, properties=dict(id=treedata.c.data_id)), private=True, lazy=False)
+ children=relation(TreeNode, primaryjoin=trees.c.parent_node_id==trees.c.node_id, lazy=None, uselist=True, cascade="delete,delete-orphan,save-update"),
+ data=relation(mapper(TreeData, treedata, properties=dict(id=treedata.c.data_id)), cascade="delete,delete-orphan,save-update", lazy=False)
), extension = TreeLoader())
-TreeNode.mapper
+session = create_session()
node2 = TreeNode('node2')
node2.append('subnode1')
print node.print_nodes()
print "\n\n\n----------------------------"
-print "Committing:"
+print "flushing:"
print "----------------------------"
-objectstore.commit()
+session.save(node)
+session.flush()
#sys.exit()
print "\n\n\n----------------------------"
print "Tree After Save:"
print "\n\n\n----------------------------"
print "Committing:"
print "----------------------------"
-objectstore.commit()
+session.flush()
print "\n\n\n----------------------------"
print "Tree After Save:"
print "tree new where root_id=%d:" % nodeid
print "----------------------------"
-objectstore.clear()
-t = TreeNode.mapper.select(TreeNode.c.root_id==nodeid, order_by=[TreeNode.c.id])[0]
+session.clear()
+
+# load some nodes. we do this based on "root id" which will load an entire sub-tree in one pass.
+# the MapperExtension will assemble the incoming nodes into a tree structure.
+t = session.query(TreeNode).select(TreeNode.c.root_id==nodeid, order_by=[TreeNode.c.id])[0]
print "\n\n\n----------------------------"
print "Full Tree:"
print "Marking root node as deleted"
print "and committing:"
print "----------------------------"
-objectstore.delete(t)
-objectstore.commit()
+session.delete(t)
+session.flush()
from sqlalchemy import *
-import sqlalchemy.attributes as attributes
-engine = create_engine('sqlite://', echo=True)
+metadata = BoundMetaData('sqlite:///', echo=True)
class Tree(object):
def __init__(self, name='', father=None):
def __repr__(self):
return self.__str__()
-table = Table('tree', engine,
+table = Table('tree', metadata,
Column('id', Integer, primary_key=True),
Column('name', String(64), nullable=False),
- Column('father_id', Integer, ForeignKey('tree.id'), nullable=True),)
+ Column('father_id', Integer, ForeignKey('tree.id'), nullable=True))
+table.create()
-assign_mapper(Tree, table,
+mapper(Tree, table,
properties={
- # set up a backref using a string
- #'father':relation(Tree, foreignkey=table.c.id,primaryjoin=table.c.father_id==table.c.id, backref='childs')},
-
- # or set up using the backref() function, which allows arguments to be passed
- 'childs':relation(Tree, foreignkey=table.c.father_id, primaryjoin=table.c.father_id==table.c.id, backref=backref('father', uselist=False, foreignkey=table.c.id))},
+ 'childs':relation(Tree, foreignkey=table.c.father_id, primaryjoin=table.c.father_id==table.c.id, backref=backref('father', uselist=False, foreignkey=table.c.id))},
)
-table.create()
root = Tree('root')
child1 = Tree('child1', root)
child2 = Tree('child2', root)
child3 = Tree('child3', child1)
-objectstore.commit()
+session = create_session()
+session.save(root)
+session.flush()
print root.childs
print child1.childs
--- /dev/null
+from sqlalchemy import *
+
+metadata = MetaData()
+
+managers_table = Table('managers', metadata,
+ Column('employee_id', Integer, primary_key=True),
+ Column('name', String(50)),
+ Column('manager_data', String(40))
+)
+
+engineers_table = Table('engineers', metadata,
+ Column('employee_id', Integer, primary_key=True),
+ Column('name', String(50)),
+ Column('engineer_info', String(40))
+)
+
+engine = create_engine('sqlite:///', echo=True)
+metadata.create_all(engine)
+
+
+class Employee(object):
+ def __init__(self, name):
+ self.name = name
+ def __repr__(self):
+ return self.__class__.__name__ + " " + self.name
+
+class Manager(Employee):
+ def __init__(self, name, manager_data):
+ self.name = name
+ self.manager_data = manager_data
+ def __repr__(self):
+ return self.__class__.__name__ + " " + self.name + " " + self.manager_data
+
+class Engineer(Employee):
+ def __init__(self, name, engineer_info):
+ self.name = name
+ self.engineer_info = engineer_info
+ def __repr__(self):
+ return self.__class__.__name__ + " " + self.name + " " + self.engineer_info
+
+
+pjoin = polymorphic_union({
+ 'manager':managers_table,
+ 'engineer':engineers_table
+}, 'type', 'pjoin')
+
+employee_mapper = mapper(Employee, pjoin, polymorphic_on=pjoin.c.type)
+manager_mapper = mapper(Manager, managers_table, inherits=employee_mapper, concrete=True, polymorphic_identity='manager')
+engineer_mapper = mapper(Engineer, engineers_table, inherits=employee_mapper, concrete=True, polymorphic_identity='engineer')
+
+
+session = create_session(bind_to=engine)
+
+m1 = Manager("pointy haired boss", "manager1")
+e1 = Engineer("wally", "engineer1")
+e2 = Engineer("dilbert", "engineer2")
+
+session.save(m1)
+session.save(e1)
+session.save(e2)
+session.flush()
+
+employees = session.query(Employee).select()
+print [e for e in employees]
+
from sqlalchemy import *
-import sys
+import sys, sets
-# this example illustrates how to create a relationship to a list of objects,
-# where each object in the list has a different type. The typed objects will
-# extend from a common base class, although this same approach can be used
-# with
+# this example illustrates a polymorphic load of two classes, where each class has a very
+# different set of properties
-db = create_engine('sqlite://', echo=True, echo_uow=False)
-#db = create_engine('postgres://user=scott&password=tiger&host=127.0.0.1&database=test', echo=True, echo_uow=False)
+metadata = BoundMetaData('sqlite://', echo='debug')
# a table to store companies
-companies = Table('companies', db,
+companies = Table('companies', metadata,
Column('company_id', Integer, primary_key=True),
- Column('name', String(50))).create()
+ Column('name', String(50)))
# we will define an inheritance relationship between the table "people" and "engineers",
# and a second inheritance relationship between the table "people" and "managers"
-people = Table('people', db,
+people = Table('people', metadata,
Column('person_id', Integer, primary_key=True),
Column('company_id', Integer, ForeignKey('companies.company_id')),
- Column('name', String(50))).create()
+ Column('name', String(50)),
+ Column('type', String(30)))
-engineers = Table('engineers', db,
+engineers = Table('engineers', metadata,
Column('person_id', Integer, ForeignKey('people.person_id'), primary_key=True),
- Column('special_description', String(50))).create()
+ Column('status', String(30)),
+ Column('engineer_name', String(50)),
+ Column('primary_language', String(50)),
+ )
-managers = Table('managers', db,
+managers = Table('managers', metadata,
Column('person_id', Integer, ForeignKey('people.person_id'), primary_key=True),
- Column('description', String(50))).create()
+ Column('status', String(30)),
+ Column('manager_name', String(50))
+ )
+
+metadata.create_all()
-
# create our classes. The Engineer and Manager classes extend from Person.
class Person(object):
+ def __init__(self, **kwargs):
+ for key, value in kwargs.iteritems():
+ setattr(self, key, value)
def __repr__(self):
return "Ordinary person %s" % self.name
class Engineer(Person):
def __repr__(self):
- return "Engineer %s, description %s" % (self.name, self.special_description)
+ return "Engineer %s, status %s, engineer_name %s, primary_language %s" % (self.name, self.status, self.engineer_name, self.primary_language)
class Manager(Person):
def __repr__(self):
- return "Manager %s, description %s" % (self.name, self.description)
+ return "Manager %s, status %s, manager_name %s" % (self.name, self.status, self.manager_name)
class Company(object):
+ def __init__(self, **kwargs):
+ for key, value in kwargs.iteritems():
+ setattr(self, key, value)
def __repr__(self):
return "Company %s" % self.name
-# next we assign Person mappers. Since these are the first mappers we are
-# creating for these classes, they automatically become the "primary mappers", which
-# define the dependency relationships between the classes, so we do a straight
-# inheritance setup, i.e. no modifications to how objects are loaded or anything like that.
-assign_mapper(Person, people)
-assign_mapper(Engineer, engineers, inherits=Person.mapper)
-assign_mapper(Manager, managers, inherits=Person.mapper)
-
-# next, we define a query that is going to load Managers and Engineers in one shot.
-# we will use a UNION ALL with an extra hardcoded column to indicate the type of object.
-# this can also be done via several LEFT OUTER JOINS but a UNION is more appropriate
-# since they are distinct result sets.
-# The select() statement is also given an alias 'pjoin', since the mapper requires
-# that all Selectables have a name.
-#
-# TECHNIQUE - when you want to load a certain set of objects from a in one query, all the
-# columns in the Selectable must have unique names. Dont worry about mappers at this point,
-# just worry about making a query where if you were to view the results, you could tell everything
-# you need to know from each row how to construct an object instance from it. this is the
-# essence of "resultset-based-mapping", which is the core ideology of SQLAlchemy.
-#
-person_join = select(
- [people, managers.c.description,column("'manager'").label('type')],
- people.c.person_id==managers.c.person_id).union_all(
- select(
- [people, engineers.c.special_description.label('description'), column("'engineer'").label('type')],
- people.c.person_id==engineers.c.person_id)).alias('pjoin')
-
-
-# lets print out what this Selectable looks like. The mapper is going to take the selectable and
-# Select off of it, with the flag "use_labels" which indicates to prefix column names with the table
-# name. So here is what our mapper will see:
-print "Person selectable:", str(person_join.select(use_labels=True)), "\n"
-
-
-# MapperExtension object.
-class PersonLoader(MapperExtension):
- def create_instance(self, mapper, row, imap, class_):
- if row['pjoin_type'] =='engineer':
- e = Engineer()
- e.special_description = row['pjoin_description']
- return e
- elif row['pjoin_type'] =='manager':
- return Manager()
- else:
- return Person()
-ext = PersonLoader()
-
-# set up the polymorphic mapper, which maps the person_join we set up to
-# the Person class, using an instance of PersonLoader.
-people_mapper = mapper(Person, person_join, extension=ext)
-
-# create a mapper for Company. the 'employees' relationship points to
-# our new people_mapper.
-#
-# the dependency relationships which take effect on commit (i.e. the order of
-# inserts/deletes) will be established against the Person class's primary
-# mapper, and when the Engineer and
-# Manager objects are found in the 'employees' list, the primary mappers
-# for those subclasses will register
-# themselves as dependent on the Person mapper's save operations.
-# (translation: it'll work)
-# TODO: get the eager loading to work (the compound select alias doesnt like being aliased itself)
-assign_mapper(Company, companies, properties={
- 'employees': relation(people_mapper, lazy=False, private=True)
-})
-c = Company(name='company1')
-c.employees.append(Manager(name='pointy haired boss', description='manager1'))
-c.employees.append(Engineer(name='dilbert', special_description='engineer1'))
-c.employees.append(Engineer(name='wally', special_description='engineer2'))
-c.employees.append(Manager(name='jsmith', description='manager2'))
-objectstore.commit()
+# create a union that represents both types of joins.
+person_join = polymorphic_union(
+ {
+ 'engineer':people.join(engineers),
+ 'manager':people.join(managers),
+ 'person':people.select(people.c.type=='person'),
+ }, None, 'pjoin')
-objectstore.clear()
+#person_mapper = mapper(Person, people, select_table=person_join, polymorphic_on=person_join.c.type, polymorphic_identity='person')
+person_mapper = mapper(Person, people, select_table=person_join,polymorphic_on=person_join.c.type, polymorphic_identity='person')
+mapper(Engineer, engineers, inherits=person_mapper, polymorphic_identity='engineer')
+mapper(Manager, managers, inherits=person_mapper, polymorphic_identity='manager')
-c = Company.get(1)
-for e in c.employees:
- print e, e._instance_key
+mapper(Company, companies, properties={
+ 'employees': relation(Person, lazy=False, private=True, backref='company')
+})
+session = create_session(echo_uow=False)
+c = Company(name='company1')
+c.employees.append(Manager(name='pointy haired boss', status='AAB', manager_name='manager1'))
+c.employees.append(Engineer(name='dilbert', status='BBA', engineer_name='engineer1', primary_language='java'))
+c.employees.append(Person(name='joesmith', status='HHH'))
+c.employees.append(Engineer(name='wally', status='CGG', engineer_name='engineer2', primary_language='python'))
+c.employees.append(Manager(name='jsmith', status='ABA', manager_name='manager2'))
+session.save(c)
+print session.new
+session.flush()
+#sys.exit()
+session.clear()
+
+c = session.query(Company).get(1)
+for e in c.employees:
+ print e, e._instance_key, e.company
+assert sets.Set([e.name for e in c.employees]) == sets.Set(['pointy haired boss', 'dilbert', 'joesmith', 'wally', 'jsmith'])
print "\n"
-dilbert = Engineer.mapper.get_by(name='dilbert')
-dilbert.special_description = 'hes dibert!'
-objectstore.commit()
+dilbert = session.query(Person).get_by(name='dilbert')
+dilbert2 = session.query(Engineer).get_by(name='dilbert')
+assert dilbert is dilbert2
+
+dilbert.engineer_name = 'hes dibert!'
-objectstore.clear()
-c = Company.get(1)
+session.flush()
+session.clear()
+
+c = session.query(Company).get(1)
for e in c.employees:
print e, e._instance_key
-objectstore.delete(c)
-objectstore.commit()
-
+session.delete(c)
+session.flush()
-managers.drop()
-engineers.drop()
-people.drop()
-companies.drop()
+metadata.drop_all()
+++ /dev/null
-from sqlalchemy import *
-import sys
-
-# this example illustrates a polymorphic load of two classes, where each class has a very
-# different set of properties
-
-db = create_engine('sqlite://', echo=True, echo_uow=False)
-
-# a table to store companies
-companies = Table('companies', db,
- Column('company_id', Integer, primary_key=True),
- Column('name', String(50))).create()
-
-# we will define an inheritance relationship between the table "people" and "engineers",
-# and a second inheritance relationship between the table "people" and "managers"
-people = Table('people', db,
- Column('person_id', Integer, primary_key=True),
- Column('company_id', Integer, ForeignKey('companies.company_id')),
- Column('name', String(50))).create()
-
-engineers = Table('engineers', db,
- Column('person_id', Integer, ForeignKey('people.person_id'), primary_key=True),
- Column('status', String(30)),
- Column('engineer_name', String(50)),
- Column('primary_language', String(50)),
- ).create()
-
-managers = Table('managers', db,
- Column('person_id', Integer, ForeignKey('people.person_id'), primary_key=True),
- Column('status', String(30)),
- Column('manager_name', String(50))
- ).create()
-
-
-# create our classes. The Engineer and Manager classes extend from Person.
-class Person(object):
- def __repr__(self):
- return "Ordinary person %s" % self.name
-class Engineer(Person):
- def __repr__(self):
- return "Engineer %s, status %s, engineer_name %s, primary_language %s" % (self.name, self.status, self.engineer_name, self.primary_language)
-class Manager(Person):
- def __repr__(self):
- return "Manager %s, status %s, manager_name %s" % (self.name, self.status, self.manager_name)
-class Company(object):
- def __repr__(self):
- return "Company %s" % self.name
-
-# assign plain vanilla mappers
-assign_mapper(Person, people)
-assign_mapper(Engineer, engineers, inherits=Person.mapper)
-assign_mapper(Manager, managers, inherits=Person.mapper)
-
-# create a union that represents both types of joins. we have to use
-# nulls to pad out the disparate columns.
-person_join = select(
- [
- people,
- managers.c.status,
- managers.c.manager_name,
- null().label('engineer_name'),
- null().label('primary_language'),
- column("'manager'").label('type')
- ],
- people.c.person_id==managers.c.person_id).union_all(
- select(
- [
- people,
- engineers.c.status,
- null().label('').label('manager_name'),
- engineers.c.engineer_name,
- engineers.c.primary_language,
- column("'engineer'").label('type')
- ],
- people.c.person_id==engineers.c.person_id)).alias('pjoin')
-
-print [c for c in person_join.c]
-
-# MapperExtension object.
-class PersonLoader(MapperExtension):
- def create_instance(self, mapper, row, imap, class_):
- if row[person_join.c.type] =='engineer':
- return Engineer()
- elif row[person_join.c.type] =='manager':
- return Manager()
- else:
- return Person()
-
- def populate_instance(self, mapper, session, instance, row, identitykey, imap, isnew):
- if row[person_join.c.type] =='engineer':
- Engineer.mapper.populate_instance(session, instance, row, identitykey, imap, isnew, frommapper=mapper)
- return False
- elif row[person_join.c.type] =='manager':
- Manager.mapper.populate_instance(session, instance, row, identitykey, imap, isnew, frommapper=mapper)
- return False
- else:
- return sqlalchemy.mapping.EXT_PASS
-
-people_mapper = mapper(Person, person_join, extension=PersonLoader())
-
-assign_mapper(Company, companies, properties={
- 'employees': relation(people_mapper, lazy=False, private=True)
-})
-
-c = Company(name='company1')
-c.employees.append(Manager(name='pointy haired boss', status='AAB', manager_name='manager1'))
-c.employees.append(Engineer(name='dilbert', status='BBA', engineer_name='engineer1', primary_language='java'))
-c.employees.append(Engineer(name='wally', status='CGG', engineer_name='engineer2', primary_language='python'))
-c.employees.append(Manager(name='jsmith', status='ABA', manager_name='manager2'))
-objectstore.commit()
-
-objectstore.clear()
-
-c = Company.get(1)
-for e in c.employees:
- print e, e._instance_key
-
-print "\n"
-
-dilbert = Engineer.mapper.get_by(name='dilbert')
-dilbert.engineer_name = 'hes dibert!'
-objectstore.commit()
-
-objectstore.clear()
-c = Company.get(1)
-for e in c.employees:
- print e, e._instance_key
-
-objectstore.delete(c)
-objectstore.commit()
-
-
-managers.drop()
-engineers.drop()
-people.drop()
-companies.drop()
--- /dev/null
+from sqlalchemy import *
+
+metadata = BoundMetaData('sqlite://', echo='debug')
+
+# a table to store companies
+companies = Table('companies', metadata,
+ Column('company_id', Integer, primary_key=True),
+ Column('name', String(50)))
+
+employees_table = Table('employees', metadata,
+ Column('employee_id', Integer, primary_key=True),
+ Column('company_id', Integer, ForeignKey('companies.company_id')),
+ Column('name', String(50)),
+ Column('type', String(20)),
+ Column('status', String(20)),
+ Column('engineer_name', String(50)),
+ Column('primary_language', String(50)),
+ Column('manager_name', String(50))
+)
+
+metadata.create_all()
+
+class Person(object):
+ def __init__(self, **kwargs):
+ for key, value in kwargs.iteritems():
+ setattr(self, key, value)
+ def __repr__(self):
+ return "Ordinary person %s" % self.name
+class Engineer(Person):
+ def __repr__(self):
+ return "Engineer %s, status %s, engineer_name %s, primary_language %s" % (self.name, self.status, self.engineer_name, self.primary_language)
+class Manager(Person):
+ def __repr__(self):
+ return "Manager %s, status %s, manager_name %s" % (self.name, self.status, self.manager_name)
+class Company(object):
+ def __init__(self, **kwargs):
+ for key, value in kwargs.iteritems():
+ setattr(self, key, value)
+ def __repr__(self):
+ return "Company %s" % self.name
+
+person_mapper = mapper(Person, employees_table, polymorphic_on=employees_table.c.type, polymorphic_identity='person')
+manager_mapper = mapper(Manager, inherits=person_mapper, polymorphic_identity='manager')
+engineer_mapper = mapper(Engineer, inherits=person_mapper, polymorphic_identity='engineer')
+
+
+
+mapper(Company, companies, properties={
+ 'employees': relation(Person, lazy=True, private=True, backref='company')
+})
+
+session = create_session()
+c = Company(name='company1')
+c.employees.append(Manager(name='pointy haired boss', status='AAB', manager_name='manager1'))
+c.employees.append(Engineer(name='dilbert', status='BBA', engineer_name='engineer1', primary_language='java'))
+c.employees.append(Person(name='joesmith', status='HHH'))
+c.employees.append(Engineer(name='wally', status='CGG', engineer_name='engineer2', primary_language='python'))
+c.employees.append(Manager(name='jsmith', status='ABA', manager_name='manager2'))
+session.save(c)
+session.flush()
+
+session.clear()
+
+c = session.query(Company).get(1)
+for e in c.employees:
+ print e, e._instance_key, e.company
+
+print "\n"
+
+dilbert = session.query(Person).get_by(name='dilbert')
+dilbert2 = session.query(Engineer).get_by(name='dilbert')
+assert dilbert is dilbert2
+
+dilbert.engineer_name = 'hes dibert!'
+
+session.flush()
+session.clear()
+
+c = session.query(Company).get(1)
+for e in c.employees:
+ print e, e._instance_key
+
+session.delete(c)
+session.flush()
+
+metadata.drop_all()
represented in distinct database rows. This allows objects to be created with dynamically changing
fields that are all persisted in a normalized fashion."""
-e = create_engine('sqlite://', echo=True)
+e = BoundMetaData('sqlite://', echo=True)
# this table represents Entity objects. each Entity gets a row in this table,
# with a primary key and a title.
object's _entities dictionary for the appropriate value, and the __setattribute__
method is overridden to set all non "_" attributes as EntityValues within the
_entities dictionary. """
- def __init__(self):
- """the constructor sets the "_entities" member to an EntityDict. A mapper
- will wrap this property with its own history-list object."""
- self._entities = EntityDict()
+
+ # establish the type of '_entities'
+ _entities = EntityDict
+
def __getattr__(self, key):
"""getattr proxies requests for attributes which dont 'exist' on the object
to the underying _entities dictionary."""
+ if key[0] == '_':
+ return super(Entity, self).__getattr__(key)
try:
return self._entities[key].value
except KeyError:
the value to the underlying datatype of its EntityField."""
def __init__(self, key=None, value=None):
if key is not None:
- self.field = class_mapper(EntityField).get_by(name=key) or EntityField(key)
+ sess = create_session()
+ self.field = sess.query(EntityField).get_by(name=key) or EntityField(key)
+ # close the session, which will make a loaded EntityField a detached instance
+ sess.close()
if self.field.datatype is None:
if isinstance(value, int):
self.field.datatype = 'int'
mapper(
EntityValue, entity_values,
properties = {
- 'field' : relation(EntityField, lazy=False)
+ 'field' : relation(EntityField, lazy=False, cascade='all')
}
)
-entitymapper = mapper(Entity, entities, properties = {
- '_entities' : relation(EntityValue, lazy=False)
+mapper(Entity, entities, properties = {
+ '_entities' : relation(EntityValue, lazy=False, cascade='save-update')
})
# create two entities. the objects can be used about as regularly as
# any object can.
+session = create_session()
entity = Entity()
entity.title = 'this is the first entity'
entity.name = 'this is the name'
entity2.data = ('hoo', 'ha')
# commit
-objectstore.commit()
+[session.save(x) for x in (entity, entity2)]
+session.flush()
# we would like to illustate loading everything totally clean from
-# the database, so we clear out the objectstore.
-objectstore.clear()
+# the database, so we clear out the session
+session.clear()
# select both objects and print
-entities = entitymapper.select()
+entities = session.query(Entity).select()
for entity in entities:
print entity.title, entity.name, entity.price, entity.data
entities[0].price=90
entities[0].title = 'another new title'
entities[1].data = {'oof':5,'lala':8}
+entity3 = Entity()
+entity3.title = 'third entity'
+entity3.name = 'new name'
+entity3.price = '$1.95'
+entity3.data = 'some data'
+session.save(entity3)
# commit changes. the correct rows are updated, nothing else.
-objectstore.commit()
+session.flush()
-# lets see if that one came through. clear object store, re-select
+# lets see if that one came through. clear the session, re-select
# and print
-objectstore.clear()
-entities = entitymapper.select()
+session.clear()
+entities = session.query(Entity).select()
for entity in entities:
print entity.title, entity.name, entity.price, entity.data
# This module is part of SQLAlchemy and is released under
# the MIT License: http://www.opensource.org/licenses/mit-license.php
-from engine import *
from types import *
from sql import *
from schema import *
-from exceptions import *
-import sqlalchemy.sql
-import sqlalchemy.mapping as mapping
-from sqlalchemy.mapping import *
-import sqlalchemy.schema
-import sqlalchemy.ext.proxy
-sqlalchemy.schema.default_engine = sqlalchemy.ext.proxy.ProxyEngine()
+from sqlalchemy.orm import *
-from sqlalchemy.mods import install_mods
+from sqlalchemy.engine import create_engine
+from sqlalchemy.schema import default_metadata
def global_connect(*args, **kwargs):
- sqlalchemy.schema.default_engine.connect(*args, **kwargs)
+ default_metadata.connect(*args, **kwargs)
\ No newline at end of file
# This module is part of SQLAlchemy and is released under
# the MIT License: http://www.opensource.org/licenses/mit-license.php
-"""defines ANSI SQL operations."""
+"""defines ANSI SQL operations. Contains default implementations for the abstract objects
+in the sql module."""
-import sqlalchemy.schema as schema
-
-from sqlalchemy.schema import *
-import sqlalchemy.sql as sql
-import sqlalchemy.engine
-from sqlalchemy.sql import *
-from sqlalchemy.util import *
+from sqlalchemy import schema, sql, engine, util
+import sqlalchemy.engine.default as default
import string, re
-ANSI_FUNCS = HashSet([
+ANSI_FUNCS = util.HashSet([
'CURRENT_TIME',
'CURRENT_TIMESTAMP',
'CURRENT_DATE',
])
-def engine(**params):
- return ANSISQLEngine(**params)
-
-class ANSISQLEngine(sqlalchemy.engine.SQLEngine):
-
- def schemagenerator(self, **params):
- return ANSISchemaGenerator(self, **params)
-
- def schemadropper(self, **params):
- return ANSISchemaDropper(self, **params)
-
- def compiler(self, statement, parameters, **kwargs):
- return ANSICompiler(statement, parameters, engine=self, **kwargs)
+def create_engine():
+ return engine.ComposedSQLEngine(None, ANSIDialect())
+class ANSIDialect(default.DefaultDialect):
def connect_args(self):
return ([],{})
def dbapi(self):
return None
+ def schemagenerator(self, *args, **params):
+ return ANSISchemaGenerator(*args, **params)
+
+ def schemadropper(self, *args, **params):
+ return ANSISchemaDropper(*args, **params)
+
+ def compiler(self, statement, parameters, **kwargs):
+ return ANSICompiler(self, statement, parameters, **kwargs)
+
+
class ANSICompiler(sql.Compiled):
"""default implementation of Compiled, which compiles ClauseElements into ANSI-compliant SQL strings."""
- def __init__(self, statement, parameters=None, typemap=None, engine=None, positional=None, paramstyle=None, **kwargs):
+ def __init__(self, dialect, statement, parameters=None, **kwargs):
"""constructs a new ANSICompiler object.
- engine - SQLEngine to compile against
+ dialect - Dialect to be used
statement - ClauseElement to be compiled
key/value pairs when the Compiled is executed, and also may affect the
actual compilation, as in the case of an INSERT where the actual columns
inserted will correspond to the keys present in the parameters."""
- sql.Compiled.__init__(self, statement, parameters, engine=engine)
+ sql.Compiled.__init__(self, dialect, statement, parameters, **kwargs)
self.binds = {}
self.froms = {}
self.wheres = {}
self.strings = {}
self.select_stack = []
- self.typemap = typemap or {}
+ self.typemap = {}
self.isinsert = False
self.isupdate = False
self.bindtemplate = ":%s"
- if engine is not None:
- self.paramstyle = engine.paramstyle
- self.positional = engine.positional
- else:
- self.positional = False
- self.paramstyle = 'named'
+ self.paramstyle = dialect.paramstyle
+ self.positional = dialect.positional
def after_compile(self):
# this re will search for params like :param
bindparams = {}
bindparams.update(params)
- d = sql.ClauseParameters(self.engine)
+ d = sql.ClauseParameters(self.dialect)
if self.positional:
for k in self.positiontup:
b = self.binds[k]
# if we are within a visit to a Select, set up the "typemap"
# for this column which is used to translate result set values
self.typemap.setdefault(column.key.lower(), column.type)
- if column.table is None or column.table.name is None:
+ if column.table is None or not column.table.named_with_column():
self.strings[column] = column.name
else:
- self.strings[column] = "%s.%s" % (column.table.name, column.name)
+ if column.table.oid_column is column:
+ n = self.dialect.oid_column_name()
+ if n is not None:
+ self.strings[column] = "%s.%s" % (column.table.name, n)
+ elif len(column.table.primary_key) != 0:
+ self.strings[column] = "%s.%s" % (column.table.name, column.table.primary_key[0].name)
+ else:
+ self.strings[column] = None
+ else:
+ self.strings[column] = "%s.%s" % (column.table.name, column.name)
def visit_fromclause(self, fromclause):
self.strings[index] = index.name
def visit_typeclause(self, typeclause):
- self.strings[typeclause] = typeclause.type.engine_impl(self.engine).get_col_spec()
+ self.strings[typeclause] = typeclause.type.dialect_impl(self.dialect).get_col_spec()
def visit_textclause(self, textclause):
if textclause.parens and len(textclause.text):
def visit_clauselist(self, list):
if list.parens:
- self.strings[list] = "(" + string.join([self.get_str(c) for c in list.clauses], ', ') + ")"
+ self.strings[list] = "(" + string.join([s for s in [self.get_str(c) for c in list.clauses] if s is not None], ', ') + ")"
else:
- self.strings[list] = string.join([self.get_str(c) for c in list.clauses], ', ')
+ self.strings[list] = string.join([s for s in [self.get_str(c) for c in list.clauses] if s is not None], ', ')
def apply_function_parens(self, func):
return func.name.upper() not in ANSI_FUNCS or len(func.clauses) > 0
# the actual list of columns to print in the SELECT column list.
# its an ordered dictionary to insure that the actual labeled column name
# is unique.
- inner_columns = OrderedDict()
+ inner_columns = util.OrderedDict()
self.select_stack.append(select)
for c in select._raw_columns:
# SQLite doesnt like selecting from a subquery where the column
# names look like table.colname, so add a label synonomous with
# the column name
- l = co.label(co.text)
+ l = co.label(co.name)
l.accept_visitor(self)
inner_columns[self.get_str(l.obj)] = l
else:
order_by = self.get_str(select.order_by_clause)
if order_by:
text += " ORDER BY " + order_by
-
+
text += self.visit_select_postclauses(select)
if select.for_update:
# case one: no parameters in the statement, no parameters in the
# compiled params - just return binds for all the table columns
if self.parameters is None and stmt.parameters is None:
- return [(c, bindparam(c.name, type=c.type)) for c in stmt.table.columns]
+ return [(c, sql.bindparam(c.name, type=c.type)) for c in stmt.table.columns]
# if we have statement parameters - set defaults in the
# compiled params
if d.has_key(c):
value = d[c]
if sql._is_literal(value):
- value = bindparam(c.name, value, type=c.type)
+ value = sql.bindparam(c.name, value, type=c.type)
values.append((c, value))
return values
return self.get_str(self.statement)
-class ANSISchemaGenerator(sqlalchemy.engine.SchemaIterator):
+class ANSISchemaGenerator(engine.SchemaIterator):
def get_column_specification(self, column, override_pk=False, first_pk=False):
raise NotImplementedError()
if isinstance(column.default.arg, str):
return repr(column.default.arg)
else:
- return str(column.default.arg.compile(self.engine))
+ return str(self._compile(column.default.arg, None))
else:
return None
+ def _compile(self, tocompile, parameters):
+ compiler = self.engine.dialect.compiler(tocompile, parameters)
+ compiler.compile()
+ return compiler
+
def visit_column(self, column):
pass
self.execute()
-class ANSISchemaDropper(sqlalchemy.engine.SchemaIterator):
+class ANSISchemaDropper(engine.SchemaIterator):
def visit_index(self, index):
self.append("\nDROP INDEX " + index.name)
self.execute()
self.execute()
-class ANSIDefaultRunner(sqlalchemy.engine.DefaultRunner):
+class ANSIDefaultRunner(engine.DefaultRunner):
pass
create_prop method on AttributeManger, which can be overridden to provide
subclasses of SmartProperty.
"""
- def __init__(self, manager, key, uselist, callable_, **kwargs):
+ def __init__(self, manager, key, uselist, callable_, typecallable, **kwargs):
self.manager = manager
self.key = key
self.uselist = uselist
self.callable_ = callable_
+ self.typecallable= typecallable
self.kwargs = kwargs
def init(self, obj, attrhist=None):
"""creates an appropriate ManagedAttribute for the given object and establishes
func = self.callable_(obj)
else:
func = None
- return self.manager.create_managed_attribute(obj, self.key, self.uselist, callable_=func, attrdict=attrhist, **self.kwargs)
+ return self.manager.create_managed_attribute(obj, self.key, self.uselist, callable_=func, attrdict=attrhist, typecallable=self.typecallable, **self.kwargs)
def __set__(self, obj, value):
self.manager.set_attribute(obj, self.key, value)
def __delete__(self, obj):
self.key = d['key']
self.__obj = weakref.ref(d['obj'])
obj = property(lambda s:s.__obj())
+ def value_changed(self, *args, **kwargs):
+ self.obj._managed_value_changed = True
+ self.do_value_changed(*args, **kwargs)
def history(self, **kwargs):
return self
def plain_init(self, *args, **kwargs):
pass
-
+ def hasparent(self, item):
+ return item.__class__._attribute_manager.attribute_history(item).get('_hasparent_' + self.key)
+ def sethasparent(self, item, value):
+ if item is not None:
+ item.__class__._attribute_manager.attribute_history(item)['_hasparent_' + self.key] = value
+
class ScalarAttribute(ManagedAttribute):
"""Used by AttributeManager to track the history of a scalar attribute
on an object instance. This is the "scalar history container" object.
so that the two objects can be called upon largely interchangeably."""
# make our own NONE to distinguish from "None"
NONE = object()
- def __init__(self, obj, key, extension=None, **kwargs):
+ def __init__(self, obj, key, extension=None, trackparent=False, **kwargs):
ManagedAttribute.__init__(self, obj, key)
self.orig = ScalarAttribute.NONE
self.extension = extension
+ self.trackparent = trackparent
def clear(self):
del self.obj.__dict__[self.key]
def history_contains(self, obj):
if self.orig is ScalarAttribute.NONE:
self.orig = orig
self.obj.__dict__[self.key] = value
+ if self.trackparent:
+ if value is not None:
+ self.sethasparent(value, True)
+ if orig is not None:
+ self.sethasparent(orig, False)
if self.extension is not None:
self.extension.set(self.obj, value, orig)
+ self.value_changed(orig, value)
def delattr(self, **kwargs):
orig = self.obj.__dict__.get(self.key, None)
if self.orig is ScalarAttribute.NONE:
self.orig = orig
self.obj.__dict__[self.key] = None
+ if self.trackparent:
+ self.sethasparent(orig, False)
if self.extension is not None:
self.extension.set(self.obj, None, orig)
+ self.value_changed(orig, None)
def append(self, obj):
self.setattr(obj)
def remove(self, obj):
self.orig = ScalarAttribute.NONE
def commit(self):
self.orig = ScalarAttribute.NONE
+ def do_value_changed(self, oldvalue, newvalue):
+ pass
def added_items(self):
if self.orig is not ScalarAttribute.NONE:
- return [self.obj.__dict__[self.key]]
+ return [self.obj.__dict__.get(self.key)]
else:
return []
def deleted_items(self):
return []
def unchanged_items(self):
if self.orig is ScalarAttribute.NONE:
- return [self.obj.__dict__[self.key]]
+ return [self.obj.__dict__.get(self.key)]
else:
return []
This is the "list history container" object.
Subclasses util.HistoryArraySet to provide "onchange" event handling as well
as a plugin point for BackrefExtension objects."""
- def __init__(self, obj, key, data=None, extension=None, **kwargs):
+ def __init__(self, obj, key, data=None, extension=None, trackparent=False, typecallable=None, **kwargs):
ManagedAttribute.__init__(self, obj, key)
self.extension = extension
+ self.trackparent = trackparent
# if we are given a list, try to behave nicely with an existing
# list that might be set on the object already
try:
except KeyError:
if data is not None:
list_ = data
+ elif typecallable is not None:
+ list_ = typecallable()
else:
list_ = []
- obj.__dict__[key] = []
-
+ obj.__dict__[key] = list_
util.HistoryArraySet.__init__(self, list_, readonly=kwargs.get('readonly', False))
- def list_value_changed(self, obj, key, item, listval, isdelete):
+ def do_value_changed(self, obj, key, item, listval, isdelete):
pass
def setattr(self, value, **kwargs):
self.obj.__dict__[self.key] = value
self.set_data(value)
def delattr(self, value, **kwargs):
pass
- def _setrecord(self, item):
- res = util.HistoryArraySet._setrecord(self, item)
- if res:
- self.list_value_changed(self.obj, self.key, item, self, False)
- if self.extension is not None:
- self.extension.append(self.obj, item)
- return res
- def _delrecord(self, item):
- res = util.HistoryArraySet._delrecord(self, item)
- if res:
- self.list_value_changed(self.obj, self.key, item, self, True)
- if self.extension is not None:
- self.extension.delete(self.obj, item)
- return res
+ def do_value_appended(self, item):
+ if self.trackparent:
+ self.sethasparent(item, True)
+ self.value_changed(self.obj, self.key, item, self, False)
+ if self.extension is not None:
+ self.extension.append(self.obj, item)
+ def do_value_deleted(self, item):
+ if self.trackparent:
+ self.sethasparent(item, False)
+ self.value_changed(self.obj, self.key, item, self, True)
+ if self.extension is not None:
+ self.extension.delete(self.obj, item)
-# deprecated
-class ListElement(ListAttribute):pass
-
class TriggeredAttribute(ManagedAttribute):
"""Used by AttributeManager to allow the attaching of a callable item, representing the future value
of a particular attribute on a particular object instance, as the current attribute on an object.
def plain_init(self, attrhist):
if not self.uselist:
- p = ScalarAttribute(self.obj, self.key, **self.kwargs)
+ p = self.manager.create_scalar(self.obj, self.key, **self.kwargs)
self.obj.__dict__[self.key] = None
else:
p = self.manager.create_list(self.obj, self.key, None, **self.kwargs)
raise AssertionError("AttributeError caught in callable prop:" + str(e.args))
self.obj.__dict__[self.key] = value
- p = ScalarAttribute(self.obj, self.key, **self.kwargs)
+ p = self.manager.create_scalar(self.obj, self.key, **self.kwargs)
else:
if not self.obj.__dict__.has_key(self.key) or len(self.obj.__dict__[self.key]) == 0:
if passive:
def __init__(self):
pass
- def value_changed(self, obj, key, value):
+ def do_value_changed(self, obj, key, value):
"""subclasses override this method to provide functionality that is triggered
upon an attribute change of value."""
pass
- def create_prop(self, class_, key, uselist, callable_, **kwargs):
+ def create_prop(self, class_, key, uselist, callable_, typecallable, **kwargs):
"""creates a scalar property object, defaulting to SmartProperty, which
will communicate change events back to this AttributeManager."""
- return SmartProperty(self, key, uselist, callable_, **kwargs)
-
- def create_list(self, obj, key, list_, **kwargs):
+ return SmartProperty(self, key, uselist, callable_, typecallable, **kwargs)
+ def create_scalar(self, obj, key, **kwargs):
+ return ScalarAttribute(obj, key, **kwargs)
+ def create_list(self, obj, key, list_, typecallable=None, **kwargs):
"""creates a history-aware list property, defaulting to a ListAttribute which
is a subclass of HistoryArrayList."""
- return ListAttribute(obj, key, list_, **kwargs)
+ return ListAttribute(obj, key, list_, typecallable=typecallable, **kwargs)
def create_callable(self, obj, key, func, uselist, **kwargs):
"""creates a callable container that will invoke a function the first
time an object property is accessed. The return value of the function
def set_attribute(self, obj, key, value, **kwargs):
"""sets the value of an object's attribute."""
self.get_unexec_history(obj, key).setattr(value, **kwargs)
- self.value_changed(obj, key, value)
def delete_attribute(self, obj, key, **kwargs):
"""deletes the value from an object's attribute."""
self.get_unexec_history(obj, key).delattr(**kwargs)
- self.value_changed(obj, key, None)
def rollback(self, *obj):
"""rolls back all attribute changes on the given list of objects,
try:
attributes = self.attribute_history(o)
for hist in attributes.values():
- hist.rollback()
+ if isinstance(hist, ManagedAttribute):
+ hist.rollback()
except KeyError:
pass
+ o._managed_value_changed = False
def commit(self, *obj):
"""commits all attribute changes on the given list of objects,
try:
attributes = self.attribute_history(o)
for hist in attributes.values():
- hist.commit()
+ if isinstance(hist, ManagedAttribute):
+ hist.commit()
except KeyError:
pass
-
+ o._managed_value_changed = False
+
+ def is_modified(self, object):
+ return getattr(object, '_managed_value_changed', False)
+
def remove(self, obj):
"""called when an object is totally being removed from memory"""
# currently a no-op since the state of the object is attached to the object itself
def is_class_managed(self, class_, key):
return hasattr(class_, key) and isinstance(getattr(class_, key), SmartProperty)
- def create_managed_attribute(self, obj, key, uselist, callable_=None, attrdict=None, **kwargs):
+ def create_managed_attribute(self, obj, key, uselist, callable_=None, attrdict=None, typecallable=None, **kwargs):
"""creates a new ManagedAttribute corresponding to the given attribute key on the
given object instance, and installs it in the attribute dictionary attached to the object."""
if callable_ is not None:
- prop = self.create_callable(obj, key, callable_, uselist=uselist, **kwargs)
+ prop = self.create_callable(obj, key, callable_, uselist=uselist, typecallable=typecallable, **kwargs)
elif not uselist:
- prop = ScalarAttribute(obj, key, **kwargs)
+ prop = self.create_scalar(obj, key, **kwargs)
else:
- prop = self.create_list(obj, key, None, **kwargs)
+ prop = self.create_list(obj, key, None, typecallable=typecallable, **kwargs)
if attrdict is None:
attrdict = self.attribute_history(obj)
attrdict[key] = prop
will be passed along to newly created ManagedAttribute."""
if not hasattr(class_, '_attribute_manager'):
class_._attribute_manager = self
- setattr(class_, key, self.create_prop(class_, key, uselist, callable_, **kwargs))
+ typecallable = getattr(class_, key, None)
+ # TODO: look at existing properties on the class, and adapt them to the SmartProperty
+ if isinstance(typecallable, SmartProperty):
+ typecallable = None
+ setattr(class_, key, self.create_prop(class_, key, uselist, callable_, typecallable=typecallable, **kwargs))
+
# the MIT License: http://www.opensource.org/licenses/mit-license.php
-import sys, StringIO, string, datetime
+import sys, StringIO, string
import sqlalchemy.sql as sql
import sqlalchemy.schema as schema
class FBDateTime(sqltypes.DateTime):
def get_col_spec(self):
return "DATE"
- def convert_bind_param(self, value, engine):
- if value is not None:
- if isinstance(value, datetime.datetime):
- seconds = float(str(value.second) + "."
- + str(value.microsecond))
- return kinterbasdb.date_conv_out((value.year, value.month, value.day,
- value.hour, value.minute, seconds))
- return kinterbasdb.timestamp_conv_in(value)
- else:
- return None
class FBText(sqltypes.TEXT):
def get_col_spec(self):
return "BLOB SUB_TYPE 2"
]}
class FBSQLEngine(ansisql.ANSISQLEngine):
- def __init__(self, opts, module=None, use_oids=False, **params):
+ def __init__(self, opts, use_ansi = True, module = None, **params):
+ self._use_ansi = use_ansi
+ self.opts = opts or {}
if module is None:
self.module = kinterbasdb
else:
self.module = module
- self.opts = self._translate_connect_args(('host', 'database', 'user', 'password'), opts)
ansisql.ANSISQLEngine.__init__(self, **params)
def do_commit(self, connection):
return self.context.last_inserted_ids
def compiler(self, statement, bindparams, **kwargs):
- return FBCompiler(statement, bindparams, engine=self, **kwargs)
+ return FBCompiler(statement, bindparams, engine=self, use_ansi=self._use_ansi, **kwargs)
def schemagenerator(self, **params):
return FBSchemaGenerator(self, **params)
class FBCompiler(ansisql.ANSICompiler):
"""firebird compiler modifies the lexical structure of Select statements to work under
non-ANSI configured Firebird databases, if the use_ansi flag is False."""
+
+ def __init__(self, engine, statement, parameters, use_ansi = True, **kwargs):
+ self._outertable = None
+ self._use_ansi = use_ansi
+ ansisql.ANSICompiler.__init__(self, engine, statement, parameters, **kwargs)
+
+ def visit_column(self, column):
+ if self._use_ansi:
+ return ansisql.ANSICompiler.visit_column(self, column)
+
+ if column.table is self._outertable:
+ self.strings[column] = "%s.%s(+)" % (column.table.name, column.name)
+ else:
+ self.strings[column] = "%s.%s" % (column.table.name, column.name)
+
def visit_function(self, func):
if len(func.clauses):
super(FBCompiler, self).visit_function(func)
""" called when building a SELECT statment, position is just before column list
Firebird puts the limit and offset right after the select...thanks for adding the
visit_select_precolumns!!!"""
- result = ''
if select.offset:
- result +=" FIRST %s " % select.offset
+ result +=" FIRST " + select.offset
if select.limit:
- result += " SKIP %s " % select.limit
+ result += " SKIP " + select.limit
if select.distinct:
result += " DISTINCT "
return result
def limit_clause(self, select):
"""Already taken care of in the visit_select_precolumns method."""
return ""
- def default_from(self):
- return ' from RDB$DATABASE '
class FBSchemaGenerator(ansisql.ANSISchemaGenerator):
def get_column_specification(self, column, override_pk=False, **kwargs):
from sqlalchemy import *
from sqlalchemy.ansisql import *
-generic_engine = ansisql.engine()
+ischema = MetaData()
-gen_schemata = schema.Table("schemata", generic_engine,
+schemata = schema.Table("schemata", ischema,
Column("catalog_name", String),
Column("schema_name", String),
Column("schema_owner", String),
schema="information_schema")
-gen_tables = schema.Table("tables", generic_engine,
+tables = schema.Table("tables", ischema,
Column("table_catalog", String),
Column("table_schema", String),
Column("table_name", String),
Column("table_type", String),
schema="information_schema")
-gen_columns = schema.Table("columns", generic_engine,
+columns = schema.Table("columns", ischema,
Column("table_schema", String),
Column("table_name", String),
Column("column_name", String),
Column("column_default", Integer),
schema="information_schema")
-gen_constraints = schema.Table("table_constraints", generic_engine,
+constraints = schema.Table("table_constraints", ischema,
Column("table_schema", String),
Column("table_name", String),
Column("constraint_name", String),
Column("constraint_type", String),
schema="information_schema")
-gen_column_constraints = schema.Table("constraint_column_usage", generic_engine,
+column_constraints = schema.Table("constraint_column_usage", ischema,
Column("table_schema", String),
Column("table_name", String),
Column("column_name", String),
Column("constraint_name", String),
schema="information_schema")
-gen_key_constraints = schema.Table("key_column_usage", generic_engine,
+pg_key_constraints = schema.Table("key_column_usage", ischema,
Column("table_schema", String),
Column("table_name", String),
Column("column_name", String),
Column("constraint_name", String),
schema="information_schema")
-gen_ref_constraints = schema.Table("referential_constraints", generic_engine,
+mysql_key_constraints = schema.Table("key_column_usage", ischema,
+ Column("table_schema", String),
+ Column("table_name", String),
+ Column("column_name", String),
+ Column("constraint_name", String),
+ Column("referenced_table_schema", String),
+ Column("referenced_table_name", String),
+ Column("referenced_column_name", String),
+ schema="information_schema")
+
+key_constraints = pg_key_constraints
+
+ref_constraints = schema.Table("referential_constraints", ischema,
Column("constraint_catalog", String),
Column("constraint_schema", String),
Column("constraint_name", String),
return self.cache[name]
-def reflecttable(engine, table, ischema_names, use_mysql=False):
- columns = gen_columns.toengine(engine)
- constraints = gen_constraints.toengine(engine)
+def reflecttable(connection, table, ischema_names, use_mysql=False):
if use_mysql:
# no idea which INFORMATION_SCHEMA spec is correct, mysql or postgres
- key_constraints = schema.Table("key_column_usage", engine,
- Column("table_schema", String),
- Column("table_name", String),
- Column("column_name", String),
- Column("constraint_name", String),
- Column("referenced_table_schema", String),
- Column("referenced_table_name", String),
- Column("referenced_column_name", String),
- schema="information_schema", useexisting=True)
+ key_constraints = mysql_key_constraints
else:
- column_constraints = gen_column_constraints.toengine(engine)
- key_constraints = gen_key_constraints.toengine(engine)
-
-
+ key_constraints = pg_key_constraints
+
if table.schema is not None:
current_schema = table.schema
else:
- current_schema = engine.get_default_schema_name()
+ current_schema = connection.default_schema_name()
s = select([columns],
sql.and_(columns.c.table_name==table.name,
columns.c.table_schema==current_schema),
order_by=[columns.c.ordinal_position])
- c = s.execute()
+ c = connection.execute(s)
while True:
row = c.fetchone()
if row is None:
s.append_whereclause(constraints.c.table_name==table.name)
s.append_whereclause(constraints.c.table_schema==current_schema)
colmap = [constraints.c.constraint_type, key_constraints.c.column_name, key_constraints.c.referenced_table_schema, key_constraints.c.referenced_table_name, key_constraints.c.referenced_column_name]
- c = s.execute()
+ c = connection.execute(s)
while True:
row = c.fetchone()
if type=='PRIMARY KEY':
table.c[constrained_column]._set_primary_key()
elif type=='FOREIGN KEY':
- remotetable = Table(referred_table, engine, autoload = True, schema=referred_schema)
+ if current_schema == referred_schema:
+ referred_schema = table.schema
+ remotetable = Table(referred_table, table.metadata, autoload=True, autoload_with=connection, schema=referred_schema)
table.c[constrained_column].append_item(schema.ForeignKey(remotetable.c[referred_column]))
super(MSSQLCompiler, self).visit_column(column)
if column.table is not None and self.tablealiases.has_key(column.table):
self.strings[column] = \
- self.strings[self.tablealiases[column.table]._get_col_by_original(column.original)]
+ self.strings[self.tablealiases[column.table].corresponding_column(column.original)]
class MSSQLSchemaGenerator(ansisql.ANSISchemaGenerator):
import sys, StringIO, string, types, re, datetime
-import sqlalchemy.sql as sql
-import sqlalchemy.engine as engine
-import sqlalchemy.schema as schema
-import sqlalchemy.ansisql as ansisql
+from sqlalchemy import sql,engine,schema,ansisql
+from sqlalchemy.engine import default
import sqlalchemy.types as sqltypes
-from sqlalchemy import *
import sqlalchemy.databases.information_schema as ischema
-from sqlalchemy.exceptions import *
+import sqlalchemy.exceptions as exceptions
try:
import MySQLdb as mysql
class MSDouble(sqltypes.Numeric):
def __init__(self, precision = None, length = None):
if (precision is None and length is not None) or (precision is not None and length is None):
- raise ArgumentError("You must specify both precision and length or omit both altogether.")
+ raise exceptions.ArgumentError("You must specify both precision and length or omit both altogether.")
super(MSDouble, self).__init__(precision, length)
def get_col_spec(self):
if self.precision is not None and self.length is not None:
class MSTime(sqltypes.Time):
def get_col_spec(self):
return "TIME"
- def convert_result_value(self, value, engine):
+ def convert_result_value(self, value, dialect):
# convert from a timedelta value
if value is not None:
return datetime.time(value.seconds/60/60, value.seconds/60%60, value.seconds - (value.seconds/60*60))
return {'name':'mysql',
'description':'MySQL',
'arguments':[
- ('user',"Database Username",None),
- ('passwd',"Database Password",None),
- ('db',"Database Name",None),
+ ('username',"Database Username",None),
+ ('password',"Database Password",None),
+ ('database',"Database Name",None),
('host',"Hostname", None),
]}
-class MySQLEngine(ansisql.ANSISQLEngine):
- def __init__(self, opts, module = None, **params):
+
+class MySQLExecutionContext(default.DefaultExecutionContext):
+ def post_exec(self, engine, proxy, compiled, parameters, **kwargs):
+ if getattr(compiled, "isinsert", False):
+ self._last_inserted_ids = [proxy().lastrowid]
+
+class MySQLDialect(ansisql.ANSIDialect):
+ def __init__(self, module = None, **kwargs):
if module is None:
self.module = mysql
- self.opts = self._translate_connect_args(('host', 'db', 'user', 'passwd'), opts)
- ansisql.ANSISQLEngine.__init__(self, **params)
+ ansisql.ANSIDialect.__init__(self, **kwargs)
+
+ def create_connect_args(self, url):
+ opts = url.translate_connect_args(['host', 'db', 'user', 'passwd', 'port'])
+ return [[], opts]
- def connect_args(self):
- return [[], self.opts]
+ def create_execution_context(self):
+ return MySQLExecutionContext(self)
def type_descriptor(self, typeobj):
return sqltypes.adapt_type(typeobj, colspecs)
- def last_inserted_ids(self):
- return self.context.last_inserted_ids
def supports_sane_rowcount(self):
return False
def compiler(self, statement, bindparams, **kwargs):
- return MySQLCompiler(statement, bindparams, engine=self, **kwargs)
+ return MySQLCompiler(self, statement, bindparams, **kwargs)
- def schemagenerator(self, **params):
- return MySQLSchemaGenerator(self, **params)
+ def schemagenerator(self, *args, **kwargs):
+ return MySQLSchemaGenerator(*args, **kwargs)
- def schemadropper(self, **params):
- return MySQLSchemaDropper(self, **params)
+ def schemadropper(self, *args, **kwargs):
+ return MySQLSchemaDropper(*args, **kwargs)
def get_default_schema_name(self):
if not hasattr(self, '_default_schema_name'):
self._default_schema_name = text("select database()", self).scalar()
return self._default_schema_name
-
- def last_inserted_ids(self):
- return self.context.last_inserted_ids
-
- def post_exec(self, proxy, compiled, parameters, **kwargs):
- if getattr(compiled, "isinsert", False):
- self.context.last_inserted_ids = [proxy().lastrowid]
- # executemany just runs normally, since we arent using rowcount at all with mysql
-# def _executemany(self, c, statement, parameters):
- # """we need accurate rowcounts for updates, inserts and deletes. mysql is *also* is not nice enough
- # to produce this correctly for an executemany, so we do our own executemany here."""
- # rowcount = 0
- # for param in parameters:
- # c.execute(statement, param)
- # rowcount += c.rowcount
- # self.context.rowcount = rowcount
-
def dbapi(self):
return self.module
- def reflecttable(self, table):
+ def has_table(self, connection, table_name):
+ cursor = connection.execute("show table status like '" + table_name + "'")
+ return bool( not not cursor.rowcount )
+
+ def reflecttable(self, connection, table):
# to use information_schema:
#ischema.reflecttable(self, table, ischema_names, use_mysql=True)
- tabletype, foreignkeyD = self.moretableinfo(table=table)
+ tabletype, foreignkeyD = self.moretableinfo(connection, table=table)
table.kwargs['mysql_engine'] = tabletype
- c = self.execute("describe " + table.name, {})
+ c = connection.execute("describe " + table.name, {})
while True:
row = c.fetchone()
if row is None:
default=default
)))
- def moretableinfo(self, table):
+ def moretableinfo(self, connection, table):
"""Return (tabletype, {colname:foreignkey,...})
execute(SHOW CREATE TABLE child) =>
CREATE TABLE `child` (
KEY `par_ind` (`parent_id`),
CONSTRAINT `child_ibfk_1` FOREIGN KEY (`parent_id`) REFERENCES `parent` (`id`) ON DELETE CASCADE\n) TYPE=InnoDB
"""
- c = self.execute("SHOW CREATE TABLE " + table.name, {})
+ c = connection.execute("SHOW CREATE TABLE " + table.name, {})
desc = c.fetchone()[1].strip()
tabletype = ''
lastparen = re.search(r'\)[^\)]*\Z', desc)
if column.primary_key:
if not override_pk:
colspec += " PRIMARY KEY"
- if not column.foreign_key and first_pk and isinstance(column.type, types.Integer):
+ if not column.foreign_key and first_pk and isinstance(column.type, sqltypes.Integer):
colspec += " AUTO_INCREMENT"
if column.foreign_key:
colspec += ", FOREIGN KEY (%s) REFERENCES %s(%s)" % (column.name, column.foreign_key.column.table.name, column.foreign_key.column.name)
def visit_index(self, index):
self.append("\nDROP INDEX " + index.name + " ON " + index.table.name)
self.execute()
+
+dialect = MySQLDialect
\ No newline at end of file
import sys, StringIO, string
+import sqlalchemy.util as util
import sqlalchemy.sql as sql
+import sqlalchemy.engine as engine
+import sqlalchemy.engine.default as default
import sqlalchemy.schema as schema
import sqlalchemy.ansisql as ansisql
-from sqlalchemy import *
import sqlalchemy.types as sqltypes
+import sqlalchemy.exceptions as exceptions
try:
import cx_Oracle
-- order multiple primary keys correctly
ORDER BY ac.constraint_name, loc.position"""
-def engine(*args, **params):
- return OracleSQLEngine(*args, **params)
def descriptor():
return {'name':'oracle',
('user', 'Username', None),
('password', 'Password', None)
]}
+
+class OracleExecutionContext(default.DefaultExecutionContext):
+ pass
-class OracleSQLEngine(ansisql.ANSISQLEngine):
- def __init__(self, opts, use_ansi = True, module = None, threaded=False, **params):
- self._use_ansi = use_ansi
- self.opts = self._translate_connect_args((None, 'dsn', 'user', 'password'), opts)
- self.opts['threaded'] = threaded
+class OracleDialect(ansisql.ANSIDialect):
+ def __init__(self, use_ansi=True, module=None, threaded=True, **kwargs):
+ self.use_ansi = use_ansi
+ self.threaded = threaded
if module is None:
self.module = cx_Oracle
else:
self.module = module
- ansisql.ANSISQLEngine.__init__(self, **params)
+ ansisql.ANSIDialect.__init__(self, **kwargs)
def dbapi(self):
return self.module
- def connect_args(self):
- return [[], self.opts]
+ def create_connect_args(self, url):
+ opts = url.translate_connect_args([None, 'dsn', 'user', 'password'])
+ opts['threaded'] = self.threaded
+ return ([], opts)
def type_descriptor(self, typeobj):
return sqltypes.adapt_type(typeobj, colspecs)
- def last_inserted_ids(self):
- return self.context.last_inserted_ids
-
def oid_column_name(self):
return "rowid"
+ def create_execution_context(self):
+ return OracleExecutionContext(self)
+
def compiler(self, statement, bindparams, **kwargs):
- return OracleCompiler(self, statement, bindparams, use_ansi=self._use_ansi, **kwargs)
-
- def schemagenerator(self, **params):
- return OracleSchemaGenerator(self, **params)
- def schemadropper(self, **params):
- return OracleSchemaDropper(self, **params)
- def defaultrunner(self, proxy):
- return OracleDefaultRunner(self, proxy)
+ return OracleCompiler(self, statement, bindparams, **kwargs)
+ def schemagenerator(self, *args, **kwargs):
+ return OracleSchemaGenerator(*args, **kwargs)
+ def schemadropper(self, *args, **kwargs):
+ return OracleSchemaDropper(*args, **kwargs)
+ def defaultrunner(self, engine, proxy):
+ return OracleDefaultRunner(engine, proxy)
+
+
+ def has_table(self, connection, table_name):
+ cursor = connection.execute("""select table_name from all_tables where table_name=:name""", {'name':table_name.upper()})
+ return bool( cursor.fetchone() is not None )
- def reflecttable(self, table):
- c = self.execute ("select COLUMN_NAME, DATA_TYPE, DATA_LENGTH, DATA_PRECISION, DATA_SCALE, NULLABLE, DATA_DEFAULT from ALL_TAB_COLUMNS where TABLE_NAME = :table_name", {'table_name':table.name.upper()})
+ def reflecttable(self, connection, table):
+ c = connection.execute ("select COLUMN_NAME, DATA_TYPE, DATA_LENGTH, DATA_PRECISION, DATA_SCALE, NULLABLE, DATA_DEFAULT from ALL_TAB_COLUMNS where TABLE_NAME = :table_name", {'table_name':table.name.upper()})
while True:
row = c.fetchone()
colargs = []
if default is not None:
- colargs.append(PassiveDefault(sql.text(default)))
+ colargs.append(schema.PassiveDefault(sql.text(default)))
name = name.lower()
table.append_item (schema.Column(name, coltype, nullable=nullable, *colargs))
- c = self.execute(constraintSQL, {'table_name' : table.name.upper()})
+ c = connection.execute(constraintSQL, {'table_name' : table.name.upper()})
while True:
row = c.fetchone()
if row is None:
table.c[local_column]._set_primary_key()
elif cons_type == 'R':
table.c[local_column].append_item(
- schema.ForeignKey(Table(remote_table,
- self,
+ schema.ForeignKey(schema.Table(remote_table,
+ table.metadata,
autoload=True).c[remote_column]
)
)
- def last_inserted_ids(self):
- return self.context.last_inserted_ids
-
- def pre_exec(self, proxy, compiled, parameters, **kwargs):
- pass
-
- def _executemany(self, c, statement, parameters):
+ def do_executemany(self, c, statement, parameters, context=None):
rowcount = 0
for param in parameters:
c.execute(statement, param)
rowcount += c.rowcount
- self.context.rowcount = rowcount
+ if context is not None:
+ context._rowcount = rowcount
class OracleCompiler(ansisql.ANSICompiler):
"""oracle compiler modifies the lexical structure of Select statements to work under
non-ANSI configured Oracle databases, if the use_ansi flag is False."""
- def __init__(self, engine, statement, parameters, use_ansi = True, **kwargs):
- self._outertable = None
- self._use_ansi = use_ansi
- ansisql.ANSICompiler.__init__(self, statement, parameters, engine=engine, **kwargs)
-
def default_from(self):
"""called when a SELECT statement has no froms, and no FROM clause is to be appended.
gives Oracle a chance to tack on a "FROM DUAL" to the string output. """
return len(func.clauses) > 0
def visit_join(self, join):
- if self._use_ansi:
+ if self.dialect.use_ansi:
return ansisql.ANSICompiler.visit_join(self, join)
self.froms[join] = self.get_from_text(join.left) + ", " + self.get_from_text(join.right)
def visit_column(self, column):
ansisql.ANSICompiler.visit_column(self, column)
- if not self._use_ansi and self._outertable is not None and column.table is self._outertable:
+ if not self.dialect.use_ansi and getattr(self, '_outertable', None) is not None and column.table is self._outertable:
self.strings[column] = self.strings[column] + "(+)"
def visit_insert(self, insert):
self.strings[select.order_by_clause] = ""
ansisql.ANSICompiler.visit_select(self, select)
return
+
if select.limit is not None or select.offset is not None:
select._oracle_visit = True
# to use ROW_NUMBER(), an ORDER BY is required.
orderby = self.strings[select.order_by_clause]
if not orderby:
orderby = select.oid_column
+ orderby.accept_visitor(self)
+ orderby = self.strings[orderby]
select.append_column(sql.ColumnClause("ROW_NUMBER() OVER (ORDER BY %s)" % orderby).label("ora_rn"))
limitselect = sql.select([c for c in select.c if c.key!='ora_rn'])
if select.offset is not None:
def visit_sequence(self, seq):
return self.proxy("SELECT " + seq.name + ".nextval FROM DUAL").fetchone()[0]
+
+dialect = OracleDialect
import sqlalchemy.util as util
import sqlalchemy.sql as sql
import sqlalchemy.engine as engine
+import sqlalchemy.engine.default as default
import sqlalchemy.schema as schema
import sqlalchemy.ansisql as ansisql
import sqlalchemy.types as sqltypes
-from sqlalchemy.exceptions import *
-from sqlalchemy import *
+import sqlalchemy.exceptions as exceptions
import information_schema as ischema
try:
def get_col_spec(self):
return "TIMESTAMP"
class PG1DateTime(sqltypes.DateTime):
- def convert_bind_param(self, value, engine):
+ def convert_bind_param(self, value, dialect):
if value is not None:
if isinstance(value, datetime.datetime):
seconds = float(str(value.second) + "."
return psycopg.TimestampFromMx(value)
else:
return None
- def convert_result_value(self, value, engine):
+ def convert_result_value(self, value, dialect):
if value is None:
return None
second_parts = str(value.second).split(".")
return datetime.datetime(value.year, value.month, value.day,
value.hour, value.minute, seconds,
microseconds)
-
def get_col_spec(self):
return "TIMESTAMP"
class PG2Date(sqltypes.Date):
def get_col_spec(self):
return "DATE"
class PG1Date(sqltypes.Date):
- def convert_bind_param(self, value, engine):
+ def convert_bind_param(self, value, dialect):
# TODO: perform appropriate postgres1 conversion between Python DateTime/MXDateTime
# this one doesnt seem to work with the "emulation" mode
if value is not None:
return psycopg.DateFromMx(value)
else:
return None
- def convert_result_value(self, value, engine):
+ def convert_result_value(self, value, dialect):
# TODO: perform appropriate postgres1 conversion between Python DateTime/MXDateTime
return value
def get_col_spec(self):
def get_col_spec(self):
return "TIME"
class PG1Time(sqltypes.Time):
- def convert_bind_param(self, value, engine):
+ def convert_bind_param(self, value, dialect):
# TODO: perform appropriate postgres1 conversion between Python DateTime/MXDateTime
# this one doesnt seem to work with the "emulation" mode
if value is not None:
return psycopg.TimeFromMx(value)
else:
return None
- def convert_result_value(self, value, engine):
+ def convert_result_value(self, value, dialect):
# TODO: perform appropriate postgres1 conversion between Python DateTime/MXDateTime
return value
def get_col_spec(self):
return {'name':'postgres',
'description':'PostGres',
'arguments':[
- ('user',"Database Username",None),
+ ('username',"Database Username",None),
('password',"Database Password",None),
('database',"Database Name",None),
('host',"Hostname", None),
]}
-class PGSQLEngine(ansisql.ANSISQLEngine):
- def __init__(self, opts, module=None, use_oids=False, **params):
+class PGExecutionContext(default.DefaultExecutionContext):
+
+ def post_exec(self, engine, proxy, compiled, parameters, **kwargs):
+ if getattr(compiled, "isinsert", False) and self.last_inserted_ids is None:
+ if not engine.dialect.use_oids:
+ pass
+ # will raise invalid error when they go to get them
+ else:
+ table = compiled.statement.table
+ cursor = proxy()
+ if cursor.lastrowid is not None and table is not None and len(table.primary_key):
+ s = sql.select(table.primary_key, table.oid_column == cursor.lastrowid)
+ c = s.compile(engine=engine)
+ cursor = proxy(str(c), c.get_params())
+ row = cursor.fetchone()
+ self._last_inserted_ids = [v for v in row]
+
+class PGDialect(ansisql.ANSIDialect):
+ def __init__(self, module=None, use_oids=False, **params):
self.use_oids = use_oids
if module is None:
- if psycopg is None:
- raise ArgumentError("Couldnt locate psycopg1 or psycopg2: specify postgres module argument")
+ #if psycopg is None:
+ # raise exceptions.ArgumentError("Couldnt locate psycopg1 or psycopg2: specify postgres module argument")
self.module = psycopg
else:
self.module = module
self.version = 1
except:
self.version = 1
- self.opts = self._translate_connect_args(('host', 'database', 'user', 'password'), opts)
- if self.opts.has_key('port'):
+ ansisql.ANSIDialect.__init__(self, **params)
+
+ def create_connect_args(self, url):
+ opts = url.translate_connect_args(['host', 'database', 'user', 'password', 'port'])
+ if opts.has_key('port'):
if self.version == 2:
- self.opts['port'] = int(self.opts['port'])
+ opts['port'] = int(opts['port'])
else:
- self.opts['port'] = str(self.opts['port'])
-
- ansisql.ANSISQLEngine.__init__(self, **params)
-
- def connect_args(self):
- return [[], self.opts]
+ opts['port'] = str(opts['port'])
+ return ([], opts)
+
+ def create_execution_context(self):
+ return PGExecutionContext(self)
def type_descriptor(self, typeobj):
if self.version == 2:
return sqltypes.adapt_type(typeobj, pg1_colspecs)
def compiler(self, statement, bindparams, **kwargs):
- return PGCompiler(statement, bindparams, engine=self, **kwargs)
-
- def schemagenerator(self, **params):
- return PGSchemaGenerator(self, **params)
-
- def schemadropper(self, **params):
- return PGSchemaDropper(self, **params)
-
- def defaultrunner(self, proxy=None):
- return PGDefaultRunner(self, proxy)
+ return PGCompiler(self, statement, bindparams, **kwargs)
+ def schemagenerator(self, *args, **kwargs):
+ return PGSchemaGenerator(*args, **kwargs)
+ def schemadropper(self, *args, **kwargs):
+ return PGSchemaDropper(*args, **kwargs)
+ def defaultrunner(self, engine, proxy):
+ return PGDefaultRunner(engine, proxy)
- def get_default_schema_name(self):
+ def get_default_schema_name(self, connection):
if not hasattr(self, '_default_schema_name'):
- self._default_schema_name = text("select current_schema()", self).scalar()
+ self._default_schema_name = connection.scalar("select current_schema()", None)
return self._default_schema_name
def last_inserted_ids(self):
if self.context.last_inserted_ids is None:
- raise InvalidRequestError("no INSERT executed, or cant use cursor.lastrowid without Postgres OIDs enabled")
+ raise exceptions.InvalidRequestError("no INSERT executed, or cant use cursor.lastrowid without Postgres OIDs enabled")
else:
return self.context.last_inserted_ids
else:
return None
- def pre_exec(self, proxy, statement, parameters, **kwargs):
- return
-
- def post_exec(self, proxy, compiled, parameters, **kwargs):
- if getattr(compiled, "isinsert", False) and self.context.last_inserted_ids is None:
- if not self.use_oids:
- pass
- # will raise invalid error when they go to get them
- else:
- table = compiled.statement.table
- cursor = proxy()
- if cursor.lastrowid is not None and table is not None and len(table.primary_key):
- s = sql.select(table.primary_key, table.oid_column == cursor.lastrowid)
- c = s.compile()
- cursor = proxy(str(c), c.get_params())
- row = cursor.fetchone()
- self.context.last_inserted_ids = [v for v in row]
-
- def _executemany(self, c, statement, parameters):
+ def do_executemany(self, c, statement, parameters, context=None):
"""we need accurate rowcounts for updates, inserts and deletes. psycopg2 is not nice enough
to produce this correctly for an executemany, so we do our own executemany here."""
rowcount = 0
for param in parameters:
- try:
- c.execute(statement, param)
- except Exception, e:
- raise exceptions.SQLError(statement, param, e)
+ c.execute(statement, param)
rowcount += c.rowcount
- self.context.rowcount = rowcount
+ if context is not None:
+ context._rowcount = rowcount
def dbapi(self):
return self.module
- def reflecttable(self, table):
+ def has_table(self, connection, table_name):
+ cursor = connection.execute("""select relname from pg_class where lower(relname) = %(name)s""", {'name':table_name.lower()})
+ return bool( not not cursor.rowcount )
+
+ def reflecttable(self, connection, table):
if self.version == 2:
ischema_names = pg2_ischema_names
else:
ischema_names = pg1_ischema_names
- # give ischema the given table's engine with which to look up
- # other tables, not 'self', since it could be a ProxyEngine
- ischema.reflecttable(table.engine, table, ischema_names)
+ ischema.reflecttable(connection, table, ischema_names)
class PGCompiler(ansisql.ANSICompiler):
-
def visit_insert_column(self, column, parameters):
# Postgres advises against OID usage and turns it off in 8.1,
return "DISTINCT ON (" + str(select.distinct) + ") "
else:
return ""
-
+
def binary_operator_string(self, binary):
if isinstance(binary.type, sqltypes.String) and binary.operator == '+':
return '||'
def get_column_specification(self, column, override_pk=False, **kwargs):
colspec = column.name
- if column.primary_key and isinstance(column.type, types.Integer) and (column.default is None or (isinstance(column.default, schema.Sequence) and column.default.optional)):
+ if column.primary_key and isinstance(column.type, sqltypes.Integer) and (column.default is None or (isinstance(column.default, schema.Sequence) and column.default.optional)):
colspec += " SERIAL"
else:
colspec += " " + column.type.engine_impl(self.engine).get_col_spec()
if isinstance(column.default, schema.PassiveDefault):
c = self.proxy("select %s" % column.default.arg)
return c.fetchone()[0]
- elif isinstance(column.type, types.Integer) and (column.default is None or (isinstance(column.default, schema.Sequence) and column.default.optional)):
+ elif isinstance(column.type, sqltypes.Integer) and (column.default is None or (isinstance(column.default, schema.Sequence) and column.default.optional)):
sch = column.table.schema
if sch is not None:
exc = "select nextval('%s.%s_%s_seq')" % (sch, column.table.name, column.name)
return c.fetchone()[0]
else:
return None
+
+dialect = PGDialect
import sys, StringIO, string, types, re
-import sqlalchemy.sql as sql
-import sqlalchemy.engine as engine
-import sqlalchemy.schema as schema
-import sqlalchemy.ansisql as ansisql
+from sqlalchemy import sql, engine, schema, ansisql, exceptions, pool
+import sqlalchemy.engine.default as default
import sqlalchemy.types as sqltypes
-from sqlalchemy.exceptions import *
-from sqlalchemy.ansisql import *
import datetime,time
pysqlite2_timesupport = False # Change this if the init.d guys ever get around to supporting time cols
class SLDateTime(sqltypes.DateTime):
def get_col_spec(self):
return "TIMESTAMP"
- def convert_bind_param(self, value, engine):
+ def convert_bind_param(self, value, dialect):
if value is not None:
return str(value)
else:
return None
- def _cvt(self, value, engine, fmt):
+ def _cvt(self, value, dialect, fmt):
if value is None:
return None
parts = value.split('.')
except ValueError:
(value, microsecond) = (value, 0)
return time.strptime(value, fmt)[0:6] + (microsecond,)
- def convert_result_value(self, value, engine):
- tup = self._cvt(value, engine, "%Y-%m-%d %H:%M:%S")
+ def convert_result_value(self, value, dialect):
+ tup = self._cvt(value, dialect, "%Y-%m-%d %H:%M:%S")
return tup and datetime.datetime(*tup)
class SLDate(SLDateTime):
def get_col_spec(self):
return "DATE"
- def convert_result_value(self, value, engine):
- tup = self._cvt(value, engine, "%Y-%m-%d")
+ def convert_result_value(self, value, dialect):
+ tup = self._cvt(value, dialect, "%Y-%m-%d")
return tup and datetime.date(*tup[0:3])
class SLTime(SLDateTime):
def get_col_spec(self):
return "TIME"
- def convert_result_value(self, value, engine):
- tup = self._cvt(value, engine, "%H:%M:%S")
+ def convert_result_value(self, value, dialect):
+ tup = self._cvt(value, dialect, "%H:%M:%S")
return tup and datetime.time(*tup[4:7])
class SLText(sqltypes.TEXT):
def get_col_spec(self):
if pysqlite2_timesupport:
colspecs.update({sqltypes.Time : SLTime})
pragma_names.update({'TIME' : SLTime})
-
-def engine(opts, **params):
- return SQLiteSQLEngine(opts, **params)
def descriptor():
return {'name':'sqlite',
'description':'SQLite',
'arguments':[
- ('filename', "Database Filename",None)
+ ('database', "Database Filename",None)
]}
-
-class SQLiteSQLEngine(ansisql.ANSISQLEngine):
- def __init__(self, opts, **params):
- if sqlite is None:
- raise ArgumentError("Couldn't import sqlite or pysqlite2")
- self.filename = opts.pop('filename', ':memory:')
- self.opts = opts or {}
- params['poolclass'] = sqlalchemy.pool.SingletonThreadPool
- ansisql.ANSISQLEngine.__init__(self, **params)
- def post_exec(self, proxy, compiled, parameters, **kwargs):
- if getattr(compiled, "isinsert", False):
- self.context.last_inserted_ids = [proxy().lastrowid]
+class SQLiteExecutionContext(default.DefaultExecutionContext):
+ def post_exec(self, engine, proxy, compiled, parameters, **kwargs):
+ if getattr(compiled, "isinsert", False):
+ self._last_inserted_ids = [proxy().lastrowid]
+
+class SQLiteDialect(ansisql.ANSIDialect):
+ def compiler(self, statement, bindparams, **kwargs):
+ return SQLiteCompiler(self, statement, bindparams, **kwargs)
+ def schemagenerator(self, *args, **kwargs):
+ return SQLiteSchemaGenerator(*args, **kwargs)
+ def create_connect_args(self, url):
+ filename = url.database or ':memory:'
+ return ([filename], {})
def type_descriptor(self, typeobj):
return sqltypes.adapt_type(typeobj, colspecs)
-
+ def create_execution_context(self):
+ return SQLiteExecutionContext(self)
def last_inserted_ids(self):
return self.context.last_inserted_ids
def connect_args(self):
return ([self.filename], self.opts)
- def compiler(self, statement, bindparams, **kwargs):
- return SQLiteCompiler(statement, bindparams, engine=self, **kwargs)
-
def dbapi(self):
+ if sqlite is None:
+ raise ArgumentError("Couldn't import sqlite or pysqlite2")
return sqlite
def push_session(self):
raise InvalidRequestError("SQLite doesn't support nested sessions")
- def schemagenerator(self, **params):
- return SQLiteSchemaGenerator(self, **params)
+ def has_table(self, connection, table_name):
+ cursor = connection.execute("PRAGMA table_info(" + table_name + ")", {})
+ row = cursor.fetchone()
+ return (row is not None)
- def reflecttable(self, table):
- c = self.execute("PRAGMA table_info(" + table.name + ")", {})
+ def reflecttable(self, connection, table):
+ c = connection.execute("PRAGMA table_info(" + table.name + ")", {})
while True:
row = c.fetchone()
if row is None:
#print "args! " +repr(args)
coltype = coltype(*[int(a) for a in args])
table.append_item(schema.Column(name, coltype, primary_key = primary_key, nullable = nullable))
- c = self.execute("PRAGMA foreign_key_list(" + table.name + ")", {})
+ c = connection.execute("PRAGMA foreign_key_list(" + table.name + ")", {})
while True:
row = c.fetchone()
if row is None:
#print "row! " + repr(row)
# look up the table based on the given table's engine, not 'self',
# since it could be a ProxyEngine
- remotetable = Table(tablename, table.engine, autoload = True)
+ remotetable = schema.Table(tablename, table.metadata, autoload=True, autoload_with=connection)
table.c[localcol].append_item(schema.ForeignKey(remotetable.c[remotecol]))
# check for UNIQUE indexes
- c = self.execute("PRAGMA index_list(" + table.name + ")", {})
+ c = connection.execute("PRAGMA index_list(" + table.name + ")", {})
unique_indexes = []
while True:
row = c.fetchone()
unique_indexes.append(row[1])
# loop thru unique indexes for one that includes the primary key
for idx in unique_indexes:
- c = self.execute("PRAGMA index_info(" + idx + ")", {})
+ c = connection.execute("PRAGMA index_info(" + idx + ")", {})
cols = []
while True:
row = c.fetchone()
table.columns[col]._set_primary_key()
class SQLiteCompiler(ansisql.ANSICompiler):
- def __init__(self, *args, **params):
- params.setdefault('paramstyle', 'named')
- ansisql.ANSICompiler.__init__(self, *args, **params)
def limit_clause(self, select):
text = ""
if select.limit is not None:
return '||'
else:
return ansisql.ANSICompiler.binary_operator_string(self, binary)
-
+
class SQLiteSchemaGenerator(ansisql.ANSISchemaGenerator):
def get_column_specification(self, column, override_pk=False, **kwargs):
colspec = column.name + " " + column.type.engine_impl(self.engine).get_col_spec()
for index in table.indexes:
self.visit_index(index)
-
+dialect = SQLiteDialect
+poolclass = pool.SingletonThreadPool
+++ /dev/null
-# engine.py
-# Copyright (C) 2005,2006 Michael Bayer mike_mp@zzzcomputing.com
-#
-# This module is part of SQLAlchemy and is released under
-# the MIT License: http://www.opensource.org/licenses/mit-license.php
-
-"""Defines the SQLEngine class, which serves as the primary "database" object
-used throughout the sql construction and object-relational mapper packages.
-A SQLEngine is a facade around a single connection pool corresponding to a
-particular set of connection parameters, and provides thread-local transactional
-methods and statement execution methods for Connection objects. It also provides
-a facade around a Cursor object to allow richer column selection for result rows
-as well as type conversion operations, known as a ResultProxy.
-
-A SQLEngine is provided to an application as a subclass that is specific to a particular type
-of DBAPI, and is the central switching point for abstracting different kinds of database
-behavior into a consistent set of behaviors. It provides a variety of factory methods
-to produce everything specific to a certain kind of database, including a Compiler,
-schema creation/dropping objects.
-
-The term "database-specific" will be used to describe any object or function that has behavior
-corresponding to a particular vendor, such as mysql-specific, sqlite-specific, etc.
-"""
-
-import sqlalchemy.pool
-import schema
-import exceptions
-import util
-import sql
-import sqlalchemy.databases
-import sqlalchemy.types as types
-import StringIO, sys, re
-from cgi import parse_qsl
-
-__all__ = ['create_engine', 'engine_descriptors']
-
-def create_engine(name, opts=None,**kwargs):
- """creates a new SQLEngine instance. There are two forms of calling this method.
-
- In the first, the "name" argument is the type of engine to load, i.e. 'sqlite', 'postgres',
- 'oracle', 'mysql'. "opts" is a dictionary of options to be sent to the underlying DBAPI module
- to create a connection, usually including a hostname, username, password, etc.
-
- In the second, the "name" argument is a URL in the form <enginename>://opt1=val1&opt2=val2.
- Where <enginename> is the name as above, and the contents of the option dictionary are
- spelled out as a URL encoded string. The "opts" argument is not used.
-
- In both cases, **kwargs represents options to be sent to the SQLEngine itself. A possibly
- partial listing of those options is as follows:
-
- pool=None : an instance of sqlalchemy.pool.DBProxy or sqlalchemy.pool.Pool to be used as the
- underlying source for connections (DBProxy/Pool is described in the previous section). If None,
- a default DBProxy will be created using the engine's own database module with the given
- arguments.
-
- echo=False : if True, the SQLEngine will log all statements as well as a repr() of their
- parameter lists to the engines logger, which defaults to sys.stdout. A SQLEngine instances'
- "echo" data member can be modified at any time to turn logging on and off. If set to the string
- 'debug', result rows will be printed to the standard output as well.
-
- logger=None : a file-like object where logging output can be sent, if echo is set to True.
- This defaults to sys.stdout.
-
- module=None : used by Oracle and Postgres, this is a reference to a DBAPI2 module to be used
- instead of the engine's default module. For Postgres, the default is psycopg2, or psycopg1 if
- 2 cannot be found. For Oracle, its cx_Oracle. For mysql, MySQLdb.
-
- use_ansi=True : used only by Oracle; when False, the Oracle driver attempts to support a
- particular "quirk" of some Oracle databases, that the LEFT OUTER JOIN SQL syntax is not
- supported, and the "Oracle join" syntax of using <column1>(+)=<column2> must be used
- in order to achieve a LEFT OUTER JOIN. Its advised that the Oracle database be configured to
- have full ANSI support instead of using this feature.
-
- """
- m = re.match(r'(\w+)://(.*)', name)
- if m is not None:
- (name, args) = m.group(1, 2)
- opts = dict( parse_qsl( args ) )
- module = getattr(__import__('sqlalchemy.databases.%s' % name).databases, name)
- return module.engine(opts, **kwargs)
-
-def engine_descriptors():
- """provides a listing of all the database implementations supported. this data
- is provided as a list of dictionaries, where each dictionary contains the following
- key/value pairs:
-
- name : the name of the engine, suitable for use in the create_engine function
-
- description: a plain description of the engine.
-
- arguments : a dictionary describing the name and description of each parameter
- used to connect to this engine's underlying DBAPI.
-
- This function is meant for usage in automated configuration tools that wish to
- query the user for database and connection information.
- """
- result = []
- for module in sqlalchemy.databases.__all__:
- module = getattr(__import__('sqlalchemy.databases.%s' % module).databases, module)
- result.append(module.descriptor())
- return result
-
-class SchemaIterator(schema.SchemaVisitor):
- """a visitor that can gather text into a buffer and execute the contents of the buffer."""
- def __init__(self, engine, **params):
- """initializes this SchemaIterator and initializes its buffer.
-
- sqlproxy - a callable function returned by SQLEngine.proxy(), which executes a
- statement plus optional parameters.
- """
- self.engine = engine
- self.buffer = StringIO.StringIO()
-
- def append(self, s):
- """appends content to the SchemaIterator's query buffer."""
- self.buffer.write(s)
-
- def execute(self):
- """executes the contents of the SchemaIterator's buffer using its sql proxy and
- clears out the buffer."""
- try:
- return self.engine.execute(self.buffer.getvalue(), None)
- finally:
- self.buffer.truncate(0)
-
-class DefaultRunner(schema.SchemaVisitor):
- def __init__(self, engine, proxy):
- self.proxy = proxy
- self.engine = engine
-
- def get_column_default(self, column):
- if column.default is not None:
- return column.default.accept_schema_visitor(self)
- else:
- return None
-
- def get_column_onupdate(self, column):
- if column.onupdate is not None:
- return column.onupdate.accept_schema_visitor(self)
- else:
- return None
-
- def visit_passive_default(self, default):
- """passive defaults by definition return None on the app side,
- and are post-fetched to get the DB-side value"""
- return None
-
- def visit_sequence(self, seq):
- """sequences are not supported by default"""
- return None
-
- def exec_default_sql(self, default):
- c = sql.select([default.arg], engine=self.engine).compile()
- return self.proxy(str(c), c.get_params()).fetchone()[0]
-
- def visit_column_onupdate(self, onupdate):
- if isinstance(onupdate.arg, sql.ClauseElement):
- return self.exec_default_sql(onupdate)
- elif callable(onupdate.arg):
- return onupdate.arg()
- else:
- return onupdate.arg
-
- def visit_column_default(self, default):
- if isinstance(default.arg, sql.ClauseElement):
- return self.exec_default_sql(default)
- elif callable(default.arg):
- return default.arg()
- else:
- return default.arg
-
-class SQLSession(object):
- """represents a a handle to the SQLEngine's connection pool. the default SQLSession maintains a distinct connection during transactions, otherwise returns connections newly retrieved from the pool each time. the Pool is usually configured to have use_threadlocal=True so if a particular connection is already checked out, youll get that same connection in the same thread. There can also be a "unique" SQLSession pushed onto the engine, which returns a connection via the unique_connection() method on Pool; this allows nested transactions to take place, or other operations upon more than one connection at a time.`"""
- def __init__(self, engine, parent=None):
- self.engine = engine
- self.parent = parent
- # if we have a parent SQLSession, then use a unique connection.
- # else we use the default connection returned by the pool.
- if parent is not None:
- self.__connection = self.engine._pool.unique_connection()
- self.__tcount = 0
- def pop(self):
- self.engine.pop_session(self)
- def _connection(self):
- try:
- return self.__transaction
- except AttributeError:
- try:
- return self.__connection
- except AttributeError:
- return self.engine._pool.connect()
- connection = property(_connection, doc="the connection represented by this SQLSession. The connection is late-connecting, meaning the call to the connection pool only occurs when it is first called (and the pool will typically only connect the first time it is called as well)")
-
- def begin(self):
- """begins a transaction on this SQLSession's connection. repeated calls to begin() will increment a counter that must be decreased by corresponding commit() statements before an actual commit occurs. this is to provide "nested" behavior of transactions so that different functions in a particular call stack can call begin()/commit() independently of each other without knowledge of an existing transaction. """
- if self.__tcount == 0:
- self.__transaction = self.connection
- self.engine.do_begin(self.connection)
- self.__tcount += 1
- def rollback(self):
- """rolls back the transaction on this SQLSession's connection. this can be called regardless of the "begin" counter value, i.e. can be called from anywhere inside a callstack. the "begin" counter is cleared."""
- if self.__tcount > 0:
- try:
- self.engine.do_rollback(self.connection)
- finally:
- del self.__transaction
- self.__tcount = 0
- def commit(self):
- """commits the transaction started by begin(). If begin() was called multiple times, a counter will be decreased for each call to commit(), with the actual commit operation occuring when the counter reaches zero. this is to provide "nested" behavior of transactions so that different functions in a particular call stack can call begin()/commit() independently of each other without knowledge of an existing transaction."""
- if self.__tcount == 1:
- try:
- self.engine.do_commit(self.connection)
- finally:
- del self.__transaction
- self.__tcount = 0
- elif self.__tcount > 1:
- self.__tcount -= 1
- def is_begun(self):
- return self.__tcount > 0
-
-class SQLEngine(schema.SchemaEngine):
- """
- The central "database" object used by an application. Subclasses of this object is used
- by the schema and SQL construction packages to provide database-specific behaviors,
- as well as an execution and thread-local transaction context.
-
- SQLEngines are constructed via the create_engine() function inside this package.
- """
-
- def __init__(self, pool=None, echo=False, logger=None, default_ordering=False, echo_pool=False, echo_uow=False, convert_unicode=False, encoding='utf-8', **params):
- """constructs a new SQLEngine. SQLEngines should be constructed via the create_engine()
- function which will construct the appropriate subclass of SQLEngine."""
- # get a handle on the connection pool via the connect arguments
- # this insures the SQLEngine instance integrates with the pool referenced
- # by direct usage of pool.manager(<module>).connect(*args, **params)
- schema.SchemaEngine.__init__(self)
- (cargs, cparams) = self.connect_args()
- if pool is None:
- params['echo'] = echo_pool
- params['use_threadlocal'] = True
- self._pool = sqlalchemy.pool.manage(self.dbapi(), **params).get_pool(*cargs, **cparams)
- elif isinstance(pool, sqlalchemy.pool.DBProxy):
- self._pool = pool.get_pool(*cargs, **cparams)
- else:
- self._pool = pool
- self.default_ordering=default_ordering
- self.echo = echo
- self.echo_uow = echo_uow
- self.convert_unicode = convert_unicode
- self.encoding = encoding
- self.context = util.ThreadLocal()
- self._ischema = None
- self._figure_paramstyle()
- self.logger = logger or util.Logger(origin='engine')
-
- def _translate_connect_args(self, names, args):
- """translates a dictionary of connection arguments to those used by a specific dbapi.
- the names parameter is a tuple of argument names in the form ('host', 'database', 'user', 'password')
- where the given strings match the corresponding argument names for the dbapi. Will return a dictionary
- with the dbapi-specific parameters, the generic ones removed, and any additional parameters still remaining,
- from the dictionary represented by args. Will return a blank dictionary if args is null."""
- if args is None:
- return {}
- a = args.copy()
- standard_names = [('host','hostname'), ('database', 'dbname'), ('user', 'username'), ('password', 'passwd', 'pw')]
- for n in names:
- sname = standard_names.pop(0)
- if n is None:
- continue
- for sn in sname:
- if sn != n and a.has_key(sn):
- a[n] = a[sn]
- del a[sn]
- return a
- def _get_ischema(self):
- # We use a property for ischema so that the accessor
- # creation only happens as needed, since otherwise we
- # have a circularity problem with the generic
- # ansisql.engine()
- if self._ischema is None:
- import sqlalchemy.databases.information_schema as ischema
- self._ischema = ischema.ISchema(self)
- return self._ischema
- ischema = property(_get_ischema, doc="""returns an ISchema object for this engine, which allows access to information_schema tables (if supported)""")
-
- def hash_key(self):
- return "%s(%s)" % (self.__class__.__name__, repr(self.connect_args()))
-
- def _get_name(self):
- return sys.modules[self.__module__].descriptor()['name']
- name = property(_get_name)
-
- def dispose(self):
- """disposes of the underlying pool manager for this SQLEngine."""
- (cargs, cparams) = self.connect_args()
- sqlalchemy.pool.manage(self.dbapi()).dispose(*cargs, **cparams)
- self._pool = None
-
- def _set_paramstyle(self, style):
- self._paramstyle = style
- self._figure_paramstyle(style)
- paramstyle = property(lambda s:s._paramstyle, _set_paramstyle)
-
- def _figure_paramstyle(self, paramstyle=None):
- db = self.dbapi()
- if paramstyle is not None:
- self._paramstyle = paramstyle
- elif db is not None:
- self._paramstyle = db.paramstyle
- else:
- self._paramstyle = 'named'
-
- if self._paramstyle == 'named':
- self.positional=False
- elif self._paramstyle == 'pyformat':
- self.positional=False
- elif self._paramstyle == 'qmark' or self._paramstyle == 'format' or self._paramstyle == 'numeric':
- # for positional, use pyformat internally, ANSICompiler will convert
- # to appropriate character upon compilation
- self.positional = True
- else:
- raise DBAPIError("Unsupported paramstyle '%s'" % self._paramstyle)
-
- def type_descriptor(self, typeobj):
- """provides a database-specific TypeEngine object, given the generic object
- which comes from the types module. Subclasses will usually use the adapt_type()
- method in the types module to make this job easy."""
- if type(typeobj) is type:
- typeobj = typeobj()
- return typeobj
-
- def _func(self):
- return sql.FunctionGenerator(self)
- func = property(_func)
-
- def text(self, text, *args, **kwargs):
- """returns a sql.text() object for performing literal queries."""
- return sql.text(text, engine=self, *args, **kwargs)
-
- def schemagenerator(self, **params):
- """returns a schema.SchemaVisitor instance that can generate schemas, when it is
- invoked to traverse a set of schema objects.
-
- schemagenerator is called via the create() method.
- """
- raise NotImplementedError()
-
- def schemadropper(self, **params):
- """returns a schema.SchemaVisitor instance that can drop schemas, when it is
- invoked to traverse a set of schema objects.
-
- schemagenerator is called via the drop() method.
- """
- raise NotImplementedError()
-
- def defaultrunner(self, proxy=None):
- """Returns a schema.SchemaVisitor instance that can execute the default values on a column.
- The base class for this visitor is the DefaultRunner class inside this module.
- This visitor will typically only receive schema.DefaultGenerator schema objects. The given
- proxy is a callable that takes a string statement and a dictionary of bind parameters
- to be executed. For engines that require positional arguments, the dictionary should
- be an instance of OrderedDict which returns its bind parameters in the proper order.
-
- defaultrunner is called within the context of the execute_compiled() method."""
- return DefaultRunner(self, proxy)
-
- def compiler(self, statement, parameters):
- """returns a sql.ClauseVisitor which will produce a string representation of the given
- ClauseElement and parameter dictionary. This object is usually a subclass of
- ansisql.ANSICompiler.
-
- compiler is called within the context of the compile() method."""
- raise NotImplementedError()
-
- def oid_column_name(self):
- """returns the oid column name for this engine, or None if the engine cant/wont support OID/ROWID."""
- return None
-
- def supports_sane_rowcount(self):
- """Provided to indicate when MySQL is being used, which does not have standard behavior
- for the "rowcount" function on a statement handle. """
- return True
-
- def create(self, entity, **params):
- """creates a table or index within this engine's database connection given a schema.Table object."""
- entity.accept_schema_visitor(self.schemagenerator(**params))
- return entity
-
- def drop(self, entity, **params):
- """drops a table or index within this engine's database connection given a schema.Table object."""
- entity.accept_schema_visitor(self.schemadropper(**params))
-
- def compile(self, statement, parameters, **kwargs):
- """given a sql.ClauseElement statement plus optional bind parameters, creates a new
- instance of this engine's SQLCompiler, compiles the ClauseElement, and returns the
- newly compiled object."""
- compiler = self.compiler(statement, parameters, **kwargs)
- compiler.compile()
- return compiler
-
- def reflecttable(self, table):
- """given a Table object, reflects its columns and properties from the database."""
- raise NotImplementedError()
-
- def get_default_schema_name(self):
- """returns the currently selected schema in the current connection."""
- return None
-
- def last_inserted_ids(self):
- """returns a thread-local list of the primary key values for the last insert statement executed.
- This does not apply to straight textual clauses; only to sql.Insert objects compiled against
- a schema.Table object, which are executed via statement.execute(). The order of items in the
- list is the same as that of the Table's 'primary_key' attribute.
-
- In some cases, this method may invoke a query back to the database to retrieve the data, based on
- the "lastrowid" value in the cursor."""
- raise NotImplementedError()
-
- def connect_args(self):
- """subclasses override this method to provide a two-item tuple containing the *args
- and **kwargs used to establish a connection."""
- raise NotImplementedError()
-
- def dbapi(self):
- """subclasses override this method to provide the DBAPI module used to establish
- connections."""
- raise NotImplementedError()
-
- def do_begin(self, connection):
- """implementations might want to put logic here for turning autocommit on/off,
- etc."""
- pass
- def do_rollback(self, connection):
- """implementations might want to put logic here for turning autocommit on/off,
- etc."""
- #print "ENGINE ROLLBACK ON ", connection.connection
- connection.rollback()
- def do_commit(self, connection):
- """implementations might want to put logic here for turning autocommit on/off, etc."""
- #print "ENGINE COMMIT ON ", connection.connection
- connection.commit()
-
- def _session(self):
- if not hasattr(self.context, 'session'):
- self.context.session = SQLSession(self)
- return self.context.session
- session = property(_session, doc="returns the current thread's SQLSession")
-
- def push_session(self):
- """pushes a new SQLSession onto this engine, temporarily replacing the previous one for the current thread. The previous session can be restored by calling pop_session(). this allows the usage of a new connection and possibly transaction within a particular block, superceding the existing one, including any transactions that are in progress. Returns the new SQLSession object."""
- sess = SQLSession(self, self.context.session)
- self.context.session = sess
- return sess
- def pop_session(self, s = None):
- """restores the current thread's SQLSession to that before the last push_session. Returns the restored SQLSession object. Raises an exception if there is no SQLSession pushed onto the stack."""
- sess = self.context.session.parent
- if sess is None:
- raise exceptions.InvalidRequestError("No SQLSession is pushed onto the stack.")
- elif s is not None and s is not self.context.session:
- raise exceptions.InvalidRequestError("Given SQLSession is not the current session on the stack")
- self.context.session = sess
- return sess
-
- def connection(self):
- """returns a managed DBAPI connection from this SQLEngine's connection pool."""
- return self.session.connection
-
- def unique_connection(self):
- """returns a DBAPI connection from this SQLEngine's connection pool that is distinct from the current thread's connection."""
- return self._pool.unique_connection()
-
- def multi_transaction(self, tables, func):
- """provides a transaction boundary across tables which may be in multiple databases.
- If you have three tables, and a function that operates upon them, providing the tables as a
- list and the function will result in a begin()/commit() pair invoked for each distinct engine
- represented within those tables, and the function executed within the context of that transaction.
- any exceptions will result in a rollback().
-
- clearly, this approach only goes so far, such as if database A commits, then database B commits
- and fails, A is already committed. Any failure conditions have to be raised before anyone
- commits for this to be useful."""
- engines = util.HashSet()
- for table in tables:
- engines.append(table.engine)
- for engine in engines:
- engine.begin()
- try:
- func()
- except:
- for engine in engines:
- engine.rollback()
- raise
- for engine in engines:
- engine.commit()
-
- def transaction(self, func, *args, **kwargs):
- """executes the given function within a transaction boundary. this is a shortcut for
- explicitly calling begin() and commit() and optionally rollback() when execptions are raised.
- The given *args and **kwargs will be passed to the function as well, which could be handy
- in constructing decorators."""
- self.begin()
- try:
- func(*args, **kwargs)
- except:
- self.rollback()
- raise
- self.commit()
-
- def begin(self):
- """ begins a transaction on the current thread SQLSession. """
- self.session.begin()
-
- def rollback(self):
- """rolls back the transaction on the current thread's SQLSession."""
- self.session.rollback()
-
- def commit(self):
- self.session.commit()
-
- def _process_defaults(self, proxy, compiled, parameters, **kwargs):
- """INSERT and UPDATE statements, when compiled, may have additional columns added to their
- VALUES and SET lists corresponding to column defaults/onupdates that are present on the
- Table object (i.e. ColumnDefault, Sequence, PassiveDefault). This method pre-execs those
- DefaultGenerator objects that require pre-execution and sets their values within the
- parameter list, and flags the thread-local state about
- PassiveDefault objects that may require post-fetching the row after it is inserted/updated.
- This method relies upon logic within the ANSISQLCompiler in its visit_insert and
- visit_update methods that add the appropriate column clauses to the statement when its
- being compiled, so that these parameters can be bound to the statement."""
- if compiled is None: return
- if getattr(compiled, "isinsert", False):
- if isinstance(parameters, list):
- plist = parameters
- else:
- plist = [parameters]
- drunner = self.defaultrunner(proxy)
- self.context.lastrow_has_defaults = False
- for param in plist:
- last_inserted_ids = []
- need_lastrowid=False
- for c in compiled.statement.table.c:
- if not param.has_key(c.name) or param[c.name] is None:
- if isinstance(c.default, schema.PassiveDefault):
- self.context.lastrow_has_defaults = True
- newid = drunner.get_column_default(c)
- if newid is not None:
- param[c.name] = newid
- if c.primary_key:
- last_inserted_ids.append(param[c.name])
- elif c.primary_key:
- need_lastrowid = True
- elif c.primary_key:
- last_inserted_ids.append(param[c.name])
- if need_lastrowid:
- self.context.last_inserted_ids = None
- else:
- self.context.last_inserted_ids = last_inserted_ids
- self.context.last_inserted_params = param
- elif getattr(compiled, 'isupdate', False):
- if isinstance(parameters, list):
- plist = parameters
- else:
- plist = [parameters]
- drunner = self.defaultrunner(proxy)
- self.context.lastrow_has_defaults = False
- for param in plist:
- for c in compiled.statement.table.c:
- if c.onupdate is not None and (not param.has_key(c.name) or param[c.name] is None):
- value = drunner.get_column_onupdate(c)
- if value is not None:
- param[c.name] = value
- self.context.last_updated_params = param
-
- def last_inserted_params(self):
- """returns a dictionary of the full parameter dictionary for the last compiled INSERT statement,
- including any ColumnDefaults or Sequences that were pre-executed. this value is thread-local."""
- return self.context.last_inserted_params
- def last_updated_params(self):
- """returns a dictionary of the full parameter dictionary for the last compiled UPDATE statement,
- including any ColumnDefaults that were pre-executed. this value is thread-local."""
- return self.context.last_updated_params
- def lastrow_has_defaults(self):
- """returns True if the last row INSERTED via a compiled insert statement contained PassiveDefaults,
- indicating that the database inserted data beyond that which we gave it. this value is thread-local."""
- return self.context.lastrow_has_defaults
-
- def pre_exec(self, proxy, compiled, parameters, **kwargs):
- """called by execute_compiled before the compiled statement is executed."""
- pass
-
- def post_exec(self, proxy, compiled, parameters, **kwargs):
- """called by execute_compiled after the compiled statement is executed."""
- pass
-
- def execute_compiled(self, compiled, parameters, connection=None, cursor=None, echo=None, **kwargs):
- """executes the given compiled statement object with the given parameters.
-
- The parameters can be a dictionary of key/value pairs, or a list of dictionaries for an
- executemany() style of execution. Engines that use positional parameters will convert
- the parameters to a list before execution.
-
- If the current thread has specified a transaction begin() for this engine, the
- statement will be executed in the context of the current transactional connection.
- Otherwise, a commit() will be performed immediately after execution, since the local
- pooled connection is returned to the pool after execution without a transaction set
- up.
-
- In all error cases, a rollback() is immediately performed on the connection before
- propigating the exception outwards.
-
- Other options include:
-
- connection - a DBAPI connection to use for the execute. If None, a connection is
- pulled from this engine's connection pool.
-
- echo - enables echo for this execution, which causes all SQL and parameters
- to be dumped to the engine's logging output before execution.
-
- typemap - a map of column names mapped to sqlalchemy.types.TypeEngine objects.
- These will be passed to the created ResultProxy to perform
- post-processing on result-set values.
-
- commit - if True, will automatically commit the statement after completion. """
-
- if connection is None:
- connection = self.connection()
-
- if cursor is None:
- cursor = connection.cursor()
-
- executemany = parameters is not None and (isinstance(parameters, list) or isinstance(parameters, tuple))
- if executemany:
- parameters = [compiled.get_params(**m) for m in parameters]
- else:
- parameters = compiled.get_params(**parameters)
- def proxy(statement=None, parameters=None):
- if statement is None:
- return cursor
-
- parameters = self._convert_compiled_params(parameters)
- self.execute(statement, parameters, connection=connection, cursor=cursor, return_raw=True)
- return cursor
-
- self.pre_exec(proxy, compiled, parameters, **kwargs)
- self._process_defaults(proxy, compiled, parameters, **kwargs)
- proxy(str(compiled), parameters)
- self.post_exec(proxy, compiled, parameters, **kwargs)
- return ResultProxy(cursor, self, typemap=compiled.typemap)
-
- def execute(self, statement, parameters=None, connection=None, cursor=None, echo=None, typemap=None, commit=False, return_raw=False, **kwargs):
- """ executes the given string-based SQL statement with the given parameters.
-
- The parameters can be a dictionary or a list, or a list of dictionaries or lists, depending
- on the paramstyle of the DBAPI.
-
- If the current thread has specified a transaction begin() for this engine, the
- statement will be executed in the context of the current transactional connection.
- Otherwise, a commit() will be performed immediately after execution, since the local
- pooled connection is returned to the pool after execution without a transaction set
- up.
-
- In all error cases, a rollback() is immediately performed on the connection before
- propagating the exception outwards.
-
- Other options include:
-
- connection - a DBAPI connection to use for the execute. If None, a connection is
- pulled from this engine's connection pool.
-
- echo - enables echo for this execution, which causes all SQL and parameters
- to be dumped to the engine's logging output before execution.
-
- typemap - a map of column names mapped to sqlalchemy.types.TypeEngine objects.
- These will be passed to the created ResultProxy to perform
- post-processing on result-set values.
-
- commit - if True, will automatically commit the statement after completion. """
-
- if connection is None:
- connection = self.connection()
-
- if cursor is None:
- cursor = connection.cursor()
-
- try:
- if echo is True or self.echo is not False:
- self.log(statement)
- self.log(repr(parameters))
- if parameters is not None and isinstance(parameters, list) and len(parameters) > 0 and (isinstance(parameters[0], list) or isinstance(parameters[0], dict)):
- self._executemany(cursor, statement, parameters)
- else:
- self._execute(cursor, statement, parameters)
- if not self.session.is_begun():
- self.do_commit(connection)
- except:
- self.do_rollback(connection)
- raise
- if return_raw:
- return cursor
- else:
- return ResultProxy(cursor, self, typemap=typemap)
-
- def _execute(self, c, statement, parameters):
- if parameters is None:
- if self.positional:
- parameters = ()
- else:
- parameters = {}
- try:
- c.execute(statement, parameters)
- except Exception, e:
- raise exceptions.SQLError(statement, parameters, e)
- self.context.rowcount = c.rowcount
- def _executemany(self, c, statement, parameters):
- c.executemany(statement, parameters)
- self.context.rowcount = c.rowcount
-
- def _convert_compiled_params(self, parameters):
- executemany = parameters is not None and isinstance(parameters, list)
- # the bind params are a CompiledParams object. but all the DBAPI's hate
- # that object (or similar). so convert it to a clean
- # dictionary/list/tuple of dictionary/tuple of list
- if parameters is not None:
- if self.positional:
- if executemany:
- parameters = [p.values() for p in parameters]
- else:
- parameters = parameters.values()
- else:
- if executemany:
- parameters = [p.get_raw_dict() for p in parameters]
- else:
- parameters = parameters.get_raw_dict()
- return parameters
-
- def proxy(self, statement=None, parameters=None):
- """returns a callable which will execute the given statement string and parameter object.
- the parameter object is expected to be the result of a call to compiled.get_params().
- This callable is a generic version of a connection/cursor-specific callable that
- is produced within the execute_compiled method, and is used for objects that require
- this style of proxy when outside of an execute_compiled method, primarily the DefaultRunner."""
- parameters = self._convert_compiled_params(parameters)
- return self.execute(statement, parameters)
-
- def log(self, msg):
- """logs a message using this SQLEngine's logger stream."""
- self.logger.write(msg)
-
-
-class ResultProxy:
- """wraps a DBAPI cursor object to provide access to row columns based on integer
- position, case-insensitive column name, or by schema.Column object. e.g.:
-
- row = fetchone()
-
- col1 = row[0] # access via integer position
-
- col2 = row['col2'] # access via name
-
- col3 = row[mytable.c.mycol] # access via Column object.
-
- ResultProxy also contains a map of TypeEngine objects and will invoke the appropriate
- convert_result_value() method before returning columns.
- """
- class AmbiguousColumn(object):
- def __init__(self, key):
- self.key = key
- def convert_result_value(self, arg, engine):
- raise InvalidRequestError("Ambiguous column name '%s' in result set! try 'use_labels' option on select statement." % (self.key))
-
- def __init__(self, cursor, engine, typemap = None):
- """ResultProxy objects are constructed via the execute() method on SQLEngine."""
- self.cursor = cursor
- self.engine = engine
- self.echo = engine.echo=="debug"
- self.rowcount = engine.context.rowcount
- metadata = cursor.description
- self.props = {}
- self.keys = []
- i = 0
- if metadata is not None:
- for item in metadata:
- # sqlite possibly prepending table name to colnames so strip
- colname = item[0].split('.')[-1].lower()
- if typemap is not None:
- rec = (typemap.get(colname, types.NULLTYPE), i)
- else:
- rec = (types.NULLTYPE, i)
- if rec[0] is None:
- raise DBAPIError("None for metadata " + colname)
- if self.props.setdefault(colname, rec) is not rec:
- self.props[colname] = (ResultProxy.AmbiguousColumn(colname), 0)
- self.keys.append(colname)
- #print "COLNAME", colname
- self.props[i] = rec
- i+=1
-
- def _get_col(self, row, key):
- if isinstance(key, schema.Column) or isinstance(key, sql.ColumnElement):
- try:
- rec = self.props[key._label.lower()]
- #print "GOT IT FROM LABEL FOR ", key._label
- except KeyError:
- try:
- rec = self.props[key.key.lower()]
- except KeyError:
- rec = self.props[key.name.lower()]
- elif isinstance(key, str):
- rec = self.props[key.lower()]
- else:
- rec = self.props[key]
- return rec[0].engine_impl(self.engine).convert_result_value(row[rec[1]], self.engine)
-
- def __iter__(self):
- while True:
- row = self.fetchone()
- if row is None:
- raise StopIteration
- else:
- yield row
-
- def last_inserted_ids(self):
- return self.engine.last_inserted_ids()
- def last_updated_params(self):
- return self.engine.last_updated_params()
- def last_inserted_params(self):
- return self.engine.last_inserted_params()
- def lastrow_has_defaults(self):
- return self.engine.lastrow_has_defaults()
- def supports_sane_rowcount(self):
- return self.engine.supports_sane_rowcount()
-
- def fetchall(self):
- """fetches all rows, just like DBAPI cursor.fetchall()."""
- l = []
- while True:
- v = self.fetchone()
- if v is None:
- return l
- l.append(v)
-
- def fetchone(self):
- """fetches one row, just like DBAPI cursor.fetchone()."""
- row = self.cursor.fetchone()
- if row is not None:
- if self.echo: self.engine.log(repr(row))
- return RowProxy(self, row)
- else:
- return None
-
-class RowProxy:
- """proxies a single cursor row for a parent ResultProxy."""
- def __init__(self, parent, row):
- """RowProxy objects are constructed by ResultProxy objects."""
- self.__parent = parent
- self.__row = row
- def __iter__(self):
- for i in range(0, len(self.__row)):
- yield self.__parent._get_col(self.__row, i)
- def __eq__(self, other):
- return (other is self) or (other == tuple([self.__parent._get_col(self.__row, key) for key in range(0, len(self.__row))]))
- def __repr__(self):
- return repr(tuple([self.__parent._get_col(self.__row, key) for key in range(0, len(self.__row))]))
- def __getitem__(self, key):
- return self.__parent._get_col(self.__row, key)
- def __getattr__(self, name):
- try:
- return self.__parent._get_col(self.__row, name)
- except KeyError:
- raise AttributeError
- def items(self):
- return [(key, getattr(self, key)) for key in self.keys()]
- def keys(self):
- return self.__parent.keys
- def values(self):
- return list(self)
- def __len__(self):
- return len(self.__row)
--- /dev/null
+# engine/__init__.py
+# Copyright (C) 2005,2006 Michael Bayer mike_mp@zzzcomputing.com
+#
+# This module is part of SQLAlchemy and is released under
+# the MIT License: http://www.opensource.org/licenses/mit-license.php
+
+import sqlalchemy.databases
+
+from base import *
+import strategies
+import re
+
+def engine_descriptors():
+ """provides a listing of all the database implementations supported. this data
+ is provided as a list of dictionaries, where each dictionary contains the following
+ key/value pairs:
+
+ name : the name of the engine, suitable for use in the create_engine function
+
+ description: a plain description of the engine.
+
+ arguments : a dictionary describing the name and description of each parameter
+ used to connect to this engine's underlying DBAPI.
+
+ This function is meant for usage in automated configuration tools that wish to
+ query the user for database and connection information.
+ """
+ result = []
+ #for module in sqlalchemy.databases.__all__:
+ for module in ['sqlite', 'postgres', 'mysql']:
+ module = getattr(__import__('sqlalchemy.databases.%s' % module).databases, module)
+ result.append(module.descriptor())
+ return result
+
+default_strategy = 'plain'
+def create_engine(*args, **kwargs):
+ """creates a new Engine instance. Using the given strategy name,
+ locates that strategy and invokes its create() method to produce the Engine.
+ The strategies themselves are instances of EngineStrategy, and the built in
+ ones are present in the sqlalchemy.engine.strategies module. Current implementations
+ include "plain" and "threadlocal". The default used by this function is "threadlocal".
+
+ "plain" provides support for a Connection object which can be used to execute SQL queries
+ with a specific underlying DBAPI connection.
+
+ "threadlocal" is similar to "plain" except that it adds support for a thread-local connection and
+ transaction context, which allows a group of engine operations to participate using the same
+ connection and transaction without the need for explicit passing of a Connection object.
+
+ The standard method of specifying the engine is via URL as the first positional
+ argument, to indicate the appropriate database dialect and connection arguments, with additional
+ keyword arguments sent as options to the dialect and resulting Engine.
+
+ The URL is in the form <dialect>://opt1=val1&opt2=val2.
+ Where <dialect> is a name such as "mysql", "oracle", "postgres", and the options indicate
+ username, password, database, etc. Supported keynames include "username", "user", "password",
+ "pw", "db", "database", "host", "filename".
+
+ **kwargs represents options to be sent to the Engine itself as well as the components of the Engine,
+ including the Dialect, the ConnectionProvider, and the Pool. A list of common options is as follows:
+
+ pool=None : an instance of sqlalchemy.pool.DBProxy or sqlalchemy.pool.Pool to be used as the
+ underlying source for connections (DBProxy/Pool is described in the previous section). If None,
+ a default DBProxy will be created using the engine's own database module with the given
+ arguments.
+
+ echo=False : if True, the Engine will log all statements as well as a repr() of their
+ parameter lists to the engines logger, which defaults to sys.stdout. A Engine instances'
+ "echo" data member can be modified at any time to turn logging on and off. If set to the string
+ 'debug', result rows will be printed to the standard output as well.
+
+ logger=None : a file-like object where logging output can be sent, if echo is set to True.
+ This defaults to sys.stdout.
+
+ encoding='utf-8' : the encoding to be used when encoding/decoding Unicode strings
+
+ convert_unicode=False : True if unicode conversion should be applied to all str types
+
+ module=None : used by Oracle and Postgres, this is a reference to a DBAPI2 module to be used
+ instead of the engine's default module. For Postgres, the default is psycopg2, or psycopg1 if
+ 2 cannot be found. For Oracle, its cx_Oracle. For mysql, MySQLdb.
+
+ use_ansi=True : used only by Oracle; when False, the Oracle driver attempts to support a
+ particular "quirk" of some Oracle databases, that the LEFT OUTER JOIN SQL syntax is not
+ supported, and the "Oracle join" syntax of using <column1>(+)=<column2> must be used
+ in order to achieve a LEFT OUTER JOIN. Its advised that the Oracle database be configured to
+ have full ANSI support instead of using this feature.
+
+ """
+ strategy = kwargs.pop('strategy', default_strategy)
+ strategy = strategies.strategies[strategy]
+ return strategy.create(*args, **kwargs)
--- /dev/null
+from sqlalchemy import exceptions, sql, schema, util, types
+import StringIO, sys, re
+
+class ConnectionProvider(object):
+ """defines an interface that returns raw Connection objects (or compatible)."""
+ def get_connection(self):
+ """this method should return a Connection or compatible object from a DBAPI which
+ also contains a close() method.
+ It is not defined what context this connection belongs to. It may be newly connected,
+ returned from a pool, part of some other kind of context such as thread-local,
+ or can be a fixed member of this object."""
+ raise NotImplementedError()
+ def dispose(self):
+ """releases all resources corresponding to this ConnectionProvider, such
+ as any underlying connection pools."""
+ raise NotImplementedError()
+
+class Dialect(sql.AbstractDialect):
+ """Adds behavior to the execution of queries to provide
+ support for column defaults, differences between paramstyles, quirks between post-execution behavior,
+ and a general consistentization of the behavior of various DBAPIs.
+
+ The Dialect should also implement the following two attributes:
+
+ positional - True if the paramstyle for this Dialect is positional
+
+ paramstyle - the paramstyle to be used (some DBAPIs support multiple paramstyles)
+
+ supports_autoclose_results - usually True; if False, indicates that rows returned by fetchone()
+ might not be just plain tuples, and may be "live" proxy objects which still require the cursor
+ to be open in order to be read (such as pyPgSQL which has active filehandles for BLOBs). in that
+ case, an auto-closing ResultProxy cannot automatically close itself after results are consumed.
+
+ convert_unicode - True if unicode conversion should be applied to all str types
+
+ encoding - type of encoding to use for unicode, usually defaults to 'utf-8'
+ """
+ def create_connect_args(self, opts):
+ """given a dictionary of key-valued connect parameters, returns a tuple
+ consisting of a *args/**kwargs suitable to send directly to the dbapi's connect function.
+ The connect args will have any number of the following keynames: host, hostname, database, dbanme,
+ user,username, password, pw, passwd, filename."""
+ raise NotImplementedError()
+ def convert_compiled_params(self, parameters):
+ """given a sql.ClauseParameters object, returns an array or dictionary suitable to pass
+ directly to this Dialect's DBAPI's execute method."""
+ def type_descriptor(self, typeobj):
+ """provides a database-specific TypeEngine object, given the generic object
+ which comes from the types module. Subclasses will usually use the adapt_type()
+ method in the types module to make this job easy."""
+ raise NotImplementedError()
+ def oid_column_name(self):
+ """returns the oid column name for this dialect, or None if the dialect cant/wont support OID/ROWID."""
+ raise NotImplementedError()
+ def supports_sane_rowcount(self):
+ """Provided to indicate when MySQL is being used, which does not have standard behavior
+ for the "rowcount" function on a statement handle. """
+ raise NotImplementedError()
+ def schemagenerator(self, engine, proxy, **params):
+ """returns a schema.SchemaVisitor instance that can generate schemas, when it is
+ invoked to traverse a set of schema objects.
+
+ schemagenerator is called via the create() method on Table, Index, and others.
+ """
+ raise NotImplementedError()
+ def schemadropper(self, engine, proxy, **params):
+ """returns a schema.SchemaVisitor instance that can drop schemas, when it is
+ invoked to traverse a set of schema objects.
+
+ schemagenerator is called via the drop() method on Table, Index, and others.
+ """
+ raise NotImplementedError()
+ def defaultrunner(self, engine, proxy, **params):
+ """returns a schema.SchemaVisitor instances that can execute defaults."""
+ raise NotImplementedError()
+ def compiler(self, statement, parameters):
+ """returns a sql.ClauseVisitor which will produce a string representation of the given
+ ClauseElement and parameter dictionary. This object is usually a subclass of
+ ansisql.ANSICompiler.
+
+ compiler is called within the context of the compile() method."""
+ raise NotImplementedError()
+ def reflecttable(self, connection, table):
+ """given an Connection and a Table object, reflects its columns and properties from the database."""
+ raise NotImplementedError()
+ def has_table(self, connection, table_name):
+ raise NotImplementedError()
+ def dbapi(self):
+ """subclasses override this method to provide the DBAPI module used to establish
+ connections."""
+ raise NotImplementedError()
+ def get_default_schema_name(self, connection):
+ """returns the currently selected schema given an connection"""
+ raise NotImplementedError()
+ def execution_context(self):
+ """returns a new ExecutionContext object."""
+ raise NotImplementedError()
+ def do_begin(self, connection):
+ """provides an implementation of connection.begin()"""
+ raise NotImplementedError()
+ def do_rollback(self, connection):
+ """provides an implementation of connection.rollback()"""
+ raise NotImplementedError()
+ def do_commit(self, connection):
+ """provides an implementation of connection.commit()"""
+ raise NotImplementedError()
+ def do_executemany(self, cursor, statement, parameters):
+ raise NotImplementedError()
+ def do_execute(self, cursor, statement, parameters):
+ raise NotImplementedError()
+
+class ExecutionContext(object):
+ """a messenger object for a Dialect that corresponds to a single execution. The Dialect
+ should provide an ExecutionContext via the create_execution_context() method.
+ The pre_exec and post_exec methods will be called for compiled statements, afterwhich
+ it is expected that the various methods last_inserted_ids, last_inserted_params, etc.
+ will contain appropriate values, if applicable."""
+ def pre_exec(self, engine, proxy, compiled, parameters):
+ """called before an execution of a compiled statement. proxy is a callable that
+ takes a string statement and a bind parameter list/dictionary."""
+ raise NotImplementedError()
+ def post_exec(self, engine, proxy, compiled, parameters):
+ """called after the execution of a compiled statement. proxy is a callable that
+ takes a string statement and a bind parameter list/dictionary."""
+ raise NotImplementedError()
+ def get_rowcount(self, cursor):
+ """returns the count of rows updated/deleted for an UPDATE/DELETE statement"""
+ raise NotImplementedError()
+ def supports_sane_rowcount(self):
+ """Provided to indicate when MySQL is being used, which does not have standard behavior
+ for the "rowcount" function on a statement handle. """
+ raise NotImplementedError()
+ def last_inserted_ids(self):
+ """returns the list of the primary key values for the last insert statement executed.
+ This does not apply to straight textual clauses; only to sql.Insert objects compiled against
+ a schema.Table object, which are executed via statement.execute(). The order of items in the
+ list is the same as that of the Table's 'primary_key' attribute.
+
+ In some cases, this method may invoke a query back to the database to retrieve the data, based on
+ the "lastrowid" value in the cursor."""
+ raise NotImplementedError()
+ def last_inserted_params(self):
+ """returns a dictionary of the full parameter dictionary for the last compiled INSERT statement,
+ including any ColumnDefaults or Sequences that were pre-executed. this value is thread-local."""
+ raise NotImplementedError()
+ def last_updated_params(self):
+ """returns a dictionary of the full parameter dictionary for the last compiled UPDATE statement,
+ including any ColumnDefaults that were pre-executed. this value is thread-local."""
+ raise NotImplementedError()
+ def lastrow_has_defaults(self):
+ """returns True if the last row INSERTED via a compiled insert statement contained PassiveDefaults,
+ indicating that the database inserted data beyond that which we gave it. this value is thread-local."""
+ raise NotImplementedError()
+
+class Connectable(object):
+ """interface for an object that can provide an Engine and a Connection object which correponds to that Engine."""
+ def contextual_connect(self):
+ """returns a Connection object which may be part of an ongoing context."""
+ raise NotImplementedError()
+ def create(self, entity, **kwargs):
+ """creates a table or index given an appropriate schema object."""
+ raise NotImplementedError()
+ def drop(self, entity, **kwargs):
+ raise NotImplementedError()
+ def execute(self, object, *multiparams, **params):
+ raise NotImplementedError()
+ def _not_impl(self):
+ raise NotImplementedError()
+ engine = property(_not_impl, doc="returns the Engine which this Connectable is associated with.")
+
+class Connection(Connectable):
+ """represents a single DBAPI connection returned from the underlying connection pool. Provides
+ execution support for string-based SQL statements as well as ClauseElement, Compiled and DefaultGenerator objects.
+ provides a begin method to return Transaction objects."""
+ def __init__(self, engine, connection=None, close_with_result=False):
+ self.__engine = engine
+ self.__connection = connection or engine.raw_connection()
+ self.__transaction = None
+ self.__close_with_result = close_with_result
+ engine = property(lambda s:s.__engine, doc="The Engine with which this Connection is associated (read only)")
+ connection = property(lambda s:s.__connection, doc="The underlying DBAPI connection managed by this Connection.")
+ should_close_with_result = property(lambda s:s.__close_with_result, doc="Indicates if this Connection should be closed when a corresponding ResultProxy is closed; this is essentially an auto-release mode.")
+ def _create_transaction(self, parent):
+ return Transaction(self, parent)
+ def connect(self):
+ """connect() is implemented to return self so that an incoming Engine or Connection object can be treated similarly."""
+ return self
+ def contextual_connect(self, **kwargs):
+ """contextual_connect() is implemented to return self so that an incoming Engine or Connection object can be treated similarly."""
+ return self
+ def begin(self):
+ if self.__transaction is None:
+ self.__transaction = self._create_transaction(None)
+ return self.__transaction
+ else:
+ return self._create_transaction(self.__transaction)
+ def _begin_impl(self):
+ if self.__engine.echo:
+ self.__engine.log("BEGIN")
+ self.__engine.dialect.do_begin(self.__connection)
+ def _rollback_impl(self):
+ if self.__engine.echo:
+ self.__engine.log("ROLLBACK")
+ self.__engine.dialect.do_rollback(self.__connection)
+ def _commit_impl(self):
+ if self.__engine.echo:
+ self.__engine.log("COMMIT")
+ self.__engine.dialect.do_commit(self.__connection)
+ def _autocommit(self, statement):
+ """when no Transaction is present, this is called after executions to provide "autocommit" behavior."""
+ # TODO: have the dialect determine if autocommit can be set on the connection directly without this
+ # extra step
+ if self.__transaction is None and re.match(r'UPDATE|INSERT|CREATE|DELETE|DROP', statement.lstrip().upper()):
+ self._commit_impl()
+ def close(self):
+ if self.__connection is not None:
+ self.__connection.close()
+ self.__connection = None
+ def scalar(self, object, parameters, **kwargs):
+ row = self.execute(object, parameters, **kwargs).fetchone()
+ if row is not None:
+ return row[0]
+ else:
+ return None
+ def execute(self, object, *multiparams, **params):
+ return Connection.executors[type(object).__mro__[-2]](self, object, *multiparams, **params)
+ def execute_default(self, default, **kwargs):
+ return default.accept_schema_visitor(self.__engine.dialect.defaultrunner(self.__engine, self.proxy, **kwargs))
+ def execute_text(self, statement, parameters=None):
+ cursor = self._execute_raw(statement, parameters)
+ return ResultProxy(self.__engine, self, cursor)
+ def _params_to_listofdicts(self, *multiparams, **params):
+ if len(multiparams) == 0:
+ return [params]
+ elif len(multiparams) == 1:
+ if multiparams[0] == None:
+ return [{}]
+ elif isinstance (multiparams[0], list) or isinstance (multiparams[0], tuple):
+ return multiparams[0]
+ else:
+ return [multiparams[0]]
+ else:
+ return multiparams
+ def execute_clauseelement(self, elem, *multiparams, **params):
+ executemany = len(multiparams) > 0
+ if executemany:
+ param = multiparams[0]
+ else:
+ param = params
+ return self.execute_compiled(elem.compile(engine=self.__engine, parameters=param), *multiparams, **params)
+ def execute_compiled(self, compiled, *multiparams, **params):
+ """executes a sql.Compiled object."""
+ cursor = self.__connection.cursor()
+ parameters = [compiled.get_params(**m) for m in self._params_to_listofdicts(*multiparams, **params)]
+ if len(parameters) == 1:
+ parameters = parameters[0]
+ def proxy(statement=None, parameters=None):
+ if statement is None:
+ return cursor
+
+ parameters = self.__engine.dialect.convert_compiled_params(parameters)
+ self._execute_raw(statement, parameters, cursor=cursor, context=context)
+ return cursor
+ context = self.__engine.dialect.create_execution_context()
+ context.pre_exec(self.__engine, proxy, compiled, parameters)
+ proxy(str(compiled), parameters)
+ context.post_exec(self.__engine, proxy, compiled, parameters)
+ return ResultProxy(self.__engine, self, cursor, context, typemap=compiled.typemap)
+
+ # poor man's multimethod/generic function thingy
+ executors = {
+ sql.ClauseElement : execute_clauseelement,
+ sql.Compiled : execute_compiled,
+ schema.SchemaItem:execute_default,
+ str.__mro__[-2] : execute_text
+ }
+
+ def create(self, entity, **kwargs):
+ """creates a table or index given an appropriate schema object."""
+ return self.__engine.create(entity, connection=self, **kwargs)
+ def drop(self, entity, **kwargs):
+ """drops a table or index given an appropriate schema object."""
+ return self.__engine.drop(entity, connection=self, **kwargs)
+ def reflecttable(self, table, **kwargs):
+ """reflects the columns in the given table from the database."""
+ return self.__engine.reflecttable(table, connection=self, **kwargs)
+ def default_schema_name(self):
+ return self.__engine.dialect.get_default_schema_name(self)
+ def run_callable(self, callable_):
+ callable_(self)
+ def _execute_raw(self, statement, parameters=None, cursor=None, echo=None, context=None, **kwargs):
+ if cursor is None:
+ cursor = self.__connection.cursor()
+ try:
+ if echo is True or self.__engine.echo is not False:
+ self.__engine.log(statement)
+ self.__engine.log(repr(parameters))
+ if parameters is not None and isinstance(parameters, list) and len(parameters) > 0 and (isinstance(parameters[0], list) or isinstance(parameters[0], dict)):
+ self._executemany(cursor, statement, parameters, context=context)
+ else:
+ self._execute(cursor, statement, parameters, context=context)
+ self._autocommit(statement)
+ except:
+ raise
+ return cursor
+
+ def _execute(self, c, statement, parameters, context=None):
+ if parameters is None:
+ if self.__engine.dialect.positional:
+ parameters = ()
+ else:
+ parameters = {}
+ try:
+ self.__engine.dialect.do_execute(c, statement, parameters, context=context)
+ except Exception, e:
+ self._rollback_impl()
+ if self.__close_with_result:
+ self.close()
+ raise exceptions.SQLError(statement, parameters, e)
+ def _executemany(self, c, statement, parameters, context=None):
+ try:
+ self.__engine.dialect.do_executemany(c, statement, parameters, context=context)
+ except Exception, e:
+ self._rollback_impl()
+ if self.__close_with_result:
+ self.close()
+ raise exceptions.SQLError(statement, parameters, e)
+ def proxy(self, statement=None, parameters=None):
+ """executes the given statement string and parameter object.
+ the parameter object is expected to be the result of a call to compiled.get_params().
+ This callable is a generic version of a connection/cursor-specific callable that
+ is produced within the execute_compiled method, and is used for objects that require
+ this style of proxy when outside of an execute_compiled method, primarily the DefaultRunner."""
+ parameters = self.__engine.dialect.convert_compiled_params(parameters)
+ return self._execute_raw(statement, parameters)
+
+class Transaction(object):
+ """represents a Transaction in progress"""
+ def __init__(self, connection, parent):
+ self.__connection = connection
+ self.__parent = parent or self
+ self.__is_active = True
+ if self.__parent is self:
+ self.__connection._begin_impl()
+ connection = property(lambda s:s.__connection, doc="The Connection object referenced by this Transaction")
+ def rollback(self):
+ if not self.__parent.__is_active:
+ raise exceptions.InvalidRequestError("This transaction is inactive")
+ if self.__parent is self:
+ self.__connection._rollback_impl()
+ self.__is_active = False
+ else:
+ self.__parent.rollback()
+ def commit(self):
+ if not self.__parent.__is_active:
+ raise exceptions.InvalidRequestError("This transaction is inactive")
+ if self.__parent is self:
+ self.__connection._commit_impl()
+ self.__is_active = False
+
+class ComposedSQLEngine(sql.Engine, Connectable):
+ """
+ Connects a ConnectionProvider, a Dialect and a CompilerFactory together to
+ provide a default implementation of SchemaEngine.
+ """
+ def __init__(self, connection_provider, dialect, echo=False, logger=None, **kwargs):
+ self.connection_provider = connection_provider
+ self.dialect=dialect
+ self.echo = echo
+ self.logger = logger or util.Logger(origin='engine')
+
+ name = property(lambda s:sys.modules[s.dialect.__module__].descriptor()['name'])
+ engine = property(lambda s:s)
+
+ def dispose(self):
+ self.connection_provider.dispose()
+ def create(self, entity, connection=None, **kwargs):
+ """creates a table or index within this engine's database connection given a schema.Table object."""
+ self._run_visitor(self.dialect.schemagenerator, entity, connection=connection, **kwargs)
+ def drop(self, entity, connection=None, **kwargs):
+ """drops a table or index within this engine's database connection given a schema.Table object."""
+ self._run_visitor(self.dialect.schemadropper, entity, connection=connection, **kwargs)
+ def execute_default(self, default, **kwargs):
+ connection = self.contextual_connect()
+ try:
+ return connection.execute_default(default, **kwargs)
+ finally:
+ connection.close()
+
+ def _func(self):
+ return sql.FunctionGenerator(self)
+ func = property(_func)
+ def text(self, text, *args, **kwargs):
+ """returns a sql.text() object for performing literal queries."""
+ return sql.text(text, engine=self, *args, **kwargs)
+
+ def _run_visitor(self, visitorcallable, element, connection=None, **kwargs):
+ if connection is None:
+ conn = self.contextual_connect()
+ else:
+ conn = connection
+ try:
+ element.accept_schema_visitor(visitorcallable(self, conn.proxy, **kwargs))
+ finally:
+ if connection is None:
+ conn.close()
+
+ def transaction(self, callable_, connection=None, *args, **kwargs):
+ if connection is None:
+ conn = self.contextual_connect()
+ else:
+ conn = connection
+ try:
+ trans = conn.begin()
+ try:
+ ret = callable_(conn, *args, **kwargs)
+ trans.commit()
+ return ret
+ except:
+ trans.rollback()
+ raise
+ finally:
+ if connection is None:
+ conn.close()
+
+ def run_callable(self, callable_, connection=None, *args, **kwargs):
+ if connection is None:
+ conn = self.contextual_connect()
+ else:
+ conn = connection
+ try:
+ return callable_(conn, *args, **kwargs)
+ finally:
+ if connection is None:
+ conn.close()
+
+ def execute(self, statement, *multiparams, **params):
+ connection = self.contextual_connect(close_with_result=True)
+ return connection.execute(statement, *multiparams, **params)
+
+ def execute_compiled(self, compiled, *multiparams, **params):
+ connection = self.contextual_connect(close_with_result=True)
+ return connection.execute_compiled(compiled, *multiparams, **params)
+
+ def compiler(self, statement, parameters, **kwargs):
+ return self.dialect.compiler(statement, parameters, engine=self, **kwargs)
+
+ def connect(self, **kwargs):
+ """returns a newly allocated Connection object."""
+ return Connection(self, **kwargs)
+
+ def contextual_connect(self, close_with_result=False, **kwargs):
+ """returns a Connection object which may be newly allocated, or may be part of some
+ ongoing context. This Connection is meant to be used by the various "auto-connecting" operations."""
+ return Connection(self, close_with_result=close_with_result, **kwargs)
+
+ def reflecttable(self, table, connection=None):
+ """given a Table object, reflects its columns and properties from the database."""
+ if connection is None:
+ conn = self.contextual_connect()
+ else:
+ conn = connection
+ try:
+ self.dialect.reflecttable(conn, table)
+ finally:
+ if connection is None:
+ conn.close()
+ def has_table(self, table_name):
+ return self.run_callable(lambda c: self.dialect.has_table(c, table_name))
+
+ def raw_connection(self):
+ """returns a DBAPI connection."""
+ return self.connection_provider.get_connection()
+
+ def log(self, msg):
+ """logs a message using this SQLEngine's logger stream."""
+ self.logger.write(msg)
+
+class ResultProxy:
+ """wraps a DBAPI cursor object to provide access to row columns based on integer
+ position, case-insensitive column name, or by schema.Column object. e.g.:
+
+ row = fetchone()
+
+ col1 = row[0] # access via integer position
+
+ col2 = row['col2'] # access via name
+
+ col3 = row[mytable.c.mycol] # access via Column object.
+
+ ResultProxy also contains a map of TypeEngine objects and will invoke the appropriate
+ convert_result_value() method before returning columns.
+ """
+ class AmbiguousColumn(object):
+ def __init__(self, key):
+ self.key = key
+ def convert_result_value(self, arg, engine):
+ raise InvalidRequestError("Ambiguous column name '%s' in result set! try 'use_labels' option on select statement." % (self.key))
+
+ def __init__(self, engine, connection, cursor, executioncontext=None, typemap=None):
+ """ResultProxy objects are constructed via the execute() method on SQLEngine."""
+ self.connection = connection
+ self.dialect = engine.dialect
+ self.cursor = cursor
+ self.engine = engine
+ self.closed = False
+ self.executioncontext = executioncontext
+ self.echo = engine.echo=="debug"
+ if executioncontext:
+ self.rowcount = executioncontext.get_rowcount(cursor)
+ else:
+ self.rowcount = cursor.rowcount
+ metadata = cursor.description
+ self.props = {}
+ self.keys = []
+ i = 0
+ if metadata is not None:
+ for item in metadata:
+ # sqlite possibly prepending table name to colnames so strip
+ colname = item[0].split('.')[-1].lower()
+ if typemap is not None:
+ rec = (typemap.get(colname, types.NULLTYPE), i)
+ else:
+ rec = (types.NULLTYPE, i)
+ if rec[0] is None:
+ raise DBAPIError("None for metadata " + colname)
+ if self.props.setdefault(colname, rec) is not rec:
+ self.props[colname] = (ResultProxy.AmbiguousColumn(colname), 0)
+ self.keys.append(colname)
+ self.props[i] = rec
+ i+=1
+ def close(self):
+ if not self.closed:
+ self.closed = True
+ if self.connection.should_close_with_result and self.dialect.supports_autoclose_results:
+ self.connection.close()
+ def _get_col(self, row, key):
+ if isinstance(key, sql.ColumnElement):
+ try:
+ rec = self.props[key._label.lower()]
+ except KeyError:
+ try:
+ rec = self.props[key.key.lower()]
+ except KeyError:
+ rec = self.props[key.name.lower()]
+ elif isinstance(key, str):
+ rec = self.props[key.lower()]
+ else:
+ rec = self.props[key]
+ return rec[0].dialect_impl(self.dialect).convert_result_value(row[rec[1]], self.dialect)
+
+ def __iter__(self):
+ while True:
+ row = self.fetchone()
+ if row is None:
+ raise StopIteration
+ else:
+ yield row
+
+ def last_inserted_ids(self):
+ return self.executioncontext.last_inserted_ids()
+ def last_updated_params(self):
+ return self.executioncontext.last_updated_params()
+ def last_inserted_params(self):
+ return self.executioncontext.last_inserted_params()
+ def lastrow_has_defaults(self):
+ return self.executioncontext.lastrow_has_defaults()
+ def supports_sane_rowcount(self):
+ return self.executioncontext.supports_sane_rowcount()
+
+ def fetchall(self):
+ """fetches all rows, just like DBAPI cursor.fetchall()."""
+ l = []
+ while True:
+ v = self.fetchone()
+ if v is None:
+ return l
+ l.append(v)
+
+ def fetchone(self):
+ """fetches one row, just like DBAPI cursor.fetchone()."""
+ row = self.cursor.fetchone()
+ if row is not None:
+ if self.echo: self.engine.log(repr(row))
+ return RowProxy(self, row)
+ else:
+ # controversy! can we auto-close the cursor after results are consumed ?
+ # what if the returned rows are still hanging around, and are "live" objects
+ # and not just plain tuples ?
+ self.close()
+ return None
+
+class RowProxy:
+ """proxies a single cursor row for a parent ResultProxy."""
+ def __init__(self, parent, row):
+ """RowProxy objects are constructed by ResultProxy objects."""
+ self.__parent = parent
+ self.__row = row
+ def close(self):
+ self.__parent.close()
+ def __iter__(self):
+ for i in range(0, len(self.__row)):
+ yield self.__parent._get_col(self.__row, i)
+ def __eq__(self, other):
+ return (other is self) or (other == tuple([self.__parent._get_col(self.__row, key) for key in range(0, len(self.__row))]))
+ def __repr__(self):
+ return repr(tuple([self.__parent._get_col(self.__row, key) for key in range(0, len(self.__row))]))
+ def __getitem__(self, key):
+ return self.__parent._get_col(self.__row, key)
+ def __getattr__(self, name):
+ try:
+ return self.__parent._get_col(self.__row, name)
+ except KeyError:
+ raise AttributeError
+ def items(self):
+ return [(key, getattr(self, key)) for key in self.keys()]
+ def keys(self):
+ return self.__parent.keys
+ def values(self):
+ return list(self)
+ def __len__(self):
+ return len(self.__row)
+
+class SchemaIterator(schema.SchemaVisitor):
+ """a visitor that can gather text into a buffer and execute the contents of the buffer."""
+ def __init__(self, engine, proxy, **params):
+ self.proxy = proxy
+ self.engine = engine
+ self.buffer = StringIO.StringIO()
+
+ def append(self, s):
+ """appends content to the SchemaIterator's query buffer."""
+ self.buffer.write(s)
+
+ def execute(self):
+ """executes the contents of the SchemaIterator's buffer using its sql proxy and
+ clears out the buffer."""
+ try:
+ return self.proxy(self.buffer.getvalue(), None)
+ finally:
+ self.buffer.truncate(0)
+
+class DefaultRunner(schema.SchemaVisitor):
+ def __init__(self, engine, proxy):
+ self.proxy = proxy
+ self.engine = engine
+
+ def get_column_default(self, column):
+ if column.default is not None:
+ return column.default.accept_schema_visitor(self)
+ else:
+ return None
+
+ def get_column_onupdate(self, column):
+ if column.onupdate is not None:
+ return column.onupdate.accept_schema_visitor(self)
+ else:
+ return None
+
+ def visit_passive_default(self, default):
+ """passive defaults by definition return None on the app side,
+ and are post-fetched to get the DB-side value"""
+ return None
+
+ def visit_sequence(self, seq):
+ """sequences are not supported by default"""
+ return None
+
+ def exec_default_sql(self, default):
+ c = sql.select([default.arg], engine=self.engine).compile()
+ return self.proxy(str(c), c.get_params()).fetchone()[0]
+
+ def visit_column_onupdate(self, onupdate):
+ if isinstance(onupdate.arg, sql.ClauseElement):
+ return self.exec_default_sql(onupdate)
+ elif callable(onupdate.arg):
+ return onupdate.arg()
+ else:
+ return onupdate.arg
+
+ def visit_column_default(self, default):
+ if isinstance(default.arg, sql.ClauseElement):
+ return self.exec_default_sql(default)
+ elif callable(default.arg):
+ return default.arg()
+ else:
+ return default.arg
--- /dev/null
+# engine/default.py
+# Copyright (C) 2005,2006 Michael Bayer mike_mp@zzzcomputing.com
+#
+# This module is part of SQLAlchemy and is released under
+# the MIT License: http://www.opensource.org/licenses/mit-license.php
+
+
+from sqlalchemy import schema, exceptions, util, sql, types
+import sqlalchemy.pool
+import StringIO, sys, re
+import base
+
+"""provides default implementations of the engine interfaces"""
+
+
+class PoolConnectionProvider(base.ConnectionProvider):
+ def __init__(self, dialect, url, poolclass=None, pool=None, **kwargs):
+ (cargs, cparams) = dialect.create_connect_args(url)
+ if pool is None:
+ kwargs.setdefault('echo', False)
+ kwargs.setdefault('use_threadlocal',True)
+ if poolclass is None:
+ poolclass = sqlalchemy.pool.QueuePool
+ dbapi = dialect.dbapi()
+ if dbapi is None:
+ raise exceptions.InvalidRequestException("Cant get DBAPI module for dialect '%s'" % dialect)
+ self._pool = poolclass(lambda: dbapi.connect(*cargs, **cparams), **kwargs)
+ else:
+ if isinstance(pool, sqlalchemy.pool.DBProxy):
+ self._pool = pool.get_pool(*cargs, **cparams)
+ else:
+ self._pool = pool
+ def get_connection(self):
+ return self._pool.connect()
+ def dispose(self):
+ self._pool.dispose()
+ if hasattr(self, '_dbproxy'):
+ self._dbproxy.dispose()
+
+class DefaultDialect(base.Dialect):
+ """default implementation of Dialect"""
+ def __init__(self, convert_unicode=False, encoding='utf-8', **kwargs):
+ self.convert_unicode = convert_unicode
+ self.supports_autoclose_results = True
+ self.encoding = encoding
+ self.positional = False
+ self.paramstyle = 'named'
+ self._ischema = None
+ self._figure_paramstyle()
+ def create_execution_context(self):
+ return DefaultExecutionContext(self)
+ def type_descriptor(self, typeobj):
+ """provides a database-specific TypeEngine object, given the generic object
+ which comes from the types module. Subclasses will usually use the adapt_type()
+ method in the types module to make this job easy."""
+ if type(typeobj) is type:
+ typeobj = typeobj()
+ return typeobj
+ def oid_column_name(self):
+ return None
+ def supports_sane_rowcount(self):
+ return True
+ def do_begin(self, connection):
+ """implementations might want to put logic here for turning autocommit on/off,
+ etc."""
+ pass
+ def do_rollback(self, connection):
+ """implementations might want to put logic here for turning autocommit on/off,
+ etc."""
+ #print "ENGINE ROLLBACK ON ", connection.connection
+ connection.rollback()
+ def do_commit(self, connection):
+ """implementations might want to put logic here for turning autocommit on/off, etc."""
+ #print "ENGINE COMMIT ON ", connection.connection
+ connection.commit()
+ def do_executemany(self, cursor, statement, parameters, **kwargs):
+ cursor.executemany(statement, parameters)
+ def do_execute(self, cursor, statement, parameters, **kwargs):
+ cursor.execute(statement, parameters)
+ def defaultrunner(self, engine, proxy):
+ return base.DefaultRunner(engine, proxy)
+
+ def _set_paramstyle(self, style):
+ self._paramstyle = style
+ self._figure_paramstyle(style)
+ paramstyle = property(lambda s:s._paramstyle, _set_paramstyle)
+
+ def convert_compiled_params(self, parameters):
+ executemany = parameters is not None and isinstance(parameters, list)
+ # the bind params are a CompiledParams object. but all the DBAPI's hate
+ # that object (or similar). so convert it to a clean
+ # dictionary/list/tuple of dictionary/tuple of list
+ if parameters is not None:
+ if self.positional:
+ if executemany:
+ parameters = [p.values() for p in parameters]
+ else:
+ parameters = parameters.values()
+ else:
+ if executemany:
+ parameters = [p.get_raw_dict() for p in parameters]
+ else:
+ parameters = parameters.get_raw_dict()
+ return parameters
+
+ def _figure_paramstyle(self, paramstyle=None):
+ db = self.dbapi()
+ if paramstyle is not None:
+ self._paramstyle = paramstyle
+ elif db is not None:
+ self._paramstyle = db.paramstyle
+ else:
+ self._paramstyle = 'named'
+
+ if self._paramstyle == 'named':
+ self.positional=False
+ elif self._paramstyle == 'pyformat':
+ self.positional=False
+ elif self._paramstyle == 'qmark' or self._paramstyle == 'format' or self._paramstyle == 'numeric':
+ # for positional, use pyformat internally, ANSICompiler will convert
+ # to appropriate character upon compilation
+ self.positional = True
+ else:
+ raise DBAPIError("Unsupported paramstyle '%s'" % self._paramstyle)
+
+ def _get_ischema(self):
+ # We use a property for ischema so that the accessor
+ # creation only happens as needed, since otherwise we
+ # have a circularity problem with the generic
+ # ansisql.engine()
+ if self._ischema is None:
+ import sqlalchemy.databases.information_schema as ischema
+ self._ischema = ischema.ISchema(self)
+ return self._ischema
+ ischema = property(_get_ischema, doc="""returns an ISchema object for this engine, which allows access to information_schema tables (if supported)""")
+
+class DefaultExecutionContext(base.ExecutionContext):
+ def __init__(self, dialect):
+ self.dialect = dialect
+ def pre_exec(self, engine, proxy, compiled, parameters):
+ self._process_defaults(engine, proxy, compiled, parameters)
+ def post_exec(self, engine, proxy, compiled, parameters):
+ pass
+ def get_rowcount(self, cursor):
+ if hasattr(self, '_rowcount'):
+ return self._rowcount
+ else:
+ return cursor.rowcount
+ def supports_sane_rowcount(self):
+ return self.dialect.supports_sane_rowcount()
+ def last_inserted_ids(self):
+ return self._last_inserted_ids
+ def last_inserted_params(self):
+ return self._last_inserted_params
+ def last_updated_params(self):
+ return self._last_updated_params
+ def lastrow_has_defaults(self):
+ return self._lastrow_has_defaults
+ def _process_defaults(self, engine, proxy, compiled, parameters):
+ """INSERT and UPDATE statements, when compiled, may have additional columns added to their
+ VALUES and SET lists corresponding to column defaults/onupdates that are present on the
+ Table object (i.e. ColumnDefault, Sequence, PassiveDefault). This method pre-execs those
+ DefaultGenerator objects that require pre-execution and sets their values within the
+ parameter list, and flags the thread-local state about
+ PassiveDefault objects that may require post-fetching the row after it is inserted/updated.
+ This method relies upon logic within the ANSISQLCompiler in its visit_insert and
+ visit_update methods that add the appropriate column clauses to the statement when its
+ being compiled, so that these parameters can be bound to the statement."""
+ if compiled is None: return
+ if getattr(compiled, "isinsert", False):
+ if isinstance(parameters, list):
+ plist = parameters
+ else:
+ plist = [parameters]
+ drunner = self.dialect.defaultrunner(engine, proxy)
+ self._lastrow_has_defaults = False
+ for param in plist:
+ last_inserted_ids = []
+ need_lastrowid=False
+ for c in compiled.statement.table.c:
+ if not param.has_key(c.name) or param[c.name] is None:
+ if isinstance(c.default, schema.PassiveDefault):
+ self._lastrow_has_defaults = True
+ newid = drunner.get_column_default(c)
+ if newid is not None:
+ param[c.name] = newid
+ if c.primary_key:
+ last_inserted_ids.append(param[c.name])
+ elif c.primary_key:
+ need_lastrowid = True
+ elif c.primary_key:
+ last_inserted_ids.append(param[c.name])
+ if need_lastrowid:
+ self._last_inserted_ids = None
+ else:
+ self._last_inserted_ids = last_inserted_ids
+ self._last_inserted_params = param
+ elif getattr(compiled, 'isupdate', False):
+ if isinstance(parameters, list):
+ plist = parameters
+ else:
+ plist = [parameters]
+ drunner = self.dialect.defaultrunner(engine, proxy)
+ self._lastrow_has_defaults = False
+ for param in plist:
+ for c in compiled.statement.table.c:
+ if c.onupdate is not None and (not param.has_key(c.name) or param[c.name] is None):
+ value = drunner.get_column_onupdate(c)
+ if value is not None:
+ param[c.name] = value
+ self._last_updated_params = param
+
+
--- /dev/null
+"""defines different strategies for creating new instances of sql.Engine.
+by default there are two, one which is the "thread-local" strategy, one which is the "plain" strategy.
+new strategies can be added via constructing a new EngineStrategy object which will add itself to the
+list of available strategies here, or replace one of the existing name.
+this can be accomplished via a mod; see the sqlalchemy/mods package for details."""
+
+
+from sqlalchemy.engine import base, default, threadlocal, url
+
+strategies = {}
+
+class EngineStrategy(object):
+ """defines a function that receives input arguments and produces an instance of sql.Engine, typically
+ an instance sqlalchemy.engine.base.ComposedSQLEngine or a subclass."""
+ def __init__(self, name):
+ """constructs a new EngineStrategy object and sets it in the list of available strategies
+ under this name."""
+ self.name = name
+ strategies[self.name] = self
+ def create(self, *args, **kwargs):
+ """given arguments, returns a new sql.Engine instance."""
+ raise NotImplementedError()
+
+
+class PlainEngineStrategy(EngineStrategy):
+ def __init__(self):
+ EngineStrategy.__init__(self, 'plain')
+ def create(self, name_or_url, **kwargs):
+ u = url.make_url(name_or_url)
+ module = u.get_module()
+
+ dialect = module.dialect(**kwargs)
+
+ poolargs = {}
+ for key in (('echo', 'echo_pool'), ('pool_size', 'pool_size'), ('max_overflow', 'max_overflow'), ('poolclass', 'poolclass'), ('pool_timeout','timeout')):
+ if kwargs.has_key(key[0]):
+ poolargs[key[1]] = kwargs[key[0]]
+ poolclass = getattr(module, 'poolclass', None)
+ if poolclass is not None:
+ poolargs.setdefault('poolclass', poolclass)
+ poolargs['use_threadlocal'] = False
+ provider = default.PoolConnectionProvider(dialect, u, **poolargs)
+
+ return base.ComposedSQLEngine(provider, dialect, **kwargs)
+PlainEngineStrategy()
+
+class ThreadLocalEngineStrategy(EngineStrategy):
+ def __init__(self):
+ EngineStrategy.__init__(self, 'threadlocal')
+ def create(self, name_or_url, **kwargs):
+ u = url.make_url(name_or_url)
+ module = u.get_module()
+
+ dialect = module.dialect(**kwargs)
+
+ poolargs = {}
+ for key in (('echo', 'echo_pool'), ('pool_size', 'pool_size'), ('max_overflow', 'max_overflow'), ('poolclass', 'poolclass'), ('pool_timeout','timeout')):
+ if kwargs.has_key(key[0]):
+ poolargs[key[1]] = kwargs[key[0]]
+ poolclass = getattr(module, 'poolclass', None)
+ if poolclass is not None:
+ poolargs.setdefault('poolclass', poolclass)
+ poolargs['use_threadlocal'] = True
+ provider = threadlocal.TLocalConnectionProvider(dialect, u, **poolargs)
+
+ return threadlocal.TLEngine(provider, dialect, **kwargs)
+ThreadLocalEngineStrategy()
+
+
+
--- /dev/null
+from sqlalchemy import schema, exceptions, util, sql, types
+import StringIO, sys, re
+import base, default
+
+"""provides a thread-local transactional wrapper around the basic ComposedSQLEngine. multiple calls to engine.connect()
+will return the same connection for the same thread. also provides begin/commit methods on the engine itself
+which correspond to a thread-local transaction."""
+
+class TLTransaction(base.Transaction):
+ def rollback(self):
+ try:
+ base.Transaction.rollback(self)
+ finally:
+ try:
+ del self.connection.engine.context.transaction
+ except AttributeError:
+ pass
+ def commit(self):
+ try:
+ base.Transaction.commit(self)
+ stack = self.connection.engine.context.transaction
+ stack.pop()
+ if len(stack) == 0:
+ del self.connection.engine.context.transaction
+ except:
+ try:
+ del self.connection.engine.context.transaction
+ except AttributeError:
+ pass
+ raise
+
+class TLConnection(base.Connection):
+ def _create_transaction(self, parent):
+ return TLTransaction(self, parent)
+ def begin(self):
+ t = base.Connection.begin(self)
+ if not hasattr(self.engine.context, 'transaction'):
+ self.engine.context.transaction = []
+ self.engine.context.transaction.append(t)
+ return t
+
+class TLEngine(base.ComposedSQLEngine):
+ """a ComposedSQLEngine that includes support for thread-local managed transactions. This engine
+ is better suited to be used with threadlocal Pool object."""
+ def __init__(self, *args, **kwargs):
+ """the TLEngine relies upon the ConnectionProvider having "threadlocal" behavior,
+ so that once a connection is checked out for the current thread, you get that same connection
+ repeatedly."""
+ base.ComposedSQLEngine.__init__(self, *args, **kwargs)
+ self.context = util.ThreadLocal()
+ def raw_connection(self):
+ """returns a DBAPI connection."""
+ return self.connection_provider.get_connection()
+ def connect(self, **kwargs):
+ """returns a Connection that is not thread-locally scoped. this is the equilvalent to calling
+ "connect()" on a ComposedSQLEngine."""
+ return base.Connection(self, self.connection_provider.unique_connection())
+ def contextual_connect(self, **kwargs):
+ """returns a TLConnection which is thread-locally scoped."""
+ return TLConnection(self, **kwargs)
+ def begin(self):
+ return self.connect().begin()
+ def commit(self):
+ if hasattr(self.context, 'transaction'):
+ self.context.transaction[-1].commit()
+ def rollback(self):
+ if hasattr(self.context, 'transaction'):
+ self.context.transaction[-1].rollback()
+ def transaction(self, func, *args, **kwargs):
+ """executes the given function within a transaction boundary. this is a shortcut for
+ explicitly calling begin() and commit() and optionally rollback() when execptions are raised.
+ The given *args and **kwargs will be passed to the function as well, which could be handy
+ in constructing decorators."""
+ trans = self.begin()
+ try:
+ func(*args, **kwargs)
+ except:
+ trans.rollback()
+ raise
+ trans.commit()
+
+class TLocalConnectionProvider(default.PoolConnectionProvider):
+ def unique_connection(self):
+ return self._pool.unique_connection()
--- /dev/null
+import re
+import cgi
+
+class URL(object):
+ def __init__(self, drivername, username=None, password=None, host=None, port=None, database=None):
+ self.drivername = drivername
+ self.username = username
+ self.password = password
+ self.host = host
+ self.port = port
+ self.database= database
+ def __str__(self):
+ s = self.drivername + "://"
+ if self.username is not None:
+ s += self.username
+ if self.password is not None:
+ s += ':' + self.password
+ s += "@"
+ if self.host is not None:
+ s += self.host
+ if self.port is not None:
+ s += ':' + self.port
+ if self.database is not None:
+ s += '/' + self.database
+ return s
+ def get_module(self):
+ return getattr(__import__('sqlalchemy.databases.%s' % self.drivername).databases, self.drivername)
+ def translate_connect_args(self, names):
+ """translates this URL's attributes into a dictionary of connection arguments used by a specific dbapi.
+ the names parameter is a list of argument names in the form ('host', 'database', 'user', 'password', 'port')
+ where the given strings match the corresponding argument names for the dbapi. Will return a dictionary
+ with the dbapi-specific parameters."""
+ a = {}
+ attribute_names = ['host', 'database', 'username', 'password', 'port']
+ for n in names:
+ sname = attribute_names.pop(0)
+ if n is None:
+ continue
+ if getattr(self, sname, None) is not None:
+ a[n] = getattr(self, sname)
+ return a
+
+
+def make_url(name_or_url):
+ if isinstance(name_or_url, str):
+ return _parse_rfc1738_args(name_or_url)
+ else:
+ return name_or_url
+
+def _parse_rfc1738_args(name):
+ pattern = re.compile(r'''
+ (\w+)://
+ (?:
+ ([^:]*)
+ (?::(.*))?
+ @)?
+ (?:
+ ([^/:]*)
+ (?::([^/]*))?
+ )?
+ (?:/(.*))?
+ '''
+ , re.X)
+
+ m = pattern.match(name)
+ if m is not None:
+ (name, username, password, host, port, database) = m.group(1, 2, 3, 4, 5, 6)
+ opts = {'username':username,'password':password,'host':host,'port':port,'database':database}
+ return URL(name, **opts)
+ else:
+ return None
+
+def _parse_keyvalue_args(name):
+ m = re.match( r'(\w+)://(.*)', name)
+ if m is not None:
+ (name, args) = m.group(1, 2)
+ opts = dict( cgi.parse_qsl( args ) )
+ return URL(name, *opts)
+ else:
+ return None
+
objects. This error generally corresponds to construction time state errors."""
pass
-class CommitError(SQLAlchemyError):
- """raised when an invalid condition is detected upon a commit()"""
+class FlushError(SQLAlchemyError):
+ """raised when an invalid condition is detected upon a flush()"""
pass
class InvalidRequestError(SQLAlchemyError):
-from sqlalchemy import assign_mapper, relation, exceptions
+from sqlalchemy import create_session, relation, mapper, join, DynamicMetaData, class_mapper
+from sqlalchemy import and_, or_
from sqlalchemy import Table, Column, ForeignKey
+from sqlalchemy.ext.sessioncontext import SessionContext
+from sqlalchemy.ext.assignmapper import assign_mapper
+from sqlalchemy import backref as create_backref
import inspect
import sys
+import sets
+
+#
+# the "proxy" to the database engine... this can be swapped out at runtime
+#
+metadata = DynamicMetaData("activemapper")
+
+#
+# thread local SessionContext
+#
+class Objectstore(SessionContext):
+ def __getattr__(self, key):
+ return getattr(self.current, key)
+ def get_session(self):
+ return self.current
+
+objectstore = Objectstore(create_session)
+
#
# declarative column declaration - this is so that we can infer the colname
class one_to_one(relationship):
def __init__(self, classname, colname=None, backref=None, private=False, lazy=True):
- relationship.__init__(self, classname, colname, backref, private, lazy, uselist=False)
+ relationship.__init__(self, classname, colname, create_backref(backref, uselist=False), private, lazy, uselist=False)
class many_to_many(relationship):
def __init__(self, classname, secondary, backref=None, lazy=True):
#
__deferred_classes__ = []
-__processed_classes__ = []
-
-def check_relationships(klass):
- #Check the class for foreign_keys recursively. If some foreign table is not found, the processing of the table
- #must be defered.
- for keyname in klass.table._foreign_keys:
- xtable = keyname._colspec[:keyname._colspec.find('.')]
- tablefound = False
- for xclass in ActiveMapperMeta.classes:
- if ActiveMapperMeta.classes[xclass].table.from_name == xtable:
- tablefound = True
- break
- if tablefound==False:
- #The refered table has not yet been created.
- return False
-
- return True
-
-
-def process_relationships(klass):
+def process_relationships(klass, was_deferred=False):
defer = False
for propname, reldesc in klass.relations.items():
- # we require that every related table has been processed first
- if not reldesc.classname in __processed_classes__:
- if not klass._classname in __deferred_classes__: __deferred_classes__.append(klass._classname)
- defer = True
-
- # check every column item to see if it points to an existing table
- # if it does not, defer...
- if not defer:
- if not check_relationships(klass):
- if not klass._classname in __deferred_classes__: __deferred_classes__.append(klass._classname)
+ if not reldesc.classname in ActiveMapperMeta.classes:
+ if not was_deferred: __deferred_classes__.append(klass)
defer = True
if not defer:
relations = {}
-
for propname, reldesc in klass.relations.items():
relclass = ActiveMapperMeta.classes[reldesc.classname]
relations[propname] = relation(relclass.mapper,
private=reldesc.private,
lazy=reldesc.lazy,
uselist=reldesc.uselist)
- if len(relations) > 0:
- assign_ok = True
- try:
- assign_mapper(klass, klass.table, properties=relations)
- except exceptions.ArgumentError:
- assign_ok = False
-
- if assign_ok:
- __processed_classes__.append(klass._classname)
- if klass._classname in __deferred_classes__: __deferred_classes__.remove(klass._classname)
- else:
- __processed_classes__.append(klass._classname)
-
+ class_mapper(klass).add_properties(relations)
+ #assign_mapper(objectstore, klass, klass.table, properties=relations,
+ # inherits=getattr(klass, "_base_mapper", None))
+ if was_deferred: __deferred_classes__.remove(klass)
+
+ if not was_deferred:
for deferred_class in __deferred_classes__:
- process_relationships(ActiveMapperMeta.classes[deferred_class])
+ process_relationships(deferred_class, was_deferred=True)
+
class ActiveMapperMeta(type):
classes = {}
-
+ metadatas = sets.Set()
def __init__(cls, clsname, bases, dict):
table_name = clsname.lower()
columns = []
relations = {}
-
+ _metadata = getattr( sys.modules[cls.__module__], "__metadata__", metadata )
+
if 'mapping' in dict:
members = inspect.getmembers(dict.get('mapping'))
for name, value in members:
if name == '__table__':
table_name = value
continue
-
+
+ if '__metadata__' == name:
+ _metadata= value
+ continue
+
if name.startswith('__'): continue
-
+
if isinstance(value, column):
if value.foreign_key:
col = Column(value.colname or name,
*value.args, **value.kwargs)
columns.append(col)
continue
-
+
if isinstance(value, relationship):
relations[name] = value
-
- cls.table = Table(table_name, redefine=True, *columns)
-
+ assert _metadata is not None, "No MetaData specified"
+ ActiveMapperMeta.metadatas.add(_metadata)
+ cls.table = Table(table_name, _metadata, *columns)
# check for inheritence
- if hasattr(bases[0], "mapping"):
- cls._base_mapper = bases[0].mapper
- assign_mapper(cls, cls.table, inherits=cls._base_mapper)
- elif len(relations) == 0:
- assign_mapper(cls, cls.table)
+ if hasattr( bases[0], "mapping" ):
+ cls._base_mapper= bases[0].mapper
+ assign_mapper(objectstore, cls, cls.table, inherits=cls._base_mapper)
+ else:
+ assign_mapper(objectstore, cls, cls.table)
cls.relations = relations
- cls._classname = clsname
ActiveMapperMeta.classes[clsname] = cls
+
process_relationships(cls)
-
+
super(ActiveMapperMeta, cls).__init__(clsname, bases, dict)
class ActiveMapper(object):
__metaclass__ = ActiveMapperMeta
-
+
def set(self, **kwargs):
for key, value in kwargs.items():
setattr(self, key, value)
#
def create_tables():
- for klass in ActiveMapperMeta.classes.values():
- klass.table.create()
-
-#
-# a utility function to drop all tables for all ActiveMapper classes
-#
+ for metadata in ActiveMapperMeta.metadatas:
+ metadata.create_all()
def drop_tables():
- for klass in ActiveMapperMeta.classes.values():
- klass.table.drop()
\ No newline at end of file
+ for metadata in ActiveMapperMeta.metadatas:
+ metadata.drop_all()
+
--- /dev/null
+from sqlalchemy import mapper, util
+import types
+
+def monkeypatch_query_method(ctx, class_, name):
+ def do(self, *args, **kwargs):
+ query = class_.mapper.query(session=ctx.current)
+ return getattr(query, name)(*args, **kwargs)
+ setattr(class_, name, classmethod(do))
+
+def monkeypatch_objectstore_method(ctx, class_, name):
+ def do(self, *args, **kwargs):
+ session = ctx.current
+ return getattr(session, name)(self, *args, **kwargs)
+ setattr(class_, name, do)
+
+def assign_mapper(ctx, class_, *args, **kwargs):
+ kwargs.setdefault("is_primary", True)
+ if not isinstance(getattr(class_, '__init__'), types.MethodType):
+ def __init__(self, **kwargs):
+ for key, value in kwargs.items():
+ setattr(self, key, value)
+ class_.__init__ = __init__
+ extension = kwargs.pop('extension', None)
+ if extension is not None:
+ extension = util.to_list(extension)
+ extension.append(ctx.mapper_extension)
+ else:
+ extension = ctx.mapper_extension
+ m = mapper(class_, extension=extension, *args, **kwargs)
+ class_.mapper = m
+ for name in ['get', 'select', 'select_by', 'selectone', 'get_by', 'join_to', 'join_via']:
+ monkeypatch_query_method(ctx, class_, name)
+ for name in ['flush', 'delete', 'expire', 'refresh', 'expunge', 'merge', 'update', 'save_or_update']:
+ monkeypatch_objectstore_method(ctx, class_, name)
from sqlalchemy import sql
from sqlalchemy.engine import create_engine
-from sqlalchemy.types import TypeEngine
-import sqlalchemy.schema as schema
-import thread, weakref
-class BaseProxyEngine(schema.SchemaEngine):
- '''
- Basis for all proxy engines
- '''
+__all__ = ['BaseProxyEngine', 'AutoConnectEngine', 'ProxyEngine']
+
+class BaseProxyEngine(sql.Engine):
+ """Basis for all proxy engines."""
def get_engine(self):
raise NotImplementedError
raise NotImplementedError
engine = property(lambda s:s.get_engine(), lambda s,e:s.set_engine(e))
-
- def reflecttable(self, table):
- return self.get_engine().reflecttable(table)
+
def execute_compiled(self, *args, **kwargs):
- return self.get_engine().execute_compiled(*args, **kwargs)
- def compiler(self, *args, **kwargs):
- return self.get_engine().compiler(*args, **kwargs)
- def schemagenerator(self, *args, **kwargs):
- return self.get_engine().schemagenerator(*args, **kwargs)
- def schemadropper(self, *args, **kwargs):
- return self.get_engine().schemadropper(*args, **kwargs)
-
- def hash_key(self):
- return "%s(%s)" % (self.__class__.__name__, id(self))
+ """this method is required to be present as it overrides the execute_compiled present in sql.Engine"""
+ return self.get_engine().execute_compiled(*args, **kwargs)
+ def compiler(self, *args, **kwargs):
+ """this method is required to be present as it overrides the compiler method present in sql.Engine"""
+ return self.get_engine().compiler(*args, **kwargs)
- def oid_column_name(self):
- # oid_column should not be requested before the engine is connected.
- # it should ideally only be called at query compilation time.
- e= self.get_engine()
- if e is None:
- return None
- return e.oid_column_name()
-
def __getattr__(self, attr):
+ """provides proxying for methods that are not otherwise present on this BaseProxyEngine. Note
+ that methods which are present on the base class sql.Engine will *not* be proxied through this,
+ and must be explicit on this class."""
# call get_engine() to give subclasses a chance to change
# connection establishment behavior
- e= self.get_engine()
+ e = self.get_engine()
if e is not None:
return getattr(e, attr)
- raise AttributeError('No connection established in ProxyEngine: '
- ' no access to %s' % attr)
+ raise AttributeError("No connection established in ProxyEngine: "
+ " no access to %s" % attr)
+
class AutoConnectEngine(BaseProxyEngine):
- '''
- An SQLEngine proxy that automatically connects when necessary.
- '''
+ """An SQLEngine proxy that automatically connects when necessary."""
- def __init__(self, dburi, opts=None, **kwargs):
+ def __init__(self, dburi, **kwargs):
BaseProxyEngine.__init__(self)
- self.dburi= dburi
- self.opts= opts
- self.kwargs= kwargs
- self._engine= None
+ self.dburi = dburi
+ self.kwargs = kwargs
+ self._engine = None
def get_engine(self):
if self._engine is None:
if callable(self.dburi):
- dburi= self.dburi()
+ dburi = self.dburi()
else:
- dburi= self.dburi
- self._engine= create_engine( dburi, self.opts, **self.kwargs )
+ dburi = self.dburi
+ self._engine = create_engine(dburi, **self.kwargs)
return self._engine
-
class ProxyEngine(BaseProxyEngine):
- """
- SQLEngine proxy. Supports lazy and late initialization by
- delegating to a real engine (set with connect()), and using proxy
- classes for TypeEngine.
+ """Engine proxy for lazy and late initialization.
+
+ This engine will delegate access to a real engine set with connect().
"""
def __init__(self, **kwargs):
self.storage.connection = {}
self.storage.engine = None
self.kwargs = kwargs
-
- def connect(self, uri, opts=None, **kwargs):
- """Establish connection to a real engine.
- """
- kw = self.kwargs.copy()
- kw.update(kwargs)
- kwargs = kw
- key = "%s(%s,%s)" % (uri, repr(opts), repr(kwargs))
+
+ def connect(self, *args, **kwargs):
+ """Establish connection to a real engine."""
+
+ kwargs.update(self.kwargs)
+ if not kwargs:
+ key = repr(args)
+ else:
+ key = "%s, %s" % (repr(args), repr(sorted(kwargs.items())))
try:
map = self.storage.connection
except AttributeError:
try:
self.engine = map[key]
except KeyError:
- map[key] = create_engine(uri, opts, **kwargs)
+ map[key] = create_engine(*args, **kwargs)
self.storage.engine = map[key]
def get_engine(self):
if self.storage.engine is None:
- raise AttributeError('No connection established')
+ raise AttributeError("No connection established")
return self.storage.engine
def set_engine(self, engine):
self.storage.engine = engine
-
-
--- /dev/null
+import sqlalchemy.sql as sql
+
+import sqlalchemy.orm as orm
+
+
+class SelectResultsExt(orm.MapperExtension):
+ def select_by(self, query, *args, **params):
+ return SelectResults(query, query.join_by(*args, **params))
+ def select(self, query, arg=None, **kwargs):
+ if arg is not None and isinstance(arg, sql.Selectable):
+ return orm.EXT_PASS
+ else:
+ return SelectResults(query, arg, ops=kwargs)
+
+class SelectResults(object):
+ def __init__(self, query, clause=None, ops={}):
+ self._query = query
+ self._clause = clause
+ self._ops = {}
+ self._ops.update(ops)
+
+ def count(self):
+ return self._query.count(self._clause)
+
+ def min(self, col):
+ return sql.select([sql.func.min(col)], self._clause, **self._ops).scalar()
+
+ def max(self, col):
+ return sql.select([sql.func.max(col)], self._clause, **self._ops).scalar()
+
+ def sum(self, col):
+ return sql.select([sql.func.sum(col)], self._clause, **self._ops).scalar()
+
+ def avg(self, col):
+ return sql.select([sql.func.avg(col)], self._clause, **self._ops).scalar()
+
+ def clone(self):
+ return SelectResults(self._query, self._clause, self._ops.copy())
+
+ def filter(self, clause):
+ new = self.clone()
+ new._clause = sql.and_(self._clause, clause)
+ return new
+
+ def order_by(self, order_by):
+ new = self.clone()
+ new._ops['order_by'] = order_by
+ return new
+
+ def limit(self, limit):
+ return self[:limit]
+
+ def offset(self, offset):
+ return self[offset:]
+
+ def list(self):
+ return list(self)
+
+ def __getitem__(self, item):
+ if isinstance(item, slice):
+ start = item.start
+ stop = item.stop
+ if (isinstance(start, int) and start < 0) or \
+ (isinstance(stop, int) and stop < 0):
+ return list(self)[item]
+ else:
+ res = self.clone()
+ if start is not None and stop is not None:
+ res._ops.update(dict(offset=self._ops.get('offset', 0)+start, limit=stop-start))
+ elif start is None and stop is not None:
+ res._ops.update(dict(limit=stop))
+ elif start is not None and stop is None:
+ res._ops.update(dict(offset=self._ops.get('offset', 0)+start))
+ if item.step is not None:
+ return list(res)[None:None:item.step]
+ else:
+ return res
+ else:
+ return list(self[item:item+1])[0]
+
+ def __iter__(self):
+ return iter(self._query.select_whereclause(self._clause, **self._ops))
--- /dev/null
+from sqlalchemy.util import ScopedRegistry
+from sqlalchemy.orm.mapper import MapperExtension
+
+__all__ = ['SessionContext', 'SessionContextExt']
+
+class SessionContext(object):
+ """A simple wrapper for ScopedRegistry that provides a "current" property
+ which can be used to get, set, or remove the session in the current scope.
+
+ By default this object provides thread-local scoping, which is the default
+ scope provided by sqlalchemy.util.ScopedRegistry.
+
+ Usage:
+ engine = create_engine(...)
+ def session_factory():
+ return Session(bind_to=engine)
+ context = SessionContext(session_factory)
+
+ s = context.current # get thread-local session
+ context.current = Session(bind_to=other_engine) # set current session
+ del context.current # discard the thread-local session (a new one will
+ # be created on the next call to context.current)
+ """
+ def __init__(self, session_factory, scopefunc=None):
+ self.registry = ScopedRegistry(session_factory, scopefunc)
+ super(SessionContext, self).__init__()
+
+ def get_current(self):
+ return self.registry()
+ def set_current(self, session):
+ self.registry.set(session)
+ def del_current(self):
+ self.registry.clear()
+ current = property(get_current, set_current, del_current,
+ """Property used to get/set/del the session in the current scope""")
+
+ def _get_mapper_extension(self):
+ try:
+ return self._extension
+ except AttributeError:
+ self._extension = ext = SessionContextExt(self)
+ return ext
+ mapper_extension = property(_get_mapper_extension,
+ doc="""get a mapper extension that implements get_session using this context""")
+
+
+class SessionContextExt(MapperExtension):
+ """a mapper extionsion that provides sessions to a mapper using SessionContext"""
+
+ def __init__(self, context):
+ MapperExtension.__init__(self)
+ self.context = context
+
+ def get_session(self):
+ return self.context.current
-from sqlalchemy import *
-
-"""
-SqlSoup provides a convenient way to access database tables without having
-to declare table or mapper classes ahead of time.
-
-Suppose we have a database with users, books, and loans tables
-(corresponding to the PyWebOff dataset, if you're curious).
-For testing purposes, we can create this db as follows:
-
->>> from sqlalchemy import create_engine
->>> e = create_engine('sqlite://filename=:memory:')
->>> for sql in _testsql: e.execute(sql)
-...
-
-Creating a SqlSoup gateway is just like creating an SqlAlchemy engine:
->>> from sqlalchemy.ext.sqlsoup import SqlSoup
->>> soup = SqlSoup('sqlite://filename=:memory:')
-
-or, you can re-use an existing engine:
->>> soup = SqlSoup(e)
-
-Loading objects is as easy as this:
->>> users = soup.users.select()
->>> users.sort()
->>> users
-[Class_Users(name='Bhargan Basepair',email='basepair@example.edu',password='basepair',classname=None,admin=1), Class_Users(name='Joe Student',email='student@example.edu',password='student',classname=None,admin=0)]
-
-Of course, letting the database do the sort is better (".c" is short for ".columns"):
->>> soup.users.select(order_by=[soup.users.c.name])
-[Class_Users(name='Bhargan Basepair',email='basepair@example.edu',password='basepair',classname=None,admin=1),
- Class_Users(name='Joe Student',email='student@example.edu',password='student',classname=None,admin=0)]
-
-Field access is intuitive:
->>> users[0].email
-u'basepair@example.edu'
-
-Of course, you don't want to load all users very often. The common case is to
-select by a key or other field:
->>> soup.users.selectone_by(name='Bhargan Basepair')
-Class_Users(name='Bhargan Basepair',email='basepair@example.edu',password='basepair',classname=None,admin=1)
-
-All the SqlAlchemy mapper select variants (select, select_by, selectone, selectone_by, selectfirst, selectfirst_by)
-are available. See the SqlAlchemy documentation for details:
-http://www.sqlalchemy.org/docs/sqlconstruction.myt
-
-Modifying objects is intuitive:
->>> user = _
->>> user.email = 'basepair+nospam@example.edu'
->>> soup.commit()
-
-(SqlSoup leverages the sophisticated SqlAlchemy unit-of-work code, so
-multiple updates to a single object will be turned into a single UPDATE
-statement when you commit.)
-
-Finally, insert and delete. Let's insert a new loan, then delete it:
->>> soup.loans.insert(book_id=soup.books.selectfirst().id, user_name=user.name)
-Class_Loans(book_id=1,user_name='Bhargan Basepair',loan_date=None)
->>> soup.commit()
-
->>> loan = soup.loans.selectone_by(book_id=1, user_name='Bhargan Basepair')
->>> soup.delete(loan)
->>> soup.commit()
-"""
-
-_testsql = """
-CREATE TABLE books (
- id integer PRIMARY KEY, -- auto-SERIAL in sqlite
- title text NOT NULL,
- published_year char(4) NOT NULL,
- authors text NOT NULL
-);
-
-CREATE TABLE users (
- name varchar(32) PRIMARY KEY,
- email varchar(128) NOT NULL,
- password varchar(128) NOT NULL,
- classname text,
- admin int NOT NULL -- 0 = false
-);
-
-CREATE TABLE loans (
- book_id int PRIMARY KEY REFERENCES books(id),
- user_name varchar(32) references users(name)
- ON DELETE SET NULL ON UPDATE CASCADE,
- loan_date date NOT NULL DEFAULT current_timestamp
-);
-
-insert into users(name, email, password, admin)
-values('Bhargan Basepair', 'basepair@example.edu', 'basepair', 1);
-insert into users(name, email, password, admin)
-values('Joe Student', 'student@example.edu', 'student', 0);
-
-insert into books(title, published_year, authors)
-values('Mustards I Have Known', '1989', 'Jones');
-insert into books(title, published_year, authors)
-values('Regional Variation in Moss', '1971', 'Flim and Flam');
-
-insert into loans(book_id, user_name)
-values (
- (select min(id) from books),
- (select name from users where name like 'Joe%'))
-;
-""".split(';')
-
-__all__ = ['NoSuchTableError', 'SqlSoup']
-
-class NoSuchTableError(SQLAlchemyError): pass
-
-# metaclass is necessary to expose class methods with getattr, e.g.
-# we want to pass db.users.select through to users._mapper.select
-class TableClassType(type):
- def insert(cls, **kwargs):
- o = cls()
- o.__dict__.update(kwargs)
- return o
- def __getattr__(cls, attr):
- if attr == '_mapper':
- # called during mapper init
- raise AttributeError()
- return getattr(cls._mapper, attr)
-
-def class_for_table(table):
- klass = TableClassType('Class_' + table.name.capitalize(), (object,), {})
- def __repr__(self):
- import locale
- encoding = locale.getdefaultlocale()[1]
- L = []
- for k in self.__class__.c.keys():
- value = getattr(self, k, '')
- if isinstance(value, unicode):
- value = value.encode(encoding)
- L.append("%s=%r" % (k, value))
- return '%s(%s)' % (self.__class__.__name__, ','.join(L))
- klass.__repr__ = __repr__
- klass._mapper = mapper(klass, table)
- return klass
-
-class SqlSoup:
- def __init__(self, *args, **kwargs):
- """
- args may either be an SQLEngine or a set of arguments suitable
- for passing to create_engine
- """
- from sqlalchemy.engine import SQLEngine
- # meh, sometimes having method overloading instead of kwargs would be easier
- if isinstance(args[0], SQLEngine):
- args = list(args)
- engine = args.pop(0)
- if args or kwargs:
- raise ArgumentError('Extra arguments not allowed when engine is given')
- else:
- engine = create_engine(*args, **kwargs)
- self._engine = engine
- self._cache = {}
- def delete(self, *args, **kwargs):
- objectstore.delete(*args, **kwargs)
- def commit(self):
- objectstore.get_session().commit()
- def rollback(self):
- objectstore.clear()
- def _reset(self):
- # for debugging
- self._cache = {}
- self.rollback()
- def __getattr__(self, attr):
- try:
- t = self._cache[attr]
- except KeyError:
- table = Table(attr, self._engine, autoload=True)
- if table.columns:
- t = class_for_table(table)
- else:
- t = None
- self._cache[attr] = t
- if not t:
- raise NoSuchTableError('%s does not exist' % attr)
- return t
-
-if __name__ == '__main__':
- import doctest
- doctest.testmod()
+from sqlalchemy import *\r
+\r
+class NoSuchTableError(SQLAlchemyError): pass\r
+\r
+# metaclass is necessary to expose class methods with getattr, e.g.\r
+# we want to pass db.users.select through to users._mapper.select\r
+class TableClassType(type):\r
+ def insert(cls, **kwargs):\r
+ o = cls()\r
+ o.__dict__.update(kwargs)\r
+ return o\r
+ def __getattr__(cls, attr):\r
+ if attr == '_mapper':\r
+ # called during mapper init\r
+ raise AttributeError()\r
+ return getattr(cls._mapper, attr)\r
+\r
+def class_for_table(table):\r
+ klass = TableClassType('Class_' + table.name.capitalize(), (object,), {})\r
+ def __repr__(self):\r
+ import locale\r
+ encoding = locale.getdefaultlocale()[1]\r
+ L = []\r
+ for k in self.__class__.c.keys():\r
+ value = getattr(self, k, '')\r
+ if isinstance(value, unicode):\r
+ value = value.encode(encoding)\r
+ L.append("%s=%r" % (k, value))\r
+ return '%s(%s)' % (self.__class__.__name__, ','.join(L))\r
+ klass.__repr__ = __repr__\r
+ klass._mapper = mapper(klass, table)\r
+ return klass\r
+\r
+class SqlSoup:\r
+ def __init__(self, *args, **kwargs):\r
+ """\r
+ args may either be an SQLEngine or a set of arguments suitable\r
+ for passing to create_engine\r
+ """\r
+ from sqlalchemy.sql import Engine\r
+ # meh, sometimes having method overloading instead of kwargs would be easier\r
+ if isinstance(args[0], Engine):\r
+ engine = args.pop(0)\r
+ if args or kwargs:\r
+ raise ArgumentError('Extra arguments not allowed when engine is given')\r
+ else:\r
+ engine = create_engine(*args, **kwargs)\r
+ self._engine = engine\r
+ self._cache = {}\r
+ def delete(self, *args, **kwargs):\r
+ objectstore.delete(*args, **kwargs)\r
+ def commit(self):\r
+ objectstore.get_session().commit()\r
+ def rollback(self):\r
+ objectstore.clear()\r
+ def _reset(self):\r
+ # for debugging\r
+ self._cache = {}\r
+ self.rollback()\r
+ def __getattr__(self, attr):\r
+ try:\r
+ t = self._cache[attr]\r
+ except KeyError:\r
+ table = Table(attr, self._engine, autoload=True)\r
+ if table.columns:\r
+ t = class_for_table(table)\r
+ else:\r
+ t = None\r
+ self._cache[attr] = t\r
+ if not t:\r
+ raise NoSuchTableError('%s does not exist' % attr)\r
+ return t\r
+++ /dev/null
-# objectstore.py
-# Copyright (C) 2005,2006 Michael Bayer mike_mp@zzzcomputing.com
-#
-# This module is part of SQLAlchemy and is released under
-# the MIT License: http://www.opensource.org/licenses/mit-license.php
-
-"""provides the Session object and a function-oriented convenience interface. This is the
-"front-end" to the Unit of Work system in unitofwork.py. Issues of "scope" are dealt with here,
-primarily through an important function "get_session()", which is where mappers and units of work go to get a handle on the current threa-local context. """
-
-from sqlalchemy import util
-from sqlalchemy.exceptions import *
-import unitofwork
-import weakref
-import sqlalchemy
-
-class Session(object):
- """Maintains a UnitOfWork instance, including transaction state."""
-
- def __init__(self, hash_key=None, new_imap=True, import_session=None):
- """Initialize the objectstore with a UnitOfWork registry. If called
- with no arguments, creates a single UnitOfWork for all operations.
-
- nest_transactions - indicates begin/commit statements can be executed in a
- "nested", defaults to False which indicates "only commit on the outermost begin/commit"
- hash_key - the hash_key used to identify objects against this session, which
- defaults to the id of the Session instance.
- """
- if import_session is not None:
- self.uow = unitofwork.UnitOfWork(identity_map=import_session.uow.identity_map)
- elif new_imap is False:
- self.uow = unitofwork.UnitOfWork(identity_map=objectstore.get_session().uow.identity_map)
- else:
- self.uow = unitofwork.UnitOfWork()
-
- self.binds = {}
- if hash_key is None:
- self.hash_key = id(self)
- else:
- self.hash_key = hash_key
- _sessions[self.hash_key] = self
-
- def bind_table(self, table, bindto):
- self.binds[table] = bindto
-
- def get_id_key(ident, class_, entity_name=None):
- """returns an identity-map key for use in storing/retrieving an item from the identity
- map, given a tuple of the object's primary key values.
-
- ident - a tuple of primary key values corresponding to the object to be stored. these
- values should be in the same order as the primary keys of the table
-
- class_ - a reference to the object's class
-
- entity_name - optional string name to further qualify the class
- """
- return (class_, tuple(ident), entity_name)
- get_id_key = staticmethod(get_id_key)
-
- def get_row_key(row, class_, primary_key, entity_name=None):
- """returns an identity-map key for use in storing/retrieving an item from the identity
- map, given a result set row.
-
- row - a sqlalchemy.dbengine.RowProxy instance or other map corresponding result-set
- column names to their values within a row.
-
- class_ - a reference to the object's class
-
- primary_key - a list of column objects that will target the primary key values
- in the given row.
-
- entity_name - optional string name to further qualify the class
- """
- return (class_, tuple([row[column] for column in primary_key]), entity_name)
- get_row_key = staticmethod(get_row_key)
-
- def engines(self, mapper):
- return [t.engine for t in mapper.tables]
-
- def flush(self, *obj):
- self.uow.flush(self, *obj)
-
- def refresh(self, *obj):
- """reloads the attributes for the given objects from the database, clears
- any changes made."""
- for o in obj:
- self.uow.refresh(o)
-
- def expire(self, *obj):
- """invalidates the data in the given objects and sets them to refresh themselves
- the next time they are requested."""
- for o in obj:
- self.uow.expire(o)
-
- def expunge(self, *obj):
- for o in obj:
- self.uow.expunge(o)
-
- def register_clean(self, obj):
- self._bind_to(obj)
- self.uow.register_clean(obj)
-
- def register_new(self, obj):
- self._bind_to(obj)
- self.uow.register_new(obj)
-
- def _bind_to(self, obj):
- """given an object, binds it to this session. changes on the object will affect
- the currently scoped UnitOfWork maintained by this session."""
- obj._sa_session_id = self.hash_key
-
- def __getattr__(self, key):
- """proxy other methods to our underlying UnitOfWork"""
- return getattr(self.uow, key)
-
- def clear(self):
- self.uow = unitofwork.UnitOfWork()
-
- def delete(self, *obj):
- """registers the given objects as to be deleted upon the next commit"""
- for o in obj:
- self.uow.register_deleted(o)
-
- def import_instance(self, instance):
- """places the given instance in the current thread's unit of work context,
- either in the current IdentityMap or marked as "new". Returns either the object
- or the current corresponding version in the Identity Map.
-
- this method should be used for any object instance that is coming from a serialized
- storage, from another thread (assuming the regular threaded unit of work model), or any
- case where the instance was loaded/created corresponding to a different base unitofwork
- than the current one."""
- if instance is None:
- return None
- key = getattr(instance, '_instance_key', None)
- mapper = object_mapper(instance)
- u = self.uow
- if key is not None:
- if u.identity_map.has_key(key):
- return u.identity_map[key]
- else:
- instance._instance_key = key
- u.identity_map[key] = instance
- self._bind_to(instance)
- else:
- u.register_new(instance)
- return instance
-
-class LegacySession(Session):
- def __init__(self, nest_on=None, hash_key=None, **kwargs):
- super(LegacySession, self).__init__(**kwargs)
- self.parent_uow = None
- self.begin_count = 0
- self.nest_on = util.to_list(nest_on)
- self.__pushed_count = 0
- def was_pushed(self):
- if self.nest_on is None:
- return
- self.__pushed_count += 1
- if self.__pushed_count == 1:
- for n in self.nest_on:
- n.push_session()
- def was_popped(self):
- if self.nest_on is None or self.__pushed_count == 0:
- return
- self.__pushed_count -= 1
- if self.__pushed_count == 0:
- for n in self.nest_on:
- n.pop_session()
- class SessionTrans(object):
- """returned by Session.begin(), denotes a transactionalized UnitOfWork instance.
- call commit() on this to commit the transaction."""
- def __init__(self, parent, uow, isactive):
- self.__parent = parent
- self.__isactive = isactive
- self.__uow = uow
- isactive = property(lambda s:s.__isactive, doc="True if this SessionTrans is the 'active' transaction marker, else its a no-op.")
- parent = property(lambda s:s.__parent, doc="returns the parent Session of this SessionTrans object.")
- uow = property(lambda s:s.__uow, doc="returns the parent UnitOfWork corresponding to this transaction.")
- def begin(self):
- """calls begin() on the underlying Session object, returning a new no-op SessionTrans object."""
- if self.parent.uow is not self.uow:
- raise InvalidRequestError("This SessionTrans is no longer valid")
- return self.parent.begin()
- def commit(self):
- """commits the transaction noted by this SessionTrans object."""
- self.__parent._trans_commit(self)
- self.__isactive = False
- def rollback(self):
- """rolls back the current UnitOfWork transaction, in the case that begin()
- has been called. The changes logged since the begin() call are discarded."""
- self.__parent._trans_rollback(self)
- self.__isactive = False
- def begin(self):
- """begins a new UnitOfWork transaction and returns a tranasaction-holding
- object. commit() or rollback() should be called on the returned object.
- commit() on the Session will do nothing while a transaction is pending, and further
- calls to begin() will return no-op transactional objects."""
- if self.parent_uow is not None:
- return Session.SessionTrans(self, self.uow, False)
- self.parent_uow = self.uow
- self.uow = unitofwork.UnitOfWork(identity_map = self.uow.identity_map)
- return Session.SessionTrans(self, self.uow, True)
- def commit(self, *objects):
- """commits the current UnitOfWork transaction. called with
- no arguments, this is only used
- for "implicit" transactions when there was no begin().
- if individual objects are submitted, then only those objects are committed, and the
- begin/commit cycle is not affected."""
- # if an object list is given, commit just those but dont
- # change begin/commit status
- if len(objects):
- self._commit_uow(*objects)
- self.uow.flush(self, *objects)
- return
- if self.parent_uow is None:
- self._commit_uow()
- def _trans_commit(self, trans):
- if trans.uow is self.uow and trans.isactive:
- try:
- self._commit_uow()
- finally:
- self.uow = self.parent_uow
- self.parent_uow = None
- def _trans_rollback(self, trans):
- if trans.uow is self.uow:
- self.uow = self.parent_uow
- self.parent_uow = None
- def _commit_uow(self, *obj):
- self.was_pushed()
- try:
- self.uow.flush(self, *obj)
- finally:
- self.was_popped()
-
-Session = LegacySession
-
-def get_id_key(ident, class_, entity_name=None):
- return Session.get_id_key(ident, class_, entity_name)
-
-def get_row_key(row, class_, primary_key, entity_name=None):
- return Session.get_row_key(row, class_, primary_key, entity_name)
-
-def begin():
- """deprecated. use s = Session(new_imap=False)."""
- return get_session().begin()
-
-def commit(*obj):
- """deprecated; use flush(*obj)"""
- get_session().flush(*obj)
-
-def flush(*obj):
- """flushes the current UnitOfWork transaction. if a transaction was begun
- via begin(), flushes only those objects that were created, modified, or deleted
- since that begin statement. otherwise flushes all objects that have been
- changed.
-
- if individual objects are submitted, then only those objects are committed, and the
- begin/commit cycle is not affected."""
- get_session().flush(*obj)
-
-def clear():
- """removes all current UnitOfWorks and IdentityMaps for this thread and
- establishes a new one. It is probably a good idea to discard all
- current mapped object instances, as they are no longer in the Identity Map."""
- get_session().clear()
-
-def refresh(*obj):
- """reloads the state of this object from the database, and cancels any in-memory
- changes."""
- get_session().refresh(*obj)
-
-def expire(*obj):
- """invalidates the data in the given objects and sets them to refresh themselves
- the next time they are requested."""
- get_session().expire(*obj)
-
-def expunge(*obj):
- get_session().expunge(*obj)
-
-def delete(*obj):
- """registers the given objects as to be deleted upon the next commit"""
- s = get_session().delete(*obj)
-
-def has_key(key):
- """returns True if the current thread-local IdentityMap contains the given instance key"""
- return get_session().has_key(key)
-
-def has_instance(instance):
- """returns True if the current thread-local IdentityMap contains the given instance"""
- return get_session().has_instance(instance)
-
-def is_dirty(obj):
- """returns True if the given object is in the current UnitOfWork's new or dirty list,
- or if its a modified list attribute on an object."""
- return get_session().is_dirty(obj)
-
-def instance_key(instance):
- """returns the IdentityMap key for the given instance"""
- return get_session().instance_key(instance)
-
-def import_instance(instance):
- return get_session().import_instance(instance)
-
-def mapper(*args, **params):
- return sqlalchemy.mapping.mapper(*args, **params)
-
-def object_mapper(obj):
- return sqlalchemy.mapping.object_mapper(obj)
-
-def class_mapper(class_):
- return sqlalchemy.mapping.class_mapper(class_)
-
-global_attributes = unitofwork.global_attributes
-
-session_registry = util.ScopedRegistry(Session) # Default session registry
-_sessions = weakref.WeakValueDictionary() # all referenced sessions (including user-created)
-
-def get_session(obj=None):
- # object-specific session ?
- if obj is not None:
- # does it have a hash key ?
- hashkey = getattr(obj, '_sa_session_id', None)
- if hashkey is not None:
- # ok, return that
- try:
- return _sessions[hashkey]
- except KeyError:
- raise InvalidRequestError("Session '%s' referenced by object '%s' no longer exists" % (hashkey, repr(obj)))
-
- return session_registry()
-
-unitofwork.get_session = get_session
-uow = get_session # deprecated
-
-def push_session(sess):
- old = get_session()
- if getattr(sess, '_previous', None) is not None:
- raise InvalidRequestError("Given Session is already pushed onto some thread's stack")
- sess._previous = old
- session_registry.set(sess)
- sess.was_pushed()
-
-def pop_session():
- sess = get_session()
- old = sess._previous
- sess._previous = None
- session_registry.set(old)
- sess.was_popped()
- return old
-
-def using_session(sess, func):
- push_session(sess)
- try:
- return func()
- finally:
- pop_session()
-
+++ /dev/null
-# mapper/util.py
-# Copyright (C) 2005,2006 Michael Bayer mike_mp@zzzcomputing.com
-#
-# This module is part of SQLAlchemy and is released under
-# the MIT License: http://www.opensource.org/licenses/mit-license.php
-
-
-import sqlalchemy.sql as sql
-
-class TableFinder(sql.ClauseVisitor):
- """given a Clause, locates all the Tables within it into a list."""
- def __init__(self, table, check_columns=False):
- self.tables = []
- self.check_columns = check_columns
- if table is not None:
- table.accept_visitor(self)
- def visit_table(self, table):
- self.tables.append(table)
- def __len__(self):
- return len(self.tables)
- def __getitem__(self, i):
- return self.tables[i]
- def __iter__(self):
- return iter(self.tables)
- def __contains__(self, obj):
- return obj in self.tables
- def __add__(self, obj):
- return self.tables + list(obj)
- def visit_column(self, column):
- if self.check_columns:
- column.table.accept_visitor(self)
-def install_mods(*mods):
- for mod in mods:
- if isinstance(mod, str):
- _mod = getattr(__import__('sqlalchemy.mods.%s' % mod).mods, mod)
- _mod.install_plugin()
- else:
- mod.install_plugin()
\ No newline at end of file
--- /dev/null
+
+import sqlalchemy.orm.objectstore as objectstore
+import sqlalchemy.orm.unitofwork as unitofwork
+import sqlalchemy.util as util
+import sqlalchemy
+
+import sqlalchemy.mods.threadlocal
+
+class LegacySession(objectstore.Session):
+ def __init__(self, nest_on=None, hash_key=None, **kwargs):
+ super(LegacySession, self).__init__(**kwargs)
+ self.parent_uow = None
+ self.begin_count = 0
+ self.nest_on = util.to_list(nest_on)
+ self.__pushed_count = 0
+ def was_pushed(self):
+ if self.nest_on is None:
+ return
+ self.__pushed_count += 1
+ if self.__pushed_count == 1:
+ for n in self.nest_on:
+ n.push_session()
+ def was_popped(self):
+ if self.nest_on is None or self.__pushed_count == 0:
+ return
+ self.__pushed_count -= 1
+ if self.__pushed_count == 0:
+ for n in self.nest_on:
+ n.pop_session()
+ class SessionTrans(object):
+ """returned by Session.begin(), denotes a transactionalized UnitOfWork instance.
+ call commit() on this to commit the transaction."""
+ def __init__(self, parent, uow, isactive):
+ self.__parent = parent
+ self.__isactive = isactive
+ self.__uow = uow
+ isactive = property(lambda s:s.__isactive, doc="True if this SessionTrans is the 'active' transaction marker, else its a no-op.")
+ parent = property(lambda s:s.__parent, doc="returns the parent Session of this SessionTrans object.")
+ uow = property(lambda s:s.__uow, doc="returns the parent UnitOfWork corresponding to this transaction.")
+ def begin(self):
+ """calls begin() on the underlying Session object, returning a new no-op SessionTrans object."""
+ if self.parent.uow is not self.uow:
+ raise InvalidRequestError("This SessionTrans is no longer valid")
+ return self.parent.begin()
+ def commit(self):
+ """commits the transaction noted by this SessionTrans object."""
+ self.__parent._trans_commit(self)
+ self.__isactive = False
+ def rollback(self):
+ """rolls back the current UnitOfWork transaction, in the case that begin()
+ has been called. The changes logged since the begin() call are discarded."""
+ self.__parent._trans_rollback(self)
+ self.__isactive = False
+ def begin(self):
+ """begins a new UnitOfWork transaction and returns a tranasaction-holding
+ object. commit() or rollback() should be called on the returned object.
+ commit() on the Session will do nothing while a transaction is pending, and further
+ calls to begin() will return no-op transactional objects."""
+ if self.parent_uow is not None:
+ return LegacySession.SessionTrans(self, self.uow, False)
+ self.parent_uow = self.uow
+ self.uow = unitofwork.UnitOfWork(identity_map = self.uow.identity_map)
+ return LegacySession.SessionTrans(self, self.uow, True)
+ def commit(self, *objects):
+ """commits the current UnitOfWork transaction. called with
+ no arguments, this is only used
+ for "implicit" transactions when there was no begin().
+ if individual objects are submitted, then only those objects are committed, and the
+ begin/commit cycle is not affected."""
+ # if an object list is given, commit just those but dont
+ # change begin/commit status
+ if len(objects):
+ self._commit_uow(*objects)
+ self.uow.flush(self, *objects)
+ return
+ if self.parent_uow is None:
+ self._commit_uow()
+ def _trans_commit(self, trans):
+ if trans.uow is self.uow and trans.isactive:
+ try:
+ self._commit_uow()
+ finally:
+ self.uow = self.parent_uow
+ self.parent_uow = None
+ def _trans_rollback(self, trans):
+ if trans.uow is self.uow:
+ self.uow = self.parent_uow
+ self.parent_uow = None
+ def _commit_uow(self, *obj):
+ self.was_pushed()
+ try:
+ self.uow.flush(self, *obj)
+ finally:
+ self.was_popped()
+
+def begin():
+ """deprecated. use s = Session(new_imap=False)."""
+ return objectstore.get_session().begin()
+
+def commit(*obj):
+ """deprecated; use flush(*obj)"""
+ objectstore.get_session().flush(*obj)
+
+def uow():
+ return objectstore.get_session()
+
+def push_session(sess):
+ old = get_session()
+ if getattr(sess, '_previous', None) is not None:
+ raise InvalidRequestError("Given Session is already pushed onto some thread's stack")
+ sess._previous = old
+ session_registry.set(sess)
+ sess.was_pushed()
+
+def pop_session():
+ sess = get_session()
+ old = sess._previous
+ sess._previous = None
+ session_registry.set(old)
+ sess.was_popped()
+ return old
+
+def using_session(sess, func):
+ push_session(sess)
+ try:
+ return func()
+ finally:
+ pop_session()
+
+def install_plugin():
+ objectstore.Session = LegacySession
+ objectstore.session_registry = util.ScopedRegistry(objectstore.Session)
+ objectstore.begin = begin
+ objectstore.commit = commit
+ objectstore.uow = uow
+ objectstore.push_session = push_session
+ objectstore.pop_session = pop_session
+ objectstore.using_session = using_session
+install_plugin()
-import sqlalchemy.sql as sql
+from sqlalchemy.ext.selectresults import *
+from sqlalchemy.orm.mapper import global_extensions
-import sqlalchemy.mapping as mapping
def install_plugin():
- mapping.global_extensions.append(SelectResultsExt)
-
-class SelectResultsExt(mapping.MapperExtension):
- def select_by(self, query, *args, **params):
- return SelectResults(query, query._by_clause(*args, **params))
- def select(self, query, arg=None, **kwargs):
- if arg is not None and isinstance(arg, sql.Selectable):
- return mapping.EXT_PASS
- else:
- return SelectResults(query, arg, ops=kwargs)
-
-MapperExtension = SelectResultsExt
-
-class SelectResults(object):
- def __init__(self, query, clause=None, ops={}):
- self._query = query
- self._clause = clause
- self._ops = {}
- self._ops.update(ops)
-
- def count(self):
- return self._query.count(self._clause)
-
- def min(self, col):
- return sql.select([sql.func.min(col)], self._clause, **self._ops).scalar()
-
- def max(self, col):
- return sql.select([sql.func.max(col)], self._clause, **self._ops).scalar()
-
- def sum(self, col):
- return sql.select([sql.func.sum(col)], self._clause, **self._ops).scalar()
-
- def avg(self, col):
- return sql.select([sql.func.avg(col)], self._clause, **self._ops).scalar()
-
- def clone(self):
- return SelectResults(self._query, self._clause, self._ops.copy())
-
- def filter(self, clause):
- new = self.clone()
- new._clause = sql.and_(self._clause, clause)
- return new
-
- def order_by(self, order_by):
- new = self.clone()
- new._ops['order_by'] = order_by
- return new
-
- def limit(self, limit):
- return self[:limit]
-
- def offset(self, offset):
- return self[offset:]
-
- def list(self):
- return list(self)
-
- def __getitem__(self, item):
- if isinstance(item, slice):
- start = item.start
- stop = item.stop
- if (isinstance(start, int) and start < 0) or \
- (isinstance(stop, int) and stop < 0):
- return list(self)[item]
- else:
- res = self.clone()
- if start is not None and stop is not None:
- res._ops.update(dict(offset=self._ops.get('offset', 0)+start, limit=stop-start))
- elif start is None and stop is not None:
- res._ops.update(dict(limit=stop))
- elif start is not None and stop is None:
- res._ops.update(dict(offset=self._ops.get('offset', 0)+start))
- if item.step is not None:
- return list(res)[None:None:item.step]
- else:
- return res
- else:
- return list(self[item:item+1])[0]
-
- def __iter__(self):
- return iter(self._query.select_whereclause(self._clause, **self._ops))
+ global_extensions.append(SelectResultsExt)
+install_plugin()
--- /dev/null
+from sqlalchemy import util, engine, mapper
+from sqlalchemy.ext.sessioncontext import SessionContext
+import sqlalchemy.ext.assignmapper as assignmapper
+from sqlalchemy.orm.mapper import global_extensions
+from sqlalchemy.orm.session import Session
+import sqlalchemy
+import sys, types
+
+"""this plugin installs thread-local behavior at the Engine and Session level.
+
+The default Engine strategy will be "threadlocal", producing TLocalEngine instances for create_engine by default.
+With this engine, connect() method will return the same connection on the same thread, if it is already checked out
+from the pool. this greatly helps functions that call multiple statements to be able to easily use just one connection
+without explicit "close" statements on result handles.
+
+on the Session side, module-level methods will be installed within the objectstore module, such as flush(), delete(), etc.
+which call this method on the thread-local session.
+
+Note: this mod creates a global, thread-local session context named sqlalchemy.objectstore. All mappers created
+while this mod is installed will reference this global context when creating new mapped object instances.
+"""
+
+class Objectstore(SessionContext):
+ def __getattr__(self, key):
+ return getattr(self.current, key)
+ def get_session(self):
+ return self.current
+
+def assign_mapper(class_, *args, **kwargs):
+ assignmapper.assign_mapper(objectstore, class_, *args, **kwargs)
+
+def _mapper_extension():
+ return SessionContext._get_mapper_extension(objectstore)
+
+objectstore = Objectstore(Session)
+def install_plugin():
+ sqlalchemy.objectstore = objectstore
+ global_extensions.append(_mapper_extension)
+ engine.default_strategy = 'threadlocal'
+ sqlalchemy.assign_mapper = assign_mapper
+
+def uninstall_plugin():
+ engine.default_strategy = 'plain'
+ global_extensions.remove(_mapper_extension)
+
+install_plugin()
the mapper package provides object-relational functionality, building upon the schema and sql
packages and tying operations to class properties and constructors.
"""
-import sqlalchemy.sql as sql
-import sqlalchemy.schema as schema
-import sqlalchemy.engine as engine
-import sqlalchemy.util as util
-import objectstore
-from exceptions import *
-import types as types
+from sqlalchemy import sql, schema, engine, util, exceptions
from mapper import *
-from properties import *
-import mapper as mapperlib
+from mapper import mapper_registry
+from query import Query
+from util import polymorphic_union
+import properties
+from session import Session as create_session
__all__ = ['relation', 'backref', 'eagerload', 'lazyload', 'noload', 'deferred', 'defer', 'undefer',
- 'mapper', 'clear_mappers', 'objectstore', 'sql', 'extension', 'class_mapper', 'object_mapper', 'MapperExtension',
- 'assign_mapper', 'cascade_mappers'
+ 'mapper', 'clear_mappers', 'sql', 'extension', 'class_mapper', 'object_mapper', 'MapperExtension', 'Query',
+ 'cascade_mappers', 'polymorphic_union', 'create_session',
]
def relation(*args, **kwargs):
"""provides a relationship of a primary Mapper to a secondary Mapper, which corresponds
to a parent-child or associative table relationship."""
if len(args) > 1 and isinstance(args[0], type):
- raise ArgumentError("relation(class, table, **kwargs) is deprecated. Please use relation(class, **kwargs) or relation(mapper, **kwargs).")
+ raise exceptions.ArgumentError("relation(class, table, **kwargs) is deprecated. Please use relation(class, **kwargs) or relation(mapper, **kwargs).")
return _relation_loader(*args, **kwargs)
def _relation_loader(mapper, secondary=None, primaryjoin=None, secondaryjoin=None, lazy=True, **kwargs):
if lazy:
- return LazyLoader(mapper, secondary, primaryjoin, secondaryjoin, **kwargs)
+ return properties.LazyLoader(mapper, secondary, primaryjoin, secondaryjoin, **kwargs)
elif lazy is None:
- return PropertyLoader(mapper, secondary, primaryjoin, secondaryjoin, **kwargs)
+ return properties.PropertyLoader(mapper, secondary, primaryjoin, secondaryjoin, **kwargs)
else:
- return EagerLoader(mapper, secondary, primaryjoin, secondaryjoin, **kwargs)
+ return properties.EagerLoader(mapper, secondary, primaryjoin, secondaryjoin, **kwargs)
def backref(name, **kwargs):
- return BackRef(name, **kwargs)
+ return properties.BackRef(name, **kwargs)
def deferred(*columns, **kwargs):
"""returns a DeferredColumnProperty, which indicates this object attributes should only be loaded
from its corresponding table column when first accessed."""
- return DeferredColumnProperty(*columns, **kwargs)
+ return properties.DeferredColumnProperty(*columns, **kwargs)
def mapper(class_, table=None, *args, **params):
- """returns a new or already cached Mapper object."""
- if table is None:
- return class_mapper(class_)
-
+ """returns a newMapper object."""
return Mapper(class_, table, *args, **params)
def clear_mappers():
def eagerload(name, **kwargs):
"""returns a MapperOption that will convert the property of the given name
into an eager load. Used with mapper.options()"""
- return EagerLazyOption(name, toeager=True, **kwargs)
+ return properties.EagerLazyOption(name, toeager=True, **kwargs)
def lazyload(name, **kwargs):
"""returns a MapperOption that will convert the property of the given name
into a lazy load. Used with mapper.options()"""
- return EagerLazyOption(name, toeager=False, **kwargs)
+ return properties.EagerLazyOption(name, toeager=False, **kwargs)
def noload(name, **kwargs):
"""returns a MapperOption that will convert the property of the given name
into a non-load. Used with mapper.options()"""
- return EagerLazyOption(name, toeager=None, **kwargs)
+ return properties.EagerLazyOption(name, toeager=None, **kwargs)
def defer(name, **kwargs):
"""returns a MapperOption that will convert the column property of the given
name into a deferred load. Used with mapper.options()"""
- return DeferredOption(name, defer=True)
+ return properties.DeferredOption(name, defer=True)
def undefer(name, **kwargs):
"""returns a MapperOption that will convert the column property of the given
name into a non-deferred (regular column) load. Used with mapper.options."""
- return DeferredOption(name, defer=False)
+ return properties.DeferredOption(name, defer=False)
-def assign_mapper(class_, *args, **params):
- params.setdefault("is_primary", True)
- if not isinstance(getattr(class_, '__init__'), types.MethodType):
- def __init__(self, **kwargs):
- for key, value in kwargs.items():
- setattr(self, key, value)
- class_.__init__ = __init__
- m = mapper(class_, *args, **params)
- class_.mapper = m
- class_.get = m.get
- class_.select = m.select
- class_.select_by = m.select_by
- class_.selectone = m.selectone
- class_.get_by = m.get_by
- def commit(self):
- objectstore.commit(self)
- def delete(self):
- objectstore.delete(self)
- def expire(self):
- objectstore.expire(self)
- def refresh(self):
- objectstore.refresh(self)
- def expunge(self):
- objectstore.expunge(self)
- class_.commit = commit
- class_.delete = delete
- class_.expire = expire
- class_.refresh = refresh
- class_.expunge = expunge
def cascade_mappers(*classes_or_mappers):
"""given a list of classes and/or mappers, identifies the foreign key relationships
--- /dev/null
+# orm/dependency.py
+# Copyright (C) 2005,2006 Michael Bayer mike_mp@zzzcomputing.com
+#
+# This module is part of SQLAlchemy and is released under
+# the MIT License: http://www.opensource.org/licenses/mit-license.php
+
+
+"""bridges the PropertyLoader (i.e. a relation()) and the UOWTransaction
+together to allow processing of scalar- and list-based dependencies at flush time."""
+
+from sync import ONETOMANY,MANYTOONE,MANYTOMANY
+from sqlalchemy import sql
+
+def create_dependency_processor(key, syncrules, cascade, secondary=None, association=None, is_backref=False, post_update=False):
+ types = {
+ ONETOMANY : OneToManyDP,
+ MANYTOONE: ManyToOneDP,
+ MANYTOMANY : ManyToManyDP,
+ }
+ if association is not None:
+ return AssociationDP(key, syncrules, cascade, secondary, association, is_backref, post_update)
+ else:
+ return types[syncrules.direction](key, syncrules, cascade, secondary, association, is_backref, post_update)
+
+class DependencyProcessor(object):
+ def __init__(self, key, syncrules, cascade, secondary=None, association=None, is_backref=False, post_update=False):
+ # TODO: update instance variable names to be more meaningful
+ self.syncrules = syncrules
+ self.cascade = cascade
+ self.mapper = syncrules.child_mapper
+ self.parent = syncrules.parent_mapper
+ self.association = association
+ self.secondary = secondary
+ self.direction = syncrules.direction
+ self.is_backref = is_backref
+ self.post_update = post_update
+ self.key = key
+
+ def register_dependencies(self, uowcommit):
+ """tells a UOWTransaction what mappers are dependent on which, with regards
+ to the two or three mappers handled by this PropertyLoader.
+
+ Also registers itself as a "processor" for one of its mappers, which
+ will be executed after that mapper's objects have been saved or before
+ they've been deleted. The process operation manages attributes and dependent
+ operations upon the objects of one of the involved mappers."""
+ raise NotImplementedError()
+
+ def whose_dependent_on_who(self, obj1, obj2):
+ """given an object pair assuming obj2 is a child of obj1, returns a tuple
+ with the dependent object second, or None if they are equal.
+ used by objectstore's object-level topological sort (i.e. cyclical
+ table dependency)."""
+ if obj1 is obj2:
+ return None
+ elif self.direction == ONETOMANY:
+ return (obj1, obj2)
+ else:
+ return (obj2, obj1)
+
+ def process_dependencies(self, task, deplist, uowcommit, delete = False):
+ """this method is called during a flush operation to synchronize data between a parent and child object.
+ it is called within the context of the various mappers and sometimes individual objects sorted according to their
+ insert/update/delete order (topological sort)."""
+ raise NotImplementedError()
+
+ def preprocess_dependencies(self, task, deplist, uowcommit, delete = False):
+ """used before the flushes' topological sort to traverse through related objects and insure every
+ instance which will require save/update/delete is properly added to the UOWTransaction."""
+ raise NotImplementedError()
+
+ def _synchronize(self, obj, child, associationrow, clearkeys):
+ """called during a flush to synchronize primary key identifier values between a parent/child object, as well as
+ to an associationrow in the case of many-to-many."""
+ raise NotImplementedError()
+
+ def get_object_dependencies(self, obj, uowcommit, passive = True):
+ """returns the list of objects that are dependent on the given object, as according to the relationship
+ this dependency processor represents"""
+ return uowcommit.uow.attributes.get_history(obj, self.key, passive = passive)
+
+
+class OneToManyDP(DependencyProcessor):
+ def register_dependencies(self, uowcommit):
+ if self.post_update:
+ stub = MapperStub(self.mapper)
+ uowcommit.register_dependency(self.mapper, stub)
+ uowcommit.register_dependency(self.parent, stub)
+ uowcommit.register_processor(stub, self, self.parent)
+ else:
+ uowcommit.register_dependency(self.parent, self.mapper)
+ uowcommit.register_processor(self.parent, self, self.parent)
+ def process_dependencies(self, task, deplist, uowcommit, delete = False):
+ #print self.mapper.table.name + " " + self.key + " " + repr(len(deplist)) + " process_dep isdelete " + repr(delete) + " direction " + repr(self.direction)
+ if delete:
+ # head object is being deleted, and we manage its list of child objects
+ # the child objects have to have their foreign key to the parent set to NULL
+ if not self.cascade.delete_orphan or self.post_update:
+ for obj in deplist:
+ childlist = self.get_object_dependencies(obj, uowcommit, passive=False)
+ for child in childlist.deleted_items():
+ if child is not None and childlist.hasparent(child) is False:
+ self._synchronize(obj, child, None, True)
+ if self.post_update:
+ uowcommit.register_object(child, postupdate=True)
+ for child in childlist.unchanged_items():
+ if child is not None:
+ self._synchronize(obj, child, None, True)
+ if self.post_update:
+ uowcommit.register_object(child, postupdate=True)
+ else:
+ for obj in deplist:
+ childlist = self.get_object_dependencies(obj, uowcommit, passive=True)
+ if childlist is not None:
+ for child in childlist.added_items():
+ self._synchronize(obj, child, None, False)
+ if child is not None and self.post_update:
+ uowcommit.register_object(child, postupdate=True)
+ for child in childlist.deleted_items():
+ if not self.cascade.delete_orphan:
+ self._synchronize(obj, child, None, True)
+
+ def preprocess_dependencies(self, task, deplist, uowcommit, delete = False):
+ #print self.mapper.table.name + " " + self.key + " " + repr(len(deplist)) + " process_dep isdelete " + repr(delete) + " direction " + repr(self.direction)
+
+ if delete:
+ # head object is being deleted, and we manage its list of child objects
+ # the child objects have to have their foreign key to the parent set to NULL
+ if self.post_update:
+ # TODO: post_update instructions should be established in this step as well
+ # (and executed in the regular traversal)
+ pass
+ elif self.cascade.delete_orphan:
+ for obj in deplist:
+ childlist = self.get_object_dependencies(obj, uowcommit, passive=False)
+ for child in childlist.deleted_items():
+ if child is not None and childlist.hasparent(child) is False:
+ uowcommit.register_object(child, isdelete=True)
+ for c in self.mapper.cascade_iterator('delete', child):
+ uowcommit.register_object(c, isdelete=True)
+ for child in childlist.unchanged_items():
+ if child is not None:
+ uowcommit.register_object(child, isdelete=True)
+ for c in self.mapper.cascade_iterator('delete', child):
+ uowcommit.register_object(c, isdelete=True)
+ else:
+ for obj in deplist:
+ childlist = self.get_object_dependencies(obj, uowcommit, passive=False)
+ for child in childlist.deleted_items():
+ if child is not None and childlist.hasparent(child) is False:
+ uowcommit.register_object(child)
+ for child in childlist.unchanged_items():
+ if child is not None:
+ uowcommit.register_object(child)
+ else:
+ for obj in deplist:
+ childlist = self.get_object_dependencies(obj, uowcommit, passive=True)
+ if childlist is not None:
+ for child in childlist.added_items():
+ if child is not None:
+ uowcommit.register_object(child)
+ for child in childlist.deleted_items():
+ if not self.cascade.delete_orphan:
+ uowcommit.register_object(child, isdelete=False)
+ elif childlist.hasparent(child) is False:
+ uowcommit.register_object(child, isdelete=True)
+ for c in self.mapper.cascade_iterator('delete', child):
+ uowcommit.register_object(c, isdelete=True)
+
+ def _synchronize(self, obj, child, associationrow, clearkeys):
+ source = obj
+ dest = child
+ if dest is None:
+ return
+ self.syncrules.execute(source, dest, obj, child, clearkeys)
+
+class ManyToOneDP(DependencyProcessor):
+ def register_dependencies(self, uowcommit):
+ if self.post_update:
+ stub = MapperStub(self.mapper)
+ uowcommit.register_dependency(self.mapper, stub)
+ uowcommit.register_dependency(self.parent, stub)
+ uowcommit.register_processor(stub, self, self.parent)
+ else:
+ uowcommit.register_dependency(self.mapper, self.parent)
+ uowcommit.register_processor(self.mapper, self, self.parent)
+ def process_dependencies(self, task, deplist, uowcommit, delete = False):
+ #print self.mapper.mapped_table.name + " " + self.key + " " + repr(len(deplist)) + " process_dep isdelete " + repr(delete) + " direction " + repr(self.direction)
+ if delete:
+ if self.post_update and not self.cascade.delete_orphan:
+ # post_update means we have to update our row to not reference the child object
+ # before we can DELETE the row
+ for obj in deplist:
+ self._synchronize(obj, None, None, True)
+ uowcommit.register_object(obj, postupdate=True)
+ else:
+ for obj in deplist:
+ childlist = self.get_object_dependencies(obj, uowcommit, passive=True)
+ if childlist is not None:
+ for child in childlist.added_items():
+ self._synchronize(obj, child, None, False)
+ if self.post_update:
+ uowcommit.register_object(obj, postupdate=True)
+
+ def preprocess_dependencies(self, task, deplist, uowcommit, delete = False):
+ #print self.mapper.table.name + " " + self.key + " " + repr(len(deplist)) + " process_dep isdelete " + repr(delete) + " direction " + repr(self.direction)
+ # TODO: post_update instructions should be established in this step as well
+ # (and executed in the regular traversal)
+ if self.post_update:
+ return
+ if delete:
+ if self.cascade.delete:
+ for obj in deplist:
+ childlist = self.get_object_dependencies(obj, uowcommit, passive=False)
+ for child in childlist.deleted_items() + childlist.unchanged_items():
+ if child is not None and childlist.hasparent(child) is False:
+ uowcommit.register_object(child, isdelete=True)
+ for c in self.mapper.cascade_iterator('delete', child):
+ uowcommit.register_object(c, isdelete=True)
+ else:
+ for obj in deplist:
+ uowcommit.register_object(obj)
+ if self.cascade.delete_orphan:
+ childlist = self.get_object_dependencies(obj, uowcommit, passive=False)
+ for child in childlist.deleted_items():
+ if childlist.hasparent(child) is False:
+ uowcommit.register_object(child, isdelete=True)
+ for c in self.mapper.cascade_iterator('delete', child):
+ uowcommit.register_object(c, isdelete=True)
+
+ def _synchronize(self, obj, child, associationrow, clearkeys):
+ source = child
+ dest = obj
+ if dest is None:
+ return
+ self.syncrules.execute(source, dest, obj, child, clearkeys)
+
+class ManyToManyDP(DependencyProcessor):
+ def register_dependencies(self, uowcommit):
+ # many-to-many. create a "Stub" mapper to represent the
+ # "middle table" in the relationship. This stub mapper doesnt save
+ # or delete any objects, but just marks a dependency on the two
+ # related mappers. its dependency processor then populates the
+ # association table.
+
+ if self.is_backref:
+ # if we are the "backref" half of a two-way backref
+ # relationship, let the other mapper handle inserting the rows
+ return
+ stub = MapperStub(self.mapper)
+ uowcommit.register_dependency(self.parent, stub)
+ uowcommit.register_dependency(self.mapper, stub)
+ uowcommit.register_processor(stub, self, self.parent)
+
+ def process_dependencies(self, task, deplist, uowcommit, delete = False):
+ #print self.mapper.table.name + " " + self.key + " " + repr(len(deplist)) + " process_dep isdelete " + repr(delete) + " direction " + repr(self.direction)
+ connection = uowcommit.transaction.connection(self.mapper)
+ secondary_delete = []
+ secondary_insert = []
+ if delete:
+ for obj in deplist:
+ childlist = self.get_object_dependencies(obj, uowcommit, passive=False)
+ for child in childlist.deleted_items() + childlist.unchanged_items():
+ associationrow = {}
+ self._synchronize(obj, child, associationrow, False)
+ secondary_delete.append(associationrow)
+ else:
+ for obj in deplist:
+ childlist = self.get_object_dependencies(obj, uowcommit)
+ if childlist is None: continue
+ for child in childlist.added_items():
+ associationrow = {}
+ self._synchronize(obj, child, associationrow, False)
+ secondary_insert.append(associationrow)
+ for child in childlist.deleted_items():
+ associationrow = {}
+ self._synchronize(obj, child, associationrow, False)
+ secondary_delete.append(associationrow)
+ if len(secondary_delete):
+ # TODO: precompile the delete/insert queries and store them as instance variables
+ # on the PropertyLoader
+ statement = self.secondary.delete(sql.and_(*[c == sql.bindparam(c.key) for c in self.secondary.c]))
+ connection.execute(statement, secondary_delete)
+ if len(secondary_insert):
+ statement = self.secondary.insert()
+ connection.execute(statement, secondary_insert)
+
+ def preprocess_dependencies(self, task, deplist, uowcommit, delete = False):
+ pass
+ def _synchronize(self, obj, child, associationrow, clearkeys):
+ dest = associationrow
+ source = None
+ if dest is None:
+ return
+ self.syncrules.execute(source, dest, obj, child, clearkeys)
+
+class AssociationDP(OneToManyDP):
+ def register_dependencies(self, uowcommit):
+ # association object. our mapper should be dependent on both
+ # the parent mapper and the association object mapper.
+ # this is where we put the "stub" as a marker, so we get
+ # association/parent->stub->self, then we process the child
+ # elments after the 'stub' save, which is before our own
+ # mapper's save.
+ stub = MapperStub(self.association)
+ uowcommit.register_dependency(self.parent, stub)
+ uowcommit.register_dependency(self.association, stub)
+ uowcommit.register_dependency(stub, self.mapper)
+ uowcommit.register_processor(stub, self, self.parent)
+ def process_dependencies(self, task, deplist, uowcommit, delete = False):
+ #print self.mapper.table.name + " " + self.key + " " + repr(len(deplist)) + " process_dep isdelete " + repr(delete) + " direction " + repr(self.direction)
+ # manage association objects.
+ for obj in deplist:
+ childlist = self.get_object_dependencies(obj, uowcommit, passive=True)
+ if childlist is None: continue
+
+ #print "DIRECTION", self.direction
+ d = {}
+ for child in childlist:
+ self._synchronize(obj, child, None, False)
+ key = self.mapper.instance_key(child)
+ #print "SYNCHRONIZED", child, "INSTANCE KEY", key
+ d[key] = child
+ uowcommit.unregister_object(child)
+
+ for child in childlist.added_items():
+ uowcommit.register_object(child)
+ key = self.mapper.instance_key(child)
+ #print "ADDED, INSTANCE KEY", key
+ d[key] = child
+
+ for child in childlist.unchanged_items():
+ key = self.mapper.instance_key(child)
+ o = d[key]
+ o._instance_key= key
+
+ for child in childlist.deleted_items():
+ key = self.mapper.instance_key(child)
+ #print "DELETED, INSTANCE KEY", key
+ if d.has_key(key):
+ o = d[key]
+ o._instance_key = key
+ uowcommit.unregister_object(child)
+ else:
+ #print "DELETE ASSOC OBJ", repr(child)
+ uowcommit.register_object(child, isdelete=True)
+ def preprocess_dependencies(self, task, deplist, uowcommit, delete = False):
+ # TODO: clean up the association step in process_dependencies and move the
+ # appropriate sections of it to here
+ pass
+
+
+class MapperStub(object):
+ """poses as a Mapper representing the association table in a many-to-many
+ join, when performing a flush().
+
+ The Task objects in the objectstore module treat it just like
+ any other Mapper, but in fact it only serves as a "dependency" placeholder
+ for the many-to-many update task."""
+ def __init__(self, mapper):
+ self.mapper = mapper
+ def register_dependencies(self, uowcommit):
+ pass
+ def save_obj(self, *args, **kwargs):
+ pass
+ def delete_obj(self, *args, **kwargs):
+ pass
+ def _primary_mapper(self):
+ return self
-# mapper/mapper.py
+# orm/mapper.py
# Copyright (C) 2005,2006 Michael Bayer mike_mp@zzzcomputing.com
#
# This module is part of SQLAlchemy and is released under
# the MIT License: http://www.opensource.org/licenses/mit-license.php
-
-import sqlalchemy.sql as sql
-import sqlalchemy.schema as schema
-import sqlalchemy.util as util
+from sqlalchemy import sql, schema, util, exceptions
+from sqlalchemy import sql_util as sqlutil
import util as mapperutil
import sync
-from sqlalchemy.exceptions import *
-import query
-import objectstore
-import sys
-import weakref
+import query as querylib
+import session as sessionlib
+import sys, weakref, sets
+
+__all__ = ['Mapper', 'MapperExtension', 'class_mapper', 'object_mapper']
# a dictionary mapping classes to their primary mappers
mapper_registry = weakref.WeakKeyDictionary()
relation() function."""
def __init__(self,
class_,
- table,
- primarytable = None,
+ local_table,
properties = None,
primary_key = None,
is_primary = False,
+ non_primary = False,
inherits = None,
inherit_condition = None,
extension = None,
entity_name = None,
always_refresh = False,
version_id_col = None,
- construct_new = False,
- **kwargs):
+ polymorphic_on=None,
+ polymorphic_map=None,
+ polymorphic_identity=None,
+ concrete=False,
+ select_table=None):
- if primarytable is not None:
- sys.stderr.write("'primarytable' argument to mapper is deprecated\n")
-
- ext = MapperExtension()
-
+ # uber-pendantic style of making mapper chain, as various testbase/
+ # threadlocal/assignmapper combinations keep putting dupes etc. in the list
+ # TODO: do something that isnt 21 lines....
+ extlist = util.HashSet()
for ext_class in global_extensions:
- ext = ext_class().chain(ext)
+ if isinstance(ext_class, MapperExtension):
+ extlist.append(ext_class)
+ else:
+ extlist.append(ext_class())
if extension is not None:
for ext_obj in util.to_list(extension):
- ext = ext_obj.chain(ext)
+ extlist.append(ext_obj)
+
+ self.extension = None
+ previous = None
+ for ext in extlist:
+ if self.extension is None:
+ self.extension = ext
+ if previous is not None:
+ previous.chain(ext)
+ previous = ext
+ if self.extension is None:
+ self.extension = MapperExtension()
- self.extension = ext
-
self.class_ = class_
self.entity_name = entity_name
self.class_key = ClassKey(class_, entity_name)
self.is_primary = is_primary
+ self.non_primary = non_primary
self.order_by = order_by
self._options = {}
self.always_refresh = always_refresh
self.version_id_col = version_id_col
- self.construct_new = construct_new
+ self._inheriting_mappers = sets.Set()
+ self.polymorphic_on = polymorphic_on
+ if polymorphic_map is None:
+ self.polymorphic_map = {}
+ else:
+ self.polymorphic_map = polymorphic_map
+ self.__surrogate_mapper = None
+ self._surrogate_parent = None
if not issubclass(class_, object):
- raise ArgumentError("Class '%s' is not a new-style class" % class_.__name__)
-
- if isinstance(table, sql.Select):
- # some db's, noteably postgres, dont want to select from a select
- # without an alias. also if we make our own alias internally, then
- # the configured properties on the mapper are not matched against the alias
- # we make, theres workarounds but it starts to get really crazy (its crazy enough
- # the SQL that gets generated) so just require an alias
- raise ArgumentError("Mapping against a Select object requires that it has a name. Use an alias to give it a name, i.e. s = select(...).alias('myselect')")
- else:
- self.table = table
+ raise exceptions.ArgumentError("Class '%s' is not a new-style class" % class_.__name__)
+
+ # set up various Selectable units:
+
+ # mapped_table - the Selectable that represents a join of the underlying Tables to be saved (or just the Table)
+ # local_table - the Selectable that was passed to this Mapper's constructor, if any
+ # select_table - the Selectable that will be used during queries. if this is specified
+ # as a constructor keyword argument, it takes precendence over mapped_table, otherwise its mapped_table
+ # unjoined_table - our Selectable, minus any joins constructed against the inherits table.
+ # this is either select_table if it was given explicitly, or in the case of a mapper that inherits
+ # its local_table
+ # tables - a collection of underlying Table objects pulled from mapped_table
+
+ for table in (local_table, select_table):
+ if table is not None and isinstance(local_table, sql.SelectBaseMixin):
+ # some db's, noteably postgres, dont want to select from a select
+ # without an alias. also if we make our own alias internally, then
+ # the configured properties on the mapper are not matched against the alias
+ # we make, theres workarounds but it starts to get really crazy (its crazy enough
+ # the SQL that gets generated) so just require an alias
+ raise exceptions.ArgumentError("Mapping against a Select object requires that it has a name. Use an alias to give it a name, i.e. s = select(...).alias('myselect')")
+
+ self.local_table = local_table
if inherits is not None:
+ if isinstance(inherits, type):
+ inherits = class_mapper(inherits)
if self.class_.__mro__[1] != inherits.class_:
- raise ArgumentError("Class '%s' does not inherit from '%s'" % (self.class_.__name__, inherits.class_.__name__))
- self.primarytable = inherits.primarytable
+ raise exceptions.ArgumentError("Class '%s' does not inherit from '%s'" % (self.class_.__name__, inherits.class_.__name__))
# inherit_condition is optional.
- if not table is inherits.noninherited_table:
- if inherit_condition is None:
- # figure out inherit condition from our table to the immediate table
- # of the inherited mapper, not its full table which could pull in other
- # stuff we dont want (allows test/inheritance.InheritTest4 to pass)
- inherit_condition = sql.join(inherits.noninherited_table, table).onclause
- self.table = sql.join(inherits.table, table, inherit_condition)
- #print "inherit condition", str(self.table.onclause)
-
- # generate sync rules. similarly to creating the on clause, specify a
- # stricter set of tables to create "sync rules" by,based on the immediate
- # inherited table, rather than all inherited tables
- self._synchronizer = sync.ClauseSynchronizer(self, self, sync.ONETOMANY)
- self._synchronizer.compile(self.table.onclause, util.HashSet([inherits.noninherited_table]), mapperutil.TableFinder(table))
- # the old rule
- #self._synchronizer.compile(self.table.onclause, inherits.tables, TableFinder(table))
+ if local_table is None:
+ self.local_table = local_table = inherits.local_table
+ if not local_table is inherits.local_table:
+ if concrete:
+ self._synchronizer= None
+ self.mapped_table = self.local_table
+ else:
+ if inherit_condition is None:
+ # figure out inherit condition from our table to the immediate table
+ # of the inherited mapper, not its full table which could pull in other
+ # stuff we dont want (allows test/inheritance.InheritTest4 to pass)
+ inherit_condition = sql.join(inherits.local_table, self.local_table).onclause
+ self.mapped_table = sql.join(inherits.mapped_table, self.local_table, inherit_condition)
+ #print "inherit condition", str(self.table.onclause)
+
+ # generate sync rules. similarly to creating the on clause, specify a
+ # stricter set of tables to create "sync rules" by,based on the immediate
+ # inherited table, rather than all inherited tables
+ self._synchronizer = sync.ClauseSynchronizer(self, self, sync.ONETOMANY)
+ self._synchronizer.compile(self.mapped_table.onclause, util.HashSet([inherits.local_table]), sqlutil.TableFinder(self.local_table))
else:
self._synchronizer = None
+ self.mapped_table = self.local_table
self.inherits = inherits
- self.noninherited_table = table
+ if polymorphic_identity is not None:
+ inherits.add_polymorphic_mapping(polymorphic_identity, self)
+ self.polymorphic_identity = polymorphic_identity
+ if self.polymorphic_on is None:
+ self.effective_polymorphic_on = inherits.effective_polymorphic_on
+ else:
+ self.effective_polymorphic_on = self.polymorphic_on
if self.order_by is False:
self.order_by = inherits.order_by
else:
- self.primarytable = self.table
- self.noninherited_table = self.table
self._synchronizer = None
self.inherits = None
+ self.mapped_table = self.local_table
+ if polymorphic_identity is not None:
+ self.add_polymorphic_mapping(polymorphic_identity, self)
+ self.polymorphic_identity = polymorphic_identity
+ self.effective_polymorphic_on = self.polymorphic_on
+
+ if select_table is not None:
+ self.select_table = select_table
+ else:
+ self.select_table = self.mapped_table
+ self.unjoined_table = self.local_table
# locate all tables contained within the "table" passed in, which
# may be a join or other construct
- self.tables = mapperutil.TableFinder(self.table)
+ self.tables = sqlutil.TableFinder(self.mapped_table)
# determine primary key columns, either passed in, or get them from our set of tables
self.pks_by_table = {}
if primary_key is not None:
for k in primary_key:
self.pks_by_table.setdefault(k.table, util.HashSet(ordered=True)).append(k)
- if k.table != self.table:
+ if k.table != self.mapped_table:
# associate pk cols from subtables to the "main" table
- self.pks_by_table.setdefault(self.table, util.HashSet(ordered=True)).append(k)
+ self.pks_by_table.setdefault(self.mapped_table, util.HashSet(ordered=True)).append(k)
+ # TODO: need local_table properly accounted for when custom primary key is sent
else:
- for t in self.tables + [self.table]:
+ for t in self.tables + [self.mapped_table]:
try:
l = self.pks_by_table[t]
except KeyError:
l = self.pks_by_table.setdefault(t, util.HashSet(ordered=True))
if not len(t.primary_key):
- raise ArgumentError("Table " + t.name + " has no primary key columns. Specify primary_key argument to mapper.")
+ raise exceptions.ArgumentError("Table " + t.name + " has no primary key columns. Specify primary_key argument to mapper.")
for k in t.primary_key:
l.append(k)
# table columns mapped to lists of MapperProperty objects
# using a list allows a single column to be defined as
# populating multiple object attributes
- self.columntoproperty = {}
+ self.columntoproperty = TranslatingDict(self.mapped_table)
# load custom properties
if properties is not None:
for key, prop in properties.iteritems():
- if sql.is_column(prop):
- try:
- prop = self.table._get_col_by_original(prop)
- except KeyError:
- raise ArgumentError("Column '%s' is not represented in mapper's table" % prop._label)
- self.columns[key] = prop
- prop = ColumnProperty(prop)
- elif isinstance(prop, list) and sql.is_column(prop[0]):
- try:
- prop = [self.table._get_col_by_original(p) for p in prop]
- except KeyError, e:
- raise ArgumentError("Column '%s' is not represented in mapper's table" % e.args[0])
- self.columns[key] = prop[0]
- prop = ColumnProperty(*prop)
- self.props[key] = prop
- if isinstance(prop, ColumnProperty):
- for col in prop.columns:
- proplist = self.columntoproperty.setdefault(col.original, [])
- proplist.append(prop)
+ self.add_property(key, prop, False)
+
+ if inherits is not None:
+ inherits._inheriting_mappers.add(self)
+ for key, prop in inherits.props.iteritems():
+ if not self.props.has_key(key):
+ p = prop.copy()
+ if p.adapt(self):
+ self.add_property(key, p, init=False)
# load properties from the main table object,
# not overriding those set up in the 'properties' argument
- for column in self.table.columns:
+ for column in self.mapped_table.columns:
+ if self.columntoproperty.has_key(column):
+ continue
if not self.columns.has_key(column.key):
self.columns[column.key] = column
- if self.columntoproperty.has_key(column.original):
- continue
-
prop = self.props.get(column.key, None)
if prop is None:
prop = ColumnProperty(column)
# column at index 0 determines which result column is used to populate the object
# attribute, in the case of mapping against a join with column names repeated
# (and particularly in an inheritance relationship)
+ # TODO: clarify this comment
prop.columns.insert(0, column)
#prop.columns.append(column)
else:
if not allow_column_override:
- raise ArgumentError("WARNING: column '%s' not being added due to property '%s'. Specify 'allow_column_override=True' to mapper() to ignore this condition." % (column.key, repr(prop)))
+ raise exceptions.ArgumentError("WARNING: column '%s' not being added due to property '%s'. Specify 'allow_column_override=True' to mapper() to ignore this condition." % (column.key, repr(prop)))
else:
continue
# its a ColumnProperty - match the ultimate table columns
# back to the property
- proplist = self.columntoproperty.setdefault(column.original, [])
+ proplist = self.columntoproperty.setdefault(column, [])
proplist.append(prop)
-
- if not mapper_registry.has_key(self.class_key) or self.is_primary or (inherits is not None and inherits._is_primary_mapper()):
- objectstore.global_attributes.reset_class_managed(self.class_)
- self._init_class()
- if inherits is not None:
- for key, prop in inherits.props.iteritems():
- if not self.props.has_key(key):
- self.props[key] = prop.copy()
- self.props[key].parent = self
- # self.props[key].key = None # force re-init
+ if not non_primary and (not mapper_registry.has_key(self.class_key) or self.is_primary or (inherits is not None and inherits._is_primary_mapper())):
+ sessionlib.global_attributes.reset_class_managed(self.class_)
+ self._init_class()
+ elif not non_primary:
+ raise exceptions.ArgumentError("Class '%s' already has a primary mapper defined. Use is_primary=True to assign a new primary mapper to the class, or use non_primary=True to create a non primary Mapper" % self.class_)
+
+ for key in self.polymorphic_map.keys():
+ if isinstance(self.polymorphic_map[key], type):
+ self.polymorphic_map[key] = class_mapper(self.polymorphic_map[key])
+
l = [(key, prop) for key, prop in self.props.iteritems()]
for key, prop in l:
if getattr(prop, 'key', None) is None:
prop.init(key, self)
- # this prints a summary of the object attributes and how they
- # will be mapped to table columns
- #print "mapper %s, columntoproperty:" % (self.class_.__name__)
- #for key, value in self.columntoproperty.iteritems():
- # print key.table.name, key.key, [(v.key, v) for v in value]
-
- def _get_query(self):
- try:
- if self._query.mapper is not self:
- self._query = query.Query(self)
- return self._query
- except AttributeError:
- self._query = query.Query(self)
- return self._query
- query = property(_get_query, doc=\
- """returns an instance of sqlalchemy.mapping.query.Query, which implements all the query-constructing
- methods such as get(), select(), select_by(), etc. The default Query object uses the global thread-local
- Session from the objectstore package. To get a Query object for a specific Session, call the
- using(session) method.""")
+ # select_table specified...set up a surrogate mapper that will be used for selects
+ # select_table has to encompass all the columns of the mapped_table either directly
+ # or through proxying relationships
+ if self.select_table is not self.mapped_table:
+ props = {}
+ if properties is not None:
+ for key, prop in properties.iteritems():
+ if sql.is_column(prop):
+ props[key] = self.select_table.corresponding_column(prop)
+ elif (isinstance(column, list) and sql.is_column(column[0])):
+ props[key] = [self.select_table.corresponding_column(c) for c in prop]
+ self.__surrogate_mapper = Mapper(self.class_, self.select_table, non_primary=True, properties=props, polymorphic_map=self.polymorphic_map, polymorphic_on=self.polymorphic_on)
+
+ def add_polymorphic_mapping(self, key, class_or_mapper, entity_name=None):
+ if isinstance(class_or_mapper, type):
+ class_or_mapper = class_mapper(class_or_mapper, entity_name=entity_name)
+ self.polymorphic_map[key] = class_or_mapper
- def get(self, *ident, **kwargs):
- """calls get() on this mapper's default Query object."""
- return self.query.get(*ident, **kwargs)
-
- def _get(self, key, ident=None, reload=False):
- return self.query._get(key, ident=ident, reload=reload)
-
- def get_by(self, *args, **params):
- """calls get_by() on this mapper's default Query object."""
- return self.query.get_by(*args, **params)
-
- def select_by(self, *args, **params):
- """calls select_by() on this mapper's default Query object."""
- return self.query.select_by(*args, **params)
-
- def selectfirst_by(self, *args, **params):
- """calls selectfirst_by() on this mapper's default Query object."""
- return self.query.selectfirst_by(*args, **params)
-
- def selectone_by(self, *args, **params):
- """calls selectone_by() on this mapper's default Query object."""
- return self.query.selectone_by(*args, **params)
-
- def count_by(self, *args, **params):
- """calls count_by() on this mapper's default Query object."""
- return self.query.count_by(*args, **params)
-
- def selectfirst(self, *args, **params):
- """calls selectfirst() on this mapper's default Query object."""
- return self.query.selectfirst(*args, **params)
-
- def selectone(self, *args, **params):
- """calls selectone() on this mapper's default Query object."""
- return self.query.selectone(*args, **params)
-
- def select(self, arg=None, **kwargs):
- """calls select() on this mapper's default Query object."""
- return self.query.select(arg=arg, **kwargs)
-
- def select_whereclause(self, whereclause=None, params=None, **kwargs):
- """calls select_whereclause() on this mapper's default Query object."""
- return self.query.select_whereclause(whereclause=whereclause, params=params, **kwargs)
-
- def count(self, whereclause=None, params=None, **kwargs):
- """calls count() on this mapper's default Query object."""
- return self.query.count(whereclause=whereclause, params=params, **kwargs)
-
- def select_statement(self, statement, **params):
- """calls select_statement() on this mapper's default Query object."""
- return self.query.select_statement(statement, **params)
-
- def select_text(self, text, **params):
- return self.query.select_text(text, **params)
+ def add_properties(self, dict_of_properties):
+ """adds the given dictionary of properties to this mapper, using add_property."""
+ for key, value in dict_of_properties.iteritems():
+ self.add_property(key, value, True)
+
+ def _create_prop_from_column(self, column, skipmissing=False):
+ if sql.is_column(column):
+ try:
+ column = self.mapped_table.corresponding_column(column)
+ except KeyError:
+ if skipmissing:
+ return
+ raise exceptions.ArgumentError("Column '%s' is not represented in mapper's table" % prop._label)
+ return ColumnProperty(column)
+ elif isinstance(column, list) and sql.is_column(column[0]):
+ try:
+ column = [self.mapped_table.corresponding_column(c) for c in column]
+ except KeyError, e:
+ # TODO: want to take the columns we have from this
+ if skipmissing:
+ return
+ raise exceptions.ArgumentError("Column '%s' is not represented in mapper's table" % e.args[0])
+ return ColumnProperty(*column)
+ else:
+ return None
- def add_property(self, key, prop):
+ def add_property(self, key, prop, init=True, skipmissing=False):
"""adds an additional property to this mapper. this is the same as if it were
specified within the 'properties' argument to the constructor. if the named
property already exists, this will replace it. Useful for
circular relationships, or overriding the parameters of auto-generated properties
such as backreferences."""
- if sql.is_column(prop):
- self.columns[key] = prop
- prop = ColumnProperty(prop)
+
+ if not isinstance(prop, MapperProperty):
+ prop = self._create_prop_from_column(prop, skipmissing=skipmissing)
+ if prop is None:
+ raise exceptions.ArgumentError("'%s' is not an instance of MapperProperty or Column" % repr(prop))
+
self.props[key] = prop
+
if isinstance(prop, ColumnProperty):
+ self.columns[key] = prop.columns[0]
for col in prop.columns:
- proplist = self.columntoproperty.setdefault(col.original, [])
+ proplist = self.columntoproperty.setdefault(col, [])
proplist.append(prop)
- prop.init(key, self)
+
+ if init:
+ prop.init(key, self)
+
+ for mapper in self._inheriting_mappers:
+ p = prop.copy()
+ if p.adapt(mapper):
+ mapper.add_property(key, p, init=False)
def __str__(self):
- return "Mapper|" + self.class_.__name__ + "|" + (self.entity_name is not None and "/%s" % self.entity_name or "") + self.primarytable.name
+ return "Mapper|" + self.class_.__name__ + "|" + (self.entity_name is not None and "/%s" % self.entity_name or "") + self.mapped_table.name
def _is_primary_mapper(self):
"""returns True if this mapper is the primary mapper for its class key (class + entity_name)"""
return mapper_registry[self.class_key]
def is_assigned(self, instance):
- """returns True if this mapper is the primary mapper for the given instance. this is dependent
+ """returns True if this mapper handles the given instance. this is dependent
not only on class assignment but the optional "entity_name" parameter as well."""
return instance.__class__ is self.class_ and getattr(instance, '_entity_name', None) == self.entity_name
+ def _assign_entity_name(self, instance):
+ """assigns this Mapper's entity name to the given instance. subsequent Mapper lookups for this
+ instance will return the primary mapper corresponding to this Mapper's class and entity name."""
+ instance._entity_name = self.entity_name
+
def _init_class(self):
- """sets up our classes' overridden __init__ method, this mappers hash key as its
- '_mapper' property, and our columns as its 'c' property. if the class already had a
- mapper, the old __init__ method is kept the same."""
+ """decorates the __init__ method on the mapped class to include auto-session attachment logic,
+ and assocites this Mapper with its class via the mapper_registry."""
oldinit = self.class_.__init__
def init(self, *args, **kwargs):
self._entity_name = kwargs.pop('_sa_entity_name', None)
# this gets the AttributeManager to do some pre-initialization,
# in order to save on KeyErrors later on
- objectstore.global_attributes.init_attr(self)
+ sessionlib.global_attributes.init_attr(self)
- nohist = kwargs.pop('_mapper_nohistory', False)
- session = kwargs.pop('_sa_session', objectstore.get_session())
- if not nohist:
- # register new with the correct session, before the object's
- # constructor is called, since further assignments within the
- # constructor would otherwise bind it to whatever get_session() is.
- session.register_new(self)
+ if kwargs.has_key('_sa_session'):
+ session = kwargs.pop('_sa_session')
+ else:
+ # works for whatever mapper the class is associated with
+ mapper = mapper_registry.get(ClassKey(self.__class__, self._entity_name))
+ if mapper is not None:
+ session = mapper.extension.get_session()
+ if session is EXT_PASS:
+ session = None
+ else:
+ session = None
+ if session is not None:
+ session._register_new(self)
if oldinit is not None:
oldinit(self, *args, **kwargs)
- # override oldinit, insuring that its not already one of our
- # own modified inits
+ # override oldinit, insuring that its not already a Mapper-decorated init method
if oldinit is None or not hasattr(oldinit, '_sa_mapper_init'):
init._sa_mapper_init = True
self.class_.__init__ = init
mapper_registry[self.class_key] = self
if self.entity_name is None:
self.class_.c = self.c
+
+ def get_session(self):
+ """returns the contextual session provided by the mapper extension chain
+
+ raises InvalidRequestError if a session cannot be retrieved from the
+ extension chain
+ """
+ s = self.extension.get_session()
+ if s is EXT_PASS:
+ raise exceptions.InvalidRequestError("No contextual Session is established. Use a MapperExtension that implements get_session or use 'import sqlalchemy.mods.threadlocal' to establish a default thread-local contextual session.")
+ return s
def has_eager(self):
"""returns True if one of the properties attached to this Mapper is eager loading"""
self.props[key] = prop
prop.init(key, self)
- def instances(self, cursor, *mappers, **kwargs):
+ def instances(self, cursor, session, *mappers, **kwargs):
"""given a cursor (ResultProxy) from an SQLEngine, returns a list of object instances
corresponding to the rows in the cursor."""
limit = kwargs.get('limit', None)
offset = kwargs.get('offset', None)
- session = kwargs.get('session', None)
- if session is None:
- session = objectstore.get_session()
populate_existing = kwargs.get('populate_existing', False)
result = util.HistoryArraySet()
# store new stuff in the identity map
for value in imap.values():
- session.register_clean(value)
+ session._register_clean(value)
if mappers:
result = [result] + otherresults
return result
- def identity_key(self, *primary_key):
- """returns the instance key for the given identity value. this is a global tracking object used by the objectstore, and is usually available off a mapped object as instance._instance_key."""
- return objectstore.get_id_key(tuple(primary_key), self.class_, self.entity_name)
-
+ def identity_key(self, primary_key):
+ """returns the instance key for the given identity value. this is a global tracking object used by the Session, and is usually available off a mapped object as instance._instance_key."""
+ return sessionlib.get_id_key(util.to_list(primary_key), self.class_, self.entity_name)
+
def instance_key(self, instance):
- """returns the instance key for the given instance. this is a global tracking object used by the objectstore, and is usually available off a mapped object as instance._instance_key."""
- return self.identity_key(*self.identity(instance))
+ """returns the instance key for the given instance. this is a global tracking object used by the Session, and is usually available off a mapped object as instance._instance_key."""
+ return self.identity_key(self.identity(instance))
def identity(self, instance):
"""returns the identity (list of primary key values) for the given instance. The list of values can be fed directly into the get() method as mapper.get(*key)."""
- return [self._getattrbycolumn(instance, column) for column in self.pks_by_table[self.table]]
+ return [self._getattrbycolumn(instance, column) for column in self.pks_by_table[self.mapped_table]]
- def compile(self, whereclause = None, **options):
- """works like select, except returns the SQL statement object without
- compiling or executing it"""
- return self.query._compile(whereclause, **options)
def copy(self, **kwargs):
mapper = Mapper.__new__(Mapper)
mapper.__dict__.update(kwargs)
mapper.props = self.props.copy()
return mapper
-
- def using(self, session):
- """returns a new Query object with the given Session."""
- if objectstore.get_session() is session:
- return self.query
- else:
- return query.Query(self, session=session)
def options(self, *options, **kwargs):
"""uses this mapper as a prototype for a new mapper with different behavior.
self._options[optkey] = mapper
return mapper
- def _get_criterion(self, key, value):
- """used by select_by to match a key/value pair against
- local properties, column names, or a matching property in this mapper's
- list of relations."""
- if self.props.has_key(key):
- return self.props[key].columns[0] == value
- elif self.table.c.has_key(key):
- return self.table.c[key] == value
- else:
- for prop in self.props.values():
- c = prop.get_criterion(key, value)
- if c is not None:
- return c
- else:
- return None
-
- def __getattr__(self, key):
- if (key.startswith('select_by_') or key.startswith('get_by_')):
- return getattr(self.query, key)
- else:
- raise AttributeError(key)
def _getpropbycolumn(self, column, raiseerror=True):
try:
- prop = self.columntoproperty[column.original]
+ prop = self.columntoproperty[column]
except KeyError:
try:
prop = self.props[column.key]
if not raiseerror:
return None
- raise InvalidRequestError("Column '%s.%s' is not available, due to conflicting property '%s':%s" % (column.table.name, column.name, column.key, repr(prop)))
+ raise exceptions.InvalidRequestError("Column '%s.%s' is not available, due to conflicting property '%s':%s" % (column.table.name, column.name, column.key, repr(prop)))
except KeyError:
if not raiseerror:
return None
- raise InvalidRequestError("No column %s.%s is configured on mapper %s..." % (column.table.name, column.name, str(self)))
+ raise exceptions.InvalidRequestError("No column %s.%s is configured on mapper %s..." % (column.table.name, column.name, str(self)))
return prop[0]
def _getattrbycolumn(self, obj, column, raiseerror=True):
return prop.getattr(obj)
def _setattrbycolumn(self, obj, column, value):
- self.columntoproperty[column.original][0].setattr(obj, value)
-
+ self.columntoproperty[column][0].setattr(obj, value)
+
+ def primary_mapper(self):
+ return mapper_registry[self.class_key]
+
def save_obj(self, objects, uow, postupdate=False):
"""called by a UnitOfWork object to save objects, which involves either an INSERT or
an UPDATE statement for each table used by this mapper, for each element of the
list."""
-
+ #print "SAVE_OBJ MAPPER", self.class_.__name__, objects
+ connection = uow.transaction.connection(self)
for table in self.tables:
- #print "SAVE_OBJ table ", table.name
+ #print "SAVE_OBJ table ", self.class_.__name__, table.name
# looping through our set of tables, which are all "real" tables, as opposed
# to our main table which might be a select statement or something non-writeable
# they are separate execs via execute(), not executemany()
if not self._has_pks(table):
+ #print "NO PKS ?", str(table)
# if we dont have a full set of primary keys for this table, we cant really
# do any CRUD with it, so skip. this occurs if we are mapping against a query
# that joins on other tables so its not really an error condition.
# time"
isinsert = not postupdate and not hasattr(obj, "_instance_key")
if isinsert:
- self.extension.before_insert(self, obj)
+ self.extension.before_insert(self, connection, obj)
else:
- self.extension.before_update(self, obj)
+ self.extension.before_update(self, connection, obj)
hasdata = False
for col in table.columns:
if col is self.version_id_col:
value = self._getattrbycolumn(obj, col)
if value is not None:
params[col.key] = value
+ elif self.effective_polymorphic_on is not None and self.effective_polymorphic_on.shares_lineage(col):
+ if isinsert:
+ value = self.polymorphic_identity
+ if col.default is None or value is not None:
+ params[col.key] = value
else:
# column is not a primary key ?
if not isinsert:
clause.clauses.append(self.version_id_col == sql.bindparam(self.version_id_col._label, type=col.type))
statement = table.update(clause)
rows = 0
+ supports_sane_rowcount = True
for rec in update:
(obj, params) = rec
- c = statement.execute(params)
- self._postfetch(table, obj, c, c.last_updated_params())
- self.extension.after_update(self, obj)
+ c = connection.execute(statement, params)
+ self._postfetch(connection, table, obj, c, c.last_updated_params())
+ self.extension.after_update(self, connection, obj)
rows += c.cursor.rowcount
if c.supports_sane_rowcount() and rows != len(update):
- raise CommitError("ConcurrencyError - updated rowcount %d does not match number of objects updated %d" % (rows, len(update)))
+ raise exceptions.FlushError("ConcurrencyError - updated rowcount %d does not match number of objects updated %d" % (rows, len(update)))
if len(insert):
statement = table.insert()
for rec in insert:
(obj, params) = rec
- c = statement.execute(**params)
+ c = connection.execute(statement, params)
primary_key = c.last_inserted_ids()
if primary_key is not None:
i = 0
if self._getattrbycolumn(obj, col) is None:
self._setattrbycolumn(obj, col, primary_key[i])
i+=1
- self._postfetch(table, obj, c, c.last_inserted_params())
+ self._postfetch(connection, table, obj, c, c.last_inserted_params())
if self._synchronizer is not None:
self._synchronizer.execute(obj, obj)
- self.extension.after_insert(self, obj)
+ self.extension.after_insert(self, connection, obj)
- def _postfetch(self, table, obj, resultproxy, params):
+ def _postfetch(self, connection, table, obj, resultproxy, params):
"""after an INSERT or UPDATE, asks the returned result if PassiveDefaults fired off on the database side
which need to be post-fetched, *or* if pre-exec defaults like ColumnDefaults were fired off
and should be populated into the instance. this is only for non-primary key columns."""
clause = sql.and_()
for p in self.pks_by_table[table]:
clause.clauses.append(p == self._getattrbycolumn(obj, p))
- row = table.select(clause).execute().fetchone()
+ row = connection.execute(table.select(clause), None).fetchone()
for c in table.c:
if self._getattrbycolumn(obj, c, False) is None:
self._setattrbycolumn(obj, c, row[c])
def delete_obj(self, objects, uow):
"""called by a UnitOfWork object to delete objects, which involves a
DELETE statement for each table used by this mapper, for each object in the list."""
+ connection = uow.transaction.connection(self)
+
for table in util.reversed(self.tables):
if not self._has_pks(table):
continue
delete = []
+ deleted_objects = []
for obj in objects:
params = {}
if not hasattr(obj, "_instance_key"):
params[col.key] = self._getattrbycolumn(obj, col)
if self.version_id_col is not None:
params[self.version_id_col.key] = self._getattrbycolumn(obj, self.version_id_col)
- self.extension.before_delete(self, obj)
+ self.extension.before_delete(self, connection, obj)
+ deleted_objects.append(obj)
if len(delete):
clause = sql.and_()
for col in self.pks_by_table[table]:
if self.version_id_col is not None:
clause.clauses.append(self.version_id_col == sql.bindparam(self.version_id_col.key, type=self.version_id_col.type))
statement = table.delete(clause)
- c = statement.execute(*delete)
+ c = connection.execute(statement, delete)
if c.supports_sane_rowcount() and c.rowcount != len(delete):
- raise CommitError("ConcurrencyError - updated rowcount %d does not match number of objects updated %d" % (c.cursor.rowcount, len(delete)))
+ raise exceptions.FlushError("ConcurrencyError - updated rowcount %d does not match number of objects updated %d" % (c.cursor.rowcount, len(delete)))
+ for obj in deleted_objects:
+ self.extension.after_delete(self, connection, obj)
def _has_pks(self, table):
try:
for k in self.pks_by_table[table]:
- if not self.columntoproperty.has_key(k.original):
+ if not self.columntoproperty.has_key(k):
return False
else:
return True
return False
def register_dependencies(self, uowcommit, *args, **kwargs):
- """called by an instance of objectstore.UOWTransaction to register
+ """called by an instance of unitofwork.UOWTransaction to register
which mappers are dependent on which, as well as DependencyProcessor
objects which will process lists of objects in between saves and deletes."""
for prop in self.props.values():
prop.register_dependencies(uowcommit, *args, **kwargs)
if self.inherits is not None:
uowcommit.register_dependency(self.inherits, self)
-
- def register_deleted(self, obj, uow):
- for prop in self.props.values():
- prop.register_deleted(obj, uow)
-
- def _identity_key(self, row):
- return objectstore.get_row_key(row, self.class_, self.pks_by_table[self.table], self.entity_name)
+ def cascade_iterator(self, type, object, recursive=None):
+ if recursive is None:
+ recursive=sets.Set()
+ if object not in recursive:
+ recursive.add(object)
+ yield object
+ for prop in self.props.values():
+ for c in prop.cascade_iterator(type, object, recursive):
+ yield c
+
+ def _row_identity_key(self, row):
+ return sessionlib.get_row_key(row, self.class_, self.pks_by_table[self.mapped_table], self.entity_name)
+ def get_select_mapper(self):
+ return self.__surrogate_mapper or self
+
def _instance(self, session, row, imap, result = None, populate_existing = False):
"""pulls an object instance from the given row and appends it to the given result
list. if the instance already exists in the given identity map, its not added. in
either case, executes all the property loaders on the instance to also process extra
information in the row."""
+
+ if self.polymorphic_on is not None:
+ discriminator = row[self.polymorphic_on]
+ mapper = self.polymorphic_map[discriminator]
+ if mapper is not self:
+ row = self.translate_row(mapper, row)
+ return mapper._instance(session, row, imap, result=result, populate_existing=populate_existing)
+
# look in main identity map. if its there, we dont do anything to it,
# including modifying any of its related items lists, as its already
# been exposed to being modified by the application.
- if session is None:
- session = objectstore.get_session()
-
populate_existing = populate_existing or self.always_refresh
- identitykey = self._identity_key(row)
+ identitykey = self._row_identity_key(row)
if session.has_key(identitykey):
instance = session._get(identitykey)
-
isnew = False
if populate_existing or session.is_expired(instance, unexpire=True):
if not imap.has_key(identitykey):
imap[identitykey] = instance
for prop in self.props.values():
prop.execute(session, instance, row, identitykey, imap, True)
- if self.extension.append_result(self, row, imap, result, instance, isnew, populate_existing=populate_existing) is EXT_PASS:
+ if self.extension.append_result(self, session, row, imap, result, instance, isnew, populate_existing=populate_existing) is EXT_PASS:
if result is not None:
result.append_nohistory(instance)
return instance
if not exists:
# check if primary key cols in the result are None - this indicates
# an instance of the object is not present in the row
- for col in self.pks_by_table[self.table]:
+ for col in self.pks_by_table[self.mapped_table]:
if row[col] is None:
return None
# plugin point
- instance = self.extension.create_instance(self, row, imap, self.class_)
+ instance = self.extension.create_instance(self, session, row, imap, self.class_)
if instance is EXT_PASS:
instance = self._create_instance(session)
imap[identitykey] = instance
# instances from the row and possibly populate this item.
if self.extension.populate_instance(self, session, instance, row, identitykey, imap, isnew) is EXT_PASS:
self.populate_instance(session, instance, row, identitykey, imap, isnew)
- if self.extension.append_result(self, row, imap, result, instance, isnew, populate_existing=populate_existing) is EXT_PASS:
+ if self.extension.append_result(self, session, row, imap, result, instance, isnew, populate_existing=populate_existing) is EXT_PASS:
if result is not None:
result.append_nohistory(instance)
return instance
def _create_instance(self, session):
- if not self.construct_new:
- return self.class_(_mapper_nohistory=True, _sa_entity_name=self.entity_name, _sa_session=session)
-
obj = self.class_.__new__(self.class_)
obj._entity_name = self.entity_name
-
+
# this gets the AttributeManager to do some pre-initialization,
# in order to save on KeyErrors later on
- objectstore.global_attributes.init_attr(obj)
-
- session._bind_to(obj)
+ sessionlib.global_attributes.init_attr(obj)
return obj
def translate_row(self, tomapper, row):
"""attempts to take a row and translate its values to a row that can
- be understood by another mapper. breaks the column references down to their
- bare keynames to accomplish this. So far this works for the various polymorphic
- examples."""
+ be understood by another mapper."""
newrow = util.DictDecorator(row)
- for c in self.table.c:
- newrow[c.name] = row[c]
- for c in tomapper.table.c:
- newrow[c] = newrow[c.name]
+ for c in tomapper.mapped_table.c:
+ c2 = self.mapped_table.corresponding_column(c, keys_ok=True, raiseerr=True)
+ newrow[c] = row[c2]
return newrow
def populate_instance(self, session, instance, row, identitykey, imap, isnew, frommapper=None):
if frommapper is not None:
row = frommapper.translate_row(self, row)
-
for prop in self.props.values():
prop.execute(session, instance, row, identitykey, imap, isnew)
+
+ # deprecated query methods. Query is constructed from Session, and the rest
+ # of these methods are called off of Query now.
+ def query(self, session=None):
+ """deprecated. use Query instead."""
+ if session is not None:
+ return querylib.Query(self, session=session)
+
+ try:
+ if self._query.mapper is not self:
+ self._query = querylib.Query(self)
+ return self._query
+ except AttributeError:
+ self._query = querylib.Query(self)
+ return self._query
+ def using(self, session):
+ """deprecated. use Query instead."""
+ return querylib.Query(self, session=session)
+ def __getattr__(self, key):
+ """deprecated. use Query instead."""
+ if (key.startswith('select_by_') or key.startswith('get_by_')):
+ return getattr(self.query(), key)
+ else:
+ raise AttributeError(key)
+ def compile(self, whereclause = None, **options):
+ """deprecated. use Query instead."""
+ return self.query()._compile(whereclause, **options)
+ def get(self, ident, **kwargs):
+ """deprecated. use Query instead."""
+ return self.query().get(ident, **kwargs)
+ def _get(self, key, ident=None, reload=False):
+ """deprecated. use Query instead."""
+ return self.query()._get(key, ident=ident, reload=reload)
+ def get_by(self, *args, **params):
+ """deprecated. use Query instead."""
+ return self.query().get_by(*args, **params)
+ def select_by(self, *args, **params):
+ """deprecated. use Query instead."""
+ return self.query().select_by(*args, **params)
+ def selectfirst_by(self, *args, **params):
+ """deprecated. use Query instead."""
+ return self.query().selectfirst_by(*args, **params)
+ def selectone_by(self, *args, **params):
+ """deprecated. use Query instead."""
+ return self.query().selectone_by(*args, **params)
+ def count_by(self, *args, **params):
+ """deprecated. use Query instead."""
+ return self.query().count_by(*args, **params)
+ def selectfirst(self, *args, **params):
+ """deprecated. use Query instead."""
+ return self.query().selectfirst(*args, **params)
+ def selectone(self, *args, **params):
+ """deprecated. use Query instead."""
+ return self.query().selectone(*args, **params)
+ def select(self, arg=None, **kwargs):
+ """deprecated. use Query instead."""
+ return self.query().select(arg=arg, **kwargs)
+ def select_whereclause(self, whereclause=None, params=None, **kwargs):
+ """deprecated. use Query instead."""
+ return self.query().select_whereclause(whereclause=whereclause, params=params, **kwargs)
+ def count(self, whereclause=None, params=None, **kwargs):
+ """deprecated. use Query instead."""
+ return self.query().count(whereclause=whereclause, params=params, **kwargs)
+ def select_statement(self, statement, **params):
+ """deprecated. use Query instead."""
+ return self.query().select_statement(statement, **params)
+ def select_text(self, text, **params):
+ """deprecated. use Query instead."""
+ return self.query().select_text(text, **params)
class MapperProperty(object):
"""an element attached to a Mapper that describes and assists in the loading and saving
"""called when the mapper receives a row. instance is the parent instance
corresponding to the row. """
raise NotImplementedError()
+ def cascade_iterator(self, type, object, recursive=None):
+ return []
def copy(self):
raise NotImplementedError()
- def get_criterion(self, key, value):
+ def get_criterion(self, query, key, value):
"""Returns a WHERE clause suitable for this MapperProperty corresponding to the
given key/value pair, where the key is a column or object property name, and value
is a value to be matched. This is only picked up by PropertyLoaders.
self.key = key
self.parent = parent
self.do_init(key, parent)
+ def adapt(self, newparent):
+ """adapts this MapperProperty to a new parent, assuming the new parent is an inheriting
+ descendant of the old parent. Should return True if the adaptation was successful, or
+ False if this MapperProperty cannot be adapted to the new parent (the case for this is,
+ the parent mapper has a polymorphic select, and this property represents a column that is not
+ represented in the new mapper's mapped table)"""
+ self.parent = newparent
+ return True
def do_init(self, key, parent):
"""template method for subclasses"""
pass
def __init__(self):
self.next = None
def chain(self, ext):
+ if ext is self:
+ raise "nu uh " + repr(self) + " " + repr(ext)
self.next = ext
- return self
+ return self
+ def get_session(self):
+ """called to retrieve a contextual Session instance with which to
+ register a new object. Note: this is not called if a session is
+ provided with the __init__ params (i.e. _sa_session)"""
+ if self.next is None:
+ return EXT_PASS
+ else:
+ return self.next.get_session()
def select_by(self, query, *args, **kwargs):
"""overrides the select_by method of the Query object"""
if self.next is None:
return EXT_PASS
else:
return self.next.select(query, *args, **kwargs)
- def create_instance(self, mapper, row, imap, class_):
+ def create_instance(self, mapper, session, row, imap, class_):
"""called when a new object instance is about to be created from a row.
the method can choose to create the instance itself, or it can return
None to indicate normal object creation should take place.
if self.next is None:
return EXT_PASS
else:
- return self.next.create_instance(mapper, row, imap, class_)
- def append_result(self, mapper, row, imap, result, instance, isnew, populate_existing=False):
+ return self.next.create_instance(mapper, session, row, imap, class_)
+ def append_result(self, mapper, session, row, imap, result, instance, isnew, populate_existing=False):
"""called when an object instance is being appended to a result list.
If this method returns True, it is assumed that the mapper should do the appending, else
if self.next is None:
return EXT_PASS
else:
- return self.next.append_result(mapper, row, imap, result, instance, isnew, populate_existing)
+ return self.next.append_result(mapper, session, row, imap, result, instance, isnew, populate_existing)
def populate_instance(self, mapper, session, instance, row, identitykey, imap, isnew):
"""called right before the mapper, after creating an instance from a row, passes the row
to its MapperProperty objects which are responsible for populating the object's attributes.
return EXT_PASS
else:
return self.next.populate_instance(mapper, session, instance, row, identitykey, imap, isnew)
- def before_insert(self, mapper, instance):
+ def before_insert(self, mapper, connection, instance):
"""called before an object instance is INSERTed into its table.
this is a good place to set up primary key values and such that arent handled otherwise."""
if self.next is not None:
- self.next.before_insert(mapper, instance)
- def before_update(self, mapper, instance):
+ self.next.before_insert(mapper, connection, instance)
+ def before_update(self, mapper, connection, instance):
"""called before an object instnace is UPDATED"""
if self.next is not None:
- self.next.before_update(mapper, instance)
- def after_update(self, mapper, instance):
+ self.next.before_update(mapper, connection, instance)
+ def after_update(self, mapper, connection, instance):
"""called after an object instnace is UPDATED"""
if self.next is not None:
- self.next.after_update(mapper, instance)
- def after_insert(self, mapper, instance):
+ self.next.after_update(mapper, connection, instance)
+ def after_insert(self, mapper, connection, instance):
"""called after an object instance has been INSERTed"""
if self.next is not None:
- self.next.after_insert(mapper, instance)
- def before_delete(self, mapper, instance):
+ self.next.after_insert(mapper, connection, instance)
+ def before_delete(self, mapper, connection, instance):
"""called before an object instance is DELETEed"""
if self.next is not None:
- self.next.before_delete(mapper, instance)
+ self.next.before_delete(mapper, connection, instance)
+ def after_delete(self, mapper, connection, instance):
+ """called after an object instance is DELETEed"""
+ if self.next is not None:
+ self.next.after_delete(mapper, connection, instance)
+class TranslatingDict(dict):
+ """a dictionary that stores ColumnElement objects as keys. incoming ColumnElement
+ keys are translated against those of an underling FromClause for all operations.
+ This way the columns from any Selectable that is derived from or underlying this
+ TranslatingDict's selectable can be used as keys."""
+ def __init__(self, selectable):
+ super(TranslatingDict, self).__init__()
+ self.selectable = selectable
+ def __translate_col(self, col):
+ ourcol = self.selectable.corresponding_column(col, keys_ok=False, raiseerr=False)
+ if ourcol is None:
+ return col
+ else:
+ return ourcol
+ def __getitem__(self, col):
+ return super(TranslatingDict, self).__getitem__(self.__translate_col(col))
+ def has_key(self, col):
+ return super(TranslatingDict, self).has_key(self.__translate_col(col))
+ def __setitem__(self, col, value):
+ return super(TranslatingDict, self).__setitem__(self.__translate_col(col), value)
+ def __contains__(self, col):
+ return self.has_key(col)
+ def setdefault(self, col, value):
+ return super(TranslatingDict, self).setdefault(self.__translate_col(col), value)
+
class ClassKey(object):
"""keys a class and an entity name to a mapper, via the mapper_registry"""
def __init__(self, class_, entity_name):
else:
return repr(obj)
-def object_mapper(object):
+def object_mapper(object, raiseerror=True, entity_name=None):
"""given an object, returns the primary Mapper associated with the object
or the object's class."""
try:
- return mapper_registry[ClassKey(object.__class__, getattr(object, '_entity_name', None))]
+ return mapper_registry[ClassKey(object.__class__, getattr(object, '_entity_name', entity_name))]
except KeyError:
- raise InvalidRequestError("Class '%s' entity name '%s' has no mapper associated with it" % (object.__class__.__name__, getattr(object, '_entity_name', None)))
+ if raiseerror:
+ raise exceptions.InvalidRequestError("Class '%s' entity name '%s' has no mapper associated with it" % (object.__class__.__name__, getattr(object, '_entity_name', None)))
+ else:
+ return None
def class_mapper(class_, entity_name=None):
"""given a ClassKey, returns the primary Mapper associated with the key."""
try:
return mapper_registry[ClassKey(class_, entity_name)]
except (KeyError, AttributeError):
- raise InvalidRequestError("Class '%s' entity name '%s' has no mapper associated with it" % (class_.__name__, entity_name))
+ raise exceptions.InvalidRequestError("Class '%s' entity name '%s' has no mapper associated with it" % (class_.__name__, entity_name))
# This module is part of SQLAlchemy and is released under
# the MIT License: http://www.opensource.org/licenses/mit-license.php
-"""defines a set of MapperProperty objects, including basic column properties as
+"""defines a set of mapper.MapperProperty objects, including basic column properties as
well as relationships. also defines some MapperOptions that can be used with the
properties."""
-from mapper import *
-import sqlalchemy.sql as sql
-import sqlalchemy.schema as schema
-import sqlalchemy.engine as engine
-import sqlalchemy.util as util
-import sqlalchemy.attributes as attributes
+from sqlalchemy import sql, schema, util, attributes, exceptions
import sync
import mapper
-import objectstore
-from sqlalchemy.exceptions import *
-import random
+import session as sessionlib
+import dependency
+import util as mapperutil
+import sets, random
-class ColumnProperty(MapperProperty):
+class ColumnProperty(mapper.MapperProperty):
"""describes an object attribute that corresponds to a table column."""
def __init__(self, *columns):
"""the list of columns describes a single object property. if there
def setattr(self, object, value):
setattr(object, self.key, value)
def get_history(self, obj, passive=False):
- return objectstore.global_attributes.get_history(obj, self.key, passive=passive)
+ return sessionlib.global_attributes.get_history(obj, self.key, passive=passive)
def copy(self):
return ColumnProperty(*self.columns)
def setup(self, key, statement, eagertable=None, **options):
for c in self.columns:
if eagertable is not None:
- statement.append_column(eagertable._get_col_by_original(c))
+ statement.append_column(eagertable.corresponding_column(c))
else:
statement.append_column(c)
def do_init(self, key, parent):
self.key = key
+ self.parent = parent
# establish a SmartProperty property manager on the object for this key
if parent._is_primary_mapper():
#print "regiser col on class %s key %s" % (parent.class_.__name__, key)
- objectstore.uow().register_attribute(parent.class_, key, uselist = False)
+ sessionlib.global_attributes.register_attribute(parent.class_, key, uselist = False)
def execute(self, session, instance, row, identitykey, imap, isnew):
if isnew:
#print "POPULATING OBJ", instance.__class__.__name__, "COL", self.columns[0]._label, "WITH DATA", row[self.columns[0]], "ROW IS A", row.__class__.__name__, "COL ID", id(self.columns[0])
# establish a SmartProperty property manager on the object for this key,
# containing a callable to load in the attribute
if self.is_primary():
- objectstore.uow().register_attribute(parent.class_, key, uselist=False, callable_=lambda i:self.setup_loader(i))
+ sessionlib.global_attributes.register_attribute(parent.class_, key, uselist=False, callable_=lambda i:self.setup_loader(i))
def setup_loader(self, instance):
+ if not self.parent.is_assigned(instance):
+ return mapper.object_mapper(instance).props[self.key].setup_loader(instance)
def lazyload():
+ session = sessionlib.object_session(instance)
+ connection = session.connection(self.parent)
clause = sql.and_()
try:
pk = self.parent.pks_by_table[self.columns[0].table]
return None
clause.clauses.append(primary_key == attr)
- if self.group is not None:
- groupcols = [p for p in self.parent.props.values() if isinstance(p, DeferredColumnProperty) and p.group==self.group]
- row = sql.select([g.columns[0] for g in groupcols], clause, use_labels=True).execute().fetchone()
- for prop in groupcols:
- if prop is self:
- continue
- instance.__dict__[prop.key] = row[prop.columns[0]]
- objectstore.global_attributes.create_history(instance, prop.key, uselist=False)
- return row[self.columns[0]]
- else:
- return sql.select([self.columns[0]], clause, use_labels=True).scalar()
+ try:
+ if self.group is not None:
+ groupcols = [p for p in self.parent.props.values() if isinstance(p, DeferredColumnProperty) and p.group==self.group]
+ row = connection.execute(sql.select([g.columns[0] for g in groupcols], clause, use_labels=True), None).fetchone()
+ for prop in groupcols:
+ if prop is self:
+ continue
+ instance.__dict__[prop.key] = row[prop.columns[0]]
+ sessionlib.global_attributes.create_history(instance, prop.key, uselist=False)
+ return row[self.columns[0]]
+ else:
+ return connection.scalar(sql.select([self.columns[0]], clause, use_labels=True),None)
+ finally:
+ connection.close()
return lazyload
def setup(self, key, statement, **options):
pass
def execute(self, session, instance, row, identitykey, imap, isnew):
if isnew:
if not self.is_primary():
- objectstore.global_attributes.create_history(instance, self.key, False, callable_=self.setup_loader(instance))
+ sessionlib.global_attributes.create_history(instance, self.key, False, callable_=self.setup_loader(instance))
else:
- objectstore.global_attributes.reset_history(instance, self.key)
+ sessionlib.global_attributes.reset_history(instance, self.key)
mapper.ColumnProperty = ColumnProperty
-class PropertyLoader(MapperProperty):
+class PropertyLoader(mapper.MapperProperty):
ONETOMANY = 0
MANYTOONE = 1
MANYTOMANY = 2
"""describes an object property that holds a single item or list of items that correspond
to a related database table."""
- def __init__(self, argument, secondary, primaryjoin, secondaryjoin, foreignkey=None, uselist=None, private=False, association=None, use_alias=None, selectalias=None, order_by=False, attributeext=None, backref=None, is_backref=False, post_update=False):
+ def __init__(self, argument, secondary, primaryjoin, secondaryjoin, foreignkey=None, uselist=None, private=False, association=None, order_by=False, attributeext=None, backref=None, is_backref=False, post_update=False, cascade=None):
self.uselist = uselist
self.argument = argument
self.secondary = secondary
# would like to have foreignkey be a list.
# however, have to figure out how to do
- # <column> in <list>, since column overrides the == operator or somethign
+ # <column> in <list>, since column overrides the == operator
# and it doesnt work
self.foreignkey = foreignkey #util.to_set(foreignkey)
if foreignkey:
self.foreigntable = foreignkey.table
else:
self.foreigntable = None
-
- self.private = private
+
+ if cascade is not None:
+ self.cascade = mapperutil.CascadeOptions(cascade)
+ else:
+ if private:
+ self.cascade = mapperutil.CascadeOptions("all, delete-orphan")
+ else:
+ self.cascade = mapperutil.CascadeOptions("save-update")
+
self.association = association
- if selectalias is not None:
- print "'selectalias' argument to relation() is deprecated. eager loads automatically alias-ize tables now."
- if use_alias is not None:
- print "'use_alias' argument to relation() is deprecated. eager loads automatically alias-ize tables now."
self.order_by = order_by
self.attributeext=attributeext
if isinstance(backref, str):
self.backref = backref
self.is_backref = is_backref
+ private = property(lambda s:s.cascade.delete_orphan)
+
+ def cascade_iterator(self, type, object, recursive=None):
+ if not type in self.cascade:
+ return
+ if recursive is None:
+ recursive = sets.Set()
+
+ childlist = sessionlib.global_attributes.get_history(object, self.key, passive=True)
+
+ for c in childlist.added_items() + childlist.deleted_items() + childlist.unchanged_items():
+ if c is not None:
+ if c not in recursive:
+ recursive.add(c)
+ yield c
+ for c2 in self.mapper.cascade_iterator(type, c, recursive):
+ yield c2
+
def copy(self):
x = self.__class__.__new__(self.__class__)
x.__dict__.update(self.__dict__)
pass
def do_init(self, key, parent):
- import sqlalchemy.mapping
+ import sqlalchemy.orm
if isinstance(self.argument, type):
- self.mapper = sqlalchemy.mapping.class_mapper(self.argument)
+ self.mapper = mapper.class_mapper(self.argument)
else:
self.mapper = self.argument
+ self.mapper = self.mapper.get_select_mapper()
+
if self.association is not None:
if isinstance(self.association, type):
- self.association = sqlalchemy.mapping.class_mapper(self.association)
+ self.association = mapper.class_mapper(self.association)
- self.target = self.mapper.table
+ self.target = self.mapper.mapped_table
self.key = key
self.parent = parent
if self.secondaryjoin is not None and self.secondary is None:
- raise ArgumentError("Property '" + self.key + "' specified with secondary join condition but no secondary argument")
+ raise exceptions.ArgumentError("Property '" + self.key + "' specified with secondary join condition but no secondary argument")
# if join conditions were not specified, figure them out based on foreign keys
if self.secondary is not None:
if self.secondaryjoin is None:
- self.secondaryjoin = sql.join(self.mapper.noninherited_table, self.secondary).onclause
+ self.secondaryjoin = sql.join(self.mapper.unjoined_table, self.secondary).onclause
if self.primaryjoin is None:
- self.primaryjoin = sql.join(parent.noninherited_table, self.secondary).onclause
+ self.primaryjoin = sql.join(parent.unjoined_table, self.secondary).onclause
else:
if self.primaryjoin is None:
- self.primaryjoin = sql.join(parent.noninherited_table, self.target).onclause
+ self.primaryjoin = sql.join(parent.unjoined_table, self.target).onclause
+
# if the foreign key wasnt specified and theres no assocaition table, try to figure
# out who is dependent on who. we dont need all the foreign keys represented in the join,
# just one of them.
if self.direction is None:
self.direction = self._get_direction()
- if self.uselist is None and self.direction == PropertyLoader.MANYTOONE:
+ if self.uselist is None and self.direction == sync.MANYTOONE:
self.uselist = False
if self.uselist is None:
self.uselist = True
self._compile_synchronizers()
+ self._dependency_processor = dependency.create_dependency_processor(self.key, self.syncrules, self.cascade, secondary=self.secondary, association=self.association, is_backref=self.is_backref, post_update=self.post_update)
# primary property handler, set up class attributes
if self.is_primary():
if self.backref is not None:
self.backref.compile(self)
- elif not objectstore.global_attributes.is_class_managed(parent.class_, key):
- raise ArgumentError("Non-primary property created for attribute '%s' on class '%s', but that attribute is not managed! Insure that the primary mapper for this class defines this property" % (key, parent.class_.__name__))
+ elif not sessionlib.global_attributes.is_class_managed(parent.class_, key):
+ raise exceptions.ArgumentError("Attempting to assign a new relation '%s' to a non-primary mapper on class '%s'. New relations can only be added to the primary mapper, i.e. the very first mapper created for class '%s' " % (key, parent.class_.__name__, parent.class_.__name__))
self.do_init_subclass(key, parent)
def _set_class_attribute(self, class_, key):
"""sets attribute behavior on our target class."""
- objectstore.uow().register_attribute(class_, key, uselist = self.uselist, deleteremoved = self.private, extension=self.attributeext)
+ sessionlib.global_attributes.register_attribute(class_, key, uselist = self.uselist, extension=self.attributeext, cascade=self.cascade, trackparent=True)
def _get_direction(self):
"""determines our 'direction', i.e. do we represent one to many, many to many, etc."""
- #print self.key, repr(self.parent.table.name), repr(self.parent.primarytable.name), repr(self.foreignkey.table.name), repr(self.target), repr(self.foreigntable.name)
+ #print self.key, repr(self.parent.mapped_table.name), repr(self.parent.primarytable.name), repr(self.foreignkey.table.name), repr(self.target), repr(self.foreigntable.name)
if self.secondaryjoin is not None:
- return PropertyLoader.MANYTOMANY
- elif self.parent.table is self.target:
+ return sync.MANYTOMANY
+ elif self.parent.mapped_table is self.target:
if self.foreignkey.primary_key:
- return PropertyLoader.MANYTOONE
+ return sync.MANYTOONE
else:
- return PropertyLoader.ONETOMANY
- elif self.foreigntable == self.mapper.noninherited_table:
- return PropertyLoader.ONETOMANY
- elif self.foreigntable == self.parent.noninherited_table:
- return PropertyLoader.MANYTOONE
+ return sync.ONETOMANY
+ elif self.foreigntable == self.mapper.unjoined_table:
+ return sync.ONETOMANY
+ elif self.foreigntable == self.parent.unjoined_table:
+ return sync.MANYTOONE
else:
- raise ArgumentError("Cant determine relation direction")
+ raise exceptions.ArgumentError("Cant determine relation direction")
def _find_dependent(self):
"""searches through the primary join condition to determine which side
# set as a reference to allow assignment from inside a first-class function
dependent = [None]
def foo(binary):
- if binary.operator != '=':
+ if binary.operator != '=' or not isinstance(binary.left, schema.Column) or not isinstance(binary.right, schema.Column):
return
- if isinstance(binary.left, schema.Column) and binary.left.primary_key:
+ if binary.left.primary_key:
if dependent[0] is binary.left.table:
- raise ArgumentError("bidirectional dependency not supported...specify foreignkey")
+ raise exceptions.ArgumentError("Could not determine the parent/child relationship for property '%s', based on join condition '%s' (table '%s' appears on both sides of the relationship, or in an otherwise ambiguous manner). please specify the 'foreignkey' keyword parameter to the relation() function indicating a column on the remote side of the relationship" % (self.key, str(self.primaryjoin), str(binary.left.table)))
dependent[0] = binary.right.table
self.foreignkey= binary.right
- elif isinstance(binary.right, schema.Column) and binary.right.primary_key:
+ elif binary.right.primary_key:
if dependent[0] is binary.right.table:
- raise ArgumentError("bidirectional dependency not supported...specify foreignkey")
+ raise exceptions.ArgumentError("Could not determine the parent/child relationship for property '%s', based on join condition '%s' (table '%s' appears on both sides of the relationship, or in an otherwise ambiguous manner). please specify the 'foreignkey' keyword parameter to the relation() function indicating a column on the remote side of the relationship" % (self.key, str(self.primaryjoin), str(binary.right.table)))
dependent[0] = binary.left.table
self.foreignkey = binary.left
visitor = BinaryVisitor(foo)
self.primaryjoin.accept_visitor(visitor)
if dependent[0] is None:
- raise ArgumentError("cant determine primary foreign key in the join relationship....specify foreignkey=<column> or foreignkey=[<columns>]")
+ raise exceptions.ArgumentError("Could not determine the parent/child relationship for property '%s', based on join condition '%s' (no relationships joining tables '%s' and '%s' could be located). please specify the 'foreignkey' keyword parameter to the relation() function indicating a column on the remote side of the relationship" % (self.key, str(self.primaryjoin), str(binary.left.table), str(binary.right.table)))
else:
self.foreigntable = dependent[0]
-
- def get_criterion(self, key, value):
- """given a key/value pair, determines if this PropertyLoader's mapper contains a key of the
- given name in its property list, or if this PropertyLoader's association mapper, if any,
- contains a key of the given name in its property list, and returns a WHERE clause against
- the given value if found.
-
- this is called by a mappers select_by method to formulate a set of key/value pairs into
- a WHERE criterion that spans multiple tables if needed."""
- # TODO: optimization: change mapper to accept a WHERE clause with separate bind parameters
- # then cache the generated WHERE clauses here, since the creation + the copy_container
- # is an extra expense
- if self.mapper.props.has_key(key):
- if self.secondaryjoin is not None:
- c = (self.mapper.props[key].columns[0]==value) & self.primaryjoin & self.secondaryjoin
- else:
- c = (self.mapper.props[key].columns[0]==value) & self.primaryjoin
- return c.copy_container()
- elif self.mapper.table.c.has_key(key):
- if self.secondaryjoin is not None:
- c = (self.mapper.table.c[key].columns[0]==value) & self.primaryjoin & self.secondaryjoin
- else:
- c = (self.mapper.table.c[key].columns[0]==value) & self.primaryjoin
- return c.copy_container()
- elif self.association is not None:
- c = self.mapper._get_criterion(key, value) & self.primaryjoin
- return c.copy_container()
- return None
-
- def register_deleted(self, obj, uow):
- if not self.private:
- return
-
- if self.uselist:
- childlist = uow.attributes.get_history(obj, self.key, passive = False)
- else:
- childlist = uow.attributes.get_history(obj, self.key)
- for child in childlist.deleted_items() + childlist.unchanged_items():
- if child is not None:
- uow.register_deleted(child)
-
- class MapperStub(object):
- """poses as a Mapper representing the association table in a many-to-many
- join, when performing a commit().
-
- The Task objects in the objectstore module treat it just like
- any other Mapper, but in fact it only serves as a "dependency" placeholder
- for the many-to-many update task."""
- def __init__(self, mapper):
- self.mapper = mapper
- def save_obj(self, *args, **kwargs):
- pass
- def delete_obj(self, *args, **kwargs):
- pass
- def _primary_mapper(self):
- return self
-
- def register_dependencies(self, uowcommit):
- """tells a UOWTransaction what mappers are dependent on which, with regards
- to the two or three mappers handled by this PropertyLoader.
-
- Also registers itself as a "processor" for one of its mappers, which
- will be executed after that mapper's objects have been saved or before
- they've been deleted. The process operation manages attributes and dependent
- operations upon the objects of one of the involved mappers."""
- if self.association is not None:
- # association object. our mapper should be dependent on both
- # the parent mapper and the association object mapper.
- # this is where we put the "stub" as a marker, so we get
- # association/parent->stub->self, then we process the child
- # elments after the 'stub' save, which is before our own
- # mapper's save.
- stub = PropertyLoader.MapperStub(self.association)
- uowcommit.register_dependency(self.parent, stub)
- uowcommit.register_dependency(self.association, stub)
- uowcommit.register_dependency(stub, self.mapper)
- uowcommit.register_processor(stub, self, self.parent, False)
- uowcommit.register_processor(stub, self, self.parent, True)
-
- elif self.direction == PropertyLoader.MANYTOMANY:
- # many-to-many. create a "Stub" mapper to represent the
- # "middle table" in the relationship. This stub mapper doesnt save
- # or delete any objects, but just marks a dependency on the two
- # related mappers. its dependency processor then populates the
- # association table.
-
- if self.is_backref:
- # if we are the "backref" half of a two-way backref
- # relationship, let the other mapper handle inserting the rows
- return
- stub = PropertyLoader.MapperStub(self.mapper)
- uowcommit.register_dependency(self.parent, stub)
- uowcommit.register_dependency(self.mapper, stub)
- uowcommit.register_processor(stub, self, self.parent, False)
- uowcommit.register_processor(stub, self, self.parent, True)
- elif self.direction == PropertyLoader.ONETOMANY:
- if self.post_update:
- stub = PropertyLoader.MapperStub(self.mapper)
- uowcommit.register_dependency(self.mapper, stub)
- uowcommit.register_dependency(self.parent, stub)
- uowcommit.register_processor(stub, self, self.parent, False)
- uowcommit.register_processor(stub, self, self.parent, True)
- else:
- uowcommit.register_dependency(self.parent, self.mapper)
- uowcommit.register_processor(self.parent, self, self.parent, False)
- uowcommit.register_processor(self.parent, self, self.parent, True)
- elif self.direction == PropertyLoader.MANYTOONE:
- if self.post_update:
- stub = PropertyLoader.MapperStub(self.mapper)
- uowcommit.register_dependency(self.mapper, stub)
- uowcommit.register_dependency(self.parent, stub)
- uowcommit.register_processor(stub, self, self.parent, False)
- uowcommit.register_processor(stub, self, self.parent, True)
- else:
- uowcommit.register_dependency(self.mapper, self.parent)
- uowcommit.register_processor(self.mapper, self, self.parent, False)
- uowcommit.register_processor(self.mapper, self, self.parent, True)
- else:
- raise AssertionError(" no foreign key ?")
-
- def get_object_dependencies(self, obj, uowcommit, passive = True):
- return uowcommit.uow.attributes.get_history(obj, self.key, passive = passive)
-
- def whose_dependent_on_who(self, obj1, obj2):
- """given an object pair assuming obj2 is a child of obj1, returns a tuple
- with the dependent object second, or None if they are equal.
- used by objectstore's object-level topological sort (i.e. cyclical
- table dependency)."""
- if obj1 is obj2:
- return None
- elif self.direction == PropertyLoader.ONETOMANY:
- return (obj1, obj2)
- else:
- return (obj2, obj1)
-
- def process_dependencies(self, task, deplist, uowcommit, delete = False):
- """this method is called during a commit operation to synchronize data between a parent and child object.
- it also can establish child or parent objects within the unit of work as "to be saved" or "deleted"
- in some cases."""
- #print self.mapper.table.name + " " + self.key + " " + repr(len(deplist)) + " process_dep isdelete " + repr(delete) + " direction " + repr(self.direction)
-
- def getlist(obj, passive=True):
- l = self.get_object_dependencies(obj, uowcommit, passive)
- uowcommit.register_saved_history(l)
- return l
-
- # plugin point
-
- if self.direction == PropertyLoader.MANYTOMANY:
- secondary_delete = []
- secondary_insert = []
- if delete:
- for obj in deplist:
- childlist = getlist(obj, False)
- for child in childlist.deleted_items() + childlist.unchanged_items():
- associationrow = {}
- self._synchronize(obj, child, associationrow, False)
- secondary_delete.append(associationrow)
- else:
- for obj in deplist:
- childlist = getlist(obj)
- if childlist is None: continue
- for child in childlist.added_items():
- associationrow = {}
- self._synchronize(obj, child, associationrow, False)
- secondary_insert.append(associationrow)
- for child in childlist.deleted_items():
- associationrow = {}
- self._synchronize(obj, child, associationrow, False)
- secondary_delete.append(associationrow)
- if len(secondary_delete):
- # TODO: precompile the delete/insert queries and store them as instance variables
- # on the PropertyLoader
- statement = self.secondary.delete(sql.and_(*[c == sql.bindparam(c.key) for c in self.secondary.c]))
- statement.execute(*secondary_delete)
- if len(secondary_insert):
- statement = self.secondary.insert()
- statement.execute(*secondary_insert)
- elif self.direction == PropertyLoader.MANYTOONE and delete:
- if self.private:
- for obj in deplist:
- childlist = getlist(obj, False)
- for child in childlist.deleted_items() + childlist.unchanged_items():
- if child is None:
- continue
- # if private child object, and is in the uow's "deleted" list,
- # insure its in the list of items to be deleted
- if child in uowcommit.uow.deleted:
- uowcommit.register_object(child, isdelete=True)
- elif self.post_update:
- # post_update means we have to update our row to not reference the child object
- # before we can DELETE the row
- for obj in deplist:
- self._synchronize(obj, None, None, True)
- uowcommit.register_object(obj, postupdate=True)
- elif self.direction == PropertyLoader.ONETOMANY and delete:
- # head object is being deleted, and we manage its list of child objects
- # the child objects have to have their foreign key to the parent set to NULL
- if self.private and not self.post_update:
- for obj in deplist:
- childlist = getlist(obj, False)
- for child in childlist.deleted_items() + childlist.unchanged_items():
- if child is None:
- continue
- # if private child object, and is in the uow's "deleted" list,
- # insure its in the list of items to be deleted
- if child in uowcommit.uow.deleted:
- uowcommit.register_object(child, isdelete=True)
- else:
- for obj in deplist:
- childlist = getlist(obj, False)
- for child in childlist.deleted_items() + childlist.unchanged_items():
- if child is not None:
- self._synchronize(obj, child, None, True)
- uowcommit.register_object(child, postupdate=self.post_update)
- elif self.association is not None:
- # manage association objects.
- for obj in deplist:
- childlist = getlist(obj, passive=True)
- if childlist is None: continue
-
- #print "DIRECTION", self.direction
- d = {}
- for child in childlist:
- self._synchronize(obj, child, None, False)
- key = self.mapper.instance_key(child)
- #print "SYNCHRONIZED", child, "INSTANCE KEY", key
- d[key] = child
- uowcommit.unregister_object(child)
-
- for child in childlist.added_items():
- uowcommit.register_object(child)
- key = self.mapper.instance_key(child)
- #print "ADDED, INSTANCE KEY", key
- d[key] = child
-
- for child in childlist.unchanged_items():
- key = self.mapper.instance_key(child)
- o = d[key]
- o._instance_key= key
-
- for child in childlist.deleted_items():
- key = self.mapper.instance_key(child)
- #print "DELETED, INSTANCE KEY", key
- if d.has_key(key):
- o = d[key]
- o._instance_key = key
- uowcommit.unregister_object(child)
- else:
- #print "DELETE ASSOC OBJ", repr(child)
- uowcommit.register_object(child, isdelete=True)
+ def get_join(self):
+ if self.secondaryjoin is not None:
+ return self.primaryjoin & self.secondaryjoin
else:
- for obj in deplist:
- childlist = getlist(obj, passive=True)
- if childlist is not None:
- for child in childlist.added_items():
- self._synchronize(obj, child, None, False)
- if self.direction == PropertyLoader.ONETOMANY and child is not None:
- uowcommit.register_object(child, postupdate=self.post_update)
- if self.direction == PropertyLoader.MANYTOONE:
- uowcommit.register_object(obj, postupdate=self.post_update)
- if self.direction != PropertyLoader.MANYTOONE:
- for child in childlist.deleted_items():
- if not self.private:
- self._synchronize(obj, child, None, True)
- uowcommit.register_object(child, isdelete=self.private)
+ return self.primaryjoin
def execute(self, session, instance, row, identitykey, imap, isnew):
if self.is_primary():
return
#print "PLAIN PROPLOADER EXEC NON-PRIAMRY", repr(id(self)), repr(self.mapper.class_), self.key
- objectstore.global_attributes.create_history(instance, self.key, self.uselist)
+ sessionlib.global_attributes.create_history(instance, self.key, self.uselist, cascade=self.cascade, trackparent=True)
+
+ def register_dependencies(self, uowcommit):
+ self._dependency_processor.register_dependencies(uowcommit)
def _compile_synchronizers(self):
"""assembles a list of 'synchronization rules', which are instructions on how to populate
The list of rules is used within commits by the _synchronize() method when dependent
objects are processed."""
-
-
- parent_tables = util.HashSet(self.parent.tables + [self.parent.primarytable])
- target_tables = util.HashSet(self.mapper.tables + [self.mapper.primarytable])
+ parent_tables = util.HashSet(self.parent.tables + [self.parent.mapped_table])
+ target_tables = util.HashSet(self.mapper.tables + [self.mapper.mapped_table])
self.syncrules = sync.ClauseSynchronizer(self.parent, self.mapper, self.direction)
- if self.direction == PropertyLoader.MANYTOMANY:
+ if self.direction == sync.MANYTOMANY:
#print "COMPILING p/c", self.parent, self.mapper
self.syncrules.compile(self.primaryjoin, parent_tables, [self.secondary], False)
self.syncrules.compile(self.secondaryjoin, target_tables, [self.secondary], True)
else:
self.syncrules.compile(self.primaryjoin, parent_tables, target_tables)
- def _synchronize(self, obj, child, associationrow, clearkeys):
- """called during a commit to execute the full list of syncrules on the
- given object/child/optional association row"""
- if self.direction == PropertyLoader.ONETOMANY:
- source = obj
- dest = child
- elif self.direction == PropertyLoader.MANYTOONE:
- source = child
- dest = obj
- elif self.direction == PropertyLoader.MANYTOMANY:
- dest = associationrow
- source = None
-
- if dest is None:
- return
-
- self.syncrules.execute(source, dest, obj, child, clearkeys)
-
class LazyLoader(PropertyLoader):
def do_init_subclass(self, key, parent):
- (self.lazywhere, self.lazybinds, self.lazyreverse) = create_lazy_clause(self.parent.noninherited_table, self.primaryjoin, self.secondaryjoin, self.foreignkey)
+ (self.lazywhere, self.lazybinds, self.lazyreverse) = create_lazy_clause(self.parent.unjoined_table, self.primaryjoin, self.secondaryjoin, self.foreignkey)
# determine if our "lazywhere" clause is the same as the mapper's
# get() clause. then we can just use mapper.get()
- self.use_get = not self.uselist and self.mapper.query._get_clause.compare(self.lazywhere)
+ self.use_get = not self.uselist and self.mapper.query()._get_clause.compare(self.lazywhere)
def _set_class_attribute(self, class_, key):
# establish a class-level lazy loader on our class
#print "SETCLASSATTR LAZY", repr(class_), key
- objectstore.global_attributes.register_attribute(class_, key, uselist = self.uselist, deleteremoved = self.private, callable_=lambda i: self.setup_loader(i), extension=self.attributeext)
+ sessionlib.global_attributes.register_attribute(class_, key, uselist = self.uselist, callable_=lambda i: self.setup_loader(i), extension=self.attributeext, cascade=self.cascade, trackparent=True)
def setup_loader(self, instance):
if not self.parent.is_assigned(instance):
- return object_mapper(instance).props[self.key].setup_loader(instance)
+ return mapper.object_mapper(instance).props[self.key].setup_loader(instance)
def lazyload():
params = {}
allparams = True
- session = objectstore.get_session(instance)
- #print "setting up loader, lazywhere", str(self.lazywhere)
- for col, bind in self.lazybinds.iteritems():
- params[bind.key] = self.parent._getattrbycolumn(instance, col)
- if params[bind.key] is None:
- allparams = False
- break
+ session = sessionlib.object_session(instance)
+ #print "setting up loader, lazywhere", str(self.lazywhere), "binds", self.lazybinds
+ if session is not None:
+ for col, bind in self.lazybinds.iteritems():
+ params[bind.key] = self.parent._getattrbycolumn(instance, col)
+ if params[bind.key] is None:
+ allparams = False
+ break
+ else:
+ allparams = False
if allparams:
# if we have a simple straight-primary key load, use mapper.get()
# to possibly save a DB round trip
if self.use_get:
ident = []
- for primary_key in self.mapper.pks_by_table[self.mapper.table]:
+ for primary_key in self.mapper.pks_by_table[self.mapper.mapped_table]:
bind = self.lazyreverse[primary_key]
ident.append(params[bind.key])
- return self.mapper.using(session).get(*ident)
+ return self.mapper.using(session).get(ident)
elif self.order_by is not False:
order_by = self.order_by
elif self.secondary is not None and self.secondary.default_order_by() is not None:
#print "EXEC NON-PRIAMRY", repr(self.mapper.class_), self.key
# we are not the primary manager for this attribute on this class - set up a per-instance lazyloader,
# which will override the class-level behavior
- objectstore.global_attributes.create_history(instance, self.key, self.uselist, callable_=self.setup_loader(instance))
+ sessionlib.global_attributes.create_history(instance, self.key, self.uselist, callable_=self.setup_loader(instance), cascade=self.cascade, trackparent=True)
else:
#print "EXEC PRIMARY", repr(self.mapper.class_), self.key
# we are the primary manager for this attribute on this class - reset its per-instance attribute state,
# so that the class-level lazy loader is executed when next referenced on this instance.
# this usually is not needed unless the constructor of the object referenced the attribute before we got
# to load data into it.
- objectstore.global_attributes.reset_history(instance, self.key)
+ sessionlib.global_attributes.reset_history(instance, self.key)
def create_lazy_clause(table, primaryjoin, secondaryjoin, foreignkey):
binds = {}
- reverselookup = {}
-
+ reverse = {}
def bind_label():
return "lazy_" + hex(random.randint(0, 65535))[2:]
-
+
def visit_binary(binary):
circular = isinstance(binary.left, schema.Column) and isinstance(binary.right, schema.Column) and binary.left.table is binary.right.table
if isinstance(binary.left, schema.Column) and isinstance(binary.right, schema.Column) and ((not circular and binary.left.table is table) or (circular and binary.right is foreignkey)):
col = binary.left
binary.left = binds.setdefault(binary.left,
sql.BindParamClause(bind_label(), None, shortname = binary.left.name))
- reverselookup[binary.right] = binds[col]
- #binary.swap()
+ reverse[binary.right] = binds[col]
if isinstance(binary.right, schema.Column) and isinstance(binary.left, schema.Column) and ((not circular and binary.right.table is table) or (circular and binary.left is foreignkey)):
col = binary.right
binary.right = binds.setdefault(binary.right,
sql.BindParamClause(bind_label(), None, shortname = binary.right.name))
- reverselookup[binary.left] = binds[col]
-
+ reverse[binary.left] = binds[col]
+
lazywhere = primaryjoin.copy_container()
li = BinaryVisitor(visit_binary)
lazywhere.accept_visitor(li)
- #print "PRIMARYJOIN", str(lazywhere), [b.key for b in binds.values()]
if secondaryjoin is not None:
lazywhere = sql.and_(lazywhere, secondaryjoin)
- return (lazywhere, binds, reverselookup)
+ return (lazywhere, binds, reverse)
-class EagerLoader(PropertyLoader):
+class EagerLoader(LazyLoader):
"""loads related objects inline with a parent query."""
def do_init_subclass(self, key, parent, recursion_stack=None):
+ if recursion_stack is None:
+ LazyLoader.do_init_subclass(self, key, parent)
parent._has_eager = True
self.eagertarget = self.target.alias()
if isinstance(prop, EagerLoader):
eagerprops.append(prop)
if len(eagerprops):
- recursion_stack[self.parent.table] = True
+ recursion_stack[self.parent.mapped_table] = True
self.mapper = self.mapper.copy()
try:
for prop in eagerprops:
continue
p = prop.copy()
self.mapper.props[prop.key] = p
-# print "we are:", id(self), self.target.name, (self.secondary and self.secondary.name or "None"), self.parent.table.name
-# print "prop is",id(prop), prop.target.name, (prop.secondary and prop.secondary.name or "None"), prop.parent.table.name
+# print "we are:", id(self), self.target.name, (self.secondary and self.secondary.name or "None"), self.parent.mapped_table.name
+# print "prop is",id(prop), prop.target.name, (prop.secondary and prop.secondary.name or "None"), prop.parent.mapped_table.name
p.do_init_subclass(prop.key, prop.parent, recursion_stack)
p._create_eager_chain(in_chain=True, recursion_stack=recursion_stack)
p.eagerprimary = p.eagerprimary.copy_container()
-# aliasizer = Aliasizer(p.parent.table, aliases={p.parent.table:self.eagertarget})
+# aliasizer = Aliasizer(p.parent.mapped_table, aliases={p.parent.mapped_table:self.eagertarget})
p.eagerprimary.accept_visitor(self.aliasizer)
- #print "new eagertqarget", p.eagertarget.name, (p.secondary and p.secondary.name or "none"), p.parent.table.name
+ #print "new eagertqarget", p.eagertarget.name, (p.secondary and p.secondary.name or "none"), p.parent.mapped_table.name
finally:
- del recursion_stack[self.parent.table]
+ del recursion_stack[self.parent.mapped_table]
self._row_decorator = self._create_decorator_row()
orderby = util.to_list(orderby)
for i in range(0, len(orderby)):
if isinstance(orderby[i], schema.Column):
- orderby[i] = self.eagertarget._get_col_by_original(orderby[i])
+ orderby[i] = self.eagertarget.corresponding_column(orderby[i])
else:
orderby[i].accept_visitor(self.aliasizer)
return orderby
if hasattr(statement, '_outerjoin'):
towrap = statement._outerjoin
else:
- towrap = self.parent.table
+ towrap = self.parent.mapped_table
# print "hello, towrap", str(towrap)
if self.secondaryjoin is not None:
"""receive a row. tell our mapper to look for a new object instance in the row, and attach
it to a list on the parent instance."""
+ decorated_row = self._decorate_row(row)
+ try:
+ # check for identity key
+ identity_key = self.mapper._row_identity_key(decorated_row)
+ except KeyError:
+ # else degrade to a lazy loader
+ LazyLoader.execute(self, session, instance, row, identitykey, imap, isnew)
+ return
+
if isnew:
# new row loaded from the database. initialize a blank container on the instance.
# this will override any per-class lazyloading type of stuff.
- h = objectstore.global_attributes.create_history(instance, self.key, self.uselist)
+ h = sessionlib.global_attributes.create_history(instance, self.key, self.uselist, cascade=self.cascade, trackparent=True)
if not self.uselist:
if isnew:
- h.setattr_clean(self._instance(session, row, imap))
+ h.setattr_clean(self.mapper._instance(session, decorated_row, imap, None))
else:
# call _instance on the row, even though the object has been created,
# so that we further descend into properties
- self._instance(session, row, imap)
+ self.mapper._instance(session, decorated_row, imap, None)
return
elif isnew:
result_list = h
else:
result_list = getattr(instance, self.key)
-
- self._instance(session, row, imap, result_list)
+ self.mapper._instance(session, decorated_row, imap, result_list)
def _create_decorator_row(self):
class DecoratorDict(object):
return map.keys()
map = {}
for c in self.eagertarget.c:
- parent = self.target._get_col_by_original(c.original)
+ parent = self.target.corresponding_column(c)
map[parent] = c
map[parent._label] = c
map[parent.name] = c
return DecoratorDict
-
- def _instance(self, session, row, imap, result_list=None):
- """gets an instance from a row, via this EagerLoader's mapper."""
+
+ def _decorate_row(self, row):
# since the EagerLoader makes an Alias of its mapper's table,
# we translate the actual result columns back to what they
# would normally be into a "virtual row" which is passed to the child mapper.
# (neither do any MapperExtensions). The row is keyed off the Column object
# (which is what mappers use) as well as its "label" (which might be what
# user-defined code is using)
- row = self._row_decorator(row)
- return self.mapper._instance(session, row, imap, result_list)
+ try:
+ return self._row_decorator(row)
+ except AttributeError:
+ self._create_eager_chain()
+ return self._row_decorator(row)
-class GenericOption(MapperOption):
+class GenericOption(mapper.MapperOption):
"""a mapper option that can handle dotted property names,
descending down through the relations of a mapper until it
reaches the target."""
"""called by the owning PropertyLoader to set up a backreference on the
PropertyLoader's mapper."""
# try to set a LazyLoader on our mapper referencing the parent mapper
+ mapper = prop.mapper.primary_mapper()
if not prop.mapper.props.has_key(self.key):
- if prop.secondaryjoin is not None:
- # if setting up a backref to a many-to-many, reverse the order
- # of the "primary" and "secondary" joins
- pj = prop.secondaryjoin
- sj = prop.primaryjoin
- else:
- pj = prop.primaryjoin
- sj = None
+ pj = self.kwargs.pop('primaryjoin', None)
+ sj = self.kwargs.pop('secondaryjoin', None)
+ # TODO: we are going to have the newly backref'd property create its
+ # primary/secondary join through normal means, and only override if they are
+ # specified to the constructor. think about if this is really going to work
+ # all the way.
+ #if pj is None:
+ # if prop.secondaryjoin is not None:
+ # # if setting up a backref to a many-to-many, reverse the order
+ # # of the "primary" and "secondary" joins
+ # pj = prop.secondaryjoin
+ # sj = prop.primaryjoin
+ # else:
+ # pj = prop.primaryjoin
+ # sj = None
lazy = self.kwargs.pop('lazy', True)
if lazy:
cls = LazyLoader
else:
cls = EagerLoader
- relation = cls(prop.parent, prop.secondary, pj, sj, backref=prop.key, is_backref=True, **self.kwargs)
- prop.mapper.add_property(self.key, relation);
+ # the backref property is set on the primary mapper
+ parent = prop.parent.primary_mapper()
+ relation = cls(parent, prop.secondary, pj, sj, backref=prop.key, is_backref=True, **self.kwargs)
+ mapper.add_property(self.key, relation);
else:
# else set one of us as the "backreference"
- if not prop.mapper.props[self.key].is_backref:
+ if not mapper.props[self.key].is_backref:
prop.is_backref=True
+ prop._dependency_processor.is_backref=True
def get_extension(self):
"""returns an attribute extension to use with this backreference."""
return attributes.GenericBackrefExtension(self.key)
for i in range(0, len(clist.clauses)):
if isinstance(clist.clauses[i], schema.Column) and self.tables.has_key(clist.clauses[i].table):
orig = clist.clauses[i]
- clist.clauses[i] = self.get_alias(clist.clauses[i].table)._get_col_by_original(clist.clauses[i])
+ clist.clauses[i] = self.get_alias(clist.clauses[i].table).corresponding_column(clist.clauses[i])
if clist.clauses[i] is None:
raise "cant get orig for " + str(orig) + " against table " + orig.table.name + " " + self.get_alias(orig.table).name
def visit_binary(self, binary):
if isinstance(binary.left, schema.Column) and self.tables.has_key(binary.left.table):
- binary.left = self.get_alias(binary.left.table)._get_col_by_original(binary.left)
+ binary.left = self.get_alias(binary.left.table).corresponding_column(binary.left)
if isinstance(binary.right, schema.Column) and self.tables.has_key(binary.right.table):
- binary.right = self.get_alias(binary.right.table)._get_col_by_original(binary.right)
+ binary.right = self.get_alias(binary.right.table).corresponding_column(binary.right)
class BinaryVisitor(sql.ClauseVisitor):
def __init__(self, func):
-# mapper/query.py
+# orm/query.py
# Copyright (C) 2005,2006 Michael Bayer mike_mp@zzzcomputing.com
#
# This module is part of SQLAlchemy and is released under
# the MIT License: http://www.opensource.org/licenses/mit-license.php
-
-import objectstore
-import sqlalchemy.sql as sql
-import sqlalchemy.util as util
+import session as sessionlib
+from sqlalchemy import sql, util, exceptions
import mapper
-from sqlalchemy.exceptions import *
class Query(object):
"""encapsulates the object-fetching operations provided by Mappers."""
- def __init__(self, mapper, **kwargs):
- self.mapper = mapper
+ def __init__(self, class_or_mapper, session=None, entity_name=None, **kwargs):
+ if isinstance(class_or_mapper, type):
+ self.mapper = class_mapper(class_or_mapper, entity_name=entity_name)
+ else:
+ self.mapper = class_or_mapper
+ self.mapper = self.mapper.get_select_mapper()
+
self.always_refresh = kwargs.pop('always_refresh', self.mapper.always_refresh)
self.order_by = kwargs.pop('order_by', self.mapper.order_by)
self.extension = kwargs.pop('extension', self.mapper.extension)
- self._session = kwargs.pop('session', None)
+ self._session = session
if not hasattr(mapper, '_get_clause'):
_get_clause = sql.and_()
for primary_key in self.mapper.pks_by_table[self.table]:
self._get_clause = self.mapper._get_clause
def _get_session(self):
if self._session is None:
- return objectstore.get_session()
+ return self.mapper.get_session()
else:
return self._session
- table = property(lambda s:s.mapper.table)
- props = property(lambda s:s.mapper.props)
+ table = property(lambda s:s.mapper.select_table)
session = property(_get_session)
- def get(self, *ident, **kwargs):
+ def get(self, ident, **kwargs):
"""returns an instance of the object based on the given identifier, or None
- if not found. The *ident argument is a
- list of primary key columns in the order of the table def's primary key columns."""
- key = self.mapper.identity_key(*ident)
- #print "key: " + repr(key) + " ident: " + repr(ident)
+ if not found. The ident argument is a scalar or tuple of primary key column values
+ in the order of the table def's primary key columns."""
+ key = self.mapper.identity_key(ident)
return self._get(key, ident, **kwargs)
+ def load(self, ident, **kwargs):
+ """returns an instance of the object based on the given identifier. If not found,
+ raises an exception. The method will *remove all pending changes* to the object
+ already existing in the Session. The ident argument is a scalar or tuple of primary
+ key column values in the order of the table def's primary key columns."""
+ key = self.mapper.identity_key(ident)
+ instance = self._get(key, ident, reload=True, **kwargs)
+ if instance is None:
+ raise exceptions.InvalidRequestError("No instance found for identity %s" % repr(ident))
+ return instance
+
def get_by(self, *args, **params):
"""returns a single object instance based on the given key/value criterion.
this is either the first value in the result list, or None if the list is
e.g. u = usermapper.get_by(user_name = 'fred')
"""
- x = self.select_whereclause(self._by_clause(*args, **params), limit=1)
+ x = self.select_whereclause(self.join_by(*args, **params), limit=1)
if x:
return x[0]
else:
"""returns an array of object instances based on the given clauses and key/value criterion.
*args is a list of zero or more ClauseElements which will be connected by AND operators.
+
**params is a set of zero or more key/value parameters which are converted into ClauseElements.
the keys are mapped to property or column names mapped by this mapper's Table, and the values
are coerced into a WHERE clause separated by AND operators. If the local property/column
ret = self.extension.select_by(self, *args, **params)
if ret is not mapper.EXT_PASS:
return ret
- return self.select_whereclause(self._by_clause(*args, **params))
+ return self.select_whereclause(self.join_by(*args, **params))
+
+ def join_by(self, *args, **params):
+ """like select_by, but returns a ClauseElement representing the WHERE clause that would normally
+ be sent to select_whereclause by select_by."""
+ clause = None
+ for arg in args:
+ if clause is None:
+ clause = arg
+ else:
+ clause &= arg
+ for key, value in params.iteritems():
+ (keys, prop) = self._locate_prop(key)
+ c = (prop.columns[0]==value) & self.join_via(keys)
+ if clause is None:
+ clause = c
+ else:
+ clause &= c
+ return clause
+
+ def _locate_prop(self, key):
+ import properties
+ keys = []
+ def search_for_prop(mapper):
+ if mapper.props.has_key(key):
+ prop = mapper.props[key]
+ if isinstance(prop, properties.PropertyLoader):
+ keys.insert(0, prop.key)
+ return prop
+ else:
+ for prop in mapper.props.values():
+ if not isinstance(prop, properties.PropertyLoader):
+ continue
+ x = search_for_prop(prop.mapper)
+ if x:
+ keys.insert(0, prop.key)
+ return x
+ else:
+ return None
+ p = search_for_prop(self.mapper)
+ if p is None:
+ raise exceptions.InvalidRequestError("Cant locate property named '%s'" % key)
+ return [keys, p]
+
+ def join_to(self, key):
+ """given the key name of a property, will recursively descend through all child properties
+ from this Query's mapper to locate the property, and will return a ClauseElement
+ representing a join from this Query's mapper to the endmost mapper."""
+ [keys, p] = self._locate_prop(key)
+ return self.join_via(keys)
+
+ def join_via(self, keys):
+ """given a list of keys that represents a path from this Query's mapper to a related mapper
+ based on names of relations from one mapper to the next, returns a
+ ClauseElement representing a join from this Query's mapper to the endmost mapper.
+ """
+ mapper = self.mapper
+ clause = None
+ for key in keys:
+ prop = mapper.props[key]
+ if clause is None:
+ clause = prop.get_join()
+ else:
+ clause &= prop.get_join()
+ mapper = prop.mapper
+
+ return clause
+
+
def selectfirst_by(self, *args, **params):
"""works like select_by(), but only returns the first result by itself, or None if no
objects returned. Synonymous with get_by()"""
def selectone_by(self, *args, **params):
"""works like selectfirst_by(), but throws an error if not exactly one result was returned."""
- ret = self.select_whereclause(self._by_clause(*args, **params), limit=2)
+ ret = self.select_whereclause(self.join_by(*args, **params), limit=2)
if len(ret) == 1:
return ret[0]
- raise InvalidRequestError('Multiple rows returned for selectone_by')
+ raise exceptions.InvalidRequestError('Multiple rows returned for selectone_by')
def count_by(self, *args, **params):
"""returns the count of instances based on the given clauses and key/value criterion.
The criterion is constructed in the same way as the select_by() method."""
- return self.count(self._by_clause(*args, **params))
+ return self.count(self.join_by(*args, **params))
def selectfirst(self, *args, **params):
"""works like select(), but only returns the first result by itself, or None if no
ret = list(self.select(*args, **params)[0:2])
if len(ret) == 1:
return ret[0]
- raise InvalidRequestError('Multiple rows returned for selectone')
+ raise exceptions.InvalidRequestError('Multiple rows returned for selectone')
def select(self, arg=None, **kwargs):
"""selects instances of the object from the database.
def count(self, whereclause=None, params=None, **kwargs):
s = self.table.count(whereclause)
- if params is not None:
- return s.scalar(**params)
- else:
- return s.scalar()
+ return self.session.scalar(self.mapper, s, params=params)
def select_statement(self, statement, **params):
return self._select_statement(statement, params=params)
def select_text(self, text, **params):
- t = sql.text(text, engine=self.mapper.primarytable.engine)
- return self.instances(t.execute(**params))
+ t = sql.text(text)
+ return self.instances(t, params=params)
+
+ def options(self, *args, **kwargs):
+ """returns a new Query object using the given MapperOptions."""
+ return self.mapper.options(*args, **kwargs).using(session=self._session)
def __getattr__(self, key):
if (key.startswith('select_by_')):
else:
raise AttributeError(key)
- def instances(self, *args, **kwargs):
- return self.mapper.instances(session=self.session, *args, **kwargs)
+ def instances(self, clauseelement, params=None, *args, **kwargs):
+ result = self.session.execute(self.mapper, clauseelement, params=params)
+ try:
+ return self.mapper.instances(result, self.session, **kwargs)
+ finally:
+ result.close()
- def _by_clause(self, *args, **params):
- clause = None
- for arg in args:
- if clause is None:
- clause = arg
- else:
- clause &= arg
- for key, value in params.iteritems():
- if value is False:
- continue
- c = self.mapper._get_criterion(key, value)
- if c is None:
- raise InvalidRequestError("Cant find criterion for property '"+ key + "'")
- if clause is None:
- clause = c
- else:
- clause &= c
- return clause
-
def _get(self, key, ident=None, reload=False):
if not reload and not self.always_refresh:
try:
if ident is None:
ident = key[1]
+ else:
+ ident = util.to_list(ident)
i = 0
params = {}
for primary_key in self.mapper.pks_by_table[self.table]:
statement.use_labels = True
if params is None:
params = {}
- return self.instances(statement.execute(**params), **kwargs)
+ return self.instances(statement, params=params, **kwargs)
def _should_nest(self, **kwargs):
"""returns True if the given statement options indicate that we should "nest" the
if order_by is False:
if self.table.default_order_by() is not None:
order_by = self.table.default_order_by()
-
+
if self._should_nest(**kwargs):
from_obj.append(self.table)
s2 = sql.select(self.table.primary_key, whereclause, use_labels=True, from_obj=from_obj, **kwargs)
--- /dev/null
+# objectstore.py
+# Copyright (C) 2005,2006 Michael Bayer mike_mp@zzzcomputing.com
+#
+# This module is part of SQLAlchemy and is released under
+# the MIT License: http://www.opensource.org/licenses/mit-license.php
+
+from sqlalchemy import util, exceptions, sql
+import unitofwork, query
+import weakref
+import sqlalchemy
+
+class SessionTransaction(object):
+ def __init__(self, session, parent=None, autoflush=True):
+ self.session = session
+ self.connections = {}
+ self.parent = parent
+ self.autoflush = autoflush
+ def connection(self, mapper_or_class, entity_name=None):
+ if isinstance(mapper_or_class, type):
+ mapper_or_class = class_mapper(mapper_or_class, entity_name=entity_name)
+ if self.parent is not None:
+ return self.parent.connection(mapper_or_class)
+ engine = self.session.get_bind(mapper_or_class)
+ return self.get_or_add(engine)
+ def _begin(self):
+ return SessionTransaction(self.session, self)
+ def add(self, connectable):
+ if self.connections.has_key(connectable.engine):
+ raise exceptions.InvalidRequestError("Session already has a Connection associated for the given Connection's Engine")
+ return self.get_or_add(connectable)
+ def get_or_add(self, connectable):
+ # we reference the 'engine' attribute on the given object, which in the case of
+ # Connection, ProxyEngine, Engine, ComposedSQLEngine, whatever, should return the original
+ # "Engine" object that is handling the connection.
+ if self.connections.has_key(connectable.engine):
+ return self.connections[connectable.engine][0]
+ e = connectable.engine
+ c = connectable.contextual_connect()
+ if not self.connections.has_key(e):
+ self.connections[e] = (c, c.begin())
+ return self.connections[e][0]
+ def commit(self):
+ if self.parent is not None:
+ return
+ if self.autoflush:
+ self.session.flush()
+ for t in self.connections.values():
+ t[1].commit()
+ self.close()
+ def rollback(self):
+ if self.parent is not None:
+ self.parent.rollback()
+ return
+ for k, t in self.connections.iteritems():
+ t[1].rollback()
+ self.close()
+ def close(self):
+ if self.parent is not None:
+ return
+ for t in self.connections.values():
+ t[0].close()
+ self.session.transaction = None
+
+class Session(object):
+ """encapsulates a set of objects being operated upon within an object-relational operation."""
+ def __init__(self, bind_to=None, hash_key=None, import_session=None, echo_uow=False):
+ if import_session is not None:
+ self.uow = unitofwork.UnitOfWork(identity_map=import_session.uow.identity_map)
+ else:
+ self.uow = unitofwork.UnitOfWork()
+
+ self.bind_to = bind_to
+ self.binds = {}
+ self.echo_uow = echo_uow
+ self.transaction = None
+ if hash_key is None:
+ self.hash_key = id(self)
+ else:
+ self.hash_key = hash_key
+ _sessions[self.hash_key] = self
+
+ def create_transaction(self, **kwargs):
+ """returns a new SessionTransaction corresponding to an existing or new transaction.
+ if the transaction is new, the returned SessionTransaction will have commit control
+ over the underlying transaction, else will have rollback control only."""
+ if self.transaction is not None:
+ return self.transaction._begin()
+ else:
+ self.transaction = SessionTransaction(self, **kwargs)
+ return self.transaction
+ def connect(self, mapper=None, **kwargs):
+ """returns a unique connection corresponding to the given mapper. this connection
+ will not be part of any pre-existing transactional context."""
+ return self.get_bind(mapper).connect(**kwargs)
+ def connection(self, mapper, **kwargs):
+ """returns a Connection corresponding to the given mapper. used by the execute()
+ method which performs select operations for Mapper and Query.
+ if this Session is transactional,
+ the connection will be in the context of this session's transaction. otherwise, the connection
+ is returned by the contextual_connect method, which some Engines override to return a thread-local
+ connection, and will have close_with_result set to True.
+
+ the given **kwargs will be sent to the engine's contextual_connect() method, if no transaction is in progress."""
+ if self.transaction is not None:
+ return self.transaction.connection(mapper)
+ else:
+ return self.get_bind(mapper).contextual_connect(**kwargs)
+ def execute(self, mapper, clause, params, **kwargs):
+ """using the given mapper to identify the appropriate Engine or Connection to be used for statement execution,
+ executes the given ClauseElement using the provided parameter dictionary. Returns a ResultProxy corresponding
+ to the execution's results. If this method allocates a new Connection for the operation, then the ResultProxy's close()
+ method will release the resources of the underlying Connection, otherwise its a no-op.
+ """
+ return self.connection(mapper, close_with_result=True).execute(clause, params, **kwargs)
+ def scalar(self, mapper, clause, params, **kwargs):
+ """works like execute() but returns a scalar result."""
+ return self.connection(mapper, close_with_result=True).scalar(clause, params, **kwargs)
+
+ def close(self):
+ """closes this Session.
+ """
+ self.clear()
+ if self.transaction is not None:
+ self.transaction.close()
+
+ def clear(self):
+ """removes all object instances from this Session. this is equivalent to calling expunge() for all
+ objects in this Session."""
+ for instance in self:
+ self._unattach(instance)
+ self.uow = unitofwork.UnitOfWork()
+
+ def mapper(self, class_, entity_name=None):
+ """given an Class, returns the primary Mapper responsible for persisting it"""
+ return class_mapper(class_, entity_name = entity_name)
+ def bind_mapper(self, mapper, bindto):
+ """binds the given Mapper to the given Engine or Connection. All subsequent operations involving this
+ Mapper will use the given bindto."""
+ self.binds[mapper] = bindto
+ def bind_table(self, table, bindto):
+ """binds the given Table to the given Engine or Connection. All subsequent operations involving this
+ Table will use the given bindto."""
+ self.binds[table] = bindto
+ def get_bind(self, mapper):
+ """given a Mapper, returns the Engine or Connection which is used to execute statements on behalf of this
+ Mapper. Calling connect() on the return result will always result in a Connection object. This method
+ disregards any SessionTransaction that may be in progress.
+
+ The order of searching is as follows:
+
+ if an Engine or Connection was bound to this Mapper specifically within this Session, returns that
+ Engine or Connection.
+
+ if an Engine or Connection was bound to this Mapper's underlying Table within this Session
+ (i.e. not to the Table directly), returns that Engine or Conneciton.
+
+ if an Engine or Connection was bound to this Session, returns that Engine or Connection.
+
+ finally, returns the Engine which was bound directly to the Table's MetaData object.
+
+ If no Engine is bound to the Table, an exception is raised.
+ """
+ if mapper is None:
+ return self.bind_to
+ elif self.binds.has_key(mapper):
+ return self.binds[mapper]
+ elif self.binds.has_key(mapper.mapped_table):
+ return self.binds[mapper.mapped_table]
+ elif self.bind_to is not None:
+ return self.bind_to
+ else:
+ e = mapper.mapped_table.engine
+ if e is None:
+ raise exceptions.InvalidRequestError("Could not locate any Engine bound to mapper '%s'" % str(mapper))
+ return e
+ def query(self, mapper_or_class, entity_name=None):
+ """given a mapper or Class, returns a new Query object corresponding to this Session and the mapper, or the classes' primary mapper."""
+ if isinstance(mapper_or_class, type):
+ return query.Query(class_mapper(mapper_or_class, entity_name=entity_name), self)
+ else:
+ return query.Query(mapper_or_class, self)
+ def _sql(self):
+ class SQLProxy(object):
+ def __getattr__(self, key):
+ def call(*args, **kwargs):
+ kwargs[engine] = self.engine
+ return getattr(sql, key)(*args, **kwargs)
+
+ sql = property(_sql)
+
+
+ def get_id_key(ident, class_, entity_name=None):
+ """returns an identity-map key for use in storing/retrieving an item from the identity
+ map, given a tuple of the object's primary key values.
+
+ ident - a tuple of primary key values corresponding to the object to be stored. these
+ values should be in the same order as the primary keys of the table
+
+ class_ - a reference to the object's class
+
+ entity_name - optional string name to further qualify the class
+ """
+ return (class_, tuple(ident), entity_name)
+ get_id_key = staticmethod(get_id_key)
+
+ def get_row_key(row, class_, primary_key, entity_name=None):
+ """returns an identity-map key for use in storing/retrieving an item from the identity
+ map, given a result set row.
+
+ row - a sqlalchemy.dbengine.RowProxy instance or other map corresponding result-set
+ column names to their values within a row.
+
+ class_ - a reference to the object's class
+
+ primary_key - a list of column objects that will target the primary key values
+ in the given row.
+
+ entity_name - optional string name to further qualify the class
+ """
+ return (class_, tuple([row[column] for column in primary_key]), entity_name)
+ get_row_key = staticmethod(get_row_key)
+
+ def begin(self, *obj):
+ """deprecated"""
+ raise exceptions.InvalidRequestError("Session.begin() is deprecated. use install_mod('legacy_session') to enable the old behavior")
+ def commit(self, *obj):
+ """deprecated"""
+ raise exceptions.InvalidRequestError("Session.commit() is deprecated. use install_mod('legacy_session') to enable the old behavior")
+
+ def flush(self, objects=None):
+ """flushes all the object modifications present in this session to the database. 'objects'
+ is a list or tuple of objects specifically to be flushed."""
+ self.uow.flush(self, objects, echo=self.echo_uow)
+
+ def get(self, class_, ident, **kwargs):
+ """returns an instance of the object based on the given identifier, or None
+ if not found. The ident argument is a scalar or tuple of primary key column values in the order of the
+ table def's primary key columns.
+
+ the entity_name keyword argument may also be specified which further qualifies the underlying
+ Mapper used to perform the query."""
+ entity_name = kwargs.get('entity_name', None)
+ return self.query(class_, entity_name=entity_name).get(ident)
+
+ def load(self, class_, ident, **kwargs):
+ """returns an instance of the object based on the given identifier. If not found,
+ raises an exception. The method will *remove all pending changes* to the object
+ already existing in the Session. The ident argument is a scalar or tuple of
+ primary key columns in the order of the table def's primary key columns.
+
+ the entity_name keyword argument may also be specified which further qualifies the underlying
+ Mapper used to perform the query."""
+ entity_name = kwargs.get('entity_name', None)
+ return self.query(class_, entity_name=entity_name).load(ident)
+
+ def refresh(self, object):
+ """reloads the attributes for the given object from the database, clears
+ any changes made."""
+ self.uow.refresh(self, object)
+
+ def expire(self, object):
+ """invalidates the data in the given object and sets them to refresh themselves
+ the next time they are requested."""
+ self.uow.expire(self, object)
+
+ def expunge(self, object):
+ """removes the given object from this Session. this will free all internal references to the object."""
+ self.uow.expunge(object)
+
+ def save(self, object, entity_name=None):
+ """
+ Adds a transient (unsaved) instance to this Session. This operation cascades the "save_or_update"
+ method to associated instances if the relation is mapped with cascade="save-update".
+
+ The 'entity_name' keyword argument will further qualify the specific Mapper used to handle this
+ instance.
+ """
+ for c in object_mapper(object, entity_name=entity_name).cascade_iterator('save-update', object):
+ if c is object:
+ self._save_impl(c, entity_name=entity_name)
+ else:
+ self.save_or_update(c, entity_name=entity_name)
+
+ def update(self, object, entity_name=None):
+ """Brings the given detached (saved) instance into this Session.
+ If there is a persistent instance with the same identifier (i.e. a saved instance already associated with this
+ Session), an exception is thrown.
+ This operation cascades the "save_or_update" method to associated instances if the relation is mapped
+ with cascade="save-update"."""
+ for c in object_mapper(object, entity_name=entity_name).cascade_iterator('save-update', object):
+ if c is object:
+ self._update_impl(c, entity_name=entity_name)
+ else:
+ self.save_or_update(c, entity_name=entity_name)
+
+ def save_or_update(self, object, entity_name=None):
+ for c in object_mapper(object, entity_name=entity_name).cascade_iterator('save-update', object):
+ key = getattr(object, '_instance_key', None)
+ if key is None:
+ self._save_impl(c, entity_name=entity_name)
+ else:
+ self._update_impl(c, entity_name=entity_name)
+
+ def delete(self, object, entity_name=None):
+ for c in object_mapper(object, entity_name=entity_name).cascade_iterator('delete', object):
+ self.uow.register_deleted(c)
+
+ def merge(self, object, entity_name=None):
+ instance = None
+ for obj in object_mapper(object, entity_name=entity_name).cascade_iterator('merge', object):
+ key = getattr(obj, '_instance_key', None)
+ if key is None:
+ mapper = object_mapper(object, entity_name=entity_name)
+ ident = mapper.identity(object)
+ for k in ident:
+ if k is None:
+ raise exceptions.InvalidRequestError("Instance '%s' does not have a full set of identity values, and does not represent a saved entity in the database. Use the add() method to add unsaved instances to this Session." % repr(obj))
+ key = mapper.identity_key(ident)
+ u = self.uow
+ if u.identity_map.has_key(key):
+ # TODO: copy the state of the given object into this one. tricky !
+ inst = u.identity_map[key]
+ else:
+ inst = self.get(object.__class__, *key[1])
+ if obj is object:
+ instance = inst
+
+ return instance
+
+ def _save_impl(self, object, **kwargs):
+ if hasattr(object, '_instance_key'):
+ if not self.uow.has_key(object._instance_key):
+ raise exceptions.InvalidRequestError("Instance '%s' is already persistent in a different Session" % repr(object))
+ else:
+ entity_name = kwargs.get('entity_name', None)
+ if entity_name is not None:
+ m = class_mapper(object.__class__, entity_name=entity_name)
+ m._assign_entity_name(object)
+ self._register_new(object)
+
+ def _update_impl(self, object, **kwargs):
+ if self._is_attached(object) and object not in self.deleted:
+ return
+ if not hasattr(object, '_instance_key'):
+ raise exceptions.InvalidRequestError("Instance '%s' is not persisted" % repr(object))
+ if global_attributes.is_modified(object):
+ self._register_dirty(object)
+ else:
+ self._register_clean(object)
+
+ def _register_changed(self, obj):
+ if hasattr(obj, '_instance_key'):
+ self._register_dirty(obj)
+ else:
+ self._register_new(obj)
+ def _register_new(self, obj):
+ self._attach(obj)
+ self.uow.register_new(obj)
+ def _register_dirty(self, obj):
+ self._attach(obj)
+ self.uow.register_dirty(obj)
+ def _register_clean(self, obj):
+ self._attach(obj)
+ self.uow.register_clean(obj)
+ def _register_deleted(self, obj):
+ self._attach(obj)
+ self.uow.register_deleted(obj)
+
+ def _attach(self, obj):
+ """given an object, attaches it to this session. """
+ if getattr(obj, '_sa_session_id', None) != self.hash_key:
+ old = getattr(obj, '_sa_session_id', None)
+ if old is not None:
+ raise exceptions.InvalidRequestError("Object '%s' is already attached to session '%s' (this is '%s')" % (repr(obj), old, id(self)))
+
+ # auto-removal from the old session is disabled. but if we decide to
+ # turn it back on, do it as below: gingerly since _sessions is a WeakValueDict
+ # and it might be affected by other threads
+ try:
+ sess = _sessions[old]
+ except KeyError:
+ sess = None
+ if sess is not None:
+ sess.expunge(old)
+ key = getattr(obj, '_instance_key', None)
+ if key is not None:
+ self.identity_map[key] = obj
+ obj._sa_session_id = self.hash_key
+
+ def _unattach(self, obj):
+ if not self._is_attached(obj): #getattr(obj, '_sa_session_id', None) != self.hash_key:
+ raise exceptions.InvalidRequestError("Object '%s' is not attached to this Session" % repr(obj))
+ del obj._sa_session_id
+
+ def _is_attached(self, obj):
+ return getattr(obj, '_sa_session_id', None) == self.hash_key
+ def __contains__(self, obj):
+ return self._is_attached(obj) and (obj in self.uow.new or self.uow.has_key(obj._instance_key))
+ def __iter__(self):
+ return iter(self.uow.identity_map.values())
+ def _get(self, key):
+ return self.uow._get(key)
+ def has_key(self, key):
+ return self.uow.has_key(key)
+ def is_expired(self, instance, **kwargs):
+ return self.uow.is_expired(instance, **kwargs)
+
+ dirty = property(lambda s:s.uow.dirty, doc="a Set of all objects marked as 'dirty' within this Session")
+ deleted = property(lambda s:s.uow.deleted, doc="a Set of all objects marked as 'deleted' within this Session")
+ new = property(lambda s:s.uow.new, doc="a Set of all objects marked as 'new' within this Session.")
+ identity_map = property(lambda s:s.uow.identity_map, doc="a WeakValueDictionary consisting of all objects within this Session keyed to their _instance_key value.")
+
+
+ def import_instance(self, *args, **kwargs):
+ """deprecated; a synynom for merge()"""
+ return self.merge(*args, **kwargs)
+
+def get_id_key(ident, class_, entity_name=None):
+ return Session.get_id_key(ident, class_, entity_name)
+
+def get_row_key(row, class_, primary_key, entity_name=None):
+ return Session.get_row_key(row, class_, primary_key, entity_name)
+
+def object_mapper(obj, **kwargs):
+ return sqlalchemy.orm.object_mapper(obj, **kwargs)
+
+def class_mapper(class_, **kwargs):
+ return sqlalchemy.orm.class_mapper(class_, **kwargs)
+
+# this is the AttributeManager instance used to provide attribute behavior on objects.
+# to all the "global variable police" out there: its a stateless object.
+global_attributes = unitofwork.global_attributes
+
+# this dictionary maps the hash key of a Session to the Session itself, and
+# acts as a Registry with which to locate Sessions. this is to enable
+# object instances to be associated with Sessions without having to attach the
+# actual Session object directly to the object instance.
+_sessions = weakref.WeakValueDictionary()
+
+def object_session(obj):
+ hashkey = getattr(obj, '_sa_session_id', None)
+ if hashkey is not None:
+ return _sessions.get(hashkey)
+ return None
+
+unitofwork.object_session = object_session
+
+
+def get_session(obj=None):
+ """deprecated"""
+ if obj is not None:
+ return object_session(obj)
+ raise exceptions.InvalidRequestError("get_session() is deprecated, and does not return the thread-local session anymore. Use the SessionContext.mapper_extension or import sqlalchemy.mod.threadlocal to establish a default thread-local context.")
import sqlalchemy.schema as schema
from sqlalchemy.exceptions import *
-"""contains the ClauseSynchronizer class which is used to map attributes between two objects
+"""contains the ClauseSynchronizer class, which is used to map attributes between two objects
in a manner corresponding to a SQL clause that compares column values."""
ONETOMANY = 0
n.cycles = Set([n])
continue
else:
- raise CommitError("Self-referential dependency detected " + repr(t))
+ raise FlushError("Self-referential dependency detected " + repr(t))
childnode = nodes[t[1]]
parentnode = nodes[t[0]]
self._add_edge(edges, (parentnode, childnode))
continue
else:
# long cycles not allowed
- raise CommitError("Circular dependency detected " + repr(edges) + repr(queue))
+ raise FlushError("Circular dependency detected " + repr(edges) + repr(queue))
node = queue.pop()
if not hasattr(node, '_cyclical'):
output.append(node)
elif parentnode.is_descendant_of(childnode):
# check for a line thats backwards with nodes in between, this is a
# circular dependency (although confirmation on this would be helpful)
- raise CommitError("Circular dependency detected")
+ raise FlushError("Circular dependency detected")
elif not childnode.is_descendant_of(parentnode):
# if relationship doesnt exist, connect nodes together
root = childnode.get_sibling_ancestor(parentnode)
-# unitofwork.py
+# orm/unitofwork.py
# Copyright (C) 2005,2006 Michael Bayer mike_mp@zzzcomputing.com
#
# This module is part of SQLAlchemy and is released under
super(UOWProperty, self).__init__(*args, **kwargs)
self.class_ = class_
property = property(lambda s:class_mapper(s.class_).props[s.key], doc="returns the MapperProperty object associated with this property")
-
-class UOWListElement(attributes.ListElement):
+
+
+class UOWListElement(attributes.ListAttribute):
"""overrides ListElement to provide unit-of-work "dirty" hooks when list attributes are modified,
plus specialzed append() method."""
- def __init__(self, obj, key, data=None, deleteremoved=False, **kwargs):
- attributes.ListElement.__init__(self, obj, key, data=data, **kwargs)
- self.deleteremoved = deleteremoved
- def list_value_changed(self, obj, key, item, listval, isdelete):
- sess = get_session(obj)
- if not isdelete and sess.deleted.contains(item):
- #raise InvalidRequestError("re-inserting a deleted value into a list")
- del sess.deleted[item]
- sess.modified_lists.append(self)
- if self.deleteremoved and isdelete:
- sess.register_deleted(item)
+ def __init__(self, obj, key, data=None, cascade=None, **kwargs):
+ attributes.ListAttribute.__init__(self, obj, key, data=data, **kwargs)
+ self.cascade = cascade
+ def do_value_changed(self, obj, key, item, listval, isdelete):
+ sess = object_session(obj)
+ if sess is not None:
+ sess._register_changed(obj)
+ if self.cascade is not None:
+ if not isdelete:
+ if self.cascade.save_update:
+ sess.save_or_update(item)
def append(self, item, _mapper_nohistory = False):
if _mapper_nohistory:
self.append_nohistory(item)
else:
- attributes.ListElement.append(self, item)
+ attributes.ListAttribute.append(self, item)
+
+class UOWScalarElement(attributes.ScalarAttribute):
+ def __init__(self, obj, key, cascade=None, **kwargs):
+ attributes.ScalarAttribute.__init__(self, obj, key, **kwargs)
+ self.cascade=cascade
+ def do_value_changed(self, oldvalue, newvalue):
+ obj = self.obj
+ sess = object_session(obj)
+ if sess is not None:
+ sess._register_changed(obj)
+ if newvalue is not None and self.cascade is not None:
+ if self.cascade.save_update:
+ sess.save_or_update(newvalue)
class UOWAttributeManager(attributes.AttributeManager):
"""overrides AttributeManager to provide unit-of-work "dirty" hooks when scalar attribues are modified, plus factory methods for UOWProperrty/UOWListElement."""
def __init__(self):
attributes.AttributeManager.__init__(self)
- def value_changed(self, obj, key, value):
- if hasattr(obj, '_instance_key'):
- get_session(obj).register_dirty(obj)
- else:
- get_session(obj).register_new(obj)
-
def create_prop(self, class_, key, uselist, callable_, **kwargs):
return UOWProperty(class_, self, key, uselist, callable_, **kwargs)
+ def create_scalar(self, obj, key, **kwargs):
+ return UOWScalarElement(obj, key, **kwargs)
+
def create_list(self, obj, key, list_, **kwargs):
return UOWListElement(obj, key, list_, **kwargs)
class UnitOfWork(object):
- """main UOW object which stores lists of dirty/new/deleted objects, as well as 'modified_lists' for list attributes. provides top-level "flush" functionality as well as the transaction boundaries with the SQLEngine(s) involved in a write operation."""
+ """main UOW object which stores lists of dirty/new/deleted objects. provides top-level "flush" functionality as well as the transaction boundaries with the SQLEngine(s) involved in a write operation."""
def __init__(self, identity_map=None):
if identity_map is not None:
self.identity_map = identity_map
self.attributes = global_attributes
self.new = util.HashSet(ordered = True)
self.dirty = util.HashSet()
- self.modified_lists = util.HashSet()
+
self.deleted = util.HashSet()
def get(self, class_, *id):
def _put(self, key, obj):
self.identity_map[key] = obj
- def refresh(self, obj):
+ def refresh(self, sess, obj):
self.rollback_object(obj)
- object_mapper(obj)._get(obj._instance_key, reload=True)
+ sess.query(obj.__class__)._get(obj._instance_key, reload=True)
- def expire(self, obj):
+ def expire(self, sess, obj):
self.rollback_object(obj)
def exp():
- object_mapper(obj)._get(obj._instance_key, reload=True)
+ sess.query(obj.__class__)._get(obj._instance_key, reload=True)
global_attributes.trigger_history(obj, exp)
def is_expired(self, obj, unexpire=False):
self.attributes.commit(obj)
def register_new(self, obj):
+ if hasattr(obj, '_instance_key'):
+ raise InvalidRequestError("Object '%s' already has an identity - it cant be registered as new" % repr(obj))
if not self.new.contains(obj):
self.new.append(obj)
+ self.unregister_deleted(obj)
def register_dirty(self, obj):
if not self.dirty.contains(obj):
self._validate_obj(obj)
self.dirty.append(obj)
-
+ self.unregister_deleted(obj)
+
def is_dirty(self, obj):
if not self.dirty.contains(obj):
return False
if not self.deleted.contains(obj):
self._validate_obj(obj)
self.deleted.append(obj)
- mapper = object_mapper(obj)
- # TODO: should the cascading delete dependency thing
- # happen wtihin PropertyLoader.process_dependencies ?
- mapper.register_deleted(obj, self)
def unregister_deleted(self, obj):
try:
except KeyError:
pass
- def flush(self, session, *objects):
+ def flush(self, session, objects=None, echo=False):
flush_context = UOWTransaction(self, session)
- if len(objects):
+ if objects is not None:
objset = util.HashSet(iter=objects)
else:
objset = None
if self.deleted.contains(obj):
continue
flush_context.register_object(obj)
- for item in self.modified_lists:
- obj = item.obj
- if objset is not None and not objset.contains(obj):
- continue
- if self.deleted.contains(obj):
- continue
- flush_context.register_object(obj, listonly = True)
- flush_context.register_saved_history(item)
-
-# for o in item.added_items() + item.deleted_items():
-# if self.deleted.contains(o):
-# continue
-# flush_context.register_object(o, listonly=True)
-
+
for obj in self.deleted:
if objset is not None and not objset.contains(obj):
continue
flush_context.register_object(obj, isdelete=True)
-
- engines = util.HashSet()
- for mapper in flush_context.mappers:
- for e in session.engines(mapper):
- engines.append(e)
-
- echo_commit = False
- for e in engines:
- echo_commit = echo_commit or e.echo_uow
- e.begin()
+
+ trans = session.create_transaction(autoflush=False)
+ flush_context.transaction = trans
try:
- flush_context.execute(echo=echo_commit)
+ flush_context.execute(echo=echo)
+ trans.commit()
except:
- for e in engines:
- e.rollback()
+ trans.rollback()
raise
- for e in engines:
- e.commit()
flush_context.post_exec()
self.mappers = util.HashSet()
self.dependencies = {}
self.tasks = {}
- self.saved_histories = util.HashSet()
self.__modified = False
+ self.__is_executing = False
def register_object(self, obj, isdelete = False, listonly = False, postupdate=False, **kwargs):
"""adds an object to this UOWTransaction to be updated in the database.
save/delete operation on the object itself, unless an additional save/delete
registration is entered for the object."""
#print "REGISTER", repr(obj), repr(getattr(obj, '_instance_key', None)), str(isdelete), str(listonly)
+
# things can get really confusing if theres duplicate instances floating around,
# so make sure everything is OK
self.uow._validate_obj(obj)
mapper = object_mapper(obj)
self.mappers.append(mapper)
task = self.get_task_by_mapper(mapper)
-
+
if postupdate:
mod = task.append_postupdate(obj)
- self.__modified = self.__modified or mod
+ if mod: self._mark_modified()
return
-
+
# for a cyclical task, things need to be sorted out already,
# so this object should have already been added to the appropriate sub-task
# can put an assertion here to make sure....
if task.circular:
return
-
+
mod = task.append(obj, listonly, isdelete=isdelete, **kwargs)
- self.__modified = self.__modified or mod
+ if mod: self._mark_modified()
def unregister_object(self, obj):
mapper = object_mapper(obj)
task = self.get_task_by_mapper(mapper)
- task.delete(obj)
+ if obj in task.objects:
+ task.delete(obj)
+ self._mark_modified()
+
+ def _mark_modified(self):
+ #if self.__is_executing:
+ # raise "test assertion failed"
self.__modified = True
def get_task_by_mapper(self, mapper):
try:
return self.tasks[mapper]
except KeyError:
- return UOWTask(self, mapper)
+ task = UOWTask(self, mapper)
+ task.mapper.register_dependencies(self)
+ return task
def register_dependency(self, mapper, dependency):
"""called by mapper.PropertyLoader to register the objects handled by
one mapper being dependent on the objects handled by another."""
# correct for primary mapper (the mapper offcially associated with the class)
self.dependencies[(mapper._primary_mapper(), dependency._primary_mapper())] = True
- self.__modified = True
+ self._mark_modified()
- def register_processor(self, mapper, processor, mapperfrom, isdeletefrom):
+ def register_processor(self, mapper, processor, mapperfrom):
"""called by mapper.PropertyLoader to register itself as a "processor", which
will be associated with a particular UOWTask, and be given a list of "dependent"
objects corresponding to another UOWTask to be processed, either after that secondary
# when the task from "mapper" executes, take the objects from the task corresponding
# to "mapperfrom"'s list of save/delete objects, and send them to "processor"
# for dependency processing
- #print "registerprocessor", str(mapper), repr(processor.key), str(mapperfrom), repr(isdeletefrom)
+ #print "registerprocessor", str(mapper), repr(processor.key), str(mapperfrom)
# correct for primary mapper (the mapper offcially associated with the class)
mapper = mapper._primary_mapper()
mapperfrom = mapperfrom._primary_mapper()
task = self.get_task_by_mapper(mapper)
targettask = self.get_task_by_mapper(mapperfrom)
- task.dependencies.append(UOWDependencyProcessor(processor, targettask, isdeletefrom))
- self.__modified = True
-
- def register_saved_history(self, listobj):
- self.saved_histories.append(listobj)
+ up = UOWDependencyProcessor(processor, targettask)
+ task.dependencies.append(up)
+ self._mark_modified()
def execute(self, echo=False):
- for task in self.tasks.values():
- task.mapper.register_dependencies(self)
-
+ # pre-execute dependency processors. this process may
+ # result in new tasks, objects and/or dependency processors being added,
+ # particularly with 'delete-orphan' cascade rules.
+ # keep running through the full list of tasks until all
+ # objects have been processed.
+ while True:
+ ret = False
+ for task in self.tasks.values():
+ for up in task.dependencies:
+ if up.preexecute(self):
+ ret = True
+ if not ret:
+ break
+
+ # flip the execution flag on. in some test cases
+ # we like to check this flag against any new objects being added, since everything
+ # should be registered by now. there is a slight exception in the case of
+ # post_update requests; this should be fixed.
+ self.__is_executing = True
+
head = self._sort_dependencies()
self.__modified = False
if LOG or echo:
print "Task dump:\n" + head.dump()
if head is not None:
head.execute(self)
+ #if self.__modified and head is not None:
+ # raise "Assertion failed ! new pre-execute dependency step should eliminate post-execute changes (except post_update stuff)."
if LOG or echo:
- if self.__modified and head is not None:
- print "\nAfter Execute:\n" + head.dump()
- else:
- print "\nExecute complete (no post-exec changes)\n"
+ print "\nExecute complete\n"
def post_exec(self):
"""after an execute/flush is completed, all of the objects and lists that have
self.uow._remove_deleted(elem.obj)
else:
self.uow.register_clean(elem.obj)
-
- for obj in self.saved_histories:
- try:
- obj.commit()
- del self.uow.modified_lists[obj]
- except KeyError:
- pass
-
- # this assertion only applies to a full flush(), not a
- # partial one
- #if len(self.uow.new) > 0 or len(self.uow.dirty) >0 or len(self.uow.modified_lists) > 0:
- # raise "assertion failed"
def _sort_dependencies(self):
"""creates a hierarchical tree of dependent tasks. the root node is returned.
mappers = util.HashSet()
for task in self.tasks.values():
mappers.append(task.mapper)
+
head = DependencySorter(self.dependencies, mappers).sort(allow_all_cycles=True)
#print str(head)
task = sort_hier(head)
"""an element within a UOWTask. corresponds to a single object instance
to be saved, deleted, or just part of the transaction as a placeholder for
further dependencies (i.e. 'listonly').
- in the case of self-referential mappers, may also store a "childtask", which is a
- UOWTask containing objects dependent on this element's object instance."""
+ in the case of self-referential mappers, may also store a list of childtasks,
+ further UOWTasks containing objects dependent on this element's object instance."""
def __init__(self, obj):
self.obj = obj
- self.listonly = True
+ self.__listonly = True
self.childtasks = []
- self.isdelete = False
- self.mapper = None
+ self.__isdelete = False
+ self.__preprocessed = {}
+ def _get_listonly(self):
+ return self.__listonly
+ def _set_listonly(self, value):
+ """set_listonly is a one-way setter, will only go from True to False."""
+ if not value and self.__listonly:
+ self.__listonly = False
+ self.clear_preprocessed()
+ def _get_isdelete(self):
+ return self.__isdelete
+ def _set_isdelete(self, value):
+ if self.__isdelete is not value:
+ self.__isdelete = value
+ self.clear_preprocessed()
+ listonly = property(_get_listonly, _set_listonly)
+ isdelete = property(_get_isdelete, _set_isdelete)
+
+ def mark_preprocessed(self, processor):
+ """marks this element as "preprocessed" by a particular UOWDependencyProcessor. preprocessing is the step
+ which sweeps through all the relationships on all the objects in the flush transaction and adds other objects
+ which are also affected, In some cases it can switch an object from "tosave" to "todelete". changes to the state
+ of this UOWTaskElement will reset all "preprocessed" flags, causing it to be preprocessed again. When all UOWTaskElements
+ have been fully preprocessed by all UOWDependencyProcessors, then the topological sort can be done."""
+ self.__preprocessed[processor] = True
+ def is_preprocessed(self, processor):
+ return self.__preprocessed.get(processor, False)
+ def clear_preprocessed(self):
+ self.__preprocessed.clear()
def __repr__(self):
return "UOWTaskElement/%d: %s/%d %s" % (id(self), self.obj.__class__.__name__, id(self.obj), (self.listonly and 'listonly' or (self.isdelete and 'delete' or 'save')) )
class UOWDependencyProcessor(object):
"""in between the saving and deleting of objects, process "dependent" data, such as filling in
a foreign key on a child item from a new primary key, or deleting association rows before a
- delete."""
- def __init__(self, processor, targettask, isdeletefrom):
+ delete. This object acts as a proxy to a DependencyProcessor."""
+ def __init__(self, processor, targettask):
self.processor = processor
self.targettask = targettask
- self.isdeletefrom = isdeletefrom
-
+
+ def preexecute(self, trans):
+ """traverses all objects handled by this dependency processor and locates additional objects which should be
+ part of the transaction, such as those affected deletes, orphans to be deleted, etc. Returns True if any
+ objects were preprocessed, or False if no objects were preprocessed."""
+ def getobj(elem):
+ elem.mark_preprocessed(self)
+ return elem.obj
+
+ ret = False
+ elements = [getobj(elem) for elem in self.targettask.tosave_elements if elem.obj is not None and not elem.is_preprocessed(self)]
+ if len(elements):
+ ret = True
+ self.processor.preprocess_dependencies(self.targettask, elements, trans, delete=False)
+
+ elements = [getobj(elem) for elem in self.targettask.todelete_elements if elem.obj is not None and not elem.is_preprocessed(self)]
+ if len(elements):
+ ret = True
+ self.processor.preprocess_dependencies(self.targettask, elements, trans, delete=True)
+ return ret
+
def execute(self, trans, delete):
if not delete:
- self.processor.process_dependencies(self.targettask, [elem.obj for elem in self.targettask.tosave_elements() if elem.obj is not None], trans, delete = delete)
+ self.processor.process_dependencies(self.targettask, [elem.obj for elem in self.targettask.tosave_elements if elem.obj is not None], trans, delete=False)
else:
- self.processor.process_dependencies(self.targettask, [elem.obj for elem in self.targettask.todelete_elements() if elem.obj is not None], trans, delete = delete)
+ self.processor.process_dependencies(self.targettask, [elem.obj for elem in self.targettask.todelete_elements if elem.obj is not None], trans, delete=True)
def get_object_dependencies(self, obj, trans, passive):
return self.processor.get_object_dependencies(obj, trans, passive=passive)
return self.processor.whose_dependent_on_who(obj, o)
def branch(self, task):
- return UOWDependencyProcessor(self.processor, task, self.isdeletefrom)
+ return UOWDependencyProcessor(self.processor, task)
class UOWTask(object):
def __init__(self, uowtransaction, mapper):
self.dependencies = []
self.cyclical_dependencies = []
self.circular = None
- self.postcircular = None
self.childtasks = []
-# print "NEW TASK", repr(self)
def is_empty(self):
return len(self.objects) == 0 and len(self.dependencies) == 0 and len(self.childtasks) == 0
-
+
def append(self, obj, listonly = False, childtask = None, isdelete = False):
"""appends an object to this task, to be either saved or deleted depending on the
'isdelete' attribute of this UOWTask. 'listonly' indicates that the object should
rec.childtasks.append(childtask)
if isdelete:
rec.isdelete = True
+ #if not childtask:
+ # rec.preprocessed = False
return retval
def append_postupdate(self, obj):
self.circular.execute(trans)
return
- self.mapper.save_obj(self.tosave_objects(), trans)
- for dep in self.cyclical_save_dependencies():
- dep.execute(trans, delete=False)
- for element in self.tosave_elements():
+ self.mapper.save_obj(self.tosave_objects, trans)
+ for dep in self.cyclical_dependencies:
+ dep.execute(trans, False)
+ for element in self.tosave_elements:
for task in element.childtasks:
task.execute(trans)
- for dep in self.save_dependencies():
- dep.execute(trans, delete=False)
- for dep in self.delete_dependencies():
- dep.execute(trans, delete=True)
- for dep in self.cyclical_delete_dependencies():
- dep.execute(trans, delete=True)
+ for dep in self.dependencies:
+ dep.execute(trans, False)
+ for dep in self.dependencies:
+ dep.execute(trans, True)
+ for dep in self.cyclical_dependencies:
+ dep.execute(trans, True)
for child in self.childtasks:
child.execute(trans)
- for element in self.todelete_elements():
+ for element in self.todelete_elements:
for task in element.childtasks:
task.execute(trans)
- self.mapper.delete_obj(self.todelete_objects(), trans)
-
- def tosave_elements(self):
- return [rec for rec in self.objects.values() if not rec.isdelete]
- def todelete_elements(self):
- return [rec for rec in self.objects.values() if rec.isdelete]
- def tosave_objects(self):
- return [rec.obj for rec in self.objects.values() if rec.obj is not None and not rec.listonly and rec.isdelete is False]
- def todelete_objects(self):
- return [rec.obj for rec in self.objects.values() if rec.obj is not None and not rec.listonly and rec.isdelete is True]
- def save_dependencies(self):
- return [dep for dep in self.dependencies if not dep.isdeletefrom]
- def cyclical_save_dependencies(self):
- return [dep for dep in self.cyclical_dependencies if not dep.isdeletefrom]
- def delete_dependencies(self):
- return [dep for dep in self.dependencies if dep.isdeletefrom]
- def cyclical_delete_dependencies(self):
- return [dep for dep in self.cyclical_dependencies if dep.isdeletefrom]
+ self.mapper.delete_obj(self.todelete_objects, trans)
+
+ tosave_elements = property(lambda self: [rec for rec in self.objects.values() if not rec.isdelete])
+ todelete_elements = property(lambda self:[rec for rec in self.objects.values() if rec.isdelete])
+ tosave_objects = property(lambda self:[rec.obj for rec in self.objects.values() if rec.obj is not None and not rec.listonly and rec.isdelete is False])
+ todelete_objects = property(lambda self:[rec.obj for rec in self.objects.values() if rec.obj is not None and not rec.listonly and rec.isdelete is True])
def _sort_circular_dependencies(self, trans, cycles):
"""for a single task, creates a hierarchical tree of "subtasks" which associate
of its object list contain dependencies on each other.
this is not the normal case; this logic only kicks in when something like
- a hierarchical tree is being represented."""
+ a hierarchical tree is being represented.
+
+ """
allobjects = []
for task in cycles:
allobjects += task.objects.keys()
tuples = []
- objecttotask = {}
-
cycles = Set(cycles)
+ #print "BEGIN CIRC SORT-------"
+ #print "PRE-CIRC:"
+ #print list(cycles)[0].dump()
+
# dependency processors that arent part of the cyclical thing
# get put here
extradeplist = []
- def get_object_task(parent, obj):
- try:
- return objecttotask[obj]
- except KeyError:
- t = UOWTask(None, parent.mapper)
- t.parent = parent
- objecttotask[obj] = t
- return t
-
+ object_to_original_task = {}
+
+ # organizes a set of new UOWTasks that will be assembled into
+ # the final tree, for the purposes of holding new UOWDependencyProcessors
+ # which process small sub-sections of dependent parent/child operations
dependencies = {}
def get_dependency_task(obj, depprocessor):
try:
dp[depprocessor] = l
return l
- # work out a list of all the "dependency processors" that
- # represent objects that have to be dependency sorted at the
- # per-object level. all other dependency processors go in
- # "extradep."
+ # organize all original UOWDependencyProcessors by their target task
deps_by_targettask = {}
for task in cycles:
for dep in task.dependencies:
for task in cycles:
for taskelement in task.objects.values():
obj = taskelement.obj
+ object_to_original_task[obj] = task
#print "OBJ", repr(obj), "TASK", repr(task)
- # create a placeholder UOWTask that may be built into the final
- # task tree
- get_object_task(task, obj)
for dep in deps_by_targettask.get(task, []):
- (processor, targettask, isdelete) = (dep.processor, dep.targettask, dep.isdeletefrom)
- if taskelement.isdelete is not dep.isdeletefrom:
+ # is this dependency involved in one of the cycles ?
+ cyclicaldep = dep.targettask in cycles and trans.get_task_by_mapper(dep.processor.mapper) in cycles
+ if not cyclicaldep:
continue
- #print "GETING LIST OFF PROC", processor.key, "OBJ", repr(obj)
-
- # traverse through the modified child items of each object. normally this
- # is done via PropertyLoader in properties.py, but we need all the info
- # up front here to do the object-level topological sort.
+
+ (processor, targettask) = (dep.processor, dep.targettask)
+ isdelete = taskelement.isdelete
# list of dependent objects from this object
childlist = dep.get_object_dependencies(obj, trans, passive = True)
# the task corresponding to the processor's objects
childtask = trans.get_task_by_mapper(processor.mapper)
- # is this dependency involved in one of the cycles ?
- cyclicaldep = dep.targettask in cycles and trans.get_task_by_mapper(dep.processor.mapper) in cycles
- if isdelete:
- childlist = childlist.unchanged_items() + childlist.deleted_items()
- else:
- childlist = childlist.added_items()
+
+# if isdelete:
+# childlist = childlist.unchanged_items() + childlist.deleted_items()
+# else:
+# childlist = childlist.added_items()
+
+ childlist = childlist.added_items() + childlist.unchanged_items() + childlist.deleted_items()
for o in childlist:
- if o is None:
- # this can be None due to the many-to-one dependency processor added
- # for deleted items, line 385 properties.py
+ if o is None or o not in childtask.objects:
continue
- if not o in childtask.objects:
- # item needs to be saved since its added, or attached to a deleted object
- childtask.append(o, isdelete=isdelete and dep.processor.private)
- if cyclicaldep:
- # cyclical, so create a placeholder UOWTask that may be built into the
- # final task tree
- t = get_object_task(childtask, o)
- if not cyclicaldep:
- # not cyclical, so we are done with this
- continue
- # cyclical, so create an ordered pair for the dependency sort
whosdep = dep.whose_dependent_on_who(obj, o)
if whosdep is not None:
tuples.append(whosdep)
- # then locate a UOWDependencyProcessor to add the object onto, which
- # will handle the modifications between saves/deletes
+ # create a UOWDependencyProcessor representing this pair of objects.
+ # append it to a UOWTask
if whosdep[0] is obj:
get_dependency_task(whosdep[0], dep).append(whosdep[0], isdelete=isdelete)
else:
#print str(head)
+ hierarchical_tasks = {}
+ def get_object_task(obj):
+ try:
+ return hierarchical_tasks[obj]
+ except KeyError:
+ originating_task = object_to_original_task[obj]
+ return hierarchical_tasks.setdefault(obj, UOWTask(None, originating_task.mapper))
+
def make_task_tree(node, parenttask):
"""takes a dependency-sorted tree of objects and creates a tree of UOWTasks"""
- t = objecttotask[node.item]
+ #print "MAKETASKTREE", node.item
+
+ t = get_object_task(node.item)
+ for n in node.children:
+ t2 = make_task_tree(n, t)
+
can_add_to_parent = t.mapper is parenttask.mapper
- if can_add_to_parent:
- parenttask.append(node.item, t.parent.objects[node.item].listonly, isdelete=t.parent.objects[node.item].isdelete, childtask=t)
- else:
- t.append(node.item, t.parent.objects[node.item].listonly, isdelete=t.parent.objects[node.item].isdelete)
- parenttask.append(None, listonly=False, isdelete=t.parent.objects[node.item].isdelete, childtask=t)
+ original_task = object_to_original_task[node.item]
+ if original_task.objects.has_key(node.item):
+ if can_add_to_parent:
+ parenttask.append(node.item, original_task.objects[node.item].listonly, isdelete=original_task.objects[node.item].isdelete, childtask=t)
+ else:
+ t.append(node.item, original_task.objects[node.item].listonly, isdelete=original_task.objects[node.item].isdelete)
+ parenttask.append(None, listonly=False, isdelete=original_task.objects[node.item].isdelete, childtask=t)
+ #else:
+ # parenttask.append(None, listonly=False, isdelete=original_task.objects[node.item].isdelete, childtask=t)
if dependencies.has_key(node.item):
for depprocessor, deptask in dependencies[node.item].iteritems():
if can_add_to_parent:
parenttask.cyclical_dependencies.append(depprocessor.branch(deptask))
else:
t.cyclical_dependencies.append(depprocessor.branch(deptask))
- for n in node.children:
- t2 = make_task_tree(n, t)
return t
# this is the new "circular" UOWTask which will execute in place of "self"
t.dependencies += [d for d in extradeplist]
t.childtasks = self.childtasks
make_task_tree(head, t)
+ #print t.dump()
return t
def dump(self):
buf.write(text)
headers[text] = True
- def _dump_processor(proc):
- if proc.isdeletefrom:
+ def _dump_processor(proc, deletes):
+ if deletes:
val = [t for t in proc.targettask.objects.values() if t.isdelete]
else:
val = [t for t in proc.targettask.objects.values() if not t.isdelete]
- buf.write(_indent() + " |- UOWDependencyProcessor(%d) %s attribute on %s (%s)\n" % (
- id(proc),
+ buf.write(_indent() + " |- %s attribute on %s (UOWDependencyProcessor(%d) processing %s)\n" % (
repr(proc.processor.key),
- (proc.isdeletefrom and
- "%s's to be deleted" % _repr_task_class(proc.targettask)
- or "saved %s's" % _repr_task_class(proc.targettask)),
+ ("%s's to be %s" % (_repr_task_class(proc.targettask), deletes and "deleted" or "saved")),
+ id(proc),
_repr_task(proc.targettask))
)
if len(val) == 0:
buf.write(_indent() + " | |-" + "(no objects)\n")
for v in val:
- buf.write(_indent() + " | |-" + _repr_task_element(v) + "\n")
+ buf.write(_indent() + " | |-" + _repr_task_element(v, proc.processor.key) + "\n")
- def _repr_task_element(te):
+ def _repr_task_element(te, attribute=None):
if te.obj is None:
objid = "(placeholder)"
else:
- objid = "%s(%d)" % (te.obj.__class__.__name__, id(te.obj))
- return "UOWTaskElement(%d): %s %s%s" % (id(te), objid, (te.listonly and '(listonly)' or (te.isdelete and '(delete' or '(save')),
- (te.mapper is not None and " w/ " + str(te.mapper) + ")" or ")")
- )
+ if attribute is not None:
+ objid = "%s(%d).%s" % (te.obj.__class__.__name__, id(te.obj), attribute)
+ else:
+ objid = "%s(%d)" % (te.obj.__class__.__name__, id(te.obj))
+ return "%s (UOWTaskElement(%d, %s))" % (objid, id(te), (te.listonly and 'listonly' or (te.isdelete and 'delete' or 'save')))
def _repr_task(task):
if task.mapper is not None:
if task.mapper.__class__.__name__ == 'Mapper':
- name = task.mapper.class_.__name__ + "/" + str(task.mapper.primarytable) + "/" + str(id(task.mapper))
+ name = task.mapper.class_.__name__ + "/" + task.mapper.local_table.name + "/" + str(task.mapper.entity_name)
else:
name = repr(task.mapper)
else:
name = '(none)'
- return ("UOWTask(%d) '%s'" % (id(task), name))
+ return ("UOWTask(%d, %s)" % (id(task), name))
def _repr_task_class(task):
if task.mapper is not None and task.mapper.__class__.__name__ == 'Mapper':
return task.mapper.class_.__name__
buf.write(i + " " + _repr_task(self))
buf.write("\n")
- for rec in self.tosave_elements():
+ for rec in self.tosave_elements:
if rec.listonly:
continue
header(buf, _indent() + " |- Save elements\n")
- buf.write(_indent() + " |- Save: " + _repr_task_element(rec) + "\n")
- for dep in self.cyclical_save_dependencies():
+ buf.write(_indent() + " |- " + _repr_task_element(rec) + "\n")
+ for dep in self.cyclical_dependencies:
header(buf, _indent() + " |- Cyclical Save dependencies\n")
- _dump_processor(dep)
- for element in self.tosave_elements():
+ _dump_processor(dep, False)
+ for element in self.tosave_elements:
for task in element.childtasks:
header(buf, _indent() + " |- Save subelements of UOWTaskElement(%s)\n" % id(element))
task._dump(buf, indent + 1)
- for dep in self.save_dependencies():
+ for dep in self.dependencies:
header(buf, _indent() + " |- Save dependencies\n")
- _dump_processor(dep)
- for dep in self.delete_dependencies():
+ _dump_processor(dep, False)
+ for dep in self.dependencies:
header(buf, _indent() + " |- Delete dependencies\n")
- _dump_processor(dep)
- for dep in self.cyclical_delete_dependencies():
+ _dump_processor(dep, True)
+ for dep in self.cyclical_dependencies:
header(buf, _indent() + " |- Cyclical Delete dependencies\n")
- _dump_processor(dep)
+ _dump_processor(dep, True)
for child in self.childtasks:
header(buf, _indent() + " |- Child tasks\n")
child._dump(buf, indent + 1)
# for obj in self.postupdate:
# header(buf, _indent() + " |- Post Update objects\n")
# buf.write(_repr(obj) + "\n")
- for element in self.todelete_elements():
+ for element in self.todelete_elements:
for task in element.childtasks:
header(buf, _indent() + " |- Delete subelements of UOWTaskElement(%s)\n" % id(element))
task._dump(buf, indent + 1)
- for rec in self.todelete_elements():
+ for rec in self.todelete_elements:
if rec.listonly:
continue
header(buf, _indent() + " |- Delete elements\n")
- buf.write(_indent() + " |- Delete: " + _repr_task_element(rec) + "\n")
+ buf.write(_indent() + " |- " + _repr_task_element(rec) + "\n")
- buf.write(_indent() + " |----\n")
+ if self.is_empty():
+ buf.write(_indent() + " |- (empty task)\n")
+ else:
+ buf.write(_indent() + " |----\n")
+
buf.write(_indent() + "\n")
def __repr__(self):
if self.mapper is not None:
if self.mapper.__class__.__name__ == 'Mapper':
- name = self.mapper.class_.__name__ + "/" + self.mapper.primarytable.name
+ name = self.mapper.class_.__name__ + "/" + self.mapper.local_table.name
else:
name = repr(self.mapper)
else:
--- /dev/null
+# mapper/util.py
+# Copyright (C) 2005,2006 Michael Bayer mike_mp@zzzcomputing.com
+#
+# This module is part of SQLAlchemy and is released under
+# the MIT License: http://www.opensource.org/licenses/mit-license.php
+
+import sets
+import sqlalchemy.sql as sql
+
+class CascadeOptions(object):
+ """keeps track of the options sent to relation().cascade"""
+ def __init__(self, arg=""):
+ values = sets.Set([c.strip() for c in arg.split(',')])
+ self.delete_orphan = "delete-orphan" in values
+ self.delete = "delete" in values or self.delete_orphan or "all" in values
+ self.save_update = "save-update" in values or "all" in values
+ self.merge = "merge" in values or "all" in values
+ self.expunge = "expunge" in values or "all" in values
+ self.refresh_expire = "refresh-expire" in values or "all" in values
+ def __contains__(self, item):
+ return getattr(self, item.replace("-", "_"), False)
+
+
+def polymorphic_union(table_map, typecolname, aliasname='p_union'):
+ colnames = sets.Set()
+ colnamemaps = {}
+
+ for key in table_map.keys():
+ table = table_map[key]
+
+ # mysql doesnt like selecting from a select; make it an alias of the select
+ if isinstance(table, sql.Select):
+ table = table.alias()
+ table_map[key] = table
+
+ m = {}
+ for c in table.c:
+ colnames.add(c.name)
+ m[c.name] = c
+ colnamemaps[table] = m
+
+ def col(name, table):
+ try:
+ return colnamemaps[table][name]
+ except KeyError:
+ return sql.null().label(name)
+
+ result = []
+ for type, table in table_map.iteritems():
+ if typecolname is not None:
+ result.append(sql.select([col(name, table) for name in colnames] + [sql.column("'%s'" % type).label(typecolname)], from_obj=[table]))
+ else:
+ result.append(sql.select([col(name, table) for name in colnames], from_obj=[table]))
+ return sql.union_all(*result).alias(aliasname)
+
\ No newline at end of file
class Pool(object):
- def __init__(self, echo = False, use_threadlocal = True, logger=None):
- self._threadconns = weakref.WeakValueDictionary()
+ def __init__(self, echo = False, use_threadlocal = True, logger=None, **kwargs):
+ self._threadconns = {} #weakref.WeakValueDictionary()
self._use_threadlocal = use_threadlocal
- self._echo = echo
+ self.echo = echo
self._logger = logger or util.Logger(origin='pool')
def unique_connection(self):
- return ConnectionFairy(self)
+ return ConnectionFairy(self).checkout()
def connect(self):
if not self._use_threadlocal:
- return ConnectionFairy(self)
+ return ConnectionFairy(self).checkout()
try:
- return self._threadconns[thread.get_ident()]
+ return self._threadconns[thread.get_ident()].checkout()
except KeyError:
- agent = ConnectionFairy(self)
+ agent = ConnectionFairy(self).checkout()
self._threadconns[thread.get_ident()] = agent
return agent
- def return_conn(self, conn):
- if self._echo:
- self.log("return connection to pool")
- self.do_return_conn(conn)
+ def return_conn(self, agent):
+ if self._use_threadlocal:
+ try:
+ del self._threadconns[thread.get_ident()]
+ except KeyError:
+ pass
+ self.do_return_conn(agent.connection)
def get(self):
- if self._echo:
- self.log("get connection from pool")
- self.log(self.status())
return self.do_get()
def return_invalid(self):
- if self._echo:
- self.log("return invalid connection to pool")
- self.log(self.status())
self.do_return_invalid()
def do_get(self):
class ConnectionFairy(object):
def __init__(self, pool, connection=None):
self.pool = pool
+ self.__counter = 0
if connection is not None:
self.connection = connection
else:
self.connection = None
self.pool.return_invalid()
raise
+ if self.pool.echo:
+ self.pool.log("Connection %s checked out from pool" % repr(self.connection))
def invalidate(self):
+ if self.pool.echo:
+ self.pool.log("Invalidate connection %s" % repr(self.connection))
+ self.connection.rollback()
self.connection = None
self.pool.return_invalid()
def cursor(self):
return CursorFairy(self, self.connection.cursor())
def __getattr__(self, key):
return getattr(self.connection, key)
+ def checkout(self):
+ if self.connection is None:
+ raise "this connection is closed"
+ self.__counter +=1
+ return self
+ def close(self):
+ self.__counter -=1
+ if self.__counter == 0:
+ self._close()
def __del__(self):
+ self._close()
+ def _close(self):
if self.connection is not None:
- self.pool.return_conn(self.connection)
+ if self.pool.echo:
+ self.pool.log("Connection %s being returned to pool" % repr(self.connection))
+ self.connection.rollback()
+ self.pool.return_conn(self)
self.pool = None
self.connection = None
class SingletonThreadPool(Pool):
"""Maintains one connection per each thread, never moving to another thread. this is
- used for SQLite and other databases with a similar restriction."""
+ used for SQLite."""
def __init__(self, creator, **params):
Pool.__init__(self, **params)
self._conns = {}
self._creator = creator
def status(self):
- return "SingletonThreadPool thread:%d size: %d" % (thread.get_ident(), len(self._conns))
+ return "SingletonThreadPool id:%d thread:%d size: %d" % (id(self), thread.get_ident(), len(self._conns))
def do_return_conn(self, conn):
- if self._conns.get(thread.get_ident(), None) is None:
- self._conns[thread.get_ident()] = conn
-
+ pass
+
def do_return_invalid(self):
try:
del self._conns[thread.get_ident()]
def do_get(self):
try:
- c = self._conns[thread.get_ident()]
- if c is None:
- return self._creator()
+ return self._conns[thread.get_ident()]
except KeyError:
c = self._creator()
- self._conns[thread.get_ident()] = None
- return c
+ self._conns[thread.get_ident()] = c
+ return c
class QueuePool(Pool):
"""uses Queue.Queue to maintain a fixed-size list of connections."""
- def __init__(self, creator, pool_size = 5, max_overflow = 10, **params):
+ def __init__(self, creator, pool_size = 5, max_overflow = 10, timeout=30, **params):
Pool.__init__(self, **params)
self._creator = creator
self._pool = Queue.Queue(pool_size)
self._overflow = 0 - pool_size
self._max_overflow = max_overflow
+ self._timeout = timeout
def do_return_conn(self, conn):
- if self._echo:
- self.log("return QP connection to pool")
try:
self._pool.put(conn, False)
except Queue.Full:
self._overflow -= 1
def do_return_invalid(self):
- if self._echo:
- self.log("return invalid connection")
if self._pool.full():
self._overflow -= 1
def do_get(self):
- if self._echo:
- self.log("get QP connection from pool")
- self.log(self.status())
try:
- return self._pool.get(self._max_overflow > -1 and self._overflow >= self._max_overflow)
+ return self._pool.get(self._max_overflow > -1 and self._overflow >= self._max_overflow, self._timeout)
except Queue.Empty:
self._overflow += 1
return self._creator()
- def __del__(self):
+ def dispose(self):
while True:
try:
conn = self._pool.get(False)
conn.close()
except Queue.Empty:
break
+ def __del__(self):
+ self.dispose()
def status(self):
tup = (self.size(), self.checkedin(), self.overflow(), self.checkedout())
the schema package "plugs in" to the SQL package.
"""
-import sql
-from util import *
-from types import *
-from exceptions import *
+from sqlalchemy import sql, types, exceptions,util
+import sqlalchemy
import copy, re, string
__all__ = ['SchemaItem', 'Table', 'Column', 'ForeignKey', 'Sequence', 'Index',
- 'SchemaEngine', 'SchemaVisitor', 'PassiveDefault', 'ColumnDefault']
-
-class SchemaMeta(type):
- """provides universal constructor arguments for all SchemaItems"""
- def __call__(self, *args, **kwargs):
- engine = kwargs.pop('engine', None)
- obj = type.__call__(self, *args, **kwargs)
- obj._engine = engine
- return obj
-
+ 'MetaData', 'BoundMetaData', 'DynamicMetaData', 'SchemaVisitor', 'PassiveDefault', 'ColumnDefault']
+
class SchemaItem(object):
"""base class for items that define a database schema."""
- __metaclass__ = SchemaMeta
def _init_items(self, *args):
for item in args:
if item is not None:
raise NotImplementedError()
def __repr__(self):
return "%s()" % self.__class__.__name__
-
-class EngineMixin(object):
- """a mixin for SchemaItems that provides an "engine" accessor."""
- def _derived_engine(self):
- """subclasses override this method to return an AbstractEngine
- bound to a parent item"""
+ def _derived_metadata(self):
+ """subclasses override this method to return a the MetaData
+ to which this item is bound"""
return None
def _get_engine(self):
- if self._engine is not None:
- return self._engine
- else:
- return self._derived_engine()
- engine = property(_get_engine)
+ return self._derived_metadata().engine
+ engine = property(lambda s:s._get_engine())
+ metadata = property(lambda s:s._derived_metadata())
-def _get_table_key(engine, name, schema):
- if schema is not None:# and schema == engine.get_default_schema_name():
- schema = None
+def _get_table_key(name, schema):
if schema is None:
return name
else:
return schema + "." + name
-class TableSingleton(SchemaMeta):
+class TableSingleton(type):
"""a metaclass used by the Table object to provide singleton behavior."""
- def __call__(self, name, engine=None, *args, **kwargs):
+ def __call__(self, name, metadata, *args, **kwargs):
try:
- if engine is not None and not isinstance(engine, SchemaEngine):
- args = [engine] + list(args)
- engine = default_engine
+ if isinstance(metadata, sql.Engine):
+ # backwards compatibility - get a BoundSchema associated with the engine
+ engine = metadata
+ if not hasattr(engine, '_legacy_metadata'):
+ engine._legacy_metadata = BoundMetaData(engine)
+ metadata = engine._legacy_metadata
name = str(name) # in case of incoming unicode
schema = kwargs.get('schema', None)
autoload = kwargs.pop('autoload', False)
+ autoload_with = kwargs.pop('autoload_with', False)
redefine = kwargs.pop('redefine', False)
mustexist = kwargs.pop('mustexist', False)
useexisting = kwargs.pop('useexisting', False)
- if not engine:
- table = type.__call__(self, name, engine, **kwargs)
- table._init_items(*args)
- return table
- key = _get_table_key(engine, name, schema)
- table = engine.tables[key]
+ key = _get_table_key(name, schema)
+ table = metadata.tables[key]
if len(args):
if redefine:
table.reload_values(*args)
elif not useexisting:
- raise ArgumentError("Table '%s.%s' is already defined. specify 'redefine=True' to remap columns, or 'useexisting=True' to use the existing table" % (schema, name))
+ raise exceptions.ArgumentError("Table '%s.%s' is already defined. specify 'redefine=True' to remap columns, or 'useexisting=True' to use the existing table" % (schema, name))
return table
except KeyError:
if mustexist:
- raise ArgumentError("Table '%s.%s' not defined" % (schema, name))
- table = type.__call__(self, name, engine, **kwargs)
- engine.tables[key] = table
+ raise exceptions.ArgumentError("Table '%s.%s' not defined" % (schema, name))
+ table = type.__call__(self, name, metadata, **kwargs)
+ table._set_parent(metadata)
# load column definitions from the database if 'autoload' is defined
# we do it after the table is in the singleton dictionary to support
# circular foreign keys
if autoload:
- engine.reflecttable(table)
+ if autoload_with:
+ autoload_with.reflecttable(table)
+ else:
+ metadata.engine.reflecttable(table)
# initialize all the column, etc. objects. done after
# reflection to allow user-overrides
table._init_items(*args)
return table
-class Table(sql.TableClause, SchemaItem):
+class Table(SchemaItem, sql.TableClause):
"""represents a relational database table. This subclasses sql.TableClause to provide
a table that is "wired" to an engine. Whereas TableClause represents a table as its
used in a SQL expression, Table represents a table as its created in the database.
Be sure to look at sqlalchemy.sql.TableImpl for additional methods defined on a Table."""
__metaclass__ = TableSingleton
- def __init__(self, name, engine, **kwargs):
+ def __init__(self, name, metadata, **kwargs):
"""Table objects can be constructed directly. The init method is actually called via
the TableSingleton metaclass. Arguments are:
Further tables constructed with the same name/schema combination will return the same
Table instance.
- engine : a SchemaEngine instance to provide services to this table. Usually a subclass of
- sql.SQLEngine.
-
*args : should contain a listing of the Column objects for this table.
**kwargs : options include:
"""
super(Table, self).__init__(name)
- self._engine = engine
+ self._metadata = metadata
self.schema = kwargs.pop('schema', None)
if self.schema is not None:
self.fullname = "%s.%s" % (self.schema, self.name)
else:
self.fullname = self.name
self.kwargs = kwargs
-
+ def _derived_metadata(self):
+ return self._metadata
def __repr__(self):
return "Table(%s)" % string.join(
- [repr(self.name)] + [repr(self.engine)] +
+ [repr(self.name)] + [repr(self.metadata)] +
[repr(x) for x in self.columns] +
["%s=%s" % (k, repr(getattr(self, k))) for k in ['schema']]
, ',\n')
def append_index(self, index):
self.indexes[index.name] = index
- def _set_parent(self, schema):
- schema.tables[self.name] = self
- self.schema = schema
+ def _set_parent(self, metadata):
+ metadata.tables[self.name] = self
+ self._metadata = metadata
def accept_schema_visitor(self, visitor):
"""traverses the given visitor across the Column objects inside this Table,
then calls the visit_table method on the visitor."""
return index
def deregister(self):
- """removes this table from it's engines table registry. this does not
+ """removes this table from it's metadata. this does not
issue a SQL DROP statement."""
- key = _get_table_key(self.engine, self.name, self.schema)
- del self.engine.tables[key]
- def create(self, **params):
- self.engine.create(self)
+ key = _get_table_key(self.name, self.schema)
+ del self.metadata.tables[key]
+ def create(self, connectable=None):
+ if connectable is not None:
+ connectable.create(self)
+ else:
+ self.engine.create(self)
return self
- def drop(self, **params):
- self.engine.drop(self)
- def toengine(self, engine, schema=None):
- """returns a singleton instance of this Table with a different engine"""
+ def drop(self, connectable=None):
+ if connectable is not None:
+ connectable.drop(self)
+ else:
+ self.engine.drop(self)
+ def tometadata(self, metadata, schema=None):
+ """returns a singleton instance of this Table with a different Schema"""
try:
if schema is None:
schema = self.schema
- key = _get_table_key(engine, self.name, schema)
- return engine.tables[key]
- except:
+ key = _get_table_key(self.name, schema)
+ return metadata.tables[key]
+ except KeyError:
args = []
for c in self.columns:
args.append(c.copy())
- return Table(self.name, engine, schema=schema, *args)
+ return Table(self.name, metadata, schema=schema, *args)
-class Column(sql.ColumnClause, SchemaItem):
+class Column(SchemaItem, sql.ColumnClause):
"""represents a column in a database table. this is a subclass of sql.ColumnClause and
represents an actual existing table in the database, in a similar fashion as TableClause/Table."""
def __init__(self, name, type, *args, **kwargs):
order of their creation.
"""
-
name = str(name) # in case of incoming unicode
super(Column, self).__init__(name, None, type)
self.args = args
self.unique = kwargs.pop('unique', None)
self.onupdate = kwargs.pop('onupdate', None)
if self.index is not None and self.unique is not None:
- raise ArgumentError("Column may not define both index and unique")
+ raise exceptions.ArgumentError("Column may not define both index and unique")
self._foreign_key = None
- self._orig = None
- self._parent = None
if len(kwargs):
- raise ArgumentError("Unknown arguments passed to Column: " + repr(kwargs.keys()))
+ raise exceptions.ArgumentError("Unknown arguments passed to Column: " + repr(kwargs.keys()))
- primary_key = SimpleProperty('_primary_key')
- foreign_key = SimpleProperty('_foreign_key')
- original = property(lambda s: s._orig or s)
- parent = property(lambda s:s._parent or s)
- engine = property(lambda s: s.table.engine)
+ primary_key = util.SimpleProperty('_primary_key')
+ foreign_key = util.SimpleProperty('_foreign_key')
columns = property(lambda self:[self])
+ def __str__(self):
+ if self.table is not None:
+ tname = self.table.displayname
+ if tname is not None:
+ return tname + "." + self.name
+ else:
+ return self.name
+ else:
+ return self.name
+
+ def _derived_metadata(self):
+ return self.table.metadata
+ def _get_engine(self):
+ return self.table.engine
+
def __repr__(self):
return "Column(%s)" % string.join(
[repr(self.name)] + [repr(self.type)] +
def _set_parent(self, table):
if getattr(self, 'table', None) is not None:
- raise ArgumentError("this Column already has a table!")
+ raise exceptions.ArgumentError("this Column already has a table!")
table.append_column(self)
if self.index or self.unique:
table.append_index_column(self, index=self.index,
fk = self.foreign_key.copy()
c = Column(name or self.name, self.type, fk, self.default, key = name or self.key, primary_key = self.primary_key, nullable = self.nullable, hidden = self.hidden)
c.table = selectable
- c._orig = self.original
+ c.orig_set = self.orig_set
c._parent = self
if not c.hidden:
selectable.columns[c.key] = c
"""Constructs a new ForeignKey object. "column" can be a schema.Column
object representing the relationship, or just its string name given as
"tablename.columnname". schema can be specified as
- "schemaname.tablename.columnname" """
+ "schema.tablename.columnname" """
if isinstance(column, unicode):
column = str(column)
self._colspec = column
def references(self, table):
"""returns True if the given table is referenced by this ForeignKey."""
- return table._get_col_by_original(self.column, False) is not None
+ return table.corresponding_column(self.column, False) is not None
def _init_column(self):
# ForeignKey inits its remote column as late as possible, so tables can
if isinstance(self._colspec, str):
m = re.match(r"^([\w_-]+)(?:\.([\w_-]+))?(?:\.([\w_-]+))?$", self._colspec)
if m is None:
- raise ArgumentError("Invalid foreign key column specification: " + self._colspec)
+ raise exceptions.ArgumentError("Invalid foreign key column specification: " + self._colspec)
if m.group(3) is None:
(tname, colname) = m.group(1, 2)
- schema = self.parent.original.table.schema
+ schema = list(self.parent.orig_set)[0].table.schema
else:
(schema,tname,colname) = m.group(1,2,3)
- table = Table(tname, self.parent.engine, mustexist=True, schema=schema)
+ table = Table(tname, list(self.parent.orig_set)[0].metadata, mustexist=True, schema=schema)
if colname is None:
key = self.parent
self._column = table.c[self.parent.key]
self.parent.foreign_key = self
self.parent.table.foreign_keys.append(self)
-class DefaultGenerator(SchemaItem, EngineMixin):
+class DefaultGenerator(SchemaItem):
"""Base class for column "default" values."""
- def __init__(self, for_update=False):
+ def __init__(self, for_update=False, metadata=None):
self.for_update = for_update
- def _derived_engine(self):
- return self.column.table.engine
+ self._metadata = metadata
+ def _derived_metadata(self):
+ try:
+ return self.column.table.metadata
+ except AttributeError:
+ return self._metadata
def _set_parent(self, column):
self.column = column
+ self._metadata = self.column.table.metadata
if self.for_update:
self.column.onupdate = self
else:
self.column.default = self
- def execute(self):
- return self.accept_schema_visitor(self.engine.defaultrunner(self.engine.proxy))
+ def execute(self, **kwargs):
+ return self.engine.execute_default(self, **kwargs)
def __repr__(self):
return "DefaultGenerator()"
return visitor.visit_sequence(self)
-class Index(SchemaItem, EngineMixin):
+class Index(SchemaItem):
"""Represents an index of columns from a database table
"""
def __init__(self, name, *columns, **kw):
self.unique = kw.pop('unique', False)
self._init_items(*columns)
- def _derived_engine(self):
- return self.table.engine
+ def _derived_metadata(self):
+ return self.table.metadata
def _init_items(self, *args):
for column in args:
self.append_column(column)
self.table.append_index(self)
elif column.table != self.table:
# all columns muse be from same table
- raise ArgumentError("All index columns must be from same table. "
+ raise exceptions.ArgumentError("All index columns must be from same table. "
"%s is from %s not %s" % (column,
column.table,
self.table))
elif column.name in [ c.name for c in self.columns ]:
- raise ArgumentError("A column may not appear twice in the "
+ raise exceptions.ArgumentError("A column may not appear twice in the "
"same index (%s already has column %s)"
% (self.name, column))
self.columns.append(column)
- def create(self):
- self.engine.create(self)
- return self
- def drop(self):
- self.engine.drop(self)
- def execute(self):
- self.create()
+ def create(self, engine=None):
+ if engine is not None:
+ engine.create(self)
+ else:
+ self.engine.create(self)
+ return self
+ def drop(self, engine=None):
+ if engine is not None:
+ engine.drop(self)
+ else:
+ self.engine.drop(self)
def accept_schema_visitor(self, visitor):
visitor.visit_index(self)
def __str__(self):
for c in self.columns]),
(self.unique and ', unique=True') or '')
-class SchemaEngine(sql.AbstractEngine):
- """a factory object used to create implementations for schema objects. This object
- is the ultimate base class for the engine.SQLEngine class."""
-
- def __init__(self):
+class MetaData(SchemaItem):
+ """represents a collection of Tables and their associated schema constructs."""
+ def __init__(self, name=None):
# a dictionary that stores Table objects keyed off their name (and possibly schema name)
self.tables = {}
- def reflecttable(self, table):
- """given a table, will query the database and populate its Column and ForeignKey
- objects."""
- raise NotImplementedError()
- def schemagenerator(self, **params):
- raise NotImplementedError()
- def schemadropper(self, **params):
- raise NotImplementedError()
-
+ self.name = name
+ def is_bound(self):
+ return False
+ def clear(self):
+ self.tables.clear()
+ def table_iterator(self, reverse=True):
+ return self._sort_tables(self.tables.values(), reverse=reverse)
+
+ def create_all(self, engine=None, tables=None):
+ if not tables:
+ tables = self.tables.values()
+
+ if engine is None and self.is_bound():
+ engine = self.engine
+
+ def do(conn):
+ e = conn.engine
+ ts = self._sort_tables( tables )
+ for table in ts:
+ if e.dialect.has_table(conn, table.name):
+ continue
+ conn.create(table)
+ engine.run_callable(do)
+
+ def drop_all(self, engine=None, tables=None):
+ if not tables:
+ tables = self.tables.values()
+
+ if engine is None and self.is_bound():
+ engine = self.engine
+
+ def do(conn):
+ e = conn.engine
+ ts = self._sort_tables( tables, reverse=True )
+ for table in ts:
+ if e.dialect.has_table(conn, table.name):
+ conn.drop(table)
+ engine.run_callable(do)
+
+ def _sort_tables(self, tables, reverse=False):
+ import sqlalchemy.sql_util
+ sorter = sqlalchemy.sql_util.TableCollection()
+ for t in self.tables.values():
+ sorter.add(t)
+ return sorter.sort(reverse=reverse)
+
+ def _derived_metadata(self):
+ return self
+ def _get_engine(self):
+ if not self.is_bound():
+ return None
+ return self._engine
+
+class BoundMetaData(MetaData):
+ """builds upon MetaData to provide the capability to bind to an Engine implementation."""
+ def __init__(self, engine_or_url, name=None, **kwargs):
+ super(BoundMetaData, self).__init__(name)
+ if isinstance(engine_or_url, str):
+ self._engine = sqlalchemy.create_engine(engine_or_url, **kwargs)
+ else:
+ self._engine = engine_or_url
+ def is_bound(self):
+ return True
+
+class DynamicMetaData(MetaData):
+ """builds upon MetaData to provide the capability to bind to multiple Engine implementations
+ on a dynamically alterable, thread-local basis."""
+ def __init__(self, name=None, threadlocal=True):
+ super(DynamicMetaData, self).__init__(name)
+ if threadlocal:
+ self.context = util.ThreadLocal()
+ else:
+ self.context = self
+ self.__engines = {}
+ def connect(self, engine_or_url, **kwargs):
+ if isinstance(engine_or_url, str):
+ try:
+ self.context._engine = self.__engines[engine_or_url]
+ except KeyError:
+ e = sqlalchemy.create_engine(engine_or_url, **kwargs)
+ self.__engines[engine_or_url] = e
+ self.context._engine = e
+ else:
+ if not self.__engines.has_key(engine_or_url):
+ self.__engines[engine_or_url] = engine_or_url
+ self.context._engine = engine_or_url
+ def is_bound(self):
+ return self.context._engine is not None
+ def dispose(self):
+ """disposes all Engines to which this DynamicMetaData has been connected."""
+ for e in self.__engines.values():
+ e.dispose()
+ engine=property(lambda s:s.context._engine)
+
class SchemaVisitor(sql.ClauseVisitor):
"""defines the visiting for SchemaItem objects"""
def visit_schema(self, schema):
"""visit a Sequence."""
pass
-
+default_metadata = DynamicMetaData('default')
+
"""defines the base components of SQL expression trees."""
-import schema
-import util
-import types as sqltypes
-from exceptions import *
-import string, re, random
+from sqlalchemy import util, exceptions
+from sqlalchemy import types as sqltypes
+import string, re, random, sets
types = __import__('types')
__all__ = ['text', 'table', 'column', 'func', 'select', 'update', 'insert', 'delete', 'join', 'and_', 'or_', 'not_', 'between_', 'case', 'cast', 'union', 'union_all', 'null', 'desc', 'asc', 'outerjoin', 'alias', 'subquery', 'literal', 'bindparam', 'exists']
text - the text of the SQL statement to be created. use :<param> to specify
bind parameters; they will be compiled to their engine-specific format.
- engine - an optional engine to be used for this text query. Alternatively, call the
- text() method off the engine directly.
+ engine - an optional engine to be used for this text query.
bindparams - a list of bindparam() instances which can be used to define the
types and/or initial values for the bind parameters within the textual statement;
def is_column(col):
return isinstance(col, ColumnElement)
-class AbstractEngine(object):
- """represents a 'thing that can produce Compiler objects an execute them'."""
+class Engine(object):
+ """represents a 'thing that can produce Compiled objects and execute them'."""
def execute_compiled(self, compiled, parameters, echo=None, **kwargs):
raise NotImplementedError()
def compiler(self, statement, parameters, **kwargs):
raise NotImplementedError()
+class AbstractDialect(object):
+ """represents the behavior of a particular database. Used by Compiled objects."""
+ pass
+
class ClauseParameters(util.OrderedDict):
"""represents a dictionary/iterator of bind parameter key names/values. Includes parameters compiled with a Compiled object as well as additional arguments passed to the Compiled object's get_params() method. Parameter values will be converted as per the TypeEngine objects present in the bind parameter objects. The non-converted value can be retrieved via the get_original method. For Compiled objects that compile positional parameters, the values() iteration of the object will return the parameter values in the correct order."""
- def __init__(self, engine=None):
+ def __init__(self, dialect):
super(ClauseParameters, self).__init__(self)
- self.engine = engine
+ self.dialect=dialect
self.binds = {}
def set_parameter(self, key, value, bindparam):
self[key] = value
self.binds[key] = bindparam
def get_original(self, key):
+ """returns the given parameter as it was originally placed in this ClauseParameters object, without any Type conversion"""
return super(ClauseParameters, self).__getitem__(key)
def __getitem__(self, key):
v = super(ClauseParameters, self).__getitem__(key)
- if self.engine is not None and self.binds.has_key(key):
- v = self.binds[key].typeprocess(v, self.engine)
+ if self.binds.has_key(key):
+ v = self.binds[key].typeprocess(v, self.dialect)
return v
def values(self):
return [self[key] for key in self]
object be dependent on the actual values of those bind parameters, even though it may
reference those values as defaults."""
- def __init__(self, statement, parameters, engine=None):
+ def __init__(self, dialect, statement, parameters, engine=None):
"""constructs a new Compiled object.
statement - ClauseElement to be compiled
clauses of an UPDATE statement. The keys of the parameter dictionary can
either be the string names of columns or ColumnClause objects.
- engine - optional SQLEngine to compile this statement against"""
- self.parameters = parameters
+ engine - optional Engine to compile this statement against"""
+ self.dialect = dialect
self.statement = statement
+ self.parameters = parameters
self.engine = engine
-
+
def __str__(self):
"""returns the string text of the generated SQL statement."""
raise NotImplementedError()
def execute(self, *multiparams, **params):
"""executes this compiled object using the AbstractEngine it is bound to."""
- if len(multiparams):
- params = multiparams
-
e = self.engine
if e is None:
- raise InvalidRequestError("This Compiled object is not bound to any engine.")
- return e.execute_compiled(self, params)
+ raise exceptions.InvalidRequestError("This Compiled object is not bound to any engine.")
+ return e.execute_compiled(self, *multiparams, **params)
def scalar(self, *multiparams, **params):
"""executes this compiled object via the execute() method, then
# in a result set is not performance-wise any different than specifying limit=1
# else we'd have to construct a copy of the select() object with the limit
# installed (else if we change the existing select, not threadsafe)
- row = self.execute(*multiparams, **params).fetchone()
- if row is not None:
- return row[0]
- else:
- return None
+ r = self.execute(*multiparams, **params)
+ row = r.fetchone()
+ try:
+ if row is not None:
+ return row[0]
+ else:
+ return None
+ finally:
+ r.close()
class Executor(object):
- """handles the compilation/execution of a ClauseElement within the context of a particular AbtractEngine. This
- AbstractEngine will usually be a SQLEngine or ConnectionProxy."""
+ """context-sensitive executor for the using() function."""
def __init__(self, clauseelement, abstractengine=None):
self.engine=abstractengine
self.clauseelement = clauseelement
def execute(self, *multiparams, **params):
- return self.compile(*multiparams, **params).execute(*multiparams, **params)
+ return self.clauseelement.execute_using(self.engine)
def scalar(self, *multiparams, **params):
- return self.compile(*multiparams, **params).scalar(*multiparams, **params)
- def compile(self, *multiparams, **params):
- if len(multiparams):
- bindparams = multiparams[0]
- else:
- bindparams = params
- compiler = self.engine.compiler(self.clauseelement, bindparams)
- compiler.compile()
- return compiler
+ return self.clauseelement.scalar_using(self.engine)
class ClauseElement(object):
"""base class for elements of a programmatically constructed SQL expression."""
else:
return None
- engine = property(lambda s: s._find_engine(), doc="attempts to locate a SQLEngine within this ClauseElement structure, or returns None if none found.")
+ engine = property(lambda s: s._find_engine(), doc="attempts to locate a Engine within this ClauseElement structure, or returns None if none found.")
def using(self, abstractengine):
return Executor(self, abstractengine)
+
+ def execute_using(self, engine, *multiparams, **params):
+ compile_params = self._conv_params(*multiparams, **params)
+ return self.compile(engine=engine, parameters=compile_params).execute(*multiparams, **params)
+ def scalar_using(self, engine, *multiparams, **params):
+ compile_params = self._conv_params(*multiparams, **params)
+ return self.compile(engine=engine, parameters=compile_params).scalar(*multiparams, **params)
+ def _conv_params(self, *multiparams, **params):
+ if len(multiparams):
+ return multiparams[0]
+ else:
+ return params
+ def compile(self, engine=None, parameters=None, compiler=None, dialect=None):
+ """compiles this SQL expression.
+
+ Uses the given Compiler, or the given AbstractDialect or Engine to create a Compiler. If no compiler
+ arguments are given, tries to use the underlying Engine this ClauseElement is bound
+ to to create a Compiler, if any. Finally, if there is no bound Engine, uses an ANSIDialect
+ to create a default Compiler.
- def compile(self, engine = None, parameters = None, typemap=None, compiler=None):
- """compiles this SQL expression using its underlying SQLEngine to produce
- a Compiled object. If no engine can be found, an ANSICompiler is used with no engine.
bindparams is a dictionary representing the default bind parameters to be used with
- the statement. """
+ the statement. if the bindparams is a list, it is assumed to be a list of dictionaries
+ and the first dictionary in the list is used with which to compile against.
+ The bind parameters can in some cases determine the output of the compilation, such as for UPDATE
+ and INSERT statements the bind parameters that are present determine the SET and VALUES clause of
+ those statements.
+ """
+
+ if (isinstance(parameters, list) or isinstance(parameters, tuple)):
+ parameters = parameters[0]
if compiler is None:
- if engine is not None:
+ if dialect is not None:
+ compiler = dialect.compiler(self, parameters)
+ elif engine is not None:
compiler = engine.compiler(self, parameters)
elif self.engine is not None:
compiler = self.engine.compiler(self, parameters)
if compiler is None:
import sqlalchemy.ansisql as ansisql
- compiler = ansisql.ANSICompiler(self, parameters=parameters)
+ compiler = ansisql.ANSIDialect().compiler(self, parameters=parameters)
compiler.compile()
return compiler
return str(self.compile())
def execute(self, *multiparams, **params):
- return self.using(self.engine).execute(*multiparams, **params)
+ return self.execute_using(self.engine, *multiparams, **params)
def scalar(self, *multiparams, **params):
- return self.using(self.engine).scalar(*multiparams, **params)
+ return self.scalar_using(self.engine, *multiparams, **params)
def __and__(self, other):
return and_(self, other)
def __div__(self, other):
return self._operate('/', other)
def __mod__(self, other):
- return self._operate('%', other)
+ return self._operate('%', other)
def __truediv__(self, other):
return self._operate('/', other)
def _bind_param(self, obj):
return BooleanExpression(self._compare_self(), null(), 'IS')
elif operator == '!=':
return BooleanExpression(self._compare_self(), null(), 'IS NOT')
- return BooleanExpression(self._compare_self(), null(), 'IS')
else:
raise exceptions.ArgumentError("Only '='/'!=' operators can be used with NULL")
elif _is_literal(obj):
obj = self._bind_param(obj)
+
return BooleanExpression(self._compare_self(), obj, operator, type=self._compare_type(obj))
def _operate(self, operator, obj):
if _is_literal(obj):
return True
class ColumnElement(Selectable, CompareMixin):
- """represents a column element within the list of a Selectable's columns. Provides
- default implementations for the things a "column" needs, including a "primary_key" flag,
- a "foreign_key" accessor, an "original" accessor which represents the ultimate column
- underlying a string of labeled/select-wrapped columns, and "columns" which returns a list
- of the single column, providing the same list-based interface as a FromClause."""
- primary_key = property(lambda self:getattr(self, '_primary_key', False))
- foreign_key = property(lambda self:getattr(self, '_foreign_key', False))
- original = property(lambda self:getattr(self, '_original', self))
- parent = property(lambda self:getattr(self, '_parent', self))
- columns = property(lambda self:[self])
+ """represents a column element within the list of a Selectable's columns.
+ A ColumnElement can either be directly associated with a TableClause, or
+ a free-standing textual column with no table, or is a "proxy" column, indicating
+ it is placed on a Selectable such as an Alias or Select statement and ultimately corresponds
+ to a TableClause-attached column (or in the case of a CompositeSelect, a proxy ColumnElement
+ may correspond to several TableClause-attached columns)."""
+
+ primary_key = property(lambda self:getattr(self, '_primary_key', False), doc="primary key flag. indicates if this Column represents part or whole of a primary key.")
+ foreign_key = property(lambda self:getattr(self, '_foreign_key', False), doc="foreign key accessor. points to a ForeignKey object which represents a Foreign Key placed on this column's ultimate ancestor.")
+ columns = property(lambda self:[self], doc="Columns accessor which just returns self, to provide compatibility with Selectable objects.")
+
+ def _get_orig_set(self):
+ try:
+ return self.__orig_set
+ except AttributeError:
+ self.__orig_set = sets.Set([self])
+ return self.__orig_set
+ def _set_orig_set(self, s):
+ if len(s) == 0:
+ s.add(self)
+ self.__orig_set = s
+ orig_set = property(_get_orig_set, _set_orig_set,doc="""a Set containing TableClause-bound, non-proxied ColumnElements for which this ColumnElement is a proxy. In all cases except for a column proxied from a Union (i.e. CompoundSelect), this set will be just one element.""")
+
+ def shares_lineage(self, othercolumn):
+ """returns True if the given ColumnElement has a common ancestor to this ColumnElement."""
+ for c in self.orig_set:
+ if c in othercolumn.orig_set:
+ return True
+ else:
+ return False
def _make_proxy(self, selectable, name=None):
"""creates a new ColumnElement representing this ColumnElement as it appears in the select list
- of an enclosing selectable. The default implementation returns a ColumnClause if a name is given,
- else just returns self. This has various mechanics with schema.Column and sql.Label so that
- Column objects as well as non-column objects like Function and BinaryClause can both appear in the
- select list of an enclosing selectable."""
+ of a descending selectable. The default implementation returns a ColumnClause if a name is given,
+ else just returns self."""
if name is not None:
co = ColumnClause(name, selectable)
+ co.orig_set = self.orig_set
selectable.columns[name]= co
return co
else:
"""represents an element that can be used within the FROM clause of a SELECT statement."""
def __init__(self, from_name = None):
self.from_name = self.name = from_name
+ def _display_name(self):
+ if self.named_with_column():
+ return self.name
+ else:
+ return None
+ displayname = property(_display_name)
def _get_from_objects(self):
# this could also be [self], at the moment it doesnt matter to the Select object
return []
def default_order_by(self):
- if not self.engine.default_ordering:
- return None
- elif self.oid_column is not None:
- return [self.oid_column]
- else:
- return self.primary_key
+ return [self.oid_column]
def accept_visitor(self, visitor):
visitor.visit_fromclause(self)
def count(self, whereclause=None, **params):
return Join(self, right, isouter = True, *args, **kwargs)
def alias(self, name=None):
return Alias(self, name)
+ def named_with_column(self):
+ """True if the name of this FromClause may be prepended to a column in a generated SQL statement"""
+ return False
def _locate_oid_column(self):
"""subclasses override this to return an appropriate OID column"""
return None
if not hasattr(self, '_oid_column'):
self._oid_column = self._locate_oid_column()
return self._oid_column
- def _get_col_by_original(self, column, raiseerr=True):
- """given a column which is a schema.Column object attached to a schema.Table object
- (i.e. an "original" column), return the Column object from this
- Selectable which corresponds to that original Column, or None if this Selectable
- does not contain the column."""
- try:
- return self.original_columns[column.original]
- except KeyError:
+ def corresponding_column(self, column, raiseerr=True, keys_ok=False):
+ """given a ColumnElement, return the ColumnElement object from this
+ Selectable which corresponds to that original Column via a proxy relationship."""
+ for c in column.orig_set:
+ try:
+ return self.original_columns[c]
+ except KeyError:
+ pass
+ else:
+ if keys_ok:
+ try:
+ return self.c[column.key]
+ except KeyError:
+ pass
if not raiseerr:
return None
else:
- raise InvalidRequestError("cant get orig for " + str(column) + " with table " + column.table.name + " from table " + self.name)
+ raise exceptions.InvalidRequestError("Given column '%s', attached to table '%s', failed to locate a corresponding column from table '%s'" % (str(column), str(column.table), self.name))
def _get_exported_attribute(self, name):
try:
c = property(lambda s:s._get_exported_attribute('_columns'))
primary_key = property(lambda s:s._get_exported_attribute('_primary_key'))
foreign_keys = property(lambda s:s._get_exported_attribute('_foreign_keys'))
- original_columns = property(lambda s:s._get_exported_attribute('_orig_cols'))
+ original_columns = property(lambda s:s._get_exported_attribute('_orig_cols'), doc="a dictionary mapping an original Table-bound column to a proxied column in this FromClause.")
oid_column = property(_get_oid_column)
def _export_columns(self):
+ """this method is called the first time any of the "exported attrbutes" are called. it receives from the Selectable
+ a list of all columns to be exported and creates "proxy" columns for each one."""
if hasattr(self, '_columns'):
# TODO: put a mutex here ? this is a key place for threading probs
return
if column.is_selectable():
for co in column.columns:
cp = self._proxy_column(co)
- self._orig_cols[co.original] = cp
- if getattr(self, 'oid_column', None):
- self._orig_cols[self.oid_column.original] = self.oid_column
+ for ci in cp.orig_set:
+ self._orig_cols[ci] = cp
+ if self.oid_column is not None:
+ for ci in self.oid_column.orig_set:
+ self._orig_cols[ci] = self.oid_column
def _exportable_columns(self):
return []
def _proxy_column(self, column):
return []
def copy_container(self):
return BindParamClause(self.key, self.value, self.shortname, self.type)
- def typeprocess(self, value, engine):
- return self.type.engine_impl(engine).convert_bind_param(value, engine)
+ def typeprocess(self, value, dialect):
+ return self.type.dialect_impl(dialect).convert_bind_param(value, dialect)
def compare(self, other):
"""compares this BindParamClause to the given clause.
self.type = type
def accept_visitor(self, visitor):
visitor.visit_typeclause(self)
- def _get_from_objects(self):
- return []
+ def _get_from_objects(self):
+ return []
+
class TextClause(ClauseElement):
"""represents literal a SQL text fragment. public constructor is the
text() function.
self.__names.append(name)
return self
def __call__(self, *c, **kwargs):
- return Function(self.__names[-1], packagenames=self.__names[0:-1], engine=self.__engine, *c, **kwargs)
+ kwargs.setdefault('engine', self.__engine)
+ return Function(self.__names[-1], packagenames=self.__names[0:-1], *c, **kwargs)
class BinaryClause(ClauseElement):
"""represents two clauses with an operator in between"""
def __init__(self, left, right, onclause=None, isouter = False):
self.left = left
self.right = right
- # TODO: if no onclause, do NATURAL JOIN
if onclause is None:
self.onclause = self._match_primaries(left, right)
else:
self.onclause = onclause
self.isouter = isouter
- name = property(lambda self: "Join on %s, %s" % (self.left.name, self.right.name))
-
+ name = property(lambda s: "Join object on " + s.left.name + " " + s.right.name)
def _locate_oid_column(self):
return self.left.oid_column
crit = []
for fk in secondary.foreign_keys:
if fk.references(primary):
- crit.append(primary._get_col_by_original(fk.column) == fk.parent)
+ crit.append(primary.corresponding_column(fk.column) == fk.parent)
self.foreignkey = fk.parent
if primary is not secondary:
for fk in primary.foreign_keys:
if fk.references(secondary):
- crit.append(secondary._get_col_by_original(fk.column) == fk.parent)
+ crit.append(secondary.corresponding_column(fk.column) == fk.parent)
self.foreignkey = fk.parent
if len(crit) == 0:
- raise ArgumentError("Cant find any foreign key relationships between '%s' and '%s'" % (primary.name, secondary.name))
+ raise exceptions.ArgumentError("Cant find any foreign key relationships between '%s' and '%s'" % (primary.name, secondary.name))
elif len(crit) == 1:
return (crit[0])
else:
self.original = baseselectable
self.selectable = selectable
if alias is None:
- n = getattr(self.original, 'name', None)
- if n is None:
- n = 'anon'
- elif len(n) > 15:
- n = n[0:15]
- alias = n + "_" + hex(random.randint(0, 65535))[2:]
+ if self.original.named_with_column():
+ alias = getattr(self.original, 'name', None)
+ if alias is None:
+ alias = 'anon'
+ elif len(alias) > 15:
+ alias = alias[0:15]
+ alias = alias + "_" + hex(random.randint(0, 65535))[2:]
self.name = alias
def _locate_oid_column(self):
return self.selectable.oid_column._make_proxy(self)
else:
return None
-
+
+ def named_with_column(self):
+ return True
def _exportable_columns(self):
+ #return self.selectable._exportable_columns()
return self.selectable.columns
def accept_visitor(self, visitor):
self.type = sqltypes.to_instance(type)
obj.parens=True
key = property(lambda s: s.name)
-
_label = property(lambda s: s.name)
- original = property(lambda s:s.obj.original)
- parent = property(lambda s:s.obj.parent)
+ orig_set = property(lambda s:s.obj.orig_set)
def accept_visitor(self, visitor):
self.obj.accept_visitor(visitor)
visitor.visit_label(self)
class ColumnClause(ColumnElement):
"""represents a textual column clause in a SQL statement. May or may not
be bound to an underlying Selectable."""
- def __init__(self, text, selectable=None, type=None):
- self.key = self.name = self.text = text
+ def __init__(self, text, selectable=None, type=None, hidden=False):
+ self.key = self.name = text
self.table = selectable
self.type = sqltypes.to_instance(type)
+ self.hidden = hidden
self.__label = None
def _get_label(self):
if self.__label is None:
- if self.table is not None and self.table.name is not None:
- self.__label = self.table.name + "_" + self.text
+ if self.table is not None and self.table.named_with_column():
+ self.__label = self.table.name + "_" + self.name
+ if self.table.c.has_key(self.__label) or len(self.__label) >= 30:
+ self.__label = self.__label[0:24] + "_" + hex(random.randint(0, 65535))[2:]
else:
- self.__label = self.text
- if (self.table is not None and self.table.c.has_key(self.__label)) or len(self.__label) >= 30:
- self.__label = self.__label[0:24] + "_" + hex(random.randint(0, 65535))[2:]
+ self.__label = self.name
return self.__label
_label = property(_get_label)
def accept_visitor(self, visitor):
for example, this could translate the column "name" from a Table object
to an Alias of a Select off of that Table object."""
- return selectable._get_col_by_original(self.original, False)
+ return selectable.corresponding_column(self.original, False)
def _get_from_objects(self):
if self.table is not None:
return [self.table]
else:
return []
def _bind_param(self, obj):
- if self.table.name is None:
- return BindParamClause(self.text, obj, shortname=self.text, type=self.type)
- else:
- return BindParamClause(self._label, obj, shortname = self.text, type=self.type)
+ return BindParamClause(self._label, obj, shortname = self.name, type=self.type)
def _make_proxy(self, selectable, name = None):
- c = ColumnClause(name or self.text, selectable)
- c._original = self.original
- selectable.columns[c.name] = c
+ c = ColumnClause(name or self.name, selectable, hidden=self.hidden)
+ c.orig_set = self.orig_set
+ if not self.hidden:
+ selectable.columns[c.name] = c
return c
def _compare_type(self, obj):
return self.type
self._primary_key = []
for c in columns:
self.append_column(c)
+ self._oid_column = ColumnClause('oid', self, hidden=True)
indexes = property(lambda s:s._indexes)
-
+
+ def named_with_column(self):
+ return True
def append_column(self, c):
- self._columns[c.text] = c
+ self._columns[c.name] = c
c.table = self
def _locate_oid_column(self):
- if self.engine is None:
- return None
- if self.engine.oid_column_name() is not None:
- _oid_column = schema.Column(self.engine.oid_column_name(), sqltypes.Integer, hidden=True)
- _oid_column._set_parent(self)
- self._orig_columns()[_oid_column.original] = _oid_column
- return _oid_column
- else:
- return None
+ return self._oid_column
def _orig_columns(self):
try:
return self._orig_cols
except AttributeError:
self._orig_cols= {}
for c in self.columns:
- self._orig_cols[c.original] = c
+ for ci in c.orig_set:
+ self._orig_cols[ci] = c
return self._orig_cols
columns = property(lambda s:s._columns)
c = property(lambda s:s._columns)
def _clear(self):
"""clears all attributes on this TableClause so that new items can be added again"""
self.columns.clear()
+ self.indexes.clear()
self.foreign_keys[:] = []
self.primary_key[:] = []
try:
class CompoundSelect(SelectBaseMixin, FromClause):
def __init__(self, keyword, *selects, **kwargs):
+ SelectBaseMixin.__init__(self)
self.keyword = keyword
self.selects = selects
self.use_labels = kwargs.pop('use_labels', False)
s.order_by(None)
self.group_by(*kwargs.get('group_by', [None]))
self.order_by(*kwargs.get('order_by', [None]))
+ self._col_map = {}
+# name = property(lambda s:s.keyword + " statement")
+ def _foo(self):
+ raise "this is a temporary assertion while we refactor SQL to not call 'name' on non-table Selectables"
+ name = property(lambda s:s._foo()) #"SELECT statement")
+
def _locate_oid_column(self):
return self.selects[0].oid_column
-
def _exportable_columns(self):
for s in self.selects:
for c in s.c:
yield c
-
def _proxy_column(self, column):
if self.use_labels:
- return column._make_proxy(self, name=column._label)
+ col = column._make_proxy(self, name=column._label)
else:
- return column._make_proxy(self, name=column.name)
+ col = column._make_proxy(self, name=column.name)
+ try:
+ colset = self._col_map[col.name]
+ except KeyError:
+ colset = sets.Set()
+ self._col_map[col.name] = colset
+ [colset.add(c) for c in col.orig_set]
+ col.orig_set = colset
+ return col
+
def accept_visitor(self, visitor):
self.order_by_clause.accept_visitor(visitor)
self.group_by_clause.accept_visitor(visitor)
"""represents a SELECT statement, with appendable clauses, as well as
the ability to execute itself and return a result set."""
def __init__(self, columns=None, whereclause = None, from_obj = [], order_by = None, group_by=None, having=None, use_labels = False, distinct=False, for_update=False, engine=None, limit=None, offset=None, scalar=False, correlate=True):
+ SelectBaseMixin.__init__(self)
self._froms = util.OrderedDict()
self.use_labels = use_labels
- self.name = None
self.whereclause = None
self.having = None
self._engine = engine
for f in from_obj:
self.append_from(f)
-
-
+
+ def _foo(self):
+ raise "this is a temporary assertion while we refactor SQL to not call 'name' on non-table Selectables"
+ name = property(lambda s:s._foo()) #"SELECT statement")
+
class CorrelatedVisitor(ClauseVisitor):
"""visits a clause, locates any Select clauses, and tells them that they should
correlate their FROM list to that of their parent."""
fromclause._process_from_dict(self._froms, True)
def _locate_oid_column(self):
for f in self._froms.values():
+ if f is self:
+ # TODO: why would we be in our own _froms list ?
+ raise exceptions.AssertionError("Select statement should not be in its own _froms list")
oid = f.oid_column
if oid is not None:
return oid
return union(self, other, **kwargs)
def union_all(self, other, **kwargs):
return union_all(self, other, **kwargs)
-
-# def scalar(self, *multiparams, **params):
- # need to set limit=1, but only in this thread.
- # we probably need to make a copy of the select(). this
- # is expensive. I think cursor.fetchone(), then discard remaining results
- # should be fine with most DBs
- # for now use base scalar() method
-
def _find_engine(self):
- """tries to return a SQLEngine, either explicitly set in this object, or searched
+ """tries to return a Engine, either explicitly set in this object, or searched
within the from clauses for one"""
if self._engine is not None:
class UpdateBase(ClauseElement):
"""forms the base for INSERT, UPDATE, and DELETE statements."""
-
def _process_colparams(self, parameters):
"""receives the "values" of an INSERT or UPDATE statement and constructs
appropriate ind parameters."""
except KeyError:
del parameters[key]
return parameters
-
def _find_engine(self):
- return self._engine
-
+ return self.table.engine
class Insert(UpdateBase):
def __init__(self, table, values=None, **params):
self.table = table
self.select = None
self.parameters = self._process_colparams(values)
- self._engine = self.table.engine
def accept_visitor(self, visitor):
if self.select is not None:
self.table = table
self.whereclause = whereclause
self.parameters = self._process_colparams(values)
- self._engine = self.table.engine
def accept_visitor(self, visitor):
if self.whereclause is not None:
def __init__(self, table, whereclause, **params):
self.table = table
self.whereclause = whereclause
- self._engine = self.table.engine
def accept_visitor(self, visitor):
if self.whereclause is not None:
--- /dev/null
+import sqlalchemy.sql as sql
+import sqlalchemy.schema as schema
+
+"""utility functions that build upon SQL and Schema constructs"""
+
+
+class TableCollection(object):
+ def __init__(self):
+ self.tables = []
+ def add(self, table):
+ self.tables.append(table)
+ def sort(self, reverse=False ):
+ import sqlalchemy.orm.topological
+ tuples = []
+ class TVisitor(schema.SchemaVisitor):
+ def visit_foreign_key(self, fkey):
+ parent_table = fkey.column.table
+ child_table = fkey.parent.table
+ tuples.append( ( parent_table, child_table ) )
+ vis = TVisitor()
+ for table in self.tables:
+ table.accept_schema_visitor(vis)
+ sorter = sqlalchemy.orm.topological.QueueDependencySorter( tuples, self.tables )
+ head = sorter.sort()
+ sequence = []
+ def to_sequence( node, seq=sequence):
+ seq.append( node.item )
+ for child in node.children:
+ to_sequence( child )
+ to_sequence( head )
+ if reverse:
+ sequence.reverse()
+ return sequence
+
+
+class TableFinder(TableCollection, sql.ClauseVisitor):
+ """given a Clause, locates all the Tables within it into a list."""
+ def __init__(self, table, check_columns=False):
+ TableCollection.__init__(self)
+ self.check_columns = check_columns
+ if table is not None:
+ table.accept_visitor(self)
+ def visit_table(self, table):
+ self.tables.append(table)
+ def __len__(self):
+ return len(self.tables)
+ def __getitem__(self, i):
+ return self.tables[i]
+ def __iter__(self):
+ return iter(self.tables)
+ def __contains__(self, obj):
+ return obj in self.tables
+ def __add__(self, obj):
+ return self.tables + list(obj)
+ def visit_column(self, column):
+ if self.check_columns:
+ column.table.accept_visitor(self)
+
+
\ No newline at end of file
self._impl_dict = {}
return self._impl_dict
impl_dict = property(_get_impl_dict)
+
class TypeEngine(AbstractType):
def __init__(self, *args, **params):
pass
def engine_impl(self, engine):
+ """deprecated; call dialect_impl with a dialect directly."""
+ return self.dialect_impl(engine.dialect)
+ def dialect_impl(self, dialect):
try:
- return self.impl_dict[engine]
- except KeyError:
- return self.impl_dict.setdefault(engine, engine.type_descriptor(self))
+ return self.impl_dict[dialect]
+ except:
+ return self.impl_dict.setdefault(dialect, dialect.type_descriptor(self))
+ def _get_impl(self):
+ if hasattr(self, '_impl'):
+ return self._impl
+ else:
+ return NULLTYPE
+ def _set_impl(self, impl):
+ self._impl = impl
+ impl = property(_get_impl, _set_impl)
def get_col_spec(self):
raise NotImplementedError()
- def convert_bind_param(self, value, engine):
+ def convert_bind_param(self, value, dialect):
return value
- def convert_result_value(self, value, engine):
+ def convert_result_value(self, value, dialect):
return value
def adapt(self, cls):
return cls()
-AbstractType.impl = TypeEngine
class TypeDecorator(AbstractType):
def __init__(self, *args, **kwargs):
+ if not hasattr(self.__class__, 'impl'):
+ raise exceptions.AssertionError("TypeDecorator implementations require a class-level variable 'impl' which refers to the class of type being decorated")
self.impl = self.__class__.impl(*args, **kwargs)
def engine_impl(self, engine):
+ return self.dialect_impl(engine.dialect)
+ def dialect_impl(self, dialect):
try:
- return self.impl_dict[engine]
+ return self.impl_dict[dialect]
except:
- typedesc = engine.type_descriptor(self.impl)
+ typedesc = dialect.type_descriptor(self.impl)
tt = self.copy()
if not isinstance(tt, self.__class__):
raise exceptions.AssertionError("Type object %s does not properly implement the copy() method, it must return an object of type %s" % (self, self.__class__))
tt.impl = typedesc
- self.impl_dict[engine] = tt
+ self.impl_dict[dialect] = tt
return tt
def get_col_spec(self):
return self.impl.get_col_spec()
- def convert_bind_param(self, value, engine):
- return self.impl.convert_bind_param(value, engine)
- def convert_result_value(self, value, engine):
- return self.impl.convert_result_value(value, engine)
+ def convert_bind_param(self, value, dialect):
+ return self.impl.convert_bind_param(value, dialect)
+ def convert_result_value(self, value, dialect):
+ return self.impl.convert_result_value(value, dialect)
def copy(self):
instance = self.__class__.__new__(self.__class__)
instance.__dict__.update(self.__dict__)
class NullTypeEngine(TypeEngine):
def get_col_spec(self):
raise NotImplementedError()
- def convert_bind_param(self, value, engine):
+ def convert_bind_param(self, value, dialect):
return value
- def convert_result_value(self, value, engine):
+ def convert_result_value(self, value, dialect):
return value
self.length = length
def adapt(self, impltype):
return impltype(length=self.length)
- def convert_bind_param(self, value, engine):
- if not engine.convert_unicode or value is None or not isinstance(value, unicode):
+ def convert_bind_param(self, value, dialect):
+ if not dialect.convert_unicode or value is None or not isinstance(value, unicode):
return value
else:
- return value.encode(engine.encoding)
- def convert_result_value(self, value, engine):
- if not engine.convert_unicode or value is None or isinstance(value, unicode):
+ return value.encode(dialect.encoding)
+ def convert_result_value(self, value, dialect):
+ if not dialect.convert_unicode or value is None or isinstance(value, unicode):
return value
else:
- return value.decode(engine.encoding)
+ return value.decode(dialect.encoding)
class Unicode(TypeDecorator):
impl = String
- def convert_bind_param(self, value, engine):
+ def convert_bind_param(self, value, dialect):
if value is not None and isinstance(value, unicode):
- return value.encode(engine.encoding)
+ return value.encode(dialect.encoding)
else:
return value
- def convert_result_value(self, value, engine):
+ def convert_result_value(self, value, dialect):
if value is not None and not isinstance(value, unicode):
- return value.decode(engine.encoding)
+ return value.decode(dialect.encoding)
else:
return value
class Binary(TypeEngine):
def __init__(self, length=None):
self.length = length
- def convert_bind_param(self, value, engine):
- return engine.dbapi().Binary(value)
- def convert_result_value(self, value, engine):
+ def convert_bind_param(self, value, dialect):
+ if value is not None:
+ return dialect.dbapi().Binary(value)
+ else:
+ return None
+ def convert_result_value(self, value, dialect):
return value
- def adap(self, impltype):
+ def adapt(self, impltype):
return impltype(length=self.length)
class PickleType(TypeDecorator):
"""allows the pickle protocol to be specified"""
self.protocol = protocol
super(PickleType, self).__init__()
- def convert_result_value(self, value, engine):
+ def convert_result_value(self, value, dialect):
if value is None:
return None
- buf = self.impl.convert_result_value(value, engine)
+ buf = self.impl.convert_result_value(value, dialect)
return pickle.loads(str(buf))
- def convert_bind_param(self, value, engine):
+ def convert_bind_param(self, value, dialect):
if value is None:
return None
- return self.impl.convert_bind_param(pickle.dumps(value, self.protocol), engine)
+ return self.impl.convert_bind_param(pickle.dumps(value, self.protocol), dialect)
class Boolean(TypeEngine):
pass
"""sets the data for this HistoryArraySet to be that of the given data.
duplicates in the incoming list will be removed."""
# first mark everything current as "deleted"
- for i in self.data:
- self.records[i] = False
+ for item in self.data:
+ self.records[item] = False
+ self.do_value_deleted(item)
# switch array
self.data = data
# TODO: fix this up, remove items from array while iterating
for i in range(0, len(self.data)):
- if not self._setrecord(self.data[i]):
- del self.data[i]
- i -= 1
+ if not self.__setrecord(self.data[i], False):
+ del self.data[i]
+ i -= 1
+ for item in self.data:
+ self.do_value_appended(item)
def history_contains(self, obj):
"""returns true if the given object exists within the history
for this HistoryArrayList."""
return self.records.has_key(obj)
def __hash__(self):
return id(self)
- def _setrecord(self, item):
- if self.readonly:
- raise InvalidRequestError("This list is read only")
+ def do_value_appended(self, value):
+ pass
+ def do_value_deleted(self, value):
+ pass
+ def __setrecord(self, item, dochanged=True):
try:
val = self.records[item]
if val is True or val is None:
return False
else:
self.records[item] = None
+ if dochanged:
+ self.do_value_appended(item)
return True
except KeyError:
self.records[item] = True
+ if dochanged:
+ self.do_value_appended(item)
return True
- def _delrecord(self, item):
- if self.readonly:
- raise InvalidRequestError("This list is read only")
+ def __delrecord(self, item, dochanged=True):
try:
val = self.records[item]
if val is None:
self.records[item] = False
+ if dochanged:
+ self.do_value_deleted(item)
return True
elif val is True:
del self.records[item]
+ if dochanged:
+ self.do_value_deleted(item)
return True
return False
except KeyError:
def has_item(self, item):
return self.records.has_key(item) and self.records[item] is not False
def __setitem__(self, i, item):
- if self._setrecord(item):
+ if self.__setrecord(item):
self.data[i] = item
def __delitem__(self, i):
- self._delrecord(self.data[i])
+ self.__delrecord(self.data[i])
del self.data[i]
def __setslice__(self, i, j, other):
+ print "HAS SETSLICE"
i = max(i, 0); j = max(j, 0)
if isinstance(other, UserList.UserList):
l = other.data
l = other
else:
l = list(other)
- g = [a for a in l if self._setrecord(a)]
+ [self.__delrecord(x) for x in self.data[i:]]
+ g = [a for a in l if self.__setrecord(a)]
self.data[i:] = g
def __delslice__(self, i, j):
i = max(i, 0); j = max(j, 0)
for a in self.data[i:j]:
- self._delrecord(a)
+ self.__delrecord(a)
del self.data[i:j]
def append(self, item):
- if self._setrecord(item):
+ if self.__setrecord(item):
self.data.append(item)
def insert(self, i, item):
- if self._setrecord(item):
+ if self.__setrecord(item):
self.data.insert(i, item)
def pop(self, i=-1):
item = self.data[i]
- if self._delrecord(item):
+ if self.__delrecord(item):
return self.data.pop(i)
def remove(self, item):
- if self._delrecord(item):
+ if self.__delrecord(item):
self.data.remove(item)
def extend(self, item_list):
for item in item_list:
from setuptools import setup, find_packages
setup(name = "SQLAlchemy",
- version = "0.1.7",
+ version = "0.1.6",
description = "Database Abstraction Library",
author = "Mike Bayer",
author_email = "mike_mp@zzzcomputing.com",
-from sqlalchemy.ext.activemapper import ActiveMapper, column, one_to_many, one_to_one
-from sqlalchemy.ext import activemapper
-from sqlalchemy import objectstore, global_connect
-from sqlalchemy import and_, or_
-from sqlalchemy import ForeignKey, String, Integer, DateTime
-from datetime import datetime
+from sqlalchemy.ext.activemapper import ActiveMapper, column, one_to_many, one_to_one, objectstore
+from sqlalchemy import and_, or_, clear_mappers
+from sqlalchemy import ForeignKey, String, Integer, DateTime
+from datetime import datetime
import unittest
+import sqlalchemy.ext.activemapper as activemapper
-#
-# application-level model objects
-#
+import testbase
-class Person(ActiveMapper):
- class mapping:
- id = column(Integer, primary_key=True)
- full_name = column(String)
- first_name = column(String)
- middle_name = column(String)
- last_name = column(String)
- birth_date = column(DateTime)
- ssn = column(String)
- gender = column(String)
- home_phone = column(String)
- cell_phone = column(String)
- work_phone = column(String)
- prefs_id = column(Integer, foreign_key=ForeignKey('preferences.id'))
- addresses = one_to_many('Address', colname='person_id', backref='person')
- preferences = one_to_one('Preferences', colname='pref_id', backref='person')
-
- def __str__(self):
- s = '%s\n' % self.full_name
- s += ' * birthdate: %s\n' % (self.birth_date or 'not provided')
- s += ' * fave color: %s\n' % (self.preferences.favorite_color or 'Unknown')
- s += ' * personality: %s\n' % (self.preferences.personality_type or 'Unknown')
-
- for address in self.addresses:
- s += ' * address: %s\n' % address.address_1
- s += ' %s, %s %s\n' % (address.city, address.state, address.postal_code)
-
- return s
+class testcase(testbase.PersistTest):
+ def setUpAll(self):
+ global Person, Preferences, Address
+
+ class Person(ActiveMapper):
+ class mapping:
+ id = column(Integer, primary_key=True)
+ full_name = column(String)
+ first_name = column(String)
+ middle_name = column(String)
+ last_name = column(String)
+ birth_date = column(DateTime)
+ ssn = column(String)
+ gender = column(String)
+ home_phone = column(String)
+ cell_phone = column(String)
+ work_phone = column(String)
+ prefs_id = column(Integer, foreign_key=ForeignKey('preferences.id'))
+ addresses = one_to_many('Address', colname='person_id', backref='person')
+ preferences = one_to_one('Preferences', colname='pref_id', backref='person')
+ def __str__(self):
+ s = '%s\n' % self.full_name
+ s += ' * birthdate: %s\n' % (self.birth_date or 'not provided')
+ s += ' * fave color: %s\n' % (self.preferences.favorite_color or 'Unknown')
+ s += ' * personality: %s\n' % (self.preferences.personality_type or 'Unknown')
-class Preferences(ActiveMapper):
- class mapping:
- __table__ = 'preferences'
- id = column(Integer, primary_key=True)
- favorite_color = column(String)
- personality_type = column(String)
+ for address in self.addresses:
+ s += ' * address: %s\n' % address.address_1
+ s += ' %s, %s %s\n' % (address.city, address.state, address.postal_code)
+ return s
-class Address(ActiveMapper):
- class mapping:
- id = column(Integer, primary_key=True)
- type = column(String)
- address_1 = column(String)
- city = column(String)
- state = column(String)
- postal_code = column(String)
- person_id = column(Integer, foreign_key=ForeignKey('person.id'))
+ class Preferences(ActiveMapper):
+ class mapping:
+ __table__ = 'preferences'
+ id = column(Integer, primary_key=True)
+ favorite_color = column(String)
+ personality_type = column(String)
+ class Address(ActiveMapper):
+ class mapping:
+ id = column(Integer, primary_key=True)
+ type = column(String)
+ address_1 = column(String)
+ city = column(String)
+ state = column(String)
+ postal_code = column(String)
+ person_id = column(Integer, foreign_key=ForeignKey('person.id'))
+ activemapper.metadata.connect(testbase.db)
+ activemapper.create_tables()
-class testcase(unittest.TestCase):
-
+ def tearDownAll(self):
+ clear_mappers()
+ activemapper.drop_tables()
+
def tearDown(self):
- people = Person.select()
- for person in people: person.delete()
+ for t in activemapper.metadata.table_iterator(reverse=True):
+ t.delete().execute()
+ #people = Person.select()
+ #for person in people: person.delete()
- addresses = Address.select()
- for address in addresses: address.delete()
+ #addresses = Address.select()
+ #for address in addresses: address.delete()
- preferences = Preferences.select()
- for preference in preferences: preference.delete()
+ #preferences = Preferences.select()
+ #for preference in preferences: preference.delete()
- objectstore.commit()
- objectstore.clear()
+ #objectstore.flush()
+ #objectstore.clear()
def create_person_one(self):
# create a person
def test_create(self):
- global_connect('sqlite:///', echo=False)
- activemapper.create_tables()
-
p1 = self.create_person_one()
-
- objectstore.commit()
+ objectstore.flush()
objectstore.clear()
results = Person.select()
def test_delete(self):
p1 = self.create_person_one()
- objectstore.commit()
+ objectstore.flush()
objectstore.clear()
results = Person.select()
self.assertEquals(len(results), 1)
results[0].delete()
- objectstore.commit()
+ objectstore.flush()
objectstore.clear()
results = Person.select()
p1 = self.create_person_one()
p2 = self.create_person_two()
- objectstore.commit()
+ objectstore.flush()
objectstore.clear()
# select and make sure we get back two results
# FIXME: I don't know why, but it seems that my backwards relationship
# on preferences still ends up being a list even though I pass
# in uselist=False...
+ # FIXED: the backref is a new PropertyLoader which needs its own "uselist".
+ # uses a function which I dont think existed when you first wrote ActiveMapper.
p1 = self.create_person_one()
self.assertEquals(p1.preferences.person, p1)
p1.delete()
- objectstore.commit()
+ objectstore.flush()
objectstore.clear()
def test_select_by(self):
# FIXME: either I don't understand select_by, or it doesn't work.
+ # FIXED (as good as we can for now): yup....everyone thinks it works that way....it only
+ # generates joins for keyword arguments, not ColumnClause args. would need a new layer of
+ # "MapperClause" objects to use properties in expressions. (MB)
p1 = self.create_person_one()
p2 = self.create_person_two()
- objectstore.commit()
+ objectstore.flush()
objectstore.clear()
- results = Person.select_by(
- Address.c.postal_code.like('30075')
+ results = Person.select(
+ Address.c.postal_code.like('30075') &
+ Person.join_to('addresses')
)
self.assertEquals(len(results), 1)
if __name__ == '__main__':
+ # go ahead and setup the database connection, and create the tables
+
+ # launch the unit tests
unittest.main()
\ No newline at end of file
import testbase
import unittest
-testbase.echo = False
-
-#test
-
def suite():
modules_to_test = (
# core utilities
- 'historyarray',
+ 'historyarray',
'attributes',
'dependency',
-
+
# connectivity, execution
'pool',
- 'engine',
+ 'transaction',
# schema/tables
'reflection',
'eagertest2',
# ORM persistence
+ 'sessioncontext',
'objectstore',
+ 'cascade',
'relationships',
# cyclical ORM persistence
'manytomany',
'onetoone',
'inheritance',
+ 'polymorph',
# extensions
'proxy_engine',
+ 'activemapper'
#'wsgi_test',
)
alltests.addTest(unittest.findTestCases(module, suiteClass=None))
return alltests
-import sys
-sys.stdout = sys.stderr
if __name__ == '__main__':
testbase.runTests(suite())
manager.register_attribute(MyTest, 'email_address', uselist = False)
x = MyTest()
x.user_id=7
- s = pickle.dumps(x)
- y = pickle.loads(s)
+ pickle.dumps(x)
def testlist(self):
class User(object):pass
--- /dev/null
+import testbase, tables
+import unittest, sys, datetime
+
+from sqlalchemy.ext.sessioncontext import SessionContext
+from sqlalchemy import *
+
+class O2MCascadeTest(testbase.AssertMixin):
+ def tearDown(self):
+ ctx.current.clear()
+ tables.delete()
+
+ def tearDownAll(self):
+ clear_mappers()
+ tables.drop()
+
+ def setUpAll(self):
+ global ctx, data
+ ctx = SessionContext(lambda: create_session(echo_uow=True))
+ tables.create()
+ mapper(tables.User, tables.users, properties = dict(
+ address = relation(mapper(tables.Address, tables.addresses), lazy = False, uselist = False, private = True),
+ orders = relation(
+ mapper(tables.Order, tables.orders, properties = dict (
+ items = relation(mapper(tables.Item, tables.orderitems), lazy = False, uselist =True, private = True)
+ )),
+ lazy = True, uselist = True, private = True)
+ ))
+
+ def setUp(self):
+ global data
+ data = [tables.User,
+ {'user_name' : 'ed',
+ 'address' : (tables.Address, {'email_address' : 'foo@bar.com'}),
+ 'orders' : (tables.Order, [
+ {'description' : 'eds 1st order', 'items' : (tables.Item, [{'item_name' : 'eds o1 item'}, {'item_name' : 'eds other o1 item'}])},
+ {'description' : 'eds 2nd order', 'items' : (tables.Item, [{'item_name' : 'eds o2 item'}, {'item_name' : 'eds other o2 item'}])}
+ ])
+ },
+ {'user_name' : 'jack',
+ 'address' : (tables.Address, {'email_address' : 'jack@jack.com'}),
+ 'orders' : (tables.Order, [
+ {'description' : 'jacks 1st order', 'items' : (tables.Item, [{'item_name' : 'im a lumberjack'}, {'item_name' : 'and im ok'}])}
+ ])
+ },
+ {'user_name' : 'foo',
+ 'address' : (tables.Address, {'email_address': 'hi@lala.com'}),
+ 'orders' : (tables.Order, [
+ {'description' : 'foo order', 'items' : (tables.Item, [])},
+ {'description' : 'foo order 2', 'items' : (tables.Item, [{'item_name' : 'hi'}])},
+ {'description' : 'foo order three', 'items' : (tables.Item, [{'item_name' : 'there'}])}
+ ])
+ }
+ ]
+
+ for elem in data[1:]:
+ u = tables.User()
+ ctx.current.save(u)
+ u.user_name = elem['user_name']
+ u.address = tables.Address()
+ u.address.email_address = elem['address'][1]['email_address']
+ u.orders = []
+ for order in elem['orders'][1]:
+ o = tables.Order()
+ o.isopen = None
+ o.description = order['description']
+ u.orders.append(o)
+ o.items = []
+ for item in order['items'][1]:
+ i = tables.Item()
+ i.item_name = item['item_name']
+ o.items.append(i)
+
+ ctx.current.flush()
+ ctx.current.clear()
+
+
+ def testdelete(self):
+ l = ctx.current.query(tables.User).select()
+ for u in l:
+ self.echo( repr(u.orders))
+ self.assert_result(l, data[0], *data[1:])
+
+ self.echo("\n\n\n")
+ ids = (l[0].user_id, l[2].user_id)
+ ctx.current.delete(l[0])
+ ctx.current.delete(l[2])
+
+ ctx.current.flush()
+ self.assert_(tables.orders.count(tables.orders.c.user_id.in_(*ids)).scalar() == 0)
+ self.assert_(tables.orderitems.count(tables.orders.c.user_id.in_(*ids) &(tables.orderitems.c.order_id==tables.orders.c.order_id)).scalar() == 0)
+ self.assert_(tables.addresses.count(tables.addresses.c.user_id.in_(*ids)).scalar() == 0)
+ self.assert_(tables.users.count(tables.users.c.user_id.in_(*ids)).scalar() == 0)
+
+
+ def testorphan(self):
+ l = ctx.current.query(tables.User).select()
+ jack = l[1]
+ jack.orders[:] = []
+
+ ids = [jack.user_id]
+ self.assert_(tables.orders.count(tables.orders.c.user_id.in_(*ids)).scalar() == 1)
+ self.assert_(tables.orderitems.count(tables.orders.c.user_id.in_(*ids) &(tables.orderitems.c.order_id==tables.orders.c.order_id)).scalar() == 2)
+
+ ctx.current.flush()
+
+ self.assert_(tables.orders.count(tables.orders.c.user_id.in_(*ids)).scalar() == 0)
+ self.assert_(tables.orderitems.count(tables.orders.c.user_id.in_(*ids) &(tables.orderitems.c.order_id==tables.orders.c.order_id)).scalar() == 0)
+
+
+class M2OCascadeTest(testbase.AssertMixin):
+ def tearDown(self):
+ ctx.current.clear()
+ for t in metadata.table_iterator(reverse=True):
+ t.delete().execute()
+
+ def tearDownAll(self):
+ clear_mappers()
+ metadata.drop_all()
+
+ def setUpAll(self):
+ global ctx, data, metadata, User, Pref
+ ctx = SessionContext(create_session)
+ metadata = BoundMetaData(testbase.db)
+ prefs = Table('prefs', metadata,
+ Column('prefs_id', Integer, Sequence('prefs_id_seq', optional=True), primary_key=True),
+ Column('prefs_data', String(40)))
+
+ users = Table('users', metadata,
+ Column('user_id', Integer, Sequence('user_id_seq', optional=True), primary_key = True),
+ Column('user_name', String(40)),
+ Column('pref_id', Integer, ForeignKey('prefs.prefs_id'))
+ )
+ class User(object):
+ pass
+ class Pref(object):
+ pass
+ metadata.create_all()
+ mapper(User, users, properties = dict(
+ pref = relation(mapper(Pref, prefs), lazy=False, cascade="all, delete-orphan")
+ ))
+
+ def setUp(self):
+ global data
+ data = [User,
+ {'user_name' : 'ed',
+ 'pref' : (Pref, {'prefs_data' : 'pref 1'}),
+ },
+ {'user_name' : 'jack',
+ 'pref' : (Pref, {'prefs_data' : 'pref 2'}),
+ },
+ {'user_name' : 'foo',
+ 'pref' : (Pref, {'prefs_data' : 'pref 3'}),
+ }
+ ]
+
+ for elem in data[1:]:
+ u = User()
+ ctx.current.save(u)
+ u.user_name = elem['user_name']
+ u.pref = Pref()
+ u.pref.prefs_data = elem['pref'][1]['prefs_data']
+
+ ctx.current.flush()
+ ctx.current.clear()
+
+ def testorphan(self):
+ l = ctx.current.query(User).select()
+ jack = l[1]
+ jack.pref = None
+ ctx.current.flush()
+
+if __name__ == "__main__":
+ testbase.main()
class SelfReferentialTest(AssertMixin):
"""tests a self-referential mapper, with an additional list of child objects."""
def setUpAll(self):
- testbase.db.tables.clear()
- global t1
- global t2
- t1 = Table('t1', testbase.db,
+ global t1, t2, metadata
+ metadata = BoundMetaData(testbase.db)
+ t1 = Table('t1', metadata,
Column('c1', Integer, primary_key=True),
Column('parent_c1', Integer, ForeignKey('t1.c1')),
Column('data', String(20))
)
- t2 = Table('t2', testbase.db,
+ t2 = Table('t2', metadata,
Column('c1', Integer, primary_key=True),
Column('c1id', Integer, ForeignKey('t1.c1')),
Column('data', String(20))
)
- t1.create()
- t2.create()
+ metadata.create_all()
def tearDownAll(self):
- t2.drop()
- t1.drop()
+ metadata.drop_all()
def setUp(self):
- objectstore.clear()
clear_mappers()
def testsingle(self):
})
a = C1('head c1')
a.c1s.append(C1('another c1'))
- objectstore.commit()
- objectstore.delete(a)
- objectstore.commit()
+ sess = create_session(echo_uow=False)
+ sess.save(a)
+ sess.flush()
+ sess.delete(a)
+ sess.flush()
def testcycle(self):
class C1(Tester):
a.c1s[0].c1s.append(C1('subchild2'))
a.c1s[1].c2s.append(C2('child2 data1'))
a.c1s[1].c2s.append(C2('child2 data2'))
- objectstore.commit()
+ sess = create_session(echo_uow=False)
+ sess.save(a)
+ sess.flush()
- objectstore.delete(a)
- objectstore.commit()
+ sess.delete(a)
+ sess.flush()
class BiDirectionalOneToManyTest(AssertMixin):
"""tests two mappers with a one-to-many relation to each other."""
def setUpAll(self):
- testbase.db.tables.clear()
- global t1
- global t2
- t1 = Table('t1', testbase.db,
+ global t1, t2, metadata
+ metadata = BoundMetaData(testbase.db)
+ t1 = Table('t1', metadata,
Column('c1', Integer, primary_key=True),
Column('c2', Integer, ForeignKey('t2.c1'))
)
- t2 = Table('t2', testbase.db,
+ t2 = Table('t2', metadata,
Column('c1', Integer, primary_key=True),
Column('c2', Integer)
)
- t2.create()
- t1.create()
+ metadata.create_all()
t2.c.c2.append_item(ForeignKey('t1.c1'))
def tearDownAll(self):
- t1.drop()
+ t1.drop()
t2.drop()
- def setUp(self):
- objectstore.clear()
- #objectstore.LOG = True
+ #metadata.drop_all()
+ def tearDown(self):
clear_mappers()
-
def testcycle(self):
class C1(object):pass
class C2(object):pass
a.c2s.append(b)
d.c1s.append(c)
b.c1s.append(c)
- objectstore.commit()
+ sess = create_session()
+ [sess.save(x) for x in [a,b,c,d,e,f]]
+ sess.flush()
class BiDirectionalOneToManyTest2(AssertMixin):
"""tests two mappers with a one-to-many relation to each other, with a second one-to-many on one of the mappers"""
def setUpAll(self):
- testbase.db.tables.clear()
- global t1
- global t2
- global t3
- t1 = Table('t1', testbase.db,
+ global t1, t2, t3, metadata
+ metadata = BoundMetaData(testbase.db)
+ t1 = Table('t1', metadata,
Column('c1', Integer, primary_key=True),
Column('c2', Integer, ForeignKey('t2.c1')),
)
- t2 = Table('t2', testbase.db,
+ t2 = Table('t2', metadata,
Column('c1', Integer, primary_key=True),
Column('c2', Integer),
)
t2.create()
t1.create()
t2.c.c2.append_item(ForeignKey('t1.c1'))
- t3 = Table('t1_data', testbase.db,
+ t3 = Table('t1_data', metadata,
Column('c1', Integer, primary_key=True),
Column('t1id', Integer, ForeignKey('t1.c1')),
Column('data', String(20)))
t3.create()
- def setUp(self):
- objectstore.clear()
- #objectstore.LOG = True
+ def tearDown(self):
clear_mappers()
def tearDownAll(self):
a.data.append(C1Data('c1data1'))
a.data.append(C1Data('c1data2'))
c.data.append(C1Data('c1data3'))
- objectstore.commit()
+ sess = create_session()
+ [sess.save(x) for x in [a,b,c,d,e,f]]
+ sess.flush()
- objectstore.delete(d)
- objectstore.delete(c)
- objectstore.commit()
+ sess.delete(d)
+ sess.delete(c)
+ sess.flush()
class OneToManyManyToOneTest(AssertMixin):
"""tests two mappers, one has a one-to-many on the other mapper, the other has a separate many-to-one relationship to the first.
two tests will have a row for each item that is dependent on the other. without the "post_update" flag, such relationships
raise an exception when dependencies are sorted."""
def setUpAll(self):
- testbase.db.tables.clear()
+ global metadata
+ metadata = BoundMetaData(testbase.db)
global person
global ball
- ball = Table('ball', db,
+ ball = Table('ball', metadata,
Column('id', Integer, Sequence('ball_id_seq', optional=True), primary_key=True),
Column('person_id', Integer),
)
- person = Table('person', db,
+ person = Table('person', metadata,
Column('id', Integer, Sequence('person_id_seq', optional=True), primary_key=True),
Column('favoriteBall_id', Integer, ForeignKey('ball.id')),
# Column('favoriteBall_id', Integer),
person.drop()
ball.drop()
- def setUp(self):
- objectstore.clear()
- #objectstore.LOG = True
+ def tearDown(self):
clear_mappers()
def testcycle(self):
b = Ball()
p = Person()
p.balls.append(b)
- objectstore.commit()
+ sess = create_session()
+ sess.save(b)
+ sess.save(b)
+ sess.flush()
def testpostupdate_m2o(self):
"""tests a cycle between two rows, with a post_update on the many-to-one"""
p.balls.append(Ball())
p.balls.append(Ball())
p.favorateBall = b
-
- self.assert_sql(db, lambda: objectstore.uow().commit(), [
+ sess = create_session()
+ sess.save(b)
+ sess.save(p)
+
+ self.assert_sql(db, lambda: sess.flush(), [
(
"INSERT INTO person (favoriteBall_id) VALUES (:favoriteBall_id)",
{'favoriteBall_id': None}
),
(
"INSERT INTO ball (person_id) VALUES (:person_id)",
- lambda:{'person_id':p.id}
+ lambda ctx:{'person_id':p.id}
),
(
"INSERT INTO ball (person_id) VALUES (:person_id)",
- lambda:{'person_id':p.id}
+ lambda ctx:{'person_id':p.id}
),
(
"INSERT INTO ball (person_id) VALUES (:person_id)",
- lambda:{'person_id':p.id}
+ lambda ctx:{'person_id':p.id}
),
(
"INSERT INTO ball (person_id) VALUES (:person_id)",
- lambda:{'person_id':p.id}
+ lambda ctx:{'person_id':p.id}
),
(
"UPDATE person SET favoriteBall_id=:favoriteBall_id WHERE person.id = :person_id",
- lambda:[{'favoriteBall_id':p.favorateBall.id,'person_id':p.id}]
+ lambda ctx:{'favoriteBall_id':p.favorateBall.id,'person_id':p.id}
)
],
with_sequences= [
(
"INSERT INTO person (id, favoriteBall_id) VALUES (:id, :favoriteBall_id)",
- lambda:{'id':db.last_inserted_ids()[0], 'favoriteBall_id': None}
+ lambda ctx:{'id':ctx.last_inserted_ids()[0], 'favoriteBall_id': None}
),
(
"INSERT INTO ball (id, person_id) VALUES (:id, :person_id)",
- lambda:{'id':db.last_inserted_ids()[0],'person_id':p.id}
+ lambda ctx:{'id':ctx.last_inserted_ids()[0],'person_id':p.id}
),
(
"INSERT INTO ball (id, person_id) VALUES (:id, :person_id)",
- lambda:{'id':db.last_inserted_ids()[0],'person_id':p.id}
+ lambda ctx:{'id':ctx.last_inserted_ids()[0],'person_id':p.id}
),
(
"INSERT INTO ball (id, person_id) VALUES (:id, :person_id)",
- lambda:{'id':db.last_inserted_ids()[0],'person_id':p.id}
+ lambda ctx:{'id':ctx.last_inserted_ids()[0],'person_id':p.id}
),
(
"INSERT INTO ball (id, person_id) VALUES (:id, :person_id)",
- lambda:{'id':db.last_inserted_ids()[0],'person_id':p.id}
+ lambda ctx:{'id':ctx.last_inserted_ids()[0],'person_id':p.id}
),
# heres the post update
(
"UPDATE person SET favoriteBall_id=:favoriteBall_id WHERE person.id = :person_id",
- lambda:[{'favoriteBall_id':p.favorateBall.id,'person_id':p.id}]
+ lambda ctx:{'favoriteBall_id':p.favorateBall.id,'person_id':p.id}
)
])
- objectstore.delete(p)
- self.assert_sql(db, lambda: objectstore.uow().commit(), [
+ sess.delete(p)
+ self.assert_sql(db, lambda: sess.flush(), [
# heres the post update (which is a pre-update with deletes)
(
"UPDATE person SET favoriteBall_id=:favoriteBall_id WHERE person.id = :person_id",
- lambda:[{'person_id': p.id, 'favoriteBall_id': None}]
+ lambda ctx:{'person_id': p.id, 'favoriteBall_id': None}
),
(
"DELETE FROM ball WHERE ball.id = :id",
None
# order cant be predicted, but something like:
- #lambda:[{'id': 1L}, {'id': 4L}, {'id': 3L}, {'id': 2L}]
+ #lambda ctx:[{'id': 1L}, {'id': 4L}, {'id': 3L}, {'id': 2L}]
),
(
"DELETE FROM person WHERE person.id = :id",
- lambda:[{'id': p.id}]
+ lambda ctx:[{'id': p.id}]
)
b4 = Ball()
p.balls.append(b4)
p.favorateBall = b
-# objectstore.commit()
- self.assert_sql(db, lambda: objectstore.uow().commit(), [
+ sess = create_session()
+ [sess.save(x) for x in [b,p,b2,b3,b4]]
+
+ self.assert_sql(db, lambda: sess.flush(), [
(
"INSERT INTO ball (person_id) VALUES (:person_id)",
{'person_id':None}
),
(
"INSERT INTO person (favoriteBall_id) VALUES (:favoriteBall_id)",
- lambda:{'favoriteBall_id':b.id}
+ lambda ctx:{'favoriteBall_id':b.id}
),
# heres the post update on each one-to-many item
(
"UPDATE ball SET person_id=:person_id WHERE ball.id = :ball_id",
- lambda:[{'person_id':p.id,'ball_id':b.id}]
+ lambda ctx:{'person_id':p.id,'ball_id':b.id}
),
(
"UPDATE ball SET person_id=:person_id WHERE ball.id = :ball_id",
- lambda:[{'person_id':p.id,'ball_id':b2.id}]
+ lambda ctx:{'person_id':p.id,'ball_id':b2.id}
),
(
"UPDATE ball SET person_id=:person_id WHERE ball.id = :ball_id",
- lambda:[{'person_id':p.id,'ball_id':b3.id}]
+ lambda ctx:{'person_id':p.id,'ball_id':b3.id}
),
(
"UPDATE ball SET person_id=:person_id WHERE ball.id = :ball_id",
- lambda:[{'person_id':p.id,'ball_id':b4.id}]
+ lambda ctx:{'person_id':p.id,'ball_id':b4.id}
),
],
with_sequences=[
(
"INSERT INTO ball (id, person_id) VALUES (:id, :person_id)",
- lambda:{'id':db.last_inserted_ids()[0], 'person_id':None}
+ lambda ctx:{'id':ctx.last_inserted_ids()[0], 'person_id':None}
),
(
"INSERT INTO ball (id, person_id) VALUES (:id, :person_id)",
- lambda:{'id':db.last_inserted_ids()[0], 'person_id':None}
+ lambda ctx:{'id':ctx.last_inserted_ids()[0], 'person_id':None}
),
(
"INSERT INTO ball (id, person_id) VALUES (:id, :person_id)",
- lambda:{'id':db.last_inserted_ids()[0], 'person_id':None}
+ lambda ctx:{'id':ctx.last_inserted_ids()[0], 'person_id':None}
),
(
"INSERT INTO ball (id, person_id) VALUES (:id, :person_id)",
- lambda:{'id':db.last_inserted_ids()[0], 'person_id':None}
+ lambda ctx:{'id':ctx.last_inserted_ids()[0], 'person_id':None}
),
(
"INSERT INTO person (id, favoriteBall_id) VALUES (:id, :favoriteBall_id)",
- lambda:{'id':db.last_inserted_ids()[0], 'favoriteBall_id':b.id}
+ lambda ctx:{'id':ctx.last_inserted_ids()[0], 'favoriteBall_id':b.id}
),
(
"UPDATE ball SET person_id=:person_id WHERE ball.id = :ball_id",
- lambda:[{'person_id':p.id,'ball_id':b.id}]
+ lambda ctx:{'person_id':p.id,'ball_id':b.id}
),
(
"UPDATE ball SET person_id=:person_id WHERE ball.id = :ball_id",
- lambda:[{'person_id':p.id,'ball_id':b2.id}]
+ lambda ctx:{'person_id':p.id,'ball_id':b2.id}
),
(
"UPDATE ball SET person_id=:person_id WHERE ball.id = :ball_id",
- lambda:[{'person_id':p.id,'ball_id':b3.id}]
+ lambda ctx:{'person_id':p.id,'ball_id':b3.id}
),
(
"UPDATE ball SET person_id=:person_id WHERE ball.id = :ball_id",
- lambda:[{'person_id':p.id,'ball_id':b4.id}]
+ lambda ctx:{'person_id':p.id,'ball_id':b4.id}
),
])
- objectstore.delete(p)
- self.assert_sql(db, lambda: objectstore.uow().commit(), [
+ sess.delete(p)
+ self.assert_sql(db, lambda: sess.flush(), [
(
"UPDATE ball SET person_id=:person_id WHERE ball.id = :ball_id",
- lambda:[{'person_id': None, 'ball_id': b.id}]
+ lambda ctx:{'person_id': None, 'ball_id': b.id}
),
(
"UPDATE ball SET person_id=:person_id WHERE ball.id = :ball_id",
- lambda:[{'person_id': None, 'ball_id': b2.id}]
+ lambda ctx:{'person_id': None, 'ball_id': b2.id}
),
(
"UPDATE ball SET person_id=:person_id WHERE ball.id = :ball_id",
- lambda:[{'person_id': None, 'ball_id': b3.id}]
+ lambda ctx:{'person_id': None, 'ball_id': b3.id}
),
(
"UPDATE ball SET person_id=:person_id WHERE ball.id = :ball_id",
- lambda:[{'person_id': None, 'ball_id': b4.id}]
+ lambda ctx:{'person_id': None, 'ball_id': b4.id}
),
(
"DELETE FROM person WHERE person.id = :id",
- lambda:[{'id':p.id}]
+ lambda ctx:[{'id':p.id}]
),
(
"DELETE FROM ball WHERE ball.id = :id",
None
# the order of deletion is not predictable, but its roughly:
- # lambda:[{'id': b.id}, {'id': b2.id}, {'id': b3.id}, {'id': b4.id}]
+ # lambda ctx:[{'id': b.id}, {'id': b2.id}, {'id': b3.id}, {'id': b4.id}]
)
])
import sqlalchemy
db = testbase.db
-testbase.echo=False
+
class DefaultTest(PersistTest):
def setUpAll(self):
use_function_defaults = db.engine.name == 'postgres' or db.engine.name == 'oracle'
is_oracle = db.engine.name == 'oracle'
- # select "count(1)" from the DB which returns different results
- # on different DBs
- currenttime = db.func.current_date(type=Date);
+ # select "count(1)" returns different results on different DBs
+ # also correct for "current_date" compatible as column default, value differences
+ currenttime = func.current_date(type=Date, engine=db);
if is_oracle:
- ts = db.func.sysdate().scalar()
+ ts = db.func.trunc(func.sysdate(), column("'DAY'")).scalar()
f = select([func.count(1) + 5], engine=db).scalar()
f2 = select([func.count(1) + 14], engine=db).scalar()
+ # TODO: engine propigation across nested functions not working
+ currenttime = func.trunc(currenttime, column("'DAY'"), engine=db)
def1 = currenttime
- def2 = text("sysdate")
+ def2 = func.trunc(text("sysdate"), column("'DAY'"))
deftype = Date
elif use_function_defaults:
f = select([func.count(1) + 5], engine=db).scalar()
t.delete().execute()
def teststandalone(self):
- x = t.c.col1.default.execute()
+ c = db.engine.contextual_connect()
+ x = c.execute(t.c.col1.default)
y = t.c.col2.default.execute()
- z = t.c.col3.default.execute()
+ z = c.execute(t.c.col3.default)
self.assert_(50 <= x <= 57)
self.assert_(y == 'imthedefault')
self.assert_(z == f)
self.assert_(5 <= z <= 6)
def testinsert(self):
- t.insert().execute()
- self.assert_(t.engine.lastrow_has_defaults())
+ r = t.insert().execute()
+ self.assert_(r.lastrow_has_defaults())
t.insert().execute()
t.insert().execute()
def testupdate(self):
- t.insert().execute()
- pk = t.engine.last_inserted_ids()[0]
+ r = t.insert().execute()
+ pk = r.last_inserted_ids()[0]
t.update(t.c.col1==pk).execute(col4=None, col5=None)
ctexec = currenttime.scalar()
self.echo("Currenttime "+ repr(ctexec))
self.assert_(14 <= f2 <= 15)
def testupdatevalues(self):
- t.insert().execute()
- pk = t.engine.last_inserted_ids()[0]
+ r = t.insert().execute()
+ pk = r.last_inserted_ids()[0]
t.update(t.c.col1==pk, values={'col3': 55}).execute()
l = t.select(t.c.col1==pk).execute()
l = l.fetchone()
@testbase.supported('postgres', 'oracle')
def teststandalone(self):
- s = Sequence("my_sequence", engine=db)
+ s = Sequence("my_sequence", metadata=testbase.db)
s.create()
try:
- x =s.execute()
+ x = s.execute()
self.assert_(x == 1)
finally:
s.drop()
from testbase import PersistTest
-import sqlalchemy.mapping.topological as topological
+import sqlalchemy.orm.topological as topological
import unittest, sys, os
return repr(self)
class DependencySortTest(PersistTest):
-
- def _assert_sort(self, tuples, allnodes, **kwargs):
-
- head = DependencySorter(tuples, allnodes).sort(**kwargs)
-
- print "\n" + str(head)
- def findnode(t, n, parent=False):
- if n.item is t[0] or (n.cycles is not None and t[0] in [c.item for c in n.cycles]):
- parent=True
- elif n.item is t[1]:
- if not parent and (n.cycles is None or t[0] not in [c.item for c in n.cycles]):
- self.assert_(False, "Node " + str(t[1]) + " not a child of " +str(t[0]))
- else:
- return
- for c in n.children:
- findnode(t, c, parent)
-
- for t in tuples:
- findnode(t, head)
-
def testsort(self):
rootnode = thingy('root')
node2 = thingy('node2')
subnode3 = thingy('subnode3')
subnode4 = thingy('subnode4')
subsubnode1 = thingy('subsubnode1')
- allnodes = [rootnode, node2,node3,node4,subnode1,subnode2,subnode3,subnode4,subsubnode1]
tuples = [
(subnode3, subsubnode1),
(node2, subnode1),
(node4, subnode3),
(node4, subnode4)
]
-
- self._assert_sort(tuples, allnodes)
+ head = DependencySorter(tuples, []).sort()
+ print "\n" + str(head)
def testsort2(self):
node1 = thingy('node1')
(node5, node6),
(node6, node2)
]
- self._assert_sort(tuples, [node1,node2,node3,node4,node5,node6,node7])
+ head = DependencySorter(tuples, [node7]).sort()
+ print "\n" + str(head)
def testsort3(self):
['Mapper|Keyword|keywords,Mapper|IKAssociation|itemkeywords', 'Mapper|Item|items,Mapper|IKAssociation|itemkeywords']
(node3, node2),
(node1,node3)
]
- self._assert_sort(tuples, [node1, node2, node3])
- self._assert_sort(tuples, [node3, node1, node2])
- self._assert_sort(tuples, [node3, node2, node1])
+ head1 = DependencySorter(tuples, [node1, node2, node3]).sort()
+ head2 = DependencySorter(tuples, [node3, node1, node2]).sort()
+ head3 = DependencySorter(tuples, [node3, node2, node1]).sort()
+ # TODO: figure out a "node == node2" function
+ #self.assert_(str(head1) == str(head2) == str(head3))
+ print "\n" + str(head1)
+ print "\n" + str(head2)
+ print "\n" + str(head3)
def testsort4(self):
node1 = thingy('keywords')
(node1, node3),
(node3, node2)
]
- self._assert_sort(tuples, [node1,node2,node3,node4])
+ head = DependencySorter(tuples, []).sort()
+ print "\n" + str(head)
def testsort5(self):
# this one, depenending on the weather,
node3,
node4
]
- self._assert_sort(tuples, allitems)
-
- def testsort6(self):
- #('tbl_c', 'tbl_d'), ('tbl_a', 'tbl_c'), ('tbl_b', 'tbl_d')
- nodea = thingy('tbl_a')
- nodeb = thingy('tbl_b')
- nodec = thingy('tbl_c')
- noded = thingy('tbl_d')
- tuples = [
- (nodec, noded),
- (nodea, nodec),
- (nodeb, noded)
- ]
- allitems = [nodea,nodeb,nodec,noded]
- self._assert_sort(tuples, allitems)
+ head = DependencySorter(tuples, allitems).sort()
+ print "\n" + str(head)
def testcircular(self):
- #print "TESTCIRCULAR"
node1 = thingy('node1')
node2 = thingy('node2')
node3 = thingy('node3')
(node3, node1),
(node4, node1)
]
- self._assert_sort(tuples, [node1,node2,node3,node4,node5], allow_all_cycles=True)
- #print "TESTCIRCULAR DONE"
+ head = DependencySorter(tuples, []).sort(allow_all_cycles=True)
+ print "\n" + str(head)
if __name__ == "__main__":
class EagerTest(AssertMixin):
def setUpAll(self):
global designType, design, part, inheritedPart
-
- designType = Table('design_types', testbase.db,
+ designType = Table('design_types', testbase.metadata,
Column('design_type_id', Integer, primary_key=True),
)
- design =Table('design', testbase.db,
+ design =Table('design', testbase.metadata,
Column('design_id', Integer, primary_key=True),
Column('design_type_id', Integer, ForeignKey('design_types.design_type_id')))
- part = Table('parts', testbase.db,
+ part = Table('parts', testbase.metadata,
Column('part_id', Integer, primary_key=True),
Column('design_id', Integer, ForeignKey('design.design_id')),
Column('design_type_id', Integer, ForeignKey('design_types.design_type_id')))
- inheritedPart = Table('inherited_part', testbase.db,
+ inheritedPart = Table('inherited_part', testbase.metadata,
Column('ip_id', Integer, primary_key=True),
Column('part_id', Integer, ForeignKey('parts.part_id')),
Column('design_id', Integer, ForeignKey('design.design_id')),
)
- designType.create()
- design.create()
- part.create()
- inheritedPart.create()
+ testbase.metadata.create_all()
def tearDownAll(self):
- inheritedPart.drop()
- part.drop()
- design.drop()
- designType.drop()
-
+ testbase.metadata.drop_all()
+ testbase.metadata.clear()
def testone(self):
class Part(object):pass
class Design(object):pass
class DesignType(object):pass
class InheritedPart(object):pass
- assign_mapper(Part, part)
+ mapper(Part, part)
- assign_mapper(InheritedPart, inheritedPart, properties=dict(
+ mapper(InheritedPart, inheritedPart, properties=dict(
part=relation(Part, lazy=False)
))
- assign_mapper(Design, design, properties=dict(
+ mapper(Design, design, properties=dict(
parts=relation(Part, private=True, backref="design"),
inheritedParts=relation(InheritedPart, private=True, backref="design"),
))
- assign_mapper(DesignType, designType, properties=dict(
+ mapper(DesignType, designType, properties=dict(
# designs=relation(Design, private=True, backref="type"),
))
- Design.mapper.add_property("type", relation(DesignType, lazy=False, backref="designs"))
- Part.mapper.add_property("design", relation(Design, lazy=False, backref="parts"))
+ class_mapper(Design).add_property("type", relation(DesignType, lazy=False, backref="designs"))
+ class_mapper(Part).add_property("design", relation(Design, lazy=False, backref="parts"))
#Part.mapper.add_property("designType", relation(DesignType))
d = Design()
- objectstore.commit()
- objectstore.clear()
- x = Design.get(1)
+ sess = create_session()
+ sess.save(d)
+ sess.flush()
+ sess.clear()
+ x = sess.query(Design).get(1)
x.inheritedParts
if __name__ == "__main__":
import unittest, sys, os
from sqlalchemy import *
import datetime
-
-db = testbase.db
+from sqlalchemy.ext.sessioncontext import SessionContext
class EagerTest(AssertMixin):
def setUpAll(self):
- objectstore.clear()
- clear_mappers()
- testbase.db.tables.clear()
-
- global companies_table, addresses_table, invoice_table, phones_table, items_table
+ global companies_table, addresses_table, invoice_table, phones_table, items_table, ctx, metadata
- companies_table = Table('companies', db,
+ metadata = BoundMetaData(testbase.db)
+ ctx = SessionContext(create_session)
+
+ companies_table = Table('companies', metadata,
Column('company_id', Integer, Sequence('company_id_seq', optional=True), primary_key = True),
Column('company_name', String(40)),
)
- addresses_table = Table('addresses', db,
+ addresses_table = Table('addresses', metadata,
Column('address_id', Integer, Sequence('address_id_seq', optional=True), primary_key = True),
Column('company_id', Integer, ForeignKey("companies.company_id")),
Column('address', String(40)),
)
- phones_table = Table('phone_numbers', db,
+ phones_table = Table('phone_numbers', metadata,
Column('phone_id', Integer, Sequence('phone_id_seq', optional=True), primary_key = True),
Column('address_id', Integer, ForeignKey('addresses.address_id')),
Column('type', String(20)),
Column('number', String(10)),
)
- invoice_table = Table('invoices', db,
+ invoice_table = Table('invoices', metadata,
Column('invoice_id', Integer, Sequence('invoice_id_seq', optional=True), primary_key = True),
Column('company_id', Integer, ForeignKey("companies.company_id")),
Column('date', DateTime),
)
- items_table = Table('items', db,
+ items_table = Table('items', metadata,
Column('item_id', Integer, Sequence('item_id_seq', optional=True), primary_key = True),
Column('invoice_id', Integer, ForeignKey('invoices.invoice_id')),
Column('code', String(20)),
Column('qty', Integer),
)
- companies_table.create()
- addresses_table.create()
- phones_table.create()
- invoice_table.create()
- items_table.create()
+ metadata.create_all()
def tearDownAll(self):
- items_table.drop()
- invoice_table.drop()
- phones_table.drop()
- addresses_table.drop()
- companies_table.drop()
+ metadata.drop_all()
def tearDown(self):
- objectstore.clear()
clear_mappers()
- items_table.delete().execute()
- invoice_table.delete().execute()
- phones_table.delete().execute()
- addresses_table.delete().execute()
- companies_table.delete().execute()
+ for t in metadata.table_iterator(reverse=True):
+ t.delete().execute()
def testone(self):
"""tests eager load of a many-to-one attached to a one-to-many. this testcase illustrated
def __repr__(self):
return "Invoice:" + repr(getattr(self, 'invoice_id', None)) + " " + repr(getattr(self, 'date', None)) + " " + repr(self.company)
- Address.mapper = mapper(Address, addresses_table, properties={
- })
- Company.mapper = mapper(Company, companies_table, properties={
- 'addresses' : relation(Address.mapper, lazy=False),
- })
- Invoice.mapper = mapper(Invoice, invoice_table, properties={
- 'company': relation(Company.mapper, lazy=False, )
- })
+ mapper(Address, addresses_table, properties={
+ }, extension=ctx.mapper_extension)
+ mapper(Company, companies_table, properties={
+ 'addresses' : relation(Address, lazy=False),
+ }, extension=ctx.mapper_extension)
+ mapper(Invoice, invoice_table, properties={
+ 'company': relation(Company, lazy=False, )
+ }, extension=ctx.mapper_extension)
c1 = Company()
c1.company_name = 'company 1'
i1.date = datetime.datetime.now()
i1.company = c1
-
- objectstore.commit()
+ ctx.current.flush()
company_id = c1.company_id
invoice_id = i1.invoice_id
- objectstore.clear()
+ ctx.current.clear()
- c = Company.mapper.get(company_id)
+ c = ctx.current.query(Company).get(company_id)
- objectstore.clear()
+ ctx.current.clear()
- i = Invoice.mapper.get(invoice_id)
+ i = ctx.current.query(Invoice).get(invoice_id)
self.echo(repr(c))
self.echo(repr(i.company))
def __repr__(self):
return "Item: " + repr(getattr(self, 'item_id', None)) + " " + repr(getattr(self, 'invoice_id', None)) + " " + repr(self.code) + " " + repr(self.qty)
- Phone.mapper = mapper(Phone, phones_table, is_primary=True)
+ mapper(Phone, phones_table, extension=ctx.mapper_extension)
- Address.mapper = mapper(Address, addresses_table, properties={
- 'phones': relation(Phone.mapper, lazy=False, backref='address')
- })
+ mapper(Address, addresses_table, properties={
+ 'phones': relation(Phone, lazy=False, backref='address')
+ }, extension=ctx.mapper_extension)
- Company.mapper = mapper(Company, companies_table, properties={
- 'addresses' : relation(Address.mapper, lazy=False, backref='company'),
- })
+ mapper(Company, companies_table, properties={
+ 'addresses' : relation(Address, lazy=False, backref='company'),
+ }, extension=ctx.mapper_extension)
- Item.mapper = mapper(Item, items_table, is_primary=True)
+ mapper(Item, items_table, extension=ctx.mapper_extension)
- Invoice.mapper = mapper(Invoice, invoice_table, properties={
- 'items': relation(Item.mapper, lazy=False, backref='invoice'),
- 'company': relation(Company.mapper, lazy=False, backref='invoices')
- })
+ mapper(Invoice, invoice_table, properties={
+ 'items': relation(Item, lazy=False, backref='invoice'),
+ 'company': relation(Company, lazy=False, backref='invoices')
+ }, extension=ctx.mapper_extension)
- objectstore.clear()
+ ctx.current.clear()
c1 = Company()
c1.company_name = 'company 1'
c1.addresses.append(a2)
- objectstore.commit()
+ ctx.current.flush()
company_id = c1.company_id
- objectstore.clear()
+ ctx.current.clear()
- a = Company.mapper.get(company_id)
+ a = ctx.current.query(Company).get(company_id)
self.echo(repr(a))
# set up an invoice
item3.qty = 3
item3.invoice = i1
- objectstore.commit()
+ ctx.current.flush()
invoice_id = i1.invoice_id
- objectstore.clear()
+ ctx.current.clear()
- c = Company.mapper.get(company_id)
+ c = ctx.current.query(Company).get(company_id)
self.echo(repr(c))
- objectstore.clear()
+ ctx.current.clear()
- i = Invoice.mapper.get(invoice_id)
+ i = ctx.current.query(Invoice).get(invoice_id)
self.echo(repr(i))
self.assert_(repr(i.company) == repr(c))
+++ /dev/null
-from sqlalchemy import *
-
-from testbase import PersistTest
-import testbase
-import unittest, re
-import tables
-
-class TransactionTest(PersistTest):
- def setUpAll(self):
- tables.create()
- def tearDownAll(self):
- tables.drop()
- def tearDown(self):
- tables.delete()
-
- def testbasic(self):
- testbase.db.begin()
- tables.users.insert().execute(user_name='jack')
- tables.users.insert().execute(user_name='fred')
- testbase.db.commit()
- l = tables.users.select().execute().fetchall()
- print l
- self.assert_(len(l) == 2)
-
- def testrollback(self):
- testbase.db.begin()
- tables.users.insert().execute(user_name='jack')
- tables.users.insert().execute(user_name='fred')
- testbase.db.rollback()
- l = tables.users.select().execute().fetchall()
- print l
- self.assert_(len(l) == 0)
-
- @testbase.unsupported('sqlite')
- def testnested(self):
- """tests nested sessions. SQLite should raise an error."""
- testbase.db.begin()
- tables.users.insert().execute(user_name='jack')
- tables.users.insert().execute(user_name='fred')
- testbase.db.push_session()
- tables.users.insert().execute(user_name='ed')
- tables.users.insert().execute(user_name='wendy')
- testbase.db.pop_session()
- testbase.db.rollback()
- l = tables.users.select().execute().fetchall()
- print l
- self.assert_(len(l) == 2)
-
- def testtwo(self):
- testbase.db.begin()
- tables.users.insert().execute(user_name='jack')
- tables.users.insert().execute(user_name='fred')
- testbase.db.commit()
- testbase.db.begin()
- tables.users.insert().execute(user_name='ed')
- tables.users.insert().execute(user_name='wendy')
- testbase.db.commit()
- testbase.db.rollback()
- l = tables.users.select().execute().fetchall()
- print l
- self.assert_(len(l) == 4)
-
-if __name__ == "__main__":
- testbase.main()
import unittest
from sqlalchemy import *
import testbase
+from sqlalchemy.ext.sessioncontext import SessionContext
from tables import *
import tables
"""tests mappers that are constructed based on "entity names", which allows the same class
to have multiple primary mappers """
def setUpAll(self):
- global user1, user2, address1, address2
- db = testbase.db
- user1 = Table('user1', db,
+ global user1, user2, address1, address2, metadata, ctx
+ metadata = BoundMetaData(testbase.db)
+ ctx = SessionContext(create_session)
+
+ user1 = Table('user1', metadata,
Column('user_id', Integer, Sequence('user1_id_seq'), primary_key=True),
Column('name', String(60), nullable=False)
- ).create()
- user2 = Table('user2', db,
+ )
+ user2 = Table('user2', metadata,
Column('user_id', Integer, Sequence('user2_id_seq'), primary_key=True),
Column('name', String(60), nullable=False)
- ).create()
- address1 = Table('address1', db,
+ )
+ address1 = Table('address1', metadata,
Column('address_id', Integer, Sequence('address1_id_seq'), primary_key=True),
Column('user_id', Integer, ForeignKey(user1.c.user_id), nullable=False),
Column('email', String(100), nullable=False)
- ).create()
- address2 = Table('address2', db,
+ )
+ address2 = Table('address2', metadata,
Column('address_id', Integer, Sequence('address2_id_seq'), primary_key=True),
Column('user_id', Integer, ForeignKey(user2.c.user_id), nullable=False),
Column('email', String(100), nullable=False)
- ).create()
+ )
+ metadata.create_all()
def tearDownAll(self):
- address1.drop()
- address2.drop()
- user1.drop()
- user2.drop()
+ metadata.drop_all()
def tearDown(self):
- address1.delete().execute()
- address2.delete().execute()
- user1.delete().execute()
- user2.delete().execute()
- objectstore.clear()
clear_mappers()
+ for t in metadata.table_iterator(reverse=True):
+ t.delete().execute()
def testbasic(self):
"""tests a pair of one-to-many mapper structures, establishing that both
class User(object):pass
class Address(object):pass
- a1mapper = mapper(Address, address1, entity_name='address1')
- a2mapper = mapper(Address, address2, entity_name='address2')
+ a1mapper = mapper(Address, address1, entity_name='address1', extension=ctx.mapper_extension)
+ a2mapper = mapper(Address, address2, entity_name='address2', extension=ctx.mapper_extension)
u1mapper = mapper(User, user1, entity_name='user1', properties ={
'addresses':relation(a1mapper)
- })
+ }, extension=ctx.mapper_extension)
u2mapper =mapper(User, user2, entity_name='user2', properties={
'addresses':relation(a2mapper)
- })
+ }, extension=ctx.mapper_extension)
u1 = User(_sa_entity_name='user1')
u1.name = 'this is user 1'
a2.email='a2@foo.com'
u2.addresses.append(a2)
- objectstore.commit()
+ ctx.current.flush()
assert user1.select().execute().fetchall() == [(u1.user_id, u1.name)]
assert user2.select().execute().fetchall() == [(u2.user_id, u2.name)]
assert address1.select().execute().fetchall() == [(u1.user_id, a1.user_id, 'a1@foo.com')]
assert address2.select().execute().fetchall() == [(u2.user_id, a2.user_id, 'a2@foo.com')]
- objectstore.clear()
- u1list = u1mapper.select()
- u2list = u2mapper.select()
+ ctx.current.clear()
+ u1list = ctx.current.query(User, entity_name='user1').select()
+ u2list = ctx.current.query(User, entity_name='user2').select()
assert len(u1list) == len(u2list) == 1
assert u1list[0] is not u2list[0]
assert len(u1list[0].addresses) == len(u2list[0].addresses) == 1
class Address1(object):pass
class Address2(object):pass
- a1mapper = mapper(Address1, address1)
- a2mapper = mapper(Address2, address2)
+ a1mapper = mapper(Address1, address1, extension=ctx.mapper_extension)
+ a2mapper = mapper(Address2, address2, extension=ctx.mapper_extension)
u1mapper = mapper(User, user1, entity_name='user1', properties ={
'addresses':relation(a1mapper)
- })
+ }, extension=ctx.mapper_extension)
u2mapper =mapper(User, user2, entity_name='user2', properties={
'addresses':relation(a2mapper)
- })
+ }, extension=ctx.mapper_extension)
u1 = User(_sa_entity_name='user1')
u1.name = 'this is user 1'
a2.email='a2@foo.com'
u2.addresses.append(a2)
- objectstore.commit()
+ ctx.current.flush()
assert user1.select().execute().fetchall() == [(u1.user_id, u1.name)]
assert user2.select().execute().fetchall() == [(u2.user_id, u2.name)]
assert address1.select().execute().fetchall() == [(u1.user_id, a1.user_id, 'a1@foo.com')]
assert address2.select().execute().fetchall() == [(u2.user_id, a2.user_id, 'a2@foo.com')]
- objectstore.clear()
- u1list = u1mapper.select()
- u2list = u2mapper.select()
+ ctx.current.clear()
+ u1list = ctx.current.query(User, entity_name='user1').select()
+ u2list = ctx.current.query(User, entity_name='user2').select()
assert len(u1list) == len(u2list) == 1
assert u1list[0] is not u2list[0]
assert len(u1list[0].addresses) == len(u2list[0].addresses) == 1
class IndexTest(testbase.AssertMixin):
def setUp(self):
- self.created = []
+ global metadata
+ metadata = BoundMetaData(testbase.db)
self.echo = testbase.db.echo
self.logger = testbase.db.logger
def tearDown(self):
testbase.db.echo = self.echo
testbase.db.logger = testbase.db.engine.logger = self.logger
- if self.created:
- self.created.reverse()
- for entity in self.created:
- entity.drop()
+ metadata.drop_all()
def test_index_create(self):
- employees = Table('employees', testbase.db,
+ employees = Table('employees', metadata,
Column('id', Integer, primary_key=True),
Column('first_name', String(30)),
Column('last_name', String(30)),
Column('email_address', String(30)))
employees.create()
- self.created.append(employees)
i = Index('employee_name_index',
employees.c.last_name, employees.c.first_name)
i.create()
- self.created.append(i)
assert employees.indexes['employee_name_index'] is i
i2 = Index('employee_email_index',
employees.c.email_address, unique=True)
i2.create()
- self.created.append(i2)
assert employees.indexes['employee_email_index'] is i2
def test_index_create_camelcase(self):
"""test that mixed-case index identifiers are legal"""
- employees = Table('companyEmployees', testbase.db,
+ employees = Table('companyEmployees', metadata,
Column('id', Integer, primary_key=True),
Column('firstName', String(30)),
Column('lastName', String(30)),
Column('emailAddress', String(30)))
employees.create()
- self.created.append(employees)
i = Index('employeeNameIndex',
employees.c.lastName, employees.c.firstName)
i.create()
- self.created.append(i)
i = Index('employeeEmailIndex',
employees.c.emailAddress, unique=True)
i.create()
- self.created.append(i)
# Check that the table is useable. This is mostly for pg,
# which can be somewhat sticky with mixed-case identifiers
stream = dummy()
stream.write = capt.append
testbase.db.logger = testbase.db.engine.logger = stream
-
- events = Table('events', testbase.db,
+ events = Table('events', metadata,
Column('id', Integer, primary_key=True),
Column('name', String(30), unique=True),
Column('location', String(30), index=True),
assert len(index_names) == 4
events.create()
- self.created.append(events)
# verify that the table is functional
events.insert().execute(id=1, name='hockey finals', location='rink',
sport='hockey', announcer='some canadian',
winner='sweden')
ss = events.select().execute().fetchall()
-
+
assert capt[0].strip().startswith('CREATE TABLE events')
- assert capt[2].strip() == \
+ assert capt[3].strip() == \
'CREATE UNIQUE INDEX ux_events_name ON events (name)'
- assert capt[4].strip() == \
- 'CREATE INDEX ix_events_location ON events (location)'
assert capt[6].strip() == \
+ 'CREATE INDEX ix_events_location ON events (location)'
+ assert capt[9].strip() == \
'CREATE UNIQUE INDEX sport_announcer ON events (sport, announcer)'
- assert capt[8].strip() == \
+ assert capt[12].strip() == \
'CREATE INDEX idx_winners ON events (winner)'
if __name__ == "__main__":
-from sqlalchemy import *
import testbase
+from sqlalchemy import *
import string
import sqlalchemy.attributes as attr
import sys
class Principal( object ):
- pass
+ def __init__(self, **kwargs):
+ for key, value in kwargs.iteritems():
+ setattr(self, key, value)
class User( Principal ):
pass
global users
global groups
global user_group_map
+ global metadata
+ metadata = BoundMetaData(testbase.db)
principals = Table(
'principals',
- testbase.db,
+ metadata,
Column('principal_id', Integer, Sequence('principal_id_seq', optional=False), primary_key=True),
Column('name', String(50), nullable=False),
)
users = Table(
'prin_users',
- testbase.db,
+ metadata,
Column('principal_id', Integer, ForeignKey('principals.principal_id'), primary_key=True),
Column('password', String(50), nullable=False),
Column('email', String(50), nullable=False),
groups = Table(
'prin_groups',
- testbase.db,
+ metadata,
Column( 'principal_id', Integer, ForeignKey('principals.principal_id'), primary_key=True),
)
user_group_map = Table(
'prin_user_group_map',
- testbase.db,
+ metadata,
Column('user_id', Integer, ForeignKey( "prin_users.principal_id"), primary_key=True ),
Column('group_id', Integer, ForeignKey( "prin_groups.principal_id"), primary_key=True ),
#Column('user_id', Integer, ForeignKey( "prin_users.principal_id"), ),
)
- principals.create()
- users.create()
- groups.create()
- user_group_map.create()
+ metadata.create_all()
+
def tearDownAll(self):
- user_group_map.drop()
- groups.drop()
- users.drop()
- principals.drop()
- testbase.db.tables.clear()
+ metadata.drop_all()
+
def setUp(self):
- objectstore.clear()
clear_mappers()
def testbasic(self):
- assign_mapper( Principal, principals )
- assign_mapper(
+ mapper( Principal, principals )
+ mapper(
User,
users,
- inherits=Principal.mapper
+ inherits=Principal
)
- assign_mapper(
+ mapper(
Group,
groups,
- inherits=Principal.mapper,
- properties=dict( users = relation(User.mapper, user_group_map, lazy=True, backref="groups") )
+ inherits=Principal,
+ properties=dict( users = relation(User, secondary=user_group_map, lazy=True, backref="groups") )
)
g = Group(name="group1")
g.users.append(User(name="user1", password="pw", email="foo@bar.com", login_id="lg1"))
-
- objectstore.commit()
+ sess = create_session()
+ sess.save(g)
+ sess.flush()
# TODO: put an assertion
class InheritTest2(testbase.AssertMixin):
"""deals with inheritance and many-to-many relationships"""
def setUpAll(self):
- engine = testbase.db
- global foo, bar, foo_bar
- foo = Table('foo', engine,
+ global foo, bar, foo_bar, metadata
+ metadata = BoundMetaData(testbase.db)
+ foo = Table('foo', metadata,
Column('id', Integer, Sequence('foo_id_seq'), primary_key=True),
Column('data', String(20)),
).create()
- bar = Table('bar', engine,
+ bar = Table('bar', metadata,
Column('bid', Integer, ForeignKey('foo.id'), primary_key=True),
#Column('fid', Integer, ForeignKey('foo.id'), )
).create()
- foo_bar = Table('foo_bar', engine,
+ foo_bar = Table('foo_bar', metadata,
Column('foo_id', Integer, ForeignKey('foo.id')),
Column('bar_id', Integer, ForeignKey('bar.bid'))).create()
-
+ metadata.create_all()
def tearDownAll(self):
- foo_bar.drop()
- bar.drop()
- foo.drop()
- testbase.db.tables.clear()
+ metadata.drop_all()
def testbasic(self):
class Foo(object):
def __repr__(self):
return str(self)
- Foo.mapper = mapper(Foo, foo)
+ mapper(Foo, foo)
class Bar(Foo):
def __str__(self):
return "Bar(%s)" % self.data
- Bar.mapper = mapper(Bar, bar, inherits=Foo.mapper, properties = {
- # the old way, you needed to explicitly set up a compound
- # column like this. but now the mapper uses SyncRules to match up
- # the parent/child inherited columns
- #'id':[bar.c.bid, foo.c.id]
- })
-
- #Bar.mapper.add_property('foos', relation(Foo.mapper, foo_bar, primaryjoin=bar.c.bid==foo_bar.c.bar_id, secondaryjoin=foo_bar.c.foo_id==foo.c.id, lazy=False))
- Bar.mapper.add_property('foos', relation(Foo.mapper, foo_bar, lazy=False))
-
- b = Bar('barfoo')
- objectstore.commit()
+ mapper(Bar, bar, inherits=Foo, properties={
+ 'foos': relation(Foo, secondary=foo_bar, lazy=False)
+ })
+
+ sess = create_session()
+ b = Bar('barfoo', _sa_session=sess)
+ sess.flush()
f1 = Foo('subfoo1')
f2 = Foo('subfoo2')
b.foos.append(f1)
b.foos.append(f2)
- objectstore.commit()
- objectstore.clear()
+ sess.flush()
+ sess.clear()
- l =b.mapper.select()
+ l = sess.query(Bar).select()
print l[0]
print l[0].foos
self.assert_result(l, Bar,
class InheritTest3(testbase.AssertMixin):
"""deals with inheritance and many-to-many relationships"""
def setUpAll(self):
- engine = testbase.db
- global foo, bar, blub, bar_foo, blub_bar, blub_foo,tables
- engine.engine.echo = 'debug'
+ global foo, bar, blub, bar_foo, blub_bar, blub_foo,metadata
+ metadata = BoundMetaData(testbase.db)
# the 'data' columns are to appease SQLite which cant handle a blank INSERT
- foo = Table('foo', engine,
+ foo = Table('foo', metadata,
Column('id', Integer, Sequence('foo_seq'), primary_key=True),
Column('data', String(20)))
- bar = Table('bar', engine,
+ bar = Table('bar', metadata,
Column('id', Integer, ForeignKey('foo.id'), primary_key=True),
Column('data', String(20)))
- blub = Table('blub', engine,
+ blub = Table('blub', metadata,
Column('id', Integer, ForeignKey('bar.id'), primary_key=True),
Column('data', String(20)))
- bar_foo = Table('bar_foo', engine,
+ bar_foo = Table('bar_foo', metadata,
Column('bar_id', Integer, ForeignKey('bar.id')),
Column('foo_id', Integer, ForeignKey('foo.id')))
- blub_bar = Table('bar_blub', engine,
+ blub_bar = Table('bar_blub', metadata,
Column('blub_id', Integer, ForeignKey('blub.id')),
Column('bar_id', Integer, ForeignKey('bar.id')))
- blub_foo = Table('blub_foo', engine,
+ blub_foo = Table('blub_foo', metadata,
Column('blub_id', Integer, ForeignKey('blub.id')),
Column('foo_id', Integer, ForeignKey('foo.id')))
-
- tables = [foo, bar, blub, bar_foo, blub_bar, blub_foo]
- for table in tables:
- table.create()
+ metadata.create_all()
def tearDownAll(self):
- for table in reversed(tables):
- table.drop()
- testbase.db.tables.clear()
+ metadata.drop_all()
def tearDown(self):
- for table in reversed(tables):
+ for table in metadata.table_iterator():
table.delete().execute()
def testbasic(self):
self.data = data
def __repr__(self):
return "Foo id %d, data %s" % (self.id, self.data)
- Foo.mapper = mapper(Foo, foo)
+ mapper(Foo, foo)
class Bar(Foo):
def __repr__(self):
return "Bar id %d, data %s" % (self.id, self.data)
- Bar.mapper = mapper(Bar, bar, inherits=Foo.mapper, properties={
- #'foos' :relation(Foo.mapper, bar_foo, primaryjoin=bar.c.id==bar_foo.c.bar_id, lazy=False)
- 'foos' :relation(Foo.mapper, bar_foo, lazy=True)
+ mapper(Bar, bar, inherits=Foo, properties={
+ 'foos' :relation(Foo, secondary=bar_foo, lazy=True)
})
- b = Bar('bar #1')
+ sess = create_session()
+ b = Bar('bar #1', _sa_session=sess)
b.foos.append(Foo("foo #1"))
b.foos.append(Foo("foo #2"))
- objectstore.commit()
+ sess.flush()
compare = repr(b) + repr(b.foos)
- objectstore.clear()
- l = Bar.mapper.select()
+ sess.clear()
+ l = sess.query(Bar).select()
self.echo(repr(l[0]) + repr(l[0].foos))
self.assert_(repr(l[0]) + repr(l[0].foos) == compare)
self.data = data
def __repr__(self):
return "Foo id %d, data %s" % (self.id, self.data)
- Foo.mapper = mapper(Foo, foo)
+ mapper(Foo, foo)
class Bar(Foo):
def __repr__(self):
return "Bar id %d, data %s" % (self.id, self.data)
- Bar.mapper = mapper(Bar, bar, inherits=Foo.mapper)
+ mapper(Bar, bar, inherits=Foo)
class Blub(Bar):
def __repr__(self):
return "Blub id %d, data %s, bars %s, foos %s" % (self.id, self.data, repr([b for b in self.bars]), repr([f for f in self.foos]))
- Blub.mapper = mapper(Blub, blub, inherits=Bar.mapper, properties={
-# 'bars':relation(Bar.mapper, blub_bar, primaryjoin=blub.c.id==blub_bar.c.blub_id, lazy=False),
-# 'foos':relation(Foo.mapper, blub_foo, primaryjoin=blub.c.id==blub_foo.c.blub_id, lazy=False),
- 'bars':relation(Bar.mapper, blub_bar, lazy=False),
- 'foos':relation(Foo.mapper, blub_foo, lazy=False),
+ mapper(Blub, blub, inherits=Bar, properties={
+ 'bars':relation(Bar, secondary=blub_bar, lazy=False),
+ 'foos':relation(Foo, secondary=blub_foo, lazy=False),
})
- useobjects = True
- if (useobjects):
- f1 = Foo("foo #1")
- b1 = Bar("bar #1")
- b2 = Bar("bar #2")
- bl1 = Blub("blub #1")
- bl1.foos.append(f1)
- bl1.bars.append(b2)
- objectstore.commit()
- compare = repr(bl1)
- blubid = bl1.id
- objectstore.clear()
- else:
- foo.insert().execute(data='foo #1')
- foo.insert().execute(data='foo #2')
- bar.insert().execute(id=1, data="bar #1")
- bar.insert().execute(id=2, data="bar #2")
- blub.insert().execute(id=1, data="blub #1")
- blub_bar.insert().execute(blub_id=1, bar_id=2)
- blub_foo.insert().execute(blub_id=1, foo_id=2)
-
- l = Blub.mapper.select()
+ sess = create_session()
+ f1 = Foo("foo #1", _sa_session=sess)
+ b1 = Bar("bar #1", _sa_session=sess)
+ b2 = Bar("bar #2", _sa_session=sess)
+ bl1 = Blub("blub #1", _sa_session=sess)
+ bl1.foos.append(f1)
+ bl1.bars.append(b2)
+ sess.flush()
+ compare = repr(bl1)
+ blubid = bl1.id
+ sess.clear()
+
+ l = sess.query(Blub).select()
self.echo(l)
self.assert_(repr(l[0]) == compare)
- objectstore.clear()
- x = Blub.mapper.get_by(id=blubid) #traceback 2
+ sess.clear()
+ x = sess.query(Blub).get_by(id=blubid)
self.echo(x)
self.assert_(repr(x) == compare)
class InheritTest4(testbase.AssertMixin):
"""deals with inheritance and one-to-many relationships"""
def setUpAll(self):
- engine = testbase.db
- global foo, bar, blub, tables
- engine.engine.echo = 'debug'
+ global foo, bar, blub, metadata
+ metadata = BoundMetaData(testbase.db)
# the 'data' columns are to appease SQLite which cant handle a blank INSERT
- foo = Table('foo', engine,
+ foo = Table('foo', metadata,
Column('id', Integer, Sequence('foo_seq'), primary_key=True),
Column('data', String(20)))
- bar = Table('bar', engine,
+ bar = Table('bar', metadata,
Column('id', Integer, ForeignKey('foo.id'), primary_key=True),
Column('data', String(20)))
- blub = Table('blub', engine,
+ blub = Table('blub', metadata,
Column('id', Integer, ForeignKey('bar.id'), primary_key=True),
Column('foo_id', Integer, ForeignKey('foo.id'), nullable=False),
Column('data', String(20)))
-
- tables = [foo, bar, blub]
- for table in tables:
- table.create()
+ metadata.create_all()
def tearDownAll(self):
- for table in reversed(tables):
- table.drop()
- testbase.db.tables.clear()
+ metadata.drop_all()
def tearDown(self):
- for table in reversed(tables):
+ for table in metadata.table_iterator():
table.delete().execute()
def testbasic(self):
self.data = data
def __repr__(self):
return "Foo id %d, data %s" % (self.id, self.data)
- Foo.mapper = mapper(Foo, foo)
+ mapper(Foo, foo)
class Bar(Foo):
def __repr__(self):
return "Bar id %d, data %s" % (self.id, self.data)
- Bar.mapper = mapper(Bar, bar, inherits=Foo.mapper)
+ mapper(Bar, bar, inherits=Foo)
class Blub(Bar):
def __repr__(self):
return "Blub id %d, data %s" % (self.id, self.data)
- Blub.mapper = mapper(Blub, blub, inherits=Bar.mapper, properties={
- # bug was raised specifically based on the order of cols in the join....
-# 'parent_foo':relation(Foo.mapper, primaryjoin=blub.c.foo_id==foo.c.id)
-# 'parent_foo':relation(Foo.mapper, primaryjoin=foo.c.id==blub.c.foo_id)
- 'parent_foo':relation(Foo.mapper)
+ mapper(Blub, blub, inherits=Bar, properties={
+ 'parent_foo':relation(Foo)
})
- b1 = Blub("blub #1")
- b2 = Blub("blub #2")
- f = Foo("foo #1")
+ sess = create_session()
+ b1 = Blub("blub #1", _sa_session=sess)
+ b2 = Blub("blub #2", _sa_session=sess)
+ f = Foo("foo #1", _sa_session=sess)
b1.parent_foo = f
b2.parent_foo = f
- objectstore.commit()
+ sess.flush()
compare = repr(b1) + repr(b2) + repr(b1.parent_foo) + repr(b2.parent_foo)
- objectstore.clear()
- l = Blub.mapper.select()
+ sess.clear()
+ l = sess.query(Blub).select()
result = repr(l[0]) + repr(l[1]) + repr(l[0].parent_foo) + repr(l[1].parent_foo)
self.echo(result)
self.assert_(compare == result)
self.assert_(l[0].parent_foo.data == 'foo #1' and l[1].parent_foo.data == 'foo #1')
-class InheritTest5(testbase.AssertMixin):
+class InheritTest5(testbase.AssertMixin):
+ """testing that construction of inheriting mappers works regardless of when extra properties
+ are added to the superclass mapper"""
def setUpAll(self):
- engine = testbase.db
- global content_type, content, product
- content_type = Table('content_type', engine,
+ global content_type, content, product, metadata
+ metadata = BoundMetaData(testbase.db)
+ content_type = Table('content_type', metadata,
Column('id', Integer, primary_key=True)
)
- content = Table('content', engine,
+ content = Table('content', metadata,
Column('id', Integer, primary_key=True),
Column('content_type_id', Integer, ForeignKey('content_type.id'))
)
- product = Table('product', engine,
+ product = Table('product', metadata,
Column('id', Integer, ForeignKey('content.id'), primary_key=True)
)
def tearDownAll(self):
- testbase.db.tables.clear()
-
+ pass
def tearDown(self):
pass
# shouldnt throw exception
products = mapper(Product, product, inherits=contents)
-
def testbackref(self):
- """this test is currently known to fail in the 0.1 series of SQLAlchemy, pending the resolution of [ticket:154]"""
+ """tests adding a property to the superclass mapper"""
class ContentType(object): pass
class Content(object): pass
class Product(Content): pass
- # this test fails currently
contents = mapper(Content, content)
products = mapper(Product, product, inherits=contents)
content_types = mapper(ContentType, content_type, properties={
p = Product()
p.contenttype = ContentType()
-
class InheritTest6(testbase.AssertMixin):
"""tests eager load/lazy load of child items off inheritance mappers, tests that
LazyLoader constructs the right query condition."""
def setUpAll(self):
- global foo, bar, bar_foo
- foo = Table('foo', testbase.db, Column('id', Integer, Sequence('foo_seq'), primary_key=True),
- Column('data', String(30))).create()
- bar = Table('bar', testbase.db, Column('id', Integer, ForeignKey('foo.id'), primary_key=True),
- Column('data', String(30))).create()
-
- bar_foo = Table('bar_foo', testbase.db,
+ global foo, bar, bar_foo, metadata
+ metadata=BoundMetaData(testbase.db)
+ foo = Table('foo', metadata, Column('id', Integer, Sequence('foo_seq'), primary_key=True),
+ Column('data', String(30)))
+ bar = Table('bar', metadata, Column('id', Integer, ForeignKey('foo.id'), primary_key=True),
+ Column('data', String(30)))
+
+ bar_foo = Table('bar_foo', metadata,
Column('bar_id', Integer, ForeignKey('bar.id')),
Column('foo_id', Integer, ForeignKey('foo.id'))
- ).create()
+ )
+ metadata.create_all()
def tearDownAll(self):
- bar_foo.drop()
- bar.drop()
- foo.drop()
+ metadata.drop_all()
def testbasic(self):
class Foo(object): pass
foos = mapper(Foo, foo)
bars = mapper(Bar, bar, inherits=foos)
bars.add_property('lazy', relation(foos, bar_foo, lazy=True))
+ print bars.props['lazy'].primaryjoin, bars.props['lazy'].secondaryjoin
bars.add_property('eager', relation(foos, bar_foo, lazy=False))
foo.insert().execute(data='foo1')
bar_foo.insert().execute(bar_id=1, foo_id=3)
bar_foo.insert().execute(bar_id=2, foo_id=4)
-
- self.assert_(len(bars.selectfirst().lazy) == 1)
- self.assert_(len(bars.selectfirst().eager) == 1)
+
+ sess = create_session()
+ q = sess.query(Bar)
+ self.assert_(len(q.selectfirst().lazy) == 1)
+ self.assert_(len(q.selectfirst().eager) == 1)
if __name__ == "__main__":
testbase.main()
class LazyTest(AssertMixin):
def setUpAll(self):
- global info_table, data_table, rel_table
- engine = testbase.db
- info_table = Table('infos', engine,
+ global info_table, data_table, rel_table, metadata
+ metadata = BoundMetaData(testbase.db)
+ info_table = Table('infos', metadata,
Column('pk', Integer, primary_key=True),
Column('info', String))
- data_table = Table('data', engine,
+ data_table = Table('data', metadata,
Column('data_pk', Integer, primary_key=True),
Column('info_pk', Integer, ForeignKey(info_table.c.pk)),
Column('timeval', Integer),
Column('data_val', String))
- rel_table = Table('rels', engine,
+ rel_table = Table('rels', metadata,
Column('rel_pk', Integer, primary_key=True),
Column('info_pk', Integer, ForeignKey(info_table.c.pk)),
Column('start', Integer),
Column('finish', Integer))
-
- info_table.create()
- rel_table.create()
- data_table.create()
+ metadata.create_all()
info_table.insert().execute(
{'pk':1, 'info':'pk_1_info'},
{'pk':2, 'info':'pk_2_info'},
def tearDownAll(self):
- data_table.drop()
- rel_table.drop()
- info_table.drop()
+ metadata.drop_all()
- def setUp(self):
- clear_mappers()
-
def testone(self):
"""tests a lazy load which has multiple join conditions, including two that are against
the same column in the child table"""
class Data(object):
pass
- # Create the basic mappers, with no frills or modifications
- Information.mapper = mapper(Information, info_table)
- Data.mapper = mapper(Data, data_table)
- Relation.mapper = mapper(Relation, rel_table)
-
- Relation.mapper.add_property('datas', relation(Data.mapper,
- primaryjoin=and_(Relation.c.info_pk==Data.c.info_pk,
- Data.c.timeval >= Relation.c.start,
- Data.c.timeval <= Relation.c.finish
- ),
- foreignkey=Data.c.info_pk))
-
- Information.mapper.add_property('rels', relation(Relation.mapper))
-
- info = Information.mapper.get(1)
- assert info
- assert len(info.rels) == 2
- assert len(info.rels[0].datas) == 3
-
- def testtwo(self):
- """same thing, but reversing the order of the cols in the join"""
- class Information(object):
- pass
-
- class Relation(object):
- pass
-
- class Data(object):
- pass
-
- # Create the basic mappers, with no frills or modifications
- Information.mapper = mapper(Information, info_table)
- Data.mapper = mapper(Data, data_table)
- Relation.mapper = mapper(Relation, rel_table)
-
- Relation.mapper.add_property('datas', relation(Data.mapper,
- primaryjoin=and_(Relation.c.info_pk==Data.c.info_pk,
- Relation.c.start <= Data.c.timeval,
- Relation.c.finish >= Data.c.timeval,
- # Data.c.timeval >= Relation.c.start,
- # Data.c.timeval <= Relation.c.finish
- ),
- foreignkey=Data.c.info_pk))
-
- Information.mapper.add_property('rels', relation(Relation.mapper))
-
- info = Information.mapper.get(1)
+ session = create_session()
+
+ mapper(Data, data_table)
+ mapper(Relation, rel_table, properties={
+
+ 'datas': relation(Data,
+ primaryjoin=and_(rel_table.c.info_pk==Data.c.info_pk,
+ Data.c.timeval >= rel_table.c.start,
+ Data.c.timeval <= rel_table.c.finish),
+ foreignkey=Data.c.info_pk)
+ }
+
+ )
+ mapper(Information, info_table, properties={
+ 'rels': relation(Relation)
+ })
+
+ info = session.query(Information).get(1)
assert info
assert len(info.rels) == 2
assert len(info.rels[0].datas) == 3
-
if __name__ == "__main__":
testbase.main()
--- /dev/null
+from testbase import PersistTest, AssertMixin
+import unittest, sys, os
+from sqlalchemy import *
+import StringIO
+import testbase
+
+from tables import *
+import tables
+
+install_mods('legacy_session')
+
+
+class LegacySessionTest(AssertMixin):
+ def setUpAll(self):
+ db.echo = False
+ users.create()
+ db.echo = testbase.echo
+ def tearDownAll(self):
+ db.echo = False
+ users.drop()
+ db.echo = testbase.echo
+ def setUp(self):
+ objectstore.get_session().clear()
+ clear_mappers()
+ tables.user_data()
+ #db.echo = "debug"
+ def tearDown(self):
+ tables.delete_user_data()
+
+ def test_nested_begin_commit(self):
+ """tests that nesting objectstore transactions with multiple commits
+ affects only the outermost transaction"""
+ class User(object):pass
+ m = mapper(User, users)
+ def name_of(id):
+ return users.select(users.c.user_id == id).execute().fetchone().user_name
+ name1 = "Oliver Twist"
+ name2 = 'Mr. Bumble'
+ self.assert_(name_of(7) != name1, msg="user_name should not be %s" % name1)
+ self.assert_(name_of(8) != name2, msg="user_name should not be %s" % name2)
+ s = objectstore.get_session()
+ trans = s.begin()
+ trans2 = s.begin()
+ m.get(7).user_name = name1
+ trans3 = s.begin()
+ m.get(8).user_name = name2
+ trans3.commit()
+ s.commit() # should do nothing
+ self.assert_(name_of(7) != name1, msg="user_name should not be %s" % name1)
+ self.assert_(name_of(8) != name2, msg="user_name should not be %s" % name2)
+ trans2.commit()
+ s.commit() # should do nothing
+ self.assert_(name_of(7) != name1, msg="user_name should not be %s" % name1)
+ self.assert_(name_of(8) != name2, msg="user_name should not be %s" % name2)
+ trans.commit()
+ self.assert_(name_of(7) == name1, msg="user_name should be %s" % name1)
+ self.assert_(name_of(8) == name2, msg="user_name should be %s" % name2)
+
+ def test_nested_rollback(self):
+ """tests that nesting objectstore transactions with a rollback inside
+ affects only the outermost transaction"""
+ class User(object):pass
+ m = mapper(User, users)
+ def name_of(id):
+ return users.select(users.c.user_id == id).execute().fetchone().user_name
+ name1 = "Oliver Twist"
+ name2 = 'Mr. Bumble'
+ self.assert_(name_of(7) != name1, msg="user_name should not be %s" % name1)
+ self.assert_(name_of(8) != name2, msg="user_name should not be %s" % name2)
+ s = objectstore.get_session()
+ trans = s.begin()
+ trans2 = s.begin()
+ m.get(7).user_name = name1
+ trans3 = s.begin()
+ m.get(8).user_name = name2
+ trans3.rollback()
+ self.assert_(name_of(7) != name1, msg="user_name should not be %s" % name1)
+ self.assert_(name_of(8) != name2, msg="user_name should not be %s" % name2)
+ trans2.commit()
+ self.assert_(name_of(7) != name1, msg="user_name should not be %s" % name1)
+ self.assert_(name_of(8) != name2, msg="user_name should not be %s" % name2)
+ trans.commit()
+ self.assert_(name_of(7) != name1, msg="user_name should not be %s" % name1)
+ self.assert_(name_of(8) != name2, msg="user_name should not be %s" % name2)
+
+ def test_true_nested(self):
+ """tests creating a new Session inside a database transaction, in
+ conjunction with an engine-level nested transaction, which uses
+ a second connection in order to achieve a nested transaction that commits, inside
+ of another engine session that rolls back."""
+# testbase.db.echo='debug'
+ class User(object):
+ pass
+ testbase.db.begin()
+ try:
+ m = mapper(User, users)
+ name1 = "Oliver Twist"
+ name2 = 'Mr. Bumble'
+ m.get(7).user_name = name1
+ s = objectstore.Session(nest_on=testbase.db)
+ m.using(s).get(8).user_name = name2
+ s.commit()
+ objectstore.commit()
+ testbase.db.rollback()
+ except:
+ testbase.db.rollback()
+ raise
+ objectstore.clear()
+ self.assert_(m.get(8).user_name == name2)
+ self.assert_(m.get(7).user_name != name1)
+
+if __name__ == "__main__":
+ testbase.main()
-from sqlalchemy import *
import testbase
+from sqlalchemy import *
import string
import sqlalchemy.attributes as attr
class M2MTest(testbase.AssertMixin):
def setUpAll(self):
- db = testbase.db
+ self.install_threadlocal()
+ metadata = testbase.metadata
global place
- place = Table('place', db,
+ place = Table('place', metadata,
Column('place_id', Integer, Sequence('pid_seq', optional=True), primary_key=True),
Column('name', String(30), nullable=False),
)
global transition
- transition = Table('transition', db,
+ transition = Table('transition', metadata,
Column('transition_id', Integer, Sequence('tid_seq', optional=True), primary_key=True),
Column('name', String(30), nullable=False),
)
global place_thingy
- place_thingy = Table('place_thingy', db,
+ place_thingy = Table('place_thingy', metadata,
Column('thingy_id', Integer, Sequence('thid_seq', optional=True), primary_key=True),
Column('place_id', Integer, ForeignKey('place.place_id'), nullable=False),
Column('name', String(30), nullable=False)
# association table #1
global place_input
- place_input = Table('place_input', db,
+ place_input = Table('place_input', metadata,
Column('place_id', Integer, ForeignKey('place.place_id')),
Column('transition_id', Integer, ForeignKey('transition.transition_id')),
)
# association table #2
global place_output
- place_output = Table('place_output', db,
+ place_output = Table('place_output', metadata,
Column('place_id', Integer, ForeignKey('place.place_id')),
Column('transition_id', Integer, ForeignKey('transition.transition_id')),
)
global place_place
- place_place = Table('place_place', db,
+ place_place = Table('place_place', metadata,
Column('pl1_id', Integer, ForeignKey('place.place_id')),
Column('pl2_id', Integer, ForeignKey('place.place_id')),
)
place.drop()
transition.drop()
#testbase.db.tables.clear()
-
+ self.uninstall_threadlocal()
+
def setUp(self):
objectstore.clear()
clear_mappers()
pp = p.places
self.echo("Place " + str(p) +" places " + repr(pp))
- objectstore.delete(p1,p2,p3,p4,p5,p6,p7)
+ [objectstore.delete(p) for p in p1,p2,p3,p4,p5,p6,p7]
objectstore.flush()
def testdouble(self):
})
Transition.mapper = mapper(Transition, transition, properties = dict(
- inputs = relation(Place.mapper, place_output, lazy=False, selectalias='op_alias'),
- outputs = relation(Place.mapper, place_input, lazy=False, selectalias='ip_alias'),
+ inputs = relation(Place.mapper, place_output, lazy=False),
+ outputs = relation(Place.mapper, place_input, lazy=False),
)
)
tran.inputs.append(Place('place1'))
tran.outputs.append(Place('place2'))
tran.outputs.append(Place('place3'))
- objectstore.commit()
+ objectstore.flush()
objectstore.clear()
r = Transition.mapper.select()
p3.inputs.append(t2)
p1.outputs.append(t1)
- objectstore.commit()
+ objectstore.flush()
self.assert_result([t1], Transition, {'outputs': (Place, [{'name':'place3'}, {'name':'place1'}])})
self.assert_result([p2], Place, {'inputs': (Transition, [{'name':'transition1'},{'name':'transition2'}])})
class M2MTest2(testbase.AssertMixin):
def setUpAll(self):
- db = testbase.db
+ self.install_threadlocal()
+ metadata = testbase.metadata
global studentTbl
- studentTbl = Table('student', db, Column('name', String(20), primary_key=True))
+ studentTbl = Table('student', metadata, Column('name', String(20), primary_key=True))
global courseTbl
- courseTbl = Table('course', db, Column('name', String(20), primary_key=True))
+ courseTbl = Table('course', metadata, Column('name', String(20), primary_key=True))
global enrolTbl
- enrolTbl = Table('enrol', db,
+ enrolTbl = Table('enrol', metadata,
Column('student_id', String(20), ForeignKey('student.name'),primary_key=True),
Column('course_id', String(20), ForeignKey('course.name'), primary_key=True))
studentTbl.drop()
courseTbl.drop()
#testbase.db.tables.clear()
-
+ self.uninstall_threadlocal()
+
def setUp(self):
objectstore.clear()
clear_mappers()
c3.students.append(s1)
self.assert_(len(s1.courses) == 3)
self.assert_(len(c1.students) == 1)
- objectstore.commit()
+ objectstore.flush()
objectstore.clear()
s = Student.mapper.get_by(name='Student1')
c = Course.mapper.get_by(name='Course3')
self.assert_(len(s.courses) == 2)
class M2MTest3(testbase.AssertMixin):
- def setUpAll(self):
- e = testbase.db
- global c, c2a1, c2a2, b, a
- c = Table('c', e,
- Column('c1', Integer, primary_key = True),
- Column('c2', String(20)),
- ).create()
-
- a = Table('a', e,
- Column('a1', Integer, primary_key=True),
- Column('a2', String(20)),
- Column('c1', Integer, ForeignKey('c.c1'))
- ).create()
-
- c2a1 = Table('ctoaone', e,
- Column('c1', Integer, ForeignKey('c.c1')),
- Column('a1', Integer, ForeignKey('a.a1'))
- ).create()
- c2a2 = Table('ctoatwo', e,
- Column('c1', Integer, ForeignKey('c.c1')),
- Column('a1', Integer, ForeignKey('a.a1'))
- ).create()
-
- b = Table('b', e,
- Column('b1', Integer, primary_key=True),
- Column('a1', Integer, ForeignKey('a.a1')),
- Column('b2', Boolean)
- ).create()
-
- def tearDownAll(self):
- b.drop()
- c2a2.drop()
- c2a1.drop()
- a.drop()
- c.drop()
- #testbase.db.tables.clear()
-
- def testbasic(self):
- class C(object):pass
- class A(object):pass
- class B(object):pass
+ def setUpAll(self):
+ self.install_threadlocal()
+ metadata = testbase.metadata
+ global c, c2a1, c2a2, b, a
+ c = Table('c', metadata,
+ Column('c1', Integer, primary_key = True),
+ Column('c2', String(20)),
+ ).create()
+
+ a = Table('a', metadata,
+ Column('a1', Integer, primary_key=True),
+ Column('a2', String(20)),
+ Column('c1', Integer, ForeignKey('c.c1'))
+ ).create()
+
+ c2a1 = Table('ctoaone', metadata,
+ Column('c1', Integer, ForeignKey('c.c1')),
+ Column('a1', Integer, ForeignKey('a.a1'))
+ ).create()
+ c2a2 = Table('ctoatwo', metadata,
+ Column('c1', Integer, ForeignKey('c.c1')),
+ Column('a1', Integer, ForeignKey('a.a1'))
+ ).create()
+
+ b = Table('b', metadata,
+ Column('b1', Integer, primary_key=True),
+ Column('a1', Integer, ForeignKey('a.a1')),
+ Column('b2', Boolean)
+ ).create()
- assign_mapper(B, b)
+ def tearDownAll(self):
+ b.drop()
+ c2a2.drop()
+ c2a1.drop()
+ a.drop()
+ c.drop()
+ #testbase.db.tables.clear()
+ self.uninstall_threadlocal()
+
+ def testbasic(self):
+ class C(object):pass
+ class A(object):pass
+ class B(object):pass
- assign_mapper(A, a,
- properties = {
- 'tbs' : relation(B, primaryjoin=and_(b.c.a1==a.c.a1, b.c.b2 == True), lazy=False),
- }
- )
+ assign_mapper(B, b)
- assign_mapper(C, c,
- properties = {
- 'a1s' : relation(A, secondary=c2a1, lazy=False),
- 'a2s' : relation(A, secondary=c2a2, lazy=False)
- }
- )
+ assign_mapper(A, a,
+ properties = {
+ 'tbs' : relation(B, primaryjoin=and_(b.c.a1==a.c.a1, b.c.b2 == True), lazy=False),
+ }
+ )
- o1 = C.get(1)
+ assign_mapper(C, c,
+ properties = {
+ 'a1s' : relation(A, secondary=c2a1, lazy=False),
+ 'a2s' : relation(A, secondary=c2a2, lazy=False)
+ }
+ )
+ o1 = C.get(1)
if __name__ == "__main__":
import testbase
import unittest, sys, os
from sqlalchemy import *
-
+import sqlalchemy.exceptions as exceptions
from tables import *
import tables
tables.drop()
db.echo = testbase.echo
def tearDown(self):
- objectstore.clear()
clear_mappers()
def setUp(self):
pass
class MapperTest(MapperSuperTest):
def testget(self):
- m = mapper(User, users)
- self.assert_(m.get(19) is None)
- u = m.get(7)
- u2 = m.get(7)
+ s = create_session()
+ mapper(User, users)
+ self.assert_(s.get(User, 19) is None)
+ u = s.get(User, 7)
+ u2 = s.get(User, 7)
self.assert_(u is u2)
- objectstore.clear()
- u2 = m.get(7)
+ s.clear()
+ u2 = s.get(User, 7)
self.assert_(u is not u2)
def testrefresh(self):
- m = mapper(User, users, properties={'addresses':relation(mapper(Address, addresses))})
- u = m.get(7)
+ mapper(User, users, properties={'addresses':relation(mapper(Address, addresses))})
+ s = create_session()
+ u = s.get(User, 7)
u.user_name = 'foo'
a = Address()
+ import sqlalchemy.orm.session
+ assert sqlalchemy.orm.session.object_session(a) is None
u.addresses.append(a)
self.assert_(a in u.addresses)
- objectstore.refresh(u)
+ s.refresh(u)
# its refreshed, so not dirty
- self.assert_(u not in objectstore.get_session().uow.dirty)
+ self.assert_(u not in s.dirty)
# username is back to the DB
self.assert_(u.user_name == 'jack')
u.user_name = 'foo'
u.addresses.append(a)
# now its dirty
- self.assert_(u in objectstore.get_session().uow.dirty)
+ self.assert_(u in s.dirty)
self.assert_(u.user_name == 'foo')
self.assert_(a in u.addresses)
- objectstore.expire(u)
+ s.expire(u)
# get the attribute, it refreshes
self.assert_(u.user_name == 'jack')
def testrefresh_lazy(self):
"""tests that when a lazy loader is set as a trigger on an object's attribute (at the attribute level, not the class level), a refresh() operation doesnt fire the lazy loader or create any problems"""
- m = mapper(User, users, properties={'addresses':relation(mapper(Address, addresses))})
- m2 = m.options(lazyload('addresses'))
- u = m2.selectfirst(users.c.user_id==8)
+ s = create_session()
+ mapper(User, users, properties={'addresses':relation(mapper(Address, addresses))})
+ q2 = s.query(User).options(lazyload('addresses'))
+ u = q2.selectfirst(users.c.user_id==8)
def go():
- objectstore.refresh(u)
+ s.refresh(u)
self.assert_sql_count(db, go, 1)
- def testexpire_eager(self):
- """tests that an eager load will populate expire()'d objects"""
- m = mapper(User, users, properties={'addresses':relation(mapper(Address, addresses))})
- [u1, u2, u3] = m.select(users.c.user_id.in_(7, 8, 9))
- self.echo([repr(x.addresses) for x in [u1, u2, u3]])
- [objectstore.expire(u) for u in [u1, u2, u3]]
- m2 = m.options(eagerload('addresses'))
- l = m2.select(users.c.user_id.in_(7,8,9))
- def go():
- u1.addresses
- u2.addresses
- u3.addresses
- self.assert_sql_count(db, go, 0)
-
- def testsessionpropigation(self):
- sess = objectstore.Session()
- m = mapper(User, users, properties={'addresses':relation(mapper(Address, addresses), lazy=True)})
- u = m.using(sess).get(7)
- assert objectstore.get_session(u) is sess
- assert objectstore.get_session(u.addresses[0]) is sess
-
def testexpire(self):
- m = mapper(User, users, properties={'addresses':relation(mapper(Address, addresses), lazy=False)})
- u = m.get(7)
+ s = create_session()
+ mapper(User, users, properties={'addresses':relation(mapper(Address, addresses), lazy=False)})
+ u = s.get(User, 7)
assert(len(u.addresses) == 1)
u.user_name = 'foo'
del u.addresses[0]
- objectstore.expire(u)
+ s.expire(u)
# test plain expire
self.assert_(u.user_name =='jack')
self.assert_(len(u.addresses) == 1)
# we're changing the database here, so if this test fails in the middle,
# it'll screw up the other tests which are hardcoded to 7/'jack'
u.user_name = 'foo'
- objectstore.commit()
+ s.flush()
# change the value in the DB
users.update(users.c.user_id==7, values=dict(user_name='jack')).execute()
- objectstore.expire(u)
+ s.expire(u)
# object isnt refreshed yet, using dict to bypass trigger
self.assert_(u.__dict__['user_name'] != 'jack')
# do a select
- m.select()
+ s.query(User).select()
# test that it refreshed
self.assert_(u.__dict__['user_name'] == 'jack')
self.assert_(u.user_name =='jack')
def testrefresh2(self):
- assign_mapper(Address, addresses)
+ s = create_session()
+ mapper(Address, addresses)
- assign_mapper(User, users, properties = dict(addresses=relation(Address.mapper,private=True,lazy=False)) )
+ mapper(User, users, properties = dict(addresses=relation(Address,private=True,lazy=False)) )
u=User()
u.user_name='Justin'
a = Address()
a.address_id=17 # to work around the hardcoded IDs in this test suite....
u.addresses.append(a)
- objectstore.commit()
- objectstore.clear()
- u = User.mapper.selectfirst()
+ s.flush()
+ s.clear()
+ u = s.query(User).selectfirst()
print u.user_name
#ok so far
- u.expire() #hangs when
+ s.expire(u) #hangs when
print u.user_name #this line runs
- u.refresh() #hangs
+ s.refresh(u) #hangs
def testmagic(self):
- m = mapper(User, users, properties = {
+ mapper(User, users, properties = {
'addresses' : relation(mapper(Address, addresses))
})
- l = m.select_by(user_name='fred')
+ sess = create_session()
+ l = sess.query(User).select_by(user_name='fred')
self.assert_result(l, User, *[{'user_id':9}])
u = l[0]
- u2 = m.get_by_user_name('fred')
+ u2 = sess.query(User).get_by_user_name('fred')
self.assert_(u is u2)
- l = m.select_by(email_address='ed@bettyboop.com')
+ l = sess.query(User).select_by(email_address='ed@bettyboop.com')
self.assert_result(l, User, *[{'user_id':8}])
- l = m.select_by(User.c.user_name=='fred', addresses.c.email_address!='ed@bettyboop.com', user_id=9)
+ l = sess.query(User).select_by(User.c.user_name=='fred', addresses.c.email_address!='ed@bettyboop.com', user_id=9)
def testprops(self):
"""tests the various attributes of the properties attached to classes"""
def testload(self):
"""tests loading rows with a mapper and producing object instances"""
- m = mapper(User, users)
- l = m.select()
+ mapper(User, users)
+ l = create_session().query(User).select()
self.assert_result(l, User, *user_result)
- l = m.select(users.c.user_name.endswith('ed'))
+ l = create_session().query(User).select(users.c.user_name.endswith('ed'))
self.assert_result(l, User, *user_result[1:3])
+ def testjoinvia(self):
+ m = mapper(User, users, properties={
+ 'orders':relation(mapper(Order, orders, properties={
+ 'items':relation(mapper(Item, orderitems))
+ }))
+ })
+
+ q = create_session().query(m)
+
+ l = q.select((orderitems.c.item_name=='item 4') & q.join_via(['orders', 'items']))
+ self.assert_result(l, User, user_result[0])
+
+ l = q.select_by(item_name='item 4')
+ self.assert_result(l, User, user_result[0])
+
+ l = q.select((orderitems.c.item_name=='item 4') & q.join_to('item_name'))
+ self.assert_result(l, User, user_result[0])
+
+ l = q.select((orderitems.c.item_name=='item 4') & q.join_to('items'))
+ self.assert_result(l, User, user_result[0])
+
def testorderby(self):
# TODO: make a unit test out of these various combinations
# m = mapper(User, users, order_by=desc(users.c.user_name))
- m = mapper(User, users, order_by=None)
-# m = mapper(User, users)
+ mapper(User, users, order_by=None)
+# mapper(User, users)
-# l = m.select(order_by=[desc(users.c.user_name), asc(users.c.user_id)])
- l = m.select()
-# l = m.select(order_by=[])
-# l = m.select(order_by=None)
+# l = create_session().query(User).select(order_by=[desc(users.c.user_name), asc(users.c.user_id)])
+ l = create_session().query(User).select()
+# l = create_session().query(User).select(order_by=[])
+# l = create_session().query(User).select(order_by=None)
def testfunction(self):
"""tests mapping to a SELECT statement that has functions in it."""
s = select([users, (users.c.user_id * 2).label('concat'), func.count(addresses.c.address_id).label('count')],
users.c.user_id==addresses.c.user_id, group_by=[c for c in users.c]).alias('myselect')
- m = mapper(User, s, primarytable=users)
- print [c.key for c in m.c]
- l = m.select()
+ mapper(User, s)
+ sess = create_session()
+ l = sess.query(User).select()
for u in l:
print "User", u.user_id, u.user_name, u.concat, u.count
- #l[1].user_name='asdf'
- #objectstore.commit()
-
+ assert l[0].concat == l[0].user_id * 2 == 14
+ assert l[1].concat == l[1].user_id * 2 == 16
+
def testcount(self):
- m = mapper(User, users)
- self.assert_(m.count()==3)
- self.assert_(m.count(users.c.user_id.in_(8,9))==2)
- self.assert_(m.count_by(user_name='fred')==1)
+ mapper(User, users)
+ q = create_session().query(User)
+ self.assert_(q.count()==3)
+ self.assert_(q.count(users.c.user_id.in_(8,9))==2)
+ self.assert_(q.count_by(user_name='fred')==1)
def testmultitable(self):
usersaddresses = sql.join(users, addresses, users.c.user_id == addresses.c.user_id)
- m = mapper(User, usersaddresses, primarytable = users, primary_key=[users.c.user_id])
- l = m.select()
+ m = mapper(User, usersaddresses, primary_key=[users.c.user_id])
+ q = create_session().query(m)
+ l = q.select()
self.assert_result(l, User, *user_result[0:2])
def testoverride(self):
'user_name' : relation(mapper(Address, addresses)),
})
self.assert_(False, "should have raised ArgumentError")
- except ArgumentError, e:
+ except exceptions.ArgumentError, e:
self.assert_(True)
-
+
+ clear_mappers()
# assert that allow_column_override cancels the error
m = mapper(User, users, properties = {
'user_name' : relation(mapper(Address, addresses))
}, allow_column_override=True)
+ clear_mappers()
# assert that the column being named else where also cancels the error
m = mapper(User, users, properties = {
'user_name' : relation(mapper(Address, addresses)),
def testeageroptions(self):
"""tests that a lazy relation can be upgraded to an eager relation via the options method"""
- m = mapper(User, users, properties = dict(
+ sess = create_session()
+ mapper(User, users, properties = dict(
addresses = relation(mapper(Address, addresses), lazy = True)
))
- l = m.options(eagerload('addresses')).select()
+ l = sess.query(User).options(eagerload('addresses')).select()
def go():
self.assert_result(l, User, *user_address_result)
self.assert_sql_count(db, go, 0)
+ def testeagerdegrade(self):
+ """tests that an eager relation automatically degrades to a lazy relation if eager columns are not available"""
+ sess = create_session()
+ usermapper = mapper(User, users, properties = dict(
+ addresses = relation(mapper(Address, addresses), lazy = False)
+ ))
+
+ # first test straight eager load, 1 statement
+ def go():
+ l = usermapper.query(sess).select()
+ self.assert_result(l, User, *user_address_result)
+ self.assert_sql_count(db, go, 1)
+
+ # then select just from users. run it into instances.
+ # then assert the data, which will launch 3 more lazy loads
+ def go():
+ r = users.select().execute()
+ l = usermapper.instances(r, sess)
+ self.assert_result(l, User, *user_address_result)
+ self.assert_sql_count(db, go, 4)
+
def testlazyoptions(self):
"""tests that an eager relation can be upgraded to a lazy relation via the options method"""
- m = mapper(User, users, properties = dict(
+ sess = create_session()
+ mapper(User, users, properties = dict(
addresses = relation(mapper(Address, addresses), lazy = False)
))
- l = m.options(lazyload('addresses')).select()
+ l = sess.query(User).options(lazyload('addresses')).select()
def go():
self.assert_result(l, User, *user_address_result)
self.assert_sql_count(db, go, 3)
def testdeepoptions(self):
- m = mapper(User, users,
+ mapper(User, users,
properties = {
'orders': relation(mapper(Order, orders, properties = {
'items' : relation(mapper(Item, orderitems, properties = {
}))
})
- m2 = m.options(eagerload('orders.items.keywords'))
- u = m.select()
+ sess = create_session()
+ q2 = sess.query(User).options(eagerload('orders.items.keywords'))
+ u = sess.query(User).select()
def go():
print u[0].orders[1].items[0].keywords[1]
self.assert_sql_count(db, go, 3)
- objectstore.clear()
- u = m2.select()
+ sess.clear()
+ u = q2.select()
self.assert_sql_count(db, go, 2)
-class PropertyTest(MapperSuperTest):
- def testbasic(self):
- """tests that you can create mappers inline with class definitions"""
- class _Address(object):
- pass
- assign_mapper(_Address, addresses)
-
- class _User(object):
- pass
- assign_mapper(_User, users, properties = dict(
- addresses = relation(_Address.mapper, lazy = False)
- ), is_primary = True)
-
- l = _User.mapper.select(_User.c.user_name == 'fred')
- self.echo(repr(l))
-
+class InheritanceTest(MapperSuperTest):
def testinherits(self):
class _Order(object):
pass
- assign_mapper(_Order, orders)
+ ordermapper = mapper(_Order, orders)
class _User(object):
pass
- assign_mapper(_User, users, properties = dict(
- orders = relation(_Order.mapper, lazy = False)
+ usermapper = mapper(_User, users, properties = dict(
+ orders = relation(ordermapper, lazy = False)
))
class AddressUser(_User):
pass
- assign_mapper(AddressUser, addresses, inherits = _User.mapper)
-
- l = AddressUser.mapper.select()
+ mapper(AddressUser, addresses, inherits = usermapper)
+
+ sess = create_session()
+ q = sess.query(AddressUser)
+ l = q.select()
jack = l[0]
self.assert_(jack.user_name=='jack')
jack.email_address = 'jack@gmail.com'
- objectstore.commit()
- objectstore.clear()
- au = AddressUser.mapper.get_by(user_name='jack')
+ sess.flush()
+ sess.clear()
+ au = q.get_by(user_name='jack')
self.assert_(au.email_address == 'jack@gmail.com')
def testinherits2(self):
pass
class AddressUser(_Address):
pass
- assign_mapper(_Order, orders)
- assign_mapper(_Address, addresses)
- assign_mapper(AddressUser, users, inherits = _Address.mapper,
+ ordermapper = mapper(_Order, orders)
+ addressmapper = mapper(_Address, addresses)
+ usermapper = mapper(AddressUser, users, inherits = addressmapper,
properties = {
- 'orders' : relation(_Order.mapper, lazy=False)
+ 'orders' : relation(ordermapper, lazy=False)
})
- l = AddressUser.mapper.select()
+ sess = create_session()
+ l = sess.query(usermapper).select()
jack = l[0]
self.assert_(jack.user_name=='jack')
jack.email_address = 'jack@gmail.com'
- objectstore.commit()
- objectstore.clear()
- au = AddressUser.mapper.get_by(user_name='jack')
+ sess.flush()
+ sess.clear()
+ au = sess.query(usermapper).get_by(user_name='jack')
self.assert_(au.email_address == 'jack@gmail.com')
o = Order()
self.assert_(o.description is None)
+ q = create_session().query(m)
def go():
- l = m.select()
+ l = q.select()
o2 = l[2]
print o2.description
+ orderby = str(orders.default_order_by()[0].compile(engine=db))
self.assert_sql(db, go, [
- ("SELECT orders.order_id AS orders_order_id, orders.user_id AS orders_user_id, orders.isopen AS orders_isopen FROM orders ORDER BY orders.%s" % orders.default_order_by()[0].key, {}),
+ ("SELECT orders.order_id AS orders_order_id, orders.user_id AS orders_user_id, orders.isopen AS orders_isopen FROM orders ORDER BY %s" % orderby, {}),
("SELECT orders.description AS orders_description FROM orders WHERE orders.order_id = :orders_order_id", {'orders_order_id':3})
])
'description':deferred(orders.c.description)
})
- l = m.select()
+ sess = create_session()
+ q = sess.query(m)
+ l = q.select()
o2 = l[2]
o2.isopen = 1
- objectstore.commit()
+ sess.flush()
def testgroup(self):
"""tests deferred load with a group"""
'description':deferred(orders.c.description, group='primary'),
'opened':deferred(orders.c.isopen, group='primary')
})
-
+ q = create_session().query(m)
def go():
- l = m.select()
+ l = q.select()
o2 = l[2]
print o2.opened, o2.description, o2.userident
+
+ orderby = str(orders.default_order_by()[0].compile(db))
self.assert_sql(db, go, [
- ("SELECT orders.order_id AS orders_order_id FROM orders ORDER BY orders.%s" % orders.default_order_by()[0].key, {}),
+ ("SELECT orders.order_id AS orders_order_id FROM orders ORDER BY %s" % orderby, {}),
("SELECT orders.user_id AS orders_user_id, orders.description AS orders_description, orders.isopen AS orders_isopen FROM orders WHERE orders.order_id = :orders_order_id", {'orders_order_id':3})
])
def testoptions(self):
"""tests using options on a mapper to create deferred and undeferred columns"""
m = mapper(Order, orders)
- m2 = m.options(defer('user_id'))
+ sess = create_session()
+ q = sess.query(m)
+ q2 = q.options(defer('user_id'))
def go():
- l = m2.select()
+ l = q2.select()
print l[2].user_id
+
+ orderby = str(orders.default_order_by()[0].compile(db))
self.assert_sql(db, go, [
- ("SELECT orders.order_id AS orders_order_id, orders.description AS orders_description, orders.isopen AS orders_isopen FROM orders ORDER BY orders.%s" % orders.default_order_by()[0].key, {}),
+ ("SELECT orders.order_id AS orders_order_id, orders.description AS orders_description, orders.isopen AS orders_isopen FROM orders ORDER BY %s" % orderby, {}),
("SELECT orders.user_id AS orders_user_id FROM orders WHERE orders.order_id = :orders_order_id", {'orders_order_id':3})
])
- objectstore.clear()
- m3 = m2.options(undefer('user_id'))
+ sess.clear()
+ q3 = q2.options(undefer('user_id'))
def go():
- l = m3.select()
+ l = q3.select()
print l[3].user_id
self.assert_sql(db, go, [
- ("SELECT orders.order_id AS orders_order_id, orders.user_id AS orders_user_id, orders.description AS orders_description, orders.isopen AS orders_isopen FROM orders ORDER BY orders.%s" % orders.default_order_by()[0].key, {}),
+ ("SELECT orders.order_id AS orders_order_id, orders.user_id AS orders_user_id, orders.description AS orders_description, orders.isopen AS orders_isopen FROM orders ORDER BY %s" % orderby, {}),
])
+
def testdeepoptions(self):
m = mapper(User, users, properties={
'orders':relation(mapper(Order, orders, properties={
}))
}))
})
- l = m.select()
+ sess = create_session()
+ q = sess.query(m)
+ l = q.select()
item = l[0].orders[1].items[1]
def go():
print item.item_name
self.assert_sql_count(db, go, 1)
self.assert_(item.item_name == 'item 4')
- objectstore.clear()
- m2 = m.options(undefer('orders.items.item_name'))
- l = m2.select()
+ sess.clear()
+ q2 = q.options(undefer('orders.items.item_name'))
+ l = q2.select()
item = l[0].orders[1].items[1]
def go():
print item.item_name
m = mapper(User, users, properties = dict(
addresses = relation(mapper(Address, addresses), lazy = True)
))
- l = m.select(users.c.user_id == 7)
+ q = create_session().query(m)
+ l = q.select(users.c.user_id == 7)
self.assert_result(l, User,
{'user_id' : 7, 'addresses' : (Address, [{'address_id' : 1}])},
)
m = mapper(User, users, properties = dict(
addresses = relation(m, lazy = True, order_by=addresses.c.email_address),
))
- l = m.select()
+ q = create_session().query(m)
+ l = q.select()
self.assert_result(l, User,
{'user_id' : 7, 'addresses' : (Address, [{'email_address' : 'jack@bean.com'}])},
{'user_id' : 9, 'addresses' : (Address, [])}
)
+ def testorderby_select(self):
+ """tests that a regular mapper select on a single table can order by a relation to a second table"""
+ m = mapper(Address, addresses)
+
+ m = mapper(User, users, properties = dict(
+ addresses = relation(m, lazy = True),
+ ))
+ q = create_session().query(m)
+ l = q.select(users.c.user_id==addresses.c.user_id, order_by=addresses.c.email_address)
+
+ self.assert_result(l, User,
+ {'user_id' : 8, 'addresses' : (Address, [{'email_address':'ed@wood.com'}, {'email_address':'ed@bettyboop.com'}, {'email_address':'ed@lala.com'}, ])},
+ {'user_id' : 7, 'addresses' : (Address, [{'email_address' : 'jack@bean.com'}])},
+ )
+
def testorderby_desc(self):
m = mapper(Address, addresses)
m = mapper(User, users, properties = dict(
addresses = relation(m, lazy = True, order_by=[desc(addresses.c.email_address)]),
))
- l = m.select()
+ q = create_session().query(m)
+ l = q.select()
self.assert_result(l, User,
{'user_id' : 7, 'addresses' : (Address, [{'email_address' : 'jack@bean.com'}])},
addresses = relation(mapper(Address, addresses), lazy = True),
orders = relation(ordermapper, primaryjoin = users.c.user_id==orders.c.user_id, lazy = True),
))
- l = m.select(limit=2, offset=1)
+ sess= create_session()
+ q = sess.query(m)
+ l = q.select(limit=2, offset=1)
self.assert_result(l, User, *user_all_result[1:3])
# use a union all to get a lot of rows to join against
u2 = users.alias('u2')
s = union_all(u2.select(use_labels=True), u2.select(use_labels=True), u2.select(use_labels=True)).alias('u')
print [key for key in s.c.keys()]
- l = m.select(s.c.u2_user_id==User.c.user_id, distinct=True)
+ l = q.select(s.c.u2_user_id==User.c.user_id, distinct=True)
self.assert_result(l, User, *user_all_result)
- objectstore.clear()
+ sess.clear()
m = mapper(Item, orderitems, is_primary=True, properties = dict(
keywords = relation(mapper(Keyword, keywords), itemkeywords, lazy = True),
))
- l = m.select((Item.c.item_name=='item 2') | (Item.c.item_name=='item 5') | (Item.c.item_name=='item 3'), order_by=[Item.c.item_id], limit=2)
+
+ l = sess.query(m).select((Item.c.item_name=='item 2') | (Item.c.item_name=='item 5') | (Item.c.item_name=='item 3'), order_by=[Item.c.item_id], limit=2)
self.assert_result(l, Item, *[item_keyword_result[1], item_keyword_result[2]])
def testonetoone(self):
m = mapper(User, users, properties = dict(
address = relation(mapper(Address, addresses), lazy = True, uselist = False)
))
- l = m.select(users.c.user_id == 7)
- self.echo(repr(l))
- self.echo(repr(l[0].address))
+ q = create_session().query(m)
+ l = q.select(users.c.user_id == 7)
+ self.assert_result(l, User, {'user_id':7, 'address' : (Address, {'address_id':1})})
def testbackwardsonetoone(self):
m = mapper(Address, addresses, properties = dict(
- user = relation(mapper(User, users, properties = {'id':users.c.user_id}), lazy = True)
+ user = relation(mapper(User, users), lazy = True)
))
- l = m.select(addresses.c.address_id == 1)
+ q = create_session().query(m)
+ l = q.select(addresses.c.address_id == 1)
self.echo(repr(l))
print repr(l[0].__dict__)
self.echo(repr(l[0].user))
closedorders = alias(orders, 'closedorders')
m = mapper(User, users, properties = dict(
addresses = relation(mapper(Address, addresses), lazy = False),
- open_orders = relation(mapper(Order, openorders), primaryjoin = and_(openorders.c.isopen == 1, users.c.user_id==openorders.c.user_id), lazy = True),
- closed_orders = relation(mapper(Order, closedorders), primaryjoin = and_(closedorders.c.isopen == 0, users.c.user_id==closedorders.c.user_id), lazy = True)
+ open_orders = relation(mapper(Order, openorders, entity_name='open'), primaryjoin = and_(openorders.c.isopen == 1, users.c.user_id==openorders.c.user_id), lazy = True),
+ closed_orders = relation(mapper(Order, closedorders,entity_name='closed'), primaryjoin = and_(closedorders.c.isopen == 0, users.c.user_id==closedorders.c.user_id), lazy = True)
))
- l = m.select()
+ q = create_session().query(m)
+ l = q.select()
self.assert_result(l, User,
{'user_id' : 7,
'addresses' : (Address, [{'address_id' : 1}]),
def testmanytomany(self):
"""tests a many-to-many lazy load"""
- assign_mapper(Item, orderitems, properties = dict(
+ mapper(Item, orderitems, properties = dict(
keywords = relation(mapper(Keyword, keywords), itemkeywords, lazy = True),
))
- l = Item.mapper.select()
+ q = create_session().query(Item)
+ l = q.select()
self.assert_result(l, Item,
{'item_id' : 1, 'keywords' : (Keyword, [{'keyword_id' : 2}, {'keyword_id' : 4}, {'keyword_id' : 6}])},
{'item_id' : 2, 'keywords' : (Keyword, [{'keyword_id' : 2}, {'keyword_id' : 5}, {'keyword_id' : 7}])},
{'item_id' : 4, 'keywords' : (Keyword, [])},
{'item_id' : 5, 'keywords' : (Keyword, [])}
)
- l = Item.mapper.select(and_(keywords.c.name == 'red', keywords.c.keyword_id == itemkeywords.c.keyword_id, Item.c.item_id==itemkeywords.c.item_id))
+ l = q.select(and_(keywords.c.name == 'red', keywords.c.keyword_id == itemkeywords.c.keyword_id, Item.c.item_id==itemkeywords.c.item_id))
self.assert_result(l, Item,
{'item_id' : 1, 'keywords' : (Keyword, [{'keyword_id' : 2}, {'keyword_id' : 4}, {'keyword_id' : 6}])},
{'item_id' : 2, 'keywords' : (Keyword, [{'keyword_id' : 2}, {'keyword_id' : 5}, {'keyword_id' : 7}])},
class EagerTest(MapperSuperTest):
def testbasic(self):
"""tests a basic one-to-many eager load"""
-
m = mapper(Address, addresses)
m = mapper(User, users, properties = dict(
addresses = relation(m, lazy = False),
))
- l = m.select()
+ q = create_session().query(m)
+ l = q.select()
self.assert_result(l, User, *user_address_result)
def testorderby(self):
m = mapper(User, users, properties = dict(
addresses = relation(m, lazy = False, order_by=addresses.c.email_address),
))
- l = m.select()
+ q = create_session().query(m)
+ l = q.select()
self.assert_result(l, User,
{'user_id' : 7, 'addresses' : (Address, [{'email_address' : 'jack@bean.com'}])},
{'user_id' : 8, 'addresses' : (Address, [{'email_address':'ed@bettyboop.com'}, {'email_address':'ed@lala.com'}, {'email_address':'ed@wood.com'}])},
m = mapper(User, users, properties = dict(
addresses = relation(m, lazy = False, order_by=[desc(addresses.c.email_address)]),
))
- l = m.select()
+ q = create_session().query(m)
+ l = q.select()
self.assert_result(l, User,
{'user_id' : 7, 'addresses' : (Address, [{'email_address' : 'jack@bean.com'}])},
addresses = relation(mapper(Address, addresses), lazy = False),
orders = relation(ordermapper, primaryjoin = users.c.user_id==orders.c.user_id, lazy = False),
))
- l = m.select(limit=2, offset=1)
+ sess = create_session()
+ q = sess.query(m)
+
+ l = q.select(limit=2, offset=1)
self.assert_result(l, User, *user_all_result[1:3])
# this is an involved 3x union of the users table to get a lot of rows.
# then see if the "distinct" works its way out. you actually get the same
# result with or without the distinct, just via less or more rows.
u2 = users.alias('u2')
s = union_all(u2.select(use_labels=True), u2.select(use_labels=True), u2.select(use_labels=True)).alias('u')
- l = m.select(s.c.u2_user_id==User.c.user_id, distinct=True)
+ l = q.select(s.c.u2_user_id==User.c.user_id, distinct=True)
self.assert_result(l, User, *user_all_result)
- objectstore.clear()
+ sess.clear()
m = mapper(Item, orderitems, is_primary=True, properties = dict(
keywords = relation(mapper(Keyword, keywords), itemkeywords, lazy = False, order_by=[keywords.c.keyword_id]),
))
- l = m.select((Item.c.item_name=='item 2') | (Item.c.item_name=='item 5') | (Item.c.item_name=='item 3'), order_by=[Item.c.item_id], limit=2)
+ q = sess.query(m)
+ l = q.select((Item.c.item_name=='item 2') | (Item.c.item_name=='item 5') | (Item.c.item_name=='item 3'), order_by=[Item.c.item_id], limit=2)
self.assert_result(l, Item, *[item_keyword_result[1], item_keyword_result[2]])
m = mapper(User, users, properties = dict(
address = relation(mapper(Address, addresses), lazy = False, uselist = False)
))
- l = m.select(users.c.user_id == 7)
+ q = create_session().query(m)
+ l = q.select(users.c.user_id == 7)
self.assert_result(l, User,
{'user_id' : 7, 'address' : (Address, {'address_id' : 1, 'email_address': 'jack@bean.com'})},
)
user = relation(mapper(User, users), lazy = False)
))
self.echo(repr(m.props['user'].uselist))
- l = m.select(addresses.c.address_id == 1)
+ q = create_session().query(m)
+ l = q.select(addresses.c.address_id == 1)
self.assert_result(l, Address,
{'address_id' : 1, 'email_address' : 'jack@bean.com',
'user' : (User, {'user_id' : 7, 'user_name' : 'jack'})
m = mapper(User, users, properties = dict(
addresses = relation(mapper(Address, addresses), primaryjoin = users.c.user_id==addresses.c.user_id, lazy = False)
))
- l = m.select(and_(addresses.c.email_address == 'ed@lala.com', addresses.c.user_id==users.c.user_id))
+ q = create_session().query(m)
+ l = q.select(and_(addresses.c.email_address == 'ed@lala.com', addresses.c.user_id==users.c.user_id))
self.assert_result(l, User,
{'user_id' : 8, 'addresses' : (Address, [{'address_id' : 2, 'email_address':'ed@wood.com'}, {'address_id':3, 'email_address':'ed@bettyboop.com'}, {'address_id':4, 'email_address':'ed@lala.com'}])},
)
def testcompile(self):
"""tests deferred operation of a pre-compiled mapper statement"""
+ session = create_session()
m = mapper(User, users, properties = dict(
addresses = relation(mapper(Address, addresses), lazy = False)
))
c = s.compile()
self.echo("\n" + str(c) + repr(c.get_params()))
- l = m.instances(s.execute(emailad = 'jack@bean.com'))
+ l = m.instances(s.execute(emailad = 'jack@bean.com'), session)
self.echo(repr(l))
def testmulti(self):
addresses = relation(mapper(Address, addresses), primaryjoin = users.c.user_id==addresses.c.user_id, lazy = False),
orders = relation(mapper(Order, orders), lazy = False),
))
- l = m.select()
+ q = create_session().query(m)
+ l = q.select()
self.assert_result(l, User,
{'user_id' : 7,
'addresses' : (Address, [{'address_id' : 1}]),
ordermapper = mapper(Order, orders)
m = mapper(User, users, properties = dict(
addresses = relation(mapper(Address, addresses), lazy = False),
- open_orders = relation(mapper(Order, openorders), primaryjoin = and_(openorders.c.isopen == 1, users.c.user_id==openorders.c.user_id), lazy = False),
- closed_orders = relation(mapper(Order, closedorders), primaryjoin = and_(closedorders.c.isopen == 0, users.c.user_id==closedorders.c.user_id), lazy = False)
+ open_orders = relation(mapper(Order, openorders, non_primary=True), primaryjoin = and_(openorders.c.isopen == 1, users.c.user_id==openorders.c.user_id), lazy = False),
+ closed_orders = relation(mapper(Order, closedorders, non_primary=True), primaryjoin = and_(closedorders.c.isopen == 0, users.c.user_id==closedorders.c.user_id), lazy = False)
))
- l = m.select()
+ q = create_session().query(m)
+ l = q.select()
self.assert_result(l, User,
{'user_id' : 7,
'addresses' : (Address, [{'address_id' : 1}]),
addresses = relation(mapper(Address, addresses), lazy = False),
orders = relation(ordermapper, primaryjoin = users.c.user_id==orders.c.user_id, lazy = False),
))
- l = m.select()
+ q = create_session().query(m)
+ l = q.select()
self.assert_result(l, User, *user_all_result)
def testmanytomany(self):
m = mapper(Item, items, properties = dict(
keywords = relation(mapper(Keyword, keywords), itemkeywords, lazy=False, order_by=[keywords.c.keyword_id]),
))
- l = m.select()
+ q = create_session().query(m)
+ l = q.select()
self.assert_result(l, Item, *item_keyword_result)
-# l = m.select()
- l = m.select(and_(keywords.c.name == 'red', keywords.c.keyword_id == itemkeywords.c.keyword_id, items.c.item_id==itemkeywords.c.item_id))
+ l = q.select(and_(keywords.c.name == 'red', keywords.c.keyword_id == itemkeywords.c.keyword_id, items.c.item_id==itemkeywords.c.item_id))
self.assert_result(l, Item,
{'item_id' : 1, 'keywords' : (Keyword, [{'keyword_id' : 2}, {'keyword_id' : 4}, {'keyword_id' : 6}])},
{'item_id' : 2, 'keywords' : (Keyword, [{'keyword_id' : 2}, {'keyword_id' : 5}, {'keyword_id' : 7}])},
)
+ def testmanytomanyoptions(self):
+ items = orderitems
+
+ m = mapper(Item, items, properties = dict(
+ keywords = relation(mapper(Keyword, keywords), itemkeywords, lazy=True, order_by=[keywords.c.keyword_id]),
+ ))
+ m2 = m.options(eagerload('keywords'))
+ q = create_session().query(m2)
+ def go():
+ l = q.select()
+ self.assert_result(l, Item, *item_keyword_result)
+ self.assert_sql_count(db, go, 1)
+
+ def go():
+ l = q.select(and_(keywords.c.name == 'red', keywords.c.keyword_id == itemkeywords.c.keyword_id, items.c.item_id==itemkeywords.c.item_id))
+ self.assert_result(l, Item,
+ {'item_id' : 1, 'keywords' : (Keyword, [{'keyword_id' : 2}, {'keyword_id' : 4}, {'keyword_id' : 6}])},
+ {'item_id' : 2, 'keywords' : (Keyword, [{'keyword_id' : 2}, {'keyword_id' : 5}, {'keyword_id' : 7}])},
+ )
+ self.assert_sql_count(db, go, 1)
+
def testoneandmany(self):
"""tests eager load for a parent object with a child object that
contains a many-to-many relationship to a third object."""
m = mapper(Order, orders, properties = dict(
items = relation(m, lazy = False)
))
- l = m.select("orders.order_id in (1,2,3)")
+ q = create_session().query(m)
+ l = q.select("orders.order_id in (1,2,3)")
self.assert_result(l, Order,
{'order_id' : 1, 'items': (Item, [])},
{'order_id' : 2, 'items': (Item, [
if manage_attributes:
attr_manager.register_attribute(User, 'id', uselist=False)
attr_manager.register_attribute(User, 'name', uselist=False)
- attr_manager.register_attribute(User, 'addresses', uselist=True)
+ attr_manager.register_attribute(User, 'addresses', uselist=True, trackparent=True)
attr_manager.register_attribute(Address, 'email', uselist=False)
now = time.time()
a.email = 'foo@bar.com'
u.addresses.append(a)
# gc.collect()
- print len(managed_attributes)
+# print len(managed_attributes)
# managed_attributes.clear()
total = time.time() - now
print "Total time", total
clear_mappers()
for x in range(1,NUM/500+1):
l = []
- for y in range(x*500-499, x*500 + 1):
+ for y in range(x*500-500, x*500):
l.append({'item_id':y, 'value':'this is item #%d' % y})
items.insert().execute(*l)
for x in range (1,NUM/100):
# this is not needed with cpython which clears non-circular refs immediately
#gc.collect()
- l = m.select(items.c.item_id.between(x*100 - 99, x*100 ))
+ l = m.select(items.c.item_id.between(x*100 - 100, x*100 - 1))
assert len(l) == 100
print "loaded ", len(l), " items "
# modifying each object will insure that the objects get placed in the "dirty" list
a.value = 'changed...'
assert len(objectstore.get_session().dirty) == len(l)
assert len(objectstore.get_session().identity_map) == len(l)
- #assert len(attributes.managed_attributes) == len(l)
+ assert len(attributes.managed_attributes) == len(l)
print len(objectstore.get_session().dirty)
print len(objectstore.get_session().identity_map)
- objectstore.expunge(*l)
+ #objectstore.expunge(*l)
if __name__ == "__main__":
testbase.main()
from sqlalchemy import *
import StringIO
import testbase
-
+from sqlalchemy.orm.mapper import global_extensions
+from sqlalchemy.ext.sessioncontext import SessionContext
+import sqlalchemy.ext.assignmapper as assignmapper
from tables import *
import tables
-class HistoryTest(AssertMixin):
+class SessionTest(AssertMixin):
def setUpAll(self):
+ global ctx, assign_mapper
+ ctx = SessionContext(create_session)
+ def assign_mapper(*args, **kwargs):
+ return assignmapper.assign_mapper(ctx, *args, **kwargs)
+ global_extensions.append(ctx.mapper_extension)
+ def tearDownAll(self):
+ global_extensions.remove(ctx.mapper_extension)
+ def tearDown(self):
+ ctx.current.clear()
+ clear_mappers()
+
+class HistoryTest(SessionTest):
+ def setUpAll(self):
+ SessionTest.setUpAll(self)
db.echo = False
users.create()
addresses.create()
addresses.drop()
users.drop()
db.echo = testbase.echo
- def setUp(self):
- objectstore.clear()
- clear_mappers()
+ SessionTest.tearDownAll(self)
def testattr(self):
"""tests the rolling back of scalar and list attributes. this kind of thing
self.assert_result([u], data[0], *data[1:])
self.echo(repr(u.addresses))
- objectstore.uow().rollback_object(u)
+ ctx.current.uow.rollback_object(u)
data = [User,
{'user_name' : None,
'addresses' : (Address, [])
self.assert_result([u], data[0], *data[1:])
def testbackref(self):
+ s = create_session()
class User(object):pass
class Address(object):pass
am = mapper(Address, addresses)
addresses = relation(am, backref='user', lazy=False))
)
- u = User()
- a = Address()
+ u = User(_sa_session=s)
+ a = Address(_sa_session=s)
a.user = u
#print repr(a.__class__._attribute_manager.get_history(a, 'user').added_items())
#print repr(u.addresses.added_items())
self.assert_(u.addresses == [a])
- objectstore.commit()
+ s.flush()
- objectstore.clear()
- u = m.select()[0]
+ s.clear()
+ u = s.query(m).select()[0]
print u.addresses[0].user
-class SessionTest(AssertMixin):
- def setUpAll(self):
- db.echo = False
- users.create()
- db.echo = testbase.echo
- def tearDownAll(self):
- db.echo = False
- users.drop()
- db.echo = testbase.echo
- def setUp(self):
- objectstore.get_session().clear()
- clear_mappers()
- tables.user_data()
- #db.echo = "debug"
- def tearDown(self):
- tables.delete_user_data()
-
- def test_nested_begin_commit(self):
- """tests that nesting objectstore transactions with multiple commits
- affects only the outermost transaction"""
- class User(object):pass
- m = mapper(User, users)
- def name_of(id):
- return users.select(users.c.user_id == id).execute().fetchone().user_name
- name1 = "Oliver Twist"
- name2 = 'Mr. Bumble'
- self.assert_(name_of(7) != name1, msg="user_name should not be %s" % name1)
- self.assert_(name_of(8) != name2, msg="user_name should not be %s" % name2)
- s = objectstore.get_session()
- trans = s.begin()
- trans2 = s.begin()
- m.get(7).user_name = name1
- trans3 = s.begin()
- m.get(8).user_name = name2
- trans3.commit()
- s.commit() # should do nothing
- self.assert_(name_of(7) != name1, msg="user_name should not be %s" % name1)
- self.assert_(name_of(8) != name2, msg="user_name should not be %s" % name2)
- trans2.commit()
- s.commit() # should do nothing
- self.assert_(name_of(7) != name1, msg="user_name should not be %s" % name1)
- self.assert_(name_of(8) != name2, msg="user_name should not be %s" % name2)
- trans.commit()
- self.assert_(name_of(7) == name1, msg="user_name should be %s" % name1)
- self.assert_(name_of(8) == name2, msg="user_name should be %s" % name2)
-
- def test_nested_rollback(self):
- """tests that nesting objectstore transactions with a rollback inside
- affects only the outermost transaction"""
- class User(object):pass
- m = mapper(User, users)
- def name_of(id):
- return users.select(users.c.user_id == id).execute().fetchone().user_name
- name1 = "Oliver Twist"
- name2 = 'Mr. Bumble'
- self.assert_(name_of(7) != name1, msg="user_name should not be %s" % name1)
- self.assert_(name_of(8) != name2, msg="user_name should not be %s" % name2)
- s = objectstore.get_session()
- trans = s.begin()
- trans2 = s.begin()
- m.get(7).user_name = name1
- trans3 = s.begin()
- m.get(8).user_name = name2
- trans3.rollback()
- self.assert_(name_of(7) != name1, msg="user_name should not be %s" % name1)
- self.assert_(name_of(8) != name2, msg="user_name should not be %s" % name2)
- trans2.commit()
- self.assert_(name_of(7) != name1, msg="user_name should not be %s" % name1)
- self.assert_(name_of(8) != name2, msg="user_name should not be %s" % name2)
- trans.commit()
- self.assert_(name_of(7) != name1, msg="user_name should not be %s" % name1)
- self.assert_(name_of(8) != name2, msg="user_name should not be %s" % name2)
-
- @testbase.unsupported('sqlite')
- def test_true_nested(self):
- """tests creating a new Session inside a database transaction, in
- conjunction with an engine-level nested transaction, which uses
- a second connection in order to achieve a nested transaction that commits, inside
- of another engine session that rolls back."""
-# testbase.db.echo='debug'
- class User(object):
- pass
- testbase.db.begin()
- try:
- m = mapper(User, users)
- name1 = "Oliver Twist"
- name2 = 'Mr. Bumble'
- m.get(7).user_name = name1
- s = objectstore.Session(nest_on=testbase.db)
- m.using(s).get(8).user_name = name2
- s.commit()
- objectstore.commit()
- testbase.db.rollback()
- except:
- testbase.db.rollback()
- raise
- objectstore.clear()
- self.assert_(m.get(8).user_name == name2)
- self.assert_(m.get(7).user_name != name1)
-class VersioningTest(AssertMixin):
+class VersioningTest(SessionTest):
def setUpAll(self):
- objectstore.clear()
+ SessionTest.setUpAll(self)
+ ctx.current.clear()
global version_table
version_table = Table('version_test', db,
- Column('id', Integer, primary_key=True),
+ Column('id', Integer, Sequence('version_test_seq'), primary_key=True ),
Column('version_id', Integer, nullable=False),
Column('value', String(40), nullable=False)
).create()
def tearDownAll(self):
version_table.drop()
+ SessionTest.tearDownAll(self)
def tearDown(self):
version_table.delete().execute()
- objectstore.clear()
- clear_mappers()
+ SessionTest.tearDown(self)
@testbase.unsupported('mysql')
def testbasic(self):
+ s = create_session()
class Foo(object):pass
assign_mapper(Foo, version_table, version_id_col=version_table.c.version_id)
- f1 =Foo(value='f1')
- f2 = Foo(value='f2')
- objectstore.commit()
+ f1 =Foo(value='f1', _sa_session=s)
+ f2 = Foo(value='f2', _sa_session=s)
+ s.flush()
f1.value='f1rev2'
- objectstore.commit()
- s = objectstore.Session()
- f1_s = Foo.mapper.using(s).get(f1.id)
+ s.flush()
+ s2 = create_session()
+ f1_s = Foo.mapper.using(s2).get(f1.id)
f1_s.value='f1rev3'
- s.commit()
+ s2.flush()
f1.value='f1rev3mine'
success = False
try:
# a concurrent session has modified this, should throw
# an exception
- objectstore.commit()
- except SQLAlchemyError:
+ s.flush()
+ except exceptions.SQLAlchemyError, e:
+ #print e
success = True
assert success
- objectstore.clear()
- f1 = Foo.mapper.get(f1.id)
- f2 = Foo.mapper.get(f2.id)
+ s.clear()
+ f1 = s.query(Foo).get(f1.id)
+ f2 = s.query(Foo).get(f2.id)
f1_s.value='f1rev4'
- s.commit()
+ s2.flush()
- objectstore.delete(f1, f2)
+ s.delete(f1, f2)
success = False
try:
- objectstore.commit()
- except SQLAlchemyError:
+ s.flush()
+ except exceptions.SQLAlchemyError, e:
+ #print e
success = True
assert success
-class UnicodeTest(AssertMixin):
+class UnicodeTest(SessionTest):
def setUpAll(self):
- objectstore.clear()
+ SessionTest.setUpAll(self)
global uni_table
uni_table = Table('uni_test', db,
Column('id', Integer, primary_key=True),
def tearDownAll(self):
uni_table.drop()
- uni_table.deregister()
+ SessionTest.tearDownAll(self)
def testbasic(self):
class Test(object):
- pass
- assign_mapper(Test, uni_table)
+ def __init__(self, id, txt):
+ self.id = id
+ self.txt = txt
+ mapper(Test, uni_table)
txt = u"\u0160\u0110\u0106\u010c\u017d"
t1 = Test(id=1, txt = txt)
self.assert_(t1.txt == txt)
- objectstore.commit()
+ ctx.current.flush()
self.assert_(t1.txt == txt)
-class PKTest(AssertMixin):
+class PKTest(SessionTest):
def setUpAll(self):
+ SessionTest.setUpAll(self)
db.echo = False
global table
global table2
table2.drop()
table3.drop()
db.echo = testbase.echo
- def setUp(self):
- objectstore.clear()
- clear_mappers()
+ SessionTest.tearDownAll(self)
@testbase.unsupported('sqlite')
def testprimarykey(self):
e.name = 'entry1'
e.value = 'this is entry 1'
e.multi_rev = 2
- objectstore.commit()
- objectstore.clear()
- e2 = Entry.mapper.get(e.multi_id, 2)
+ ctx.current.flush()
+ ctx.current.clear()
+ e2 = Entry.mapper.get((e.multi_id, 2))
self.assert_(e is not e2 and e._instance_key == e2._instance_key)
def testmanualpk(self):
class Entry(object):
e.pk_col_1 = 'pk1'
e.pk_col_2 = 'pk1_related'
e.data = 'im the data'
- objectstore.commit()
+ ctx.current.flush()
def testkeypks(self):
import datetime
class Entity(object):
e.secondary = 'pk2'
e.assigned = datetime.date.today()
e.data = 'some more data'
- objectstore.commit()
+ ctx.current.flush()
-class PrivateAttrTest(AssertMixin):
+class PrivateAttrTest(SessionTest):
"""tests various things to do with private=True mappers"""
def setUpAll(self):
+ SessionTest.setUpAll(self)
global a_table, b_table
a_table = Table('a',testbase.db,
Column('a_id', Integer, Sequence('next_a_id'), primary_key=True),
def tearDownAll(self):
b_table.drop()
a_table.drop()
- def setUp(self):
- objectstore.clear()
- clear_mappers()
+ SessionTest.tearDownAll(self)
def testsinglecommit(self):
"""tests that a commit of a single object deletes private relationships"""
class B(object):pass
assign_mapper(B,b_table)
- assign_mapper(A,a_table,properties= {'bs' : relation
- (B.mapper,private=True)})
+ assign_mapper(A,a_table,properties= {'bs' : relation(B.mapper,private=True)})
# create some objects
a = A(data='a1')
a.bs.append(b2)
# inserts both A and Bs
- objectstore.commit(a)
+ ctx.current.flush([a])
- objectstore.delete(a)
- objectstore.commit(a)
+ ctx.current.delete(a)
+ ctx.current.flush([a])
assert b_table.count().scalar() == 0
a2 = A(data='testa2')
b = B(data='testb')
b.a = a1
- objectstore.commit()
- objectstore.clear()
- sess = objectstore.get_session()
+ ctx.current.flush()
+ ctx.current.clear()
+ sess = ctx.current
a1 = A.mapper.get(a1.a_id)
a2 = A.mapper.get(a2.a_id)
assert a1.bs[0].a is a1
b = a1.bs[0]
b.a = a2
assert b not in sess.deleted
- objectstore.commit()
+ ctx.current.flush()
assert b in sess.identity_map.values()
-class DefaultTest(AssertMixin):
+class DefaultTest(SessionTest):
"""tests that when saving objects whose table contains DefaultGenerators, either python-side, preexec or database-side,
the newly saved instances receive all the default values either through a post-fetch or getting the pre-exec'ed
defaults back from the engine."""
def setUpAll(self):
+ SessionTest.setUpAll(self)
#db.echo = 'debug'
use_string_defaults = db.engine.__module__.endswith('postgres') or db.engine.__module__.endswith('oracle') or db.engine.__module__.endswith('sqlite')
self.table.create()
def tearDownAll(self):
self.table.drop()
+ SessionTest.tearDownAll(self)
def setUp(self):
self.table = Table('default_test', db)
def testinsert(self):
h3 = Hoho(hoho=self.althohoval, counter=12)
h4 = Hoho()
h5 = Hoho(foober='im the new foober')
- objectstore.commit()
+ ctx.current.flush()
self.assert_(h1.hoho==self.althohoval)
self.assert_(h3.hoho==self.althohoval)
self.assert_(h2.hoho==h4.hoho==h5.hoho==self.hohoval)
self.assert_(h1.counter == h4.counter==h5.counter==7)
self.assert_(h2.foober == h3.foober == h4.foober == 'im foober')
self.assert_(h5.foober=='im the new foober')
- objectstore.clear()
+ ctx.current.clear()
l = Hoho.mapper.select()
(h1, h2, h3, h4, h5) = l
self.assert_(h1.hoho==self.althohoval)
class Hoho(object):pass
assign_mapper(Hoho, self.table)
h1 = Hoho(hoho="15", counter="15")
- objectstore.commit()
+ ctx.current.flush()
self.assert_(h1.hoho=="15")
self.assert_(h1.counter=="15")
self.assert_(h1.foober=="im foober")
class Hoho(object):pass
assign_mapper(Hoho, self.table)
h1 = Hoho()
- objectstore.commit()
+ ctx.current.flush()
self.assert_(h1.foober == 'im foober')
h1.counter = 19
- objectstore.commit()
+ ctx.current.flush()
self.assert_(h1.foober == 'im the update')
-class SaveTest(AssertMixin):
+class SaveTest(SessionTest):
def setUpAll(self):
+ SessionTest.setUpAll(self)
db.echo = False
tables.create()
db.echo = testbase.echo
def tearDownAll(self):
db.echo = False
- db.commit()
tables.drop()
db.echo = testbase.echo
+ SessionTest.tearDownAll(self)
def setUp(self):
db.echo = False
- # remove all history/identity maps etc.
- objectstore.clear()
- # remove all mapperes
- clear_mappers()
keywords.insert().execute(
dict(name='blue'),
dict(name='red'),
dict(name='round'),
dict(name='square')
)
- db.commit()
db.echo = testbase.echo
def tearDown(self):
db.echo = False
- db.commit()
tables.delete()
db.echo = testbase.echo
- self.assert_(len(objectstore.uow().new) == 0)
- self.assert_(len(objectstore.uow().dirty) == 0)
- self.assert_(len(objectstore.uow().modified_lists) == 0)
-
+ #self.assert_(len(ctx.current.new) == 0)
+ #self.assert_(len(ctx.current.dirty) == 0)
+ SessionTest.tearDown(self)
+
def testbasic(self):
# save two users
u = User()
u.user_name = 'savetester'
-
m = mapper(User, users)
u2 = User()
u2.user_name = 'savetester2'
- objectstore.uow().register_new(u)
+ ctx.current.save(u)
- objectstore.uow().commit(u)
- objectstore.uow().commit()
+ ctx.current.flush([u])
+ ctx.current.flush()
# assert the first one retreives the same from the identity map
nu = m.get(u.user_id)
self.assert_(u is nu)
# clear out the identity map, so next get forces a SELECT
- objectstore.clear()
+ ctx.current.clear()
# check it again, identity should be different but ids the same
nu = m.get(u.user_id)
self.assert_(u is not nu and u.user_id == nu.user_id and nu.user_name == 'savetester')
# change first users name and save
+ ctx.current.update(u)
u.user_name = 'modifiedname'
- objectstore.uow().commit()
+ assert u in ctx.current.dirty
+ ctx.current.flush()
# select both
- #objectstore.clear()
+ #ctx.current.clear()
userlist = m.select(users.c.user_id.in_(u.user_id, u2.user_id), order_by=[users.c.user_name])
print repr(u.user_id), repr(userlist[0].user_id), repr(userlist[0].user_name)
self.assert_(u.user_id == userlist[0].user_id and userlist[0].user_name == 'modifiedname')
u.addresses.append(Address())
u.addresses.append(Address())
u.addresses.append(Address())
- objectstore.commit()
- objectstore.clear()
+ ctx.current.flush()
+ ctx.current.clear()
ulist = m1.select()
u1 = ulist[0]
u1.user_name = 'newname'
- objectstore.commit()
+ ctx.current.flush()
self.assert_(len(u1.addresses) == 4)
def testinherits(self):
)
au = AddressUser()
- objectstore.commit()
- objectstore.clear()
+ ctx.current.flush()
+ ctx.current.clear()
l = AddressUser.mapper.selectone()
self.assert_(l.user_id == au.user_id and l.address_id == au.address_id)
"""tests a save of an object where each instance spans two tables. also tests
redefinition of the keynames for the column properties."""
usersaddresses = sql.join(users, addresses, users.c.user_id == addresses.c.user_id)
- print usersaddresses._get_col_by_original(users.c.user_id)
+ print usersaddresses.corresponding_column(users.c.user_id)
print repr(usersaddresses._orig_cols)
- m = mapper(User, usersaddresses, primarytable = users,
+ m = mapper(User, usersaddresses,
properties = dict(
email = addresses.c.email_address,
foo_id = [users.c.user_id, addresses.c.user_id],
u.user_name = 'multitester'
u.email = 'multi@test.org'
- objectstore.uow().commit()
+ ctx.current.flush()
usertable = users.select(users.c.user_id.in_(u.foo_id)).execute().fetchall()
self.assertEqual(usertable[0].values(), [u.foo_id, 'multitester'])
u.email = 'lala@hey.com'
u.user_name = 'imnew'
- objectstore.uow().commit()
+ ctx.current.flush()
usertable = users.select(users.c.user_id.in_(u.foo_id)).execute().fetchall()
self.assertEqual(usertable[0].values(), [u.foo_id, 'imnew'])
u.user_name = 'one2onetester'
u.address = Address()
u.address.email_address = 'myonlyaddress@foo.com'
- objectstore.uow().commit()
+ ctx.current.flush()
u.user_name = 'imnew'
- objectstore.uow().commit()
+ ctx.current.flush()
u.address.email_address = 'imnew@foo.com'
- objectstore.uow().commit()
+ ctx.current.flush()
+ def testchildmove(self):
+ """tests moving a child from one parent to the other, then deleting the first parent, properly
+ updates the child with the new parent. this tests the 'trackparent' option in the attributes module."""
+ m = mapper(User, users, properties = dict(
+ addresses = relation(mapper(Address, addresses), lazy = True, private = False)
+ ))
+ u1 = User()
+ u1.user_name = 'user1'
+ u2 = User()
+ u2.user_name = 'user2'
+ a = Address()
+ a.email_address = 'address1'
+ u1.addresses.append(a)
+ ctx.current.flush()
+ del u1.addresses[0]
+ u2.addresses.append(a)
+ ctx.current.delete(u1)
+ ctx.current.flush()
+ ctx.current.clear()
+ u2 = m.get(u2.user_id)
+ assert len(u2.addresses) == 1
+
def testdelete(self):
m = mapper(User, users, properties = dict(
address = relation(mapper(Address, addresses), lazy = True, uselist = False, private = False)
u.user_name = 'one2onetester'
u.address = a
u.address.email_address = 'myonlyaddress@foo.com'
- objectstore.uow().commit()
+ ctx.current.flush()
self.echo("\n\n\n")
- objectstore.uow().register_deleted(u)
- objectstore.uow().commit()
- self.assert_(a.address_id is not None and a.user_id is None and not objectstore.uow().identity_map.has_key(u._instance_key) and objectstore.uow().identity_map.has_key(a._instance_key))
-
- def testcascadingdelete(self):
- m = mapper(User, users, properties = dict(
- address = relation(mapper(Address, addresses), lazy = False, uselist = False, private = True),
- orders = relation(
- mapper(Order, orders, properties = dict (
- items = relation(mapper(Item, orderitems), lazy = False, uselist =True, private = True)
- )),
- lazy = True, uselist = True, private = True)
- ))
-
- data = [User,
- {'user_name' : 'ed',
- 'address' : (Address, {'email_address' : 'foo@bar.com'}),
- 'orders' : (Order, [
- {'description' : 'eds 1st order', 'items' : (Item, [{'item_name' : 'eds o1 item'}, {'item_name' : 'eds other o1 item'}])},
- {'description' : 'eds 2nd order', 'items' : (Item, [{'item_name' : 'eds o2 item'}, {'item_name' : 'eds other o2 item'}])}
- ])
- },
- {'user_name' : 'jack',
- 'address' : (Address, {'email_address' : 'jack@jack.com'}),
- 'orders' : (Order, [
- {'description' : 'jacks 1st order', 'items' : (Item, [{'item_name' : 'im a lumberjack'}, {'item_name' : 'and im ok'}])}
- ])
- },
- {'user_name' : 'foo',
- 'address' : (Address, {'email_address': 'hi@lala.com'}),
- 'orders' : (Order, [
- {'description' : 'foo order', 'items' : (Item, [])},
- {'description' : 'foo order 2', 'items' : (Item, [{'item_name' : 'hi'}])},
- {'description' : 'foo order three', 'items' : (Item, [{'item_name' : 'there'}])}
- ])
- }
- ]
-
- for elem in data[1:]:
- u = User()
- u.user_name = elem['user_name']
- u.address = Address()
- u.address.email_address = elem['address'][1]['email_address']
- u.orders = []
- for order in elem['orders'][1]:
- o = Order()
- o.isopen = None
- o.description = order['description']
- u.orders.append(o)
- o.items = []
- for item in order['items'][1]:
- i = Item()
- i.item_name = item['item_name']
- o.items.append(i)
-
- objectstore.uow().commit()
- objectstore.clear()
-
- l = m.select()
- for u in l:
- self.echo( repr(u.orders))
- self.assert_result(l, data[0], *data[1:])
-
- self.echo("\n\n\n")
- objectstore.uow().register_deleted(l[0])
- objectstore.uow().register_deleted(l[2])
- objectstore.commit()
- return
- res = self.capture_exec(db, lambda: objectstore.uow().commit())
- state = None
-
- for line in res.split('\n'):
- if line == "DELETE FROM items WHERE items.item_id = :item_id":
- self.assert_(state is None or state == 'addresses')
- elif line == "DELETE FROM orders WHERE orders.order_id = :order_id":
- state = 'orders'
- elif line == "DELETE FROM email_addresses WHERE email_addresses.address_id = :address_id":
- if state is None:
- state = 'addresses'
- elif line == "DELETE FROM users WHERE users.user_id = :user_id":
- self.assert_(state is not None)
+ ctx.current.delete(u)
+ ctx.current.flush()
+ self.assert_(a.address_id is not None and a.user_id is None and not ctx.current.identity_map.has_key(u._instance_key) and ctx.current.identity_map.has_key(a._instance_key))
def testbackwardsonetoone(self):
# test 'backwards'
a.user.user_name = elem['user_name']
objects.append(a)
- objectstore.uow().commit()
+ ctx.current.flush()
objects[2].email_address = 'imnew@foo.bar'
objects[3].user = User()
objects[3].user.user_name = 'imnewlyadded'
- self.assert_sql(db, lambda: objectstore.uow().commit(), [
+ self.assert_sql(db, lambda: ctx.current.flush(), [
(
"INSERT INTO users (user_name) VALUES (:user_name)",
{'user_name': 'imnewlyadded'}
),
{
"UPDATE email_addresses SET email_address=:email_address WHERE email_addresses.address_id = :email_addresses_address_id":
- lambda: [{'email_address': 'imnew@foo.bar', 'email_addresses_address_id': objects[2].address_id}]
+ lambda ctx: {'email_address': 'imnew@foo.bar', 'email_addresses_address_id': objects[2].address_id}
,
"UPDATE email_addresses SET user_id=:user_id WHERE email_addresses.address_id = :email_addresses_address_id":
- lambda: [{'user_id': objects[3].user.user_id, 'email_addresses_address_id': objects[3].address_id}]
+ lambda ctx: {'user_id': objects[3].user.user_id, 'email_addresses_address_id': objects[3].address_id}
},
],
with_sequences=[
(
"INSERT INTO users (user_id, user_name) VALUES (:user_id, :user_name)",
- lambda:{'user_name': 'imnewlyadded', 'user_id':db.last_inserted_ids()[0]}
+ lambda ctx:{'user_name': 'imnewlyadded', 'user_id':ctx.last_inserted_ids()[0]}
),
{
"UPDATE email_addresses SET email_address=:email_address WHERE email_addresses.address_id = :email_addresses_address_id":
- lambda: [{'email_address': 'imnew@foo.bar', 'email_addresses_address_id': objects[2].address_id}]
+ lambda ctx: {'email_address': 'imnew@foo.bar', 'email_addresses_address_id': objects[2].address_id}
,
"UPDATE email_addresses SET user_id=:user_id WHERE email_addresses.address_id = :email_addresses_address_id":
- lambda: [{'user_id': objects[3].user.user_id, 'email_addresses_address_id': objects[3].address_id}]
+ lambda ctx: {'user_id': objects[3].user.user_id, 'email_addresses_address_id': objects[3].address_id}
},
])
u.addresses.append(a2)
self.echo( repr(u.addresses))
self.echo( repr(u.addresses.added_items()))
- objectstore.uow().commit()
+ ctx.current.flush()
usertable = users.select(users.c.user_id.in_(u.user_id)).execute().fetchall()
self.assertEqual(usertable[0].values(), [u.user_id, 'one2manytester'])
a2.email_address = 'somethingnew@foo.com'
- objectstore.uow().commit()
+ ctx.current.flush()
addresstable = addresses.select(addresses.c.address_id == addressid).execute().fetchall()
dict(user_id = 8, user_name = 'ed'),
dict(user_id = 9, user_name = 'fred')
)
- db.commit()
# mapper with just users table
assign_mapper(User, users)
u[0].addresses[0].email_address='hi'
# insure that upon commit, the new mapper with the address relation is used
- self.assert_sql(db, lambda: objectstore.commit(),
+ self.assert_sql(db, lambda: ctx.current.flush(),
[
(
"INSERT INTO email_addresses (user_id, email_address) VALUES (:user_id, :email_address)",
with_sequences=[
(
"INSERT INTO email_addresses (address_id, user_id, email_address) VALUES (:address_id, :user_id, :email_address)",
- lambda:{'email_address': 'hi', 'user_id': 7, 'address_id':db.last_inserted_ids()[0]}
+ lambda ctx:{'email_address': 'hi', 'user_id': 7, 'address_id':ctx.last_inserted_ids()[0]}
),
]
)
a3 = Address()
a3.email_address = 'emailaddress3'
- objectstore.commit()
+ ctx.current.flush()
self.echo("\n\n\n")
# modify user2 directly, append an address to user1.
u2.user_name = 'user2modified'
u1.addresses.append(a3)
del u1.addresses[0]
- self.assert_sql(db, lambda: objectstore.commit(),
+ self.assert_sql(db, lambda: ctx.current.flush(),
[
(
"UPDATE users SET user_name=:user_name WHERE users.user_id = :users_user_id",
- [{'users_user_id': u2.user_id, 'user_name': 'user2modified'}]
+ {'users_user_id': u2.user_id, 'user_name': 'user2modified'}
),
(
"UPDATE email_addresses SET user_id=:user_id WHERE email_addresses.address_id = :email_addresses_address_id",
- [{'user_id': u1.user_id, 'email_addresses_address_id': a3.address_id}]
+ {'user_id': u1.user_id, 'email_addresses_address_id': a3.address_id}
),
("UPDATE email_addresses SET user_id=:user_id WHERE email_addresses.address_id = :email_addresses_address_id",
- [{'user_id': None, 'email_addresses_address_id': a1.address_id}]
+ {'user_id': None, 'email_addresses_address_id': a1.address_id}
)
])
u1.user_name='user1'
a1.user = u1
- objectstore.commit()
+ ctx.current.flush()
self.echo("\n\n\n")
- objectstore.delete(u1)
+ ctx.current.delete(u1)
a1.user = None
- objectstore.commit()
+ ctx.current.flush()
def _testalias(self):
"""tests that an alias of a table can be used in a mapper.
def testmanytomany(self):
items = orderitems
+ keywordmapper = mapper(Keyword, keywords)
+
items.select().execute()
m = mapper(Item, items, properties = dict(
- keywords = relation(mapper(Keyword, keywords), itemkeywords, lazy = False),
+ keywords = relation(keywordmapper, itemkeywords, lazy = False),
))
- keywordmapper = mapper(Keyword, keywords)
data = [Item,
{'item_name': 'mm_item1', 'keywords' : (Keyword,[{'name': 'big'},{'name': 'green'}, {'name': 'purple'},{'name': 'round'}])},
k.name = kname
item.keywords.append(k)
- objectstore.uow().commit()
+ ctx.current.flush()
l = m.select(items.c.item_name.in_(*[e['item_name'] for e in data[1:]]), order_by=[items.c.item_name, keywords.c.name])
self.assert_result(l, *data)
k = Keyword()
k.name = 'yellow'
objects[5].keywords.append(k)
- self.assert_sql(db, lambda:objectstore.commit(), [
+ self.assert_sql(db, lambda:ctx.current.flush(), [
{
"UPDATE items SET item_name=:item_name WHERE items.item_id = :items_item_id":
- [{'item_name': 'item4updated', 'items_item_id': objects[4].item_id}]
+ {'item_name': 'item4updated', 'items_item_id': objects[4].item_id}
,
"INSERT INTO keywords (name) VALUES (:name)":
{'name': 'yellow'}
},
("INSERT INTO itemkeywords (item_id, keyword_id) VALUES (:item_id, :keyword_id)",
- lambda: [{'item_id': objects[5].item_id, 'keyword_id': k.keyword_id}]
+ lambda ctx: [{'item_id': objects[5].item_id, 'keyword_id': k.keyword_id}]
)
],
with_sequences = [
{
"UPDATE items SET item_name=:item_name WHERE items.item_id = :items_item_id":
- [{'item_name': 'item4updated', 'items_item_id': objects[4].item_id}]
+ {'item_name': 'item4updated', 'items_item_id': objects[4].item_id}
,
"INSERT INTO keywords (keyword_id, name) VALUES (:keyword_id, :name)":
- lambda: {'name': 'yellow', 'keyword_id':db.last_inserted_ids()[0]}
+ lambda ctx: {'name': 'yellow', 'keyword_id':ctx.last_inserted_ids()[0]}
},
("INSERT INTO itemkeywords (item_id, keyword_id) VALUES (:item_id, :keyword_id)",
- lambda: [{'item_id': objects[5].item_id, 'keyword_id': k.keyword_id}]
+ lambda ctx: [{'item_id': objects[5].item_id, 'keyword_id': k.keyword_id}]
)
]
)
objects[2].keywords.append(k)
dkid = objects[5].keywords[1].keyword_id
del objects[5].keywords[1]
- self.assert_sql(db, lambda:objectstore.commit(), [
+ self.assert_sql(db, lambda:ctx.current.flush(), [
(
"DELETE FROM itemkeywords WHERE itemkeywords.item_id = :item_id AND itemkeywords.keyword_id = :keyword_id",
[{'item_id': objects[5].item_id, 'keyword_id': dkid}]
),
(
"INSERT INTO itemkeywords (item_id, keyword_id) VALUES (:item_id, :keyword_id)",
- lambda: [{'item_id': objects[2].item_id, 'keyword_id': k.keyword_id}]
+ lambda ctx: [{'item_id': objects[2].item_id, 'keyword_id': k.keyword_id}]
)
])
- objectstore.delete(objects[3])
- objectstore.commit()
+ ctx.current.delete(objects[3])
+ ctx.current.flush()
def testassociation(self):
class IKAssociation(object):
# the reorganization of mapper construction affected this, but was fixed again
m = mapper(Item, items, properties = dict(
keywords = relation(mapper(IKAssociation, itemkeywords, properties = dict(
- keyword = relation(mapper(Keyword, keywords), lazy = False, uselist = False)
+ keyword = relation(mapper(Keyword, keywords, non_primary=True), lazy = False, uselist = False)
), primary_key = [itemkeywords.c.item_id, itemkeywords.c.keyword_id]),
lazy = False)
))
ik.keyword = k
item.keywords.append(ik)
- objectstore.uow().commit()
- objectstore.clear()
+ ctx.current.flush()
+ ctx.current.clear()
l = m.select(items.c.item_name.in_(*[e['item_name'] for e in data[1:]]), order_by=[items.c.item_name, keywords.c.name])
self.assert_result(l, *data)
a = Address()
a.email_address = 'testaddress'
a.user = u
- objectstore.commit()
+ ctx.current.flush()
print repr(u.addresses)
x = False
try:
if x:
self.assert_(False, "User addresses element should be scalar based")
- objectstore.delete(u)
- objectstore.commit()
+ ctx.current.delete(u)
+ ctx.current.flush()
def testdoublerelation(self):
m2 = mapper(Address, addresses)
u.boston_addresses.append(a)
u.newyork_addresses.append(b)
- objectstore.commit()
+ ctx.current.flush()
-class SaveTest2(AssertMixin):
+class SaveTest2(SessionTest):
def setUp(self):
db.echo = False
- objectstore.clear()
+ ctx.current.clear()
clear_mappers()
self.users = Table('users', db,
Column('user_id', Integer, Sequence('user_id_seq', optional=True), primary_key = True),
self.addresses.drop()
self.users.drop()
db.echo = testbase.echo
+ SessionTest.tearDown(self)
def testbackwardsnonmatch(self):
m = mapper(Address, self.addresses, properties = dict(
a.user = User()
a.user.user_name = elem['user_name']
objects.append(a)
- self.assert_sql(db, lambda: objectstore.commit(), [
+ self.assert_sql(db, lambda: ctx.current.flush(), [
(
"INSERT INTO users (user_name) VALUES (:user_name)",
{'user_name': 'thesub'}
with_sequences = [
(
"INSERT INTO users (user_id, user_name) VALUES (:user_id, :user_name)",
- lambda: {'user_name': 'thesub', 'user_id':db.last_inserted_ids()[0]}
+ lambda ctx: {'user_name': 'thesub', 'user_id':ctx.last_inserted_ids()[0]}
),
(
"INSERT INTO users (user_id, user_name) VALUES (:user_id, :user_name)",
- lambda: {'user_name': 'assdkfj', 'user_id':db.last_inserted_ids()[0]}
+ lambda ctx: {'user_name': 'assdkfj', 'user_id':ctx.last_inserted_ids()[0]}
),
(
"INSERT INTO email_addresses (address_id, rel_user_id, email_address) VALUES (:address_id, :rel_user_id, :email_address)",
- lambda:{'rel_user_id': 1, 'email_address': 'bar@foo.com', 'address_id':db.last_inserted_ids()[0]}
+ lambda ctx:{'rel_user_id': 1, 'email_address': 'bar@foo.com', 'address_id':ctx.last_inserted_ids()[0]}
),
(
"INSERT INTO email_addresses (address_id, rel_user_id, email_address) VALUES (:address_id, :rel_user_id, :email_address)",
- lambda:{'rel_user_id': 2, 'email_address': 'thesdf@asdf.com', 'address_id':db.last_inserted_ids()[0]}
+ lambda ctx:{'rel_user_id': 2, 'email_address': 'thesdf@asdf.com', 'address_id':ctx.last_inserted_ids()[0]}
)
]
)
from sqlalchemy import *
import testbase
+from sqlalchemy.ext.sessioncontext import SessionContext
class Jack(object):
def __repr__(self):
class O2OTest(testbase.AssertMixin):
def setUpAll(self):
- global jack, port
- jack = Table('jack', testbase.db,
+ global jack, port, metadata, ctx
+ metadata = BoundMetaData(testbase.db)
+ ctx = SessionContext(create_session)
+ jack = Table('jack', metadata,
Column('id', Integer, primary_key=True),
#Column('room_id', Integer, ForeignKey("room.id")),
Column('number', String(50)),
)
- port = Table('port', testbase.db,
+ port = Table('port', metadata,
Column('id', Integer, primary_key=True),
#Column('device_id', Integer, ForeignKey("device.id")),
Column('name', String(30)),
Column('description', String(100)),
Column('jack_id', Integer, ForeignKey("jack.id")),
)
- jack.create()
- port.create()
+ metadata.create_all()
def setUp(self):
- objectstore.clear()
+ pass
def tearDown(self):
clear_mappers()
def tearDownAll(self):
- port.drop()
- jack.drop()
+ metadata.drop_all()
def test1(self):
- assign_mapper(Port, port)
- assign_mapper(Jack, jack, order_by=[jack.c.number],properties = {
- 'port': relation(Port.mapper, backref='jack', uselist=False, lazy=True),
- })
+ mapper(Port, port, extension=ctx.mapper_extension)
+ mapper(Jack, jack, order_by=[jack.c.number],properties = {
+ 'port': relation(Port, backref='jack', uselist=False, lazy=True),
+ }, extension=ctx.mapper_extension)
j=Jack(number='101')
p=Port(name='fa0/1')
j.port=p
- objectstore.commit()
+ ctx.current.flush()
jid = j.id
pid = p.id
- j=Jack.get(jid)
- p=Port.get(pid)
+ j=ctx.current.query(Jack).get(jid)
+ p=ctx.current.query(Port).get(pid)
print p.jack
- print j.port
+ assert p.jack is not None
+ assert p.jack is j
+ assert j.port is not None
p.jack=None
assert j.port is None #works
- objectstore.clear()
+ ctx.current.clear()
- j=Jack.get(jid)
- p=Port.get(pid)
+ j=ctx.current.query(Jack).get(jid)
+ p=ctx.current.query(Port).get(pid)
j.port=None
self.assert_(p.jack is None)
- objectstore.commit()
+ ctx.current.flush()
- j.delete()
- objectstore.commit()
+ ctx.current.delete(j)
+ ctx.current.flush()
if __name__ == "__main__":
testbase.main()
--- /dev/null
+from testbase import PersistTest
+import sqlalchemy.engine.url as url
+import unittest
+
+class ParseConnectTest(PersistTest):
+ def testrfc1738(self):
+ for text in (
+ 'dbtype://username:password@hostspec:110//usr/db_file.db',
+ 'dbtype://username:password@hostspec/database',
+ 'dbtype://username:password@hostspec',
+ 'dbtype://username:password@/database',
+ 'dbtype://username@hostspec',
+ 'dbtype://username:password@127.0.0.1:1521',
+ 'dbtype://hostspec/database',
+ 'dbtype://hostspec',
+ 'dbtype:///database',
+ 'dbtype:///:memory:',
+ 'dbtype:///foo/bar/im/a/file',
+ 'dbtype:///E:/work/src/LEM/db/hello.db',
+ 'dbtype://'
+ ):
+ u = url.make_url(text)
+ print u, text
+ assert str(u) == text
+
+
+if __name__ == "__main__":
+ unittest.main()
+
\ No newline at end of file
--- /dev/null
+import testbase
+from sqlalchemy import *
+import sets
+
+# test classes
+class Person(object):
+ def __init__(self, **kwargs):
+ for key, value in kwargs.iteritems():
+ setattr(self, key, value)
+ def get_name(self):
+ try:
+ return getattr(self, 'person_name')
+ except AttributeError:
+ return getattr(self, 'name')
+ def __repr__(self):
+ return "Ordinary person %s" % self.get_name()
+class Engineer(Person):
+ def __repr__(self):
+ return "Engineer %s, status %s, engineer_name %s, primary_language %s" % (self.get_name(), self.status, self.engineer_name, self.primary_language)
+class Manager(Person):
+ def __repr__(self):
+ return "Manager %s, status %s, manager_name %s" % (self.get_name(), self.status, self.manager_name)
+class Company(object):
+ def __init__(self, **kwargs):
+ for key, value in kwargs.iteritems():
+ setattr(self, key, value)
+ def __repr__(self):
+ return "Company %s" % self.name
+
+class MultipleTableTest(testbase.PersistTest):
+ def setUpAll(self, use_person_column=False):
+ global companies, people, engineers, managers, metadata
+ metadata = BoundMetaData(testbase.db)
+
+ # a table to store companies
+ companies = Table('companies', metadata,
+ Column('company_id', Integer, primary_key=True),
+ Column('name', String(50)))
+
+ # we will define an inheritance relationship between the table "people" and "engineers",
+ # and a second inheritance relationship between the table "people" and "managers"
+ people = Table('people', metadata,
+ Column('person_id', Integer, primary_key=True),
+ Column('company_id', Integer, ForeignKey('companies.company_id')),
+ Column('name', String(50)),
+ Column('type', String(30)))
+
+ engineers = Table('engineers', metadata,
+ Column('person_id', Integer, ForeignKey('people.person_id'), primary_key=True),
+ Column('status', String(30)),
+ Column('engineer_name', String(50)),
+ Column('primary_language', String(50)),
+ )
+
+ managers = Table('managers', metadata,
+ Column('person_id', Integer, ForeignKey('people.person_id'), primary_key=True),
+ Column('status', String(30)),
+ Column('manager_name', String(50))
+ )
+
+ metadata.create_all()
+
+ def tearDownAll(self):
+ metadata.drop_all()
+
+ def tearDown(self):
+ clear_mappers()
+ for t in metadata.table_iterator(reverse=True):
+ t.delete().execute()
+
+ def test_f_f_f(self):
+ self.do_test(False, False, False)
+ def test_f_f_t(self):
+ self.do_test(False, False, True)
+ def test_f_t_f(self):
+ self.do_test(False, True, False)
+ def test_f_t_t(self):
+ self.do_test(False, True, True)
+ def test_t_f_f(self):
+ self.do_test(True, False, False)
+ def test_t_f_t(self):
+ self.do_test(True, False, True)
+ def test_t_t_f(self):
+ self.do_test(True, True, False)
+ def test_t_t_t(self):
+ self.do_test(True, True, True)
+
+
+ def do_test(self, include_base=False, lazy_relation=True, redefine_colprop=False):
+ """tests the polymorph.py example, with several options:
+
+ include_base - whether or not to include the base 'person' type in the union.
+ lazy_relation - whether or not the Company relation to People is lazy or eager.
+ redefine_colprop - if we redefine the 'name' column to be 'people_name' on the base Person class
+ """
+ # create a union that represents both types of joins.
+ if include_base:
+ person_join = polymorphic_union(
+ {
+ 'engineer':people.join(engineers),
+ 'manager':people.join(managers),
+ 'person':people.select(people.c.type=='person'),
+ }, None, 'pjoin')
+ else:
+ person_join = polymorphic_union(
+ {
+ 'engineer':people.join(engineers),
+ 'manager':people.join(managers),
+ }, None, 'pjoin')
+
+ if redefine_colprop:
+ person_mapper = mapper(Person, people, select_table=person_join, polymorphic_on=person_join.c.type, polymorphic_identity='person', properties= {'person_name':people.c.name})
+ else:
+ person_mapper = mapper(Person, people, select_table=person_join, polymorphic_on=person_join.c.type, polymorphic_identity='person')
+
+ mapper(Engineer, engineers, inherits=person_mapper, polymorphic_identity='engineer')
+ mapper(Manager, managers, inherits=person_mapper, polymorphic_identity='manager')
+
+ mapper(Company, companies, properties={
+ 'employees': relation(Person, lazy=lazy_relation, private=True, backref='company')
+ })
+
+ if redefine_colprop:
+ person_attribute_name = 'person_name'
+ else:
+ person_attribute_name = 'name'
+
+ session = create_session()
+ c = Company(name='company1')
+ c.employees.append(Manager(status='AAB', manager_name='manager1', **{person_attribute_name:'pointy haired boss'}))
+ c.employees.append(Engineer(status='BBA', engineer_name='engineer1', primary_language='java', **{person_attribute_name:'dilbert'}))
+ if include_base:
+ c.employees.append(Person(status='HHH', **{person_attribute_name:'joesmith'}))
+ c.employees.append(Engineer(status='CGG', engineer_name='engineer2', primary_language='python', **{person_attribute_name:'wally'}))
+ c.employees.append(Manager(status='ABA', manager_name='manager2', **{person_attribute_name:'jsmith'}))
+ session.save(c)
+ print session.new
+ session.flush()
+ session.clear()
+ id = c.company_id
+ c = session.query(Company).get(id)
+ for e in c.employees:
+ print e, e._instance_key, e.company
+ if include_base:
+ assert sets.Set([e.get_name() for e in c.employees]) == sets.Set(['pointy haired boss', 'dilbert', 'joesmith', 'wally', 'jsmith'])
+ else:
+ assert sets.Set([e.get_name() for e in c.employees]) == sets.Set(['pointy haired boss', 'dilbert', 'wally', 'jsmith'])
+ print "\n"
+
+
+ dilbert = session.query(Person).selectfirst(person_join.c.name=='dilbert')
+ dilbert2 = session.query(Engineer).selectfirst(people.c.name=='dilbert')
+ assert dilbert is dilbert2
+
+ dilbert.engineer_name = 'hes dibert!'
+
+ session.flush()
+ session.clear()
+
+ c = session.query(Company).get(id)
+ for e in c.employees:
+ print e, e._instance_key
+
+ session.delete(c)
+ session.flush()
+
+if __name__ == "__main__":
+ testbase.main()
+
from testbase import PersistTest
-import unittest, sys, os
+import unittest, sys, os, time
from pysqlite2 import dbapi2 as sqlite
import sqlalchemy.pool as pool
self.assert_(connection.cursor() is not None)
self.assert_(connection is not connection2)
- def testqueuepool(self):
+ def testqueuepool_del(self):
+ self._do_testqueuepool(useclose=False)
+
+ def testqueuepool_close(self):
+ self._do_testqueuepool(useclose=True)
+
+ def _do_testqueuepool(self, useclose=False):
+
p = pool.QueuePool(creator = lambda: sqlite.connect('foo.db'), pool_size = 3, max_overflow = -1, use_threadlocal = False, echo = False)
def status(pool):
self.assert_(status(p) == (3,0,2,5))
c6 = p.connect()
self.assert_(status(p) == (3,0,3,6))
- c4 = c3 = c2 = None
+ if useclose:
+ c4.close()
+ c3.close()
+ c2.close()
+ else:
+ c4 = c3 = c2 = None
self.assert_(status(p) == (3,3,3,3))
- c1 = c5 = c6 = None
+ if useclose:
+ c1.close()
+ c5.close()
+ c6.close()
+ else:
+ c1 = c5 = c6 = None
self.assert_(status(p) == (3,3,0,0))
c1 = p.connect()
c2 = p.connect()
self.assert_(status(p) == (3, 1, 0, 2))
- c2 = None
+ if useclose:
+ c2.close()
+ else:
+ c2 = None
self.assert_(status(p) == (3, 2, 0, 1))
- def testthreadlocal(self):
+ def test_timeout(self):
+ p = pool.QueuePool(creator = lambda: sqlite.connect('foo.db'), pool_size = 3, max_overflow = 0, use_threadlocal = False, echo = False, timeout=2)
+ c1 = p.get()
+ c2 = p.get()
+ c3 = p.get()
+ now = time.time()
+ c4 = p.get()
+ assert int(time.time() - now) == 2
+
+ def testthreadlocal_del(self):
+ self._do_testthreadlocal(useclose=False)
+
+ def testthreadlocal_close(self):
+ self._do_testthreadlocal(useclose=True)
+
+ def _do_testthreadlocal(self, useclose=False):
for p in (
pool.QueuePool(creator = lambda: sqlite.connect('foo.db'), pool_size = 3, max_overflow = -1, use_threadlocal = True, echo = False),
pool.SingletonThreadPool(creator = lambda: sqlite.connect('foo.db'), use_threadlocal = True)
- ):
+ ):
c1 = p.connect()
c2 = p.connect()
self.assert_(c1 is c2)
c3 = p.unique_connection()
self.assert_(c3 is not c1)
- c2 = None
+ if useclose:
+ c2.close()
+ else:
+ c2 = None
c2 = p.connect()
self.assert_(c1 is c2)
self.assert_(c3 is not c1)
+ if useclose:
+ c2.close()
+ else:
+ c2 = None
+
+ if useclose:
+ c1 = p.connect()
+ c2 = p.connect()
+ c3 = p.connect()
+ c3.close()
+ c2.close()
+ self.assert_(c1.connection is not None)
def tearDown(self):
pool.clear_managers()
+import os
+
from sqlalchemy import *
from sqlalchemy.ext.proxy import ProxyEngine
from testbase import PersistTest
import testbase
-import os
#
# Define an engine, table and mapper at the module level, to show that the
#
-module_engine = ProxyEngine(echo=testbase.echo)
-users = Table('users', module_engine,
- Column('user_id', Integer, primary_key=True),
- Column('user_name', String(16)),
- Column('password', String(20))
- )
+class ProxyTestBase(PersistTest):
+ def setUpAll(self):
+
+ global users, User, module_engine, module_metadata
+
+ module_engine = ProxyEngine(echo=testbase.echo)
+ module_metadata = MetaData()
-class User(object):
- pass
+ users = Table('users', module_metadata,
+ Column('user_id', Integer, primary_key=True),
+ Column('user_name', String(16)),
+ Column('password', String(20))
+ )
+ class User(object):
+ pass
-class ConstructTest(PersistTest):
+ User.mapper = mapper(User, users)
+ def tearDownAll(self):
+ clear_mappers()
+
+class ConstructTest(ProxyTestBase):
"""tests that we can build SQL constructs without engine-specific parameters, particulary
oid_column, being needed, as the proxy engine is usually not connected yet."""
+
def test_join(self):
engine = ProxyEngine()
t = Table('table1', engine,
Column('col2', Integer, ForeignKey('table1.col1')))
j = join(t, t2)
-class ProxyEngineTest1(PersistTest):
- def setUp(self):
- clear_mappers()
- objectstore.clear()
-
+class ProxyEngineTest1(ProxyTestBase):
+
def test_engine_connect(self):
# connect to a real engine
module_engine.connect(testbase.db_uri)
- users.create()
- assign_mapper(User, users)
+ module_metadata.create_all(module_engine)
+
+ session = create_session(bind_to=module_engine)
try:
- trans = objectstore.begin()
user = User()
user.user_name='fred'
user.password='*'
- trans.commit()
+
+ session.save(user)
+ session.flush()
+
+ query = session.query(User)
# select
- sqluser = User.select_by(user_name='fred')[0]
+ sqluser = query.select_by(user_name='fred')[0]
assert sqluser.user_name == 'fred'
# modify
sqluser.user_name = 'fred jones'
- # commit - saves everything that changed
- objectstore.commit()
+ # flush - saves everything that changed
+ session.flush()
- allusers = [ user.user_name for user in User.select() ]
- assert allusers == [ 'fred jones' ]
+ allusers = [ user.user_name for user in query.select() ]
+ assert allusers == ['fred jones']
+
finally:
- users.drop()
+ module_metadata.drop_all(module_engine)
+
+
+class ThreadProxyTest(ProxyTestBase):
-class ThreadProxyTest(PersistTest):
- def setUp(self):
- assign_mapper(User, users)
- def tearDown(self):
- clear_mappers()
def tearDownAll(self):
- pass
os.remove('threadtesta.db')
os.remove('threadtestb.db')
try:
module_engine.connect(db_uri)
- users.create()
+ module_metadata.create_all(module_engine)
try:
- trans = objectstore.begin()
+ session = create_session(bind_to=module_engine)
+
+ query = session.query(User)
- all = User.select()[:]
+ all = list(query.select())
assert all == []
u = User()
u.user_name = uname
u.password = 'whatever'
- trans.commit()
- names = [ us.user_name for us in User.select() ]
- assert names == [ uname ]
+ session.save(u)
+ session.flush()
+
+ names = [u.user_name for u in query.select()]
+ assert names == [uname]
finally:
- users.drop()
- module_engine.dispose()
+ module_metadata.drop_all(module_engine)
except Exception, e:
import traceback
traceback.print_exc()
# NOTE: I'm not sure how to give the test runner the option to
# override these uris, or how to safely clear them after test runs
- a = Thread(target=run('sqlite://filename=threadtesta.db', 'jim', qa))
- b = Thread(target=run('sqlite://filename=threadtestb.db', 'joe', qb))
+ a = Thread(target=run('sqlite:///threadtesta.db', 'jim', qa))
+ b = Thread(target=run('sqlite:///threadtestb.db', 'joe', qb))
a.start()
b.start()
if res != False:
raise res
-class ProxyEngineTest2(PersistTest):
- def setUp(self):
- clear_mappers()
- objectstore.clear()
+class ProxyEngineTest2(ProxyTestBase):
def test_table_singleton_a(self):
"""set up for table singleton check
Column('cat_name', String))
engine.connect(testbase.db_uri)
- cats.create()
- cats.drop()
+
+ cats.create(engine)
+ cats.drop(engine)
ProxyEngineTest2.cats_table_a = cats
assert isinstance(cats, Table)
# this will fail because the old reference's local storage will
# not have the default attributes
engine.connect(testbase.db_uri)
- cats.create()
- cats.drop()
-
- def test_type_engine_caching(self):
- from sqlalchemy.engine import SQLEngine
- import sqlalchemy.types as sqltypes
-
- class EngineA(SQLEngine):
- def __init__(self):
- pass
-
- def hash_key(self):
- return 'a'
-
- def type_descriptor(self, typeobj):
- if isinstance(typeobj, types.Integer):
- return TypeEngineX2()
- else:
- return TypeEngineSTR()
-
- class EngineB(SQLEngine):
- def __init__(self):
- pass
-
- def hash_key(self):
- return 'b'
-
- def type_descriptor(self, typeobj):
- return TypeEngineMonkey()
-
- class TypeEngineX2(sqltypes.TypeEngine):
- def convert_bind_param(self, value, engine):
- return value * 2
-
- class TypeEngineSTR(sqltypes.TypeEngine):
- def convert_bind_param(self, value, engine):
- return repr(str(value))
-
- class TypeEngineMonkey(sqltypes.TypeEngine):
- def convert_bind_param(self, value, engine):
- return 'monkey'
-
- engine = ProxyEngine()
- engine.storage.engine = EngineA()
-
- a = sqltypes.Integer().engine_impl(engine)
- assert a.convert_bind_param(12, engine) == 24
- assert a.convert_bind_param([1,2,3], engine) == [1, 2, 3, 1, 2, 3]
-
- a2 = sqltypes.String().engine_impl(engine)
- assert a2.convert_bind_param(12, engine) == "'12'"
- assert a2.convert_bind_param([1,2,3], engine) == "'[1, 2, 3]'"
-
- engine.storage.engine = EngineB()
- b = sqltypes.Integer().engine_impl(engine)
- assert b.convert_bind_param(12, engine) == 'monkey'
- assert b.convert_bind_param([1,2,3], engine) == 'monkey'
-
-
- def test_type_engine_autoincrement(self):
- engine = ProxyEngine()
- dogs = Table('dogs', engine,
- Column('dog_id', Integer, primary_key=True),
- Column('breed', String),
- Column('name', String))
-
- class Dog(object):
- pass
-
- assign_mapper(Dog, dogs)
-
- engine.connect(testbase.db_uri)
- dogs.create()
- try:
- spot = Dog()
- spot.breed = 'beagle'
- spot.name = 'Spot'
+ cats.create(engine)
+ cats.drop(engine)
- rover = Dog()
- rover.breed = 'spaniel'
- rover.name = 'Rover'
-
- objectstore.commit()
-
- assert spot.dog_id > 0, "Spot did not get an id"
- assert rover.dog_id != spot.dog_id
- finally:
- dogs.drop()
-
- def test_type_proxy_schema_gen(self):
- from sqlalchemy.databases.postgres import PGSchemaGenerator
-
- engine = ProxyEngine()
- lizards = Table('lizards', engine,
- Column('id', Integer, primary_key=True),
- Column('name', String))
-
- # this doesn't really CONNECT to pg, just establishes pg as the
- # actual engine so that we can determine that it gets the right
- # answer
- engine.connect('postgres://database=test&port=5432&host=127.0.0.1&user=scott&password=tiger')
-
- sg = PGSchemaGenerator(engine)
- id_spec = sg.get_column_specification(lizards.c.id)
- assert id_spec == 'id SERIAL NOT NULL PRIMARY KEY'
-
-
if __name__ == "__main__":
testbase.main()
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
import tables
db = testbase.db
-#db.echo='debug'
from sqlalchemy import *
from sqlalchemy.engine import ResultProxy, RowProxy
def testdelete(self):
- c = db.connection()
-
self.users.insert().execute(user_id = 7, user_name = 'jack')
self.users.insert().execute(user_id = 8, user_name = 'fred')
print repr(self.users.select().execute().fetchall())
print repr(self.users.select().execute().fetchall())
- def testtransaction(self):
- def dostuff():
- self.users.insert().execute(user_id = 7, user_name = 'john')
- self.users.insert().execute(user_id = 8, user_name = 'jack')
-
- db.transaction(dostuff)
- print repr(self.users.select().execute().fetchall())
-
def testselectlimit(self):
self.users.insert().execute(user_id=1, user_name='john')
self.users.insert().execute(user_id=2, user_name='jack')
self.users.insert().execute(user_id=1, user_name='foo')
r = self.users.select().execute().fetchone()
self.assertEqual(len(r), 2)
+ r.close()
r = db.execute('select user_name, user_id from query_users', {}).fetchone()
self.assertEqual(len(r), 2)
+ r.close()
r = db.execute('select user_name from query_users', {}).fetchone()
self.assertEqual(len(r), 1)
+ r.close()
def test_column_order_with_simple_query(self):
# should return values in column definition order
self.assertEqual(r[1], 1)
self.assertEqual(r.keys(), ['user_name', 'user_id'])
self.assertEqual(r.values(), ['foo', 1])
-
+
+ @testbase.unsupported('oracle')
def test_column_accessor_shadow(self):
- if db.engine.__module__.endswith('oracle'):
- return
-
shadowed = Table('test_shadowed', db,
Column('shadow_id', INT, primary_key = True),
Column('shadow_name', VARCHAR(20)),
self.fail('Should not allow access to private attributes')
except AttributeError:
pass # expected
+ r.close()
finally:
shadowed.drop()
import sqlalchemy.ansisql as ansisql
import sqlalchemy.databases.postgres as postgres
-import sqlalchemy.databases.oracle as oracle
-import sqlalchemy.databases.sqlite as sqllite
from sqlalchemy import *
def testbasic(self):
# really trip it up with a circular reference
- use_function_defaults = testbase.db.engine.__module__.endswith('postgres') or testbase.db.engine.__module__.endswith('oracle')
+ use_function_defaults = testbase.db.engine.name == 'postgres' or testbase.db.engine.name == 'oracle'
use_string_defaults = use_function_defaults or testbase.db.engine.__module__.endswith('sqlite')
table.drop()
def testtoengine(self):
- db = ansisql.engine()
+ meta = MetaData('md1')
+ meta2 = MetaData('md2')
- table = Table('mytable', db,
+ table = Table('mytable', meta,
Column('myid', Integer, key = 'id'),
Column('name', String, key = 'name', nullable=False),
Column('description', String, key = 'description'),
print repr(table)
- pgdb = postgres.engine({})
+ table2 = table.tometadata(meta2)
- pgtable = table.toengine(pgdb)
+ print repr(table2)
- print repr(pgtable)
- assert pgtable.c.id.nullable
- assert not pgtable.c.name.nullable
- assert pgtable.c.description.nullable
+ assert table is not table2
+ assert table2.c.id.nullable
+ assert not table2.c.name.nullable
+ assert table2.c.description.nullable
def testoverride(self):
table = Table(
self.assert_(isinstance(table.c.col4.type, String))
finally:
table.drop()
+
+class CreateDropTest(PersistTest):
+ def setUpAll(self):
+ global metadata
+ metadata = MetaData()
+ users = Table('users', metadata,
+ Column('user_id', Integer, Sequence('user_id_seq', optional=True), primary_key = True),
+ Column('user_name', String(40)),
+ )
+
+ addresses = Table('email_addresses', metadata,
+ Column('address_id', Integer, Sequence('address_id_seq', optional=True), primary_key = True),
+ Column('user_id', Integer, ForeignKey(users.c.user_id)),
+ Column('email_address', String(40)),
+
+ )
+
+ orders = Table('orders', metadata,
+ Column('order_id', Integer, Sequence('order_id_seq', optional=True), primary_key = True),
+ Column('user_id', Integer, ForeignKey(users.c.user_id)),
+ Column('description', String(50)),
+ Column('isopen', Integer),
+
+ )
+
+ orderitems = Table('items', metadata,
+ Column('item_id', INT, Sequence('items_id_seq', optional=True), primary_key = True),
+ Column('order_id', INT, ForeignKey("orders")),
+ Column('item_name', VARCHAR(50)),
+
+ )
+
+ def test_sorter( self ):
+ tables = metadata._sort_tables(metadata.tables.values())
+ table_names = [t.name for t in tables]
+ self.assert_( table_names == ['users', 'orders', 'items', 'email_addresses'] or table_names == ['users', 'email_addresses', 'orders', 'items'])
+
+
+ def test_createdrop(self):
+ metadata.create_all(engine=testbase.db)
+ self.assertEqual( testbase.db.has_table('items'), True )
+ self.assertEqual( testbase.db.has_table('email_addresses'), True )
+ metadata.create_all(engine=testbase.db)
+ self.assertEqual( testbase.db.has_table('items'), True )
+
+ metadata.drop_all(engine=testbase.db)
+ self.assertEqual( testbase.db.has_table('items'), False )
+ self.assertEqual( testbase.db.has_table('email_addresses'), False )
+ metadata.drop_all(engine=testbase.db)
+ self.assertEqual( testbase.db.has_table('items'), False )
+
+
if __name__ == "__main__":
testbase.main()
global tbl_b
global tbl_c
global tbl_d
- tbl_a = Table("tbl_a", db,
+ metadata = MetaData()
+ tbl_a = Table("tbl_a", metadata,
Column("id", Integer, primary_key=True),
Column("name", String),
)
- tbl_b = Table("tbl_b", db,
+ tbl_b = Table("tbl_b", metadata,
Column("id", Integer, primary_key=True),
Column("name", String),
)
- tbl_c = Table("tbl_c", db,
+ tbl_c = Table("tbl_c", metadata,
Column("id", Integer, primary_key=True),
Column("tbl_a_id", Integer, ForeignKey("tbl_a.id"), nullable=False),
Column("name", String),
)
- tbl_d = Table("tbl_d", db,
+ tbl_d = Table("tbl_d", metadata,
Column("id", Integer, primary_key=True),
Column("tbl_c_id", Integer, ForeignKey("tbl_c.id"), nullable=False),
Column("tbl_b_id", Integer, ForeignKey("tbl_b.id")),
Column("name", String),
)
def setUp(self):
- tbl_a.create()
- tbl_b.create()
- tbl_c.create()
- tbl_d.create()
-
- objectstore.clear()
- clear_mappers()
+ global session
+ session = create_session(bind_to=testbase.db)
+ conn = session.connect()
+ conn.create(tbl_a)
+ conn.create(tbl_b)
+ conn.create(tbl_c)
+ conn.create(tbl_d)
class A(object):
pass
b = B(); b.name = "b1"
c = C(); c.name = "c1"; c.a_row = a
# we must have more than one d row or it won't fail
- d = D(); d.name = "d1"; d.b_row = b; d.c_row = c
- d = D(); d.name = "d2"; d.b_row = b; d.c_row = c
- d = D(); d.name = "d3"; d.b_row = b; d.c_row = c
-
+ d1 = D(); d1.name = "d1"; d1.b_row = b; d1.c_row = c
+ d2 = D(); d2.name = "d2"; d2.b_row = b; d2.c_row = c
+ d3 = D(); d3.name = "d3"; d3.b_row = b; d3.c_row = c
+ session.save_or_update(a)
+ session.save_or_update(b)
+
def tearDown(self):
- tbl_d.drop()
- tbl_c.drop()
- tbl_b.drop()
- tbl_a.drop()
+ conn = session.connect()
+ conn.drop(tbl_d)
+ conn.drop(tbl_c)
+ conn.drop(tbl_b)
+ conn.drop(tbl_a)
def tearDownAll(self):
- testbase.db.tables.clear()
+ testbase.metadata.tables.clear()
def testDeleteRootTable(self):
- session = objectstore.get_session()
- session.commit()
+ session.flush()
session.delete(a) # works as expected
- session.commit()
-
+ session.flush()
+
def testDeleteMiddleTable(self):
- session = objectstore.get_session()
- session.commit()
+ session.flush()
session.delete(c) # fails
- session.commit()
+ session.flush()
if __name__ == "__main__":
from sqlalchemy import *
-import sqlalchemy.ansisql as ansisql
-import sqlalchemy.databases.postgres as postgres
-import sqlalchemy.databases.oracle as oracle
-import sqlalchemy.databases.sqlite as sqlite
-import sqlalchemy.databases.mysql as mysql
-
-db = ansisql.engine()
-#db = create_engine('mssql')
-
+from sqlalchemy.databases import sqlite, postgres, mysql, oracle
from testbase import PersistTest
import unittest, re
column('otherstuff'),
)
+metadata = MetaData()
table4 = Table(
- 'remotetable', db,
+ 'remotetable', metadata,
Column('rem_id', Integer, primary_key=True),
Column('datatype_id', Integer),
Column('value', String(20)),
)
class SQLTest(PersistTest):
- def runtest(self, clause, result, engine = None, params = None, checkparams = None):
- c = clause.compile(parameters=params, engine=engine)
+ def runtest(self, clause, result, dialect = None, params = None, checkparams = None):
+ c = clause.compile(parameters=params, dialect=dialect)
self.echo("\nSQL String:\n" + str(c) + repr(c.get_params()))
cc = re.sub(r'\n', '', str(c))
self.assert_(cc == result, str(c) + "\n does not match \n" + result)
"""tests placing select statements in the column clause of another select, for the
purposes of selecting from the exported columns of that select."""
s = select([table1], table1.c.name == 'jack')
- #print [key for key in s.c.keys()]
self.runtest(
select(
[s],
self.runtest(
select([users, s.c.street], from_obj=[s]),
"""SELECT users.user_id, users.user_name, users.password, s.street FROM users, (SELECT addresses.street AS street FROM addresses WHERE addresses.user_id = users.user_id) AS s""")
-
def testcolumnsubquery(self):
s = select([table1.c.myid], scalar=True, correlate=False)
)
self.runtest(
- literal("a") + literal("b") * literal("c"), ":literal + (:liter_1 * :liter_2)", db
+ literal("a") + literal("b") * literal("c"), ":literal + (:liter_1 * :liter_2)"
)
def testmultiparam(self):
)
def testoraclelimit(self):
- e = create_engine('oracle')
- users = Table('users', e, Column('name', String(10), key='username'))
- self.runtest(select([users.c.username], limit=5), "SELECT name FROM (SELECT users.name AS name, ROW_NUMBER() OVER (ORDER BY users.rowid ASC) AS ora_rn FROM users) WHERE ora_rn<=5", engine=e)
+ metadata = MetaData()
+ users = Table('users', metadata, Column('name', String(10), key='username'))
+ self.runtest(select([users.c.username], limit=5), "SELECT name FROM (SELECT users.name AS name, ROW_NUMBER() OVER (ORDER BY users.rowid) AS ora_rn FROM users) WHERE ora_rn<=5", dialect=oracle.dialect())
def testgroupby_and_orderby(self):
self.runtest(
def testtext(self):
self.runtest(
text("select * from foo where lala = bar") ,
- "select * from foo where lala = bar",
- engine = db
+ "select * from foo where lala = bar"
)
self.runtest(select(
["foobar(a)", "pk_foo_bar(syslaal)"],
"a = 12",
- from_obj = ["foobar left outer join lala on foobar.foo = lala.foo"],
- engine = db
+ from_obj = ["foobar left outer join lala on foobar.foo = lala.foo"]
),
"SELECT foobar(a), pk_foo_bar(syslaal) FROM foobar left outer join lala on foobar.foo = lala.foo WHERE a = 12")
s.append_whereclause("column2=19")
s.order_by("column1")
s.append_from("table1")
- self.runtest(s, "SELECT column1, column2 FROM table1 WHERE column1=12 AND column2=19 ORDER BY column1", db)
+ self.runtest(s, "SELECT column1, column2 FROM table1 WHERE column1=12 AND column2=19 ORDER BY column1")
def testtextbinds(self):
self.runtest(
- db.text("select * from foo where lala=:bar and hoho=:whee"),
+ text("select * from foo where lala=:bar and hoho=:whee"),
"select * from foo where lala=:bar and hoho=:whee",
checkparams={'bar':4, 'whee': 7},
params={'bar':4, 'whee': 7, 'hoho':10},
- engine=db
)
- engine = postgres.engine({})
+ dialect = postgres.dialect()
self.runtest(
- engine.text("select * from foo where lala=:bar and hoho=:whee"),
+ text("select * from foo where lala=:bar and hoho=:whee"),
"select * from foo where lala=%(bar)s and hoho=%(whee)s",
checkparams={'bar':4, 'whee': 7},
params={'bar':4, 'whee': 7, 'hoho':10},
- engine=engine
+ dialect=dialect
)
- engine = sqlite.engine({})
+ dialect = sqlite.dialect()
self.runtest(
- engine.text("select * from foo where lala=:bar and hoho=:whee"),
+ text("select * from foo where lala=:bar and hoho=:whee"),
"select * from foo where lala=? and hoho=?",
checkparams=[4, 7],
params={'bar':4, 'whee': 7, 'hoho':10},
- engine=engine
+ dialect=dialect
)
def testtextmix(self):
"SELECT foo.bar.lala(:lala)")
# test a dotted func off the engine itself
- self.runtest(db.func.lala.hoho(7), "lala.hoho(:hoho)")
+ self.runtest(func.lala.hoho(7), "lala.hoho(:hoho)")
def testjoin(self):
self.runtest(
FROM mytable UNION SELECT myothertable.otherid, myothertable.othername \
FROM myothertable UNION SELECT thirdtable.userid, thirdtable.otherstuff FROM thirdtable")
+ u = union(
+ select([table1]),
+ select([table2]),
+ select([table3])
+ )
+ assert u.corresponding_column(table2.c.otherid) is u.c.otherid
+
def testouterjoin(self):
# test an outer join. the oracle module should take the ON clause of the join and
WHERE mytable.name = %(mytable_name)s AND mytable.myid = %(mytable_myid)s AND \
myothertable.othername != %(myothertable_othername)s AND \
EXISTS (select yay from foo where boo = lar)",
- engine = postgres.engine({}))
-
+ dialect=postgres.dialect()
+ )
self.runtest(query,
"SELECT mytable.myid, mytable.name, mytable.description, myothertable.otherid, myothertable.othername \
FROM mytable, myothertable WHERE mytable.myid = myothertable.otherid(+) AND \
mytable.name = :mytable_name AND mytable.myid = :mytable_myid AND \
myothertable.othername != :myothertable_othername AND EXISTS (select yay from foo where boo = lar)",
- engine = oracle.engine({}, use_ansi = False))
+ dialect=oracle.OracleDialect(use_ansi = False))
query = table1.outerjoin(table2, table1.c.myid==table2.c.otherid).outerjoin(table3, table3.c.userid==table2.c.otherid)
self.runtest(query.select(), "SELECT mytable.myid, mytable.name, mytable.description, myothertable.otherid, myothertable.othername, thirdtable.userid, thirdtable.otherstuff FROM mytable LEFT OUTER JOIN myothertable ON mytable.myid = myothertable.otherid LEFT OUTER JOIN thirdtable ON thirdtable.userid = myothertable.otherid")
- self.runtest(query.select(), "SELECT mytable.myid, mytable.name, mytable.description, myothertable.otherid, myothertable.othername, thirdtable.userid, thirdtable.otherstuff FROM mytable, myothertable, thirdtable WHERE mytable.myid = myothertable.otherid(+) AND thirdtable.userid(+) = myothertable.otherid", engine=oracle.engine({}, use_ansi=False))
+ self.runtest(query.select(), "SELECT mytable.myid, mytable.name, mytable.description, myothertable.otherid, myothertable.othername, thirdtable.userid, thirdtable.otherstuff FROM mytable, myothertable, thirdtable WHERE mytable.myid = myothertable.otherid(+) AND thirdtable.userid(+) = myothertable.otherid", dialect=oracle.dialect(use_ansi=False))
def testbindparam(self):
self.runtest(select(
# check that the bind params sent along with a compile() call
# get preserved when the params are retreived later
s = select([table1], table1.c.myid == bindparam('test'))
- c = s.compile(parameters = {'test' : 7}, engine=db)
+ c = s.compile(parameters = {'test' : 7})
self.assert_(c.get_params() == {'test' : 7})
Column('ts', TIMESTAMP),
)
- def check_results(engine, expected_results, literal):
+ def check_results(dialect, expected_results, literal):
self.assertEqual(len(expected_results), 5, 'Incorrect number of expected results')
- self.assertEqual(str(cast(tbl.c.v1, Numeric, engine=engine)), 'CAST(casttest.v1 AS %s)' %expected_results[0])
- self.assertEqual(str(cast(tbl.c.v1, Numeric(12, 9), engine=engine)), 'CAST(casttest.v1 AS %s)' %expected_results[1])
- self.assertEqual(str(cast(tbl.c.ts, Date, engine=engine)), 'CAST(casttest.ts AS %s)' %expected_results[2])
- self.assertEqual(str(cast(1234, TEXT, engine=engine)), 'CAST(%s AS %s)' %(literal, expected_results[3]))
- self.assertEqual(str(cast('test', String(20), engine=engine)), 'CAST(%s AS %s)' %(literal, expected_results[4]))
-
- sel = select([tbl, cast(tbl.c.v1, Numeric)], engine=engine)
- self.assertEqual(str(sel), "SELECT casttest.id, casttest.v1, casttest.v2, casttest.ts, CAST(casttest.v1 AS NUMERIC(10, 2)) \nFROM casttest")
-
+ self.assertEqual(str(cast(tbl.c.v1, Numeric).compile(dialect=dialect)), 'CAST(casttest.v1 AS %s)' %expected_results[0])
+ self.assertEqual(str(cast(tbl.c.v1, Numeric(12, 9)).compile(dialect=dialect)), 'CAST(casttest.v1 AS %s)' %expected_results[1])
+ self.assertEqual(str(cast(tbl.c.ts, Date).compile(dialect=dialect)), 'CAST(casttest.ts AS %s)' %expected_results[2])
+ self.assertEqual(str(cast(1234, TEXT).compile(dialect=dialect)), 'CAST(%s AS %s)' %(literal, expected_results[3]))
+ self.assertEqual(str(cast('test', String(20)).compile(dialect=dialect)), 'CAST(%s AS %s)' %(literal, expected_results[4]))
+ sel = select([tbl, cast(tbl.c.v1, Numeric)]).compile(dialect=dialect)
+ self.assertEqual(str(sel), "SELECT casttest.id, casttest.v1, casttest.v2, casttest.ts, CAST(casttest.v1 AS NUMERIC(10, 2)) \nFROM casttest")
# first test with Postgres engine
- check_results(postgres.engine({}), ['NUMERIC(10, 2)', 'NUMERIC(12, 9)', 'DATE', 'TEXT', 'VARCHAR(20)'], '%(literal)s')
+ check_results(postgres.dialect(), ['NUMERIC(10, 2)', 'NUMERIC(12, 9)', 'DATE', 'TEXT', 'VARCHAR(20)'], '%(literal)s')
# then the Oracle engine
- check_results(oracle.engine({}, use_ansi = False), ['NUMERIC(10, 2)', 'NUMERIC(12, 9)', 'DATE', 'CLOB', 'VARCHAR(20)'], ':literal')
+# check_results(oracle.OracleDialect(), ['NUMERIC(10, 2)', 'NUMERIC(12, 9)', 'DATE', 'CLOB', 'VARCHAR(20)'], ':literal')
# then the sqlite engine
- check_results(sqlite.engine({}), ['NUMERIC(10, 2)', 'NUMERIC(12, 9)', 'DATE', 'TEXT', 'VARCHAR(20)'], '?')
+ check_results(sqlite.dialect(), ['NUMERIC(10, 2)', 'NUMERIC(12, 9)', 'DATE', 'TEXT', 'VARCHAR(20)'], '?')
# and the MySQL engine
- check_results(mysql.engine({}), ['NUMERIC(10, 2)', 'NUMERIC(12, 9)', 'DATE', 'TEXT', 'VARCHAR(20)'], '%s')
+ check_results(mysql.dialect(), ['NUMERIC(10, 2)', 'NUMERIC(12, 9)', 'DATE', 'TEXT', 'VARCHAR(20)'], '%s')
class CRUDTest(SQLTest):
def testinsert(self):
self.runtest(update(table1, table1.c.myid == 12, values = {table1.c.name : table1.c.myid}), "UPDATE mytable SET name=mytable.myid, description=:description WHERE mytable.myid = :mytable_myid", params = {'description':'test'})
self.runtest(update(table1, table1.c.myid == 12, values = {table1.c.myid : 9}), "UPDATE mytable SET myid=:myid, description=:description WHERE mytable.myid = :mytable_myid", params = {'mytable_myid': 12, 'myid': 9, 'description': 'test'})
s = table1.update(table1.c.myid == 12, values = {table1.c.name : 'lala'})
- c = s.compile(parameters = {'mytable_id':9,'name':'h0h0'}, engine=db)
- print str(c)
+ c = s.compile(parameters = {'mytable_id':9,'name':'h0h0'})
self.assert_(str(s) == str(c))
def testupdateexpression(self):
s = select([table2], table2.c.otherid == table1.c.myid)
u = update(table1, table1.c.name == 'jack', values = {table1.c.name : s})
self.runtest(u, "UPDATE mytable SET name=(SELECT myothertable.otherid, myothertable.othername FROM myothertable WHERE myothertable.otherid = mytable.myid) WHERE mytable.name = :mytable_name")
-
+
# test a correlated WHERE clause
s = select([table2.c.othername], table2.c.otherid == 7)
u = update(table1, table1.c.name==s)
Column('col1', Integer, primary_key=True),\r
Column('col2', String(20)),\r
Column('col3', Integer),\r
+ Column('colx', Integer),\r
redefine=True\r
)\r
\r
Column('col1', Integer, primary_key=True),\r
Column('col2', Integer, ForeignKey('table1.col1')),\r
Column('col3', String(20)),\r
+ Column('coly', Integer),\r
redefine=True\r
)\r
\r
-table3 = Table('table3', db, \r
- Column('col1', Integer, ForeignKey('table1.col1'), primary_key=True),\r
- Column('col2', Integer),\r
- Column('col3', String(20)),\r
- redefine=True\r
- )\r
-\r
class SelectableTest(testbase.AssertMixin):\r
def testtablealias(self):\r
a = table.alias('a')\r
print str(j)\r
self.assert_(criterion.compare(j.onclause))\r
\r
- def testjoinpks(self):\r
- a = join(table, table3)\r
- b = join(table, table3, table.c.col1==table3.c.col2)\r
- c = join(table, table3, table.c.col2==table3.c.col2)\r
- d = join(table, table3, table.c.col2==table3.c.col1)\r
- \r
- self.assert_(a.primary_key==[table.c.col1])\r
- self.assert_(b.primary_key==[table.c.col1, table3.c.col1])\r
- self.assert_(c.primary_key==[table.c.col1, table3.c.col1])\r
- self.assert_(d.primary_key==[table.c.col1, table3.c.col1])\r
+ def testunion(self):\r
+ # tests that we can correspond a column in a Select statement with a certain Table, against\r
+ # a column in a Union where one of its underlying Selects matches to that same Table\r
+ u = select([table.c.col1, table.c.col2, table.c.col3, table.c.colx, null().label('coly')]).union(\r
+ select([table2.c.col1, table2.c.col2, table2.c.col3, null().label('colx'), table2.c.coly])\r
+ )\r
+ s1 = table.select(use_labels=True)\r
+ s2 = table2.select(use_labels=True)\r
+ print ["%d %s" % (id(c),c.key) for c in u.c]\r
+ c = u.corresponding_column(s1.c.table1_col2)\r
+ print "%d %s" % (id(c), c.key)\r
+ assert u.corresponding_column(s1.c.table1_col2) is u.c.col2\r
+ assert u.corresponding_column(s2.c.table2_col2) is u.c.col2\r
+\r
+ def testaliasunion(self):\r
+ # same as testunion, except its an alias of the union\r
+ u = select([table.c.col1, table.c.col2, table.c.col3, table.c.colx, null().label('coly')]).union(\r
+ select([table2.c.col1, table2.c.col2, table2.c.col3, null().label('colx'), table2.c.coly])\r
+ ).alias('analias')\r
+ s1 = table.select(use_labels=True)\r
+ s2 = table2.select(use_labels=True)\r
+ assert u.corresponding_column(s1.c.table1_col2) is u.c.col2\r
+ assert u.corresponding_column(s2.c.table2_col2) is u.c.col2\r
+ assert u.corresponding_column(s2.c.table2_coly) is u.c.coly\r
+ assert s2.corresponding_column(u.c.coly) is s2.c.table2_coly\r
+\r
+ def testselectunion(self):\r
+ # like testaliasunion, but off a Select off the union.\r
+ u = select([table.c.col1, table.c.col2, table.c.col3, table.c.colx, null().label('coly')]).union(\r
+ select([table2.c.col1, table2.c.col2, table2.c.col3, null().label('colx'), table2.c.coly])\r
+ ).alias('analias')\r
+ s = select([u])\r
+ s1 = table.select(use_labels=True)\r
+ s2 = table2.select(use_labels=True)\r
+ assert s.corresponding_column(s1.c.table1_col2) is s.c.col2\r
+ assert s.corresponding_column(s2.c.table2_col2) is s.c.col2\r
+\r
+ def testunionagainstjoin(self):\r
+ # same as testunion, except its an alias of the union\r
+ u = select([table.c.col1, table.c.col2, table.c.col3, table.c.colx, null().label('coly')]).union(\r
+ select([table2.c.col1, table2.c.col2, table2.c.col3, null().label('colx'), table2.c.coly])\r
+ ).alias('analias')\r
+ j1 = table.join(table2)\r
+ assert u.corresponding_column(j1.c.table1_colx) is u.c.colx\r
+ assert j1.corresponding_column(u.c.colx) is j1.c.table1_colx\r
\r
def testjoin(self):\r
a = join(table, table2)\r
from sqlalchemy import *
-from sqlalchemy.mods.selectresults import SelectResultsExt
+from sqlalchemy.ext.selectresults import SelectResultsExt
class Foo(object):
pass
class SelectResultsTest(PersistTest):
def setUpAll(self):
+ self.install_threadlocal()
global foo
foo = Table('foo', testbase.db,
Column('id', Integer, Sequence('foo_id_seq'), primary_key=True),
- Column('bar', Integer))
+ Column('bar', Integer),
+ Column('range', Integer))
assign_mapper(Foo, foo, extension=SelectResultsExt())
foo.create()
for i in range(100):
- Foo(bar=i)
- objectstore.commit()
+ Foo(bar=i, range=i%10)
+ objectstore.flush()
def setUp(self):
- self.orig = Foo.mapper.select_whereclause()
- self.res = Foo.select()
+ self.query = Foo.mapper.query()
+ self.orig = self.query.select_whereclause()
+ self.res = self.query.select()
def tearDownAll(self):
global foo
foo.drop()
+ self.uninstall_threadlocal()
+
+ def test_selectby(self):
+ res = self.query.select_by(range=5)
+ assert res.order_by([Foo.c.bar])[0].bar == 5
+ assert res.order_by([desc(Foo.c.bar)])[0].bar == 95
def test_slice(self):
assert self.res[1] == self.orig[1]
--- /dev/null
+
+# test merging a composed object.
+
+# test that when cascading an operation, like "merge", lazy-loaded scalar and list attributes that werent already loaded on the given object remain not loaded.
+
+# test putting an object in session A, "moving" it to session B, insure its in B and not in A
+
--- /dev/null
+from testbase import PersistTest, AssertMixin
+import unittest, sys, os
+from sqlalchemy.ext.sessioncontext import SessionContext
+from sqlalchemy.orm.session import object_session, Session
+from sqlalchemy import *
+import testbase
+
+metadata = MetaData()
+users = Table('users', metadata,
+ Column('user_id', Integer, Sequence('user_id_seq', optional=True), primary_key = True),
+ Column('user_name', String(40)),
+ mysql_engine='innodb'
+)
+
+class SessionContextTest(AssertMixin):
+ def setUp(self):
+ clear_mappers()
+
+ def do_test(self, class_, context):
+ """test session assignment on object creation"""
+ obj = class_()
+ assert context.current == object_session(obj)
+
+ # keep a reference so the old session doesn't get gc'd
+ old_session = context.current
+
+ context.current = Session()
+ assert context.current != object_session(obj)
+ assert old_session == object_session(obj)
+
+ new_session = context.current
+ del context.current
+ assert context.current != new_session
+ assert old_session == object_session(obj)
+
+ obj2 = class_()
+ assert context.current == object_session(obj2)
+
+ def test_mapper_extension(self):
+ context = SessionContext(Session)
+ class User(object): pass
+ User.mapper = mapper(User, users, extension=context.mapper_extension)
+ self.do_test(User, context)
+
+
+if __name__ == "__main__":
+ testbase.main()
ECHO = testbase.echo
db = testbase.db
+metadata = BoundMetaData(db)
-
-users = Table('users', db,
+users = Table('users', metadata,
Column('user_id', Integer, Sequence('user_id_seq', optional=True), primary_key = True),
Column('user_name', String(40)),
mysql_engine='innodb'
)
-addresses = Table('email_addresses', db,
+addresses = Table('email_addresses', metadata,
Column('address_id', Integer, Sequence('address_id_seq', optional=True), primary_key = True),
Column('user_id', Integer, ForeignKey(users.c.user_id)),
Column('email_address', String(40)),
)
-orders = Table('orders', db,
+orders = Table('orders', metadata,
Column('order_id', Integer, Sequence('order_id_seq', optional=True), primary_key = True),
Column('user_id', Integer, ForeignKey(users.c.user_id)),
Column('description', String(50)),
)
-orderitems = Table('items', db,
+orderitems = Table('items', metadata,
Column('item_id', INT, Sequence('items_id_seq', optional=True), primary_key = True),
Column('order_id', INT, ForeignKey("orders")),
Column('item_name', VARCHAR(50)),
)
-keywords = Table('keywords', db,
+keywords = Table('keywords', metadata,
Column('keyword_id', Integer, Sequence('keyword_id_seq', optional=True), primary_key = True),
Column('name', VARCHAR(50)),
)
-itemkeywords = Table('itemkeywords', db,
+itemkeywords = Table('itemkeywords', metadata,
Column('item_id', INT, ForeignKey("items")),
Column('keyword_id', INT, ForeignKey("keywords")),
)
def create():
- users.create()
- addresses.create()
- orders.create()
- orderitems.create()
- keywords.create()
- itemkeywords.create()
-
+ metadata.create_all()
def drop():
- itemkeywords.drop()
- keywords.drop()
- orderitems.drop()
- orders.drop()
- addresses.drop()
- users.drop()
- db.commit()
-
+ metadata.drop_all()
def delete():
- itemkeywords.delete().execute()
- keywords.delete().execute()
- orderitems.delete().execute()
- orders.delete().execute()
- addresses.delete().execute()
- users.delete().execute()
- db.commit()
-
+ for t in metadata.table_iterator(reverse=True):
+ t.delete().execute()
def user_data():
users.insert().execute(
dict(user_id = 7, user_name = 'jack'),
)
def delete_user_data():
users.delete().execute()
- db.commit()
def data():
delete()
dict(keyword_id=7, item_id=2),
dict(keyword_id=6, item_id=3)
)
-
- db.commit()
class User(object):
def __init__(self):
import StringIO
import sqlalchemy.engine as engine
import sqlalchemy.ext.proxy as proxy
-import sqlalchemy.schema as schema
+import sqlalchemy.pool as pool
+#import sqlalchemy.schema as schema
import re, sys
+import sqlalchemy
+import optparse
+
-echo = True
-#echo = False
-#echo = 'debug'
db = None
+metadata = None
db_uri = None
+echo = True
+
+# redefine sys.stdout so all those print statements go to the echo func
+local_stdout = sys.stdout
+class Logger(object):
+ def write(self, msg):
+ if echo:
+ local_stdout.write(msg)
+sys.stdout = Logger()
+
+def echo_text(text):
+ print text
def parse_argv():
# we are using the unittest main runner, so we are just popping out the
# arguments we need instead of using our own getopt type of thing
- global db, db_uri
+ global db, db_uri, metadata
DBTYPE = 'sqlite'
PROXY = False
+
+
+ parser = optparse.OptionParser(usage = "usage: %prog [options] files...")
+ parser.add_option("--dburi", action="store", dest="dburi", help="database uri (overrides --db)")
+ parser.add_option("--db", action="store", dest="db", default="sqlite", help="prefab database uri (sqlite, sqlite_file, postgres, mysql, oracle, oracle8, mssql)")
+ parser.add_option("--mockpool", action="store_true", dest="mockpool", help="use mock pool")
+ parser.add_option("--verbose", action="store_true", dest="verbose", help="full debug echoing")
+ parser.add_option("--quiet", action="store_true", dest="quiet", help="be totally quiet")
+ parser.add_option("--nothreadlocal", action="store_true", dest="nothreadlocal", help="dont use thread-local mod")
+ parser.add_option("--enginestrategy", action="store", default=None, dest="enginestrategy", help="engine strategy (plain or threadlocal, defaults to SA default)")
+
+ (options, args) = parser.parse_args()
+ sys.argv[1:] = args
- if len(sys.argv) >= 3:
- if sys.argv[1] == '--dburi':
- (param, db_uri) = (sys.argv.pop(1), sys.argv.pop(1))
- elif sys.argv[1] == '--db':
- (param, DBTYPE) = (sys.argv.pop(1), sys.argv.pop(1))
+ if options.dburi:
+ db_uri = param = options.dburi
+ elif options.db:
+ DBTYPE = param = options.db
+
opts = {}
if (None == db_uri):
- p = DBTYPE.split('.')
- if len(p) > 1:
- arg = p[0]
- DBTYPE = p[1]
- if arg == 'proxy':
- PROXY = True
if DBTYPE == 'sqlite':
- db_uri = 'sqlite://filename=:memory:'
+ db_uri = 'sqlite:///:memory:'
elif DBTYPE == 'sqlite_file':
- db_uri = 'sqlite://filename=querytest.db'
+ db_uri = 'sqlite:///querytest.db'
elif DBTYPE == 'postgres':
- db_uri = 'postgres://database=test&port=5432&host=127.0.0.1&user=scott&password=tiger'
+ db_uri = 'postgres://scott:tiger@127.0.0.1:5432/test'
elif DBTYPE == 'mysql':
- db_uri = 'mysql://database=test&host=127.0.0.1&user=scott&password=tiger'
+ db_uri = 'mysql://scott:tiger@127.0.0.1/test'
elif DBTYPE == 'oracle':
- db_uri = 'oracle://user=scott&password=tiger'
+ db_uri = 'oracle://scott:tiger@127.0.0.1:1521'
elif DBTYPE == 'oracle8':
- db_uri = 'oracle://user=scott&password=tiger'
+ db_uri = 'oracle://scott:tiger@127.0.0.1:1521'
opts = {'use_ansi':False}
elif DBTYPE == 'mssql':
- db_uri = 'mssql://database=test&user=scott&password=tiger'
+ db_uri = 'mssql://scott:tiger@/test'
if not db_uri:
raise "Could not create engine. specify --db <sqlite|sqlite_file|postgres|mysql|oracle|oracle8|mssql> to test runner."
- if PROXY:
- db = proxy.ProxyEngine(echo=echo, default_ordering=True, **opts)
- db.connect(db_uri)
+ if not options.nothreadlocal:
+ __import__('sqlalchemy.mods.threadlocal')
+ sqlalchemy.mods.threadlocal.uninstall_plugin()
+
+ global echo
+ echo = options.verbose and not options.quiet
+
+ global quiet
+ quiet = options.quiet
+
+ if options.enginestrategy is not None:
+ opts['strategy'] = options.enginestrategy
+ if options.mockpool:
+ db = engine.create_engine(db_uri, echo=True, default_ordering=True, poolclass=MockPool, **opts)
else:
- db = engine.create_engine(db_uri, echo=echo, default_ordering=True, **opts)
+ db = engine.create_engine(db_uri, echo=True, default_ordering=True, **opts)
db = EngineAssert(db)
-
+ metadata = sqlalchemy.BoundMetaData(db)
+
def unsupported(*dbs):
"""a decorator that marks a test as unsupported by one or more database implementations"""
def decorate(func):
return lala
return decorate
-def echo_text(text):
- print text
class PersistTest(unittest.TestCase):
"""persist base class, provides default setUpAll, tearDownAll and echo functionality"""
def __init__(self, *args, **params):
unittest.TestCase.__init__(self, *args, **params)
def echo(self, text):
- if echo:
- echo_text(text)
+ echo_text(text)
+ def install_threadlocal(self):
+ sqlalchemy.mods.threadlocal.install_plugin()
+ def uninstall_threadlocal(self):
+ sqlalchemy.mods.threadlocal.uninstall_plugin()
def setUpAll(self):
pass
def tearDownAll(self):
pass
+ def shortDescription(self):
+ """overridden to not return docstrings"""
+ return None
+
+class MockPool(pool.Pool):
+ """this pool is hardcore about only one connection being used at a time."""
+ def __init__(self, creator, **params):
+ pool.Pool.__init__(self, **params)
+ self.connection = creator()
+ self._conn = self.connection
+
+ def status(self):
+ return "MockPool"
+
+ def do_return_conn(self, conn):
+ assert conn is self._conn and self.connection is None
+ self.connection = conn
+
+ def do_return_invalid(self):
+ raise "Invalid"
+ def do_get(self):
+ if getattr(self, 'breakpoint', False):
+ raise "breakpoint"
+ assert self.connection is not None
+ c = self.connection
+ self.connection = None
+ return c
class AssertMixin(PersistTest):
"""given a list-based structure of keys/properties which represent information within an object structure, and
"""decorates a SQLEngine object to match the incoming queries against a set of assertions."""
def __init__(self, engine):
self._engine = engine
- self.realexec = engine.post_exec
- self.realexec.im_self.post_exec = self.post_exec
+
+ self.real_execution_context = engine.dialect.create_execution_context
+ engine.dialect.create_execution_context = self.execution_context
+
self.logger = engine.logger
self.set_assert_list(None, None)
self.sql_count = 0
return self._engine
def set_engine(self, e):
self._engine = e
-# def __getattr__(self, key):
- # return getattr(self.engine, key)
def set_assert_list(self, unittest, list):
self.unittest = unittest
self.assert_list = list
def _set_echo(self, echo):
self.engine.echo = echo
echo = property(lambda s: s.engine.echo, _set_echo)
- def post_exec(self, proxy, compiled, parameters, **kwargs):
- self.engine.logger = self.logger
- statement = str(compiled)
- statement = re.sub(r'\n', '', statement)
-
- if self.assert_list is not None:
- item = self.assert_list[-1]
- if not isinstance(item, dict):
- item = self.assert_list.pop()
- else:
- # asserting a dictionary of statements->parameters
- # this is to specify query assertions where the queries can be in
- # multiple orderings
- if not item.has_key('_converted'):
- for key in item.keys():
- ckey = self.convert_statement(key)
- item[ckey] = item[key]
- if ckey != key:
- del item[key]
- item['_converted'] = True
- try:
- entry = item.pop(statement)
- if len(item) == 1:
- self.assert_list.pop()
- item = (statement, entry)
- except KeyError:
- self.unittest.assert_(False, "Testing for one of the following queries: %s, received '%s'" % (repr([k for k in item.keys()]), statement))
-
- (query, params) = item
- if callable(params):
- params = params()
-
- query = self.convert_statement(query)
-
- self.unittest.assert_(statement == query and (params is None or params == parameters), "Testing for query '%s' params %s, received '%s' with params %s" % (query, repr(params), statement, repr(parameters)))
- self.sql_count += 1
- return self.realexec(proxy, compiled, parameters, **kwargs)
+
+ def execution_context(self):
+ def post_exec(engine, proxy, compiled, parameters, **kwargs):
+ ctx = e
+ self.engine.logger = self.logger
+ statement = str(compiled)
+ statement = re.sub(r'\n', '', statement)
+ if self.assert_list is not None:
+ item = self.assert_list[-1]
+ if not isinstance(item, dict):
+ item = self.assert_list.pop()
+ else:
+ # asserting a dictionary of statements->parameters
+ # this is to specify query assertions where the queries can be in
+ # multiple orderings
+ if not item.has_key('_converted'):
+ for key in item.keys():
+ ckey = self.convert_statement(key)
+ item[ckey] = item[key]
+ if ckey != key:
+ del item[key]
+ item['_converted'] = True
+ try:
+ entry = item.pop(statement)
+ if len(item) == 1:
+ self.assert_list.pop()
+ item = (statement, entry)
+ except KeyError:
+ self.unittest.assert_(False, "Testing for one of the following queries: %s, received '%s'" % (repr([k for k in item.keys()]), statement))
+
+ (query, params) = item
+ if callable(params):
+ params = params(ctx)
+ if params is not None and isinstance(params, list) and len(params) == 1:
+ params = params[0]
+
+ query = self.convert_statement(query)
+ self.unittest.assert_(statement == query and (params is None or params == parameters), "Testing for query '%s' params %s, received '%s' with params %s" % (query, repr(params), statement, repr(parameters)))
+ self.sql_count += 1
+ return realexec(ctx, proxy, compiled, parameters, **kwargs)
+
+ e = self.real_execution_context()
+ realexec = e.post_exec
+ realexec.im_self.post_exec = post_exec
+ return e
+
def convert_statement(self, query):
- paramstyle = self.engine.paramstyle
+ paramstyle = self.engine.dialect.paramstyle
if paramstyle == 'named':
pass
elif paramstyle =='pyformat':
def runTests(suite):
- runner = unittest.TextTestRunner(verbosity = 2, descriptions =1)
+ runner = unittest.TextTestRunner(verbosity = quiet and 1 or 2)
runner.run(suite)
def main():
- unittest.main()
+ suite = unittest.TestLoader().loadTestsFromModule(__import__('__main__'))
+ runTests(suite)
import string,datetime, re, sys
from testbase import PersistTest, AssertMixin
import testbase
-
+import sqlalchemy.engine.url as url
+
db = testbase.db
class MyType(types.TypeEngine):
class AdaptTest(PersistTest):
def testadapt(self):
- e1 = create_engine('postgres://')
- e2 = create_engine('sqlite://')
- e3 = create_engine('mysql://')
+ e1 = url.URL('postgres').get_module().dialect()
+ e2 = url.URL('mysql').get_module().dialect()
+ e3 = url.URL('sqlite').get_module().dialect()
type = String(40)
- t1 = type.engine_impl(e1)
- t2 = type.engine_impl(e2)
- t3 = type.engine_impl(e3)
+ t1 = type.dialect_impl(e1)
+ t2 = type.dialect_impl(e2)
+ t3 = type.dialect_impl(e3)
assert t1 != t2
assert t2 != t3
assert t3 != t1
)
for aCol in testTable.c:
- self.assertEquals(expectedResults[aCol.name], db.schemagenerator().get_column_specification(aCol))
+ self.assertEquals(expectedResults[aCol.name], db.dialect.schemagenerator(db, None).get_column_specification(aCol))
class UnicodeTest(AssertMixin):
"""tests the Unicode type. also tests the TypeDecorator with instances in the types package."""
unicode_table.create()
def tearDownAll(self):
unicode_table.drop()
- def testwhereclause(self):
- l = unicode_table.select(unicode_table.c.unicode_data==u'this is also unicode').execute()
- def testmapperwhere(self):
- class Foo(object):pass
- m = mapper(Foo, unicode_table)
- l = m.get_by(unicode_data=unicode('this is also unicode'))
- l = m.get_by(plain_data=unicode('this is also unicode'))
def testbasic(self):
rawdata = 'Alors vous imaginez ma surprise, au lever du jour, quand une dr\xc3\xb4le de petit voix m\xe2\x80\x99a r\xc3\xa9veill\xc3\xa9. Elle disait: \xc2\xab S\xe2\x80\x99il vous pla\xc3\xaet\xe2\x80\xa6 dessine-moi un mouton! \xc2\xbb\n'
unicodedata = rawdata.decode('utf-8')
self.assert_(isinstance(x['unicode_data'], unicode) and x['unicode_data'] == unicodedata)
if isinstance(x['plain_data'], unicode):
# SQLLite returns even non-unicode data as unicode
- self.assert_(sys.modules[db.engine.__module__].descriptor()['name'] == 'sqlite')
+ self.assert_(db.name == 'sqlite')
self.echo("its sqlite !")
else:
self.assert_(not isinstance(x['plain_data'], unicode) and x['plain_data'] == rawdata)
def testengineparam(self):
"""tests engine-wide unicode conversion"""
- prev_unicode = db.engine.convert_unicode
+ prev_unicode = db.engine.dialect.convert_unicode
try:
- db.engine.convert_unicode = True
+ db.engine.dialect.convert_unicode = True
rawdata = 'Alors vous imaginez ma surprise, au lever du jour, quand une dr\xc3\xb4le de petit voix m\xe2\x80\x99a r\xc3\xa9veill\xc3\xa9. Elle disait: \xc2\xab S\xe2\x80\x99il vous pla\xc3\xaet\xe2\x80\xa6 dessine-moi un mouton! \xc2\xbb\n'
unicodedata = rawdata.decode('utf-8')
unicode_table.insert().execute(unicode_data=unicodedata, plain_data=rawdata)
self.assert_(isinstance(x['unicode_data'], unicode) and x['unicode_data'] == unicodedata)
self.assert_(isinstance(x['plain_data'], unicode) and x['plain_data'] == unicodedata)
finally:
- db.engine.convert_unicode = prev_unicode
-
+ db.engine.dialect.convert_unicode = prev_unicode
class Foo(object):
def __init__(self, moredata):
self.moredata = moredata
def __eq__(self, other):
return other.data == self.data and other.stuff == self.stuff and other.moredata==self.moredata
-
+
class BinaryTest(AssertMixin):
def setUpAll(self):
global binary_table
Column('data', Binary),
Column('data_slice', Binary(100)),
Column('misc', String(30)),
- Column('pickled', PickleType))
+ Column('pickled', PickleType)
+ )
binary_table.create()
def tearDownAll(self):
binary_table.drop()
def testbinary(self):
testobj1 = Foo('im foo 1')
testobj2 = Foo('im foo 2')
-
+
stream1 =self.get_module_stream('sqlalchemy.sql')
- stream2 =self.get_module_stream('sqlalchemy.engine')
+ stream2 =self.get_module_stream('sqlalchemy.schema')
binary_table.insert().execute(primary_id=1, misc='sql.pyc', data=stream1, data_slice=stream1[0:100], pickled=testobj1)
- binary_table.insert().execute(primary_id=2, misc='engine.pyc', data=stream2, data_slice=stream2[0:99], pickled=testobj2)
+ binary_table.insert().execute(primary_id=2, misc='schema.pyc', data=stream2, data_slice=stream2[0:99], pickled=testobj2)
l = binary_table.select().execute().fetchall()
- print type(l[0]['data'])
- return
print len(stream1), len(l[0]['data']), len(l[0]['data_slice'])
self.assert_(list(stream1) == list(l[0]['data']))
self.assert_(list(stream1[0:100]) == list(l[0]['data_slice']))
collist = [Column('user_id', INT, primary_key = True), Column('user_name', VARCHAR(20)), Column('user_datetime', DateTime),
Column('user_date', Date), Column('user_time', Time)]
- if db.engine.__module__.endswith('mysql') or db.engine.__module__.endswith('mssql'):
+ if db.engine.name == 'mysql' or db.engine.name == 'mssql':
# strip microseconds -- not supported by this engine (should be an easier way to detect this)
for d in insert_data:
if d[2] is not None:
--- /dev/null
+
+import testbase
+import unittest, sys, datetime
+import tables
+db = testbase.db
+from sqlalchemy import *
+
+class TransactionTest(testbase.PersistTest):
+ def setUpAll(self):
+ global users, metadata
+ metadata = MetaData()
+ users = Table('query_users', metadata,
+ Column('user_id', INT, primary_key = True),
+ Column('user_name', VARCHAR(20)),
+ )
+ users.create(testbase.db)
+
+ def tearDown(self):
+ testbase.db.connect().execute(users.delete())
+ def tearDownAll(self):
+ users.drop(testbase.db)
+
+ @testbase.unsupported('mysql')
+ def testrollback(self):
+ """test a basic rollback"""
+ connection = testbase.db.connect()
+ transaction = connection.begin()
+ connection.execute(users.insert(), user_id=1, user_name='user1')
+ connection.execute(users.insert(), user_id=2, user_name='user2')
+ connection.execute(users.insert(), user_id=3, user_name='user3')
+ transaction.rollback()
+
+ result = connection.execute("select * from query_users")
+ assert len(result.fetchall()) == 0
+ connection.close()
+
+class AutoRollbackTest(testbase.PersistTest):
+ def setUpAll(self):
+ global metadata
+ metadata = MetaData()
+
+ def tearDownAll(self):
+ metadata.drop_all(testbase.db)
+
+ def testrollback_deadlock(self):
+ """test that returning connections to the pool clears any object locks."""
+ conn1 = testbase.db.connect()
+ conn2 = testbase.db.connect()
+ users = Table('deadlock_users', metadata,
+ Column('user_id', INT, primary_key = True),
+ Column('user_name', VARCHAR(20)),
+ )
+ users.create(conn1)
+ conn1.execute("select * from deadlock_users")
+ conn1.close()
+ # without auto-rollback in the connection pool's return() logic, this deadlocks in Postgres,
+ # because conn1 is returned to the pool but still has a lock on "deadlock_users"
+ # comment out the rollback in pool/ConnectionFairy._close() to see !
+ users.drop(conn2)
+ conn2.close()
+
+if __name__ == "__main__":
+ testbase.main()
projects / thirdparty / sqlalchemy / sqlalchemy.git / commitdiff
