From: Michael Trier Date: Tue, 23 Feb 2010 06:15:24 +0000 (+0000) Subject: Major cleanup work in the docs to link class names into the API docs. Fixes #1702. X-Git-Tag: rel_0_6beta2~141 X-Git-Url: http://git.ipfire.org/gitweb.cgi?a=commitdiff_plain;h=81410467f4c5f1dbecc5c3d5f0f183c56da2a4dc;p=thirdparty%2Fsqlalchemy%2Fsqlalchemy.git Major cleanup work in the docs to link class names into the API docs. Fixes #1702. --- diff --git a/CHANGES b/CHANGES index ff260e1bb1..6da0f58de6 100644 --- a/CHANGES +++ b/CHANGES @@ -204,7 +204,11 @@ CHANGES does a lookup among any number of potential attributes more efficiently by grouping several into a common structure. Both FromCache and RelationCache are simpler individually. - + +- documentation + - Major cleanup work in the docs to link class names into + the API docs. [ticket:1700/1702] + 0.6beta1 ======== - Major Release diff --git a/doc/build/dbengine.rst b/doc/build/dbengine.rst index cc61b255af..28218e3f4e 100644 --- a/doc/build/dbengine.rst +++ b/doc/build/dbengine.rst @@ -15,12 +15,12 @@ The general structure is this:: \---| Dialect |---/ |__________| +-----------+ (__________) -Where above, a :class:`~sqlalchemy.engine.Engine` references both a :class:`~sqlalchemy.engine.Dialect` and :class:`~sqlalchemy.pool.Pool`, which together interpret the DBAPI's module functions as well as the behavior of the database. +Where above, a :class:`~sqlalchemy.engine.base.Engine` references both a :class:`~sqlalchemy.engine.base.Dialect` and :class:`~sqlalchemy.pool.Pool`, which together interpret the DBAPI's module functions as well as the behavior of the database. Creating an engine is just a matter of issuing a single call, :func:`create_engine()`:: engine = create_engine('postgresql://scott:tiger@localhost:5432/mydatabase') - + The above engine invokes the ``postgresql`` dialect and a connection pool which references ``localhost:5432``. The engine can be used directly to issue SQL to the database. The most generic way is to use connections, which you get via the ``connect()`` method:: @@ -30,29 +30,29 @@ The engine can be used directly to issue SQL to the database. The most generic for row in result: print "username:", row['username'] connection.close() - -The connection is an instance of :class:`~sqlalchemy.engine.Connection`, which is a **proxy** object for an actual DBAPI connection. The returned result is an instance of :class:`~sqlalchemy.engine.ResultProxy`, which acts very much like a DBAPI cursor. -When you say ``engine.connect()``, a new ``Connection`` object is created, and a DBAPI connection is retrieved from the connection pool. Later, when you call ``connection.close()``, the DBAPI connection is returned to the pool; nothing is actually "closed" from the perspective of the database. +The connection is an instance of :class:`~sqlalchemy.engine.base.Connection`, which is a **proxy** object for an actual DBAPI connection. The returned result is an instance of :class:`~sqlalchemy.engine.ResultProxy`, which acts very much like a DBAPI cursor. + +When you say ``engine.connect()``, a new :class:`~sqlalchemy.engine.base.Connection` object is created, and a DBAPI connection is retrieved from the connection pool. Later, when you call ``connection.close()``, the DBAPI connection is returned to the pool; nothing is actually "closed" from the perspective of the database. -To execute some SQL more quickly, you can skip the ``Connection`` part and just say:: +To execute some SQL more quickly, you can skip the :class:`~sqlalchemy.engine.base.Connection` part and just say:: result = engine.execute("select username from users") for row in result: print "username:", row['username'] result.close() -Where above, the ``execute()`` method on the ``Engine`` does the ``connect()`` part for you, and returns the ``ResultProxy`` directly. The actual ``Connection`` is *inside* the ``ResultProxy``, waiting for you to finish reading the result. In this case, when you ``close()`` the ``ResultProxy``, the underlying ``Connection`` is closed, which returns the DBAPI connection to the pool. +Where above, the ``execute()`` method on the :class:`~sqlalchemy.engine.base.Engine` does the ``connect()`` part for you, and returns the :class:`~sqlalchemy.engine.base.ResultProxy` directly. The actual :class:`~sqlalchemy.engine.base.Connection` is *inside* the :class:`~sqlalchemy.engine.base.ResultProxy`, waiting for you to finish reading the result. In this case, when you ``close()`` the :class:`~sqlalchemy.engine.base.ResultProxy`, the underlying :class:`~sqlalchemy.engine.base.Connection` is closed, which returns the DBAPI connection to the pool. -To summarize the above two examples, when you use a ``Connection`` object, it's known as **explicit execution**. When you don't see the ``Connection`` object, but you still use the ``execute()`` method on the ``Engine``, it's called **explicit, connectionless execution**. A third variant of execution also exists called **implicit execution**; this will be described later. +To summarize the above two examples, when you use a :class:`~sqlalchemy.engine.base.Connection` object, it's known as **explicit execution**. When you don't see the :class:`~sqlalchemy.engine.base.Connection` object, but you still use the ``execute()`` method on the :class:`~sqlalchemy.engine.base.Engine`, it's called **explicit, connectionless execution**. A third variant of execution also exists called **implicit execution**; this will be described later. -The ``Engine`` and ``Connection`` can do a lot more than what we illustrated above; SQL strings are only its most rudimentary function. Later chapters will describe how "constructed SQL" expressions can be used with engines; in many cases, you don't have to deal with the ``Engine`` at all after it's created. The Object Relational Mapper (ORM), an optional feature of SQLAlchemy, also uses the ``Engine`` in order to get at connections; that's also a case where you can often create the engine once, and then forget about it. +The :class:`~sqlalchemy.engine.base.Engine` and :class:`~sqlalchemy.engine.base.Connection` can do a lot more than what we illustrated above; SQL strings are only its most rudimentary function. Later chapters will describe how "constructed SQL" expressions can be used with engines; in many cases, you don't have to deal with the :class:`~sqlalchemy.engine.base.Engine` at all after it's created. The Object Relational Mapper (ORM), an optional feature of SQLAlchemy, also uses the :class:`~sqlalchemy.engine.base.Engine` in order to get at connections; that's also a case where you can often create the engine once, and then forget about it. .. _supported_dbapis: -Supported Databases +Supported Databases ==================== -Recall that the ``Dialect`` is used to describe how to talk to a specific kind of database. Dialects are included with SQLAlchemy for many different backends; these can be seen as a Python package within the :mod:`~sqlalchemy.dialect` package. Each dialect requires the appropriate DBAPI drivers to be installed separately. +Recall that the :class:`~sqlalchemy.engine.base.Dialect` is used to describe how to talk to a specific kind of database. Dialects are included with SQLAlchemy for many different backends; these can be seen as a Python package within the :mod:`~sqlalchemy.dialect` package. Each dialect requires the appropriate DBAPI drivers to be installed separately. Dialects included with SQLAlchemy fall under one of three categories: supported, experimental, and third party. Supported drivers are those which work against the most common databases available in the open source world, including SQLite, PostgreSQL, MySQL, and Firebird. Very popular commercial databases which provide easy access to test platforms are also supported, these currently include MSSQL and Oracle. These dialects are tested frequently and the level of support should be close to 100% for each. @@ -89,7 +89,7 @@ Downloads for each DBAPI at the time of this writing are as follows: The SQLAlchemy Wiki contains a page of database notes, describing whatever quirks and behaviors have been observed. Its a good place to check for issues with specific databases. `Database Notes `_ -create_engine() URL Arguments +create_engine() URL Arguments ============================== SQLAlchemy indicates the source of an Engine strictly via `RFC-1738 `_ style URLs, combined with optional keyword arguments to specify options for the Engine. The form of the URL is: @@ -107,7 +107,7 @@ Dialect names include the identifying name of the SQLAlchemy dialect which inclu # postgresql on Jython pg_db = create_engine('postgresql+zxjdbc://scott:tiger@localhost/mydatabase') - + # mysql - MySQLdb (mysql-python) is the default driver mysql_db = create_engine('mysql://scott:tiger@localhost/foo') mysql_db = create_engine('mysql+mysqldb://scott:tiger@localhost/foo') @@ -138,12 +138,12 @@ SQLite connects to file based databases. The same URL format is used, omitting # or absolute, starting with a slash: sqlite_db = create_engine('sqlite:////absolute/path/to/foo.db') - + To use a SQLite ``:memory:`` database, specify an empty URL:: sqlite_memory_db = create_engine('sqlite://') -The :class:`~sqlalchemy.engine.base.Engine` will ask the connection pool for a connection when the ``connect()`` or ``execute()`` methods are called. The default connection pool, :class:`~sqlalchemy.pool.QueuePool`, as well as the default connection pool used with SQLite, :class:`~sqlalchemy.pool.SingletonThreadPool`, will open connections to the database on an as-needed basis. As concurrent statements are executed, :class:`~sqlalchemy.pool.QueuePool` will grow its pool of connections to a default size of five, and will allow a default "overflow" of ten. Since the ``Engine`` is essentially "home base" for the connection pool, it follows that you should keep a single :class:`~sqlalchemy.engine.base.Engine` per database established within an application, rather than creating a new one for each connection. +The :class:`~sqlalchemy.engine.base.Engine` will ask the connection pool for a connection when the ``connect()`` or ``execute()`` methods are called. The default connection pool, :class:`~sqlalchemy.pool.QueuePool`, as well as the default connection pool used with SQLite, :class:`~sqlalchemy.pool.SingletonThreadPool`, will open connections to the database on an as-needed basis. As concurrent statements are executed, :class:`~sqlalchemy.pool.QueuePool` will grow its pool of connections to a default size of five, and will allow a default "overflow" of ten. Since the :class:`~sqlalchemy.engine.base.Engine` is essentially "home base" for the connection pool, it follows that you should keep a single :class:`~sqlalchemy.engine.base.Engine` per database established within an application, rather than creating a new one for each connection. Custom DBAPI connect() arguments -------------------------------- @@ -155,7 +155,7 @@ Custom arguments used when issuing the ``connect()`` call to the underlying DBAP db = create_engine('postgresql://scott:tiger@localhost/test?argument1=foo&argument2=bar') If SQLAlchemy's database connector is aware of a particular query argument, it may convert its type from string to its proper type. - + ``create_engine`` also takes an argument ``connect_args`` which is an additional dictionary that will be passed to ``connect()``. This can be used when arguments of a type other than string are required, and SQLAlchemy's database connector has no type conversion logic present for that parameter: .. sourcecode:: python+sql @@ -173,7 +173,7 @@ The most customizable connection method of all is to pass a ``creator`` argument .. _create_engine_args: -Database Engine Options +Database Engine Options ======================== Keyword options can also be specified to ``create_engine()``, following the string URL as follows: @@ -184,12 +184,12 @@ Keyword options can also be specified to ``create_engine()``, following the stri Options common to all database dialects are described at :func:`~sqlalchemy.create_engine`. -More On Connections +More On Connections ==================== -Recall from the beginning of this section that the Engine provides a ``connect()`` method which returns a ``Connection`` object. ``Connection`` is a *proxy* object which maintains a reference to a DBAPI connection instance. The ``close()`` method on ``Connection`` does not actually close the DBAPI connection, but instead returns it to the connection pool referenced by the ``Engine``. ``Connection`` will also automatically return its resources to the connection pool when the object is garbage collected, i.e. its ``__del__()`` method is called. When using the standard C implementation of Python, this method is usually called immediately as soon as the object is dereferenced. With other Python implementations such as Jython, this is not so guaranteed. - -The ``execute()`` methods on both ``Engine`` and ``Connection`` can also receive SQL clause constructs as well:: +Recall from the beginning of this section that the Engine provides a ``connect()`` method which returns a :class:`~sqlalchemy.engine.base.Connection` object. :class:`~sqlalchemy.engine.base.Connection` is a *proxy* object which maintains a reference to a DBAPI connection instance. The ``close()`` method on :class:`~sqlalchemy.engine.base.Connection` does not actually close the DBAPI connection, but instead returns it to the connection pool referenced by the :class:`~sqlalchemy.engine.base.Engine`. :class:`~sqlalchemy.engine.base.Connection` will also automatically return its resources to the connection pool when the object is garbage collected, i.e. its ``__del__()`` method is called. When using the standard C implementation of Python, this method is usually called immediately as soon as the object is dereferenced. With other Python implementations such as Jython, this is not so guaranteed. + +The ``execute()`` methods on both :class:`~sqlalchemy.engine.base.Engine` and :class:`~sqlalchemy.engine.base.Connection` can also receive SQL clause constructs as well:: connection = engine.connect() result = connection.execute(select([table1], table1.c.col1==5)) @@ -199,17 +199,17 @@ The ``execute()`` methods on both ``Engine`` and ``Connection`` can also receive The above SQL construct is known as a ``select()``. The full range of SQL constructs available are described in :ref:`sqlexpression_toplevel`. -Both ``Connection`` and ``Engine`` fulfill an interface known as ``Connectable`` which specifies common functionality between the two objects, namely being able to call ``connect()`` to return a ``Connection`` object (``Connection`` just returns itself), and being able to call ``execute()`` to get a result set. Following this, most SQLAlchemy functions and objects which accept an ``Engine`` as a parameter or attribute with which to execute SQL will also accept a ``Connection``. This argument is named ``bind``:: +Both :class:`~sqlalchemy.engine.base.Connection` and :class:`~sqlalchemy.engine.base.Engine` fulfill an interface known as :class:`~sqlalchemy.engine.base.Connectable` which specifies common functionality between the two objects, namely being able to call ``connect()`` to return a :class:`~sqlalchemy.engine.base.Connection` object (:class:`~sqlalchemy.engine.base.Connection` just returns itself), and being able to call ``execute()`` to get a result set. Following this, most SQLAlchemy functions and objects which accept an :class:`~sqlalchemy.engine.base.Engine` as a parameter or attribute with which to execute SQL will also accept a :class:`~sqlalchemy.engine.base.Connection`. This argument is named ``bind``:: engine = create_engine('sqlite:///:memory:') - + # specify some Table metadata metadata = MetaData() table = Table('sometable', metadata, Column('col1', Integer)) - + # create the table with the Engine table.create(bind=engine) - + # drop the table with a Connection off the Engine connection = engine.connect() table.drop(bind=connection) @@ -221,11 +221,11 @@ Connection facts: * the Connection object is **not thread-safe**. While a Connection can be shared among threads using properly synchronized access, this is also not recommended as many DBAPIs have issues with, if not outright disallow, sharing of connection state between threads. * The Connection object represents a single dbapi connection checked out from the connection pool. In this state, the connection pool has no affect upon the connection, including its expiration or timeout state. For the connection pool to properly manage connections, **connections should be returned to the connection pool (i.e. ``connection.close()``) whenever the connection is not in use**. If your application has a need for management of multiple connections or is otherwise long running (this includes all web applications, threaded or not), don't hold a single connection open at the module level. - -Using Transactions with Connection + +Using Transactions with Connection =================================== -The ``Connection`` object provides a ``begin()`` method which returns a ``Transaction`` object. This object is usually used within a try/except clause so that it is guaranteed to ``rollback()`` or ``commit()``:: +The :class:`~sqlalchemy.engine.base.Connection` object provides a ``begin()`` method which returns a :class:`~sqlalchemy.engine.base.Transaction` object. This object is usually used within a try/except clause so that it is guaranteed to ``rollback()`` or ``commit()``:: trans = connection.begin() try: @@ -236,7 +236,7 @@ The ``Connection`` object provides a ``begin()`` method which returns a ``Transa trans.rollback() raise -The ``Transaction`` object also handles "nested" behavior by keeping track of the outermost begin/commit pair. In this example, two functions both issue a transaction on a Connection, but only the outermost Transaction object actually takes effect when it is committed. +The :class:`~sqlalchemy.engine.base.Transaction` object also handles "nested" behavior by keeping track of the outermost begin/commit pair. In this example, two functions both issue a transaction on a Connection, but only the outermost Transaction object actually takes effect when it is committed. .. sourcecode:: python+sql @@ -262,7 +262,7 @@ The ``Transaction`` object also handles "nested" behavior by keeping track of th raise # open a Connection and call method_a - conn = engine.connect() + conn = engine.connect() method_a(conn) conn.close() @@ -281,7 +281,7 @@ Understanding Autocommit ------------------------ -The above transaction example illustrates how to use ``Transaction`` so that several executions can take part in the same transaction. What happens when we issue an INSERT, UPDATE or DELETE call without using ``Transaction``? The answer is **autocommit**. While many DBAPIs implement a flag called ``autocommit``, the current SQLAlchemy behavior is such that it implements its own autocommit. This is achieved by detecting statements which represent data-changing operations, i.e. INSERT, UPDATE, DELETE, etc., and then issuing a COMMIT automatically if no transaction is in progress. The detection is based on compiled statement attributes, or in the case of a text-only statement via regular expressions. +The above transaction example illustrates how to use :class:`~sqlalchemy.engine.base.Transaction` so that several executions can take part in the same transaction. What happens when we issue an INSERT, UPDATE or DELETE call without using :class:`~sqlalchemy.engine.base.Transaction`? The answer is **autocommit**. While many DBAPIs implement a flag called ``autocommit``, the current SQLAlchemy behavior is such that it implements its own autocommit. This is achieved by detecting statements which represent data-changing operations, i.e. INSERT, UPDATE, DELETE, etc., and then issuing a COMMIT automatically if no transaction is in progress. The detection is based on compiled statement attributes, or in the case of a text-only statement via regular expressions. .. sourcecode:: python+sql @@ -290,22 +290,22 @@ The above transaction example illustrates how to use ``Transaction`` so that sev .. _dbengine_implicit: -Connectionless Execution, Implicit Execution +Connectionless Execution, Implicit Execution ============================================= -Recall from the first section we mentioned executing with and without a ``Connection``. ``Connectionless`` execution refers to calling the ``execute()`` method on an object which is not a ``Connection``, which could be on the ``Engine`` itself, or could be a constructed SQL object. When we say "implicit", we mean that we are calling the ``execute()`` method on an object which is neither a ``Connection`` nor an ``Engine`` object; this can only be used with constructed SQL objects which have their own ``execute()`` method, and can be "bound" to an ``Engine``. A description of "constructed SQL objects" may be found in :ref:`sqlexpression_toplevel`. +Recall from the first section we mentioned executing with and without a :class:`~sqlalchemy.engine.base.Connection`. ``Connectionless`` execution refers to calling the ``execute()`` method on an object which is not a :class:`~sqlalchemy.engine.base.Connection`, which could be on the :class:`~sqlalchemy.engine.base.Engine` itself, or could be a constructed SQL object. When we say "implicit", we mean that we are calling the ``execute()`` method on an object which is neither a :class:`~sqlalchemy.engine.base.Connection` nor an :class:`~sqlalchemy.engine.base.Engine` object; this can only be used with constructed SQL objects which have their own ``execute()`` method, and can be "bound" to an :class:`~sqlalchemy.engine.base.Engine`. A description of "constructed SQL objects" may be found in :ref:`sqlexpression_toplevel`. -A summary of all three methods follows below. First, assume the usage of the following ``MetaData`` and ``Table`` objects; while we haven't yet introduced these concepts, for now you only need to know that we are representing a database table, and are creating an "executable" SQL construct which issues a statement to the database. These objects are described in :ref:`metadata_toplevel`. +A summary of all three methods follows below. First, assume the usage of the following :class:`~sqlalchemy.schema.MetaData` and :class:`~sqlalchemy.schema.Table` objects; while we haven't yet introduced these concepts, for now you only need to know that we are representing a database table, and are creating an "executable" SQL construct which issues a statement to the database. These objects are described in :ref:`metadata_toplevel`. .. sourcecode:: python+sql meta = MetaData() - users_table = Table('users', meta, - Column('id', Integer, primary_key=True), + users_table = Table('users', meta, + Column('id', Integer, primary_key=True), Column('name', String(50)) ) - -Explicit execution delivers the SQL text or constructed SQL expression to the ``execute()`` method of ``Connection``: + +Explicit execution delivers the SQL text or constructed SQL expression to the ``execute()`` method of :class:`~sqlalchemy.engine.base.Connection`: .. sourcecode:: python+sql @@ -316,7 +316,7 @@ Explicit execution delivers the SQL text or constructed SQL expression to the `` # .... connection.close() -Explicit, connectionless execution delivers the expression to the ``execute()`` method of ``Engine``: +Explicit, connectionless execution delivers the expression to the ``execute()`` method of :class:`~sqlalchemy.engine.base.Engine`: .. sourcecode:: python+sql @@ -326,7 +326,7 @@ Explicit, connectionless execution delivers the expression to the ``execute()`` # .... result.close() -Implicit execution is also connectionless, and calls the ``execute()`` method on the expression itself, utilizing the fact that either an ``Engine`` or ``Connection`` has been *bound* to the expression object (binding is discussed further in the next section, :ref:`metadata_toplevel`): +Implicit execution is also connectionless, and calls the ``execute()`` method on the expression itself, utilizing the fact that either an :class:`~sqlalchemy.engine.base.Engine` or :class:`~sqlalchemy.engine.base.Connection` has been *bound* to the expression object (binding is discussed further in the next section, :ref:`metadata_toplevel`): .. sourcecode:: python+sql @@ -336,23 +336,23 @@ Implicit execution is also connectionless, and calls the ``execute()`` method on for row in result: # .... result.close() - -In both "connectionless" examples, the ``Connection`` is created behind the scenes; the ``ResultProxy`` returned by the ``execute()`` call references the ``Connection`` used to issue the SQL statement. When we issue ``close()`` on the ``ResultProxy``, or if the result set object falls out of scope and is garbage collected, the underlying ``Connection`` is closed for us, resulting in the DBAPI connection being returned to the pool. + +In both "connectionless" examples, the :class:`~sqlalchemy.engine.base.Connection` is created behind the scenes; the :class:`~sqlalchemy.engine.base.ResultProxy` returned by the ``execute()`` call references the :class:`~sqlalchemy.engine.base.Connection` used to issue the SQL statement. When we issue ``close()`` on the :class:`~sqlalchemy.engine.base.ResultProxy`, or if the result set object falls out of scope and is garbage collected, the underlying :class:`~sqlalchemy.engine.base.Connection` is closed for us, resulting in the DBAPI connection being returned to the pool. .. _threadlocal_strategy: -Using the Threadlocal Execution Strategy +Using the Threadlocal Execution Strategy ----------------------------------------- -The "threadlocal" engine strategy is used by non-ORM applications which wish to bind a transaction to the current thread, such that all parts of the application can participate in that transaction implicitly without the need to explicitly reference a ``Connection``. "threadlocal" is designed for a very specific pattern of use, and is not appropriate unless this very specfic pattern, described below, is what's desired. It has **no impact** on the "thread safety" of SQLAlchemy components or one's application. It also should not be used when using an ORM ``Session`` object, as the ``Session`` itself represents an ongoing transaction and itself handles the job of maintaining connection and transactional resources. +The "threadlocal" engine strategy is used by non-ORM applications which wish to bind a transaction to the current thread, such that all parts of the application can participate in that transaction implicitly without the need to explicitly reference a :class:`~sqlalchemy.engine.base.Connection`. "threadlocal" is designed for a very specific pattern of use, and is not appropriate unless this very specfic pattern, described below, is what's desired. It has **no impact** on the "thread safety" of SQLAlchemy components or one's application. It also should not be used when using an ORM :class:`~sqlalchemy.orm.session.Session` object, as the :class:`~sqlalchemy.orm.session.Session` itself represents an ongoing transaction and itself handles the job of maintaining connection and transactional resources. Enabling ``threadlocal`` is achieved as follows: .. sourcecode:: python+sql db = create_engine('mysql://localhost/test', strategy='threadlocal') - -When the engine above is used in a "connectionless" style, meaning ``engine.execute()`` is called, a DBAPI connection is retrieved from the connection pool and then associated with the current thread. Subsequent operations on the ``Engine`` while the DBAPI connection remains checked out will make use of the *same* DBAPI connection object. The connection stays allocated until all returned ``ResultProxy`` objects are closed, which occurs for a particular ``ResultProxy`` after all pending results are fetched, or immediately for an operation which returns no rows (such as an INSERT). + +When the engine above is used in a "connectionless" style, meaning ``engine.execute()`` is called, a DBAPI connection is retrieved from the connection pool and then associated with the current thread. Subsequent operations on the :class:`~sqlalchemy.engine.base.Engine` while the DBAPI connection remains checked out will make use of the *same* DBAPI connection object. The connection stays allocated until all returned :class:`~sqlalchemy.engine.base.ResultProxy` objects are closed, which occurs for a particular :class:`~sqlalchemy.engine.base.ResultProxy` after all pending results are fetched, or immediately for an operation which returns no rows (such as an INSERT). .. sourcecode:: python+sql @@ -377,7 +377,7 @@ When the engine above is used in a "connectionless" style, meaning ``engine.exec The above example does not illustrate any pattern that is particularly useful, as it is not a frequent occurence that two execute/result fetching operations "leapfrog" one another. There is a slight savings of connection pool checkout overhead between the two operations, and an implicit sharing of the same transactional context, but since there is no explicitly declared transaction, this association is short lived. -The real usage of "threadlocal" comes when we want several operations to occur within the scope of a shared transaction. The ``Engine`` now has ``begin()``, ``commit()`` and ``rollback()`` methods which will retrieve a connection resource from the pool and establish a new transaction, maintaining the connection against the current thread until the transaction is committed or rolled back: +The real usage of "threadlocal" comes when we want several operations to occur within the scope of a shared transaction. The :class:`~sqlalchemy.engine.base.Engine` now has ``begin()``, ``commit()`` and ``rollback()`` methods which will retrieve a connection resource from the pool and establish a new transaction, maintaining the connection against the current thread until the transaction is committed or rolled back: .. sourcecode:: python+sql @@ -388,18 +388,18 @@ The real usage of "threadlocal" comes when we want several operations to occur w db.commit() except: db.rollback() - -``call_operation1()`` and ``call_operation2()`` can make use of the ``Engine`` as a global variable, using the "connectionless" execution style, and their operations will participate in the same transaction: + +``call_operation1()`` and ``call_operation2()`` can make use of the :class:`~sqlalchemy.engine.base.Engine` as a global variable, using the "connectionless" execution style, and their operations will participate in the same transaction: .. sourcecode:: python+sql def call_operation1(): engine.execute("insert into users values (?, ?)", 1, "john") - + def call_operation2(): users.update(users.c.user_id==5).execute(name='ed') - -When using threadlocal, operations that do call upon the ``engine.connect()`` method will receive a ``Connection`` that is **outside** the scope of the transaction. This can be used for operations such as logging the status of an operation regardless of transaction success: + +When using threadlocal, operations that do call upon the ``engine.connect()`` method will receive a :class:`~sqlalchemy.engine.base.Connection` that is **outside** the scope of the transaction. This can be used for operations such as logging the status of an operation regardless of transaction success: .. sourcecode:: python+sql @@ -417,22 +417,22 @@ When using threadlocal, operations that do call upon the ``engine.connect()`` me finally: conn.close() -Functions which are written to use an explicit ``Connection`` object, but wish to participate in the threadlocal transaction, can receive their ``Connection`` object from the ``contextual_connect()`` method, which returns a ``Connection`` that is **inside** the scope of the transaction: +Functions which are written to use an explicit :class:`~sqlalchemy.engine.base.Connection` object, but wish to participate in the threadlocal transaction, can receive their :class:`~sqlalchemy.engine.base.Connection` object from the ``contextual_connect()`` method, which returns a :class:`~sqlalchemy.engine.base.Connection` that is **inside** the scope of the transaction: .. sourcecode:: python+sql conn = db.contextual_connect() call_operation3(conn) conn.close() - + Calling ``close()`` on the "contextual" connection does not release the connection resources to the pool if other resources are making use of it. A resource-counting mechanism is employed so that the connection is released back to the pool only when all users of that connection, including the transaction established by ``engine.begin()``, have been completed. So remember - if you're not sure if you need to use ``strategy="threadlocal"`` or not, the answer is **no** ! It's driven by a specific programming pattern that is generally not the norm. -Configuring Logging +Configuring Logging ==================== -Python's standard `logging `_ module is used to implement informational and debug log output with SQLAlchemy. This allows SQLAlchemy's logging to integrate in a standard way with other applications and libraries. The ``echo`` and ``echo_pool`` flags that are present on ``create_engine()``, as well as the ``echo_uow`` flag used on ``Session``, all interact with regular loggers. +Python's standard `logging `_ module is used to implement informational and debug log output with SQLAlchemy. This allows SQLAlchemy's logging to integrate in a standard way with other applications and libraries. The ``echo`` and ``echo_pool`` flags that are present on ``create_engine()``, as well as the ``echo_uow`` flag used on :class:`~sqlalchemy.orm.session.Session`, all interact with regular loggers. This section assumes familiarity with the above linked logging module. All logging performed by SQLAlchemy exists underneath the ``sqlalchemy`` namespace, as used by ``logging.getLogger('sqlalchemy')``. When logging has been configured (i.e. such as via ``logging.basicConfig()``), the general namespace of SA loggers that can be turned on is as follows: @@ -450,11 +450,11 @@ For example, to log SQL queries as well as unit of work debugging: .. sourcecode:: python+sql import logging - + logging.basicConfig() logging.getLogger('sqlalchemy.engine').setLevel(logging.INFO) logging.getLogger('sqlalchemy.orm.unitofwork').setLevel(logging.DEBUG) - + By default, the log level is set to ``logging.ERROR`` within the entire ``sqlalchemy`` namespace so that no log operations occur, even within an application that has logging enabled otherwise. -The ``echo`` flags present as keyword arguments to ``create_engine()`` and others as well as the ``echo`` property on ``Engine``, when set to ``True``, will first attempt to ensure that logging is enabled. Unfortunately, the ``logging`` module provides no way of determining if output has already been configured (note we are referring to if a logging configuration has been set up, not just that the logging level is set). For this reason, any ``echo=True`` flags will result in a call to ``logging.basicConfig()`` using sys.stdout as the destination. It also sets up a default format using the level name, timestamp, and logger name. Note that this configuration has the affect of being configured **in addition** to any existing logger configurations. Therefore, **when using Python logging, ensure all echo flags are set to False at all times**, to avoid getting duplicate log lines. +The ``echo`` flags present as keyword arguments to ``create_engine()`` and others as well as the ``echo`` property on :class:`~sqlalchemy.engine.base.Engine`, when set to ``True``, will first attempt to ensure that logging is enabled. Unfortunately, the ``logging`` module provides no way of determining if output has already been configured (note we are referring to if a logging configuration has been set up, not just that the logging level is set). For this reason, any ``echo=True`` flags will result in a call to ``logging.basicConfig()`` using sys.stdout as the destination. It also sets up a default format using the level name, timestamp, and logger name. Note that this configuration has the affect of being configured **in addition** to any existing logger configurations. Therefore, **when using Python logging, ensure all echo flags are set to False at all times**, to avoid getting duplicate log lines. diff --git a/doc/build/intro.rst b/doc/build/intro.rst index f0bf9ebbc4..d4fdfccdfd 100644 --- a/doc/build/intro.rst +++ b/doc/build/intro.rst @@ -25,8 +25,8 @@ Main Documentation * :ref:`datamapping_toplevel` - A comprehensive walkthrough of major ORM patterns and techniques. * :ref:`session_toplevel` - A detailed description of SQLAlchemy's Session object -* :ref:`engines_toplevel` - Describes SQLAlchemy's database-connection facilities, including connection documentation and working with connections and transactions. -* :ref:`metadata_toplevel` - All about schema management using ``MetaData`` and ``Table`` objects; reading database schemas into your application, creating and dropping tables, constraints, defaults, sequences, indexes. +* :ref:`engines_toplevel` - Describes SQLAlchemy's database-connection facilities, including connection documentation and working with connections and transactions. +* :ref:`metadata_toplevel` - All about schema management using :class:`~sqlalchemy.schema.MetaData` and :class:`~sqlalchemy.schema.Table` objects; reading database schemas into your application, creating and dropping tables, constraints, defaults, sequences, indexes. * :ref:`pooling_toplevel` - Further detail about SQLAlchemy's connection pool library. * :ref:`types` - Datatypes included with SQLAlchemy, their functions, as well as how to create your own types. * :ref:`plugins` - Included addons for SQLAlchemy @@ -45,7 +45,7 @@ Installing SQLAlchemy ====================== Installing SQLAlchemy from scratch is most easily achieved with `setuptools `_. Assuming it's installed, just run this from the command-line: - + .. sourcecode:: none # easy_install SQLAlchemy @@ -62,7 +62,7 @@ Otherwise, you can install from the distribution using the ``setup.py`` script: # python setup.py install -Installing a Database API +Installing a Database API ========================== SQLAlchemy is designed to operate with a `DB-API `_ implementation built for a particular database, and includes support for the most popular databases. The current list is at :ref:`supported_dbapis`. @@ -78,7 +78,7 @@ This documentation covers SQLAlchemy version 0.6. If you're working on a system >>> sqlalchemy.__version__ # doctest: +SKIP 0.6.0 -0.5 to 0.6 Migration +0.5 to 0.6 Migration ===================== Notes on what's changed from 0.5 to 0.6 is available on the SQLAlchemy wiki at `06Migration `_. diff --git a/doc/build/mappers.rst b/doc/build/mappers.rst index bafefa3758..07a76a613b 100644 --- a/doc/build/mappers.rst +++ b/doc/build/mappers.rst @@ -8,10 +8,10 @@ This section references most major configurational patterns involving the :func: Mapper Configuration ==================== -Customizing Column Properties +Customizing Column Properties ------------------------------ -The default behavior of a ``mapper`` is to assemble all the columns in the mapped ``Table`` into mapped object attributes. This behavior can be modified in several ways, as well as enhanced by SQL expressions. +The default behavior of a ``mapper`` is to assemble all the columns in the mapped :class:`~sqlalchemy.schema.Table` into mapped object attributes. This behavior can be modified in several ways, as well as enhanced by SQL expressions. To load only a part of the columns referenced by a table as attributes, use the ``include_properties`` and ``exclude_properties`` arguments:: @@ -19,7 +19,7 @@ To load only a part of the columns referenced by a table as attributes, use the mapper(Address, addresses_table, exclude_properties=['street', 'city', 'state', 'zip']) -To change the name of the attribute mapped to a particular column, place the ``Column`` object in the ``properties`` dictionary with the desired key:: +To change the name of the attribute mapped to a particular column, place the :class:`~sqlalchemy.schema.Column` object in the ``properties`` dictionary with the desired key:: mapper(User, users_table, properties={ 'id': users_table.c.user_id, @@ -32,7 +32,7 @@ To change the names of all attributes using a prefix, use the ``column_prefix`` The above will place attribute names such as ``_user_id``, ``_user_name``, ``_password`` etc. on the mapped ``User`` class. -To place multiple columns which are known to be "synonymous" based on foreign key relationship or join condition into the same mapped attribute, put them together using a list, as below where we map to a ``Join``:: +To place multiple columns which are known to be "synonymous" based on foreign key relationship or join condition into the same mapped attribute, put them together using a list, as below where we map to a :class:`~sqlalchemy.sql.expression.Join`:: # join users and addresses usersaddresses = sql.join(users_table, addresses_table, \ @@ -42,7 +42,7 @@ To place multiple columns which are known to be "synonymous" based on foreign ke 'id':[users_table.c.user_id, addresses_table.c.user_id], }) -Deferred Column Loading +Deferred Column Loading ------------------------ This feature allows particular columns of a table to not be loaded by default, instead being loaded later on when first referenced. It is essentially "column-level lazy loading". This feature is useful when one wants to avoid loading a large text or binary field into memory when it's not needed. Individual columns can be lazy loaded by themselves or placed into groups that lazy-load together:: @@ -91,7 +91,7 @@ Deferred columns can be placed into groups so that they load together:: 'photo3': deferred(book_excerpts.c.photo3, group='photos') }) -You can defer or undefer columns at the ``Query`` level using the ``defer`` and ``undefer`` options:: +You can defer or undefer columns at the :class:`~sqlalchemy.orm.query.Query` level using the ``defer`` and ``undefer`` options:: query = session.query(Book) query.options(defer('summary')).all() @@ -102,10 +102,10 @@ And an entire "deferred group", i.e. which uses the ``group`` keyword argument t query = session.query(Book) query.options(undefer_group('photos')).all() -SQL Expressions as Mapped Attributes +SQL Expressions as Mapped Attributes ------------------------------------- -To add a SQL clause composed of local or external columns as a read-only, mapped column attribute, use the :func:`~sqlalchemy.orm.column_property()` function. Any scalar-returning ``ClauseElement`` may be used, as long as it has a ``name`` attribute; usually, you'll want to call ``label()`` to give it a specific name:: +To add a SQL clause composed of local or external columns as a read-only, mapped column attribute, use the :func:`~sqlalchemy.orm.column_property()` function. Any scalar-returning :class:`~sqlalchemy.sql.expression.ClauseElement` may be used, as long as it has a ``name`` attribute; usually, you'll want to call ``label()`` to give it a specific name:: mapper(User, users_table, properties={ 'fullname': column_property( @@ -126,11 +126,11 @@ Correlated subqueries may be used as well: ) }) -Changing Attribute Behavior +Changing Attribute Behavior ---------------------------- -Simple Validators +Simple Validators ~~~~~~~~~~~~~~~~~~ @@ -138,19 +138,19 @@ A quick way to add a "validation" routine to an attribute is to use the :func:`~ .. sourcecode:: python+sql - addresses_table = Table('addresses', metadata, + addresses_table = Table('addresses', metadata, Column('id', Integer, primary_key=True), Column('email', String) ) - + class EmailAddress(object): @validates('email') def validate_email(self, key, address): assert '@' in address return address - + mapper(EmailAddress, addresses_table) - + Validators also receive collection events, when items are added to a collection: .. sourcecode:: python+sql @@ -162,8 +162,8 @@ Validators also receive collection events, when items are added to a collection: return address .. _synonyms: - -Using Descriptors + +Using Descriptors ~~~~~~~~~~~~~~~~~~ A more comprehensive way to produce modified behavior for an attribute is to use descriptors. These are commonly used in Python using the ``property()`` function. The standard SQLAlchemy technique for descriptors is to create a plain descriptor, and to have it read/write from a mapped attribute with a different name. To have the descriptor named the same as a column, map the column under a different name, i.e.: @@ -180,8 +180,8 @@ A more comprehensive way to produce modified behavior for an attribute is to use mapper(MyAddress, addresses_table, properties={ '_email': addresses_table.c.email }) - -However, the approach above is not complete. While our ``EmailAddress`` object will shuttle the value through the ``email`` descriptor and into the ``_email`` mapped attribute, the class level ``EmailAddress.email`` attribute does not have the usual expression semantics usable with ``Query``. To provide these, we instead use the ``synonym()`` function as follows: + +However, the approach above is not complete. While our ``EmailAddress`` object will shuttle the value through the ``email`` descriptor and into the ``_email`` mapped attribute, the class level ``EmailAddress.email`` attribute does not have the usual expression semantics usable with :class:`~sqlalchemy.orm.query.Query`. To provide these, we instead use the ``synonym()`` function as follows: .. sourcecode:: python+sql @@ -204,10 +204,10 @@ If the mapped class does not provide a property, the ``synonym()`` construct wil .. _custom_comparators: -Custom Comparators +Custom Comparators ~~~~~~~~~~~~~~~~~~~ -The expressions returned by comparison operations, such as ``User.name=='ed'``, can be customized. SQLAlchemy attributes generate these expressions using :class:`~sqlalchemy.orm.interfaces.PropComparator` objects, which provide common Python expression overrides including ``__eq__()``, ``__ne__()``, ``__lt__()``, and so on. Any mapped attribute can be passed a user-defined class via the ``comparator_factory`` keyword argument, which subclasses the appropriate ``PropComparator`` in use, which can provide any or all of these methods: +The expressions returned by comparison operations, such as ``User.name=='ed'``, can be customized. SQLAlchemy attributes generate these expressions using :class:`~sqlalchemy.orm.interfaces.PropComparator` objects, which provide common Python expression overrides including ``__eq__()``, ``__ne__()``, ``__lt__()``, and so on. Any mapped attribute can be passed a user-defined class via the ``comparator_factory`` keyword argument, which subclasses the appropriate :class:`~sqlalchemy.orm.interfaces.PropComparator` in use, which can provide any or all of these methods: .. sourcecode:: python+sql @@ -236,11 +236,11 @@ There are four kinds of ``Comparator`` classes which may be subclassed, as accor * ``relation()`` attribute - ``sqlalchemy.orm.properties.RelationProperty.Comparator`` * ``comparable_property()`` attribute - ``sqlalchemy.orm.interfaces.PropComparator`` -When using ``comparable_property()``, which is a mapper property that isn't tied to any column or mapped table, the ``__clause_element__()`` method of ``PropComparator`` should also be implemented. - +When using ``comparable_property()``, which is a mapper property that isn't tied to any column or mapped table, the ``__clause_element__()`` method of :class:`~sqlalchemy.orm.interfaces.PropComparator` should also be implemented. + The ``comparator_factory`` argument is accepted by all ``MapperProperty``-producing functions: ``column_property()``, ``composite()``, ``comparable_property()``, ``synonym()``, ``relation()``, ``backref()``, ``deferred()``, and ``dynamic_loader()``. -Composite Column Types +Composite Column Types ----------------------- Sets of columns can be associated with a single datatype. The ORM treats the group of columns like a single column which accepts and returns objects using the custom datatype you provide. In this example, we'll create a table ``vertices`` which stores a pair of x/y coordinates, and a custom datatype ``Point`` which is a composite type of an x and y column: @@ -309,7 +309,7 @@ The "equals" comparison operation by default produces an AND of all correspondin 'end': composite(Point, vertices.c.x2, vertices.c.y2, comparator_factory=PointComparator) }) -Controlling Ordering +Controlling Ordering --------------------- The ORM does not generate ordering for any query unless explicitly configured. @@ -327,7 +327,7 @@ Note that when using eager loaders with relations, the tables used by the eager session.query(User).join('addresses').order_by(Address.street) -Ordering for rows loaded through ``Query`` is usually specified using the ``order_by()`` generative method. There is also an option to set a default ordering for Queries which are against a single mapped entity and where there was no explicit ``order_by()`` stated, which is the ``order_by`` keyword argument to ``mapper()``:: +Ordering for rows loaded through :class:`~sqlalchemy.orm.query.Query` is usually specified using the ``order_by()`` generative method. There is also an option to set a default ordering for Queries which are against a single mapped entity and where there was no explicit ``order_by()`` stated, which is the ``order_by`` keyword argument to ``mapper()``:: # order by a column mapper(User, users_table, order_by=users_table.c.user_id) @@ -335,11 +335,11 @@ Ordering for rows loaded through ``Query`` is usually specified using the ``orde # order by multiple items mapper(User, users_table, order_by=[users_table.c.user_id, users_table.c.user_name.desc()]) -Above, a ``Query`` issued for the ``User`` class will use the value of the mapper's ``order_by`` setting if the ``Query`` itself has no ordering specified. +Above, a :class:`~sqlalchemy.orm.query.Query` issued for the ``User`` class will use the value of the mapper's ``order_by`` setting if the :class:`~sqlalchemy.orm.query.Query` itself has no ordering specified. .. _datamapping_inheritance: -Mapping Class Inheritance Hierarchies +Mapping Class Inheritance Hierarchies -------------------------------------- SQLAlchemy supports three forms of inheritance: *single table inheritance*, where several types of classes are stored in one table, *concrete table inheritance*, where each type of class is stored in its own table, and *joined table inheritance*, where the parent/child classes are stored in their own tables that are joined together in a select. Whereas support for single and joined table inheritance is strong, concrete table inheritance is a less common scenario with some particular problems so is not quite as flexible. @@ -370,7 +370,7 @@ For the following sections, assume this class relationship: def __repr__(self): return self.__class__.__name__ + " " + self.name + " " + self.engineer_info -Joined Table Inheritance +Joined Table Inheritance ~~~~~~~~~~~~~~~~~~~~~~~~~ In joined table inheritance, each class along a particular classes' list of parents is represented by a unique table. The total set of attributes for a particular instance is represented as a join along all tables in its inheritance path. Here, we first define a table to represent the ``Employee`` class. This table will contain a primary key column (or columns), and a column for each attribute that's represented by ``Employee``. In this case it's just ``name``:: @@ -407,10 +407,10 @@ We then configure mappers as usual, except we use some additional arguments to i And that's it. Querying against ``Employee`` will return a combination of ``Employee``, ``Engineer`` and ``Manager`` objects. Newly saved ``Engineer``, ``Manager``, and ``Employee`` objects will automatically populate the ``employees.type`` column with ``engineer``, ``manager``, or ``employee``, as appropriate. -Controlling Which Tables are Queried +Controlling Which Tables are Queried +++++++++++++++++++++++++++++++++++++ -The ``with_polymorphic()`` method of ``Query`` affects the specific subclass tables which the Query selects from. Normally, a query such as this: +The ``with_polymorphic()`` method of :class:`~sqlalchemy.orm.query.Query` affects the specific subclass tables which the Query selects from. Normally, a query such as this: .. sourcecode:: python+sql @@ -439,7 +439,7 @@ As attributes are requested from those ``Employee`` objects which are represente WHERE ? = engineers.employee_id [2] -This behavior works well when issuing searches for small numbers of items, such as when using ``get()``, since the full range of joined tables are not pulled in to the SQL statement unnecessarily. But when querying a larger span of rows which are known to be of many types, you may want to actively join to some or all of the joined tables. The ``with_polymorphic`` feature of ``Query`` and ``mapper`` provides this. +This behavior works well when issuing searches for small numbers of items, such as when using ``get()``, since the full range of joined tables are not pulled in to the SQL statement unnecessarily. But when querying a larger span of rows which are known to be of many types, you may want to actively join to some or all of the joined tables. The ``with_polymorphic`` feature of :class:`~sqlalchemy.orm.query.Query` and ``mapper`` provides this. Telling our query to polymorphically load ``Engineer`` and ``Manager`` objects: @@ -463,13 +463,13 @@ produces a query which joins the ``employees`` table to both the ``engineers`` a # join to the engineers table query.with_polymorphic(Engineer) - + # join to the engineers and managers tables query.with_polymorphic([Engineer, Manager]) - + # join to all subclass tables query.with_polymorphic('*') - + It also accepts a second argument ``selectable`` which replaces the automatic join creation and instead selects directly from the selectable given. This feature is normally used with "concrete" inheritance, described later, but can be used with any kind of inheritance setup in the case that specialized SQL should be used to load polymorphically: .. sourcecode:: python+sql @@ -483,7 +483,7 @@ It also accepts a second argument ``selectable`` which replaces the automatic jo session.query(Employee).with_polymorphic([Engineer, Manager]).\ filter(or_(Engineer.engineer_info=='w', Manager.manager_data=='q')) - + Note that if you only need to load a single subtype, such as just the ``Engineer`` objects, ``with_polymorphic()`` is not needed since you would query against the ``Engineer`` class directly. The mapper also accepts ``with_polymorphic`` as a configurational argument so that the joined-style load will be issued automatically. This argument may be the string ``'*'``, a list of classes, or a tuple consisting of either, followed by a selectable. @@ -497,9 +497,9 @@ The mapper also accepts ``with_polymorphic`` as a configurational argument so th The above mapping will produce a query similar to that of ``with_polymorphic('*')`` for every query of ``Employee`` objects. -Using ``with_polymorphic()`` with ``Query`` will override the mapper-level ``with_polymorphic`` setting. +Using ``with_polymorphic()`` with :class:`~sqlalchemy.orm.query.Query` will override the mapper-level ``with_polymorphic`` setting. -Creating Joins to Specific Subtypes +Creating Joins to Specific Subtypes ++++++++++++++++++++++++++++++++++++ The ``of_type()`` method is a helper which allows the construction of joins along ``relation`` paths while narrowing the criterion to specific subclasses. Suppose the ``employees`` table represents a collection of employees which are associated with a ``Company`` object. We'll add a ``company_id`` column to the ``employees`` table and a new table ``companies``: @@ -659,7 +659,7 @@ Upon select, the polymorphic union produces a query like this: ) AS pjoin [] -Using Relations with Inheritance +Using Relations with Inheritance ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Both joined-table and single table inheritance scenarios produce mappings which are usable in :func:`~sqlalchemy.orm.relation` functions; that is, it's possible to map a parent object to a child object which is polymorphic. Similarly, inheriting mappers can have :func:`~sqlalchemy.orm.relation` objects of their own at any level, which are inherited to each child class. The only requirement for relations is that there is a table relationship between parent and child. An example is the following modification to the joined table inheritance example, which sets a bi-directional relationship between ``Employee`` and ``Company``: @@ -726,23 +726,23 @@ The big limitation with concrete table inheritance is that :func:`~sqlalchemy.or 'a':a_table, 'b':b_table }, 'type', 'ajoin') - - mapper(A, a_table, with_polymorphic=('*', ajoin), - polymorphic_on=ajoin.c.type, polymorphic_identity='a', + + mapper(A, a_table, with_polymorphic=('*', ajoin), + polymorphic_on=ajoin.c.type, polymorphic_identity='a', properties={ 'some_c':relation(C, back_populates='many_a') }) - mapper(B, b_table,inherits=A, concrete=True, - polymorphic_identity='b', + mapper(B, b_table,inherits=A, concrete=True, + polymorphic_identity='b', properties={ 'some_c':relation(C, back_populates='many_a') }) mapper(C, c_table, properties={ 'many_a':relation(A, collection_class=set, back_populates='some_c'), }) - -Mapping a Class against Multiple Tables + +Mapping a Class against Multiple Tables ---------------------------------------- Mappers can be constructed against arbitrary relational units (called ``Selectables``) as well as plain ``Tables``. For example, The ``join`` keyword from the SQL package creates a neat selectable unit comprised of multiple tables, complete with its own composite primary key, which can be passed in to a mapper as the table. @@ -784,7 +784,7 @@ A second example: In both examples above, "composite" columns were added as properties to the mappers; these are aggregations of multiple columns into one mapper property, which instructs the mapper to keep both of those columns set at the same value. -Mapping a Class against Arbitrary Selects +Mapping a Class against Arbitrary Selects ------------------------------------------ @@ -805,7 +805,7 @@ Similar to mapping against a join, a plain select() object can be used with a ma Above, the "customers" table is joined against the "orders" table to produce a full row for each customer row, the total count of related rows in the "orders" table, and the highest price in the "orders" table, grouped against the full set of columns in the "customers" table. That query is then mapped against the Customer class. New instances of Customer will contain attributes for each column in the "customers" table as well as an "order_count" and "highest_order" attribute. Updates to the Customer object will only be reflected in the "customers" table and not the "orders" table. This is because the primary key columns of the "orders" table are not represented in this mapper and therefore the table is not affected by save or delete operations. -Multiple Mappers for One Class +Multiple Mappers for One Class ------------------------------- The first mapper created for a certain class is known as that class's "primary mapper." Other mappers can be created as well on the "load side" - these are called **secondary mappers**. This is a mapper that must be constructed with the keyword argument ``non_primary=True``, and represents a load-only mapper. Objects that are loaded with a secondary mapper will have their save operation processed by the primary mapper. It is also invalid to add new ``relation()`` objects to a non-primary mapper. To use this mapper with the Session, specify it to the ``query`` method: @@ -823,16 +823,16 @@ example: # select result = session.query(othermapper).select() -The "non primary mapper" is a rarely needed feature of SQLAlchemy; in most cases, the ``Query`` object can produce any kind of query that's desired. It's recommended that a straight ``Query`` be used in place of a non-primary mapper unless the mapper approach is absolutely needed. Current use cases for the "non primary mapper" are when you want to map the class to a particular select statement or view to which additional query criterion can be added, and for when the particular mapped select statement or view is to be placed in a ``relation()`` of a parent mapper. +The "non primary mapper" is a rarely needed feature of SQLAlchemy; in most cases, the :class:`~sqlalchemy.orm.query.Query` object can produce any kind of query that's desired. It's recommended that a straight :class:`~sqlalchemy.orm.query.Query` be used in place of a non-primary mapper unless the mapper approach is absolutely needed. Current use cases for the "non primary mapper" are when you want to map the class to a particular select statement or view to which additional query criterion can be added, and for when the particular mapped select statement or view is to be placed in a ``relation()`` of a parent mapper. -Multiple "Persistence" Mappers for One Class +Multiple "Persistence" Mappers for One Class --------------------------------------------- The non_primary mapper defines alternate mappers for the purposes of loading objects. What if we want the same class to be *persisted* differently, such as to different tables ? SQLAlchemy refers to this as the "entity name" pattern, and in Python one can use a recipe which creates anonymous subclasses which are distinctly mapped. See the recipe at `Entity Name `_. -Constructors and Object Initialization +Constructors and Object Initialization --------------------------------------- Mapping imposes no restrictions or requirements on the constructor (``__init__``) method for the class. You are free to require any arguments for the function @@ -859,7 +859,7 @@ recreating transient properties that are normally assigned in your ``__init__``: self.stuff = [] When ``obj = MyMappedClass()`` is executed, Python calls the ``__init__`` method as normal and the ``data`` argument is required. When instances are loaded -during a ``Query`` operation as in ``query(MyMappedClass).one()``, ``init_on_load`` is called instead. +during a :class:`~sqlalchemy.orm.query.Query` operation as in ``query(MyMappedClass).one()``, ``init_on_load`` is called instead. Any method may be tagged as the ``reconstructor``, even the ``__init__`` method. SQLAlchemy will call the reconstructor method with no arguments. Scalar (non-collection) database-mapped attributes of the instance will be available for use within the function. Eagerly-loaded collections are generally not yet @@ -867,12 +867,12 @@ available and will usually only contain the first element. ORM state changes mad operation, so the activity within a reconstructor should be conservative. While the ORM does not call your ``__init__`` method, it will modify the class's ``__init__`` slightly. The method is lightly wrapped to act as a trigger for -the ORM, allowing mappers to be compiled automatically and will fire a ``init_instance`` event that ``MapperExtension`` objectss may listen for. -``MapperExtension`` objects can also listen for a ``reconstruct_instance`` event, analogous to the ``reconstructor`` decorator above. +the ORM, allowing mappers to be compiled automatically and will fire a ``init_instance`` event that :class:`~sqlalchemy.orm.interfaces.MapperExtension` objects may listen for. +:class:`~sqlalchemy.orm.interfaces.MapperExtension` objects can also listen for a ``reconstruct_instance`` event, analogous to the ``reconstructor`` decorator above. .. _extending_mapper: -Extending Mapper +Extending Mapper ----------------- Mappers can have functionality augmented or replaced at many points in its execution via the usage of the MapperExtension class. This class is just a series of "hooks" where various functionality takes place. An application can make its own MapperExtension objects, overriding only the methods it needs. Methods that are not overridden return the special value ``sqlalchemy.orm.EXT_CONTINUE`` to allow processing to continue to the next MapperExtension or simply proceed normally if there are no more extensions. @@ -889,15 +889,15 @@ Multiple extensions will be chained together and processed in order; they are sp .. _advdatamapping_relation: -Relation Configuration +Relation Configuration ======================= -Basic Relational Patterns +Basic Relational Patterns -------------------------- A quick walkthrough of the basic relational patterns. -One To Many +One To Many ~~~~~~~~~~~~ A one to many relationship places a foreign key in the child table referencing the parent. SQLAlchemy creates the relationship as a collection on the parent object containing instances of the child object. @@ -935,7 +935,7 @@ To establish a bi-directional relationship in one-to-many, where the "reverse" s ``Child`` will get a ``parent`` attribute with many-to-one semantics. -Many To One +Many To One ~~~~~~~~~~~~ @@ -965,7 +965,7 @@ Many to one places a foreign key in the parent table referencing the child. The Backref behavior is available here as well, where ``backref="parents"`` will place a one-to-many collection on the ``Child`` class. -One To One +One To One ~~~~~~~~~~~ @@ -985,7 +985,7 @@ Or to turn many-to-one into one-to-one: 'child': relation(Child, backref=backref('parent', uselist=False)) }) -Many To Many +Many To Many ~~~~~~~~~~~~~ @@ -1083,7 +1083,7 @@ To enhance the association object pattern such that direct access to the ``Assoc **Important Note**: it is strongly advised that the ``secondary`` table argument not be combined with the Association Object pattern, unless the ``relation()`` which contains the ``secondary`` argument is marked ``viewonly=True``. Otherwise, SQLAlchemy may persist conflicting data to the underlying association table since it is represented by two conflicting mappings. The Association Proxy pattern should be favored in the case where access to the underlying association data is only sometimes needed. -Adjacency List Relationships +Adjacency List Relationships ----------------------------- @@ -1127,7 +1127,7 @@ SQLAlchemy's ``mapper()`` configuration for a self-referential one-to-many relat 'children': relation(Node) }) -To create a many-to-one relationship from child to parent, an extra indicator of the "remote side" is added, which contains the ``Column`` object or objects indicating the remote side of the relation: +To create a many-to-one relationship from child to parent, an extra indicator of the "remote side" is added, which contains the :class:`~sqlalchemy.schema.Column` object or objects indicating the remote side of the relation: .. sourcecode:: python+sql @@ -1145,7 +1145,7 @@ And the bi-directional version combines both: There are several examples included with SQLAlchemy illustrating self-referential strategies; these include :ref:`examples_adjacencylist` and :ref:`examples_xmlpersistence`. -Self-Referential Query Strategies +Self-Referential Query Strategies ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -1156,7 +1156,7 @@ Querying self-referential structures is done in the same way as any other query # get all nodes named 'child2' session.query(Node).filter(Node.data=='child2') -On the subject of joins, i.e. those described in `datamapping_joins`, self-referential structures require the usage of aliases so that the same table can be referenced multiple times within the FROM clause of the query. Aliasing can be done either manually using the ``nodes`` ``Table`` object as a source of aliases: +On the subject of joins, i.e. those described in `datamapping_joins`, self-referential structures require the usage of aliases so that the same table can be referenced multiple times within the FROM clause of the query. Aliasing can be done either manually using the ``nodes`` :class:`~sqlalchemy.schema.Table` object as a source of aliases: .. sourcecode:: python+sql @@ -1194,7 +1194,7 @@ To add criterion to multiple points along a longer join, use ``from_joinpoint=Tr WHERE treenodes.data = ? AND treenodes_1.data = ? AND treenodes_2.data = ? ['subchild1', 'child2', 'root'] -Configuring Eager Loading +Configuring Eager Loading ~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -1211,27 +1211,27 @@ Eager loading of relations occurs using joins or outerjoins from parent to child FROM treenodes LEFT OUTER JOIN treenodes AS treenodes_2 ON treenodes.id = treenodes_2.parent_id LEFT OUTER JOIN treenodes AS treenodes_1 ON treenodes_2.id = treenodes_1.parent_id [] -Specifying Alternate Join Conditions to relation() +Specifying Alternate Join Conditions to relation() --------------------------------------------------- The ``relation()`` function uses the foreign key relationship between the parent and child tables to formulate the **primary join condition** between parent and child; in the case of a many-to-many relationship it also formulates the **secondary join condition**:: one to many/many to one: ------------------------ - + parent_table --> parent_table.c.id == child_table.c.parent_id --> child_table primaryjoin - + many to many: ------------- - - parent_table --> parent_table.c.id == secondary_table.c.parent_id --> + + parent_table --> parent_table.c.id == secondary_table.c.parent_id --> primaryjoin - + secondary_table.c.child_id == child_table.c.id --> child_table secondaryjoin -If you are working with a ``Table`` which has no ``ForeignKey`` objects on it (which can be the case when using reflected tables with MySQL), or if the join condition cannot be expressed by a simple foreign key relationship, use the ``primaryjoin`` and possibly ``secondaryjoin`` conditions to create the appropriate relationship. +If you are working with a :class:`~sqlalchemy.schema.Table` which has no :class:`~sqlalchemy.schema.ForeignKey` objects on it (which can be the case when using reflected tables with MySQL), or if the join condition cannot be expressed by a simple foreign key relationship, use the ``primaryjoin`` and possibly ``secondaryjoin`` conditions to create the appropriate relationship. In this example we create a relation ``boston_addresses`` which will only load the user addresses with a city of "Boston": @@ -1265,11 +1265,11 @@ Many to many relationships can be customized by one or both of ``primaryjoin`` a ) }) -Specifying Foreign Keys +Specifying Foreign Keys ~~~~~~~~~~~~~~~~~~~~~~~~ -When using ``primaryjoin`` and ``secondaryjoin``, SQLAlchemy also needs to be aware of which columns in the relation reference the other. In most cases, a ``Table`` construct will have ``ForeignKey`` constructs which take care of this; however, in the case of reflected tables on a database that does not report FKs (like MySQL ISAM) or when using join conditions on columns that don't have foreign keys, the ``relation()`` needs to be told specifically which columns are "foreign" using the ``foreign_keys`` collection: +When using ``primaryjoin`` and ``secondaryjoin``, SQLAlchemy also needs to be aware of which columns in the relation reference the other. In most cases, a :class:`~sqlalchemy.schema.Table` construct will have :class:`~sqlalchemy.schema.ForeignKey` constructs which take care of this; however, in the case of reflected tables on a database that does not report FKs (like MySQL ISAM) or when using join conditions on columns that don't have foreign keys, the ``relation()`` needs to be told specifically which columns are "foreign" using the ``foreign_keys`` collection: .. sourcecode:: python+sql @@ -1280,11 +1280,11 @@ When using ``primaryjoin`` and ``secondaryjoin``, SQLAlchemy also needs to be aw foreign_keys=[addresses_table.c.user_id]) }) -Building Query-Enabled Properties +Building Query-Enabled Properties ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Very ambitious custom join conditions may fail to be directly persistable, and in some cases may not even load correctly. To remove the persistence part of the equation, use the flag ``viewonly=True`` on the ``relation()``, which establishes it as a read-only attribute (data written to the collection will be ignored on flush()). However, in extreme cases, consider using a regular Python property in conjunction with ``Query`` as follows: +Very ambitious custom join conditions may fail to be directly persistable, and in some cases may not even load correctly. To remove the persistence part of the equation, use the flag ``viewonly=True`` on the ``relation()``, which establishes it as a read-only attribute (data written to the collection will be ignored on flush()). However, in extreme cases, consider using a regular Python property in conjunction with :class:`~sqlalchemy.orm.query.Query` as follows: .. sourcecode:: python+sql @@ -1293,7 +1293,7 @@ Very ambitious custom join conditions may fail to be directly persistable, and i return object_session(self).query(Address).with_parent(self).filter(...).all() addresses = property(_get_addresses) -Multiple Relations against the Same Parent/Child +Multiple Relations against the Same Parent/Child ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -1350,7 +1350,7 @@ When a structure using the above mapping is flushed, the "widget" row will be IN .. _advdatamapping_entitycollections: -Alternate Collection Implementations +Alternate Collection Implementations ------------------------------------- Mapping a one-to-many or many-to-many relationship results in a collection of values accessible through an attribute on the parent instance. By default, this collection is a ``list``: @@ -1380,12 +1380,12 @@ Collections are not limited to lists. Sets, mutable sequences and almost any ot assert child in parent.children -Custom Collection Implementations +Custom Collection Implementations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ You can use your own types for collections as well. For most cases, simply inherit from ``list`` or ``set`` and add the custom behavior. -Collections in SQLAlchemy are transparently *instrumented*. Instrumentation means that normal operations on the collection are tracked and result in changes being written to the database at flush time. Additionally, collection operations can fire *events* which indicate some secondary operation must take place. Examples of a secondary operation include saving the child item in the parent's ``Session`` (i.e. the ``save-update`` cascade), as well as synchronizing the state of a bi-directional relationship (i.e. a ``backref``). +Collections in SQLAlchemy are transparently *instrumented*. Instrumentation means that normal operations on the collection are tracked and result in changes being written to the database at flush time. Additionally, collection operations can fire *events* which indicate some secondary operation must take place. Examples of a secondary operation include saving the child item in the parent's :class:`~sqlalchemy.orm.session.Session` (i.e. the ``save-update`` cascade), as well as synchronizing the state of a bi-directional relationship (i.e. a ``backref``). The collections package understands the basic interface of lists, sets and dicts and will automatically apply instrumentation to those built-in types and their subclasses. Object-derived types that implement a basic collection interface are detected and instrumented via duck-typing: @@ -1427,7 +1427,7 @@ This class looks list-like because of ``append``, but ``__emulates__`` forces it But this class won't work quite yet: a little glue is needed to adapt it for use by SQLAlchemy. The ORM needs to know which methods to use to append, remove and iterate over members of the collection. When using a type like ``list`` or ``set``, the appropriate methods are well-known and used automatically when present. This set-like class does not provide the expected ``add`` method, so we must supply an explicit mapping for the ORM via a decorator. -Annotating Custom Collections via Decorators +Annotating Custom Collections via Decorators ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -1470,7 +1470,7 @@ And that's all that's needed to complete the example. SQLAlchemy will add insta There is no requirement to be list-, or set-like at all. Collection classes can be any shape, so long as they have the append, remove and iterate interface marked for SQLAlchemy's use. Append and remove methods will be called with a mapped entity as the single argument, and iterator methods are called with no arguments and must return an iterator. -Dictionary-Based Collections +Dictionary-Based Collections ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -1512,7 +1512,7 @@ The :class:`sqlalchemy.orm.collections.MappedCollection` class can be used as a The ORM understands the ``dict`` interface just like lists and sets, and will automatically instrument all dict-like methods if you choose to subclass ``dict`` or provide dict-like collection behavior in a duck-typed class. You must decorate appender and remover methods, however- there are no compatible methods in the basic dictionary interface for SQLAlchemy to use by default. Iteration will go through ``itervalues()`` unless otherwise decorated. -Instrumentation and Custom Types +Instrumentation and Custom Types ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -1531,10 +1531,10 @@ The ORM uses this approach for built-ins, quietly substituting a trivial subclas The collections package provides additional decorators and support for authoring custom types. See the :mod:`sqlalchemy.orm.collections` package for more information and discussion of advanced usage and Python 2.3-compatible decoration options. -Configuring Loader Strategies: Lazy Loading, Eager Loading +Configuring Loader Strategies: Lazy Loading, Eager Loading ----------------------------------------------------------- -In the :ref:`ormtutorial_toplevel`, we introduced the concept of **Eager Loading**. We used an ``option`` in conjunction with the ``Query`` object in order to indicate that a relation should be loaded at the same time as the parent, within a single SQL query: +In the :ref:`ormtutorial_toplevel`, we introduced the concept of **Eager Loading**. We used an ``option`` in conjunction with the :class:`~sqlalchemy.orm.query.Query` object in order to indicate that a relation should be loaded at the same time as the parent, within a single SQL query: .. sourcecode:: python+sql @@ -1604,12 +1604,12 @@ or more simply just use ``eagerload_all()``: There are two other loader strategies available, **dynamic loading** and **no loading**; these are described in :ref:`largecollections`. -Routing Explicit Joins/Statements into Eagerly Loaded Collections +Routing Explicit Joins/Statements into Eagerly Loaded Collections ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The behavior of :func:`~sqlalchemy.orm.eagerload()` is such that joins are created automatically, the results of which are routed into collections and scalar references on loaded objects. It is often the case that a query already includes the necessary joins which represent a particular collection or scalar reference, and the joins added by the eagerload feature are redundant - yet you'd still like the collections/references to be populated. -For this SQLAlchemy supplies the :func:`~sqlalchemy.orm.contains_eager()` option. This option is used in the same manner as the :func:`~sqlalchemy.orm.eagerload()` option except it is assumed that the ``Query`` will specify the appropriate joins explicitly. Below it's used with a ``from_statement`` load:: +For this SQLAlchemy supplies the :func:`~sqlalchemy.orm.contains_eager()` option. This option is used in the same manner as the :func:`~sqlalchemy.orm.eagerload()` option except it is assumed that the :class:`~sqlalchemy.orm.query.Query` will specify the appropriate joins explicitly. Below it's used with a ``from_statement`` load:: # mapping is the users->addresses mapping mapper(User, users_table, properties={ @@ -1629,7 +1629,7 @@ It works just as well with an inline ``Query.join()`` or ``Query.outerjoin()``:: session.query(User).outerjoin(User.addresses).options(contains_eager(User.addresses)).all() -If the "eager" portion of the statement is "aliased", the ``alias`` keyword argument to ``contains_eager()`` may be used to indicate it. This is a string alias name or reference to an actual ``Alias`` (or other selectable) object: +If the "eager" portion of the statement is "aliased", the ``alias`` keyword argument to ``contains_eager()`` may be used to indicate it. This is a string alias name or reference to an actual :class:`~sqlalchemy.sql.expression.Alias` (or other selectable) object: .. sourcecode:: python+sql @@ -1647,14 +1647,14 @@ If the "eager" portion of the statement is "aliased", the ``alias`` keyword argu adalias.user_id AS adalias_user_id, adalias.email_address AS adalias_email_address, (...other columns...) FROM users LEFT OUTER JOIN email_addresses AS email_addresses_1 ON users.user_id = email_addresses_1.user_id -The ``alias`` argument is used only as a source of columns to match up to the result set. You can use it even to match up the result to arbitrary label names in a string SQL statement, by passing a selectable() which links those labels to the mapped ``Table``:: +The ``alias`` argument is used only as a source of columns to match up to the result set. You can use it even to match up the result to arbitrary label names in a string SQL statement, by passing a selectable() which links those labels to the mapped :class:`~sqlalchemy.schema.Table`:: # label the columns of the addresses table eager_columns = select([ - addresses.c.address_id.label('a1'), - addresses.c.email_address.label('a2'), + addresses.c.address_id.label('a1'), + addresses.c.email_address.label('a2'), addresses.c.user_id.label('a3')]) - + # select from a raw SQL statement which uses those label names for the # addresses table. contains_eager() matches them up. query = session.query(User).\ @@ -1688,16 +1688,16 @@ A variant on ``contains_eager()`` is the ``contains_alias()`` option, which is u .. _largecollections: -Working with Large Collections +Working with Large Collections ------------------------------- The default behavior of ``relation()`` is to fully load the collection of items in, as according to the loading strategy of the relation. Additionally, the Session by default only knows how to delete objects which are actually present within the session. When a parent instance is marked for deletion and flushed, the Session loads its full list of child items in so that they may either be deleted as well, or have their foreign key value set to null; this is to avoid constraint violations. For large collections of child items, there are several strategies to bypass full loading of child items both at load time as well as deletion time. -Dynamic Relation Loaders +Dynamic Relation Loaders ~~~~~~~~~~~~~~~~~~~~~~~~~ -The most useful by far is the ``dynamic_loader()`` relation. This is a variant of ``relation()`` which returns a ``Query`` object in place of a collection when accessed. ``filter()`` criterion may be applied as well as limits and offsets, either explicitly or via array slices: +The most useful by far is the ``dynamic_loader()`` relation. This is a variant of ``relation()`` which returns a :class:`~sqlalchemy.orm.query.Query` object in place of a collection when accessed. ``filter()`` criterion may be applied as well as limits and offsets, either explicitly or via array slices: .. sourcecode:: python+sql @@ -1732,7 +1732,7 @@ To place a dynamic relation on a backref, use ``lazy='dynamic'``: Note that eager/lazy loading options cannot be used in conjunction dynamic relations at this time. -Setting Noload +Setting Noload ~~~~~~~~~~~~~~~ The opposite of the dynamic relation is simply "noload", specified using ``lazy=None``: @@ -1745,7 +1745,7 @@ The opposite of the dynamic relation is simply "noload", specified using ``lazy= Above, the ``children`` collection is fully writeable, and changes to it will be persisted to the database as well as locally available for reading at the time they are added. However when instances of ``MyClass`` are freshly loaded from the database, the ``children`` collection stays empty. -Using Passive Deletes +Using Passive Deletes ~~~~~~~~~~~~~~~~~~~~~~ Use ``passive_deletes=True`` to disable child object loading on a DELETE operation, in conjunction with "ON DELETE (CASCADE|SET NULL)" on your database to automatically cascade deletes to child objects. Note that "ON DELETE" is not supported on SQLite, and requires ``InnoDB`` tables when using MySQL: @@ -1770,7 +1770,7 @@ Use ``passive_deletes=True`` to disable child object loading on a DELETE operati When ``passive_deletes`` is applied, the ``children`` relation will not be loaded into memory when an instance of ``MyClass`` is marked for deletion. The ``cascade="all, delete-orphan"`` *will* take effect for instances of ``MyOtherClass`` which are currently present in the session; however for instances of ``MyOtherClass`` which are not loaded, SQLAlchemy assumes that "ON DELETE CASCADE" rules will ensure that those rows are deleted by the database and that no foreign key violation will occur. -Mutable Primary Keys / Update Cascades +Mutable Primary Keys / Update Cascades --------------------------------------- When the primary key of an entity changes, related items which reference the primary key must also be updated as well. For databases which enforce referential integrity, it's required to use the database's ON UPDATE CASCADE functionality in order to propagate primary key changes. For those which don't, the ``passive_updates`` flag can be set to ``False`` which instructs SQLAlchemy to issue UPDATE statements individually. The ``passive_updates`` flag can also be ``False`` in conjunction with ON UPDATE CASCADE functionality, although in that case it issues UPDATE statements unnecessarily. diff --git a/doc/build/metadata.rst b/doc/build/metadata.rst index f4c79884f5..09386f5f5e 100644 --- a/doc/build/metadata.rst +++ b/doc/build/metadata.rst @@ -7,23 +7,23 @@ Database Meta Data Describing Databases with MetaData ================================== -The core of SQLAlchemy's query and object mapping operations are supported by *database metadata*, which is comprised of Python objects that describe tables and other schema-level objects. These objects are at the core of three major types of operations - issuing CREATE and DROP statements (known as *DDL*), constructing SQL queries, and expressing information about structures that already exist within the database. +The core of SQLAlchemy's query and object mapping operations are supported by *database metadata*, which is comprised of Python objects that describe tables and other schema-level objects. These objects are at the core of three major types of operations - issuing CREATE and DROP statements (known as *DDL*), constructing SQL queries, and expressing information about structures that already exist within the database. -Database metadata can be expressed by explicitly naming the various components and their properties, using constructs such as ``Table``, ``Column``, ``ForeignKey`` and ``Sequence``, all of which are imported from the ``sqlalchemy.schema`` package. It can also be generated by SQLAlchemy using a process called *reflection*, which means you start with a single object such as ``Table``, assign it a name, and then instruct SQLAlchemy to load all the additional information related to that name from a particular engine source. +Database metadata can be expressed by explicitly naming the various components and their properties, using constructs such as :class:`~sqlalchemy.schema.Table`, :class:`~sqlalchemy.schema.Column`, :class:`~sqlalchemy.schema.ForeignKey` and :class:`~sqlalchemy.schema.Sequence`, all of which are imported from the ``sqlalchemy.schema`` package. It can also be generated by SQLAlchemy using a process called *reflection*, which means you start with a single object such as :class:`~sqlalchemy.schema.Table`, assign it a name, and then instruct SQLAlchemy to load all the additional information related to that name from a particular engine source. A key feature of SQLAlchemy's database metadata constructs is that they are designed to be used in a *declarative* style which closely resembles that of real DDL. They are therefore most intuitive to those who have some background in creating real schema generation scripts. -A collection of metadata entities is stored in an object aptly named ``MetaData``:: +A collection of metadata entities is stored in an object aptly named :class:`~sqlalchemy.schema.MetaData`:: from sqlalchemy import * - + metadata = MetaData() -``MetaData`` is a container object that keeps together many different features of a database (or multiple databases) being described. +:class:`~sqlalchemy.schema.MetaData` is a container object that keeps together many different features of a database (or multiple databases) being described. -To represent a table, use the ``Table`` class. Its two primary arguments are the table name, then the ``MetaData`` object which it will be associated with. The remaining positional arguments are mostly ``Column`` objects describing each column:: +To represent a table, use the :class:`~sqlalchemy.schema.Table` class. Its two primary arguments are the table name, then the :class:`~sqlalchemy.schema.MetaData` object which it will be associated with. The remaining positional arguments are mostly :class:`~sqlalchemy.schema.Column` objects describing each column:: - user = Table('user', metadata, + user = Table('user', metadata, Column('user_id', Integer, primary_key = True), Column('user_name', String(16), nullable = False), Column('email_address', String(60)), @@ -37,7 +37,7 @@ Note also that each column describes its datatype using objects corresponding to Accessing Tables and Columns ---------------------------- -The ``MetaData`` object contains all of the schema constructs we've associated with it. It supports a few methods of accessing these table objects, such as the ``sorted_tables`` accessor which returns a list of each ``Table`` object in order of foreign key dependency (that is, each table is preceded by all tables which it references):: +The :class:`~sqlalchemy.schema.MetaData` object contains all of the schema constructs we've associated with it. It supports a few methods of accessing these table objects, such as the ``sorted_tables`` accessor which returns a list of each :class:`~sqlalchemy.schema.Table` object in order of foreign key dependency (that is, each table is preceded by all tables which it references):: >>> for t in metadata.sorted_tables: ... print t.name @@ -46,104 +46,104 @@ The ``MetaData`` object contains all of the schema constructs we've associated w invoice invoice_item -In most cases, individual ``Table`` objects have been explicitly declared, and these objects are typically accessed directly as module-level variables in an application. -Once a ``Table`` has been defined, it has a full set of accessors which allow inspection of its properties. Given the following ``Table`` definition:: +In most cases, individual :class:`~sqlalchemy.schema.Table` objects have been explicitly declared, and these objects are typically accessed directly as module-level variables in an application. +Once a :class:`~sqlalchemy.schema.Table` has been defined, it has a full set of accessors which allow inspection of its properties. Given the following :class:`~sqlalchemy.schema.Table` definition:: - employees = Table('employees', metadata, + employees = Table('employees', metadata, Column('employee_id', Integer, primary_key=True), Column('employee_name', String(60), nullable=False), Column('employee_dept', Integer, ForeignKey("departments.department_id")) ) -Note the ``ForeignKey`` object used in this table - this construct defines a reference to a remote table, and is fully described in :ref:`metadata_foreignkeys`. Methods of accessing information about this table include:: +Note the :class:`~sqlalchemy.schema.ForeignKey` object used in this table - this construct defines a reference to a remote table, and is fully described in :ref:`metadata_foreignkeys`. Methods of accessing information about this table include:: # access the column "EMPLOYEE_ID": employees.columns.employee_id - + # or just employees.c.employee_id - + # via string employees.c['employee_id'] - + # iterate through all columns for c in employees.c: print c - + # get the table's primary key columns for primary_key in employees.primary_key: print primary_key - + # get the table's foreign key objects: for fkey in employees.foreign_keys: print fkey - + # access the table's MetaData: employees.metadata - + # access the table's bound Engine or Connection, if its MetaData is bound: employees.bind - + # access a column's name, type, nullable, primary key, foreign key employees.c.employee_id.name employees.c.employee_id.type employees.c.employee_id.nullable employees.c.employee_id.primary_key employees.c.employee_dept.foreign_key - - # get the "key" of a column, which defaults to its name, but can + + # get the "key" of a column, which defaults to its name, but can # be any user-defined string: employees.c.name.key - + # access a column's table: employees.c.employee_id.table is employees - + # get the table related by a foreign key fcolumn = employees.c.employee_dept.foreign_key.column.table .. _metadata_binding: -Creating and Dropping Database Tables +Creating and Dropping Database Tables ------------------------------------- -Once you've defined some ``Table`` objects, assuming you're working with a brand new database one thing you might want to do is issue CREATE statements for those tables and their related constructs (as an aside, it's also quite possible that you *don't* want to do this, if you already have some preferred methodology such as tools included with your database or an existing scripting system - if that's the case, feel free to skip this section - SQLAlchemy has no requirement that it be used to create your tables). +Once you've defined some :class:`~sqlalchemy.schema.Table` objects, assuming you're working with a brand new database one thing you might want to do is issue CREATE statements for those tables and their related constructs (as an aside, it's also quite possible that you *don't* want to do this, if you already have some preferred methodology such as tools included with your database or an existing scripting system - if that's the case, feel free to skip this section - SQLAlchemy has no requirement that it be used to create your tables). -The usual way to issue CREATE is to use ``create_all()`` on the ``MetaData`` object. This method will issue queries that first check for the existence of each individual table, and if not found will issue the CREATE statements: +The usual way to issue CREATE is to use ``create_all()`` on the :class:`~sqlalchemy.schema.MetaData` object. This method will issue queries that first check for the existence of each individual table, and if not found will issue the CREATE statements: .. sourcecode:: python+sql engine = create_engine('sqlite:///:memory:') - + metadata = MetaData() - - user = Table('user', metadata, + + user = Table('user', metadata, Column('user_id', Integer, primary_key = True), Column('user_name', String(16), nullable = False), Column('email_address', String(60), key='email'), Column('password', String(20), nullable = False) ) - - user_prefs = Table('user_prefs', metadata, + + user_prefs = Table('user_prefs', metadata, Column('pref_id', Integer, primary_key=True), Column('user_id', Integer, ForeignKey("user.user_id"), nullable=False), Column('pref_name', String(40), nullable=False), Column('pref_value', String(100)) ) - + {sql}metadata.create_all(engine) PRAGMA table_info(user){} CREATE TABLE user( - user_id INTEGER NOT NULL PRIMARY KEY, - user_name VARCHAR(16) NOT NULL, - email_address VARCHAR(60), + user_id INTEGER NOT NULL PRIMARY KEY, + user_name VARCHAR(16) NOT NULL, + email_address VARCHAR(60), password VARCHAR(20) NOT NULL ) PRAGMA table_info(user_prefs){} CREATE TABLE user_prefs( - pref_id INTEGER NOT NULL PRIMARY KEY, - user_id INTEGER NOT NULL REFERENCES user(user_id), - pref_name VARCHAR(40) NOT NULL, + pref_id INTEGER NOT NULL PRIMARY KEY, + user_id INTEGER NOT NULL REFERENCES user(user_id), + pref_name VARCHAR(40) NOT NULL, pref_value VARCHAR(100) ) @@ -151,7 +151,7 @@ The usual way to issue CREATE is to use ``create_all()`` on the ``MetaData`` obj Dropping all tables is similarly achieved using the ``drop_all()`` method. This method does the exact opposite of ``create_all()`` - the presence of each table is checked first, and tables are dropped in reverse order of dependency. -Creating and dropping individual tables can be done via the ``create()`` and ``drop()`` methods of ``Table``. These methods by default issue the CREATE or DROP regardless of the table being present: +Creating and dropping individual tables can be done via the ``create()`` and ``drop()`` methods of :class:`~sqlalchemy.schema.Table`. These methods by default issue the CREATE or DROP regardless of the table being present: .. sourcecode:: python+sql @@ -159,7 +159,7 @@ Creating and dropping individual tables can be done via the ``create()`` and ``d meta = MetaData() - employees = Table('employees', meta, + employees = Table('employees', meta, Column('employee_id', Integer, primary_key=True), Column('employee_name', String(60), nullable=False, key='name'), Column('employee_dept', Integer, ForeignKey("departments.department_id")) @@ -170,7 +170,7 @@ Creating and dropping individual tables can be done via the ``create()`` and ``d employee_name VARCHAR(60) NOT NULL, employee_dept INTEGER REFERENCES departments(department_id) ) - {} + {} ``drop()`` method: @@ -178,33 +178,33 @@ Creating and dropping individual tables can be done via the ``create()`` and ``d {sql}employees.drop(engine) DROP TABLE employees - {} + {} To enable the "check first for the table existing" logic, add the ``checkfirst=True`` argument to ``create()`` or ``drop()``:: employees.create(engine, checkfirst=True) employees.drop(engine, checkfirst=False) - -Binding MetaData to an Engine or Connection + +Binding MetaData to an Engine or Connection -------------------------------------------- -Notice in the previous section the creator/dropper methods accept an argument for the database engine in use. When a schema construct is combined with an ``Engine`` object, or an individual ``Connection`` object, we call this the *bind*. In the above examples the bind is associated with the schema construct only for the duration of the operation. However, the option exists to persistently associate a bind with a set of schema constructs via the ``MetaData`` object's ``bind`` attribute:: +Notice in the previous section the creator/dropper methods accept an argument for the database engine in use. When a schema construct is combined with an :class:`~sqlalchemy.engine.base.Engine` object, or an individual :class:`~sqlalchemy.engine.base.Connection` object, we call this the *bind*. In the above examples the bind is associated with the schema construct only for the duration of the operation. However, the option exists to persistently associate a bind with a set of schema constructs via the :class:`~sqlalchemy.schema.MetaData` object's ``bind`` attribute:: engine = create_engine('sqlite://') - - # create MetaData + + # create MetaData meta = MetaData() # bind to an engine meta.bind = engine -We can now call methods like ``create_all()`` without needing to pass the ``Engine``:: +We can now call methods like ``create_all()`` without needing to pass the :class:`~sqlalchemy.engine.base.Engine`:: meta.create_all() - + The MetaData's bind is used for anything that requires an active connection, such as loading the definition of a table from the database automatically (called *reflection*):: - + # describe a table called 'users', query the database for its columns users_table = Table('users', meta, autoload=True) @@ -221,32 +221,32 @@ Binding the MetaData to the Engine is a **completely optional** feature. The a # generate a SELECT statement and execute result = engine.execute(users_table.select()) -Should you use bind ? It's probably best to start without it. If you find yourself constantly needing to specify the same ``Engine`` object throughout the entire application, consider binding as a convenience feature which is applicable to applications that don't have multiple engines in use and don't have the need to reference connections explicitly. It should also be noted that an application which is focused on using the SQLAlchemy ORM will not be dealing explicitly with ``Engine`` or ``Connection`` objects very much in any case, so it's probably less confusing and more "future proof" to not use the `bind` attribute. +Should you use bind ? It's probably best to start without it. If you find yourself constantly needing to specify the same :class:`~sqlalchemy.engine.base.Engine` object throughout the entire application, consider binding as a convenience feature which is applicable to applications that don't have multiple engines in use and don't have the need to reference connections explicitly. It should also be noted that an application which is focused on using the SQLAlchemy ORM will not be dealing explicitly with :class:`~sqlalchemy.engine.base.Engine` or :class:`~sqlalchemy.engine.base.Connection` objects very much in any case, so it's probably less confusing and more "future proof" to not use the `bind` attribute. Reflecting Tables ----------------- -A ``Table`` object can be instructed to load information about itself from the corresponding database schema object already existing within the database. This process is called *reflection*. Most simply you need only specify the table name, a ``MetaData`` object, and the ``autoload=True`` flag. If the ``MetaData`` is not persistently bound, also add the ``autoload_with`` argument:: +A :class:`~sqlalchemy.schema.Table` object can be instructed to load information about itself from the corresponding database schema object already existing within the database. This process is called *reflection*. Most simply you need only specify the table name, a :class:`~sqlalchemy.schema.MetaData` object, and the ``autoload=True`` flag. If the :class:`~sqlalchemy.schema.MetaData` is not persistently bound, also add the ``autoload_with`` argument:: >>> messages = Table('messages', meta, autoload=True, autoload_with=engine) >>> [c.name for c in messages.columns] ['message_id', 'message_name', 'date'] -The above operation will use the given engine to query the database for information about the ``messages`` table, and will then generate ``Column``, ``ForeignKey``, and other objects corresponding to this information as though the ``Table`` object were hand-constructed in Python. +The above operation will use the given engine to query the database for information about the ``messages`` table, and will then generate :class:`~sqlalchemy.schema.Column`, :class:`~sqlalchemy.schema.ForeignKey`, and other objects corresponding to this information as though the :class:`~sqlalchemy.schema.Table` object were hand-constructed in Python. -When tables are reflected, if a given table references another one via foreign key, a second ``Table`` object is created within the ``MetaData`` object representing the connection. Below, assume the table ``shopping_cart_items`` references a table named ``shopping_carts``. Reflecting the ``shopping_cart_items`` table has the effect such that the ``shopping_carts`` table will also be loaded:: +When tables are reflected, if a given table references another one via foreign key, a second :class:`~sqlalchemy.schema.Table` object is created within the :class:`~sqlalchemy.schema.MetaData` object representing the connection. Below, assume the table ``shopping_cart_items`` references a table named ``shopping_carts``. Reflecting the ``shopping_cart_items`` table has the effect such that the ``shopping_carts`` table will also be loaded:: >>> shopping_cart_items = Table('shopping_cart_items', meta, autoload=True, autoload_with=engine) >>> 'shopping_carts' in meta.tables: True - -The ``MetaData`` has an interesting "singleton-like" behavior such that if you requested both tables individually, ``MetaData`` will ensure that exactly one ``Table`` object is created for each distinct table name. The ``Table`` constructor actually returns to you the already-existing ``Table`` object if one already exists with the given name. Such as below, we can access the already generated ``shopping_carts`` table just by naming it:: + +The :class:`~sqlalchemy.schema.MetaData` has an interesting "singleton-like" behavior such that if you requested both tables individually, :class:`~sqlalchemy.schema.MetaData` will ensure that exactly one :class:`~sqlalchemy.schema.Table` object is created for each distinct table name. The :class:`~sqlalchemy.schema.Table` constructor actually returns to you the already-existing :class:`~sqlalchemy.schema.Table` object if one already exists with the given name. Such as below, we can access the already generated ``shopping_carts`` table just by naming it:: shopping_carts = Table('shopping_carts', meta) -Of course, it's a good idea to use ``autoload=True`` with the above table regardless. This is so that the table's attributes will be loaded if they have not been already. The autoload operation only occurs for the table if it hasn't already been loaded; once loaded, new calls to ``Table`` with the same name will not re-issue any reflection queries. +Of course, it's a good idea to use ``autoload=True`` with the above table regardless. This is so that the table's attributes will be loaded if they have not been already. The autoload operation only occurs for the table if it hasn't already been loaded; once loaded, new calls to :class:`~sqlalchemy.schema.Table` with the same name will not re-issue any reflection queries. -Overriding Reflected Columns +Overriding Reflected Columns ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Individual columns can be overridden with explicit values when reflecting tables; this is handy for specifying custom datatypes, constraints such as primary keys that may not be configured within the database, etc.:: @@ -262,8 +262,8 @@ Reflecting Views The reflection system can also reflect views. Basic usage is the same as that of a table:: my_view = Table("some_view", metadata, autoload=True) - -Above, ``my_view`` is a ``Table`` object with ``Column`` objects representing the names and types + +Above, ``my_view`` is a :class:`~sqlalchemy.schema.Table` object with :class:`~sqlalchemy.schema.Column` objects representing the names and types of each column within the view "some_view". Usually, it's desired to have at least a primary key constraint when reflecting a view, if not @@ -272,22 +272,22 @@ foreign keys as well. View reflection doesn't extrapolate these constraints. Use the "override" technique for this, specifying explicitly those columns which are part of the primary key or have foreign key constraints:: - my_view = Table("some_view", metadata, + my_view = Table("some_view", metadata, Column("view_id", Integer, primary_key=True), Column("related_thing", Integer, ForeignKey("othertable.thing_id")), autoload=True ) -Reflecting All Tables at Once +Reflecting All Tables at Once ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -The ``MetaData`` object can also get a listing of tables and reflect the full set. This is achieved by using the ``reflect()`` method. After calling it, all located tables are present within the ``MetaData`` object's dictionary of tables:: +The :class:`~sqlalchemy.schema.MetaData` object can also get a listing of tables and reflect the full set. This is achieved by using the ``reflect()`` method. After calling it, all located tables are present within the :class:`~sqlalchemy.schema.MetaData` object's dictionary of tables:: meta = MetaData() meta.reflect(bind=someengine) users_table = meta.tables['users'] addresses_table = meta.tables['addresses'] - + ``metadata.reflect()`` also provides a handy way to clear or delete all the rows in a database:: meta = MetaData() @@ -295,10 +295,10 @@ The ``MetaData`` object can also get a listing of tables and reflect the full se for table in reversed(meta.sorted_tables): someengine.execute(table.delete()) -Specifying the Schema Name +Specifying the Schema Name --------------------------- -Some databases support the concept of multiple schemas. A ``Table`` can reference this by specifying the ``schema`` keyword argument:: +Some databases support the concept of multiple schemas. A :class:`~sqlalchemy.schema.Table` can reference this by specifying the ``schema`` keyword argument:: financial_info = Table('financial_info', meta, Column('id', Integer, primary_key=True), @@ -306,16 +306,16 @@ Some databases support the concept of multiple schemas. A ``Table`` can referen schema='remote_banks' ) -Within the ``MetaData`` collection, this table will be identified by the combination of ``financial_info`` and ``remote_banks``. If another table called ``financial_info`` is referenced without the ``remote_banks`` schema, it will refer to a different ``Table``. ``ForeignKey`` objects can specify references to columns in this table using the form ``remote_banks.financial_info.id``. +Within the :class:`~sqlalchemy.schema.MetaData` collection, this table will be identified by the combination of ``financial_info`` and ``remote_banks``. If another table called ``financial_info`` is referenced without the ``remote_banks`` schema, it will refer to a different :class:`~sqlalchemy.schema.Table`. :class:`~sqlalchemy.schema.ForeignKey` objects can specify references to columns in this table using the form ``remote_banks.financial_info.id``. The ``schema`` argument should be used for any name qualifiers required, including Oracle's "owner" attribute and similar. It also can accommodate a dotted name for longer schemes:: schema="dbo.scott" -Backend-Specific Options +Backend-Specific Options ------------------------ -``Table`` supports database-specific options. For example, MySQL has different table backend types, including "MyISAM" and "InnoDB". This can be expressed with ``Table`` using ``mysql_engine``:: +:class:`~sqlalchemy.schema.Table` supports database-specific options. For example, MySQL has different table backend types, including "MyISAM" and "InnoDB". This can be expressed with :class:`~sqlalchemy.schema.Table` using ``mysql_engine``:: addresses = Table('engine_email_addresses', meta, Column('address_id', Integer, primary_key = True), @@ -323,21 +323,21 @@ Backend-Specific Options Column('email_address', String(20)), mysql_engine='InnoDB' ) - + Other backends may support table-level options as well. See the API documentation for each backend for further details. -Column Insert/Update Defaults +Column Insert/Update Defaults ============================== SQLAlchemy provides a very rich featureset regarding column level events which take place during INSERT and UPDATE statements. Options include: - + * Scalar values used as defaults during INSERT and UPDATE operations * Python functions which execute upon INSERT and UPDATE operations * SQL expressions which are embedded in INSERT statements (or in some cases execute beforehand) * SQL expressions which are embedded in UPDATE statements * Server side default values used during INSERT * Markers for server-side triggers used during UPDATE - + The general rule for all insert/update defaults is that they only take effect if no value for a particular column is passed as an ``execute()`` parameter; otherwise, the given value is used. Scalar Defaults @@ -348,7 +348,7 @@ The simplest kind of default is a scalar value used as the default value of a co Table("mytable", meta, Column("somecolumn", Integer, default=12) ) - + Above, the value "12" will be bound as the column value during an INSERT if no other value is supplied. A scalar value may also be associated with an UPDATE statement, though this is not very common (as UPDATE statements are usually looking for dynamic defaults):: @@ -358,7 +358,7 @@ A scalar value may also be associated with an UPDATE statement, though this is n ) -Python-Executed Functions +Python-Executed Functions ------------------------- The ``default`` and ``onupdate`` keyword arguments also accept Python functions. These functions are invoked at the time of insert or update if no other value for that column is supplied, and the value returned is used for the column's value. Below illustrates a crude "sequence" that assigns an incrementing counter to a primary key column:: @@ -370,19 +370,19 @@ The ``default`` and ``onupdate`` keyword arguments also accept Python functions. i += 1 return i - t = Table("mytable", meta, + t = Table("mytable", meta, Column('id', Integer, primary_key=True, default=mydefault), ) -It should be noted that for real "incrementing sequence" behavior, the built-in capabilities of the database should normally be used, which may include sequence objects or other autoincrementing capabilities. For primary key columns, SQLAlchemy will in most cases use these capabilities automatically. See the API documentation for ``Column`` including the ``autoincrement`` flag, as well as the section on ``Sequence`` later in this chapter for background on standard primary key generation techniques. +It should be noted that for real "incrementing sequence" behavior, the built-in capabilities of the database should normally be used, which may include sequence objects or other autoincrementing capabilities. For primary key columns, SQLAlchemy will in most cases use these capabilities automatically. See the API documentation for :class:`~sqlalchemy.schema.Column` including the ``autoincrement`` flag, as well as the section on :class:`~sqlalchemy.schema.Sequence` later in this chapter for background on standard primary key generation techniques. To illustrate onupdate, we assign the Python ``datetime`` function ``now`` to the ``onupdate`` attribute:: import datetime - - t = Table("mytable", meta, + + t = Table("mytable", meta, Column('id', Integer, primary_key=True), - + # define 'last_updated' to be populated with datetime.now() Column('last_updated', DateTime, onupdate=datetime.datetime.now), ) @@ -396,7 +396,7 @@ The Python functions used by ``default`` and ``onupdate`` may also make use of t def mydefault(context): return context.current_parameters['counter'] + 12 - + t = Table('mytable', meta, Column('counter', Integer), Column('counter_plus_twelve', Integer, default=mydefault, onupdate=mydefault) @@ -406,17 +406,17 @@ Above we illustrate a default function which will execute for all INSERT and UPD While the context object passed to the default function has many attributes, the ``current_parameters`` member is a special member provided only during the execution of a default function for the purposes of deriving defaults from its existing values. For a single statement that is executing many sets of bind parameters, the user-defined function is called for each set of parameters, and ``current_parameters`` will be provided with each individual parameter set for each execution. -SQL Expressions +SQL Expressions --------------- The "default" and "onupdate" keywords may also be passed SQL expressions, including select statements or direct function calls:: - t = Table("mytable", meta, + t = Table("mytable", meta, Column('id', Integer, primary_key=True), - + # define 'create_date' to default to now() Column('create_date', DateTime, default=func.now()), - + # define 'key' to pull its default from the 'keyvalues' table Column('key', String(20), default=keyvalues.select(keyvalues.c.type='type1', limit=1)), @@ -440,11 +440,11 @@ The above SQL functions are usually executed "inline" with the INSERT or UPDATE * the ``inline=True`` flag is not set on the ``Insert()`` or ``Update()`` construct, and the statement has not defined an explicit `returning()` clause. -Whether or not the default generation clause "pre-executes" is not something that normally needs to be considered, unless it is being addressed for performance reasons. +Whether or not the default generation clause "pre-executes" is not something that normally needs to be considered, unless it is being addressed for performance reasons. -When the statement is executed with a single set of parameters (that is, it is not an "executemany" style execution), the returned ``ResultProxy`` will contain a collection accessible via ``result.postfetch_cols()`` which contains a list of all ``Column`` objects which had an inline-executed default. Similarly, all parameters which were bound to the statement, including all Python and SQL expressions which were pre-executed, are present in the ``last_inserted_params()`` or ``last_updated_params()`` collections on ``ResultProxy``. The ``inserted_primary_key`` collection contains a list of primary key values for the row inserted (a list so that single-column and composite-column primary keys are represented in the same format). +When the statement is executed with a single set of parameters (that is, it is not an "executemany" style execution), the returned :class:`~sqlalchemy.engine.base.ResultProxy` will contain a collection accessible via ``result.postfetch_cols()`` which contains a list of all :class:`~sqlalchemy.schema.Column` objects which had an inline-executed default. Similarly, all parameters which were bound to the statement, including all Python and SQL expressions which were pre-executed, are present in the ``last_inserted_params()`` or ``last_updated_params()`` collections on :class:`~sqlalchemy.engine.base.ResultProxy`. The ``inserted_primary_key`` collection contains a list of primary key values for the row inserted (a list so that single-column and composite-column primary keys are represented in the same format). -Server Side Defaults +Server Side Defaults -------------------- A variant on the SQL expression default is the ``server_default``, which gets placed in the CREATE TABLE statement during a ``create()`` operation: @@ -465,7 +465,7 @@ A create call for the above table will produce:: The behavior of ``server_default`` is similar to that of a regular SQL default; if it's placed on a primary key column for a database which doesn't have a way to "postfetch" the ID, and the statement is not "inlined", the SQL expression is pre-executed; otherwise, SQLAlchemy lets the default fire off on the database side normally. -Triggered Columns +Triggered Columns ------------------ Columns with values set by a database trigger or other external process may be called out with a marker:: @@ -477,14 +477,14 @@ Columns with values set by a database trigger or other external process may be c These markers do not emit a "default" clause when the table is created, however they do set the same internal flags as a static ``server_default`` clause, providing hints to higher-level tools that a "post-fetch" of these rows should be performed after an insert or update. -Defining Sequences +Defining Sequences ------------------- -SQLAlchemy represents database sequences using the ``Sequence`` object, which is considered to be a special case of "column default". It only has an effect on databases which have explicit support for sequences, which currently includes Postgresql, Oracle, and Firebird. The ``Sequence`` object is otherwise ignored. +SQLAlchemy represents database sequences using the :class:`~sqlalchemy.schema.Sequence` object, which is considered to be a special case of "column default". It only has an effect on databases which have explicit support for sequences, which currently includes Postgresql, Oracle, and Firebird. The :class:`~sqlalchemy.schema.Sequence` object is otherwise ignored. -The ``Sequence`` may be placed on any column as a "default" generator to be used during INSERT operations, and can also be configured to fire off during UPDATE operations if desired. It is most commonly used in conjunction with a single integer primary key column:: +The :class:`~sqlalchemy.schema.Sequence` may be placed on any column as a "default" generator to be used during INSERT operations, and can also be configured to fire off during UPDATE operations if desired. It is most commonly used in conjunction with a single integer primary key column:: - table = Table("cartitems", meta, + table = Table("cartitems", meta, Column("cart_id", Integer, Sequence('cart_id_seq'), primary_key=True), Column("description", String(40)), Column("createdate", DateTime()) @@ -492,16 +492,16 @@ The ``Sequence`` may be placed on any column as a "default" generator to be used Where above, the table "cartitems" is associated with a sequence named "cart_id_seq". When INSERT statements take place for "cartitems", and no value is passed for the "cart_id" column, the "cart_id_seq" sequence will be used to generate a value. -When the ``Sequence`` is associated with a table, CREATE and DROP statements issued for that table will also issue CREATE/DROP for the sequence object as well, thus "bundling" the sequence object with its parent table. +When the :class:`~sqlalchemy.schema.Sequence` is associated with a table, CREATE and DROP statements issued for that table will also issue CREATE/DROP for the sequence object as well, thus "bundling" the sequence object with its parent table. -The ``Sequence`` object also implements special functionality to accommodate Postgresql's SERIAL datatype. The SERIAL type in PG automatically generates a sequence that is used implicitly during inserts. This means that if a ``Table`` object defines a ``Sequence`` on its primary key column so that it works with Oracle and Firebird, the ``Sequence`` would get in the way of the "implicit" sequence that PG would normally use. For this use case, add the flag ``optional=True`` to the ``Sequence`` object - this indicates that the ``Sequence`` should only be used if the database provides no other option for generating primary key identifiers. +The :class:`~sqlalchemy.schema.Sequence` object also implements special functionality to accommodate Postgresql's SERIAL datatype. The SERIAL type in PG automatically generates a sequence that is used implicitly during inserts. This means that if a :class:`~sqlalchemy.schema.Table` object defines a :class:`~sqlalchemy.schema.Sequence` on its primary key column so that it works with Oracle and Firebird, the :class:`~sqlalchemy.schema.Sequence` would get in the way of the "implicit" sequence that PG would normally use. For this use case, add the flag ``optional=True`` to the :class:`~sqlalchemy.schema.Sequence` object - this indicates that the :class:`~sqlalchemy.schema.Sequence` should only be used if the database provides no other option for generating primary key identifiers. -The ``Sequence`` object also has the ability to be executed standalone like a SQL expression, which has the effect of calling its "next value" function:: +The :class:`~sqlalchemy.schema.Sequence` object also has the ability to be executed standalone like a SQL expression, which has the effect of calling its "next value" function:: seq = Sequence('some_sequence') nextid = connection.execute(seq) -Defining Constraints and Indexes +Defining Constraints and Indexes ================================= .. _metadata_foreignkeys: @@ -511,9 +511,9 @@ Defining Foreign Keys A *foreign key* in SQL is a table-level construct that constrains one or more columns in that table to only allow values that are present in a different set of columns, typically but not always located on a different table. We call the columns which are constrained the *foreign key* columns and the columns which they are constrained towards the *referenced* columns. The referenced columns almost always define the primary key for their owning table, though there are exceptions to this. The foreign key is the "joint" that connects together pairs of rows which have a relationship with each other, and SQLAlchemy assigns very deep importance to this concept in virtually every area of its operation. -In SQLAlchemy as well as in DDL, foreign key constraints can be defined as additional attributes within the table clause, or for single-column foreign keys they may optionally be specified within the definition of a single column. The single column foreign key is more common, and at the column level is specified by constructing a ``ForeignKey`` object as an argument to a ``Column`` object:: +In SQLAlchemy as well as in DDL, foreign key constraints can be defined as additional attributes within the table clause, or for single-column foreign keys they may optionally be specified within the definition of a single column. The single column foreign key is more common, and at the column level is specified by constructing a :class:`~sqlalchemy.schema.ForeignKey` object as an argument to a :class:`~sqlalchemy.schema.Column` object:: - user_preference = Table('user_preference', metadata, + user_preference = Table('user_preference', metadata, Column('pref_id', Integer, primary_key=True), Column('user_id', Integer, ForeignKey("user.user_id"), nullable=False), Column('pref_name', String(40), nullable=False), @@ -522,15 +522,15 @@ In SQLAlchemy as well as in DDL, foreign key constraints can be defined as addit Above, we define a new table ``user_preference`` for which each row must contain a value in the ``user_id`` column that also exists in the ``user`` table's ``user_id`` column. -The argument to ``ForeignKey`` is most commonly a string of the form *.*, or for a table in a remote schema or "owner" of the form *..*. It may also be an actual ``Column`` object, which as we'll see later is accessed from an existing ``Table`` object via its ``c`` collection:: +The argument to :class:`~sqlalchemy.schema.ForeignKey` is most commonly a string of the form *.*, or for a table in a remote schema or "owner" of the form *..*. It may also be an actual :class:`~sqlalchemy.schema.Column` object, which as we'll see later is accessed from an existing :class:`~sqlalchemy.schema.Table` object via its ``c`` collection:: ForeignKey(user.c.user_id) The advantage to using a string is that the in-python linkage between ``user`` and ``user_preference`` is resolved only when first needed, so that table objects can be easily spread across multiple modules and defined in any order. -Foreign keys may also be defined at the table level, using the ``ForeignKeyConstraint`` object. This object can describe a single- or multi-column foreign key. A multi-column foreign key is known as a *composite* foreign key, and almost always references a table that has a composite primary key. Below we define a table ``invoice`` which has a composite primary key:: +Foreign keys may also be defined at the table level, using the :class:`~sqlalchemy.schema.ForeignKeyConstraint` object. This object can describe a single- or multi-column foreign key. A multi-column foreign key is known as a *composite* foreign key, and almost always references a table that has a composite primary key. Below we define a table ``invoice`` which has a composite primary key:: - invoice = Table('invoice', metadata, + invoice = Table('invoice', metadata, Column('invoice_id', Integer, primary_key=True), Column('ref_num', Integer, primary_key=True), Column('description', String(60), nullable=False) @@ -538,58 +538,58 @@ Foreign keys may also be defined at the table level, using the ``ForeignKeyConst And then a table ``invoice_item`` with a composite foreign key referencing ``invoice``:: - invoice_item = Table('invoice_item', metadata, + invoice_item = Table('invoice_item', metadata, Column('item_id', Integer, primary_key=True), Column('item_name', String(60), nullable=False), Column('invoice_id', Integer, nullable=False), Column('ref_num', Integer, nullable=False), ForeignKeyConstraint(['invoice_id', 'ref_num'], ['invoice.invoice_id', 'invoice.ref_num']) ) - -It's important to note that the ``ForeignKeyConstraint`` is the only way to define a composite foreign key. While we could also have placed individual ``ForeignKey`` objects on both the ``invoice_item.invoice_id`` and ``invoice_item.ref_num`` columns, SQLAlchemy would not be aware that these two values should be paired together - it would be two individual foreign key constraints instead of a single composite foreign key referencing two columns. + +It's important to note that the :class:`~sqlalchemy.schema.ForeignKeyConstraint` is the only way to define a composite foreign key. While we could also have placed individual :class:`~sqlalchemy.schema.ForeignKey` objects on both the ``invoice_item.invoice_id`` and ``invoice_item.ref_num`` columns, SQLAlchemy would not be aware that these two values should be paired together - it would be two individual foreign key constraints instead of a single composite foreign key referencing two columns. Creating/Dropping Foreign Key Constraints via ALTER ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -In all the above examples, the ``ForeignKey`` object causes the "REFERENCES" keyword to be added inline to a column definition within a "CREATE TABLE" statement when ``create_all()`` is issued, and ``ForeignKeyConstraint`` invokes the "CONSTRAINT" keyword inline with "CREATE TABLE". There are some cases where this is undesireable, particularly when two tables reference each other mutually, each with a foreign key referencing the other. In such a situation at least one of the foreign key constraints must be generated after both tables have been built. To support such a scheme, ``ForeignKey`` and ``ForeignKeyConstraint`` offer the flag ``use_alter=True``. When using this flag, the constraint will be generated using a definition similar to "ALTER TABLE ADD CONSTRAINT ...". Since a name is required, the ``name`` attribute must also be specified. For example:: +In all the above examples, the :class:`~sqlalchemy.schema.ForeignKey` object causes the "REFERENCES" keyword to be added inline to a column definition within a "CREATE TABLE" statement when ``create_all()`` is issued, and :class:`~sqlalchemy.schema.ForeignKeyConstraint` invokes the "CONSTRAINT" keyword inline with "CREATE TABLE". There are some cases where this is undesireable, particularly when two tables reference each other mutually, each with a foreign key referencing the other. In such a situation at least one of the foreign key constraints must be generated after both tables have been built. To support such a scheme, :class:`~sqlalchemy.schema.ForeignKey` and :class:`~sqlalchemy.schema.ForeignKeyConstraint` offer the flag ``use_alter=True``. When using this flag, the constraint will be generated using a definition similar to "ALTER TABLE ADD CONSTRAINT ...". Since a name is required, the ``name`` attribute must also be specified. For example:: node = Table('node', meta, Column('node_id', Integer, primary_key=True), - Column('primary_element', Integer, + Column('primary_element', Integer, ForeignKey('element.element_id', use_alter=True, name='fk_node_element_id') ) ) - + element = Table('element', meta, Column('element_id', Integer, primary_key=True), Column('parent_node_id', Integer), ForeignKeyConstraint( - ['parent_node_id'], - ['node.node_id'], - use_alter=True, + ['parent_node_id'], + ['node.node_id'], + use_alter=True, name='fk_element_parent_node_id' ) ) -ON UPDATE and ON DELETE +ON UPDATE and ON DELETE ~~~~~~~~~~~~~~~~~~~~~~~ -Most databases support *cascading* of foreign key values, that is the when a parent row is updated the new value is placed in child rows, or when the parent row is deleted all corresponding child rows are set to null or deleted. In data definition language these are specified using phrases like "ON UPDATE CASCADE", "ON DELETE CASCADE", and "ON DELETE SET NULL", corresponding to foreign key constraints. The phrase after "ON UPDATE" or "ON DELETE" may also other allow other phrases that are specific to the database in use. The ``ForeignKey`` and ``ForeignKeyConstraint`` objects support the generation of this clause via the ``onupdate`` and ``ondelete`` keyword arguments. The value is any string which will be output after the appropriate "ON UPDATE" or "ON DELETE" phrase:: +Most databases support *cascading* of foreign key values, that is the when a parent row is updated the new value is placed in child rows, or when the parent row is deleted all corresponding child rows are set to null or deleted. In data definition language these are specified using phrases like "ON UPDATE CASCADE", "ON DELETE CASCADE", and "ON DELETE SET NULL", corresponding to foreign key constraints. The phrase after "ON UPDATE" or "ON DELETE" may also other allow other phrases that are specific to the database in use. The :class:`~sqlalchemy.schema.ForeignKey` and :class:`~sqlalchemy.schema.ForeignKeyConstraint` objects support the generation of this clause via the ``onupdate`` and ``ondelete`` keyword arguments. The value is any string which will be output after the appropriate "ON UPDATE" or "ON DELETE" phrase:: child = Table('child', meta, - Column('id', Integer, - ForeignKey('parent.id', onupdate="CASCADE", ondelete="CASCADE"), + Column('id', Integer, + ForeignKey('parent.id', onupdate="CASCADE", ondelete="CASCADE"), primary_key=True ) ) - + composite = Table('composite', meta, Column('id', Integer, primary_key=True), Column('rev_id', Integer), Column('note_id', Integer), ForeignKeyConstraint( ['rev_id', 'note_id'], - ['revisions.id', 'revisions.note_id'], + ['revisions.id', 'revisions.note_id'], onupdate="CASCADE", ondelete="SET NULL" ) ) @@ -599,19 +599,19 @@ Note that these clauses are not supported on SQLite, and require ``InnoDB`` tabl UNIQUE Constraint ----------------- -Unique constraints can be created anonymously on a single column using the ``unique`` keyword on ``Column``. Explicitly named unique constraints and/or those with multiple columns are created via the ``UniqueConstraint`` table-level construct. +Unique constraints can be created anonymously on a single column using the ``unique`` keyword on :class:`~sqlalchemy.schema.Column`. Explicitly named unique constraints and/or those with multiple columns are created via the :class:`~sqlalchemy.schema.UniqueConstraint` table-level construct. .. sourcecode:: python+sql meta = MetaData() mytable = Table('mytable', meta, - + # per-column anonymous unique constraint Column('col1', Integer, unique=True), - + Column('col2', Integer), Column('col3', Integer), - + # explicit/composite unique constraint. 'name' is optional. UniqueConstraint('col2', 'col3', name='uix_1') ) @@ -619,7 +619,7 @@ Unique constraints can be created anonymously on a single column using the ``uni CHECK Constraint ---------------- -Check constraints can be named or unnamed and can be created at the Column or Table level, using the ``CheckConstraint`` construct. The text of the check constraint is passed directly through to the database, so there is limited "database independent" behavior. Column level check constraints generally should only refer to the column to which they are placed, while table level constraints can refer to any columns in the table. +Check constraints can be named or unnamed and can be created at the Column or Table level, using the :class:`~sqlalchemy.schema.CheckConstraint` construct. The text of the check constraint is passed directly through to the database, so there is limited "database independent" behavior. Column level check constraints generally should only refer to the column to which they are placed, while table level constraints can refer to any columns in the table. Note that some databases do not actively support check constraints such as MySQL and SQLite. @@ -627,33 +627,33 @@ Note that some databases do not actively support check constraints such as MySQL meta = MetaData() mytable = Table('mytable', meta, - + # per-column CHECK constraint Column('col1', Integer, CheckConstraint('col1>5')), - + Column('col2', Integer), Column('col3', Integer), - + # table level CHECK constraint. 'name' is optional. CheckConstraint('col2 > col3 + 5', name='check1') ) - + {sql}mytable.create(engine) CREATE TABLE mytable ( - col1 INTEGER CHECK (col1>5), - col2 INTEGER, - col3 INTEGER, + col1 INTEGER CHECK (col1>5), + col2 INTEGER, + col3 INTEGER, CONSTRAINT check1 CHECK (col2 > col3 + 5) ){stop} - + Indexes ------- -Indexes can be created anonymously (using an auto-generated name ``ix_``) for a single column using the inline ``index`` keyword on ``Column``, which also modifies the usage of ``unique`` to apply the uniqueness to the index itself, instead of adding a separate UNIQUE constraint. For indexes with specific names or which encompass more than one column, use the ``Index`` construct, which requires a name. +Indexes can be created anonymously (using an auto-generated name ``ix_``) for a single column using the inline ``index`` keyword on :class:`~sqlalchemy.schema.Column`, which also modifies the usage of ``unique`` to apply the uniqueness to the index itself, instead of adding a separate UNIQUE constraint. For indexes with specific names or which encompass more than one column, use the :class:`~sqlalchemy.schema.Index` construct, which requires a name. -Note that the ``Index`` construct is created **externally** to the table which it corresponds, using ``Column`` objects and not strings. +Note that the :class:`~sqlalchemy.schema.Index` construct is created **externally** to the table which it corresponds, using :class:`~sqlalchemy.schema.Column` objects and not strings. -Below we illustrate a ``Table`` with several ``Index`` objects associated. The DDL for "CREATE INDEX" is issued right after the create statements for the table: +Below we illustrate a :class:`~sqlalchemy.schema.Table` with several :class:`~sqlalchemy.schema.Index` objects associated. The DDL for "CREATE INDEX" is issued right after the create statements for the table: .. sourcecode:: python+sql @@ -680,11 +680,11 @@ Below we illustrate a ``Table`` with several ``Index`` objects associated. The {sql}mytable.create(engine) CREATE TABLE mytable ( - col1 INTEGER, - col2 INTEGER, - col3 INTEGER, - col4 INTEGER, - col5 INTEGER, + col1 INTEGER, + col2 INTEGER, + col3 INTEGER, + col4 INTEGER, + col5 INTEGER, col6 INTEGER ) CREATE INDEX ix_mytable_col1 ON mytable (col1) @@ -692,18 +692,18 @@ Below we illustrate a ``Table`` with several ``Index`` objects associated. The CREATE UNIQUE INDEX myindex ON mytable (col5, col6) CREATE INDEX idx_col34 ON mytable (col3, col4){stop} -The ``Index`` object also supports its own ``create()`` method: +The :class:`~sqlalchemy.schema.Index` object also supports its own ``create()`` method: .. sourcecode:: python+sql i = Index('someindex', mytable.c.col5) {sql}i.create(engine) CREATE INDEX someindex ON mytable (col5){stop} - + Customizing DDL =============== -In the preceding sections we've discussed a variety of schema constructs including ``Table``, ``ForeignKeyConstraint``, ``CheckConstraint``, and ``Sequence``. Throughout, we've relied upon the ``create()`` and ``create_all()`` methods of ``Table`` and ``MetaData`` in order to issue data definition language (DDL) for all constructs. When issued, a pre-determined order of operations is invoked, and DDL to create each table is created unconditionally including all constraints and other objects associated with it. For more complex scenarios where database-specific DDL is required, SQLAlchemy offers two techniques which can be used to add any DDL based on any condition, either accompanying the standard generation of tables or by itself. +In the preceding sections we've discussed a variety of schema constructs including :class:`~sqlalchemy.schema.Table`, :class:`~sqlalchemy.schema.ForeignKeyConstraint`, :class:`~sqlalchemy.schema.CheckConstraint`, and :class:`~sqlalchemy.schema.Sequence`. Throughout, we've relied upon the ``create()`` and ``create_all()`` methods of :class:`~sqlalchemy.schema.Table` and :class:`~sqlalchemy.schema.MetaData` in order to issue data definition language (DDL) for all constructs. When issued, a pre-determined order of operations is invoked, and DDL to create each table is created unconditionally including all constraints and other objects associated with it. For more complex scenarios where database-specific DDL is required, SQLAlchemy offers two techniques which can be used to add any DDL based on any condition, either accompanying the standard generation of tables or by itself. Controlling DDL Sequences ------------------------- @@ -715,17 +715,17 @@ The ``sqlalchemy.schema`` package contains SQL expression constructs that provid from sqlalchemy.schema import CreateTable {sql}engine.execute(CreateTable(mytable)) CREATE TABLE mytable ( - col1 INTEGER, - col2 INTEGER, - col3 INTEGER, - col4 INTEGER, - col5 INTEGER, + col1 INTEGER, + col2 INTEGER, + col3 INTEGER, + col4 INTEGER, + col5 INTEGER, col6 INTEGER ){stop} - + Above, the ``CreateTable`` construct works like any other expression construct (such as ``select()``, ``table.insert()``, etc.). A full reference of available constructs is in :ref:`schema_api_ddl`. -The DDL constructs all extend a common base class which provides the capability to be associated with an individual ``Table`` or ``MetaData`` object, to be invoked upon create/drop events. Consider the example of a table which contains a CHECK constraint: +The DDL constructs all extend a common base class which provides the capability to be associated with an individual :class:`~sqlalchemy.schema.Table` or :class:`~sqlalchemy.schema.MetaData` object, to be invoked upon create/drop events. Consider the example of a table which contains a CHECK constraint: .. sourcecode:: python+sql @@ -737,20 +737,20 @@ The DDL constructs all extend a common base class which provides the capability {sql}users.create(engine) CREATE TABLE users ( - user_id SERIAL NOT NULL, - user_name VARCHAR(40) NOT NULL, - PRIMARY KEY (user_id), + user_id SERIAL NOT NULL, + user_name VARCHAR(40) NOT NULL, + PRIMARY KEY (user_id), CONSTRAINT cst_user_name_length CHECK (length(user_name) >= 8) ){stop} -The above table contains a column "user_name" which is subject to a CHECK constraint that validates that the length of the string is at least eight characters. When a ``create()`` is issued for this table, DDL for the ``CheckConstraint`` will also be issued inline within the table definition. +The above table contains a column "user_name" which is subject to a CHECK constraint that validates that the length of the string is at least eight characters. When a ``create()`` is issued for this table, DDL for the :class:`~sqlalchemy.schema.CheckConstraint` will also be issued inline within the table definition. -The ``CheckConstraint`` construct can also be constructed externally and associated with the ``Table`` afterwards:: +The :class:`~sqlalchemy.schema.CheckConstraint` construct can also be constructed externally and associated with the :class:`~sqlalchemy.schema.Table` afterwards:: constraint = CheckConstraint('length(user_name) >= 8',name="cst_user_name_length") users.append_constraint(constraint) - -So far, the effect is the same. However, if we create DDL elements corresponding to the creation and removal of this constraint, and associate them with the ``Table`` as events, these new events will take over the job of issuing DDL for the constraint. Additionally, the constraint will be added via ALTER: + +So far, the effect is the same. However, if we create DDL elements corresponding to the creation and removal of this constraint, and associate them with the :class:`~sqlalchemy.schema.Table` as events, these new events will take over the job of issuing DDL for the constraint. Additionally, the constraint will be added via ALTER: .. sourcecode:: python+sql @@ -759,8 +759,8 @@ So far, the effect is the same. However, if we create DDL elements correspondin {sql}users.create(engine) CREATE TABLE users ( - user_id SERIAL NOT NULL, - user_name VARCHAR(40) NOT NULL, + user_id SERIAL NOT NULL, + user_name VARCHAR(40) NOT NULL, PRIMARY KEY (user_id) ) @@ -769,18 +769,18 @@ So far, the effect is the same. However, if we create DDL elements correspondin {sql}users.drop(engine) ALTER TABLE users DROP CONSTRAINT cst_user_name_length DROP TABLE users{stop} - -The real usefulness of the above becomes clearer once we illustrate the ``on`` attribute of a DDL event. The ``on`` parameter is part of the constructor, and may be a string name of a database dialect name, a tuple containing dialect names, or a Python callable. This will limit the execution of the item to just those dialects, or when the return value of the callable is ``True``. So if our ``CheckConstraint`` was only supported by Postgresql and not other databases, we could limit it to just that dialect:: + +The real usefulness of the above becomes clearer once we illustrate the ``on`` attribute of a DDL event. The ``on`` parameter is part of the constructor, and may be a string name of a database dialect name, a tuple containing dialect names, or a Python callable. This will limit the execution of the item to just those dialects, or when the return value of the callable is ``True``. So if our :class:`~sqlalchemy.schema.CheckConstraint` was only supported by Postgresql and not other databases, we could limit it to just that dialect:: AddConstraint(constraint, on='postgresql').execute_at("after-create", users) DropConstraint(constraint, on='postgresql').execute_at("before-drop", users) - + Or to any set of dialects:: AddConstraint(constraint, on=('postgresql', 'mysql')).execute_at("after-create", users) DropConstraint(constraint, on=('postgresql', 'mysql')).execute_at("before-drop", users) - -When using a callable, the callable is passed the ddl element, event name, the ``Table`` or ``MetaData`` object whose "create" or "drop" event is in progress, and the ``Connection`` object being used for the operation, as well as additional information as keyword arguments. The callable can perform checks, such as whether or not a given item already exists. Below we define ``should_create()`` and ``should_drop()`` callables that check for the presence of our named constraint: + +When using a callable, the callable is passed the ddl element, event name, the :class:`~sqlalchemy.schema.Table` or :class:`~sqlalchemy.schema.MetaData` object whose "create" or "drop" event is in progress, and the :class:`~sqlalchemy.engine.base.Connection` object being used for the operation, as well as additional information as keyword arguments. The callable can perform checks, such as whether or not a given item already exists. Below we define ``should_create()`` and ``should_drop()`` callables that check for the presence of our named constraint: .. sourcecode:: python+sql @@ -790,20 +790,20 @@ When using a callable, the callable is passed the ddl element, event name, the ` def should_drop(ddl, event, target, connection, **kw): return not should_create(ddl, event, target, connection, **kw) - + AddConstraint(constraint, on=should_create).execute_at("after-create", users) DropConstraint(constraint, on=should_drop).execute_at("before-drop", users) {sql}users.create(engine) CREATE TABLE users ( - user_id SERIAL NOT NULL, - user_name VARCHAR(40) NOT NULL, + user_id SERIAL NOT NULL, + user_name VARCHAR(40) NOT NULL, PRIMARY KEY (user_id) ) select conname from pg_constraint where conname='cst_user_name_length' ALTER TABLE users ADD CONSTRAINT cst_user_name_length CHECK (length(user_name) >= 8){stop} - + {sql}users.drop(engine) select conname from pg_constraint where conname='cst_user_name_length' ALTER TABLE users DROP CONSTRAINT cst_user_name_length @@ -823,22 +823,22 @@ text to be emitted: A more comprehensive method of creating libraries of DDL constructs is to use the :ref:`sqlalchemy.ext.compiler_toplevel` extension. See that chapter for full details. -Adapting Tables to Alternate Metadata +Adapting Tables to Alternate Metadata ====================================== -A ``Table`` object created against a specific ``MetaData`` object can be re-created against a new MetaData using the ``tometadata`` method: +A :class:`~sqlalchemy.schema.Table` object created against a specific :class:`~sqlalchemy.schema.MetaData` object can be re-created against a new MetaData using the ``tometadata`` method: .. sourcecode:: python+sql # create two metadata meta1 = MetaData('sqlite:///querytest.db') meta2 = MetaData() - + # load 'users' from the sqlite engine users_table = Table('users', meta1, autoload=True) - + # create the same Table object for the plain metadata users_table_2 = users_table.tometadata(meta2) - - + + diff --git a/doc/build/ormtutorial.rst b/doc/build/ormtutorial.rst index 9ea6a3021b..4b7eaffeb0 100644 --- a/doc/build/ormtutorial.rst +++ b/doc/build/ormtutorial.rst @@ -26,7 +26,7 @@ The ``echo`` flag is a shortcut to setting up SQLAlchemy logging, which is accom Define and Create a Table ========================== -Next we want to tell SQLAlchemy about our tables. We will start with just a single table called ``users``, which will store records for the end-users using our application (lets assume it's a website). We define our tables within a catalog called ``MetaData``, using the ``Table`` construct, which is used in a manner similar to SQL's CREATE TABLE syntax:: +Next we want to tell SQLAlchemy about our tables. We will start with just a single table called ``users``, which will store records for the end-users using our application (lets assume it's a website). We define our tables within a catalog called :class:`~sqlalchemy.schema.MetaData`, using the :class:`~sqlalchemy.schema.Table` construct, which is used in a manner similar to SQL's CREATE TABLE syntax:: >>> from sqlalchemy import Table, Column, Integer, String, MetaData, ForeignKey >>> metadata = MetaData() @@ -37,7 +37,7 @@ Next we want to tell SQLAlchemy about our tables. We will start with just a sin ... Column('password', String) ... ) -All about how to define ``Table`` objects, as well as how to load their definition from an existing database (known as **reflection**), is described in :ref:`metadata_toplevel`. +All about how to define :class:`~sqlalchemy.schema.Table` objects, as well as how to load their definition from an existing database (known as **reflection**), is described in :ref:`metadata_toplevel`. Next, we can issue CREATE TABLE statements derived from our table metadata, by calling ``create_all()`` and passing it the ``engine`` instance which points to our database. This will check for the presence of a table first before creating, so it's safe to call multiple times: @@ -71,12 +71,12 @@ Next, we can issue CREATE TABLE statements derived from our table metadata, by c Additionally, Firebird and Oracle require sequences to generate new primary key identifiers, and SQLAlchemy doesn't generate or assume these - without being instructed. For that, you use the ``Sequence`` construct:: + without being instructed. For that, you use the :class:`~sqlalchemy.schema.Sequence` construct:: from sqlalchemy import Sequence Column('id', Integer, Sequence('user_id_seq'), primary_key=True) - A full, foolproof ``Table`` is therefore:: + A full, foolproof :class:`~sqlalchemy.schema.Table` is therefore:: users_table = Table('users', metadata, Column('id', Integer, Sequence('user_id_seq'), primary_key=True), @@ -87,7 +87,7 @@ Next, we can issue CREATE TABLE statements derived from our table metadata, by c Define a Python Class to be Mapped =================================== -While the ``Table`` object defines information about our database, it does not say anything about the definition or behavior of the business objects used by our application; SQLAlchemy views this as a separate concern. To correspond to our ``users`` table, let's create a rudimentary ``User`` class. It only need subclass Python's built-in ``object`` class (i.e. it's a new style class):: +While the :class:`~sqlalchemy.schema.Table` object defines information about our database, it does not say anything about the definition or behavior of the business objects used by our application; SQLAlchemy views this as a separate concern. To correspond to our ``users`` table, let's create a rudimentary ``User`` class. It only need subclass Python's built-in ``object`` class (i.e. it's a new style class):: >>> class User(object): ... def __init__(self, name, fullname, password): @@ -108,7 +108,7 @@ With our ``users_table`` and ``User`` class, we now want to map the two together >>> mapper(User, users_table) # doctest:+ELLIPSIS,+NORMALIZE_WHITESPACE -The ``mapper()`` function creates a new ``Mapper`` object and stores it away for future reference, associated with our class. Let's now create and inspect a ``User`` object:: +The ``mapper()`` function creates a new :class:`~sqlalchemy.orm.mapper.Mapper` object and stores it away for future reference, associated with our class. Let's now create and inspect a ``User`` object:: >>> ed_user = User('ed', 'Ed Jones', 'edspassword') >>> ed_user.name @@ -118,13 +118,13 @@ The ``mapper()`` function creates a new ``Mapper`` object and stores it away for >>> str(ed_user.id) 'None' -The ``id`` attribute, which while not defined by our ``__init__()`` method, exists due to the ``id`` column present within the ``users_table`` object. By default, the ``mapper`` creates class attributes for all columns present within the ``Table``. These class attributes exist as Python descriptors, and define **instrumentation** for the mapped class. The functionality of this instrumentation is very rich and includes the ability to track modifications and automatically load new data from the database when needed. +The ``id`` attribute, which while not defined by our ``__init__()`` method, exists due to the ``id`` column present within the ``users_table`` object. By default, the ``mapper`` creates class attributes for all columns present within the :class:`~sqlalchemy.schema.Table`. These class attributes exist as Python descriptors, and define **instrumentation** for the mapped class. The functionality of this instrumentation is very rich and includes the ability to track modifications and automatically load new data from the database when needed. Since we have not yet told SQLAlchemy to persist ``Ed Jones`` within the database, its id is ``None``. When we persist the object later, this attribute will be populated with a newly generated value. Creating Table, Class and Mapper All at Once Declaratively =========================================================== -The preceding approach to configuration involving a ``Table``, user-defined class, and ``mapper()`` call illustrate classical SQLAlchemy usage, which values the highest separation of concerns possible. A large number of applications don't require this degree of separation, and for those SQLAlchemy offers an alternate "shorthand" configurational style called **declarative**. For many applications, this is the only style of configuration needed. Our above example using this style is as follows:: +The preceding approach to configuration involving a :class:`~sqlalchemy.schema.Table`, user-defined class, and ``mapper()`` call illustrate classical SQLAlchemy usage, which values the highest separation of concerns possible. A large number of applications don't require this degree of separation, and for those SQLAlchemy offers an alternate "shorthand" configurational style called **declarative**. For many applications, this is the only style of configuration needed. Our above example using this style is as follows:: >>> from sqlalchemy.ext.declarative import declarative_base @@ -145,13 +145,13 @@ The preceding approach to configuration involving a ``Table``, user-defined clas ... def __repr__(self): ... return "" % (self.name, self.fullname, self.password) -Above, the ``declarative_base()`` function defines a new class which we name ``Base``, from which all of our ORM-enabled classes will derive. Note that we define ``Column`` objects with no "name" field, since it's inferred from the given attribute name. +Above, the ``declarative_base()`` function defines a new class which we name ``Base``, from which all of our ORM-enabled classes will derive. Note that we define :class:`~sqlalchemy.schema.Column` objects with no "name" field, since it's inferred from the given attribute name. -The underlying ``Table`` object created by our ``declarative_base()`` version of ``User`` is accessible via the ``__table__`` attribute:: +The underlying :class:`~sqlalchemy.schema.Table` object created by our ``declarative_base()`` version of ``User`` is accessible via the ``__table__`` attribute:: >>> users_table = User.__table__ -and the owning ``MetaData`` object is available as well:: +and the owning :class:`~sqlalchemy.schema.MetaData` object is available as well:: >>> metadata = Base.metadata @@ -160,42 +160,42 @@ Yet another "declarative" method is available for SQLAlchemy as a third party li Creating a Session ================== -We're now ready to start talking to the database. The ORM's "handle" to the database is the ``Session``. When we first set up the application, at the same level as our ``create_engine()`` statement, we define a ``Session`` class which will serve as a factory for new ``Session`` objects: +We're now ready to start talking to the database. The ORM's "handle" to the database is the :class:`~sqlalchemy.orm.session.Session`. When we first set up the application, at the same level as our ``create_engine()`` statement, we define a :class:`~sqlalchemy.orm.session.Session` class which will serve as a factory for new :class:`~sqlalchemy.orm.session.Session` objects: .. sourcecode:: python+sql >>> from sqlalchemy.orm import sessionmaker >>> Session = sessionmaker(bind=engine) -In the case where your application does not yet have an ``Engine`` when you define your module-level objects, just set it up like this: +In the case where your application does not yet have an :class:`~sqlalchemy.engine.base.Engine` when you define your module-level objects, just set it up like this: .. sourcecode:: python+sql >>> Session = sessionmaker() -Later, when you create your engine with ``create_engine()``, connect it to the ``Session`` using ``configure()``: +Later, when you create your engine with ``create_engine()``, connect it to the :class:`~sqlalchemy.orm.session.Session` using ``configure()``: .. sourcecode:: python+sql >>> Session.configure(bind=engine) # once engine is available -This custom-made ``Session`` class will create new ``Session`` objects which are bound to our database. Other transactional characteristics may be defined when calling ``sessionmaker()`` as well; these are described in a later chapter. Then, whenever you need to have a conversation with the database, you instantiate a ``Session``:: +This custom-made :class:`~sqlalchemy.orm.session.Session` class will create new :class:`~sqlalchemy.orm.session.Session` objects which are bound to our database. Other transactional characteristics may be defined when calling ``sessionmaker()`` as well; these are described in a later chapter. Then, whenever you need to have a conversation with the database, you instantiate a :class:`~sqlalchemy.orm.session.Session`:: >>> session = Session() -The above ``Session`` is associated with our SQLite ``engine``, but it hasn't opened any connections yet. When it's first used, it retrieves a connection from a pool of connections maintained by the ``engine``, and holds onto it until we commit all changes and/or close the session object. +The above :class:`~sqlalchemy.orm.session.Session` is associated with our SQLite ``engine``, but it hasn't opened any connections yet. When it's first used, it retrieves a connection from a pool of connections maintained by the ``engine``, and holds onto it until we commit all changes and/or close the session object. Adding new Objects ================== -To persist our ``User`` object, we ``add()`` it to our ``Session``:: +To persist our ``User`` object, we ``add()`` it to our :class:`~sqlalchemy.orm.session.Session`:: >>> ed_user = User('ed', 'Ed Jones', 'edspassword') >>> session.add(ed_user) -At this point, the instance is **pending**; no SQL has yet been issued. The ``Session`` will issue the SQL to persist ``Ed Jones`` as soon as is needed, using a process known as a **flush**. If we query the database for ``Ed Jones``, all pending information will first be flushed, and the query is issued afterwards. +At this point, the instance is **pending**; no SQL has yet been issued. The :class:`~sqlalchemy.orm.session.Session` will issue the SQL to persist ``Ed Jones`` as soon as is needed, using a process known as a **flush**. If we query the database for ``Ed Jones``, all pending information will first be flushed, and the query is issued afterwards. -For example, below we create a new ``Query`` object which loads instances of ``User``. We "filter by" the ``name`` attribute of ``ed``, and indicate that we'd like only the first result in the full list of rows. A ``User`` instance is returned which is equivalent to that which we've added: +For example, below we create a new :class:`~sqlalchemy.orm.query.Query` object which loads instances of ``User``. We "filter by" the ``name`` attribute of ``ed``, and indicate that we'd like only the first result in the full list of rows. A ``User`` instance is returned which is equivalent to that which we've added: .. sourcecode:: python+sql @@ -211,12 +211,12 @@ For example, below we create a new ``Query`` object which loads instances of ``U {stop}>>> our_user -In fact, the ``Session`` has identified that the row returned is the **same** row as one already represented within its internal map of objects, so we actually got back the identical instance as that which we just added:: +In fact, the :class:`~sqlalchemy.orm.session.Session` has identified that the row returned is the **same** row as one already represented within its internal map of objects, so we actually got back the identical instance as that which we just added:: >>> ed_user is our_user True -The ORM concept at work here is known as an **identity map** and ensures that all operations upon a particular row within a ``Session`` operate upon the same set of data. Once an object with a particular primary key is present in the ``Session``, all SQL queries on that ``Session`` will always return the same Python object for that particular primary key; it also will raise an error if an attempt is made to place a second, already-persisted object with the same primary key within the session. +The ORM concept at work here is known as an **identity map** and ensures that all operations upon a particular row within a :class:`~sqlalchemy.orm.session.Session` operate upon the same set of data. Once an object with a particular primary key is present in the :class:`~sqlalchemy.orm.session.Session`, all SQL queries on that :class:`~sqlalchemy.orm.session.Session` will always return the same Python object for that particular primary key; it also will raise an error if an attempt is made to place a second, already-persisted object with the same primary key within the session. We can add more ``User`` objects at once using ``add_all()``: @@ -233,7 +233,7 @@ Also, Ed has already decided his password isn't too secure, so lets change it: >>> ed_user.password = 'f8s7ccs' -The ``Session`` is paying attention. It knows, for example, that ``Ed Jones`` has been modified: +The :class:`~sqlalchemy.orm.session.Session` is paying attention. It knows, for example, that ``Ed Jones`` has been modified: .. sourcecode:: python+sql @@ -249,7 +249,7 @@ and that three new ``User`` objects are pending: , ]) -We tell the ``Session`` that we'd like to issue all remaining changes to the database and commit the transaction, which has been in progress throughout. We do this via ``commit()``: +We tell the :class:`~sqlalchemy.orm.session.Session` that we'd like to issue all remaining changes to the database and commit the transaction, which has been in progress throughout. We do this via ``commit()``: .. sourcecode:: python+sql @@ -278,11 +278,11 @@ If we look at Ed's ``id`` attribute, which earlier was ``None``, it now has a va [1] {stop}1 -After the ``Session`` inserts new rows in the database, all newly generated identifiers and database-generated defaults become available on the instance, either immediately or via load-on-first-access. In this case, the entire row was re-loaded on access because a new transaction was begun after we issued ``commit()``. SQLAlchemy by default refreshes data from a previous transaction the first time it's accessed within a new transaction, so that the most recent state is available. The level of reloading is configurable as is described in the chapter on Sessions. +After the :class:`~sqlalchemy.orm.session.Session` inserts new rows in the database, all newly generated identifiers and database-generated defaults become available on the instance, either immediately or via load-on-first-access. In this case, the entire row was re-loaded on access because a new transaction was begun after we issued ``commit()``. SQLAlchemy by default refreshes data from a previous transaction the first time it's accessed within a new transaction, so that the most recent state is available. The level of reloading is configurable as is described in the chapter on Sessions. Rolling Back ============ -Since the ``Session`` works within a transaction, we can roll back changes made too. Let's make two changes that we'll revert; ``ed_user``'s user name gets set to ``Edwardo``: +Since the :class:`~sqlalchemy.orm.session.Session` works within a transaction, we can roll back changes made too. Let's make two changes that we'll revert; ``ed_user``'s user name gets set to ``Edwardo``: .. sourcecode:: python+sql @@ -344,7 +344,7 @@ issuing a SELECT illustrates the changes made to the database: Querying ======== -A ``Query`` is created using the ``query()`` function on ``Session``. This function takes a variable number of arguments, which can be any combination of classes and class-instrumented descriptors. Below, we indicate a ``Query`` which loads ``User`` instances. When evaluated in an iterative context, the list of ``User`` objects present is returned: +A :class:`~sqlalchemy.orm.query.Query` is created using the ``query()`` function on :class:`~sqlalchemy.orm.session.Session`. This function takes a variable number of arguments, which can be any combination of classes and class-instrumented descriptors. Below, we indicate a :class:`~sqlalchemy.orm.query.Query` which loads ``User`` instances. When evaluated in an iterative context, the list of ``User`` objects present is returned: .. sourcecode:: python+sql @@ -359,7 +359,7 @@ A ``Query`` is created using the ``query()`` function on ``Session``. This func mary Mary Contrary fred Fred Flinstone -The ``Query`` also accepts ORM-instrumented descriptors as arguments. Any time multiple class entities or column-based entities are expressed as arguments to the ``query()`` function, the return result is expressed as tuples: +The :class:`~sqlalchemy.orm.query.Query` also accepts ORM-instrumented descriptors as arguments. Any time multiple class entities or column-based entities are expressed as arguments to the ``query()`` function, the return result is expressed as tuples: .. sourcecode:: python+sql @@ -373,7 +373,7 @@ The ``Query`` also accepts ORM-instrumented descriptors as arguments. Any time mary Mary Contrary fred Fred Flinstone -The tuples returned by ``Query`` are *named* tuples, and can be treated much like an ordinary Python object. The names are the same as the attribute's name for an attribute, and the class name for a class: +The tuples returned by :class:`~sqlalchemy.orm.query.Query` are *named* tuples, and can be treated much like an ordinary Python object. The names are the same as the attribute's name for an attribute, and the class name for a class: .. sourcecode:: python+sql @@ -403,7 +403,7 @@ You can control the names using the ``label()`` construct for scalar attributes mary fred -Basic operations with ``Query`` include issuing LIMIT and OFFSET, most conveniently using Python array slices and typically in conjunction with ORDER BY: +Basic operations with :class:`~sqlalchemy.orm.query.Query` include issuing LIMIT and OFFSET, most conveniently using Python array slices and typically in conjunction with ORDER BY: .. sourcecode:: python+sql @@ -438,7 +438,7 @@ and filtering results, which is accomplished either with ``filter_by()``, which ['Ed Jones'] {stop}ed -The ``Query`` object is fully *generative*, meaning that most method calls return a new ``Query`` object upon which further criteria may be added. For example, to query for users named "ed" with a full name of "Ed Jones", you can call ``filter()`` twice, which joins criteria using ``AND``: +The :class:`~sqlalchemy.orm.query.Query` object is fully *generative*, meaning that most method calls return a new :class:`~sqlalchemy.orm.query.Query` object upon which further criteria may be added. For example, to query for users named "ed" with a full name of "Ed Jones", you can call ``filter()`` twice, which joins criteria using ``AND``: .. sourcecode:: python+sql @@ -473,9 +473,9 @@ Here's a rundown of some of the most common operators used in ``filter()``: query.filter(User.name.in_(['ed', 'wendy', 'jack'])) # works with query objects too: - + query.filter(User.name.in_(session.query(User.name).filter(User.name.like('%ed%')))) - + * NOT IN:: query.filter(~User.name.in_(['ed', 'wendy', 'jack'])) @@ -618,20 +618,20 @@ Counting .. sourcecode:: python+sql {sql}>>> session.query(User).filter(User.name.like('%ed')).count() - SELECT count(1) AS count_1 - FROM users + SELECT count(1) AS count_1 + FROM users WHERE users.name LIKE ? ['%ed'] {stop}2 - + The :meth:`~sqlalchemy.orm.query.Query.count()` method is used to determine how many rows the SQL statement would return, and is mainly intended to return a simple count of a single type of entity, in this case ``User``. For more complicated sets of columns or entities where the "thing to be counted" needs to be indicated more specifically, :meth:`~sqlalchemy.orm.query.Query.count()` is probably not what you want. Below, a query for individual columns does return the expected result: .. sourcecode:: python+sql {sql}>>> session.query(User.id, User.name).filter(User.name.like('%ed')).count() - SELECT count(1) AS count_1 - FROM (SELECT users.id AS users_id, users.name AS users_name - FROM users + SELECT count(1) AS count_1 + FROM (SELECT users.id AS users_id, users.name AS users_name + FROM users WHERE users.name LIKE ?) AS anon_1 ['%ed'] {stop}2 @@ -641,8 +641,8 @@ The :meth:`~sqlalchemy.orm.query.Query.count()` method is used to determine how .. sourcecode:: python+sql {sql}>>> session.query(User.name).group_by(User.name).count() - SELECT count(1) AS count_1 - FROM (SELECT users.name AS users_name + SELECT count(1) AS count_1 + FROM (SELECT users.name AS users_name FROM users GROUP BY users.name) AS anon_1 [] {stop}4 @@ -650,10 +650,10 @@ The :meth:`~sqlalchemy.orm.query.Query.count()` method is used to determine how We don't want the number ``4``, we wanted some rows back. So for detailed queries where you need to count something specific, use the ``func.count()`` function as a column expression: .. sourcecode:: python+sql - + >>> from sqlalchemy import func {sql}>>> session.query(func.count(User.name), User.name).group_by(User.name).all() - SELECT count(users.name) AS count_1, users.name AS users_name + SELECT count(users.name) AS count_1, users.name AS users_name FROM users GROUP BY users.name {stop}[] [(1, u'ed'), (1, u'fred'), (1, u'mary'), (1, u'wendy')] @@ -701,7 +701,7 @@ When using the ``declarative`` extension, :func:`~sqlalchemy.orm.relation()` giv .... addresses = relation("Address", order_by="Address.id", backref="user") -When ``declarative`` is not in use, you typically define your :func:`~sqlalchemy.orm.mapper()` well after the target classes and ``Table`` objects have been defined, so string expressions are not needed. +When ``declarative`` is not in use, you typically define your :func:`~sqlalchemy.orm.mapper()` well after the target classes and :class:`~sqlalchemy.schema.Table` objects have been defined, so string expressions are not needed. We'll need to create the ``addresses`` table in the database, so we will issue another CREATE from our metadata, which will skip over tables which have already been created: @@ -797,10 +797,10 @@ If you want to reduce the number of queries (dramatically, in many cases), we ca >>> from sqlalchemy.orm import eagerload {sql}>>> jack = session.query(User).options(eagerload('addresses')).filter_by(name='jack').one() #doctest: +NORMALIZE_WHITESPACE - SELECT users.id AS users_id, users.name AS users_name, users.fullname AS users_fullname, - users.password AS users_password, addresses_1.id AS addresses_1_id, addresses_1.email_address - AS addresses_1_email_address, addresses_1.user_id AS addresses_1_user_id - FROM users LEFT OUTER JOIN addresses AS addresses_1 ON users.id = addresses_1.user_id + SELECT users.id AS users_id, users.name AS users_name, users.fullname AS users_fullname, + users.password AS users_password, addresses_1.id AS addresses_1_id, addresses_1.email_address + AS addresses_1_email_address, addresses_1.user_id AS addresses_1_user_id + FROM users LEFT OUTER JOIN addresses AS addresses_1 ON users.id = addresses_1.user_id WHERE users.name = ? ORDER BY addresses_1.id ['jack'] @@ -880,7 +880,7 @@ The above would produce SQL something like ``foo JOIN bars ON JOIN ba Using Aliases ------------- -When querying across multiple tables, if the same table needs to be referenced more than once, SQL typically requires that the table be *aliased* with another name, so that it can be distinguished against other occurrences of that table. The ``Query`` supports this most explicitly using the ``aliased`` construct. Below we join to the ``Address`` entity twice, to locate a user who has two distinct email addresses at the same time: +When querying across multiple tables, if the same table needs to be referenced more than once, SQL typically requires that the table be *aliased* with another name, so that it can be distinguished against other occurrences of that table. The :class:`~sqlalchemy.orm.query.Query` supports this most explicitly using the ``aliased`` construct. Below we join to the ``Address`` entity twice, to locate a user who has two distinct email addresses at the same time: .. sourcecode:: python+sql @@ -904,20 +904,20 @@ When querying across multiple tables, if the same table needs to be referenced m Using Subqueries ---------------- -The ``Query`` is suitable for generating statements which can be used as subqueries. Suppose we wanted to load ``User`` objects along with a count of how many ``Address`` records each user has. The best way to generate SQL like this is to get the count of addresses grouped by user ids, and JOIN to the parent. In this case we use a LEFT OUTER JOIN so that we get rows back for those users who don't have any addresses, e.g.:: +The :class:`~sqlalchemy.orm.query.Query` is suitable for generating statements which can be used as subqueries. Suppose we wanted to load ``User`` objects along with a count of how many ``Address`` records each user has. The best way to generate SQL like this is to get the count of addresses grouped by user ids, and JOIN to the parent. In this case we use a LEFT OUTER JOIN so that we get rows back for those users who don't have any addresses, e.g.:: SELECT users.*, adr_count.address_count FROM users LEFT OUTER JOIN (SELECT user_id, count(*) AS address_count FROM addresses GROUP BY user_id) AS adr_count ON users.id=adr_count.user_id -Using the ``Query``, we build a statement like this from the inside out. The ``statement`` accessor returns a SQL expression representing the statement generated by a particular ``Query`` - this is an instance of a ``select()`` construct, which are described in :ref:`sqlexpression_toplevel`:: +Using the :class:`~sqlalchemy.orm.query.Query`, we build a statement like this from the inside out. The ``statement`` accessor returns a SQL expression representing the statement generated by a particular :class:`~sqlalchemy.orm.query.Query` - this is an instance of a ``select()`` construct, which are described in :ref:`sqlexpression_toplevel`:: >>> from sqlalchemy.sql import func >>> stmt = session.query(Address.user_id, func.count('*').label('address_count')).group_by(Address.user_id).subquery() -The ``func`` keyword generates SQL functions, and the ``subquery()`` method on ``Query`` produces a SQL expression construct representing a SELECT statement embedded within an alias (it's actually shorthand for ``query.statement.alias()``). +The ``func`` keyword generates SQL functions, and the ``subquery()`` method on :class:`~sqlalchemy.orm.query.Query` produces a SQL expression construct representing a SELECT statement embedded within an alias (it's actually shorthand for ``query.statement.alias()``). -Once we have our statement, it behaves like a ``Table`` construct, such as the one we created for ``users`` at the start of this tutorial. The columns on the statement are accessible through an attribute called ``c``: +Once we have our statement, it behaves like a :class:`~sqlalchemy.schema.Table` construct, such as the one we created for ``users`` at the start of this tutorial. The columns on the statement are accessible through an attribute called ``c``: .. sourcecode:: python+sql @@ -948,11 +948,11 @@ Above, we just selected a result that included a column from a subquery. What i >>> adalias = aliased(Address, stmt) >>> for user, address in session.query(User, adalias).join((adalias, User.addresses)): # doctest: +NORMALIZE_WHITESPACE ... print user, address - SELECT users.id AS users_id, users.name AS users_name, users.fullname AS users_fullname, - users.password AS users_password, anon_1.id AS anon_1_id, - anon_1.email_address AS anon_1_email_address, anon_1.user_id AS anon_1_user_id - FROM users JOIN (SELECT addresses.id AS id, addresses.email_address AS email_address, addresses.user_id AS user_id - FROM addresses + SELECT users.id AS users_id, users.name AS users_name, users.fullname AS users_fullname, + users.password AS users_password, anon_1.id AS anon_1_id, + anon_1.email_address AS anon_1_email_address, anon_1.user_id AS anon_1_user_id + FROM users JOIN (SELECT addresses.id AS id, addresses.email_address AS email_address, addresses.user_id AS user_id + FROM addresses WHERE addresses.email_address != ?) AS anon_1 ON users.id = anon_1.user_id ['j25@yahoo.com'] {stop} @@ -978,7 +978,7 @@ There is an explicit EXISTS construct, which looks like this: [] {stop}jack -The ``Query`` features several operators which make usage of EXISTS automatically. Above, the statement can be expressed along the ``User.addresses`` relation using ``any()``: +The :class:`~sqlalchemy.orm.query.Query` features several operators which make usage of EXISTS automatically. Above, the statement can be expressed along the ``User.addresses`` relation using ``any()``: .. sourcecode:: python+sql @@ -1223,7 +1223,7 @@ The declarative setup is as follows: ... def __init__(self, keyword): ... self.keyword = keyword -Above, the many-to-many relation is ``BlogPost.keywords``. The defining feature of a many-to-many relation is the ``secondary`` keyword argument which references a ``Table`` object representing the association table. This table only contains columns which reference the two sides of the relation; if it has *any* other columns, such as its own primary key, or foreign keys to other tables, SQLAlchemy requires a different usage pattern called the "association object", described at :ref:`association_pattern`. +Above, the many-to-many relation is ``BlogPost.keywords``. The defining feature of a many-to-many relation is the ``secondary`` keyword argument which references a :class:`~sqlalchemy.schema.Table` object representing the association table. This table only contains columns which reference the two sides of the relation; if it has *any* other columns, such as its own primary key, or foreign keys to other tables, SQLAlchemy requires a different usage pattern called the "association object", described at :ref:`association_pattern`. The many-to-many relation is also bi-directional using the ``backref`` keyword. This is the one case where usage of ``backref`` is generally required, since if a separate ``posts`` relation were added to the ``Keyword`` entity, both relations would independently add and remove rows from the ``post_keywords`` table and produce conflicts. @@ -1310,10 +1310,10 @@ We can now look up all blog posts with the keyword 'firstpost'. We'll use the [2, "Wendy's Blog Post", 'This is a test'] INSERT INTO post_keywords (post_id, keyword_id) VALUES (?, ?) [[1, 1], [1, 2]] - SELECT posts.id AS posts_id, posts.user_id AS posts_user_id, posts.headline AS posts_headline, posts.body AS posts_body - FROM posts - WHERE EXISTS (SELECT 1 - FROM post_keywords, keywords + SELECT posts.id AS posts_id, posts.user_id AS posts_user_id, posts.headline AS posts_headline, posts.body AS posts_body + FROM posts + WHERE EXISTS (SELECT 1 + FROM post_keywords, keywords WHERE posts.id = post_keywords.post_id AND keywords.id = post_keywords.keyword_id AND keywords.keyword = ?) ['firstpost'] {stop}[BlogPost("Wendy's Blog Post", 'This is a test', )] diff --git a/doc/build/session.rst b/doc/build/session.rst index 9fa03a7833..55e371da4b 100644 --- a/doc/build/session.rst +++ b/doc/build/session.rst @@ -9,16 +9,16 @@ The `Mapper` is the entrypoint to the configurational API of the SQLAlchemy obje What does the Session do ? ========================== -In the most general sense, the ``Session`` establishes all conversations with the database and represents a "holding zone" for all the mapped instances which you've loaded or created during its lifespan. It implements the `Unit of Work `_ pattern, which means it keeps track of all changes which occur, and is capable of **flushing** those changes to the database as appropriate. Another important facet of the ``Session`` is that it's also maintaining **unique** copies of each instance, where "unique" means "only one object with a particular primary key" - this pattern is called the `Identity Map `_. +In the most general sense, the :class:`~sqlalchemy.orm.session.Session` establishes all conversations with the database and represents a "holding zone" for all the mapped instances which you've loaded or created during its lifespan. It implements the `Unit of Work `_ pattern, which means it keeps track of all changes which occur, and is capable of **flushing** those changes to the database as appropriate. Another important facet of the :class:`~sqlalchemy.orm.session.Session` is that it's also maintaining **unique** copies of each instance, where "unique" means "only one object with a particular primary key" - this pattern is called the `Identity Map `_. -Beyond that, the ``Session`` implements an interface which lets you move objects in or out of the session in a variety of ways, it provides the entryway to a ``Query`` object which is used to query the database for data, and it also provides a transactional context for SQL operations which rides on top of the transactional capabilities of ``Engine`` and ``Connection`` objects. +Beyond that, the :class:`~sqlalchemy.orm.session.Session` implements an interface which lets you move objects in or out of the session in a variety of ways, it provides the entryway to a :class:`~sqlalchemy.orm.query.Query` object which is used to query the database for data, and it also provides a transactional context for SQL operations which rides on top of the transactional capabilities of :class:`~sqlalchemy.engine.base.Engine` and :class:`~sqlalchemy.engine.base.Connection` objects. Getting a Session ================= -``Session`` is a regular Python class which can be directly instantiated. However, to standardize how sessions are configured and acquired, the ``sessionmaker()`` function is normally used to create a top level ``Session`` configuration which can then be used throughout an application without the need to repeat the configurational arguments. +:class:`~sqlalchemy.orm.session.Session` is a regular Python class which can be directly instantiated. However, to standardize how sessions are configured and acquired, the ``sessionmaker()`` function is normally used to create a top level :class:`~sqlalchemy.orm.session.Session` configuration which can then be used throughout an application without the need to repeat the configurational arguments. -Using a sessionmaker() Configuration +Using a sessionmaker() Configuration ------------------------------------ The usage of ``sessionmaker()`` is illustrated below: @@ -26,86 +26,86 @@ The usage of ``sessionmaker()`` is illustrated below: .. sourcecode:: python+sql from sqlalchemy.orm import sessionmaker - + # create a configured "Session" class Session = sessionmaker(bind=some_engine) # create a Session session = Session() - + # work with sess myobject = MyObject('foo', 'bar') session.add(myobject) session.commit() - + # close when finished session.close() -Above, the ``sessionmaker`` call creates a class for us, which we assign to the name ``Session``. This class is a subclass of the actual ``sqlalchemy.orm.session.Session`` class, which will instantiate with a particular bound engine. +Above, the ``sessionmaker`` call creates a class for us, which we assign to the name :class:`~sqlalchemy.orm.session.Session`. This class is a subclass of the actual ``sqlalchemy.orm.session.Session`` class, which will instantiate with a particular bound engine. -When you write your application, place the call to ``sessionmaker()`` somewhere global, and then make your new ``Session`` class available to the rest of your application. +When you write your application, place the call to ``sessionmaker()`` somewhere global, and then make your new :class:`~sqlalchemy.orm.session.Session` class available to the rest of your application. -Binding Session to an Engine +Binding Session to an Engine ---------------------------- -In our previous example regarding ``sessionmaker()``, we specified a ``bind`` for a particular ``Engine``. If we'd like to construct a ``sessionmaker()`` without an engine available and bind it later on, or to specify other options to an existing ``sessionmaker()``, we may use the ``configure()`` method:: +In our previous example regarding ``sessionmaker()``, we specified a ``bind`` for a particular :class:`~sqlalchemy.engine.base.Engine`. If we'd like to construct a ``sessionmaker()`` without an engine available and bind it later on, or to specify other options to an existing ``sessionmaker()``, we may use the ``configure()`` method:: # configure Session class with desired options Session = sessionmaker() # later, we create the engine engine = create_engine('postgresql://...') - + # associate it with our custom Session class Session.configure(bind=engine) # work with the session session = Session() -It's actually entirely optional to bind a Session to an engine. If the underlying mapped ``Table`` objects use "bound" metadata, the ``Session`` will make use of the bound engine instead (or will even use multiple engines if multiple binds are present within the mapped tables). "Bound" metadata is described at :ref:`metadata_binding`. +It's actually entirely optional to bind a Session to an engine. If the underlying mapped :class:`~sqlalchemy.schema.Table` objects use "bound" metadata, the :class:`~sqlalchemy.orm.session.Session` will make use of the bound engine instead (or will even use multiple engines if multiple binds are present within the mapped tables). "Bound" metadata is described at :ref:`metadata_binding`. -The ``Session`` also has the ability to be bound to multiple engines explicitly. Descriptions of these scenarios are described in :ref:`session_partitioning`. +The :class:`~sqlalchemy.orm.session.Session` also has the ability to be bound to multiple engines explicitly. Descriptions of these scenarios are described in :ref:`session_partitioning`. -Binding Session to a Connection +Binding Session to a Connection ------------------------------- -The ``Session`` can also be explicitly bound to an individual database ``Connection``. Reasons for doing this may include to join a ``Session`` with an ongoing transaction local to a specific ``Connection`` object, or to bypass connection pooling by just having connections persistently checked out and associated with distinct, long running sessions:: +The :class:`~sqlalchemy.orm.session.Session` can also be explicitly bound to an individual database :class:`~sqlalchemy.engine.base.Connection`. Reasons for doing this may include to join a :class:`~sqlalchemy.orm.session.Session` with an ongoing transaction local to a specific :class:`~sqlalchemy.engine.base.Connection` object, or to bypass connection pooling by just having connections persistently checked out and associated with distinct, long running sessions:: # global application scope. create Session class, engine Session = sessionmaker() engine = create_engine('postgresql://...') - + ... - + # local scope, such as within a controller function - + # connect to the database connection = engine.connect() - + # bind an individual Session to the connection session = Session(bind=connection) -Using create_session() +Using create_session() ---------------------- -As an alternative to ``sessionmaker()``, ``create_session()`` is a function which calls the normal ``Session`` constructor directly. All arguments are passed through and the new ``Session`` object is returned:: +As an alternative to ``sessionmaker()``, ``create_session()`` is a function which calls the normal :class:`~sqlalchemy.orm.session.Session` constructor directly. All arguments are passed through and the new :class:`~sqlalchemy.orm.session.Session` object is returned:: session = create_session(bind=myengine, autocommit=True, autoflush=False) Note that ``create_session()`` disables all optional "automation" by default. Called with no arguments, the session produced is not autoflushing, does not auto-expire, and does not maintain a transaction (i.e. it begins and commits a new transaction for each ``flush()``). SQLAlchemy uses ``create_session()`` extensively within its own unit tests. -Configurational Arguments +Configurational Arguments ------------------------- -Configurational arguments accepted by ``sessionmaker()`` and ``create_session()`` are the same as that of the ``Session`` class itself, and are described at :func:`sqlalchemy.orm.sessionmaker`. +Configurational arguments accepted by ``sessionmaker()`` and ``create_session()`` are the same as that of the :class:`~sqlalchemy.orm.session.Session` class itself, and are described at :func:`sqlalchemy.orm.sessionmaker`. Note that the defaults of ``create_session()`` are the opposite of that of ``sessionmaker()``: autoflush and expire_on_commit are False, autocommit is True. It is recommended to use the ``sessionmaker()`` function instead of ``create_session()``. ``create_session()`` is used to get a session with no automation turned on and is useful for testing. -Using the Session +Using the Session ================== -Quickie Intro to Object States +Quickie Intro to Object States ------------------------------ It's helpful to know the states which an instance can have within a session: @@ -118,36 +118,36 @@ It's helpful to know the states which an instance can have within a session: * *Detached* - an instance which has a record in the database, but is not in any session. There's nothing wrong with this, and you can use objects normally when they're detached, **except** they will not be able to issue any SQL in order to load collections or attributes which are not yet loaded, or were marked as "expired". -Knowing these states is important, since the ``Session`` tries to be strict about ambiguous operations (such as trying to save the same object to two different sessions at the same time). +Knowing these states is important, since the :class:`~sqlalchemy.orm.session.Session` tries to be strict about ambiguous operations (such as trying to save the same object to two different sessions at the same time). -Frequently Asked Questions +Frequently Asked Questions -------------------------- * When do I make a ``sessionmaker`` ? Just one time, somewhere in your application's global scope. It should be looked upon as part of your application's configuration. If your application has three .py files in a package, you could, for example, place the ``sessionmaker`` line in your ``__init__.py`` file; from that point on your other modules say "from mypackage import Session". That way, everyone else just uses ``Session()``, and the configuration of that session is controlled by that central point. - If your application starts up, does imports, but does not know what database it's going to be connecting to, you can bind the ``Session`` at the "class" level to the engine later on, using ``configure()``. + If your application starts up, does imports, but does not know what database it's going to be connecting to, you can bind the :class:`~sqlalchemy.orm.session.Session` at the "class" level to the engine later on, using ``configure()``. In the examples in this section, we will frequently show the ``sessionmaker`` being created right above the line where we actually invoke ``Session()``. But that's just for example's sake ! In reality, the ``sessionmaker`` would be somewhere at the module level, and your individual ``Session()`` calls would be sprinkled all throughout your app, such as in a web application within each controller method. -* When do I make a ``Session`` ? +* When do I make a :class:`~sqlalchemy.orm.session.Session` ? You typically invoke ``Session()`` when you first need to talk to your database, and want to save some objects or load some existing ones. Then, you work with it, save your changes, and then dispose of it....or at the very least ``close()`` it. It's not a "global" kind of object, and should be handled more like a "local variable", as it's generally **not** safe to use with concurrent threads. Sessions are very inexpensive to make, and don't use any resources whatsoever until they are first used...so create some ! - There is also a pattern whereby you're using a **contextual session**, this is described later in :ref:`unitofwork_contextual`. In this pattern, a helper object is maintaining a ``Session`` for you, most commonly one that is local to the current thread (and sometimes also local to an application instance). SQLAlchemy has worked this pattern out such that it still *looks* like you're creating a new session as you need one...so in that case, it's still a guaranteed win to just say ``Session()`` whenever you want a session. + There is also a pattern whereby you're using a **contextual session**, this is described later in :ref:`unitofwork_contextual`. In this pattern, a helper object is maintaining a :class:`~sqlalchemy.orm.session.Session` for you, most commonly one that is local to the current thread (and sometimes also local to an application instance). SQLAlchemy has worked this pattern out such that it still *looks* like you're creating a new session as you need one...so in that case, it's still a guaranteed win to just say ``Session()`` whenever you want a session. + +* Is the Session a cache ? -* Is the Session a cache ? + Yeee...no. It's somewhat used as a cache, in that it implements the identity map pattern, and stores objects keyed to their primary key. However, it doesn't do any kind of query caching. This means, if you say ``session.query(Foo).filter_by(name='bar')``, even if ``Foo(name='bar')`` is right there, in the identity map, the session has no idea about that. It has to issue SQL to the database, get the rows back, and then when it sees the primary key in the row, *then* it can look in the local identity map and see that the object is already there. It's only when you say ``query.get({some primary key})`` that the :class:`~sqlalchemy.orm.session.Session` doesn't have to issue a query. - Yeee...no. It's somewhat used as a cache, in that it implements the identity map pattern, and stores objects keyed to their primary key. However, it doesn't do any kind of query caching. This means, if you say ``session.query(Foo).filter_by(name='bar')``, even if ``Foo(name='bar')`` is right there, in the identity map, the session has no idea about that. It has to issue SQL to the database, get the rows back, and then when it sees the primary key in the row, *then* it can look in the local identity map and see that the object is already there. It's only when you say ``query.get({some primary key})`` that the ``Session`` doesn't have to issue a query. - Additionally, the Session stores object instances using a weak reference by default. This also defeats the purpose of using the Session as a cache, unless the ``weak_identity_map`` flag is set to ``False``. - The ``Session`` is not designed to be a global object from which everyone consults as a "registry" of objects. That is the job of a **second level cache**. A good library for implementing second level caching is `Memcached `_. It *is* possible to "sort of" use the ``Session`` in this manner, if you set it to be non-transactional and it never flushes any SQL, but it's not a terrific solution, since if concurrent threads load the same objects at the same time, you may have multiple copies of the same objects present in collections. + The :class:`~sqlalchemy.orm.session.Session` is not designed to be a global object from which everyone consults as a "registry" of objects. That is the job of a **second level cache**. A good library for implementing second level caching is `Memcached `_. It *is* possible to "sort of" use the :class:`~sqlalchemy.orm.session.Session` in this manner, if you set it to be non-transactional and it never flushes any SQL, but it's not a terrific solution, since if concurrent threads load the same objects at the same time, you may have multiple copies of the same objects present in collections. -* How can I get the ``Session`` for a certain object ? +* How can I get the :class:`~sqlalchemy.orm.session.Session` for a certain object ? - Use the ``object_session()`` classmethod available on ``Session``:: + Use the ``object_session()`` classmethod available on :class:`~sqlalchemy.orm.session.Session`:: session = Session.object_session(someobject) @@ -160,11 +160,11 @@ Frequently Asked Questions Nope. It has no thread synchronization of any kind built in, and particularly when you do a flush operation, it definitely is not open to concurrent threads accessing it, because it holds onto a single database connection at that point. If you use a session which is non-transactional for read operations only, it's still not thread-"safe", but you also wont get any catastrophic failures either, since it opens and closes connections on an as-needed basis; it's just that different threads might load the same objects independently of each other, but only one will wind up in the identity map (however, the other one might still live in a collection somewhere). But the bigger point here is, you should not *want* to use the session with multiple concurrent threads. That would be like having everyone at a restaurant all eat from the same plate. The session is a local "workspace" that you use for a specific set of tasks; you don't want to, or need to, share that session with other threads who are doing some other task. If, on the other hand, there are other threads participating in the same task you are, such as in a desktop graphical application, then you would be sharing the session with those threads, but you also will have implemented a proper locking scheme (or your graphical framework does) so that those threads do not collide. - + Querying -------- -The ``query()`` function takes one or more *entities* and returns a new ``Query`` object which will issue mapper queries within the context of this Session. An entity is defined as a mapped class, a ``Mapper`` object, an orm-enabled *descriptor*, or an ``AliasedClass`` object:: +The ``query()`` function takes one or more *entities* and returns a new :class:`~sqlalchemy.orm.query.Query` object which will issue mapper queries within the context of this Session. An entity is defined as a mapped class, a :class:`~sqlalchemy.orm.mapper.Mapper` object, an orm-enabled *descriptor*, or an ``AliasedClass`` object:: # query from a class session.query(User).filter_by(name='ed').all() @@ -174,12 +174,12 @@ The ``query()`` function takes one or more *entities* and returns a new ``Query` # query using orm-enabled descriptors session.query(User.name, User.fullname).all() - + # query from a mapper user_mapper = class_mapper(User) session.query(user_mapper) -When ``Query`` returns results, each object instantiated is stored within the identity map. When a row matches an object which is already present, the same object is returned. In the latter case, whether or not the row is populated onto an existing object depends upon whether the attributes of the instance have been *expired* or not. A default-configured ``Session`` automatically expires all instances along transaction boundaries, so that with a normally isolated transaction, there shouldn't be any issue of instances representing data which is stale with regards to the current transaction. +When :class:`~sqlalchemy.orm.query.Query` returns results, each object instantiated is stored within the identity map. When a row matches an object which is already present, the same object is returned. In the latter case, whether or not the row is populated onto an existing object depends upon whether the attributes of the instance have been *expired* or not. A default-configured :class:`~sqlalchemy.orm.session.Session` automatically expires all instances along transaction boundaries, so that with a normally isolated transaction, there shouldn't be any issue of instances representing data which is stale with regards to the current transaction. Adding New or Existing Items ---------------------------- @@ -190,7 +190,7 @@ Adding New or Existing Items user2 = User(name='user2') session.add(user1) session.add(user2) - + session.commit() # write changes to the database To add a list of items to the session at once, use ``add_all()``:: @@ -216,10 +216,10 @@ When given an instance, it follows these steps: With ``merge()``, the given instance is not placed within the session, and can be associated with a different session or detached. ``merge()`` is very useful for taking the state of any kind of object structure without regard for its origins or current session associations and placing that state within a session. Here's two examples: - * An application which reads an object structure from a file and wishes to save it to the database might parse the file, build up the structure, and then use ``merge()`` to save it to the database, ensuring that the data within the file is used to formulate the primary key of each element of the structure. Later, when the file has changed, the same process can be re-run, producing a slightly different object structure, which can then be ``merged()`` in again, and the ``Session`` will automatically update the database to reflect those changes. + * An application which reads an object structure from a file and wishes to save it to the database might parse the file, build up the structure, and then use ``merge()`` to save it to the database, ensuring that the data within the file is used to formulate the primary key of each element of the structure. Later, when the file has changed, the same process can be re-run, producing a slightly different object structure, which can then be ``merged()`` in again, and the :class:`~sqlalchemy.orm.session.Session` will automatically update the database to reflect those changes. * A web application stores mapped entities within an HTTP session object. When each request starts up, the serialized data can be merged into the session, so that the original entity may be safely shared among requests and threads. -``merge()`` is frequently used by applications which implement their own second level caches. This refers to an application which uses an in memory dictionary, or an tool like Memcached to store objects over long running spans of time. When such an object needs to exist within a ``Session``, ``merge()`` is a good choice since it leaves the original cached object untouched. For this use case, merge provides a keyword option called ``load=False``. When this boolean flag is set to ``False``, ``merge()`` will not issue any SQL to reconcile the given object against the current state of the database, thereby reducing query overhead. The limitation is that the given object and all of its children may not contain any pending changes, and it's also of course possible that newer information in the database will not be present on the merged object, since no load is issued. +``merge()`` is frequently used by applications which implement their own second level caches. This refers to an application which uses an in memory dictionary, or an tool like Memcached to store objects over long running spans of time. When such an object needs to exist within a :class:`~sqlalchemy.orm.session.Session`, ``merge()`` is a good choice since it leaves the original cached object untouched. For this use case, merge provides a keyword option called ``load=False``. When this boolean flag is set to ``False``, ``merge()`` will not issue any SQL to reconcile the given object against the current state of the database, thereby reducing query overhead. The limitation is that the given object and all of its children may not contain any pending changes, and it's also of course possible that newer information in the database will not be present on the merged object, since no load is issued. Deleting -------- @@ -255,24 +255,24 @@ Deleting based on Filter Criterion The caveat with ``Session.delete()`` is that you need to have an object handy already in order to delete. The Query includes a ``delete()`` method which deletes based on filtering criteria:: session.query(User).filter(User.id==7).delete() - -The ``Query.delete()`` method includes functionality to "expire" objects already in the session which -match the criteria. However it does have some caveats, including that "delete" and "delete-orphan" + +The ``Query.delete()`` method includes functionality to "expire" objects already in the session which +match the criteria. However it does have some caveats, including that "delete" and "delete-orphan" cascades won't be fully expressed for collections which are already loaded. See the API docs for :meth:`~sqlalchemy.orm.query.Query.delete` for more details. Flushing -------- -When the ``Session`` is used with its default configuration, the flush step is nearly always done transparently. Specifically, the flush occurs before any individual ``Query`` is issued, as well as within the ``commit()`` call before the transaction is committed. It also occurs before a SAVEPOINT is issued when ``begin_nested()`` is used. +When the :class:`~sqlalchemy.orm.session.Session` is used with its default configuration, the flush step is nearly always done transparently. Specifically, the flush occurs before any individual :class:`~sqlalchemy.orm.query.Query` is issued, as well as within the ``commit()`` call before the transaction is committed. It also occurs before a SAVEPOINT is issued when ``begin_nested()`` is used. Regardless of the autoflush setting, a flush can always be forced by issuing ``flush()``:: session.flush() - + The "flush-on-Query" aspect of the behavior can be disabled by constructing ``sessionmaker()`` with the flag ``autoflush=False``:: Session = sessionmaker(autoflush=False) - + Additionally, autoflush can be temporarily disabled by setting the ``autoflush`` flag at any time:: mysession = Session() @@ -280,16 +280,16 @@ Additionally, autoflush can be temporarily disabled by setting the ``autoflush`` Some autoflush-disable recipes are available at `DisableAutoFlush `_. -The flush process *always* occurs within a transaction, even if the ``Session`` has been configured with ``autocommit=True``, a setting that disables the session's persistent transactional state. If no transaction is present, ``flush()`` creates its own transaction and commits it. Any failures during flush will always result in a rollback of whatever transaction is present. If the Session is not in ``autocommit=True`` mode, an explicit call to ``rollback()`` is required after a flush fails, even though the underlying transaction will have been rolled back already - this is so that the overall nesting pattern of so-called "subtransactions" is consistently maintained. +The flush process *always* occurs within a transaction, even if the :class:`~sqlalchemy.orm.session.Session` has been configured with ``autocommit=True``, a setting that disables the session's persistent transactional state. If no transaction is present, ``flush()`` creates its own transaction and commits it. Any failures during flush will always result in a rollback of whatever transaction is present. If the Session is not in ``autocommit=True`` mode, an explicit call to ``rollback()`` is required after a flush fails, even though the underlying transaction will have been rolled back already - this is so that the overall nesting pattern of so-called "subtransactions" is consistently maintained. Committing ---------- -``commit()`` is used to commit the current transaction. It always issues ``flush()`` beforehand to flush any remaining state to the database; this is independent of the "autoflush" setting. If no transaction is present, it raises an error. Note that the default behavior of the ``Session`` is that a transaction is always present; this behavior can be disabled by setting ``autocommit=True``. In autocommit mode, a transaction can be initiated by calling the ``begin()`` method. +``commit()`` is used to commit the current transaction. It always issues ``flush()`` beforehand to flush any remaining state to the database; this is independent of the "autoflush" setting. If no transaction is present, it raises an error. Note that the default behavior of the :class:`~sqlalchemy.orm.session.Session` is that a transaction is always present; this behavior can be disabled by setting ``autocommit=True``. In autocommit mode, a transaction can be initiated by calling the ``begin()`` method. -Another behavior of ``commit()`` is that by default it expires the state of all instances present after the commit is complete. This is so that when the instances are next accessed, either through attribute access or by them being present in a ``Query`` result set, they receive the most recent state. To disable this behavior, configure ``sessionmaker()`` with ``expire_on_commit=False``. +Another behavior of ``commit()`` is that by default it expires the state of all instances present after the commit is complete. This is so that when the instances are next accessed, either through attribute access or by them being present in a :class:`~sqlalchemy.orm.query.Query` result set, they receive the most recent state. To disable this behavior, configure ``sessionmaker()`` with ``expire_on_commit=False``. -Normally, instances loaded into the ``Session`` are never changed by subsequent queries; the assumption is that the current transaction is isolated so the state most recently loaded is correct as long as the transaction continues. Setting ``autocommit=True`` works against this model to some degree since the ``Session`` behaves in exactly the same way with regard to attribute state, except no transaction is present. +Normally, instances loaded into the :class:`~sqlalchemy.orm.session.Session` are never changed by subsequent queries; the assumption is that the current transaction is isolated so the state most recently loaded is correct as long as the transaction continues. Setting ``autocommit=True`` works against this model to some degree since the :class:`~sqlalchemy.orm.session.Session` behaves in exactly the same way with regard to attribute state, except no transaction is present. Rolling Back ------------ @@ -297,13 +297,13 @@ Rolling Back ``rollback()`` rolls back the current transaction. With a default configured session, the post-rollback state of the session is as follows: * All connections are rolled back and returned to the connection pool, unless the Session was bound directly to a Connection, in which case the connection is still maintained (but still rolled back). - * Objects which were initially in the *pending* state when they were added to the ``Session`` within the lifespan of the transaction are expunged, corresponding to their INSERT statement being rolled back. The state of their attributes remains unchanged. + * Objects which were initially in the *pending* state when they were added to the :class:`~sqlalchemy.orm.session.Session` within the lifespan of the transaction are expunged, corresponding to their INSERT statement being rolled back. The state of their attributes remains unchanged. * Objects which were marked as *deleted* within the lifespan of the transaction are promoted back to the *persistent* state, corresponding to their DELETE statement being rolled back. Note that if those objects were first *pending* within the transaction, that operation takes precedence instead. - * All objects not expunged are fully expired. + * All objects not expunged are fully expired. -With that state understood, the ``Session`` may safely continue usage after a rollback occurs. +With that state understood, the :class:`~sqlalchemy.orm.session.Session` may safely continue usage after a rollback occurs. -When a ``flush()`` fails, typically for reasons like primary key, foreign key, or "not nullable" constraint violations, a ``rollback()`` is issued automatically (it's currently not possible for a flush to continue after a partial failure). However, the flush process always uses its own transactional demarcator called a *subtransaction*, which is described more fully in the docstrings for ``Session``. What it means here is that even though the database transaction has been rolled back, the end user must still issue ``rollback()`` to fully reset the state of the ``Session``. +When a ``flush()`` fails, typically for reasons like primary key, foreign key, or "not nullable" constraint violations, a ``rollback()`` is issued automatically (it's currently not possible for a flush to continue after a partial failure). However, the flush process always uses its own transactional demarcator called a *subtransaction*, which is described more fully in the docstrings for :class:`~sqlalchemy.orm.session.Session`. What it means here is that even though the database transaction has been rolled back, the end user must still issue ``rollback()`` to fully reset the state of the :class:`~sqlalchemy.orm.session.Session`. Expunging --------- @@ -313,7 +313,7 @@ Expunge removes an object from the Session, sending persistent instances to the .. sourcecode:: python+sql session.expunge(obj1) - + To remove all items, call ``session.expunge_all()`` (this method was formerly known as ``clear()``). Closing @@ -329,7 +329,7 @@ To assist with the Session's "sticky" behavior of instances which are present, i # immediately re-load attributes on obj1, obj2 session.refresh(obj1) session.refresh(obj2) - + # expire objects obj1, obj2, attributes will be reloaded # on the next access: session.expire(obj1) @@ -340,7 +340,7 @@ To assist with the Session's "sticky" behavior of instances which are present, i # immediately re-load the attributes 'hello', 'world' on obj1, obj2 session.refresh(obj1, ['hello', 'world']) session.refresh(obj2, ['hello', 'world']) - + # expire the attributes 'hello', 'world' objects obj1, obj2, attributes will be reloaded # on the next access: session.expire(obj1, ['hello', 'world']) @@ -350,12 +350,12 @@ The full contents of the session may be expired at once using ``expire_all()``:: session.expire_all() -``refresh()`` and ``expire()`` are usually not needed when working with a default-configured ``Session``. The usual need is when an UPDATE or DELETE has been issued manually within the transaction using ``Session.execute()``. +``refresh()`` and ``expire()`` are usually not needed when working with a default-configured :class:`~sqlalchemy.orm.session.Session`. The usual need is when an UPDATE or DELETE has been issued manually within the transaction using ``Session.execute()``. -Session Attributes +Session Attributes ------------------ -The ``Session`` itself acts somewhat like a set-like collection. All items present may be accessed using the iterator interface:: +The :class:`~sqlalchemy.orm.session.Session` itself acts somewhat like a set-like collection. All items present may be accessed using the iterator interface:: for obj in session: print obj @@ -377,7 +377,7 @@ The session is also keeping track of all newly created (i.e. pending) objects, a # persistent objects that have been marked as deleted via session.delete(obj) session.deleted -Note that objects within the session are by default *weakly referenced*. This means that when they are dereferenced in the outside application, they fall out of scope from within the ``Session`` as well and are subject to garbage collection by the Python interpreter. The exceptions to this include objects which are pending, objects which are marked as deleted, or persistent objects which have pending changes on them. After a full flush, these collections are all empty, and all objects are again weakly referenced. To disable the weak referencing behavior and force all objects within the session to remain until explicitly expunged, configure ``sessionmaker()`` with the ``weak_identity_map=False`` setting. +Note that objects within the session are by default *weakly referenced*. This means that when they are dereferenced in the outside application, they fall out of scope from within the :class:`~sqlalchemy.orm.session.Session` as well and are subject to garbage collection by the Python interpreter. The exceptions to this include objects which are pending, objects which are marked as deleted, or persistent objects which have pending changes on them. After a full flush, these collections are all empty, and all objects are again weakly referenced. To disable the weak referencing behavior and force all objects within the session to remain until explicitly expunged, configure ``sessionmaker()`` with the ``weak_identity_map=False`` setting. .. _unitofwork_cascades: @@ -406,9 +406,9 @@ The default value for ``cascade`` on :func:`~sqlalchemy.orm.relation()` is ``sav Managing Transactions ===================== -The ``Session`` manages transactions across all engines associated with it. As the ``Session`` receives requests to execute SQL statements using a particular ``Engine`` or ``Connection``, it adds each individual ``Engine`` encountered to its transactional state and maintains an open connection for each one (note that a simple application normally has just one ``Engine``). At commit time, all unflushed data is flushed, and each individual transaction is committed. If the underlying databases support two-phase semantics, this may be used by the Session as well if two-phase transactions are enabled. +The :class:`~sqlalchemy.orm.session.Session` manages transactions across all engines associated with it. As the :class:`~sqlalchemy.orm.session.Session` receives requests to execute SQL statements using a particular :class:`~sqlalchemy.engine.base.Engine` or :class:`~sqlalchemy.engine.base.Connection`, it adds each individual :class:`~sqlalchemy.engine.base.Engine` encountered to its transactional state and maintains an open connection for each one (note that a simple application normally has just one :class:`~sqlalchemy.engine.base.Engine`). At commit time, all unflushed data is flushed, and each individual transaction is committed. If the underlying databases support two-phase semantics, this may be used by the Session as well if two-phase transactions are enabled. -Normal operation ends the transactional state using the ``rollback()`` or ``commit()`` methods. After either is called, the ``Session`` starts a new transaction:: +Normal operation ends the transactional state using the ``rollback()`` or ``commit()`` methods. After either is called, the :class:`~sqlalchemy.orm.session.Session` starts a new transaction:: Session = sessionmaker() session = Session() @@ -417,7 +417,7 @@ Normal operation ends the transactional state using the ``rollback()`` or ``comm item2 = session.query(Item).get(2) item1.foo = 'bar' item2.bar = 'foo' - + # commit- will immediately go into a new transaction afterwards session.commit() except: @@ -449,7 +449,7 @@ The ``begin()`` method also returns a transactional token which is compatible wi item1.foo = 'bar' item2.bar = 'foo' -Using SAVEPOINT +Using SAVEPOINT --------------- SAVEPOINT transactions, if supported by the underlying engine, may be delineated using the ``begin_nested()`` method:: @@ -465,32 +465,32 @@ SAVEPOINT transactions, if supported by the underlying engine, may be delineated session.commit() # commits u1 and u2 -``begin_nested()`` may be called any number of times, which will issue a new SAVEPOINT with a unique identifier for each call. For each ``begin_nested()`` call, a corresponding ``rollback()`` or ``commit()`` must be issued. +``begin_nested()`` may be called any number of times, which will issue a new SAVEPOINT with a unique identifier for each call. For each ``begin_nested()`` call, a corresponding ``rollback()`` or ``commit()`` must be issued. -When ``begin_nested()`` is called, a ``flush()`` is unconditionally issued (regardless of the ``autoflush`` setting). This is so that when a ``rollback()`` occurs, the full state of the session is expired, thus causing all subsequent attribute/instance access to reference the full state of the ``Session`` right before ``begin_nested()`` was called. +When ``begin_nested()`` is called, a ``flush()`` is unconditionally issued (regardless of the ``autoflush`` setting). This is so that when a ``rollback()`` occurs, the full state of the session is expired, thus causing all subsequent attribute/instance access to reference the full state of the :class:`~sqlalchemy.orm.session.Session` right before ``begin_nested()`` was called. -Enabling Two-Phase Commit +Enabling Two-Phase Commit ------------------------- Finally, for MySQL, PostgreSQL, and soon Oracle as well, the session can be instructed to use two-phase commit semantics. This will coordinate the committing of transactions across databases so that the transaction is either committed or rolled back in all databases. You can also ``prepare()`` the session for interacting with transactions not managed by SQLAlchemy. To use two phase transactions set the flag ``twophase=True`` on the session:: engine1 = create_engine('postgresql://db1') engine2 = create_engine('postgresql://db2') - + Session = sessionmaker(twophase=True) # bind User operations to engine 1, Account operations to engine 2 Session.configure(binds={User:engine1, Account:engine2}) session = Session() - + # .... work with accounts and users - + # commit. session will issue a flush to all DBs, and a prepare step to all DBs, # before committing both transactions session.commit() -Embedding SQL Insert/Update Expressions into a Flush +Embedding SQL Insert/Update Expressions into a Flush ===================================================== This feature allows the value of a database column to be set to a SQL expression instead of a literal value. It's especially useful for atomic updates, calling stored procedures, etc. All you do is assign an expression to an attribute:: @@ -498,40 +498,40 @@ This feature allows the value of a database column to be set to a SQL expression class SomeClass(object): pass mapper(SomeClass, some_table) - + someobject = session.query(SomeClass).get(5) - + # set 'value' attribute to a SQL expression adding one someobject.value = some_table.c.value + 1 - + # issues "UPDATE some_table SET value=value+1" session.commit() - -This technique works both for INSERT and UPDATE statements. After the flush/commit operation, the ``value`` attribute on ``someobject`` above is expired, so that when next accessed the newly generated value will be loaded from the database. -Using SQL Expressions with Sessions +This technique works both for INSERT and UPDATE statements. After the flush/commit operation, the ``value`` attribute on ``someobject`` above is expired, so that when next accessed the newly generated value will be loaded from the database. + +Using SQL Expressions with Sessions ==================================== -SQL expressions and strings can be executed via the ``Session`` within its transactional context. This is most easily accomplished using the ``execute()`` method, which returns a ``ResultProxy`` in the same manner as an ``Engine`` or ``Connection``:: +SQL expressions and strings can be executed via the :class:`~sqlalchemy.orm.session.Session` within its transactional context. This is most easily accomplished using the ``execute()`` method, which returns a :class:`~sqlalchemy.engine.base.ResultProxy` in the same manner as an :class:`~sqlalchemy.engine.base.Engine` or :class:`~sqlalchemy.engine.base.Connection`:: Session = sessionmaker(bind=engine) session = Session() - + # execute a string statement result = session.execute("select * from table where id=:id", {'id':7}) - + # execute a SQL expression construct result = session.execute(select([mytable]).where(mytable.c.id==7)) -The current ``Connection`` held by the ``Session`` is accessible using the ``connection()`` method:: +The current :class:`~sqlalchemy.engine.base.Connection` held by the :class:`~sqlalchemy.orm.session.Session` is accessible using the ``connection()`` method:: connection = session.connection() -The examples above deal with a ``Session`` that's bound to a single ``Engine`` or ``Connection``. To execute statements using a ``Session`` which is bound either to multiple engines, or none at all (i.e. relies upon bound metadata), both ``execute()`` and ``connection()`` accept a ``mapper`` keyword argument, which is passed a mapped class or ``Mapper`` instance, which is used to locate the proper context for the desired engine:: +The examples above deal with a :class:`~sqlalchemy.orm.session.Session` that's bound to a single :class:`~sqlalchemy.engine.base.Engine` or :class:`~sqlalchemy.engine.base.Connection`. To execute statements using a :class:`~sqlalchemy.orm.session.Session` which is bound either to multiple engines, or none at all (i.e. relies upon bound metadata), both ``execute()`` and ``connection()`` accept a ``mapper`` keyword argument, which is passed a mapped class or :class:`~sqlalchemy.orm.mapper.Mapper` instance, which is used to locate the proper context for the desired engine:: Session = sessionmaker() session = Session() - + # need to specify mapper or class when executing result = session.execute("select * from table where id=:id", {'id':7}, mapper=MyMappedClass) @@ -539,80 +539,80 @@ The examples above deal with a ``Session`` that's bound to a single ``Engine`` o connection = session.connection(MyMappedClass) -Joining a Session into an External Transaction +Joining a Session into an External Transaction =============================================== -If a ``Connection`` is being used which is already in a transactional state (i.e. has a ``Transaction``), a ``Session`` can be made to participate within that transaction by just binding the ``Session`` to that ``Connection``:: +If a :class:`~sqlalchemy.engine.base.Connection` is being used which is already in a transactional state (i.e. has a :class:`~sqlalchemy.engine.base.Transaction`), a :class:`~sqlalchemy.orm.session.Session` can be made to participate within that transaction by just binding the :class:`~sqlalchemy.orm.session.Session` to that :class:`~sqlalchemy.engine.base.Connection`:: Session = sessionmaker() - + # non-ORM connection + transaction conn = engine.connect() trans = conn.begin() - + # create a Session, bind to the connection session = Session(bind=conn) - + # ... work with session - + session.commit() # commit the session session.close() # close it out, prohibit further actions - + trans.commit() # commit the actual transaction -Note that above, we issue a ``commit()`` both on the ``Session`` as well as the ``Transaction``. This is an example of where we take advantage of ``Connection``'s ability to maintain *subtransactions*, or nested begin/commit pairs. The ``Session`` is used exactly as though it were managing the transaction on its own; its ``commit()`` method issues its ``flush()``, and commits the subtransaction. The subsequent transaction the ``Session`` starts after commit will not begin until it's next used. Above we issue a ``close()`` to prevent this from occurring. Finally, the actual transaction is committed using ``Transaction.commit()``. +Note that above, we issue a ``commit()`` both on the :class:`~sqlalchemy.orm.session.Session` as well as the :class:`~sqlalchemy.engine.base.Transaction`. This is an example of where we take advantage of :class:`~sqlalchemy.engine.base.Connection`'s ability to maintain *subtransactions*, or nested begin/commit pairs. The :class:`~sqlalchemy.orm.session.Session` is used exactly as though it were managing the transaction on its own; its ``commit()`` method issues its ``flush()``, and commits the subtransaction. The subsequent transaction the :class:`~sqlalchemy.orm.session.Session` starts after commit will not begin until it's next used. Above we issue a ``close()`` to prevent this from occurring. Finally, the actual transaction is committed using ``Transaction.commit()``. -When using the ``threadlocal`` engine context, the process above is simplified; the ``Session`` uses the same connection/transaction as everyone else in the current thread, whether or not you explicitly bind it:: +When using the ``threadlocal`` engine context, the process above is simplified; the :class:`~sqlalchemy.orm.session.Session` uses the same connection/transaction as everyone else in the current thread, whether or not you explicitly bind it:: engine = create_engine('postgresql://mydb', strategy="threadlocal") engine.begin() - + session = Session() # session takes place in the transaction like everyone else - + # ... go nuts - + engine.commit() # commit the transaction .. _unitofwork_contextual: -Contextual/Thread-local Sessions +Contextual/Thread-local Sessions ================================= -A common need in applications, particularly those built around web frameworks, is the ability to "share" a ``Session`` object among disparate parts of an application, without needing to pass the object explicitly to all method and function calls. What you're really looking for is some kind of "global" session object, or at least "global" to all the parts of an application which are tasked with servicing the current request. For this pattern, SQLAlchemy provides the ability to enhance the ``Session`` class generated by ``sessionmaker()`` to provide auto-contextualizing support. This means that whenever you create a ``Session`` instance with its constructor, you get an *existing* ``Session`` object which is bound to some "context". By default, this context is the current thread. This feature is what previously was accomplished using the ``sessioncontext`` SQLAlchemy extension. +A common need in applications, particularly those built around web frameworks, is the ability to "share" a :class:`~sqlalchemy.orm.session.Session` object among disparate parts of an application, without needing to pass the object explicitly to all method and function calls. What you're really looking for is some kind of "global" session object, or at least "global" to all the parts of an application which are tasked with servicing the current request. For this pattern, SQLAlchemy provides the ability to enhance the :class:`~sqlalchemy.orm.session.Session` class generated by ``sessionmaker()`` to provide auto-contextualizing support. This means that whenever you create a :class:`~sqlalchemy.orm.session.Session` instance with its constructor, you get an *existing* :class:`~sqlalchemy.orm.session.Session` object which is bound to some "context". By default, this context is the current thread. This feature is what previously was accomplished using the ``sessioncontext`` SQLAlchemy extension. -Creating a Thread-local Context +Creating a Thread-local Context ------------------------------- -The ``scoped_session()`` function wraps around the ``sessionmaker()`` function, and produces an object which behaves the same as the ``Session`` subclass returned by ``sessionmaker()``:: +The ``scoped_session()`` function wraps around the ``sessionmaker()`` function, and produces an object which behaves the same as the :class:`~sqlalchemy.orm.session.Session` subclass returned by ``sessionmaker()``:: from sqlalchemy.orm import scoped_session, sessionmaker Session = scoped_session(sessionmaker()) - -However, when you instantiate this ``Session`` "class", in reality the object is pulled from a threadlocal variable, or if it doesn't exist yet, it's created using the underlying class generated by ``sessionmaker()``:: + +However, when you instantiate this :class:`~sqlalchemy.orm.session.Session` "class", in reality the object is pulled from a threadlocal variable, or if it doesn't exist yet, it's created using the underlying class generated by ``sessionmaker()``:: >>> # call Session() the first time. the new Session instance is created. >>> session = Session() - + >>> # later, in the same application thread, someone else calls Session() >>> session2 = Session() - + >>> # the two Session objects are *the same* object >>> session is session2 True -Since the ``Session()`` constructor now returns the same ``Session`` object every time within the current thread, the object returned by ``scoped_session()`` also implements most of the ``Session`` methods and properties at the "class" level, such that you don't even need to instantiate ``Session()``:: +Since the ``Session()`` constructor now returns the same :class:`~sqlalchemy.orm.session.Session` object every time within the current thread, the object returned by ``scoped_session()`` also implements most of the :class:`~sqlalchemy.orm.session.Session` methods and properties at the "class" level, such that you don't even need to instantiate ``Session()``:: # create some objects u1 = User() u2 = User() - + # save to the contextual session, without instantiating Session.add(u1) Session.add(u2) - + # view the "new" attribute assert u1 in Session.new - + # commit changes Session.commit() @@ -621,41 +621,41 @@ The contextual session may be disposed of by calling ``Session.remove()``:: # remove current contextual session Session.remove() -After ``remove()`` is called, the next operation with the contextual session will start a new ``Session`` for the current thread. +After ``remove()`` is called, the next operation with the contextual session will start a new :class:`~sqlalchemy.orm.session.Session` for the current thread. .. _session_lifespan: -Lifespan of a Contextual Session +Lifespan of a Contextual Session -------------------------------- A (really, really) common question is when does the contextual session get created, when does it get disposed ? We'll consider a typical lifespan as used in a web application:: Web Server Web Framework User-defined Controller Call -------------- -------------- ------------------------------ - web request -> + web request -> call controller -> # call Session(). this establishes a new, # contextual Session. session = Session() - + # load some objects, save some changes objects = session.query(MyClass).all() - - # some other code calls Session, it's the + + # some other code calls Session, it's the # same contextual session as "sess" session2 = Session() session2.add(foo) session2.commit() - + # generate content to be returned return generate_content() Session.remove() <- - web response <- + web response <- The above example illustrates an explicit call to ``Session.remove()``. This has the effect such that each web request starts fresh with a brand new session. When integrating with a web framework, there's actually many options on how to proceed for this step: -* Session.remove() - this is the most cut and dry approach; the ``Session`` is thrown away, all of its transactional/connection resources are closed out, everything within it is explicitly gone. A new ``Session`` will be used on the next request. -* Session.close() - Similar to calling ``remove()``, in that all objects are explicitly expunged and all transactional/connection resources closed, except the actual ``Session`` object hangs around. It doesn't make too much difference here unless the start of the web request would like to pass specific options to the initial construction of ``Session()``, such as a specific ``Engine`` to bind to. -* Session.commit() - In this case, the behavior is that any remaining changes pending are flushed, and the transaction is committed. The full state of the session is expired, so that when the next web request is started, all data will be reloaded. In reality, the contents of the ``Session`` are weakly referenced anyway so its likely that it will be empty on the next request in any case. +* Session.remove() - this is the most cut and dry approach; the :class:`~sqlalchemy.orm.session.Session` is thrown away, all of its transactional/connection resources are closed out, everything within it is explicitly gone. A new :class:`~sqlalchemy.orm.session.Session` will be used on the next request. +* Session.close() - Similar to calling ``remove()``, in that all objects are explicitly expunged and all transactional/connection resources closed, except the actual :class:`~sqlalchemy.orm.session.Session` object hangs around. It doesn't make too much difference here unless the start of the web request would like to pass specific options to the initial construction of ``Session()``, such as a specific :class:`~sqlalchemy.engine.base.Engine` to bind to. +* Session.commit() - In this case, the behavior is that any remaining changes pending are flushed, and the transaction is committed. The full state of the session is expired, so that when the next web request is started, all data will be reloaded. In reality, the contents of the :class:`~sqlalchemy.orm.session.Session` are weakly referenced anyway so its likely that it will be empty on the next request in any case. * Session.rollback() - Similar to calling commit, except we assume that the user would have called commit explicitly if that was desired; the ``rollback()`` ensures that no transactional state remains and expires all data, in the case that the request was aborted and did not roll back itself. * do nothing - this is a valid option as well. The controller code is responsible for doing one of the above steps at the end of the request. @@ -698,11 +698,11 @@ Basic usage is similar to :class:`~sqlalchemy.orm.interfaces.MapperExtension`:: class MySessionExtension(SessionExtension): def before_commit(self, session): print "before commit!" - + Session = sessionmaker(extension=MySessionExtension()) - + or with :func:`~sqlalchemy.orm.create_session()`:: session = create_session(extension=MySessionExtension()) - -The same ``SessionExtension`` instance can be used with any number of sessions. + +The same :class:`~sqlalchemy.orm.interfaces.SessionExtension` instance can be used with any number of sessions. diff --git a/doc/build/sqlexpression.rst b/doc/build/sqlexpression.rst index 613f4551de..b1b0ba575e 100644 --- a/doc/build/sqlexpression.rst +++ b/doc/build/sqlexpression.rst @@ -17,7 +17,7 @@ A quick check to verify that we are on at least **version 0.6** of SQLAlchemy: >>> import sqlalchemy >>> sqlalchemy.__version__ # doctest:+SKIP 0.6.0 - + Connecting ========== @@ -28,16 +28,16 @@ For this tutorial we will use an in-memory-only SQLite database. This is an ea >>> from sqlalchemy import create_engine >>> engine = create_engine('sqlite:///:memory:', echo=True) - + The ``echo`` flag is a shortcut to setting up SQLAlchemy logging, which is accomplished via Python's standard ``logging`` module. With it enabled, we'll see all the generated SQL produced. If you are working through this tutorial and want less output generated, set it to ``False``. This tutorial will format the SQL behind a popup window so it doesn't get in our way; just click the "SQL" links to see what's being generated. - -Define and Create Tables + +Define and Create Tables ========================= -The SQL Expression Language constructs its expressions in most cases against table columns. In SQLAlchemy, a column is most often represented by an object called ``Column``, and in all cases a ``Column`` is associated with a ``Table``. A collection of ``Table`` objects and their associated child objects is referred to as **database metadata**. In this tutorial we will explicitly lay out several ``Table`` objects, but note that SA can also "import" whole sets of ``Table`` objects automatically from an existing database (this process is called **table reflection**). +The SQL Expression Language constructs its expressions in most cases against table columns. In SQLAlchemy, a column is most often represented by an object called :class:`~sqlalchemy.schema.Column`, and in all cases a :class:`~sqlalchemy.schema.Column` is associated with a :class:`~sqlalchemy.schema.Table`. A collection of :class:`~sqlalchemy.schema.Table` objects and their associated child objects is referred to as **database metadata**. In this tutorial we will explicitly lay out several :class:`~sqlalchemy.schema.Table` objects, but note that SA can also "import" whole sets of :class:`~sqlalchemy.schema.Table` objects automatically from an existing database (this process is called **table reflection**). -We define our tables all within a catalog called ``MetaData``, using the ``Table`` construct, which resembles regular SQL CREATE TABLE statements. We'll make two tables, one of which represents "users" in an application, and another which represents zero or more "email addreses" for each row in the "users" table: +We define our tables all within a catalog called :class:`~sqlalchemy.schema.MetaData`, using the :class:`~sqlalchemy.schema.Table` construct, which resembles regular SQL CREATE TABLE statements. We'll make two tables, one of which represents "users" in an application, and another which represents zero or more "email addreses" for each row in the "users" table: .. sourcecode:: pycon+sql @@ -49,15 +49,15 @@ We define our tables all within a catalog called ``MetaData``, using the ``Table ... Column('fullname', String), ... ) - >>> addresses = Table('addresses', metadata, + >>> addresses = Table('addresses', metadata, ... Column('id', Integer, primary_key=True), ... Column('user_id', None, ForeignKey('users.id')), ... Column('email_address', String, nullable=False) ... ) -All about how to define ``Table`` objects, as well as how to create them from an existing database automatically, is described in :ref:`metadata_toplevel`. +All about how to define :class:`~sqlalchemy.schema.Table` objects, as well as how to create them from an existing database automatically, is described in :ref:`metadata_toplevel`. -Next, to tell the ``MetaData`` we'd actually like to create our selection of tables for real inside the SQLite database, we use ``create_all()``, passing it the ``engine`` instance which points to our database. This will check for the presence of each table first before creating, so it's safe to call multiple times: +Next, to tell the :class:`~sqlalchemy.schema.MetaData` we'd actually like to create our selection of tables for real inside the SQLite database, we use ``create_all()``, passing it the ``engine`` instance which points to our database. This will check for the presence of each table first before creating, so it's safe to call multiple times: .. sourcecode:: pycon+sql @@ -67,18 +67,18 @@ Next, to tell the ``MetaData`` we'd actually like to create our selection of tab PRAGMA table_info("addresses") () CREATE TABLE users ( - id INTEGER NOT NULL, - name VARCHAR, - fullname VARCHAR, + id INTEGER NOT NULL, + name VARCHAR, + fullname VARCHAR, PRIMARY KEY (id) ) () COMMIT CREATE TABLE addresses ( - id INTEGER NOT NULL, - user_id INTEGER, - email_address VARCHAR NOT NULL, - PRIMARY KEY (id), + id INTEGER NOT NULL, + user_id INTEGER, + email_address VARCHAR NOT NULL, + PRIMARY KEY (id), FOREIGN KEY(user_id) REFERENCES users (id) ) () @@ -99,12 +99,12 @@ Next, to tell the ``MetaData`` we'd actually like to create our selection of tab Additionally, Firebird and Oracle require sequences to generate new primary key identifiers, and SQLAlchemy doesn't generate or assume these - without being instructed. For that, you use the ``Sequence`` construct:: + without being instructed. For that, you use the :class:`~sqlalchemy.schema.Sequence` construct:: from sqlalchemy import Sequence Column('id', Integer, Sequence('user_id_seq'), primary_key=True) - A full, foolproof ``Table`` is therefore:: + A full, foolproof :class:`~sqlalchemy.schema.Table` is therefore:: users_table = Table('users', metadata, Column('id', Integer, Sequence('user_id_seq'), primary_key=True), @@ -116,7 +116,7 @@ Next, to tell the ``MetaData`` we'd actually like to create our selection of tab Insert Expressions ================== -The first SQL expression we'll create is the ``Insert`` construct, which represents an INSERT statement. This is typically created relative to its target table:: +The first SQL expression we'll create is the :class:`~sqlalchemy.sql.expression.Insert` construct, which represents an INSERT statement. This is typically created relative to its target table:: >>> ins = users.insert() @@ -124,28 +124,28 @@ To see a sample of the SQL this construct produces, use the ``str()`` function:: >>> str(ins) 'INSERT INTO users (id, name, fullname) VALUES (:id, :name, :fullname)' - + Notice above that the INSERT statement names every column in the ``users`` table. This can be limited by using the ``values()`` method, which establishes the VALUES clause of the INSERT explicitly:: >>> ins = users.insert().values(name='jack', fullname='Jack Jones') >>> str(ins) 'INSERT INTO users (name, fullname) VALUES (:name, :fullname)' - -Above, while the ``values`` method limited the VALUES clause to just two columns, the actual data we placed in ``values`` didn't get rendered into the string; instead we got named bind parameters. As it turns out, our data *is* stored within our ``Insert`` construct, but it typically only comes out when the statement is actually executed; since the data consists of literal values, SQLAlchemy automatically generates bind parameters for them. We can peek at this data for now by looking at the compiled form of the statement:: + +Above, while the ``values`` method limited the VALUES clause to just two columns, the actual data we placed in ``values`` didn't get rendered into the string; instead we got named bind parameters. As it turns out, our data *is* stored within our :class:`~sqlalchemy.sql.expression.Insert` construct, but it typically only comes out when the statement is actually executed; since the data consists of literal values, SQLAlchemy automatically generates bind parameters for them. We can peek at this data for now by looking at the compiled form of the statement:: >>> ins.compile().params #doctest: +NORMALIZE_WHITESPACE - {'fullname': 'Jack Jones', 'name': 'jack'} + {'fullname': 'Jack Jones', 'name': 'jack'} -Executing +Executing ========== -The interesting part of an ``Insert`` is executing it. In this tutorial, we will generally focus on the most explicit method of executing a SQL construct, and later touch upon some "shortcut" ways to do it. The ``engine`` object we created is a repository for database connections capable of issuing SQL to the database. To acquire a connection, we use the ``connect()`` method:: +The interesting part of an :class:`~sqlalchemy.sql.expression.Insert` is executing it. In this tutorial, we will generally focus on the most explicit method of executing a SQL construct, and later touch upon some "shortcut" ways to do it. The ``engine`` object we created is a repository for database connections capable of issuing SQL to the database. To acquire a connection, we use the ``connect()`` method:: >>> conn = engine.connect() >>> conn #doctest: +ELLIPSIS -The ``Connection`` object represents an actively checked out DBAPI connection resource. Lets feed it our ``Insert`` object and see what happens: +The :class:`~sqlalchemy.engine.base.Connection` object represents an actively checked out DBAPI connection resource. Lets feed it our :class:`~sqlalchemy.sql.expression.Insert` object and see what happens: .. sourcecode:: pycon+sql @@ -154,7 +154,7 @@ The ``Connection`` object represents an actively checked out DBAPI connection re ['jack', 'Jack Jones'] COMMIT -So the INSERT statement was now issued to the database. Although we got positional "qmark" bind parameters instead of "named" bind parameters in the output. How come ? Because when executed, the ``Connection`` used the SQLite **dialect** to help generate the statement; when we use the ``str()`` function, the statement isn't aware of this dialect, and falls back onto a default which uses named parameters. We can view this manually as follows: +So the INSERT statement was now issued to the database. Although we got positional "qmark" bind parameters instead of "named" bind parameters in the output. How come ? Because when executed, the :class:`~sqlalchemy.engine.base.Connection` used the SQLite **dialect** to help generate the statement; when we use the ``str()`` function, the statement isn't aware of this dialect, and falls back onto a default which uses named parameters. We can view this manually as follows: .. sourcecode:: pycon+sql @@ -162,20 +162,20 @@ So the INSERT statement was now issued to the database. Although we got positio >>> str(ins) 'INSERT INTO users (name, fullname) VALUES (?, ?)' -What about the ``result`` variable we got when we called ``execute()`` ? As the SQLAlchemy ``Connection`` object references a DBAPI connection, the result, known as a ``ResultProxy`` object, is analogous to the DBAPI cursor object. In the case of an INSERT, we can get important information from it, such as the primary key values which were generated from our statement: +What about the ``result`` variable we got when we called ``execute()`` ? As the SQLAlchemy :class:`~sqlalchemy.engine.base.Connection` object references a DBAPI connection, the result, known as a :class:`~sqlalchemy.engine.base.ResultProxy` object, is analogous to the DBAPI cursor object. In the case of an INSERT, we can get important information from it, such as the primary key values which were generated from our statement: .. sourcecode:: pycon+sql >>> result.inserted_primary_key [1] - -The value of ``1`` was automatically generated by SQLite, but only because we did not specify the ``id`` column in our ``Insert`` statement; otherwise, our explicit value would have been used. In either case, SQLAlchemy always knows how to get at a newly generated primary key value, even though the method of generating them is different across different databases; each databases' ``Dialect`` knows the specific steps needed to determine the correct value (or values; note that ``inserted_primary_key`` returns a list so that it supports composite primary keys). -Executing Multiple Statements +The value of ``1`` was automatically generated by SQLite, but only because we did not specify the ``id`` column in our :class:`~sqlalchemy.sql.expression.Insert` statement; otherwise, our explicit value would have been used. In either case, SQLAlchemy always knows how to get at a newly generated primary key value, even though the method of generating them is different across different databases; each databases' :class:`~sqlalchemy.engine.base.Dialect` knows the specific steps needed to determine the correct value (or values; note that ``inserted_primary_key`` returns a list so that it supports composite primary keys). + +Executing Multiple Statements ============================== -Our insert example above was intentionally a little drawn out to show some various behaviors of expression language constructs. In the usual case, an ``Insert`` statement is usually compiled against the parameters sent to the ``execute()`` method on ``Connection``, so that there's no need to use the ``values`` keyword with ``Insert``. Lets create a generic ``Insert`` statement again and use it in the "normal" way: +Our insert example above was intentionally a little drawn out to show some various behaviors of expression language constructs. In the usual case, an :class:`~sqlalchemy.sql.expression.Insert` statement is usually compiled against the parameters sent to the ``execute()`` method on :class:`~sqlalchemy.engine.base.Connection`, so that there's no need to use the ``values`` keyword with :class:`~sqlalchemy.sql.expression.Insert`. Lets create a generic :class:`~sqlalchemy.sql.expression.Insert` statement again and use it in the "normal" way: .. sourcecode:: pycon+sql @@ -186,7 +186,7 @@ Our insert example above was intentionally a little drawn out to show some vario COMMIT {stop} -Above, because we specified all three columns in the the ``execute()`` method, the compiled ``Insert`` included all three columns. The ``Insert`` statement is compiled at execution time based on the parameters we specified; if we specified fewer parameters, the ``Insert`` would have fewer entries in its VALUES clause. +Above, because we specified all three columns in the the ``execute()`` method, the compiled :class:`~sqlalchemy.sql.expression.Insert` included all three columns. The :class:`~sqlalchemy.sql.expression.Insert` statement is compiled at execution time based on the parameters we specified; if we specified fewer parameters, the :class:`~sqlalchemy.sql.expression.Insert` would have fewer entries in its VALUES clause. To issue many inserts using DBAPI's ``executemany()`` method, we can send in a list of dictionaries each containing a distinct set of parameters to be inserted, as we do here to add some email addresses: @@ -205,13 +205,13 @@ To issue many inserts using DBAPI's ``executemany()`` method, we can send in a l Above, we again relied upon SQLite's automatic generation of primary key identifiers for each ``addresses`` row. -When executing multiple sets of parameters, each dictionary must have the **same** set of keys; i.e. you cant have fewer keys in some dictionaries than others. This is because the ``Insert`` statement is compiled against the **first** dictionary in the list, and it's assumed that all subsequent argument dictionaries are compatible with that statement. +When executing multiple sets of parameters, each dictionary must have the **same** set of keys; i.e. you cant have fewer keys in some dictionaries than others. This is because the :class:`~sqlalchemy.sql.expression.Insert` statement is compiled against the **first** dictionary in the list, and it's assumed that all subsequent argument dictionaries are compatible with that statement. -Connectionless / Implicit Execution +Connectionless / Implicit Execution ==================================== -We're executing our ``Insert`` using a ``Connection``. There's two options that allow you to not have to deal with the connection part. You can execute in the **connectionless** style, using the engine, which opens and closes a connection for you: +We're executing our :class:`~sqlalchemy.sql.expression.Insert` using a :class:`~sqlalchemy.engine.base.Connection`. There's two options that allow you to not have to deal with the connection part. You can execute in the **connectionless** style, using the engine, which opens and closes a connection for you: .. sourcecode:: pycon+sql @@ -219,8 +219,8 @@ We're executing our ``Insert`` using a ``Connection``. There's two options that INSERT INTO users (name, fullname) VALUES (?, ?) ['fred', 'Fred Flintstone'] COMMIT - -and you can save even more steps than that, if you connect the ``Engine`` to the ``MetaData`` object we created earlier. When this is done, all SQL expressions which involve tables within the ``MetaData`` object will be automatically **bound** to the ``Engine``. In this case, we call it **implicit execution**: + +and you can save even more steps than that, if you connect the :class:`~sqlalchemy.engine.base.Engine` to the :class:`~sqlalchemy.schema.MetaData` object we created earlier. When this is done, all SQL expressions which involve tables within the :class:`~sqlalchemy.schema.MetaData` object will be automatically **bound** to the :class:`~sqlalchemy.engine.base.Engine`. In this case, we call it **implicit execution**: .. sourcecode:: pycon+sql @@ -230,15 +230,15 @@ and you can save even more steps than that, if you connect the ``Engine`` to the ['mary', 'Mary Contrary'] COMMIT -When the ``MetaData`` is bound, statements will also compile against the engine's dialect. Since a lot of the examples here assume the default dialect, we'll detach the engine from the metadata which we just attached: +When the :class:`~sqlalchemy.schema.MetaData` is bound, statements will also compile against the engine's dialect. Since a lot of the examples here assume the default dialect, we'll detach the engine from the metadata which we just attached: .. sourcecode:: pycon+sql >>> metadata.bind = None Detailed examples of connectionless and implicit execution are available in the "Engines" chapter: :ref:`dbengine_implicit`. - -Selecting + +Selecting ========== @@ -249,11 +249,11 @@ We began with inserts just so that our test database had some data in it. The m >>> from sqlalchemy.sql import select >>> s = select([users]) >>> result = conn.execute(s) - {opensql}SELECT users.id, users.name, users.fullname + {opensql}SELECT users.id, users.name, users.fullname FROM users [] -Above, we issued a basic ``select()`` call, placing the ``users`` table within the COLUMNS clause of the select, and then executing. SQLAlchemy expanded the ``users`` table into the set of each of its columns, and also generated a FROM clause for us. The result returned is again a ``ResultProxy`` object, which acts much like a DBAPI cursor, including methods such as ``fetchone()`` and ``fetchall()``. The easiest way to get rows from it is to just iterate: +Above, we issued a basic ``select()`` call, placing the ``users`` table within the COLUMNS clause of the select, and then executing. SQLAlchemy expanded the ``users`` table into the set of each of its columns, and also generated a FROM clause for us. The result returned is again a :class:`~sqlalchemy.engine.base.ResultProxy` object, which acts much like a DBAPI cursor, including methods such as ``fetchone()`` and ``fetchall()``. The easiest way to get rows from it is to just iterate: .. sourcecode:: pycon+sql @@ -269,10 +269,10 @@ Above, we see that printing each row produces a simple tuple-like result. We ha .. sourcecode:: pycon+sql {sql}>>> result = conn.execute(s) - SELECT users.id, users.name, users.fullname + SELECT users.id, users.name, users.fullname FROM users [] - + {stop}>>> row = result.fetchone() >>> print "name:", row['name'], "; fullname:", row['fullname'] name: jack ; fullname: Jack Jones @@ -285,13 +285,13 @@ Integer indexes work as well: >>> print "name:", row[1], "; fullname:", row[2] name: wendy ; fullname: Wendy Williams -But another way, whose usefulness will become apparent later on, is to use the ``Column`` objects directly as keys: +But another way, whose usefulness will become apparent later on, is to use the :class:`~sqlalchemy.schema.Column` objects directly as keys: .. sourcecode:: pycon+sql {sql}>>> for row in conn.execute(s): ... print "name:", row[users.c.name], "; fullname:", row[users.c.fullname] - SELECT users.id, users.name, users.fullname + SELECT users.id, users.name, users.fullname FROM users [] {stop}name: jack ; fullname: Jack Jones @@ -299,19 +299,19 @@ But another way, whose usefulness will become apparent later on, is to use the ` name: fred ; fullname: Fred Flintstone name: mary ; fullname: Mary Contrary -Result sets which have pending rows remaining should be explicitly closed before discarding. While the resources referenced by the ``ResultProxy`` will be closed when the object is garbage collected, it's better to make it explicit as some database APIs are very picky about such things: +Result sets which have pending rows remaining should be explicitly closed before discarding. While the resources referenced by the :class:`~sqlalchemy.engine.base.ResultProxy` will be closed when the object is garbage collected, it's better to make it explicit as some database APIs are very picky about such things: .. sourcecode:: pycon+sql >>> result.close() -If we'd like to more carefully control the columns which are placed in the COLUMNS clause of the select, we reference individual ``Column`` objects from our ``Table``. These are available as named attributes off the ``c`` attribute of the ``Table`` object: +If we'd like to more carefully control the columns which are placed in the COLUMNS clause of the select, we reference individual :class:`~sqlalchemy.schema.Column` objects from our :class:`~sqlalchemy.schema.Table`. These are available as named attributes off the ``c`` attribute of the :class:`~sqlalchemy.schema.Table` object: .. sourcecode:: pycon+sql >>> s = select([users.c.name, users.c.fullname]) {sql}>>> result = conn.execute(s) - SELECT users.name, users.fullname + SELECT users.name, users.fullname FROM users [] {stop}>>> for row in result: #doctest: +NORMALIZE_WHITESPACE @@ -320,14 +320,14 @@ If we'd like to more carefully control the columns which are placed in the COLUM (u'wendy', u'Wendy Williams') (u'fred', u'Fred Flintstone') (u'mary', u'Mary Contrary') - + Lets observe something interesting about the FROM clause. Whereas the generated statement contains two distinct sections, a "SELECT columns" part and a "FROM table" part, our ``select()`` construct only has a list containing columns. How does this work ? Let's try putting *two* tables into our ``select()`` statement: .. sourcecode:: pycon+sql {sql}>>> for row in conn.execute(select([users, addresses])): ... print row - SELECT users.id, users.name, users.fullname, addresses.id, addresses.user_id, addresses.email_address + SELECT users.id, users.name, users.fullname, addresses.id, addresses.user_id, addresses.email_address FROM users, addresses [] {stop}(1, u'jack', u'Jack Jones', 1, 1, u'jack@yahoo.com') @@ -354,8 +354,8 @@ It placed **both** tables into the FROM clause. But also, it made a real mess. >>> s = select([users, addresses], users.c.id==addresses.c.user_id) {sql}>>> for row in conn.execute(s): ... print row - SELECT users.id, users.name, users.fullname, addresses.id, addresses.user_id, addresses.email_address - FROM users, addresses + SELECT users.id, users.name, users.fullname, addresses.id, addresses.user_id, addresses.email_address + FROM users, addresses WHERE users.id = addresses.user_id [] {stop}(1, u'jack', u'Jack Jones', 1, 1, u'jack@yahoo.com') @@ -363,7 +363,7 @@ It placed **both** tables into the FROM clause. But also, it made a real mess. (2, u'wendy', u'Wendy Williams', 3, 2, u'www@www.org') (2, u'wendy', u'Wendy Williams', 4, 2, u'wendy@aol.com') -So that looks a lot better, we added an expression to our ``select()`` which had the effect of adding ``WHERE users.id = addresses.user_id`` to our statement, and our results were managed down so that the join of ``users`` and ``addresses`` rows made sense. But let's look at that expression? It's using just a Python equality operator between two different ``Column`` objects. It should be clear that something is up. Saying ``1==1`` produces ``True``, and ``1==2`` produces ``False``, not a WHERE clause. So lets see exactly what that expression is doing: +So that looks a lot better, we added an expression to our ``select()`` which had the effect of adding ``WHERE users.id = addresses.user_id`` to our statement, and our results were managed down so that the join of ``users`` and ``addresses`` rows made sense. But let's look at that expression? It's using just a Python equality operator between two different :class:`~sqlalchemy.schema.Column` objects. It should be clear that something is up. Saying ``1==1`` produces ``True``, and ``1==2`` produces ``False``, not a WHERE clause. So lets see exactly what that expression is doing: .. sourcecode:: pycon+sql @@ -377,9 +377,9 @@ Wow, surprise ! This is neither a ``True`` nor a ``False``. Well what is it ? >>> str(users.c.id==addresses.c.user_id) 'users.id = addresses.user_id' -As you can see, the ``==`` operator is producing an object that is very much like the ``Insert`` and ``select()`` objects we've made so far, thanks to Python's ``__eq__()`` builtin; you call ``str()`` on it and it produces SQL. By now, one can see that everything we are working with is ultimately the same type of object. SQLAlchemy terms the base class of all of these expressions as ``sqlalchemy.sql.ClauseElement``. +As you can see, the ``==`` operator is producing an object that is very much like the :class:`~sqlalchemy.sql.expression.Insert` and ``select()`` objects we've made so far, thanks to Python's ``__eq__()`` builtin; you call ``str()`` on it and it produces SQL. By now, one can see that everything we are working with is ultimately the same type of object. SQLAlchemy terms the base class of all of these expressions as ``sqlalchemy.sql.ClauseElement``. -Operators +Operators ========== @@ -389,44 +389,44 @@ Since we've stumbled upon SQLAlchemy's operator paradigm, let's go through some >>> print users.c.id==addresses.c.user_id users.id = addresses.user_id - + If we use a literal value (a literal meaning, not a SQLAlchemy clause object), we get a bind parameter: .. sourcecode:: pycon+sql >>> print users.c.id==7 users.id = :id_1 - -The ``7`` literal is embedded in ``ClauseElement``; we can use the same trick we did with the ``Insert`` object to see it: + +The ``7`` literal is embedded in :class:`~sqlalchemy.sql.expression.ClauseElement`; we can use the same trick we did with the :class:`~sqlalchemy.sql.expression.Insert` object to see it: .. sourcecode:: pycon+sql >>> (users.c.id==7).compile().params {u'id_1': 7} - + Most Python operators, as it turns out, produce a SQL expression here, like equals, not equals, etc.: .. sourcecode:: pycon+sql >>> print users.c.id != 7 users.id != :id_1 - + >>> # None converts to IS NULL >>> print users.c.name == None users.name IS NULL - - >>> # reverse works too + + >>> # reverse works too >>> print 'fred' > users.c.name users.name < :name_1 - + If we add two integer columns together, we get an addition expression: .. sourcecode:: pycon+sql >>> print users.c.id + addresses.c.id users.id + addresses.id - -Interestingly, the type of the ``Column`` is important ! If we use ``+`` with two string based columns (recall we put types like ``Integer`` and ``String`` on our ``Column`` objects at the beginning), we get something different: + +Interestingly, the type of the :class:`~sqlalchemy.schema.Column` is important ! If we use ``+`` with two string based columns (recall we put types like ``Integer`` and ``String`` on our :class:`~sqlalchemy.schema.Column` objects at the beginning), we get something different: .. sourcecode:: pycon+sql @@ -440,7 +440,7 @@ Where ``||`` is the string concatenation operator used on most databases. But n >>> print (users.c.name + users.c.fullname).compile(bind=create_engine('mysql://')) concat(users.name, users.fullname) -The above illustrates the SQL that's generated for an ``Engine`` that's connected to a MySQL database; the ``||`` operator now compiles as MySQL's ``concat()`` function. +The above illustrates the SQL that's generated for an :class:`~sqlalchemy.engine.base.Engine` that's connected to a MySQL database; the ``||`` operator now compiles as MySQL's ``concat()`` function. If you have come across an operator which really isn't available, you can always use the ``op()`` method; this generates whatever operator you need: @@ -448,14 +448,14 @@ If you have come across an operator which really isn't available, you can always >>> print users.c.name.op('tiddlywinks')('foo') users.name tiddlywinks :name_1 - + This function can also be used to make bitwise operators explicit. For example:: somecolumn.op('&')(0xff) is a bitwise AND of the value in `somecolumn`. - -Conjunctions + +Conjunctions ============= @@ -464,11 +464,11 @@ We'd like to show off some of our operators inside of ``select()`` constructs. .. sourcecode:: pycon+sql >>> from sqlalchemy.sql import and_, or_, not_ - >>> print and_(users.c.name.like('j%'), users.c.id==addresses.c.user_id, #doctest: +NORMALIZE_WHITESPACE + >>> print and_(users.c.name.like('j%'), users.c.id==addresses.c.user_id, #doctest: +NORMALIZE_WHITESPACE ... or_(addresses.c.email_address=='wendy@aol.com', addresses.c.email_address=='jack@yahoo.com'), ... not_(users.c.id>5)) - users.name LIKE :name_1 AND users.id = addresses.user_id AND - (addresses.email_address = :email_address_1 OR addresses.email_address = :email_address_2) + users.name LIKE :name_1 AND users.id = addresses.user_id AND + (addresses.email_address = :email_address_1 OR addresses.email_address = :email_address_2) AND users.id <= :id_1 And you can also use the re-jiggered bitwise AND, OR and NOT operators, although because of Python operator precedence you have to watch your parenthesis: @@ -478,37 +478,37 @@ And you can also use the re-jiggered bitwise AND, OR and NOT operators, although >>> print users.c.name.like('j%') & (users.c.id==addresses.c.user_id) & \ ... ((addresses.c.email_address=='wendy@aol.com') | (addresses.c.email_address=='jack@yahoo.com')) \ ... & ~(users.c.id>5) # doctest: +NORMALIZE_WHITESPACE - users.name LIKE :name_1 AND users.id = addresses.user_id AND - (addresses.email_address = :email_address_1 OR addresses.email_address = :email_address_2) + users.name LIKE :name_1 AND users.id = addresses.user_id AND + (addresses.email_address = :email_address_1 OR addresses.email_address = :email_address_2) AND users.id <= :id_1 So with all of this vocabulary, let's select all users who have an email address at AOL or MSN, whose name starts with a letter between "m" and "z", and we'll also generate a column containing their full name combined with their email address. We will add two new constructs to this statement, ``between()`` and ``label()``. ``between()`` produces a BETWEEN clause, and ``label()`` is used in a column expression to produce labels using the ``AS`` keyword; it's recommended when selecting from expressions that otherwise would not have a name: .. sourcecode:: pycon+sql - >>> s = select([(users.c.fullname + ", " + addresses.c.email_address).label('title')], - ... and_( - ... users.c.id==addresses.c.user_id, - ... users.c.name.between('m', 'z'), + >>> s = select([(users.c.fullname + ", " + addresses.c.email_address).label('title')], + ... and_( + ... users.c.id==addresses.c.user_id, + ... users.c.name.between('m', 'z'), ... or_( - ... addresses.c.email_address.like('%@aol.com'), + ... addresses.c.email_address.like('%@aol.com'), ... addresses.c.email_address.like('%@msn.com') ... ) ... ) - ... ) + ... ) >>> print conn.execute(s).fetchall() #doctest: +NORMALIZE_WHITESPACE - SELECT users.fullname || ? || addresses.email_address AS title - FROM users, addresses - WHERE users.id = addresses.user_id AND users.name BETWEEN ? AND ? AND + SELECT users.fullname || ? || addresses.email_address AS title + FROM users, addresses + WHERE users.id = addresses.user_id AND users.name BETWEEN ? AND ? AND (addresses.email_address LIKE ? OR addresses.email_address LIKE ?) [', ', 'm', 'z', '%@aol.com', '%@msn.com'] [(u'Wendy Williams, wendy@aol.com',)] -Once again, SQLAlchemy figured out the FROM clause for our statement. In fact it will determine the FROM clause based on all of its other bits; the columns clause, the where clause, and also some other elements which we haven't covered yet, which include ORDER BY, GROUP BY, and HAVING. +Once again, SQLAlchemy figured out the FROM clause for our statement. In fact it will determine the FROM clause based on all of its other bits; the columns clause, the where clause, and also some other elements which we haven't covered yet, which include ORDER BY, GROUP BY, and HAVING. .. _sqlexpression_text: -Using Text +Using Text =========== Our last example really became a handful to type. Going from what one understands to be a textual SQL expression into a Python construct which groups components together in a programmatic style can be hard. That's why SQLAlchemy lets you just use strings too. The ``text()`` construct represents any textual statement. To use bind parameters with ``text()``, always use the named colon format. Such as below, we create a ``text()`` and execute it, feeding in the bind parameters to the ``execute()`` method: @@ -516,41 +516,41 @@ Our last example really became a handful to type. Going from what one understan .. sourcecode:: pycon+sql >>> from sqlalchemy.sql import text - >>> s = text("""SELECT users.fullname || ', ' || addresses.email_address AS title - ... FROM users, addresses - ... WHERE users.id = addresses.user_id AND users.name BETWEEN :x AND :y AND + >>> s = text("""SELECT users.fullname || ', ' || addresses.email_address AS title + ... FROM users, addresses + ... WHERE users.id = addresses.user_id AND users.name BETWEEN :x AND :y AND ... (addresses.email_address LIKE :e1 OR addresses.email_address LIKE :e2) ... """) {sql}>>> print conn.execute(s, x='m', y='z', e1='%@aol.com', e2='%@msn.com').fetchall() # doctest:+NORMALIZE_WHITESPACE - SELECT users.fullname || ', ' || addresses.email_address AS title - FROM users, addresses - WHERE users.id = addresses.user_id AND users.name BETWEEN ? AND ? AND + SELECT users.fullname || ', ' || addresses.email_address AS title + FROM users, addresses + WHERE users.id = addresses.user_id AND users.name BETWEEN ? AND ? AND (addresses.email_address LIKE ? OR addresses.email_address LIKE ?) ['m', 'z', '%@aol.com', '%@msn.com'] {stop}[(u'Wendy Williams, wendy@aol.com',)] -To gain a "hybrid" approach, the `select()` construct accepts strings for most of its arguments. Below we combine the usage of strings with our constructed ``select()`` object, by using the ``select()`` object to structure the statement, and strings to provide all the content within the structure. For this example, SQLAlchemy is not given any ``Column`` or ``Table`` objects in any of its expressions, so it cannot generate a FROM clause. So we also give it the ``from_obj`` keyword argument, which is a list of ``ClauseElements`` (or strings) to be placed within the FROM clause: +To gain a "hybrid" approach, the `select()` construct accepts strings for most of its arguments. Below we combine the usage of strings with our constructed ``select()`` object, by using the ``select()`` object to structure the statement, and strings to provide all the content within the structure. For this example, SQLAlchemy is not given any :class:`~sqlalchemy.schema.Column` or :class:`~sqlalchemy.schema.Table` objects in any of its expressions, so it cannot generate a FROM clause. So we also give it the ``from_obj`` keyword argument, which is a list of ``ClauseElements`` (or strings) to be placed within the FROM clause: .. sourcecode:: pycon+sql - >>> s = select(["users.fullname || ', ' || addresses.email_address AS title"], - ... and_( - ... "users.id = addresses.user_id", + >>> s = select(["users.fullname || ', ' || addresses.email_address AS title"], + ... and_( + ... "users.id = addresses.user_id", ... "users.name BETWEEN 'm' AND 'z'", ... "(addresses.email_address LIKE :x OR addresses.email_address LIKE :y)" ... ), ... from_obj=['users', 'addresses'] ... ) {sql}>>> print conn.execute(s, x='%@aol.com', y='%@msn.com').fetchall() #doctest: +NORMALIZE_WHITESPACE - SELECT users.fullname || ', ' || addresses.email_address AS title - FROM users, addresses + SELECT users.fullname || ', ' || addresses.email_address AS title + FROM users, addresses WHERE users.id = addresses.user_id AND users.name BETWEEN 'm' AND 'z' AND (addresses.email_address LIKE ? OR addresses.email_address LIKE ?) ['%@aol.com', '%@msn.com'] {stop}[(u'Wendy Williams, wendy@aol.com',)] Going from constructed SQL to text, we lose some capabilities. We lose the capability for SQLAlchemy to compile our expression to a specific target database; above, our expression won't work with MySQL since it has no ``||`` construct. It also becomes more tedious for SQLAlchemy to be made aware of the datatypes in use; for example, if our bind parameters required UTF-8 encoding before going in, or conversion from a Python ``datetime`` into a string (as is required with SQLite), we would have to add extra information to our ``text()`` construct. Similar issues arise on the result set side, where SQLAlchemy also performs type-specific data conversion in some cases; still more information can be added to ``text()`` to work around this. But what we really lose from our statement is the ability to manipulate it, transform it, and analyze it. These features are critical when using the ORM, which makes heavy usage of relational transformations. To show off what we mean, we'll first introduce the ALIAS construct and the JOIN construct, just so we have some juicier bits to play with. -Using Aliases +Using Aliases ============== The alias corresponds to a "renamed" version of a table or arbitrary relation, which occurs anytime you say "SELECT .. FROM sometable AS someothername". The ``AS`` creates a new name for the table. Aliases are super important in SQL as they allow you to reference the same table more than once. Scenarios where you need to do this include when you self-join a table to itself, or more commonly when you need to join from a parent table to a child table multiple times. For example, we know that our user ``jack`` has two email addresses. How can we locate jack based on the combination of those two addresses? We need to join twice to it. Let's construct two distinct aliases for the ``addresses`` table and join: @@ -560,14 +560,14 @@ The alias corresponds to a "renamed" version of a table or arbitrary relation, w >>> a1 = addresses.alias('a1') >>> a2 = addresses.alias('a2') >>> s = select([users], and_( - ... users.c.id==a1.c.user_id, - ... users.c.id==a2.c.user_id, - ... a1.c.email_address=='jack@msn.com', + ... users.c.id==a1.c.user_id, + ... users.c.id==a2.c.user_id, + ... a1.c.email_address=='jack@msn.com', ... a2.c.email_address=='jack@yahoo.com' ... )) {sql}>>> print conn.execute(s).fetchall() - SELECT users.id, users.name, users.fullname - FROM users, addresses AS a1, addresses AS a2 + SELECT users.id, users.name, users.fullname + FROM users, addresses AS a1, addresses AS a2 WHERE users.id = a1.user_id AND users.id = a2.user_id AND a1.email_address = ? AND a2.email_address = ? ['jack@msn.com', 'jack@yahoo.com'] {stop}[(1, u'jack', u'Jack Jones')] @@ -579,14 +579,14 @@ Easy enough. One thing that we're going for with the SQL Expression Language is >>> a1 = addresses.alias() >>> a2 = addresses.alias() >>> s = select([users], and_( - ... users.c.id==a1.c.user_id, - ... users.c.id==a2.c.user_id, - ... a1.c.email_address=='jack@msn.com', + ... users.c.id==a1.c.user_id, + ... users.c.id==a2.c.user_id, + ... a1.c.email_address=='jack@msn.com', ... a2.c.email_address=='jack@yahoo.com' ... )) {sql}>>> print conn.execute(s).fetchall() - SELECT users.id, users.name, users.fullname - FROM users, addresses AS addresses_1, addresses AS addresses_2 + SELECT users.id, users.name, users.fullname + FROM users, addresses AS addresses_1, addresses AS addresses_2 WHERE users.id = addresses_1.user_id AND users.id = addresses_2.user_id AND addresses_1.email_address = ? AND addresses_2.email_address = ? ['jack@msn.com', 'jack@yahoo.com'] {stop}[(1, u'jack', u'Jack Jones')] @@ -600,15 +600,15 @@ Aliases can of course be used for anything which you can SELECT from, including >>> a1 = s.correlate(None).alias() >>> s = select([users.c.name], users.c.id==a1.c.id) {sql}>>> print conn.execute(s).fetchall() - SELECT users.name - FROM users, (SELECT users.id AS id, users.name AS name, users.fullname AS fullname - FROM users, addresses AS addresses_1, addresses AS addresses_2 - WHERE users.id = addresses_1.user_id AND users.id = addresses_2.user_id AND addresses_1.email_address = ? AND addresses_2.email_address = ?) AS anon_1 + SELECT users.name + FROM users, (SELECT users.id AS id, users.name AS name, users.fullname AS fullname + FROM users, addresses AS addresses_1, addresses AS addresses_2 + WHERE users.id = addresses_1.user_id AND users.id = addresses_2.user_id AND addresses_1.email_address = ? AND addresses_2.email_address = ?) AS anon_1 WHERE users.id = anon_1.id ['jack@msn.com', 'jack@yahoo.com'] {stop}[(u'jack',)] - -Using Joins + +Using Joins ============ @@ -618,8 +618,8 @@ We're halfway along to being able to construct any SELECT expression. The next >>> print users.join(addresses) users JOIN addresses ON users.id = addresses.user_id - -The alert reader will see more surprises; SQLAlchemy figured out how to JOIN the two tables ! The ON condition of the join, as it's called, was automatically generated based on the ``ForeignKey`` object which we placed on the ``addresses`` table way at the beginning of this tutorial. Already the ``join()`` construct is looking like a much better way to join tables. + +The alert reader will see more surprises; SQLAlchemy figured out how to JOIN the two tables ! The ON condition of the join, as it's called, was automatically generated based on the :class:`~sqlalchemy.schema.ForeignKey` object which we placed on the ``addresses`` table way at the beginning of this tutorial. Already the ``join()`` construct is looking like a much better way to join tables. Of course you can join on whatever expression you want, such as if we want to join on all users who use the same name in their email address as their username: @@ -636,7 +636,7 @@ When we create a ``select()`` construct, SQLAlchemy looks around at the tables w ... users.join(addresses, addresses.c.email_address.like(users.c.name + '%')) ... ]) {sql}>>> print conn.execute(s).fetchall() - SELECT users.fullname + SELECT users.fullname FROM users JOIN addresses ON addresses.email_address LIKE users.name || ? ['%'] {stop}[(u'Jack Jones',), (u'Jack Jones',), (u'Wendy Williams',)] @@ -647,7 +647,7 @@ The ``outerjoin()`` function just creates ``LEFT OUTER JOIN`` constructs. It's >>> s = select([users.c.fullname], from_obj=[users.outerjoin(addresses)]) >>> print s - SELECT users.fullname + SELECT users.fullname FROM users LEFT OUTER JOIN addresses ON users.id = addresses.user_id That's the output ``outerjoin()`` produces, unless, of course, you're stuck in a gig using Oracle prior to version 9, and you've set up your engine (which would be using ``OracleDialect``) to use Oracle-specific SQL: @@ -656,13 +656,13 @@ That's the output ``outerjoin()`` produces, unless, of course, you're stuck in a >>> from sqlalchemy.dialects.oracle import dialect as OracleDialect >>> print s.compile(dialect=OracleDialect(use_ansi=False)) - SELECT users.fullname - FROM users, addresses + SELECT users.fullname + FROM users, addresses WHERE users.id = addresses.user_id(+) If you don't know what that SQL means, don't worry ! The secret tribe of Oracle DBAs don't want their black magic being found out ;). -Intro to Generative Selects and Transformations +Intro to Generative Selects and Transformations ================================================ @@ -674,46 +674,46 @@ To support this, the ``select()`` construct we've been working with supports pie >>> query = users.select() >>> print query - SELECT users.id, users.name, users.fullname + SELECT users.id, users.name, users.fullname FROM users - + We encounter search criterion of "name='jack'". So we apply WHERE criterion stating such: .. sourcecode:: pycon+sql >>> query = query.where(users.c.name=='jack') - + Next, we encounter that they'd like the results in descending order by full name. We apply ORDER BY, using an extra modifier ``desc``: .. sourcecode:: pycon+sql >>> query = query.order_by(users.c.fullname.desc()) - + We also come across that they'd like only users who have an address at MSN. A quick way to tack this on is by using an EXISTS clause, which we correlate to the ``users`` table in the enclosing SELECT: .. sourcecode:: pycon+sql >>> from sqlalchemy.sql import exists >>> query = query.where( - ... exists([addresses.c.id], + ... exists([addresses.c.id], ... and_(addresses.c.user_id==users.c.id, addresses.c.email_address.like('%@msn.com')) ... ).correlate(users)) - + And finally, the application also wants to see the listing of email addresses at once; so to save queries, we outerjoin the ``addresses`` table (using an outer join so that users with no addresses come back as well; since we're programmatic, we might not have kept track that we used an EXISTS clause against the ``addresses`` table too...). Additionally, since the ``users`` and ``addresses`` table both have a column named ``id``, let's isolate their names from each other in the COLUMNS clause by using labels: .. sourcecode:: pycon+sql >>> query = query.column(addresses).select_from(users.outerjoin(addresses)).apply_labels() - + Let's bake for .0001 seconds and see what rises: .. sourcecode:: pycon+sql >>> conn.execute(query).fetchall() - {opensql}SELECT users.id AS users_id, users.name AS users_name, users.fullname AS users_fullname, addresses.id AS addresses_id, addresses.user_id AS addresses_user_id, addresses.email_address AS addresses_email_address - FROM users LEFT OUTER JOIN addresses ON users.id = addresses.user_id - WHERE users.name = ? AND (EXISTS (SELECT addresses.id - FROM addresses + {opensql}SELECT users.id AS users_id, users.name AS users_name, users.fullname AS users_fullname, addresses.id AS addresses_id, addresses.user_id AS addresses_user_id, addresses.email_address AS addresses_email_address + FROM users LEFT OUTER JOIN addresses ON users.id = addresses.user_id + WHERE users.name = ? AND (EXISTS (SELECT addresses.id + FROM addresses WHERE addresses.user_id = users.id AND addresses.email_address LIKE ?)) ORDER BY users.fullname DESC ['jack', '%@msn.com'] {stop}[(1, u'jack', u'Jack Jones', 1, 1, u'jack@yahoo.com'), (1, u'jack', u'Jack Jones', 2, 1, u'jack@msn.com')] @@ -725,22 +725,22 @@ So we started small, added one little thing at a time, and at the end we have a >>> a1 = addresses.alias() >>> query = query.replace_selectable(addresses, a1) >>> print query - {opensql}SELECT users.id AS users_id, users.name AS users_name, users.fullname AS users_fullname, addresses_1.id AS addresses_1_id, addresses_1.user_id AS addresses_1_user_id, addresses_1.email_address AS addresses_1_email_address - FROM users LEFT OUTER JOIN addresses AS addresses_1 ON users.id = addresses_1.user_id - WHERE users.name = :name_1 AND (EXISTS (SELECT addresses_1.id - FROM addresses AS addresses_1 + {opensql}SELECT users.id AS users_id, users.name AS users_name, users.fullname AS users_fullname, addresses_1.id AS addresses_1_id, addresses_1.user_id AS addresses_1_user_id, addresses_1.email_address AS addresses_1_email_address + FROM users LEFT OUTER JOIN addresses AS addresses_1 ON users.id = addresses_1.user_id + WHERE users.name = :name_1 AND (EXISTS (SELECT addresses_1.id + FROM addresses AS addresses_1 WHERE addresses_1.user_id = users.id AND addresses_1.email_address LIKE :email_address_1)) ORDER BY users.fullname DESC -One more thing though, with automatic labeling applied as well as anonymous aliasing, how do we retrieve the columns from the rows for this thing ? The label for the ``email_addresses`` column is now the generated name ``addresses_1_email_address``; and in another statement might be something different ! This is where accessing by result columns by ``Column`` object becomes very useful: +One more thing though, with automatic labeling applied as well as anonymous aliasing, how do we retrieve the columns from the rows for this thing ? The label for the ``email_addresses`` column is now the generated name ``addresses_1_email_address``; and in another statement might be something different ! This is where accessing by result columns by :class:`~sqlalchemy.schema.Column` object becomes very useful: .. sourcecode:: pycon+sql - + {sql}>>> for row in conn.execute(query): ... print "Name:", row[users.c.name], "; Email Address", row[a1.c.email_address] - SELECT users.id AS users_id, users.name AS users_name, users.fullname AS users_fullname, addresses_1.id AS addresses_1_id, addresses_1.user_id AS addresses_1_user_id, addresses_1.email_address AS addresses_1_email_address - FROM users LEFT OUTER JOIN addresses AS addresses_1 ON users.id = addresses_1.user_id - WHERE users.name = ? AND (EXISTS (SELECT addresses_1.id - FROM addresses AS addresses_1 + SELECT users.id AS users_id, users.name AS users_name, users.fullname AS users_fullname, addresses_1.id AS addresses_1_id, addresses_1.user_id AS addresses_1_user_id, addresses_1.email_address AS addresses_1_email_address + FROM users LEFT OUTER JOIN addresses AS addresses_1 ON users.id = addresses_1.user_id + WHERE users.name = ? AND (EXISTS (SELECT addresses_1.id + FROM addresses AS addresses_1 WHERE addresses_1.user_id = users.id AND addresses_1.email_address LIKE ?)) ORDER BY users.fullname DESC ['jack', '%@msn.com'] {stop}Name: jack ; Email Address jack@yahoo.com @@ -748,12 +748,12 @@ One more thing though, with automatic labeling applied as well as anonymous alia The above example, by its end, got significantly more intense than the typical end-user constructed SQL will usually be. However when writing higher-level tools such as ORMs, they become more significant. SQLAlchemy's ORM relies very heavily on techniques like this. -Everything Else +Everything Else ================ The concepts of creating SQL expressions have been introduced. What's left are more variants of the same themes. So now we'll catalog the rest of the important things we'll need to know. -Bind Parameter Objects +Bind Parameter Objects ---------------------- @@ -764,8 +764,8 @@ Throughout all these examples, SQLAlchemy is busy creating bind parameters where >>> from sqlalchemy.sql import bindparam >>> s = users.select(users.c.name==bindparam('username')) {sql}>>> conn.execute(s, username='wendy').fetchall() - SELECT users.id, users.name, users.fullname - FROM users + SELECT users.id, users.name, users.fullname + FROM users WHERE users.name = ? ['wendy'] {stop}[(2, u'wendy', u'Wendy Williams')] @@ -776,29 +776,29 @@ Another important aspect of bind parameters is that they may be assigned a type. >>> s = users.select(users.c.name.like(bindparam('username', type_=String) + text("'%'"))) {sql}>>> conn.execute(s, username='wendy').fetchall() - SELECT users.id, users.name, users.fullname - FROM users + SELECT users.id, users.name, users.fullname + FROM users WHERE users.name LIKE ? || '%' ['wendy'] {stop}[(2, u'wendy', u'Wendy Williams')] - - + + Bind parameters of the same name can also be used multiple times, where only a single named value is needed in the execute parameters: .. sourcecode:: pycon+sql - >>> s = select([users, addresses], - ... users.c.name.like(bindparam('name', type_=String) + text("'%'")) | - ... addresses.c.email_address.like(bindparam('name', type_=String) + text("'@%'")), + >>> s = select([users, addresses], + ... users.c.name.like(bindparam('name', type_=String) + text("'%'")) | + ... addresses.c.email_address.like(bindparam('name', type_=String) + text("'@%'")), ... from_obj=[users.outerjoin(addresses)]) {sql}>>> conn.execute(s, name='jack').fetchall() - SELECT users.id, users.name, users.fullname, addresses.id, addresses.user_id, addresses.email_address - FROM users LEFT OUTER JOIN addresses ON users.id = addresses.user_id + SELECT users.id, users.name, users.fullname, addresses.id, addresses.user_id, addresses.email_address + FROM users LEFT OUTER JOIN addresses ON users.id = addresses.user_id WHERE users.name LIKE ? || '%' OR addresses.email_address LIKE ? || '@%' ['jack', 'jack'] {stop}[(1, u'jack', u'Jack Jones', 1, 1, u'jack@yahoo.com'), (1, u'jack', u'Jack Jones', 2, 1, u'jack@msn.com')] -Functions +Functions --------- @@ -812,19 +812,19 @@ SQL functions are created using the ``func`` keyword, which generates functions >>> print func.concat('x', 'y') concat(:param_1, :param_2) - + By "generates", we mean that **any** SQL function is created based on the word you choose:: >>> print func.xyz_my_goofy_function() # doctest: +NORMALIZE_WHITESPACE - xyz_my_goofy_function() - + xyz_my_goofy_function() + Certain function names are known by SQLAlchemy, allowing special behavioral rules to be applied. Some for example are "ANSI" functions, which mean they don't get the parenthesis added after them, such as CURRENT_TIMESTAMP: .. sourcecode:: pycon+sql >>> print func.current_timestamp() CURRENT_TIMESTAMP - + Functions are most typically used in the columns clause of a select statement, and can also be labeled as well as given a type. Labeling a function is recommended so that the result can be targeted in a result row based on a string name, and assigning it a type is required when you need result-set processing to occur, such as for Unicode conversion and date conversions. Below, we use the result function ``scalar()`` to just read the first column of the first row and then close the result; the label, even though present, is not important in this case: .. sourcecode:: pycon+sql @@ -832,38 +832,38 @@ Functions are most typically used in the columns clause of a select statement, a >>> print conn.execute( ... select([func.max(addresses.c.email_address, type_=String).label('maxemail')]) ... ).scalar() - {opensql}SELECT max(addresses.email_address) AS maxemail + {opensql}SELECT max(addresses.email_address) AS maxemail FROM addresses [] {stop}www@www.org - + Databases such as PostgreSQL and Oracle which support functions that return whole result sets can be assembled into selectable units, which can be used in statements. Such as, a database function ``calculate()`` which takes the parameters ``x`` and ``y``, and returns three columns which we'd like to name ``q``, ``z`` and ``r``, we can construct using "lexical" column objects as well as bind parameters: .. sourcecode:: pycon+sql >>> from sqlalchemy.sql import column - >>> calculate = select([column('q'), column('z'), column('r')], + >>> calculate = select([column('q'), column('z'), column('r')], ... from_obj=[func.calculate(bindparam('x'), bindparam('y'))]) - + >>> print select([users], users.c.id > calculate.c.z) - SELECT users.id, users.name, users.fullname - FROM users, (SELECT q, z, r - FROM calculate(:x, :y)) + SELECT users.id, users.name, users.fullname + FROM users, (SELECT q, z, r + FROM calculate(:x, :y)) WHERE users.id > z - + If we wanted to use our ``calculate`` statement twice with different bind parameters, the ``unique_params()`` function will create copies for us, and mark the bind parameters as "unique" so that conflicting names are isolated. Note we also make two separate aliases of our selectable: .. sourcecode:: pycon+sql >>> s = select([users], users.c.id.between( - ... calculate.alias('c1').unique_params(x=17, y=45).c.z, + ... calculate.alias('c1').unique_params(x=17, y=45).c.z, ... calculate.alias('c2').unique_params(x=5, y=12).c.z)) - + >>> print s - SELECT users.id, users.name, users.fullname - FROM users, (SELECT q, z, r - FROM calculate(:x_1, :y_1)) AS c1, (SELECT q, z, r - FROM calculate(:x_2, :y_2)) AS c2 + SELECT users.id, users.name, users.fullname + FROM users, (SELECT q, z, r + FROM calculate(:x_1, :y_1)) AS c1, (SELECT q, z, r + FROM calculate(:x_2, :y_2)) AS c2 WHERE users.id BETWEEN c1.z AND c2.z >>> s.compile().params @@ -871,7 +871,7 @@ If we wanted to use our ``calculate`` statement twice with different bind parame See also :attr:`sqlalchemy.sql.expression.func`. -Unions and Other Set Operations +Unions and Other Set Operations ------------------------------- Unions come in two flavors, UNION and UNION ALL, which are available via module level functions: @@ -885,14 +885,14 @@ Unions come in two flavors, UNION and UNION ALL, which are available via module ... ).order_by(addresses.c.email_address) {sql}>>> print conn.execute(u).fetchall() - SELECT addresses.id, addresses.user_id, addresses.email_address - FROM addresses - WHERE addresses.email_address = ? UNION SELECT addresses.id, addresses.user_id, addresses.email_address - FROM addresses + SELECT addresses.id, addresses.user_id, addresses.email_address + FROM addresses + WHERE addresses.email_address = ? UNION SELECT addresses.id, addresses.user_id, addresses.email_address + FROM addresses WHERE addresses.email_address LIKE ? ORDER BY addresses.email_address ['foo@bar.com', '%@yahoo.com'] {stop}[(1, 1, u'jack@yahoo.com')] - + Also available, though not supported on all databases, are ``intersect()``, ``intersect_all()``, ``except_()``, and ``except_all()``: .. sourcecode:: pycon+sql @@ -904,10 +904,10 @@ Also available, though not supported on all databases, are ``intersect()``, ``in ... ) {sql}>>> print conn.execute(u).fetchall() - SELECT addresses.id, addresses.user_id, addresses.email_address - FROM addresses - WHERE addresses.email_address LIKE ? EXCEPT SELECT addresses.id, addresses.user_id, addresses.email_address - FROM addresses + SELECT addresses.id, addresses.user_id, addresses.email_address + FROM addresses + WHERE addresses.email_address LIKE ? EXCEPT SELECT addresses.id, addresses.user_id, addresses.email_address + FROM addresses WHERE addresses.email_address LIKE ? ['%@%.com', '%@msn.com'] {stop}[(1, 1, u'jack@yahoo.com'), (4, 2, u'wendy@aol.com')] @@ -926,20 +926,20 @@ the "union" to be stated as a subquery: ... addresses.select(addresses.c.email_address.like('%@msn.com')) ... ) {sql}>>> print conn.execute(u).fetchall() # doctest: +NORMALIZE_WHITESPACE - SELECT anon_1.id, anon_1.user_id, anon_1.email_address - FROM (SELECT addresses.id AS id, addresses.user_id AS user_id, - addresses.email_address AS email_address FROM addresses - WHERE addresses.email_address LIKE ? UNION SELECT addresses.id AS id, - addresses.user_id AS user_id, addresses.email_address AS email_address - FROM addresses WHERE addresses.email_address LIKE ?) AS anon_1 EXCEPT - SELECT addresses.id, addresses.user_id, addresses.email_address - FROM addresses + SELECT anon_1.id, anon_1.user_id, anon_1.email_address + FROM (SELECT addresses.id AS id, addresses.user_id AS user_id, + addresses.email_address AS email_address FROM addresses + WHERE addresses.email_address LIKE ? UNION SELECT addresses.id AS id, + addresses.user_id AS user_id, addresses.email_address AS email_address + FROM addresses WHERE addresses.email_address LIKE ?) AS anon_1 EXCEPT + SELECT addresses.id, addresses.user_id, addresses.email_address + FROM addresses WHERE addresses.email_address LIKE ? ['%@yahoo.com', '%@msn.com', '%@msn.com'] {stop}[(1, 1, u'jack@yahoo.com')] -Scalar Selects +Scalar Selects -------------- @@ -948,12 +948,12 @@ To embed a SELECT in a column expression, use ``as_scalar()``: .. sourcecode:: pycon+sql {sql}>>> print conn.execute(select([ # doctest: +NORMALIZE_WHITESPACE - ... users.c.name, + ... users.c.name, ... select([func.count(addresses.c.id)], users.c.id==addresses.c.user_id).as_scalar() ... ])).fetchall() SELECT users.name, (SELECT count(addresses.id) AS count_1 - FROM addresses - WHERE users.id = addresses.user_id) AS anon_1 + FROM addresses + WHERE users.id = addresses.user_id) AS anon_1 FROM users [] {stop}[(u'jack', 2), (u'wendy', 2), (u'fred', 0), (u'mary', 0)] @@ -963,72 +963,72 @@ Alternatively, applying a ``label()`` to a select evaluates it as a scalar as we .. sourcecode:: pycon+sql {sql}>>> print conn.execute(select([ # doctest: +NORMALIZE_WHITESPACE - ... users.c.name, + ... users.c.name, ... select([func.count(addresses.c.id)], users.c.id==addresses.c.user_id).label('address_count') ... ])).fetchall() SELECT users.name, (SELECT count(addresses.id) AS count_1 - FROM addresses - WHERE users.id = addresses.user_id) AS address_count + FROM addresses + WHERE users.id = addresses.user_id) AS address_count FROM users [] {stop}[(u'jack', 2), (u'wendy', 2), (u'fred', 0), (u'mary', 0)] -Correlated Subqueries +Correlated Subqueries --------------------- Notice in the examples on "scalar selects", the FROM clause of each embedded select did not contain the ``users`` table in its FROM clause. This is because SQLAlchemy automatically attempts to correlate embedded FROM objects to that of an enclosing query. To disable this, or to specify explicit FROM clauses to be correlated, use ``correlate()``:: >>> s = select([users.c.name], users.c.id==select([users.c.id]).correlate(None)) >>> print s - SELECT users.name - FROM users - WHERE users.id = (SELECT users.id + SELECT users.name + FROM users + WHERE users.id = (SELECT users.id FROM users) >>> s = select([users.c.name, addresses.c.email_address], users.c.id== ... select([users.c.id], users.c.id==addresses.c.user_id).correlate(addresses) ... ) >>> print s - SELECT users.name, addresses.email_address - FROM users, addresses - WHERE users.id = (SELECT users.id - FROM users + SELECT users.name, addresses.email_address + FROM users, addresses + WHERE users.id = (SELECT users.id + FROM users WHERE users.id = addresses.user_id) -Ordering, Grouping, Limiting, Offset...ing... +Ordering, Grouping, Limiting, Offset...ing... --------------------------------------------- -The ``select()`` function can take keyword arguments ``order_by``, ``group_by`` (as well as ``having``), ``limit``, and ``offset``. There's also ``distinct=True``. These are all also available as generative functions. ``order_by()`` expressions can use the modifiers ``asc()`` or ``desc()`` to indicate ascending or descending. +The ``select()`` function can take keyword arguments ``order_by``, ``group_by`` (as well as ``having``), ``limit``, and ``offset``. There's also ``distinct=True``. These are all also available as generative functions. ``order_by()`` expressions can use the modifiers ``asc()`` or ``desc()`` to indicate ascending or descending. .. sourcecode:: pycon+sql >>> s = select([addresses.c.user_id, func.count(addresses.c.id)]).\ ... group_by(addresses.c.user_id).having(func.count(addresses.c.id)>1) {sql}>>> print conn.execute(s).fetchall() - SELECT addresses.user_id, count(addresses.id) AS count_1 - FROM addresses GROUP BY addresses.user_id + SELECT addresses.user_id, count(addresses.id) AS count_1 + FROM addresses GROUP BY addresses.user_id HAVING count(addresses.id) > ? [1] {stop}[(1, 2), (2, 2)] - + >>> s = select([addresses.c.email_address, addresses.c.id]).distinct().\ ... order_by(addresses.c.email_address.desc(), addresses.c.id) {sql}>>> conn.execute(s).fetchall() - SELECT DISTINCT addresses.email_address, addresses.id + SELECT DISTINCT addresses.email_address, addresses.id FROM addresses ORDER BY addresses.email_address DESC, addresses.id [] {stop}[(u'www@www.org', 3), (u'wendy@aol.com', 4), (u'jack@yahoo.com', 1), (u'jack@msn.com', 2)] - + >>> s = select([addresses]).offset(1).limit(1) {sql}>>> print conn.execute(s).fetchall() # doctest: +NORMALIZE_WHITESPACE - SELECT addresses.id, addresses.user_id, addresses.email_address - FROM addresses + SELECT addresses.id, addresses.user_id, addresses.email_address + FROM addresses LIMIT 1 OFFSET 1 [] {stop}[(2, 1, u'jack@msn.com')] - -Updates + +Updates ======== @@ -1042,7 +1042,7 @@ Finally, we're back to UPDATE. Updates work a lot like INSERTS, except there is ['ed', 'jack'] COMMIT {stop} - + >>> # use bind parameters >>> u = users.update().where(users.c.name==bindparam('oldname')).values(name=bindparam('newname')) {sql}>>> conn.execute(u, oldname='jack', newname='ed') #doctest: +ELLIPSIS @@ -1058,7 +1058,7 @@ Finally, we're back to UPDATE. Updates work a lot like INSERTS, except there is COMMIT {stop} -Correlated Updates +Correlated Updates ------------------ A correlated update lets you update a table using selection from another table, or the same table: @@ -1067,15 +1067,15 @@ A correlated update lets you update a table using selection from another table, >>> s = select([addresses.c.email_address], addresses.c.user_id==users.c.id).limit(1) {sql}>>> conn.execute(users.update().values(fullname=s)) #doctest: +ELLIPSIS,+NORMALIZE_WHITESPACE - UPDATE users SET fullname=(SELECT addresses.email_address - FROM addresses - WHERE addresses.user_id = users.id + UPDATE users SET fullname=(SELECT addresses.email_address + FROM addresses + WHERE addresses.user_id = users.id LIMIT 1 OFFSET 0) [] COMMIT {stop} -Deletes +Deletes ======== @@ -1088,14 +1088,14 @@ Finally, a delete. Easy enough: [] COMMIT {stop} - + {sql}>>> conn.execute(users.delete().where(users.c.name > 'm')) #doctest: +ELLIPSIS DELETE FROM users WHERE users.name > ? ['m'] COMMIT {stop} -Further Reference +Further Reference ================== API docs: :mod:`sqlalchemy.sql.expression` @@ -1106,4 +1106,4 @@ Engine/Connection/Execution Reference: :ref:`engines_toplevel` SQL Types: :ref:`types` - +