From: Mike Bayer Date: Mon, 12 Aug 2013 15:33:17 +0000 (-0400) Subject: - more "when do i create the session" docs, get specific X-Git-Tag: rel_0_8_3~73 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=7779c22e401642ae40702d8d0523580ca65bad6e;p=thirdparty%2Fsqlalchemy%2Fsqlalchemy.git - more "when do i create the session" docs, get specific - more glossary terms - turn the session FAQ into rst sections at last --- diff --git a/doc/build/glossary.rst b/doc/build/glossary.rst index 5646686913..7c497df769 100644 --- a/doc/build/glossary.rst +++ b/doc/build/glossary.rst @@ -95,6 +95,23 @@ Glossary class which each represent a particular database column or relationship to a related class. + identity map + A mapping between Python objects and their database identities. + The identity map is a collection that's associated with an + ORM :term:`session` object, and maintains a single instance + of every database object keyed to its identity. The advantage + to this pattern is that all operations which occur for a particular + database identity are transparently coordinated onto a single + object instance. When using an identity map in conjunction with + an :term:`isolated` transaction, having a reference + to an object that's known to have a particular primary key can + be considered from a practical standpoint to be a + proxy to the actual database row. + + .. seealso:: + + Martin Fowler - Identity Map - http://martinfowler.com/eaaCatalog/identityMap.html + lazy load lazy loads In object relational mapping, a "lazy load" refers to an @@ -263,6 +280,16 @@ Glossary :doc:`orm/session` + Session + The container or scope for ORM database operations. Sessions + load instances from the database, track changes to mapped + instances and persist changes in a single unit of work when + flushed. + + .. seealso:: + + :doc:`orm/session` + columns clause The portion of the ``SELECT`` statement which enumerates the SQL expressions to be returned in the result set. The expressions @@ -411,3 +438,86 @@ Glossary query via its ``FROM`` clause is not possible, because the correlation can only proceed once the original source rows from the enclosing statement's FROM clause are available. + + ACID + ACID model + An acronym for "Atomicity, Consistency, Isolation, + Durability"; a set of properties that guarantee that + database transactions are processed reliably. + (via Wikipedia) + + .. seealso:: + + :term:`atomicity` + + :term:`consistency` + + :term:`isolation` + + :term:`durability` + + http://en.wikipedia.org/wiki/ACID_Model + + atomicity + Atomicity is one of the components of the :term:`ACID` model, + and requires that each transaction is "all or nothing": + if one part of the transaction fails, the entire transaction + fails, and the database state is left unchanged. An atomic + system must guarantee atomicity in each and every situation, + including power failures, errors, and crashes. + (via Wikipedia) + + .. seealso:: + + :term:`ACID` + + http://en.wikipedia.org/wiki/Atomicity_(database_systems) + + consistency + Consistency is one of the compoments of the :term:`ACID` model, + and ensures that any transaction will + bring the database from one valid state to another. Any data + written to the database must be valid according to all defined + rules, including but not limited to :term:`constraints`, cascades, + triggers, and any combination thereof. + (via Wikipedia) + + .. seealso:: + + :term:`ACID` + + http://en.wikipedia.org/wiki/Consistency_(database_systems) + + isolation + isolated + The isolation property of the :term:`ACID` model + ensures that the concurrent execution + of transactions results in a system state that would be + obtained if transactions were executed serially, i.e. one + after the other. Each transaction must execute in total + isolation i.e. if T1 and T2 execute concurrently then each + should remain independent of the other. + (via Wikipedia) + + .. seealso:: + + :term:`ACID` + + http://en.wikipedia.org/wiki/Isolation_(database_systems) + + durability + Durability is a property of the :term:`ACID` model + which means that once a transaction has been committed, + it will remain so, even in the event of power loss, crashes, + or errors. In a relational database, for instance, once a + group of SQL statements execute, the results need to be stored + permanently (even if the database crashes immediately + thereafter). + (via Wikipedia) + + .. seealso:: + + :term:`ACID` + + http://en.wikipedia.org/wiki/Durability_(database_systems) + diff --git a/doc/build/orm/session.rst b/doc/build/orm/session.rst index 6774af2d94..52c7b1d18d 100644 --- a/doc/build/orm/session.rst +++ b/doc/build/orm/session.rst @@ -203,190 +203,293 @@ at the same time). Session Frequently Asked Questions ----------------------------------- -* When do I make a :class:`.sessionmaker` ? - - Just one time, somewhere in your application's global scope. It should be - looked upon as part of your application's configuration. If your - application has three .py files in a package, you could, for example, - place the :class:`.sessionmaker` line in your ``__init__.py`` file; from - that point on your other modules say "from mypackage import Session". That - way, everyone else just uses :class:`.Session()`, - and the configuration of that session is controlled by that central point. - - If your application starts up, does imports, but does not know what - database it's going to be connecting to, you can bind the - :class:`.Session` at the "class" level to the - engine later on, using :meth:`.sessionmaker.configure`. - - In the examples in this section, we will frequently show the - :class:`.sessionmaker` being created right above the line where we actually - invoke :class:`.Session`. But that's just for - example's sake! In reality, the :class:`.sessionmaker` would be somewhere - at the module level. The calls to instantiate :class:`.Session` - would then be placed at the point in the application where database - conversations begin. - -* When do I construct a :class:`.Session`, when do I commit it, and when do I close it ? - - A :class:`.Session` is typically constructed at the beginning of a logical - operation where database access is potentially anticipated. - - The :class:`.Session`, whenever it is used to talk to the database, - begins a database transaction as soon as it starts communicating. - Assuming the ``autocommit`` flag is left at its recommended default - of ``False``, this transaction remains in progress until the :class:`.Session` - is rolled back, committed, or closed. The :class:`.Session` will - begin a new transaction if it is used again, subsequent to the previous - transaction ending; from this it follows that the :class:`.Session` - is capable of having a lifespan across many transactions, though only - one at a time. We refer to these two concepts as **transaction scope** - and **session scope**. - - The implication here is that the SQLAlchemy ORM is encouraging the - developer to establish these two scopes in his or her application, - including not only when the scopes begin and end, but also the - expanse of those scopes, for example should a single - :class:`.Session` instance be local to the execution flow within a - function or method, should it be a global object used by the - entire application, or somewhere in between these two. - - The burden placed on the developer to determine this scope is one - area where the SQLAlchemy ORM necessarily has a strong opinion - about how the database should be used. The unit-of-work pattern - is specifically one of accumulating changes over time and flushing - them periodically, keeping in-memory state in sync with what's - known to be present in a local transaction. This pattern is only - effective when meaningful transaction scopes are in place. - - It's usually not very hard to determine the best points at which - to begin and end the scope of a :class:`.Session`, though the wide - variety of application architectures possible can introduce - challenging situations. - - A common choice is to tear down the :class:`.Session` at the same - time the transaction ends, meaning the transaction and session scopes - are the same. This is a great choice to start out with as it - removes the need to consider session scope as separate from transaction - scope. - - While there's no one-size-fits-all recommendation for how transaction - scope should be determined, there are common patterns. Especially - if one is writing a web application, the choice is pretty much established. - - A web application is the easiest case because such an appication is already - constructed around a single, consistent scope - this is the **request**, - which represents an incoming request from a browser, the processing - of that request to formulate a response, and finally the delivery of that - response back to the client. Integrating web applications with the - :class:`.Session` is then the straightforward task of linking the - scope of the :class:`.Session` to that of the request. The :class:`.Session` - can be established as the request begins, or using a **lazy initialization** - pattern which establishes one as soon as it is needed. The request - then proceeds, with some system in place where application logic can access - the current :class:`.Session` in a manner associated with how the actual - request object is accessed. As the request ends, the :class:`.Session` - is torn down as well, usually through the usage of event hooks provided - by the web framework. The transaction used by the :class:`.Session` - may also be committed at this point, or alternatively the application may - opt for an explicit commit pattern, only committing for those requests - where one is warranted, but still always tearing down the :class:`.Session` - unconditionally at the end. - - Most web frameworks include infrastructure to establish a single - :class:`.Session`, associated with the request, which is correctly - constructed and torn down corresponding - torn down at the end of a request. Such infrastructure pieces - include products such as `Flask-SQLAlchemy `_, - for usage in conjunction with the Flask web framework, - and `Zope-SQLAlchemy `_, - for usage in conjunction with the Pyramid and Zope frameworks. - SQLAlchemy strongly recommends that these products be used as - available. - - In those situations where integration libraries are not available, - SQLAlchemy includes its own "helper" class known as - :class:`.scoped_session`. A tutorial on the usage of this object - is at :ref:`unitofwork_contextual`. It provides both a quick way - to associate a :class:`.Session` with the current thread, as well as - patterns to associate :class:`.Session` objects with other kinds of - scopes. - - As mentioned before, for non-web applications there is no one clear - pattern, as applications themselves don't have just one pattern - of architecture. The best strategy is to attempt to demarcate - "operations", points at which a particular thread begins to perform - a series of operations for some period of time, which can be committed - at the end. Some examples: - - * A background daemon which spawns off child forks - would want to create a :class:`.Session` local to each child - process work with that :class:`.Session` through the life of the "job" - that the fork is handling, then tear it down when the job is completed. - - * For a command-line script, the application would create a single, global - :class:`.Session` that is established when the program begins to do its - work, and commits it right as the program is completing its task. - - * For a GUI interface-driven application, the scope of the :class:`.Session` - may best be within the scope of a user-generated event, such as a button - push. Or, the scope may correspond to explicit user interaction, such as - the user "opening" a series of records, then "saving" them. - -* Is the Session a cache ? - - Yeee...no. It's somewhat used as a cache, in that it implements the - identity map pattern, and stores objects keyed to their primary key. - However, it doesn't do any kind of query caching. This means, if you say - ``session.query(Foo).filter_by(name='bar')``, even if ``Foo(name='bar')`` - is right there, in the identity map, the session has no idea about that. - It has to issue SQL to the database, get the rows back, and then when it - sees the primary key in the row, *then* it can look in the local identity - map and see that the object is already there. It's only when you say - ``query.get({some primary key})`` that the - :class:`~sqlalchemy.orm.session.Session` doesn't have to issue a query. - - Additionally, the Session stores object instances using a weak reference - by default. This also defeats the purpose of using the Session as a cache. - - The :class:`.Session` is not designed to be a - global object from which everyone consults as a "registry" of objects. - That's more the job of a **second level cache**. SQLAlchemy provides - a pattern for implementing second level caching using `dogpile.cache `_, - via the :ref:`examples_caching` example. - -* How can I get the :class:`~sqlalchemy.orm.session.Session` for a certain object ? - - Use the :meth:`~.Session.object_session` classmethod - available on :class:`~sqlalchemy.orm.session.Session`:: - - session = Session.object_session(someobject) - -* Is the session thread-safe? - - The :class:`.Session` is very much intended to be used in a - **non-concurrent** fashion, which usually means in only one thread at a - time. - - The :class:`.Session` should be used in such a way that one - instance exists for a single series of operations within a single - transaction. One expedient way to get this effect is by associating - a :class:`.Session` with the current thread (see :ref:`unitofwork_contextual` - for background). Another is to use a pattern - where the :class:`.Session` is passed between functions and is otherwise - not shared with other threads. - - The bigger point is that you should not *want* to use the session - with multiple concurrent threads. That would be like having everyone at a - restaurant all eat from the same plate. The session is a local "workspace" - that you use for a specific set of tasks; you don't want to, or need to, - share that session with other threads who are doing some other task. - - If there are in fact multiple threads participating - in the same task, then you may consider sharing the session between - those threads, though this would be an extremely unusual scenario. - In this case it would be necessary - to implement a proper locking scheme so that the :class:`.Session` is still not - exposed to concurrent access. +When do I make a :class:`.sessionmaker`? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Just one time, somewhere in your application's global scope. It should be +looked upon as part of your application's configuration. If your +application has three .py files in a package, you could, for example, +place the :class:`.sessionmaker` line in your ``__init__.py`` file; from +that point on your other modules say "from mypackage import Session". That +way, everyone else just uses :class:`.Session()`, +and the configuration of that session is controlled by that central point. + +If your application starts up, does imports, but does not know what +database it's going to be connecting to, you can bind the +:class:`.Session` at the "class" level to the +engine later on, using :meth:`.sessionmaker.configure`. + +In the examples in this section, we will frequently show the +:class:`.sessionmaker` being created right above the line where we actually +invoke :class:`.Session`. But that's just for +example's sake! In reality, the :class:`.sessionmaker` would be somewhere +at the module level. The calls to instantiate :class:`.Session` +would then be placed at the point in the application where database +conversations begin. + +When do I construct a :class:`.Session`, when do I commit it, and when do I close it? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. topic:: tl;dr; + + As a general rule, keep the lifecycle of the session **separate and + external** from functions and objects that access and/or manipulate + database data. + +A :class:`.Session` is typically constructed at the beginning of a logical +operation where database access is potentially anticipated. + +The :class:`.Session`, whenever it is used to talk to the database, +begins a database transaction as soon as it starts communicating. +Assuming the ``autocommit`` flag is left at its recommended default +of ``False``, this transaction remains in progress until the :class:`.Session` +is rolled back, committed, or closed. The :class:`.Session` will +begin a new transaction if it is used again, subsequent to the previous +transaction ending; from this it follows that the :class:`.Session` +is capable of having a lifespan across many transactions, though only +one at a time. We refer to these two concepts as **transaction scope** +and **session scope**. + +The implication here is that the SQLAlchemy ORM is encouraging the +developer to establish these two scopes in his or her application, +including not only when the scopes begin and end, but also the +expanse of those scopes, for example should a single +:class:`.Session` instance be local to the execution flow within a +function or method, should it be a global object used by the +entire application, or somewhere in between these two. + +The burden placed on the developer to determine this scope is one +area where the SQLAlchemy ORM necessarily has a strong opinion +about how the database should be used. The :term:`unit of work` pattern +is specifically one of accumulating changes over time and flushing +them periodically, keeping in-memory state in sync with what's +known to be present in a local transaction. This pattern is only +effective when meaningful transaction scopes are in place. + +It's usually not very hard to determine the best points at which +to begin and end the scope of a :class:`.Session`, though the wide +variety of application architectures possible can introduce +challenging situations. + +A common choice is to tear down the :class:`.Session` at the same +time the transaction ends, meaning the transaction and session scopes +are the same. This is a great choice to start out with as it +removes the need to consider session scope as separate from transaction +scope. + +While there's no one-size-fits-all recommendation for how transaction +scope should be determined, there are common patterns. Especially +if one is writing a web application, the choice is pretty much established. + +A web application is the easiest case because such an appication is already +constructed around a single, consistent scope - this is the **request**, +which represents an incoming request from a browser, the processing +of that request to formulate a response, and finally the delivery of that +response back to the client. Integrating web applications with the +:class:`.Session` is then the straightforward task of linking the +scope of the :class:`.Session` to that of the request. The :class:`.Session` +can be established as the request begins, or using a :term:`lazy initialization` +pattern which establishes one as soon as it is needed. The request +then proceeds, with some system in place where application logic can access +the current :class:`.Session` in a manner associated with how the actual +request object is accessed. As the request ends, the :class:`.Session` +is torn down as well, usually through the usage of event hooks provided +by the web framework. The transaction used by the :class:`.Session` +may also be committed at this point, or alternatively the application may +opt for an explicit commit pattern, only committing for those requests +where one is warranted, but still always tearing down the :class:`.Session` +unconditionally at the end. + +Most web frameworks include infrastructure to establish a single +:class:`.Session`, associated with the request, which is correctly +constructed and torn down corresponding +torn down at the end of a request. Such infrastructure pieces +include products such as `Flask-SQLAlchemy `_, +for usage in conjunction with the Flask web framework, +and `Zope-SQLAlchemy `_, +for usage in conjunction with the Pyramid and Zope frameworks. +SQLAlchemy strongly recommends that these products be used as +available. + +In those situations where integration libraries are not available, +SQLAlchemy includes its own "helper" class known as +:class:`.scoped_session`. A tutorial on the usage of this object +is at :ref:`unitofwork_contextual`. It provides both a quick way +to associate a :class:`.Session` with the current thread, as well as +patterns to associate :class:`.Session` objects with other kinds of +scopes. + +As mentioned before, for non-web applications there is no one clear +pattern, as applications themselves don't have just one pattern +of architecture. The best strategy is to attempt to demarcate +"operations", points at which a particular thread begins to perform +a series of operations for some period of time, which can be committed +at the end. Some examples: + +* A background daemon which spawns off child forks + would want to create a :class:`.Session` local to each child + process, work with that :class:`.Session` through the life of the "job" + that the fork is handling, then tear it down when the job is completed. + +* For a command-line script, the application would create a single, global + :class:`.Session` that is established when the program begins to do its + work, and commits it right as the program is completing its task. + +* For a GUI interface-driven application, the scope of the :class:`.Session` + may best be within the scope of a user-generated event, such as a button + push. Or, the scope may correspond to explicit user interaction, such as + the user "opening" a series of records, then "saving" them. + +As a general rule, the application should manage the lifecycle of the +session *externally* to functions that deal with specific data. This is a +fundamental separation of concerns which keeps data-specific operations +agnostic of the context in which they access and manipulate that data. + +E.g. **don't do this**:: + + ### this is the **wrong way to do it** ### + + class ThingOne(object): + def go(self): + session = Session() + try: + session.query(FooBar).update({"x": 5}) + session.commit() + except: + session.rollback() + raise + + class ThingTwo(object): + def go(self): + session = Session() + try: + session.query(Widget).update({"q": 18}) + session.commit() + except: + session.rollback() + raise + + def run_my_program(): + ThingOne().go() + ThingTwo().go() + +Keep the lifecycle of the session (and usually the transaction) +**separate and external**:: + + ### this is a **better** (but not the only) way to do it ### + + class ThingOne(object): + def go(self, session): + session.query(FooBar).update({"x": 5}) + + class ThingTwo(object): + def go(self): + session.query(Widget).update({"q": 18}) + + def run_my_program(): + session = Session() + try: + ThingOne().go(session) + ThingTwo().go(session) + + session.commit() + except: + session.rollback() + raise + finally: + session.close() + +The advanced developer will try to keep the details of session, transaction +and exception management as far as possible from the details of the program +doing its work. For example, we can further separate concerns using a `context manager `_:: + + ### another way (but again *not the only way*) to do it ### + + from contextlib import contextmanager + + @contextmanager + def session_scope(): + """Provide a transactional scope around a series of operations.""" + session = Session() + try: + yield session + session.commit() + except: + session.rollback() + raise + finally: + session.close() + + + def run_my_program(): + with session_scope() as session: + ThingOne().go(session) + ThingTwo().go(session) + + +Is the Session a cache? +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Yeee...no. It's somewhat used as a cache, in that it implements the +:term:`identity map` pattern, and stores objects keyed to their primary key. +However, it doesn't do any kind of query caching. This means, if you say +``session.query(Foo).filter_by(name='bar')``, even if ``Foo(name='bar')`` +is right there, in the identity map, the session has no idea about that. +It has to issue SQL to the database, get the rows back, and then when it +sees the primary key in the row, *then* it can look in the local identity +map and see that the object is already there. It's only when you say +``query.get({some primary key})`` that the +:class:`~sqlalchemy.orm.session.Session` doesn't have to issue a query. + +Additionally, the Session stores object instances using a weak reference +by default. This also defeats the purpose of using the Session as a cache. + +The :class:`.Session` is not designed to be a +global object from which everyone consults as a "registry" of objects. +That's more the job of a **second level cache**. SQLAlchemy provides +a pattern for implementing second level caching using `dogpile.cache `_, +via the :ref:`examples_caching` example. + +How can I get the :class:`~sqlalchemy.orm.session.Session` for a certain object? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Use the :meth:`~.Session.object_session` classmethod +available on :class:`~sqlalchemy.orm.session.Session`:: + + session = Session.object_session(someobject) + +The newer :ref:`core_inspection_toplevel` system can also be used:: + + from sqlalchemy import inspect + session = inspect(object).session + +Is the session thread-safe? +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The :class:`.Session` is very much intended to be used in a +**non-concurrent** fashion, which usually means in only one thread at a +time. + +The :class:`.Session` should be used in such a way that one +instance exists for a single series of operations within a single +transaction. One expedient way to get this effect is by associating +a :class:`.Session` with the current thread (see :ref:`unitofwork_contextual` +for background). Another is to use a pattern +where the :class:`.Session` is passed between functions and is otherwise +not shared with other threads. + +The bigger point is that you should not *want* to use the session +with multiple concurrent threads. That would be like having everyone at a +restaurant all eat from the same plate. The session is a local "workspace" +that you use for a specific set of tasks; you don't want to, or need to, +share that session with other threads who are doing some other task. + +If there are in fact multiple threads participating +in the same task, then you may consider sharing the session between +those threads, though this would be an extremely unusual scenario. +In this case it would be necessary +to implement a proper locking scheme so that the :class:`.Session` is still not +exposed to concurrent access. Querying