{sql}>>> our_user = session.query(User).filter_by(name='ed').first() # doctest:+ELLIPSIS,+NORMALIZE_WHITESPACE
BEGIN
INSERT INTO users (name, fullname, password) VALUES (?, ?, ?)
- ['ed', 'Ed Jones', 'edspassword']
+ ('ed', 'Ed Jones', 'edspassword')
SELECT users.id AS users_id, users.name AS users_name, users.fullname AS users_fullname, users.password AS users_password
FROM users
WHERE users.name = ?
LIMIT 1 OFFSET 0
- ['ed']
+ ('ed',)
{stop}>>> our_user
<User('ed','Ed Jones', 'edspassword')>
{sql}>>> session.commit()
UPDATE users SET password=? WHERE users.id = ?
- ['f8s7ccs', 1]
+ ('f8s7ccs', 1)
INSERT INTO users (name, fullname, password) VALUES (?, ?, ?)
- ['wendy', 'Wendy Williams', 'foobar']
+ ('wendy', 'Wendy Williams', 'foobar')
INSERT INTO users (name, fullname, password) VALUES (?, ?, ?)
- ['mary', 'Mary Contrary', 'xxg527']
+ ('mary', 'Mary Contrary', 'xxg527')
INSERT INTO users (name, fullname, password) VALUES (?, ?, ?)
- ['fred', 'Fred Flinstone', 'blah']
+ ('fred', 'Fred Flinstone', 'blah')
COMMIT
``commit()`` flushes whatever remaining changes remain to the database, and commits the transaction. The connection resources referenced by the session are now returned to the connection pool. Subsequent operations with this session will occur in a **new** transaction, which will again re-acquire connection resources when first needed.
SELECT users.id AS users_id, users.name AS users_name, users.fullname AS users_fullname, users.password AS users_password
FROM users
WHERE users.id = ?
- [1]
+ (1,)
{stop}1
After the :class:`~sqlalchemy.orm.session.Session` inserts new rows in the database, all newly generated identifiers and database-generated defaults become available on the instance, either immediately or via load-on-first-access. In this case, the entire row was re-loaded on access because a new transaction was begun after we issued ``commit()``. SQLAlchemy by default refreshes data from a previous transaction the first time it's accessed within a new transaction, so that the most recent state is available. The level of reloading is configurable as is described in the chapter on Sessions.
{sql}>>> session.query(User).filter(User.name.in_(['Edwardo', 'fakeuser'])).all() #doctest: +NORMALIZE_WHITESPACE
UPDATE users SET name=? WHERE users.id = ?
- ['Edwardo', 1]
+ ('Edwardo', 1)
INSERT INTO users (name, fullname, password) VALUES (?, ?, ?)
- ['fakeuser', 'Invalid', '12345']
+ ('fakeuser', 'Invalid', '12345')
SELECT users.id AS users_id, users.name AS users_name, users.fullname AS users_fullname, users.password AS users_password
FROM users
WHERE users.name IN (?, ?)
- ['Edwardo', 'fakeuser']
+ ('Edwardo', 'fakeuser')
{stop}[<User('Edwardo','Ed Jones', 'f8s7ccs')>, <User('fakeuser','Invalid', '12345')>]
Rolling back, we can see that ``ed_user``'s name is back to ``ed``, and ``fake_user`` has been kicked out of the session:
SELECT users.id AS users_id, users.name AS users_name, users.fullname AS users_fullname, users.password AS users_password
FROM users
WHERE users.id = ?
- [1]
+ (1,)
{stop}u'ed'
>>> fake_user in session
False
SELECT users.id AS users_id, users.name AS users_name, users.fullname AS users_fullname, users.password AS users_password
FROM users
WHERE users.name IN (?, ?)
- ['ed', 'fakeuser']
+ ('ed', 'fakeuser')
{stop}[<User('ed','Ed Jones', 'f8s7ccs')>]
.. _ormtutorial_querying:
SELECT users.id AS users_id, users.name AS users_name,
users.fullname AS users_fullname, users.password AS users_password
FROM users ORDER BY users.id
- []
+ ()
{stop}ed Ed Jones
wendy Wendy Williams
mary Mary Contrary
... print name, fullname
SELECT users.name AS users_name, users.fullname AS users_fullname
FROM users
- []
+ ()
{stop}ed Ed Jones
wendy Wendy Williams
mary Mary Contrary
... print row.User, row.name
SELECT users.id AS users_id, users.name AS users_name, users.fullname AS users_fullname, users.password AS users_password
FROM users
- []
+ ()
{stop}<User('ed','Ed Jones', 'f8s7ccs')> ed
<User('wendy','Wendy Williams', 'foobar')> wendy
<User('mary','Mary Contrary', 'xxg527')> mary
... print row.user_alias, row.name_label
SELECT users_1.id AS users_1_id, users_1.name AS users_1_name, users_1.fullname AS users_1_fullname, users_1.password AS users_1_password, users_1.name AS name_label
FROM users AS users_1
- []{stop}
+ (){stop}
<User('ed','Ed Jones', 'f8s7ccs')> ed
<User('wendy','Wendy Williams', 'foobar')> wendy
<User('mary','Mary Contrary', 'xxg527')> mary
SELECT users.id AS users_id, users.name AS users_name, users.fullname AS users_fullname, users.password AS users_password
FROM users ORDER BY users.id
LIMIT 2 OFFSET 1
- []{stop}
+ (){stop}
<User('wendy','Wendy Williams', 'foobar')>
<User('mary','Mary Contrary', 'xxg527')>
... print name
SELECT users.name AS users_name FROM users
WHERE users.fullname = ?
- ['Ed Jones']
+ ('Ed Jones',)
{stop}ed
...or ``filter()``, which uses more flexible SQL expression language constructs. These allow you to use regular Python operators with the class-level attributes on your mapped class:
... print name
SELECT users.name AS users_name FROM users
WHERE users.fullname = ?
- ['Ed Jones']
+ ('Ed Jones',)
{stop}ed
The :class:`~sqlalchemy.orm.query.Query` object is fully *generative*, meaning that most method calls return a new :class:`~sqlalchemy.orm.query.Query` object upon which further criteria may be added. For example, to query for users named "ed" with a full name of "Ed Jones", you can call ``filter()`` twice, which joins criteria using ``AND``:
SELECT users.id AS users_id, users.name AS users_name, users.fullname AS users_fullname, users.password AS users_password
FROM users
WHERE users.name = ? AND users.fullname = ?
- ['ed', 'Ed Jones']
+ ('ed', 'Ed Jones')
{stop}<User('ed','Ed Jones', 'f8s7ccs')>
SELECT users.id AS users_id, users.name AS users_name, users.fullname AS users_fullname, users.password AS users_password
FROM users
WHERE users.name LIKE ? ORDER BY users.id
- ['%ed']
+ ('%ed',)
{stop}[<User('ed','Ed Jones', 'f8s7ccs')>, <User('fred','Fred Flinstone', 'blah')>]
:meth:`~sqlalchemy.orm.query.Query.first()` applies a limit of one and returns the first result as a scalar:
FROM users
WHERE users.name LIKE ? ORDER BY users.id
LIMIT 1 OFFSET 0
- ['%ed']
+ ('%ed',)
{stop}<User('ed','Ed Jones', 'f8s7ccs')>
:meth:`~sqlalchemy.orm.query.Query.one()`, fully fetches all rows, and if not exactly one object identity or composite row is present in the result, raises an error:
SELECT users.id AS users_id, users.name AS users_name, users.fullname AS users_fullname, users.password AS users_password
FROM users
WHERE users.name LIKE ? ORDER BY users.id
- ['%ed']
+ ('%ed',)
{stop}Multiple rows were found for one()
.. sourcecode:: python+sql
SELECT users.id AS users_id, users.name AS users_name, users.fullname AS users_fullname, users.password AS users_password
FROM users
WHERE users.name LIKE ? AND users.id = ? ORDER BY users.id
- ['%ed', 99]
+ ('%ed', 99)
{stop}No row was found for one()
Using Literal SQL
SELECT users.id AS users_id, users.name AS users_name, users.fullname AS users_fullname, users.password AS users_password
FROM users
WHERE id<224 ORDER BY id
- []
+ ()
{stop}ed
wendy
mary
SELECT users.id AS users_id, users.name AS users_name, users.fullname AS users_fullname, users.password AS users_password
FROM users
WHERE id<? and name=? ORDER BY users.id
- [224, 'fred']
+ (224, 'fred')
{stop}<User('fred','Fred Flinstone', 'blah')>
To use an entirely string-based statement, using :meth:`~sqlalchemy.orm.query.Query.from_statement()`; just ensure that the columns clause of the statement contains the column names normally used by the mapper (below illustrated using an asterisk):
{sql}>>> session.query(User).from_statement("SELECT * FROM users where name=:name").params(name='ed').all()
SELECT * FROM users where name=?
- ['ed']
+ ('ed',)
{stop}[<User('ed','Ed Jones', 'f8s7ccs')>]
You can use :meth:`~sqlalchemy.orm.query.Query.from_statement()` to go completely "raw", using string names to identify desired columns:
{sql}>>> session.query("id", "name", "thenumber12").from_statement("SELECT id, name, 12 as thenumber12 FROM users where name=:name").params(name='ed').all()
SELECT id, name, 12 as thenumber12 FROM users where name=?
- ['ed']
+ ('ed',)
{stop}[(1, u'ed', 12)]
Counting
.. sourcecode:: python+sql
- {sql}>>> session.query(User).filter(User.name.like('%ed')).count()
+ {sql}>>> session.query(User).filter(User.name.like('%ed')).count() #doctest: +NORMALIZE_WHITESPACE
SELECT count(1) AS count_1
FROM users
WHERE users.name LIKE ?
- ['%ed']
+ ('%ed',)
{stop}2
The :meth:`~sqlalchemy.orm.query.Query.count()` method is used to determine how many rows the SQL statement would return, and is mainly intended to return a simple count of a single type of entity, in this case ``User``. For more complicated sets of columns or entities where the "thing to be counted" needs to be indicated more specifically, :meth:`~sqlalchemy.orm.query.Query.count()` is probably not what you want. Below, a query for individual columns does return the expected result:
.. sourcecode:: python+sql
- {sql}>>> session.query(User.id, User.name).filter(User.name.like('%ed')).count()
+ {sql}>>> session.query(User.id, User.name).filter(User.name.like('%ed')).count() #doctest: +NORMALIZE_WHITESPACE
SELECT count(1) AS count_1
FROM (SELECT users.id AS users_id, users.name AS users_name
FROM users
WHERE users.name LIKE ?) AS anon_1
- ['%ed']
+ ('%ed',)
{stop}2
...but if you look at the generated SQL, SQLAlchemy saw that we were placing individual column expressions and decided to wrap whatever it was we were doing in a subquery, so as to be assured that it returns the "number of rows". This defensive behavior is not really needed here and in other cases is not what we want at all, such as if we wanted a grouping of counts per name:
.. sourcecode:: python+sql
- {sql}>>> session.query(User.name).group_by(User.name).count()
+ {sql}>>> session.query(User.name).group_by(User.name).count() #doctest: +NORMALIZE_WHITESPACE
SELECT count(1) AS count_1
FROM (SELECT users.name AS users_name
FROM users GROUP BY users.name) AS anon_1
- []
+ ()
{stop}4
We don't want the number ``4``, we wanted some rows back. So for detailed queries where you need to count something specific, use the ``func.count()`` function as a column expression:
.. sourcecode:: python+sql
>>> from sqlalchemy import func
- {sql}>>> session.query(func.count(User.name), User.name).group_by(User.name).all()
+ {sql}>>> session.query(func.count(User.name), User.name).group_by(User.name).all() #doctest: +NORMALIZE_WHITESPACE
SELECT count(users.name) AS count_1, users.name AS users_name
FROM users GROUP BY users.name
- {stop}[]
+ {stop}()
[(1, u'ed'), (1, u'fred'), (1, u'mary'), (1, u'wendy')]
Building a Relation
>>> session.add(jack)
{sql}>>> session.commit()
INSERT INTO users (name, fullname, password) VALUES (?, ?, ?)
- ['jack', 'Jack Bean', 'gjffdd']
+ ('jack', 'Jack Bean', 'gjffdd')
INSERT INTO addresses (email_address, user_id) VALUES (?, ?)
- ['jack@google.com', 5]
+ ('jack@google.com', 5)
INSERT INTO addresses (email_address, user_id) VALUES (?, ?)
- ['j25@yahoo.com', 5]
+ ('j25@yahoo.com', 5)
COMMIT
Querying for Jack, we get just Jack back. No SQL is yet issued for Jack's addresses:
SELECT users.id AS users_id, users.name AS users_name, users.fullname AS users_fullname, users.password AS users_password
FROM users
WHERE users.name = ?
- ['jack']
+ ('jack',)
{stop}>>> jack
<User('jack','Jack Bean', 'gjffdd')>
SELECT addresses.id AS addresses_id, addresses.email_address AS addresses_email_address, addresses.user_id AS addresses_user_id
FROM addresses
WHERE ? = addresses.user_id ORDER BY addresses.id
- [5]
+ (5,)
{stop}[<Address('jack@google.com')>, <Address('j25@yahoo.com')>]
When we accessed the ``addresses`` collection, SQL was suddenly issued. This is an example of a **lazy loading relation**. The ``addresses`` collection is now loaded and behaves just like an ordinary list.
AS addresses_1_email_address, addresses_1.user_id AS addresses_1_user_id
FROM users LEFT OUTER JOIN addresses AS addresses_1 ON users.id = addresses_1.user_id
WHERE users.name = ? ORDER BY addresses_1.id
- ['jack']
+ ('jack',)
{stop}>>> jack
<User('jack','Jack Bean', 'gjffdd')>
addresses.email_address AS addresses_email_address, addresses.user_id AS addresses_user_id
FROM users, addresses
WHERE users.id = addresses.user_id AND addresses.email_address = ?
- ['jack@google.com']
+ ('jack@google.com',)
{stop}<User('jack','Jack Bean', 'gjffdd')> <Address('jack@google.com')>
Or we can make a real JOIN construct; one way to do so is to use the ORM :func:`~sqlalchemy.orm.join()` function, and tell :class:`~sqlalchemy.orm.query.Query` to "select from" this join:
SELECT users.id AS users_id, users.name AS users_name, users.fullname AS users_fullname, users.password AS users_password
FROM users JOIN addresses ON users.id = addresses.user_id
WHERE addresses.email_address = ?
- ['jack@google.com']
+ ('jack@google.com',)
{stop}[<User('jack','Jack Bean', 'gjffdd')>]
:func:`~sqlalchemy.orm.join()` knows how to join between ``User`` and ``Address`` because there's only one foreign key between them. If there were no foreign keys, or several, :func:`~sqlalchemy.orm.join()` would require a third argument indicating the ON clause of the join, in one of the following forms:
SELECT users.id AS users_id, users.name AS users_name, users.fullname AS users_fullname, users.password AS users_password
FROM users JOIN addresses ON users.id = addresses.user_id
WHERE addresses.email_address = ?
- ['jack@google.com']
+ ('jack@google.com',)
{stop}[<User('jack','Jack Bean', 'gjffdd')>]
To explicitly specify the target of the join, use tuples to form an argument list similar to the standalone join. This becomes more important when using aliases and similar constructs:
FROM users JOIN addresses AS addresses_1 ON users.id = addresses_1.user_id
JOIN addresses AS addresses_2 ON users.id = addresses_2.user_id
WHERE addresses_1.email_address = ? AND addresses_2.email_address = ?
- ['jack@google.com', 'j25@yahoo.com']
+ ('jack@google.com', 'j25@yahoo.com')
{stop}jack jack@google.com j25@yahoo.com
Using Subqueries
FROM users LEFT OUTER JOIN (SELECT addresses.user_id AS user_id, count(?) AS address_count
FROM addresses GROUP BY addresses.user_id) AS anon_1 ON users.id = anon_1.user_id
ORDER BY users.id
- ['*']
+ ('*',)
{stop}<User('ed','Ed Jones', 'f8s7ccs')> None
<User('wendy','Wendy Williams', 'foobar')> None
<User('mary','Mary Contrary', 'xxg527')> None
FROM users JOIN (SELECT addresses.id AS id, addresses.email_address AS email_address, addresses.user_id AS user_id
FROM addresses
WHERE addresses.email_address != ?) AS anon_1 ON users.id = anon_1.user_id
- ['j25@yahoo.com']
+ ('j25@yahoo.com',)
{stop}<User('jack','Jack Bean', 'gjffdd')> <Address('jack@google.com')>
Using EXISTS
WHERE EXISTS (SELECT *
FROM addresses
WHERE addresses.user_id = users.id)
- []
+ ()
{stop}jack
The :class:`~sqlalchemy.orm.query.Query` features several operators which make usage of EXISTS automatically. Above, the statement can be expressed along the ``User.addresses`` relation using ``any()``:
WHERE EXISTS (SELECT 1
FROM addresses
WHERE users.id = addresses.user_id)
- []
+ ()
{stop}jack
``any()`` takes criterion as well, to limit the rows matched:
WHERE EXISTS (SELECT 1
FROM addresses
WHERE users.id = addresses.user_id AND addresses.email_address LIKE ?)
- ['%google%']
+ ('%google%',)
{stop}jack
``has()`` is the same operator as ``any()`` for many-to-one relations (note the ``~`` operator here too, which means "NOT"):
WHERE NOT (EXISTS (SELECT 1
FROM users
WHERE users.id = addresses.user_id AND users.name = ?))
- ['jack']
+ ('jack',)
{stop}[]
Common Relation Operators
>>> session.delete(jack)
{sql}>>> session.query(User).filter_by(name='jack').count() # doctest: +NORMALIZE_WHITESPACE
UPDATE addresses SET user_id=? WHERE addresses.id = ?
- [None, 1]
+ (None, 1)
UPDATE addresses SET user_id=? WHERE addresses.id = ?
- [None, 2]
+ (None, 2)
DELETE FROM users WHERE users.id = ?
- [5]
+ (5,)
SELECT count(1) AS count_1
FROM users
WHERE users.name = ?
- ['jack']
+ ('jack',)
{stop}0
So far, so good. How about Jack's ``Address`` objects ?
SELECT count(1) AS count_1
FROM addresses
WHERE addresses.email_address IN (?, ?)
- ['jack@google.com', 'j25@yahoo.com']
+ ('jack@google.com', 'j25@yahoo.com')
{stop}2
Uh oh, they're still there ! Analyzing the flush SQL, we can see that the ``user_id`` column of each address was set to NULL, but the rows weren't deleted. SQLAlchemy doesn't assume that deletes cascade, you have to tell it to do so.
SELECT users.id AS users_id, users.name AS users_name, users.fullname AS users_fullname, users.password AS users_password
FROM users
WHERE users.id = ?
- [5]
+ (5,)
{stop}
# remove one Address (lazy load fires off)
SELECT addresses.id AS addresses_id, addresses.email_address AS addresses_email_address, addresses.user_id AS addresses_user_id
FROM addresses
WHERE ? = addresses.user_id
- [5]
+ (5,)
{stop}
# only one address remains
... Address.email_address.in_(['jack@google.com', 'j25@yahoo.com'])
... ).count() # doctest: +NORMALIZE_WHITESPACE
DELETE FROM addresses WHERE addresses.id = ?
- [2]
+ (2,)
SELECT count(1) AS count_1
FROM addresses
WHERE addresses.email_address IN (?, ?)
- ['jack@google.com', 'j25@yahoo.com']
+ ('jack@google.com', 'j25@yahoo.com')
{stop}1
Deleting Jack will delete both Jack and his remaining ``Address``:
{sql}>>> session.query(User).filter_by(name='jack').count() # doctest: +NORMALIZE_WHITESPACE
DELETE FROM addresses WHERE addresses.id = ?
- [1]
+ (1,)
DELETE FROM users WHERE users.id = ?
- [5]
+ (5,)
SELECT count(1) AS count_1
FROM users
WHERE users.name = ?
- ['jack']
+ ('jack',)
{stop}0
{sql}>>> session.query(Address).filter(
SELECT count(1) AS count_1
FROM addresses
WHERE addresses.email_address IN (?, ?)
- ['jack@google.com', 'j25@yahoo.com']
+ ('jack@google.com', 'j25@yahoo.com')
{stop}0
Building a Many To Many Relation
SELECT users.id AS users_id, users.name AS users_name, users.fullname AS users_fullname, users.password AS users_password
FROM users
WHERE users.name = ?
- ['wendy']
+ ('wendy',)
{stop}
>>> post = BlogPost("Wendy's Blog Post", "This is a test", wendy)
>>> session.add(post)
{sql}>>> session.query(BlogPost).filter(BlogPost.keywords.any(keyword='firstpost')).all() #doctest: +NORMALIZE_WHITESPACE
INSERT INTO keywords (keyword) VALUES (?)
- ['wendy']
+ ('wendy',)
INSERT INTO keywords (keyword) VALUES (?)
- ['firstpost']
+ ('firstpost',)
INSERT INTO posts (user_id, headline, body) VALUES (?, ?, ?)
- [2, "Wendy's Blog Post", 'This is a test']
+ (2, "Wendy's Blog Post", 'This is a test')
INSERT INTO post_keywords (post_id, keyword_id) VALUES (?, ?)
- [[1, 1], [1, 2]]
+ ((1, 1), (1, 2))
SELECT posts.id AS posts_id, posts.user_id AS posts_user_id, posts.headline AS posts_headline, posts.body AS posts_body
FROM posts
WHERE EXISTS (SELECT 1
FROM post_keywords, keywords
WHERE posts.id = post_keywords.post_id AND keywords.id = post_keywords.keyword_id AND keywords.keyword = ?)
- ['firstpost']
+ ('firstpost',)
{stop}[BlogPost("Wendy's Blog Post", 'This is a test', <User('wendy','Wendy Williams', 'foobar')>)]
If we want to look up just Wendy's posts, we can tell the query to narrow down to her as a parent:
WHERE ? = posts.user_id AND (EXISTS (SELECT 1
FROM post_keywords, keywords
WHERE posts.id = post_keywords.post_id AND keywords.id = post_keywords.keyword_id AND keywords.keyword = ?))
- [2, 'firstpost']
+ (2, 'firstpost')
{stop}[BlogPost("Wendy's Blog Post", 'This is a test', <User('wendy','Wendy Williams', 'foobar')>)]
Or we can use Wendy's own ``posts`` relation, which is a "dynamic" relation, to query straight from there:
WHERE ? = posts.user_id AND (EXISTS (SELECT 1
FROM post_keywords, keywords
WHERE posts.id = post_keywords.post_id AND keywords.id = post_keywords.keyword_id AND keywords.keyword = ?))
- [2, 'firstpost']
+ (2, 'firstpost')
{stop}[BlogPost("Wendy's Blog Post", 'This is a test', <User('wendy','Wendy Williams', 'foobar')>)]
Further Reference
>>> result = conn.execute(ins)
{opensql}INSERT INTO users (name, fullname) VALUES (?, ?)
- ['jack', 'Jack Jones']
+ ('jack', 'Jack Jones')
COMMIT
So the INSERT statement was now issued to the database. Although we got positional "qmark" bind parameters instead of "named" bind parameters in the output. How come ? Because when executed, the :class:`~sqlalchemy.engine.base.Connection` used the SQLite **dialect** to help generate the statement; when we use the ``str()`` function, the statement isn't aware of this dialect, and falls back onto a default which uses named parameters. We can view this manually as follows:
>>> ins = users.insert()
>>> conn.execute(ins, id=2, name='wendy', fullname='Wendy Williams') # doctest: +ELLIPSIS
{opensql}INSERT INTO users (id, name, fullname) VALUES (?, ?, ?)
- [2, 'wendy', 'Wendy Williams']
+ (2, 'wendy', 'Wendy Williams')
COMMIT
{stop}<sqlalchemy.engine.base.ResultProxy object at 0x...>
... {'user_id': 2, 'email_address' : 'wendy@aol.com'},
... ])
{opensql}INSERT INTO addresses (user_id, email_address) VALUES (?, ?)
- [[1, 'jack@yahoo.com'], [1, 'jack@msn.com'], [2, 'www@www.org'], [2, 'wendy@aol.com']]
+ ((1, 'jack@yahoo.com'), (1, 'jack@msn.com'), (2, 'www@www.org'), (2, 'wendy@aol.com'))
COMMIT
{stop}<sqlalchemy.engine.base.ResultProxy object at 0x...>
{sql}>>> result = engine.execute(users.insert(), name='fred', fullname="Fred Flintstone")
INSERT INTO users (name, fullname) VALUES (?, ?)
- ['fred', 'Fred Flintstone']
+ ('fred', 'Fred Flintstone')
COMMIT
and you can save even more steps than that, if you connect the :class:`~sqlalchemy.engine.base.Engine` to the :class:`~sqlalchemy.schema.MetaData` object we created earlier. When this is done, all SQL expressions which involve tables within the :class:`~sqlalchemy.schema.MetaData` object will be automatically **bound** to the :class:`~sqlalchemy.engine.base.Engine`. In this case, we call it **implicit execution**:
>>> metadata.bind = engine
{sql}>>> result = users.insert().execute(name="mary", fullname="Mary Contrary")
INSERT INTO users (name, fullname) VALUES (?, ?)
- ['mary', 'Mary Contrary']
+ ('mary', 'Mary Contrary')
COMMIT
When the :class:`~sqlalchemy.schema.MetaData` is bound, statements will also compile against the engine's dialect. Since a lot of the examples here assume the default dialect, we'll detach the engine from the metadata which we just attached:
>>> from sqlalchemy.sql import select
>>> s = select([users])
- >>> result = conn.execute(s)
+ >>> result = conn.execute(s) # doctest: +NORMALIZE_WHITESPACE
{opensql}SELECT users.id, users.name, users.fullname
FROM users
- []
+ ()
Above, we issued a basic ``select()`` call, placing the ``users`` table within the COLUMNS clause of the select, and then executing. SQLAlchemy expanded the ``users`` table into the set of each of its columns, and also generated a FROM clause for us. The result returned is again a :class:`~sqlalchemy.engine.base.ResultProxy` object, which acts much like a DBAPI cursor, including methods such as :func:`~sqlalchemy.engine.base.ResultProxy.fetchone` and :func:`~sqlalchemy.engine.base.ResultProxy.fetchall`. The easiest way to get rows from it is to just iterate:
.. sourcecode:: pycon+sql
- {sql}>>> result = conn.execute(s)
+ {sql}>>> result = conn.execute(s) # doctest: +NORMALIZE_WHITESPACE
SELECT users.id, users.name, users.fullname
FROM users
- []
+ ()
{stop}>>> row = result.fetchone()
>>> print "name:", row['name'], "; fullname:", row['fullname']
.. sourcecode:: pycon+sql
- {sql}>>> for row in conn.execute(s):
+ {sql}>>> for row in conn.execute(s): # doctest: +NORMALIZE_WHITESPACE
... print "name:", row[users.c.name], "; fullname:", row[users.c.fullname]
SELECT users.id, users.name, users.fullname
FROM users
- []
+ ()
{stop}name: jack ; fullname: Jack Jones
name: wendy ; fullname: Wendy Williams
name: fred ; fullname: Fred Flintstone
.. sourcecode:: pycon+sql
>>> s = select([users.c.name, users.c.fullname])
- {sql}>>> result = conn.execute(s)
+ {sql}>>> result = conn.execute(s) # doctest: +NORMALIZE_WHITESPACE
SELECT users.name, users.fullname
FROM users
- []
+ ()
{stop}>>> for row in result: #doctest: +NORMALIZE_WHITESPACE
... print row
(u'jack', u'Jack Jones')
.. sourcecode:: pycon+sql
{sql}>>> for row in conn.execute(select([users, addresses])):
- ... print row
+ ... print row # doctest: +NORMALIZE_WHITESPACE
SELECT users.id, users.name, users.fullname, addresses.id, addresses.user_id, addresses.email_address
FROM users, addresses
- []
+ ()
{stop}(1, u'jack', u'Jack Jones', 1, 1, u'jack@yahoo.com')
(1, u'jack', u'Jack Jones', 2, 1, u'jack@msn.com')
(1, u'jack', u'Jack Jones', 3, 2, u'www@www.org')
>>> s = select([users, addresses], users.c.id==addresses.c.user_id)
{sql}>>> for row in conn.execute(s):
- ... print row
+ ... print row # doctest: +NORMALIZE_WHITESPACE
SELECT users.id, users.name, users.fullname, addresses.id, addresses.user_id, addresses.email_address
FROM users, addresses
WHERE users.id = addresses.user_id
- []
+ ()
{stop}(1, u'jack', u'Jack Jones', 1, 1, u'jack@yahoo.com')
(1, u'jack', u'Jack Jones', 2, 1, u'jack@msn.com')
(2, u'wendy', u'Wendy Williams', 3, 2, u'www@www.org')
FROM users, addresses
WHERE users.id = addresses.user_id AND users.name BETWEEN ? AND ? AND
(addresses.email_address LIKE ? OR addresses.email_address LIKE ?)
- [', ', 'm', 'z', '%@aol.com', '%@msn.com']
+ (', ', 'm', 'z', '%@aol.com', '%@msn.com')
[(u'Wendy Williams, wendy@aol.com',)]
Once again, SQLAlchemy figured out the FROM clause for our statement. In fact it will determine the FROM clause based on all of its other bits; the columns clause, the where clause, and also some other elements which we haven't covered yet, which include ORDER BY, GROUP BY, and HAVING.
FROM users, addresses
WHERE users.id = addresses.user_id AND users.name BETWEEN ? AND ? AND
(addresses.email_address LIKE ? OR addresses.email_address LIKE ?)
- ['m', 'z', '%@aol.com', '%@msn.com']
+ ('m', 'z', '%@aol.com', '%@msn.com')
{stop}[(u'Wendy Williams, wendy@aol.com',)]
To gain a "hybrid" approach, the `select()` construct accepts strings for most of its arguments. Below we combine the usage of strings with our constructed ``select()`` object, by using the ``select()`` object to structure the statement, and strings to provide all the content within the structure. For this example, SQLAlchemy is not given any :class:`~sqlalchemy.schema.Column` or :class:`~sqlalchemy.schema.Table` objects in any of its expressions, so it cannot generate a FROM clause. So we also give it the ``from_obj`` keyword argument, which is a list of ``ClauseElements`` (or strings) to be placed within the FROM clause:
SELECT users.fullname || ', ' || addresses.email_address AS title
FROM users, addresses
WHERE users.id = addresses.user_id AND users.name BETWEEN 'm' AND 'z' AND (addresses.email_address LIKE ? OR addresses.email_address LIKE ?)
- ['%@aol.com', '%@msn.com']
+ ('%@aol.com', '%@msn.com')
{stop}[(u'Wendy Williams, wendy@aol.com',)]
Going from constructed SQL to text, we lose some capabilities. We lose the capability for SQLAlchemy to compile our expression to a specific target database; above, our expression won't work with MySQL since it has no ``||`` construct. It also becomes more tedious for SQLAlchemy to be made aware of the datatypes in use; for example, if our bind parameters required UTF-8 encoding before going in, or conversion from a Python ``datetime`` into a string (as is required with SQLite), we would have to add extra information to our ``text()`` construct. Similar issues arise on the result set side, where SQLAlchemy also performs type-specific data conversion in some cases; still more information can be added to ``text()`` to work around this. But what we really lose from our statement is the ability to manipulate it, transform it, and analyze it. These features are critical when using the ORM, which makes heavy usage of relational transformations. To show off what we mean, we'll first introduce the ALIAS construct and the JOIN construct, just so we have some juicier bits to play with.
... a1.c.email_address=='jack@msn.com',
... a2.c.email_address=='jack@yahoo.com'
... ))
- {sql}>>> print conn.execute(s).fetchall()
+ {sql}>>> print conn.execute(s).fetchall() # doctest: +NORMALIZE_WHITESPACE
SELECT users.id, users.name, users.fullname
FROM users, addresses AS a1, addresses AS a2
WHERE users.id = a1.user_id AND users.id = a2.user_id AND a1.email_address = ? AND a2.email_address = ?
- ['jack@msn.com', 'jack@yahoo.com']
+ ('jack@msn.com', 'jack@yahoo.com')
{stop}[(1, u'jack', u'Jack Jones')]
Easy enough. One thing that we're going for with the SQL Expression Language is the melding of programmatic behavior with SQL generation. Coming up with names like ``a1`` and ``a2`` is messy; we really didn't need to use those names anywhere, it's just the database that needed them. Plus, we might write some code that uses alias objects that came from several different places, and it's difficult to ensure that they all have unique names. So instead, we just let SQLAlchemy make the names for us, using "anonymous" aliases:
... a1.c.email_address=='jack@msn.com',
... a2.c.email_address=='jack@yahoo.com'
... ))
- {sql}>>> print conn.execute(s).fetchall()
+ {sql}>>> print conn.execute(s).fetchall() # doctest: +NORMALIZE_WHITESPACE
SELECT users.id, users.name, users.fullname
FROM users, addresses AS addresses_1, addresses AS addresses_2
WHERE users.id = addresses_1.user_id AND users.id = addresses_2.user_id AND addresses_1.email_address = ? AND addresses_2.email_address = ?
- ['jack@msn.com', 'jack@yahoo.com']
+ ('jack@msn.com', 'jack@yahoo.com')
{stop}[(1, u'jack', u'Jack Jones')]
One super-huge advantage of anonymous aliases is that not only did we not have to guess up a random name, but we can also be guaranteed that the above SQL string is **deterministically** generated to be the same every time. This is important for databases such as Oracle which cache compiled "query plans" for their statements, and need to see the same SQL string in order to make use of it.
>>> a1 = s.correlate(None).alias()
>>> s = select([users.c.name], users.c.id==a1.c.id)
- {sql}>>> print conn.execute(s).fetchall()
+ {sql}>>> print conn.execute(s).fetchall() # doctest: +NORMALIZE_WHITESPACE
SELECT users.name
FROM users, (SELECT users.id AS id, users.name AS name, users.fullname AS fullname
FROM users, addresses AS addresses_1, addresses AS addresses_2
WHERE users.id = addresses_1.user_id AND users.id = addresses_2.user_id AND addresses_1.email_address = ? AND addresses_2.email_address = ?) AS anon_1
WHERE users.id = anon_1.id
- ['jack@msn.com', 'jack@yahoo.com']
+ ('jack@msn.com', 'jack@yahoo.com')
{stop}[(u'jack',)]
Using Joins
>>> s = select([users.c.fullname], from_obj=[
... users.join(addresses, addresses.c.email_address.like(users.c.name + '%'))
... ])
- {sql}>>> print conn.execute(s).fetchall()
+ {sql}>>> print conn.execute(s).fetchall() # doctest: +NORMALIZE_WHITESPACE
SELECT users.fullname
FROM users JOIN addresses ON addresses.email_address LIKE users.name || ?
- ['%']
+ ('%',)
{stop}[(u'Jack Jones',), (u'Jack Jones',), (u'Wendy Williams',)]
The ``outerjoin()`` function just creates ``LEFT OUTER JOIN`` constructs. It's used just like ``join()``:
.. sourcecode:: pycon+sql
>>> s = select([users.c.fullname], from_obj=[users.outerjoin(addresses)])
- >>> print s
+ >>> print s # doctest: +NORMALIZE_WHITESPACE
SELECT users.fullname
FROM users LEFT OUTER JOIN addresses ON users.id = addresses.user_id
.. sourcecode:: pycon+sql
>>> from sqlalchemy.dialects.oracle import dialect as OracleDialect
- >>> print s.compile(dialect=OracleDialect(use_ansi=False))
+ >>> print s.compile(dialect=OracleDialect(use_ansi=False)) # doctest: +NORMALIZE_WHITESPACE
SELECT users.fullname
FROM users, addresses
WHERE users.id = addresses.user_id(+)
.. sourcecode:: pycon+sql
>>> query = users.select()
- >>> print query
+ >>> print query # doctest: +NORMALIZE_WHITESPACE
SELECT users.id, users.name, users.fullname
FROM users
.. sourcecode:: pycon+sql
- >>> conn.execute(query).fetchall()
+ >>> conn.execute(query).fetchall() # doctest: +NORMALIZE_WHITESPACE
{opensql}SELECT users.id AS users_id, users.name AS users_name, users.fullname AS users_fullname, addresses.id AS addresses_id, addresses.user_id AS addresses_user_id, addresses.email_address AS addresses_email_address
FROM users LEFT OUTER JOIN addresses ON users.id = addresses.user_id
WHERE users.name = ? AND (EXISTS (SELECT addresses.id
FROM addresses
WHERE addresses.user_id = users.id AND addresses.email_address LIKE ?)) ORDER BY users.fullname DESC
- ['jack', '%@msn.com']
+ ('jack', '%@msn.com')
{stop}[(1, u'jack', u'Jack Jones', 1, 1, u'jack@yahoo.com'), (1, u'jack', u'Jack Jones', 2, 1, u'jack@msn.com')]
So we started small, added one little thing at a time, and at the end we have a huge statement..which actually works. Now let's do one more thing; the searching function wants to add another ``email_address`` criterion on, however it doesn't want to construct an alias of the ``addresses`` table; suppose many parts of the application are written to deal specifically with the ``addresses`` table, and to change all those functions to support receiving an arbitrary alias of the address would be cumbersome. We can actually *convert* the ``addresses`` table within the *existing* statement to be an alias of itself, using :func:`~sqlalchemy.sql.expression.FromClause.replace_selectable`:
>>> a1 = addresses.alias()
>>> query = query.replace_selectable(addresses, a1)
- >>> print query
+ >>> print query # doctest: +NORMALIZE_WHITESPACE
{opensql}SELECT users.id AS users_id, users.name AS users_name, users.fullname AS users_fullname, addresses_1.id AS addresses_1_id, addresses_1.user_id AS addresses_1_user_id, addresses_1.email_address AS addresses_1_email_address
FROM users LEFT OUTER JOIN addresses AS addresses_1 ON users.id = addresses_1.user_id
WHERE users.name = :name_1 AND (EXISTS (SELECT addresses_1.id
.. sourcecode:: pycon+sql
{sql}>>> for row in conn.execute(query):
- ... print "Name:", row[users.c.name], "; Email Address", row[a1.c.email_address]
+ ... print "Name:", row[users.c.name], "; Email Address", row[a1.c.email_address] # doctest: +NORMALIZE_WHITESPACE
SELECT users.id AS users_id, users.name AS users_name, users.fullname AS users_fullname, addresses_1.id AS addresses_1_id, addresses_1.user_id AS addresses_1_user_id, addresses_1.email_address AS addresses_1_email_address
FROM users LEFT OUTER JOIN addresses AS addresses_1 ON users.id = addresses_1.user_id
WHERE users.name = ? AND (EXISTS (SELECT addresses_1.id
FROM addresses AS addresses_1
WHERE addresses_1.user_id = users.id AND addresses_1.email_address LIKE ?)) ORDER BY users.fullname DESC
- ['jack', '%@msn.com']
+ ('jack', '%@msn.com')
{stop}Name: jack ; Email Address jack@yahoo.com
Name: jack ; Email Address jack@msn.com
>>> from sqlalchemy.sql import bindparam
>>> s = users.select(users.c.name==bindparam('username'))
- {sql}>>> conn.execute(s, username='wendy').fetchall()
+ {sql}>>> conn.execute(s, username='wendy').fetchall() # doctest: +NORMALIZE_WHITESPACE
SELECT users.id, users.name, users.fullname
FROM users
WHERE users.name = ?
- ['wendy']
+ ('wendy',)
{stop}[(2, u'wendy', u'Wendy Williams')]
Another important aspect of bind parameters is that they may be assigned a type. The type of the bind parameter will determine its behavior within expressions and also how the data bound to it is processed before being sent off to the database:
.. sourcecode:: pycon+sql
>>> s = users.select(users.c.name.like(bindparam('username', type_=String) + text("'%'")))
- {sql}>>> conn.execute(s, username='wendy').fetchall()
+ {sql}>>> conn.execute(s, username='wendy').fetchall() # doctest: +NORMALIZE_WHITESPACE
SELECT users.id, users.name, users.fullname
FROM users
WHERE users.name LIKE ? || '%'
- ['wendy']
+ ('wendy',)
{stop}[(2, u'wendy', u'Wendy Williams')]
... users.c.name.like(bindparam('name', type_=String) + text("'%'")) |
... addresses.c.email_address.like(bindparam('name', type_=String) + text("'@%'")),
... from_obj=[users.outerjoin(addresses)])
- {sql}>>> conn.execute(s, name='jack').fetchall()
+ {sql}>>> conn.execute(s, name='jack').fetchall() # doctest: +NORMALIZE_WHITESPACE
SELECT users.id, users.name, users.fullname, addresses.id, addresses.user_id, addresses.email_address
FROM users LEFT OUTER JOIN addresses ON users.id = addresses.user_id
WHERE users.name LIKE ? || '%' OR addresses.email_address LIKE ? || '@%'
- ['jack', 'jack']
+ ('jack', 'jack')
{stop}[(1, u'jack', u'Jack Jones', 1, 1, u'jack@yahoo.com'), (1, u'jack', u'Jack Jones', 2, 1, u'jack@msn.com')]
Functions
>>> print conn.execute(
... select([func.max(addresses.c.email_address, type_=String).label('maxemail')])
- ... ).scalar()
+ ... ).scalar() # doctest: +NORMALIZE_WHITESPACE
{opensql}SELECT max(addresses.email_address) AS maxemail
FROM addresses
- []
+ ()
{stop}www@www.org
Databases such as PostgreSQL and Oracle which support functions that return whole result sets can be assembled into selectable units, which can be used in statements. Such as, a database function ``calculate()`` which takes the parameters ``x`` and ``y``, and returns three columns which we'd like to name ``q``, ``z`` and ``r``, we can construct using "lexical" column objects as well as bind parameters:
>>> calculate = select([column('q'), column('z'), column('r')],
... from_obj=[func.calculate(bindparam('x'), bindparam('y'))])
- >>> print select([users], users.c.id > calculate.c.z)
+ >>> print select([users], users.c.id > calculate.c.z) # doctest: +NORMALIZE_WHITESPACE
SELECT users.id, users.name, users.fullname
FROM users, (SELECT q, z, r
FROM calculate(:x, :y))
... calculate.alias('c1').unique_params(x=17, y=45).c.z,
... calculate.alias('c2').unique_params(x=5, y=12).c.z))
- >>> print s
+ >>> print s # doctest: +NORMALIZE_WHITESPACE
SELECT users.id, users.name, users.fullname
FROM users, (SELECT q, z, r
FROM calculate(:x_1, :y_1)) AS c1, (SELECT q, z, r
... addresses.select(addresses.c.email_address.like('%@yahoo.com')),
... ).order_by(addresses.c.email_address)
- {sql}>>> print conn.execute(u).fetchall()
+ {sql}>>> print conn.execute(u).fetchall() # doctest: +NORMALIZE_WHITESPACE
SELECT addresses.id, addresses.user_id, addresses.email_address
FROM addresses
WHERE addresses.email_address = ? UNION SELECT addresses.id, addresses.user_id, addresses.email_address
FROM addresses
WHERE addresses.email_address LIKE ? ORDER BY addresses.email_address
- ['foo@bar.com', '%@yahoo.com']
+ ('foo@bar.com', '%@yahoo.com')
{stop}[(1, 1, u'jack@yahoo.com')]
Also available, though not supported on all databases, are ``intersect()``, ``intersect_all()``, ``except_()``, and ``except_all()``:
... addresses.select(addresses.c.email_address.like('%@msn.com'))
... )
- {sql}>>> print conn.execute(u).fetchall()
+ {sql}>>> print conn.execute(u).fetchall() # doctest: +NORMALIZE_WHITESPACE
SELECT addresses.id, addresses.user_id, addresses.email_address
FROM addresses
WHERE addresses.email_address LIKE ? EXCEPT SELECT addresses.id, addresses.user_id, addresses.email_address
FROM addresses
WHERE addresses.email_address LIKE ?
- ['%@%.com', '%@msn.com']
+ ('%@%.com', '%@msn.com')
{stop}[(1, 1, u'jack@yahoo.com'), (4, 2, u'wendy@aol.com')]
A common issue with so-called "compound" selectables arises due to the fact that they nest with parenthesis. SQLite in particular doesn't like a statement that starts with parenthesis. So when nesting a "compound" inside a "compound", it's often necessary to apply
SELECT addresses.id, addresses.user_id, addresses.email_address
FROM addresses
WHERE addresses.email_address LIKE ?
- ['%@yahoo.com', '%@msn.com', '%@msn.com']
+ ('%@yahoo.com', '%@msn.com', '%@msn.com')
{stop}[(1, 1, u'jack@yahoo.com')]
FROM addresses
WHERE users.id = addresses.user_id) AS anon_1
FROM users
- []
+ ()
{stop}[(u'jack', 2), (u'wendy', 2), (u'fred', 0), (u'mary', 0)]
Alternatively, applying a ``label()`` to a select evaluates it as a scalar as well:
FROM addresses
WHERE users.id = addresses.user_id) AS address_count
FROM users
- []
+ ()
{stop}[(u'jack', 2), (u'wendy', 2), (u'fred', 0), (u'mary', 0)]
Correlated Subqueries
Notice in the examples on "scalar selects", the FROM clause of each embedded select did not contain the ``users`` table in its FROM clause. This is because SQLAlchemy automatically attempts to correlate embedded FROM objects to that of an enclosing query. To disable this, or to specify explicit FROM clauses to be correlated, use ``correlate()``::
>>> s = select([users.c.name], users.c.id==select([users.c.id]).correlate(None))
- >>> print s
+ >>> print s # doctest: +NORMALIZE_WHITESPACE
SELECT users.name
FROM users
WHERE users.id = (SELECT users.id
>>> s = select([users.c.name, addresses.c.email_address], users.c.id==
... select([users.c.id], users.c.id==addresses.c.user_id).correlate(addresses)
... )
- >>> print s
+ >>> print s # doctest: +NORMALIZE_WHITESPACE
SELECT users.name, addresses.email_address
FROM users, addresses
WHERE users.id = (SELECT users.id
>>> s = select([addresses.c.user_id, func.count(addresses.c.id)]).\
... group_by(addresses.c.user_id).having(func.count(addresses.c.id)>1)
- {sql}>>> print conn.execute(s).fetchall()
+ {sql}>>> print conn.execute(s).fetchall() # doctest: +NORMALIZE_WHITESPACE
SELECT addresses.user_id, count(addresses.id) AS count_1
FROM addresses GROUP BY addresses.user_id
HAVING count(addresses.id) > ?
- [1]
+ (1,)
{stop}[(1, 2), (2, 2)]
>>> s = select([addresses.c.email_address, addresses.c.id]).distinct().\
... order_by(addresses.c.email_address.desc(), addresses.c.id)
- {sql}>>> conn.execute(s).fetchall()
+ {sql}>>> conn.execute(s).fetchall() # doctest: +NORMALIZE_WHITESPACE
SELECT DISTINCT addresses.email_address, addresses.id
FROM addresses ORDER BY addresses.email_address DESC, addresses.id
- []
+ ()
{stop}[(u'www@www.org', 3), (u'wendy@aol.com', 4), (u'jack@yahoo.com', 1), (u'jack@msn.com', 2)]
>>> s = select([addresses]).offset(1).limit(1)
SELECT addresses.id, addresses.user_id, addresses.email_address
FROM addresses
LIMIT 1 OFFSET 1
- []
+ ()
{stop}[(2, 1, u'jack@msn.com')]
Updates
>>> # change 'jack' to 'ed'
{sql}>>> conn.execute(users.update().where(users.c.name=='jack').values(name='ed')) #doctest: +ELLIPSIS
UPDATE users SET name=? WHERE users.name = ?
- ['ed', 'jack']
+ ('ed', 'jack')
COMMIT
{stop}<sqlalchemy.engine.base.ResultProxy object at 0x...>
>>> u = users.update().where(users.c.name==bindparam('oldname')).values(name=bindparam('newname'))
{sql}>>> conn.execute(u, oldname='jack', newname='ed') #doctest: +ELLIPSIS
UPDATE users SET name=? WHERE users.name = ?
- ['ed', 'jack']
+ ('ed', 'jack')
COMMIT
{stop}<sqlalchemy.engine.base.ResultProxy object at 0x...>
>>> # update a column to an expression.:
{sql}>>> conn.execute(users.update().values(fullname="Fullname: " + users.c.name)) #doctest: +ELLIPSIS
UPDATE users SET fullname=(? || users.name)
- ['Fullname: ']
+ ('Fullname: ',)
COMMIT
{stop}<sqlalchemy.engine.base.ResultProxy object at 0x...>
FROM addresses
WHERE addresses.user_id = users.id
LIMIT 1 OFFSET 0)
- []
+ ()
COMMIT
{stop}<sqlalchemy.engine.base.ResultProxy object at 0x...>
{sql}>>> conn.execute(addresses.delete()) #doctest: +ELLIPSIS
DELETE FROM addresses
- []
+ ()
COMMIT
{stop}<sqlalchemy.engine.base.ResultProxy object at 0x...>
{sql}>>> conn.execute(users.delete().where(users.c.name > 'm')) #doctest: +ELLIPSIS
DELETE FROM users WHERE users.name > ?
- ['m']
+ ('m',)
COMMIT
{stop}<sqlalchemy.engine.base.ResultProxy object at 0x...>
projects / thirdparty / sqlalchemy / sqlalchemy.git / commitdiff
