Current Migration Guide
-----------------------
+For SQLAlchemy 2.0, there are two separate documents; the "Major Migration
+Guide" details how to update a SQLAlchemy 1.4 application to be compatible
+under SQLAlchemy 2.0. The "What's New?" document details major new features,
+capabilities and behaviors in SQLAlchemy 2.0.
+
.. toctree::
:titlesonly:
migration_20
+ whatsnew_20
Change logs
-----------
.. _migration_20_toplevel:
-=============================
-What's New in SQLAlchemy 2.0?
-=============================
+======================================
+SQLAlchemy 2.0 - Major Migration Guide
+======================================
+
+.. admonition:: Note for Readers
+
+ SQLAlchemy 2.0's transition documents are separated into **two**
+ documents - one which details major API shifts from the 1.x to 2.x
+ series, and the other which details new features and behaviors relative
+ to SQLAlchemy 1.4:
+
+ * :ref:`migration_20_toplevel` - this document, 1.x to 2.x API shifts
+ * :ref:`whatsnew_20_toplevel` - new features and behaviors for SQLAlchemy 2.0
+
+ Readers who have already updated their 1.4 application to follow
+ SQLAlchemy 2.0 engine and ORM conventions may navigate to
+ :ref:`whatsnew_20_toplevel` for an overview of new features and
+ capabilities.
.. admonition:: About this document
--- /dev/null
+.. _whatsnew_20_toplevel:
+
+=============================
+What's New in SQLAlchemy 2.0?
+=============================
+
+.. admonition:: Note for Readers
+
+ SQLAlchemy 2.0's transition documents are separated into **two**
+ documents - one which details major API shifts from the 1.x to 2.x
+ series, and the other which details new features and behaviors relative
+ to SQLAlchemy 1.4:
+
+ * :ref:`migration_20_toplevel` - 1.x to 2.x API shifts
+ * :ref:`whatsnew_20_toplevel` - this document, new features and behaviors for SQLAlchemy 2.0
+
+ Readers who have not yet updated their 1.4 application to follow
+ SQLAlchemy 2.0 engine and ORM conventions may navigate to
+ :ref:`migration_20_toplevel` for a guide to ensuring SQLAlchemy 2.0
+ compatibility, which is a prerequisite for having working code under
+ version 2.0.
+
+
+.. admonition:: About this Document
+
+ This document describes changes between SQLAlchemy version 1.4
+ and SQLAlchemy version 2.0, **independent** of the major changes between
+ :term:`1.x style` and :term:`2.0 style` usage. Readers should start
+ with the :ref:`migration_20_toplevel` document to get an overall picture
+ of the major compatibility changes between the 1.x and 2.x series.
+
+ Aside from the major 1.x->2.x migration path, the next largest
+ paradigm shift in SQLAlchemy 2.0 is deep integration with :pep:`484` typing
+ practices and current capabilities, particularly within the ORM. New
+ type-driven ORM declarative styles inspired by Python dataclasses_, as well
+ as new integrations with dataclasses themselves, complement an overall
+ approach that no longer requires stubs and also goes very far towards
+ providing a type-aware method chain from SQL statement to result set.
+
+ The prominence of Python typing is significant not only so that type checkers
+ like mypy_ can run without plugins; more significantly it allows IDEs
+ like vscode_ and pycharm_ to take a much more active role in assisting
+ with the composition of a SQLAlchemy application.
+
+
+.. _typeshed: https://github.com/python/typeshed
+
+.. _dataclasses: https://docs.python.org/3/library/dataclasses.html
+
+.. _mypy: https://mypy.readthedocs.io/en/stable/
+
+.. _vscode: https://code.visualstudio.com/
+
+.. _pylance: https://github.com/microsoft/pylance-release
+
+.. _pycharm: https://www.jetbrains.com/pycharm/
+
+
+New Typing Support in Core and ORM - Stubs / Extensions no longer used
+=======================================================================
+
+
+The approach to typing for Core and ORM has been completely reworked, compared
+to the interim approach that was provided in version 1.4 via the
+sqlalchemy2-stubs_ package. The new approach begins at the most fundamental
+element in SQLAlchemy which is the :class:`_schema.Column`, or more
+accurately the :class:`.ColumnElement` that underlies all SQL
+expressions that have a type. This expression-level typing then extends into the area of
+statement construction, statement execution, and result sets, and finally into the ORM
+where new :ref:`declarative <orm_declarative_mapper_config_toplevel>` forms allow
+for fully typed ORM models that integrate all the way from statement to
+result set.
+
+SQL Expression / Statement / Result Set Typing
+----------------------------------------------
+
+This section provides background and examples for SQLAlchemy's new
+SQL expression typing approach, which extends from base :class:`.ColumnElement`
+constructs through SQL statements and result sets and into realm of ORM mapping.
+
+Rationale and Overview
+^^^^^^^^^^^^^^^^^^^^^^
+
+.. tip::
+
+ This section is an architectural discussion. Skip ahead to
+ :ref:`whatsnew_20_expression_typing_examples` to just see what the new typing
+ looks like.
+
+In sqlalchemy2-stubs_, SQL expressions were typed as generics_ that then
+referred to a :class:`.TypeEngine` object such as :class:`.Integer`,
+:class:`.DateTime`, or :class:`.String` as their generic argument
+(such as ``Column[Integer]``). This was itself a departure from what
+the original Dropbox sqlalchemy-stubs_ package did, where
+:class:`.Column` and its foundational constructs were directly generic on
+Python types, such as ``int``, ``datetime`` and ``str``. It was hoped
+that since :class:`.Integer` / :class:`.DateTime` / :class:`.String` themselves
+are generic against ``int`` / ``datetime`` / ``str``, there would be ways
+to maintain both levels of information and to be able to extract the Python
+type from a column expression via the :class:`.TypeEngine` as an intermediary
+construct. However, this is not the case, as :pep:`484`
+doesn't really have a rich enough feature set for this to be viable,
+lacking capabilities such as
+`higher kinded TypeVars <https://github.com/python/typing/issues/548>`_.
+
+So after a `deep assessment <https://github.com/python/typing/discussions/999>`_
+of the current capabilities of :pep:`484`, SQLAlchemy 2.0 has realized the
+original wisdom of sqlalchemy-stubs_ in this area and returned to linking
+column expressions directly to Python types. This does mean that if one
+has SQL expressions to different subtypes, like ``Column(VARCHAR)`` vs.
+``Column(Unicode)``, the specifics of those two :class:`.String` subtypes
+is not carried along as the type only carries along ``str``,
+but in practice this is usually not an issue and it is generally vastly more
+useful that the Python type is immediately present, as it represents the
+in-Python data one will be storing and receiving for this column directly.
+
+Concretely, this means that an expression like ``Column('id', Integer)``
+is typed as ``Column[int]``. This allows for a viable pipeline of
+SQLAlchemy construct -> Python datatype to be set up, without the need for
+typing plugins. Crucially, it allows full interoperability with
+the ORM's paradigm of using :func:`_sql.select` and :class:`_engine.Row`
+constructs that reference ORM mapped class types (e.g. a :class:`_engine.Row`
+containing instances of user-mapped instances, such as the ``User`` and
+``Address`` examples used in our tutorials). While Python typing currently has very limited
+support for customization of tuple-types (where :pep:`646`, the first pep that
+attempts to deal with tuple-like objects, was `intentionally limited
+in its functionality <https://mail.python.org/archives/list/typing-sig@python.org/message/G2PNHRR32JMFD3JR7ACA2NDKWTDSEPUG/>`_
+and by itself is not yet viable for arbitrary tuple
+manipulation),
+a fairly decent approach has been devised that allows for basic
+:func:`_sql.select()` -> :class:`_engine.Result` -> :class:`_engine.Row` typing
+to function, including for ORM classes, where at the point at which a
+:class:`_engine.Row` object is to be unpacked into individual column entries,
+a small typing-oriented accessor is added that allows the individual Python
+values to maintain the Python type linked to the SQL expression from which
+they originated (translation: it works).
+
+.. _sqlalchemy-stubs: https://github.com/dropbox/sqlalchemy-stubs
+
+.. _sqlalchemy2-stubs: https://github.com/sqlalchemy/sqlalchemy2-stubs
+
+.. _generics: https://peps.python.org/pep-0484/#generics
+
+.. _whatsnew_20_expression_typing_examples:
+
+SQL Expression Typing - Examples
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+A brief tour of typing behaviors. Comments
+indicate what one would see hovering over the code in vscode_ (or roughly
+what typing tools would display when using the `reveal_type() <https://mypy.readthedocs.io/en/latest/common_issues.html?highlight=reveal_type#reveal-type>`_
+helper):
+
+* Simple Python Types Assigned to SQL Expressions
+
+ ::
+
+ # (variable) str_col: ColumnClause[str]
+ str_col = column("a", String)
+
+ # (variable) int_col: ColumnClause[int]
+ int_col = column("a", Integer)
+
+ # (variable) expr1: ColumnElement[str]
+ expr1 = str_col + "x"
+
+ # (variable) expr2: ColumnElement[int]
+ expr2 = int_col + 10
+
+ # (variable) expr3: ColumnElement[bool]
+ expr3 = int_col == 15
+
+* Individual SQL expressions assigned to :func:`_sql.select` constructs, as well as any
+ row-returning construct, including row-returning DML
+ such as :class:`_sql.Insert` with :meth:`_sql.Insert.returning`, are packed
+ into a ``Tuple[]`` type which retains the Python type for each element.
+
+ ::
+
+ # (variable) stmt: Select[Tuple[str, int]]
+ stmt = select(str_col, int_col)
+
+ # (variable) stmt: ReturningInsert[Tuple[str, int]]
+ ins_stmt = insert(table('t')).returning(str_col, int_col)
+
+* The ``Tuple[]`` type from any row returning construct, when invoked with an
+ ``.execute()`` method, carries through to :class:`_engine.Result`
+ and :class:`_engine.Row`. In order to unpack the :class:`_engine.Row`
+ object as a tuple, the :meth:`_engine.Row.tuple` or :attr:`_engine.Row.t`
+ accessor essentially casts the :class:`_engine.Row` into the corresponding
+ ``Tuple[]`` (though remains the same :class:`_engine.Row` object at runtime).
+
+ ::
+
+ with engine.connect() as conn:
+
+ # (variable) stmt: Select[Tuple[str, int]]
+ stmt = select(str_col, int_col)
+
+ # (variable) result: Result[Tuple[str, int]]
+ result = conn.execute(stmt)
+
+ # (variable) row: Row[Tuple[str, int]] | None
+ row = result.first()
+
+ if row is not None:
+ # for typed tuple unpacking or indexed access,
+ # use row.tuple() or row.t (this is the small typing-oriented accessor)
+ strval, intval = row.t
+
+ # (variable) strval: str
+ strval
+
+ # (variable) intval: int
+ intval
+
+* Scalar values for single-column statements do the right thing with
+ methods like :meth:`_engine.Connection.scalar`, :meth:`_engine.Result.scalars`,
+ etc.
+
+ ::
+
+ # (variable) data: Sequence[str]
+ data = connection.execute(select(str_col)).scalars().all()
+
+* The above support for row-returning constructs works the best with
+ ORM mapped classes, as a mapped class can list out specific types
+ for its members. The example below sets up a class using
+ :ref:`new type-aware syntaxes <whatsnew_20_orm_declarative_typing>`,
+ described in the following section::
+
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import mapped_column
+
+
+ class Base(DeclarativeBase):
+ pass
+
+
+ class User(Base):
+ __tablename__ = 'user_account'
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+ name: Mapped[str]
+ addresses: Mapped[List["Address"]] = relationship()
+
+ class Address(Base):
+ __tablename__ = "address"
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+ email_address: Mapped[str]
+ user_id = mapped_column(ForeignKey("user_account.id"))
+
+
+ With the above mapping, the attributes are typed and express themselves
+ all the way from statement to result set::
+
+ with Session(engine) as session:
+
+ # (variable) stmt: Select[Tuple[int, str]]
+ stmt_1 = select(User.id, User.name)
+
+ # (variable) result_1: Result[Tuple[int, str]]
+ result_1 = session.execute(stmt_1)
+
+ # (variable) intval: int
+ # (variable) strval: str
+ intval, strval = result_1.one().t
+
+ Mapped classes themselves are also types, and behave the same way, such
+ as a SELECT against two mapped classes::
+
+ with Session(engine) as session:
+
+ # (variable) stmt: Select[Tuple[User, Address]]
+ stmt_2 = select(User, Address).join_from(User, Address)
+
+ # (variable) result_2: Result[Tuple[User, Address]]
+ result_2 = session.execute(stmt_2)
+
+ # (variable) user_obj: User
+ # (variable) address_obj: Address
+ user_obj, address_obj = result_2.one().t
+
+ When selecting mapped classes, constructs like :class:`_orm.aliased` work
+ as well, maintaining the column-level attributes of the original mapped
+ class as well as the return type expected from a statement::
+
+ with Session(engine) as session:
+
+ # this is in fact an Annotated type, but typing tools don't
+ # generally display this
+
+ # (variable) u1: Type[User]
+ u1 = aliased(User)
+
+ # (variable) stmt: Select[Tuple[User, User, str]]
+ stmt = select(User, u1, User.name).filter(User.id == 5)
+
+ # (variable) result: Result[Tuple[User, User, str]]
+ result = session.execute(stmt)
+
+* Core Table does not yet have a decent way to maintain typing of
+ :class:`_schema.Column` objects when accessing them via the :attr:`.Table.c` accessor.
+
+ Since :class:`.Table` is set up as an instance of a class, and the
+ :attr:`.Table.c` accessor typically accesses :class:`.Column` objects
+ dynamically by name, there's not yet an established typing approach for this; some
+ alternative syntax would be needed.
+
+* ORM classes, scalars, etc. work great.
+
+ The typical use case of selecting ORM classes, as scalars or tuples,
+ all works, both 2.0 and 1.x style queries, getting back the exact type
+ either by itself or contained within the appropriate container such
+ as ``Sequence[]``, ``List[]`` or ``Iterator[]``::
+
+ # (variable) users1: Sequence[User]
+ users1 = session.scalars(select(User)).all()
+
+ # (variable) user: User
+ user = session.query(User).one()
+
+ # (variable) user_iter: Iterator[User]
+ user_iter = iter(session.scalars(select(User)))
+
+* Legacy :class:`_orm.Query` gains tuple typing as well.
+
+ The typing support for :class:`_orm.Query` goes well beyond what
+ sqlalchemy-stubs_ or sqlalchemy2-stubs_ offered, where both scalar-object
+ as well as tuple-typed :class:`_orm.Query` objects will retain result level
+ typing for most cases::
+
+ # (variable) q1: RowReturningQuery[Tuple[int, str]]
+ q1 = session.query(User.id, User.name)
+
+ # (variable) rows: List[Row[Tuple[int, str]]]
+ rows = q1.all()
+
+ # (variable) q2: Query[User]
+ q2 = session.query(User)
+
+ # (variable) users: List[User]
+ users = q2.all()
+
+the catch - all stubs must be uninstalled
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+A key caveat with the typing support is that **all SQLAlchemy stubs packages
+must be uninstalled** for typing to work. When running mypy_ against a
+Python virtualenv, this is only a matter of uninstalling those packages.
+However, a SQLAlchemy stubs package is also currently part of typeshed_, which
+itself is bundled into some typing tools such as Pylance_, so it may be
+necessary in some cases to locate the files for these packages and delete them,
+if they are in fact interfering with the new typing working correctly.
+
+Once SQLAlchemy 2.0 is released in final status, typeshed will remove
+SQLAlchemy from its own stubs source.
+
+
+
+.. _whatsnew_20_orm_declarative_typing:
+
+ORM Declarative Models
+----------------------
+
+SQLAlchemy 1.4 introduced the first SQLAlchemy-native ORM typing support
+using a combination of sqlalchemy2-stubs_ and the :ref:`Mypy Plugin <mypy_toplevel>`.
+In SQLAlchemy 2.0, the Mypy plugin **remains available, and has been updated
+to work with SQLAlchemy 2.0's typing system**. However, it should now be
+considered **deprecated**, as applications now have a straightforward path to adopting the
+new typing support that does not use plugins or stubs.
+
+Overview
+^^^^^^^^
+
+The fundamental approach for the new system is that mapped column declarations,
+when using a fully :ref:`Declarative <orm_declarative_table>` model (that is,
+not :ref:`hybrid declarative <orm_imperative_table_configuration>` or
+:ref:`imperative <orm_imperative_mapping>` configurations, which are unchanged),
+are first derived at runtime by inspecting the type annotation on the left side
+of each attribute declaration, if present. Left hand type annotations are
+expected to be contained within the
+:class:`_orm.Mapped` generic type, otherwise the attribute is not considered
+to be a mapped attribute. The attribute declaration may then refer to
+the :func:`_orm.mapped_column` construct on the right hand side, which is used
+to provide additional Core-level schema information about the
+:class:`_schema.Column` to be produced and mapped. This right hand side
+declaration is optional if a :class:`_orm.Mapped` annotation is present on the
+left side; if no annotation is present on the left side, then the
+:func:`_orm.mapped_column` may be used as an exact replacement for the
+:class:`_schema.Column` directive where it will provide for more accurate (but
+not exact) typing behavior of the attribute, even though no annotation is
+present.
+
+The approach is inspired by the approach of Python dataclasses_ which starts
+with an annotation on the left, then allows for an optional
+``dataclasses.field()`` specification on the right; the key difference from the
+dataclasses approach is that SQLAlchemy's approach is strictly **opt-in**,
+where existing mappings that use :class:`_schema.Column` without any type
+annotations continue to work as they always have, and the
+:func:`_orm.mapped_column` construct may be used as a direct replacement for
+:class:`_schema.Column` without any explicit type annotations. Only for exact
+attribute-level Python types to be present is the use of explicit annotations
+with :class:`_orm.Mapped` required. These annotations may be used on an
+as-needed, per-attribute basis for those attributes where specific types are
+helpful; non-annotated attributes that use :func:`_orm.mapped_column` will be
+typed as ``Any`` at the instance level.
+
+Migrating an Existing Mapping
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Transitioning to the new ORM approach begins as more verbose, but becomes more
+succinct than was previously possible as the available new features are used
+fully. The following steps detail a typical transition and then continue
+on to illustrate some more options.
+
+
+Step one - :func:`_orm.declarative_base` is superseded by :class:`_orm.DeclarativeBase`.
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+One observed limitation in Python typing is that there seems to be
+no ability to have a class dynamically generated from a function which then
+is understood by typing tools as a base for new classes. To solve this problem
+without plugins, the usual call to :func:`_orm.declarative_base` can be replaced
+with using the :class:`_orm.DeclarativeBase` class, which produces the same
+``Base`` object as usual, except that typing tools understand it::
+
+ from sqlalchemy.orm import DeclarativeBase
+
+ class Base(DeclarativeBase):
+ pass
+
+Step two - replace Declarative use of :class:`_schema.Column` with :func:`_orm.mapped_column`
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The :func:`_orm.mapped_column` is an ORM-typing aware construct that can
+be swapped directly for the use of :class:`_schema.Column`. Given a
+1.x style mapping as::
+
+ from sqlalchemy import Column
+ from sqlalchemy.orm import relationship
+ from sqlalchemy.orm import DeclarativeBase
+
+ class Base(DeclarativeBase):
+ pass
+
+ class User(Base):
+ __tablename__ = 'user_account'
+
+ id = Column(Integer, primary_key=True)
+ name = Column(String(30), nullable=False)
+ fullname = Column(String)
+ addresses = relationship("Address", back_populates="user")
+
+ class Address(Base):
+ __tablename__ = "address"
+
+ id = Column(Integer, primary_key=True)
+ email_address = Column(String, nullable=False)
+ user_id = Column(ForeignKey("user_account.id"), nullable=False)
+ user = relationship("User", back_populates="addresses")
+
+We replace :class:`_schema.Column` with :func:`_orm.mapped_column`; no
+arguments need to change::
+
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import mapped_column
+ from sqlalchemy.orm import relationship
+
+ class Base(DeclarativeBase):
+ pass
+
+ class User(Base):
+ __tablename__ = 'user_account'
+
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(30), nullable=False)
+ fullname = mapped_column(String)
+ addresses = relationship("Address", back_populates="user")
+
+ class Address(Base):
+ __tablename__ = "address"
+
+ id = mapped_column(Integer, primary_key=True)
+ email_address = mapped_column(String, nullable=False)
+ user_id = mapped_column(ForeignKey("user_account.id"), nullable=False)
+ user = relationship("User", back_populates="addresses")
+
+The individual columns above are **not yet typed with Python types**,
+and are instead typed as ``Mapped[Any]``; this is because we can declare any
+column either with ``Optional`` or not, and there's no way to have a
+"guess" in place that won't cause typing errors when we type it
+explicitly.
+
+However, at this step, our above mapping has appropriate :term:`descriptor` types
+set up for all attributes and may be used in queries as well as for
+instance-level manipulation, all of which will **pass mypy --strict mode** with no
+plugins.
+
+Step three - apply exact Python types as needed using :class:`_orm.Mapped`.
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This can be done for all attributes for which exact typing is desired;
+attributes that are fine being left as ``Any`` may be skipped. For
+context we also illustrate :class:`_orm.Mapped` being used for a
+:func:`_orm.relationship` where we apply an exact type.
+The mapping within this interim step
+will be more verbose, however with proficiency, this step can
+be combined with subsequent steps to update mappings more directly::
+
+ from typing import List
+ from typing import Optional
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import mapped_column
+ from sqlalchemy.orm import relationship
+
+ class Base(DeclarativeBase):
+ pass
+
+ class User(Base):
+ __tablename__ = 'user_account'
+
+ id: Mapped[int] = mapped_column(Integer, primary_key=True)
+ name: Mapped[str] = mapped_column(String(30), nullable=False)
+ fullname: Mapped[Optional[str]] = mapped_column(String)
+ addresses: Mapped[List["Address"]] = relationship("Address", back_populates="user")
+
+ class Address(Base):
+ __tablename__ = "address"
+
+ id: Mapped[int] = mapped_column(Integer, primary_key=True)
+ email_address: Mapped[str] = mapped_column(String, nullable=False)
+ user_id: Mapped[int] = mapped_column(ForeignKey("user_account.id"), nullable=False)
+ user: Mapped["User"] = relationship("User", back_populates="addresses")
+
+At this point, our ORM mapping is fully typed and will produce exact-typed
+:func:`_sql.select`, :class:`_orm.Query` and :class:`_engine.Result`
+constructs. We now can proceed to pare down redundancy in the mapping
+declaration.
+
+Step four - remove :func:`_orm.mapped_column` directives where no longer needed
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+All ``nullable`` parameters can be implied using ``Optional[]``; in
+the absence of ``Optional[]``, ``nullable`` defaults to ``False``. All SQL
+types without arguments such as ``Integer`` and ``String`` can be expressed
+as a Python annotation alone. A :func:`_orm.mapped_column` directive with no
+parameters can be removed entirely. :func:`_orm.relationship` now derives its
+class from the left hand annotation, supporting forward references as well
+(as :func:`_orm.relationship` has supported string-based forward references
+for ten years already ;) )::
+
+ from typing import List
+ from typing import Optional
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import mapped_column
+ from sqlalchemy.orm import relationship
+
+ class Base(DeclarativeBase):
+ pass
+
+ class User(Base):
+ __tablename__ = 'user_account'
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+ name: Mapped[str] = mapped_column(String(30))
+ fullname: Mapped[Optional[str]]
+ addresses: Mapped[List["Address"]] = relationship(back_populates="user")
+
+ class Address(Base):
+ __tablename__ = "address"
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+ email_address: Mapped[str]
+ user_id: Mapped[int] = mapped_column(ForeignKey("user_account.id"))
+ user: Mapped["User"] = relationship(back_populates="addresses")
+
+
+Step five - make use of pep-593 ``Annotated`` to package common directives into types
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This is a radical new
+capability that presents an alternative, or complementary approach, to
+:ref:`declarative mixins <orm_mixins_toplevel>` as a means to provide type
+oriented configuration, and also replaces the need for
+:class:`_orm.declared_attr` decorated functions in most cases.
+
+First, the Declarative mapping allows the mapping of Python type to
+SQL type, such as ``str`` to :class:`_types.String`, to be customized
+using :paramref:`_orm.registry.type_annotation_map`. Using :pep:`593`
+``Annotated`` allows us to create variants of a particular Python type so that
+the same type, such as ``str``, may be used which each provide variants
+of :class:`_types.String`, as below where use of an ``Annotated`` ``str`` called
+``str50`` will indicate ``String(50)``::
+
+ from typing_extensions import Annotated
+ from sqlalchemy.orm import DeclarativeBase
+
+ str50 = Annotated[str, 50]
+
+ # declarative base with a type-level override, using a type that is
+ # expected to be used in multiple places
+ class Base(DeclarativeBase):
+ registry = registry(type_annotation_map={
+ str50: String(50),
+ })
+
+
+Second, Declarative will extract full
+:func:`_orm.mapped_column` definitions from the left hand type if
+``Annotated[]`` is used, by passing a :func:`_orm.mapped_column` construct
+as any argument to the ``Annotated[]`` construct (credit to `@adriangb01 <https://twitter.com/adriangb01/status/1532841383647657988>`_
+for illustrating this idea). This capability may be extended in future releases
+to also include :func:`_orm.relationship`, :func:`_orm.composite` and other
+constructs, but currently is limited to :func:`_orm.mapped_column`. The
+example below adds additional ``Annotated`` types in addition to our
+``str50`` example to illustrate this feature::
+
+ from typing_extensions import Annotated
+ from typing import List
+ from typing import Optional
+ from sqlalchemy import ForeignKey
+ from sqlalchemy import String
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import mapped_column
+ from sqlalchemy.orm import relationship
+
+ # declarative base from previous example
+ str50 = Annotated[str, 50]
+
+ class Base(DeclarativeBase):
+ registry = registry(type_annotation_map={
+ str50: String(50),
+ })
+
+ # set up mapped_column() overrides, using whole column styles that are
+ # expected to be used in multiple places
+ intpk = Annotated[int, mapped_column(primary_key=True)]
+ user_fk = Annotated[int, mapped_column(ForeignKey('user_account.id'))]
+
+
+ class User(Base):
+ __tablename__ = 'user_account'
+
+ id: Mapped[intpk]
+ name: Mapped[str50]
+ fullname: Mapped[Optional[str]]
+ addresses: Mapped[List["Address"]] = relationship(back_populates="user")
+
+ class Address(Base):
+ __tablename__ = "address"
+
+ id: Mapped[intpk]
+ email_address: Mapped[str50]
+ user_id: Mapped[user_fk]
+ user: Mapped["User"] = relationship(back_populates="addresses")
+
+Above, columns that are mapped with ``Mapped[str50]``, ``Mapped[intpk]``,
+or ``Mapped[user_fk]`` draw from both the
+:paramref:`_orm.registry.type_annotation_map` as well as the
+``Annotated`` construct directly in order to re-use pre-established typing
+and column configurations.
+
+Step six - turn mapped classes into dataclasses_
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+We can turn mapped classes into dataclasses_, where a key advantage
+is that we can build a strictly-typed ``__init__()`` method with explicit
+positional, keyword only, and default arguments, not to mention we get methods
+such as ``__str__()`` and ``__repr__()`` for free. The next section
+:ref:`whatsnew_20_dataclasses` illustrates further transformation of the above
+model.
+
+
+Typing is supported from step 3 onwards
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+With the above examples, any example from "step 3" on forward will include
+that the attributes
+of the model are typed
+and will populate through to :func:`_sql.select`, :class:`_orm.Query`,
+and :class:`_engine.Row` objects::
+
+ # (variable) stmt: Select[Tuple[int, str]]
+ stmt = select(User.id, User.name)
+
+ with Session(e) as sess:
+ for row in sess.execute(stmt):
+ # (variable) row: Row[Tuple[int, str]]
+ print(row)
+
+ # (variable) users: Sequence[User]
+ users = sess.scalars(select(User)).all()
+
+ # (variable) users_legacy: List[User]
+ users_legacy = sess.query(User).all()
+
+.. seealso::
+
+ :ref:`orm_declarative_table` - Updated Declarative documentation for
+ Declarative generation and mapping of :class:`.Table` columns.
+
+.. _whatsnew_20_dataclasses:
+
+Native Support for Dataclasses Mapped as ORM Models
+-----------------------------------------------------
+
+The new ORM Declarative features introduced above at
+:ref:`whatsnew_20_orm_declarative_typing` introduced the
+new :func:`_orm.mapped_column` construct and illustrated type-centric
+mapping with optional use of :pep:`593` ``Annotated``. We can take
+the mapping one step further by integrating with with Python
+dataclasses_. This new feature is made possible via :pep:`681` which
+allows for type checkers to recognize classes that are dataclass compatible,
+or are fully dataclasses, but were declared through alternate APIs.
+
+Using the dataclasses feature, mapped classes gain an ``__init__()`` method
+that supports positional arguments as well as customizable default values
+for optional keyword arguments. As mentioned previously, dataclasses also
+generate many useful methods such as ``__str__()``, ``__eq__()``. Dataclass
+serialization methods such as
+`dataclasses.asdict() <https://docs.python.org/3/library/dataclasses.html#dataclasses.asdict>`_ and
+`dataclasses.astuple() <https://docs.python.org/3/library/dataclasses.html#dataclasses.astuple>`_
+also work, but don't currently accommodate for self-referential structures, which
+makes them less viable for mappings that have bidirectional relationships.
+
+SQLAlchemy's current integration approach converts the user-defined class
+into a **real dataclass** to provide runtime functionality; the feature
+makes use of the existing dataclass feature introduced in SQLAlchemy 1.4 at
+:ref:`change_5027` to produce an equivalent runtime mapping with a fully integrated
+configuration style, which is also more correctly typed than was possible
+with the previous approach.
+
+To support dataclasses in compliance with :pep:`681`, ORM constructs like
+:func:`_orm.mapped_column` and :func:`_orm.relationship` accept additional
+:pep:`681` arguments ``init``, ``default``, and ``default_factory`` which
+are passed along to the dataclass creation process. These
+arguments currently must be present in an explicit directive on the right side,
+just as they would be used with ``dataclasses.field()``; they currently
+can't be local to an ``Annotated`` construct on the left side. To support
+the convenient use of ``Annotated`` while still supporting dataclass
+configuration, :func:`_orm.mapped_column` can merge
+a minimal set of right-hand arguments with that of an existing
+:func:`_orm.mapped_column` construct located on the left side within an ``Annotated``
+construct, so that most of the succinctness is maintained, as will be seen
+below.
+
+To enable dataclasses using class inheritance we make
+use of the :class:`.MappedAsDataclass` mixin, either directly on each class, or
+on the ``Base`` class, as illustrated below where we further modify the
+example mapping from "Step 5" of :ref:`whatsnew_20_orm_declarative_typing`::
+
+ from typing_extensions import Annotated
+ from typing import List
+ from typing import Optional
+ from sqlalchemy import ForeignKey
+ from sqlalchemy import String
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import MappedAsDataclass
+ from sqlalchemy.orm import mapped_column
+ from sqlalchemy.orm import relationship
+
+
+ class Base(MappedAsDataclass, DeclarativeBase):
+ """subclasses will be converted to dataclasses"""
+
+ intpk = Annotated[int, mapped_column(primary_key=True)]
+ str30 = Annotated[str, mapped_column(String(30))]
+ user_fk = Annotated[int, mapped_column(ForeignKey("user_account.id"))]
+
+
+ class User(Base):
+ __tablename__ = "user_account"
+
+ id: Mapped[intpk] = mapped_column(init=False)
+ name: Mapped[str30]
+ fullname: Mapped[Optional[str]] = mapped_column(default=None)
+ addresses: Mapped[List["Address"]] = relationship(
+ back_populates="user", default_factory=list
+ )
+
+
+ class Address(Base):
+ __tablename__ = "address"
+
+ id: Mapped[intpk] = mapped_column(init=False)
+ email_address: Mapped[str]
+ user_id: Mapped[user_fk] = mapped_column(init=False)
+ user: Mapped["User"] = relationship(
+ back_populates="addresses", default=None
+ )
+
+The above mapping has used the ``@dataclasses.dataclass`` decorator directly
+on each mapped class at the same time that the declarative mapping was
+set up, internally setting up each ``dataclasses.field()`` directive as
+indicated. ``User`` / ``Address`` structures can be created using
+positional arguments as configured::
+
+ >>> u1 = User("username", fullname="full name", addresses=[Address("email@address")])
+ >>> u1
+ User(id=None, name='username', fullname='full name', addresses=[Address(id=None, email_address='email@address', user_id=None, user=...)])
+
+
+.. seealso::
+
+ :ref:`orm_declarative_native_dataclasses`
+
autodoc_class_signature = "separated"
+# enable "annotation" indicator. doesn't actually use this
+# link right now, it's just a png image
+zzzeeksphinx_annotation_key = "glossary#annotated-example"
# to use this, we need:
# 1. fix sphinx-paramlinks to work with "description" typing
:ref:`migration_20_toplevel`
+ mixin class
+ mixin classes
+
+ A common object-oriented pattern where a class that contains methods or
+ attributes for use by other classes without having to be the parent class
+ of those other classes.
+
+ .. seealso::
+
+ `Mixin (via Wikipedia) <https://en.wikipedia.org/wiki/Mixin>`_
+
+
reflection
reflected
In SQLAlchemy, this term refers to the feature of querying a database's
:ref:`metadata_reflection_toplevel` - complete background on
database reflection.
+ :ref:`orm_declarative_reflected` - background on integrating
+ ORM mappings with reflected tables.
+
imperative
declarative
.. container::
* :doc:`Migrating to SQLAlchemy 2.0 <changelog/migration_20>` - Complete background on migrating from 1.3 or 1.4 to 2.0
+ * :doc:`What's New in SQLAlchemy 2.0? <changelog/whatsnew_20>` - New 2.0 features and behaviors beyond the 1.x migration
* :doc:`Changelog catalog <changelog/index>` - Detailed changelogs for all SQLAlchemy Versions
Starting with the following example::
from sqlalchemy import Column, ForeignKey, Integer, String
- from sqlalchemy.orm import declarative_base, relationship
+ from sqlalchemy.orm import DeclarativeBase, relationship
- Base = declarative_base()
+ class Base(DeclarativeBase):
+ pass
class User(Base):
__tablename__ = "user"
- id = Column(Integer, primary_key=True)
- name = Column(String)
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String)
addresses = relationship("Address", backref="user")
class Address(Base):
__tablename__ = "address"
- id = Column(Integer, primary_key=True)
- email = Column(String)
- user_id = Column(Integer, ForeignKey("user.id"))
+ id = mapped_column(Integer, primary_key=True)
+ email = mapped_column(String)
+ user_id = mapped_column(Integer, ForeignKey("user.id"))
The above configuration establishes a collection of ``Address`` objects on ``User`` called
``User.addresses``. It also establishes a ``.user`` attribute on ``Address`` which will
it's equivalent to the following::
from sqlalchemy import Column, ForeignKey, Integer, String
- from sqlalchemy.orm import declarative_base, relationship
+ from sqlalchemy.orm import DeclarativeBase, relationship
- Base = declarative_base()
+ class Base(DeclarativeBase):
+ pass
class User(Base):
__tablename__ = "user"
- id = Column(Integer, primary_key=True)
- name = Column(String)
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String)
addresses = relationship("Address", back_populates="user")
class Address(Base):
__tablename__ = "address"
- id = Column(Integer, primary_key=True)
- email = Column(String)
- user_id = Column(Integer, ForeignKey("user.id"))
+ id = mapped_column(Integer, primary_key=True)
+ email = mapped_column(String)
+ user_id = mapped_column(Integer, ForeignKey("user.id"))
user = relationship("User", back_populates="addresses")
which also includes the :paramref:`_orm.relationship.backref` keyword::
from sqlalchemy import Column, ForeignKey, Integer, String
- from sqlalchemy.orm import declarative_base, relationship
+ from sqlalchemy.orm import DeclarativeBase, relationship
- Base = declarative_base()
+ class Base(DeclarativeBase):
+ pass
class User(Base):
__tablename__ = "user"
- id = Column(Integer, primary_key=True)
- name = Column(String)
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String)
addresses = relationship(
"Address",
class Address(Base):
__tablename__ = "address"
- id = Column(Integer, primary_key=True)
- email = Column(String)
- user_id = Column(Integer, ForeignKey("user.id"))
+ id = mapped_column(Integer, primary_key=True)
+ email = mapped_column(String)
+ user_id = mapped_column(Integer, ForeignKey("user.id"))
When the "backref" is generated, the :paramref:`_orm.relationship.primaryjoin`
condition is copied to the new :func:`_orm.relationship` as well::
class User(Base):
__tablename__ = "user"
- id = Column(Integer, primary_key=True)
- name = Column(String)
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String)
addresses = relationship(
"Address",
Basic Relationship Patterns
---------------------------
-A quick walkthrough of the basic relational patterns.
+A quick walkthrough of the basic relational patterns, which in this section are illustrated
+using :ref:`Declarative <orm_explicit_declarative_base>` style mappings
+based on the use of the :class:`_orm.Mapped` annotation type.
-The imports used for each of the following sections is as follows::
+The setup for each of the following sections is as follows::
- from sqlalchemy import Column, ForeignKey, Integer, Table
- from sqlalchemy.orm import declarative_base, relationship
+ from sqlalchemy import ForeignKey
+ from sqlalchemy import Integer
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import mapped_column
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import relationship
- Base = declarative_base()
+ class Base(DeclarativeBase):
+ pass
+
+
+Declarative vs. Imperative Forms
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+As SQLAlchemy has evolved, different ORM configurational styles have emerged.
+For examples in this section and others that use annotated
+:ref:`Declarative <orm_explicit_declarative_base>` mappings with
+:class:`_orm.Mapped`, the corresponding non-annotated form should use the
+desired class, or string class name, as the first argument passed to
+:func:`_orm.relationship`. The example below illustrates the form used in
+this document, which is a fully Declarative example using :pep:`484` annotations,
+where the :func:`_orm.relationship` construct is also deriving the target
+class and collection type from the :class:`_orm.Mapped` annotation,
+which is the most modern form of SQLAlchemy Declarative mapping::
+
+ class Parent(Base):
+ __tablename__ = "parent"
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+ children: Mapped[list["Child"]] = relationship(back_populates="parent")
+
+
+ class Child(Base):
+ __tablename__ = "child"
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+ parent_id: Mapped[int] = mapped_column(ForeignKey("parent.id"))
+ parent: Mapped["Parent"] = relationship(back_populates="children")
+
+In contrast, using a Declarative mapping **without** annotations is
+the more "classic" form of mapping, where :func:`_orm.relationship`
+requires all parameters passed to it directly, as in the example below::
+
+ class Parent(Base):
+ __tablename__ = "parent"
+
+ id = mapped_column(Integer, primary_key=True)
+ children = relationship("Child", back_populates="parent")
+
+
+ class Child(Base):
+ __tablename__ = "child"
+
+ id = mapped_column(Integer, primary_key=True)
+ parent_id = mapped_column(ForeignKey("parent.id"))
+ parent = relationship("Parent", back_populates="children")
+
+Finally, using :ref:`Imperative Mapping <orm_imperative_mapping>`, which
+is SQLAlchemy's original mapping form before Declarative was made (which
+nonetheless remains preferred by a vocal minority of users), the above
+configuration looks like::
+
+ registry.map_imperatively(
+ Parent, parent_table, properties={
+ "children": relationship(
+ "Child", back_populates="parent"
+ )
+ }
+ )
+
+ registry.map_imperatively(
+ Child, child_table, properties={
+ "parent": relationship("Parent", back_populates="children")
+ }
+ )
+
+Additionally, the default collection style for non-annotated mappings is
+``list``. To use a ``set`` or other collection without annotations, indicate
+it using the :paramref:`_orm.relationship.collection_class` parameter::
+
+ class Parent(Base):
+ __tablename__ = "parent"
+
+ id = mapped_column(Integer, primary_key=True)
+ children = relationship("Child", collection_class=set, ...)
+
+Detail on collection configuration for :func:`_orm.relationship` is at
+:ref:`custom_collections`.
+
+Additional differences between annotated and non-annotated / imperative
+styles will be noted as needed.
.. _relationship_patterns_o2m:
class Parent(Base):
__tablename__ = "parent"
- id = Column(Integer, primary_key=True)
- children = relationship("Child")
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+ children: Mapped[list["Child"]] = relationship()
class Child(Base):
__tablename__ = "child"
- id = Column(Integer, primary_key=True)
- parent_id = Column(Integer, ForeignKey("parent.id"))
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+ parent_id: Mapped[int] = mapped_column(ForeignKey("parent.id"))
To establish a bidirectional relationship in one-to-many, where the "reverse"
side is a many to one, specify an additional :func:`_orm.relationship` and connect
class Parent(Base):
__tablename__ = "parent"
- id = Column(Integer, primary_key=True)
- children = relationship("Child", back_populates="parent")
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+ children: Mapped[list["Child"]] = relationship(back_populates="parent")
class Child(Base):
__tablename__ = "child"
- id = Column(Integer, primary_key=True)
- parent_id = Column(Integer, ForeignKey("parent.id"))
- parent = relationship("Parent", back_populates="children")
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+ parent_id: Mapped[int] = mapped_column(ForeignKey("parent.id"))
+ parent: Mapped["Parent"] = relationship(back_populates="children")
``Child`` will get a ``parent`` attribute with many-to-one semantics.
Alternatively, the :paramref:`_orm.relationship.backref` option may be used
on a single :func:`_orm.relationship` instead of using
-:paramref:`_orm.relationship.back_populates`::
+:paramref:`_orm.relationship.back_populates`; in this form, the ``Child.parent``
+relationship is generated implicitly::
+
+ class Parent(Base):
+ __tablename__ = "parent"
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+ children: Mapped[list["Child"]] = relationship(backref="parent")
+
+ class Child(Base):
+ __tablename__ = "child"
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+ parent_id: Mapped[int] = mapped_column(ForeignKey("parent.id"))
+
+.. note::
+
+ Using :paramref:`_orm.relationship.backref` will not provide
+ adequate information to :pep:`484` typing tools such that they will be
+ correctly aware of the ``Child.parent`` attribute, as it is not
+ explicitly present. For modern Python styles,
+ :paramref:`_orm.relationship.back_populates` with explicit use of
+ :func:`_orm.relationship` on both classes in a bi-directional relationship
+ should be preferred.
+
+.. _relationship_patterns_o2m_collection:
+
+Using Sets, Lists, or other Collection Types for One To Many
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Using annotated Declarative mappings, the type of collection used for the
+:func:`_orm.relationship` is derived from the collection type passed to the
+:class:`_orm.Mapped` container type. The example from the previous section
+may be written to use a ``set`` rather than a ``list`` for the
+``Parent.children`` collection using ``Mapped[set["Child"]]``::
class Parent(Base):
__tablename__ = "parent"
- id = Column(Integer, primary_key=True)
- children = relationship("Child", backref="parent")
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+ children: Mapped[set["Child"]] = relationship(back_populates="parent")
+
+When using non-annotated forms including imperative mappings, the Python
+class to use as a collection may be passed using the
+:paramref:`_orm.relationship.collection_class` parameter.
+
+.. seealso::
+
+ :ref:`custom_collections` - contains further detail on collection
+ configuration including some techniques to map :func:`_orm.relationship`
+ to dictionaries.
+
Configuring Delete Behavior for One to Many
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
class Parent(Base):
__tablename__ = "parent"
- id = Column(Integer, primary_key=True)
- child_id = Column(Integer, ForeignKey("child.id"))
- child = relationship("Child")
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+ child_id: Mapped[int] = mapped_column(ForeignKey("child.id"))
+ child: Mapped["Child"] = relationship()
class Child(Base):
__tablename__ = "child"
- id = Column(Integer, primary_key=True)
+
+ id: Mapped[int] = mapped_column(primary_key=True)
Bidirectional behavior is achieved by adding a second :func:`_orm.relationship`
and applying the :paramref:`_orm.relationship.back_populates` parameter
class Parent(Base):
__tablename__ = "parent"
- id = Column(Integer, primary_key=True)
- child_id = Column(Integer, ForeignKey("child.id"))
- child = relationship("Child", back_populates="parents")
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+ child_id: Mapped[int] = mapped_column(ForeignKey("child.id"))
+ child: Mapped["Child"] = relationship(back_populates="parents")
class Child(Base):
__tablename__ = "child"
- id = Column(Integer, primary_key=True)
- parents = relationship("Parent", back_populates="child")
-Alternatively, the :paramref:`_orm.relationship.backref` parameter
-may be applied to a single :func:`_orm.relationship`, such as ``Parent.child``::
+ id: Mapped[int] = mapped_column(primary_key=True)
+ parents: Mapped[list["Parent"]] = relationship(back_populates="child")
+
+As is the case with :ref:`relationship_patterns_o2m`, the
+:paramref:`_orm.relationship.backref` parameter may be used in place of
+:paramref:`_orm.relationship.back_populates`, however :paramref:`_orm.relationship.back_populates`
+is preferred for its explicitness.
- class Parent(Base):
- __tablename__ = "parent"
- id = Column(Integer, primary_key=True)
- child_id = Column(Integer, ForeignKey("child.id"))
- child = relationship("Child", backref="parents")
.. _relationships_one_to_one:
One To One
~~~~~~~~~~
-One To One is essentially a bidirectional relationship with a scalar
-attribute on both sides. Within the ORM, "one-to-one" is considered as a
-convention where the ORM expects that only one related row will exist
-for any parent row.
+One To One is essentially a :ref:`relationship_patterns_o2m`
+relationship from a foreign key perspective, but indicates that there will
+only be one row at any time that refers to a particular parent row.
-The "one-to-one" convention is achieved by applying a value of
-``False`` to the :paramref:`_orm.relationship.uselist` parameter of the
-:func:`_orm.relationship` construct, or in some cases the :func:`_orm.backref`
-construct, applying it on the "one-to-many" or "collection" side of a
-relationship.
-
-In the example below we present a bidirectional relationship that includes
-both :ref:`one-to-many <relationship_patterns_o2m>` (``Parent.children``) and
-a :ref:`many-to-one <relationship_patterns_m2o>` (``Child.parent``)
-relationships::
+When using annotated mappings with :class:`_orm.Mapped`, the "one-to-one"
+convention is achieved by applying a non-collection type to the
+:class:`_orm.Mapped` annotation on both sides of the relationship, which will
+imply to the ORM that a collection should not be used on either side, as in the
+example below::
class Parent(Base):
__tablename__ = "parent"
- id = Column(Integer, primary_key=True)
- # one-to-many collection
- children = relationship("Child", back_populates="parent")
+ id: Mapped[int] = mapped_column(primary_key=True)
+ child: Mapped["Child"] = relationship(back_populates="parent")
class Child(Base):
__tablename__ = "child"
- id = Column(Integer, primary_key=True)
- parent_id = Column(Integer, ForeignKey("parent.id"))
- # many-to-one scalar
- parent = relationship("Parent", back_populates="children")
-
-Above, ``Parent.children`` is the "one-to-many" side referring to a collection,
-and ``Child.parent`` is the "many-to-one" side referring to a single object.
-To convert this to "one-to-one", the "one-to-many" or "collection" side
-is converted into a scalar relationship using the ``uselist=False`` flag,
-renaming ``Parent.children`` to ``Parent.child`` for clarity::
-
- class Parent(Base):
- __tablename__ = "parent"
- id = Column(Integer, primary_key=True)
-
- # previously one-to-many Parent.children is now
- # one-to-one Parent.child
- child = relationship("Child", back_populates="parent", uselist=False)
-
-
- class Child(Base):
- __tablename__ = "child"
- id = Column(Integer, primary_key=True)
- parent_id = Column(Integer, ForeignKey("parent.id"))
-
- # many-to-one side remains, see tip below
- parent = relationship("Parent", back_populates="child")
+ id: Mapped[int] = mapped_column(primary_key=True)
+ parent_id: Mapped[int] = mapped_column(ForeignKey("parent.id"))
+ parent: Mapped["Parent"] = relationship(back_populates="child")
Above, when we load a ``Parent`` object, the ``Parent.child`` attribute
will refer to a single ``Child`` object rather than a collection. If we
constraint on the ``Child.parent_id`` column would ensure that only
one ``Child`` row may refer to a particular ``Parent`` row at a time.
+.. versionadded:: 2.0 The :func:`_orm.relationship` construct can derive
+ the effective value of the :paramref:`_orm.relationship.uselist`
+ parameter from a given :class:`_orm.Mapped` annotation.
-In the case where the :paramref:`_orm.relationship.backref`
-parameter is used to define the "one-to-many" side, this can be converted
-to the "one-to-one" convention using the :func:`_orm.backref`
-function which allows the relationship generated by the
-:paramref:`_orm.relationship.backref` parameter to receive custom parameters,
-in this case the ``uselist`` parameter::
+Setting uselist=False for non-annotated configurations
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- from sqlalchemy.orm import backref
+When using :func:`_orm.relationship` without the benefit of :class:`_orm.Mapped`
+annotations, the one-to-one pattern can be enabled using the
+:paramref:`_orm.relationship.uselist` parameter set to ``False`` on what
+would normally be the "many" side, illustrated in a non-annotated
+Declarative configuration below::
class Parent(Base):
__tablename__ = "parent"
- id = Column(Integer, primary_key=True)
+
+ id = mapped_column(Integer, primary_key=True)
+ child = relationship("Child", uselist=False, back_populates="parent")
class Child(Base):
__tablename__ = "child"
- id = Column(Integer, primary_key=True)
- parent_id = Column(Integer, ForeignKey("parent.id"))
- parent = relationship("Parent", backref=backref("child", uselist=False))
+
+ id = mapped_column(Integer, primary_key=True)
+ parent_id = mapped_column(ForeignKey("parent.id"))
+ parent = relationship("Parent", back_populates="child")
+
.. _relationships_many_to_many:
~~~~~~~~~~~~
Many to Many adds an association table between two classes. The association
-table is indicated by the :paramref:`_orm.relationship.secondary` argument to
-:func:`_orm.relationship`. Usually, the :class:`_schema.Table` uses the
-:class:`_schema.MetaData` object associated with the declarative base
-class, so that the :class:`_schema.ForeignKey` directives can locate the
-remote tables with which to link::
-
+table is nearly always given as a Core :class:`_schema.Table` object or
+other Core selectable such as a :class:`_sql.Join` object, and is
+indicated by the :paramref:`_orm.relationship.secondary` argument to
+:func:`_orm.relationship`. Usually, the :class:`_schema.Table` uses the
+:class:`_schema.MetaData` object associated with the declarative base class, so
+that the :class:`_schema.ForeignKey` directives can locate the remote tables
+with which to link::
+
+ from sqlalchemy import Column
+ from sqlalchemy import Table
+
+ # note for a Core table, we use the sqlalchemy.Column construct,
+ # not sqlalchemy.orm.mapped_column
association_table = Table(
"association",
Base.metadata,
class Parent(Base):
__tablename__ = "left"
- id = Column(Integer, primary_key=True)
- children = relationship("Child", secondary=association_table)
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+ children: Mapped[list["Child"]] = relationship(secondary=association_table)
class Child(Base):
__tablename__ = "right"
- id = Column(Integer, primary_key=True)
+
+ id: Mapped[int] = mapped_column(primary_key=True)
.. tip::
Column("right_id", ForeignKey("right.id"), primary_key=True),
)
+Setting Bi-Directional Many-to-many
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
For a bidirectional relationship, both sides of the relationship contain a
collection. Specify using :paramref:`_orm.relationship.back_populates`, and
for each :func:`_orm.relationship` specify the common association table::
+ from sqlalchemy import Column
+ from sqlalchemy import Table
+
association_table = Table(
"association",
Base.metadata,
class Parent(Base):
__tablename__ = "left"
- id = Column(Integer, primary_key=True)
- children = relationship(
- "Child", secondary=association_table, back_populates="parents"
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+ children: Mapped[list["Child"]] = relationship(
+ secondary=association_table, back_populates="parents"
)
class Child(Base):
__tablename__ = "right"
- id = Column(Integer, primary_key=True)
- parents = relationship(
- "Parent", secondary=association_table, back_populates="children"
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+ parents: Mapped[list["Parent"]] = relationship(
+ secondary=association_table, back_populates="children"
)
use the same :paramref:`_orm.relationship.secondary` argument for the
reverse relationship::
+ from sqlalchemy import Column
+ from sqlalchemy import Table
+
association_table = Table(
"association",
Base.metadata,
- Column("left_id", ForeignKey("left.id"), primary_key=True),
- Column("right_id", ForeignKey("right.id"), primary_key=True),
+ mapped_column("left_id", ForeignKey("left.id"), primary_key=True),
+ mapped_column("right_id", ForeignKey("right.id"), primary_key=True),
)
class Parent(Base):
__tablename__ = "left"
- id = Column(Integer, primary_key=True)
- children = relationship(
- "Child", secondary=association_table, backref="parents"
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+ children: Mapped[list["Child"]] = relationship(
+ secondary=association_table, backref="parents"
)
class Child(Base):
__tablename__ = "right"
- id = Column(Integer, primary_key=True)
+ id: Mapped[int] = mapped_column(primary_key=True)
-The :paramref:`_orm.relationship.secondary` argument of
-:func:`_orm.relationship` also accepts a callable that returns the ultimate
-argument, which is evaluated only when mappers are first used. Using this, we
-can define the ``association_table`` at a later point, as long as it's
-available to the callable after all module initialization is complete::
+Using a late-evaluated form for the "secondary" argument
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- class Parent(Base):
- __tablename__ = "left"
- id = Column(Integer, primary_key=True)
- children = relationship(
- "Child",
- secondary=lambda: association_table,
- backref="parents",
- )
+The :paramref:`_orm.relationship.secondary` parameter of
+:func:`_orm.relationship` also accepts two different "late evaluated" forms,
+including string table name as well as lambda callable. See the section
+:ref:`orm_declarative_relationship_secondary_eval` for background and
+examples.
-With the declarative extension in use, the traditional "string name of the table"
-is accepted as well, matching the name of the table as stored in ``Base.metadata.tables``::
+
+Using Sets, Lists, or other Collection Types for Many To Many
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Configuration of collections for a Many to Many relationship is identical
+to that of :ref:`relationship_patterns_o2m`, as described at
+:ref:`relationship_patterns_o2m_collection`. For an annotated mapping
+using :class:`_orm.Mapped`, the collection can be indicated by the
+type of collection used within the :class:`_orm.Mapped` generic class,
+such as ``set``::
class Parent(Base):
__tablename__ = "left"
- id = Column(Integer, primary_key=True)
- children = relationship("Child", secondary="association", backref="parents")
-.. warning:: When passed as a Python-evaluable string, the
- :paramref:`_orm.relationship.secondary` argument is interpreted using Python's
- ``eval()`` function. **DO NOT PASS UNTRUSTED INPUT TO THIS STRING**. See
- :ref:`declarative_relationship_eval` for details on declarative
- evaluation of :func:`_orm.relationship` arguments.
+ id: Mapped[int] = mapped_column(primary_key=True)
+ children: Mapped[set["Child"]] = relationship(secondary=association_table)
+When using non-annotated forms including imperative mappings, as is
+the case with one-to-many, the Python
+class to use as a collection may be passed using the
+:paramref:`_orm.relationship.collection_class` parameter.
+
+.. seealso::
+
+ :ref:`custom_collections` - contains further detail on collection
+ configuration including some techniques to map :func:`_orm.relationship`
+ to dictionaries.
.. _relationships_many_to_many_deletion:
Association Object
~~~~~~~~~~~~~~~~~~
-The association object pattern is a variant on many-to-many: it's used
-when your association table contains additional columns beyond those
-which are foreign keys to the left and right tables. Instead of using
-the :paramref:`_orm.relationship.secondary` argument, you map a new class
-directly to the association table. The left side of the relationship
-references the association object via one-to-many, and the association
-class references the right side via many-to-one. Below we illustrate
-an association table mapped to the ``Association`` class which
-includes a column called ``extra_data``, which is a string value that
+The association object pattern is a variant on many-to-many: it's used when an
+association table contains additional columns beyond those which are foreign
+keys to the parent and child (or left and right) tables, columns which are most
+ideally mapped to their own ORM mapped class. This mapped class is mapped
+against the :class:`.Table` that would otherwise be noted as
+:paramref:`_orm.relationship.secondary` when using the many-to-many pattern.
+
+In the association object pattern, the :paramref:`_orm.relationship.secondary`
+parameter is not used; instead, a class is mapped directly to the association
+table. Two individual :func:`_orm.relationship` constructs then link first the
+parent side to the mapped association class via one to many, and then the
+mapped association class to the child side via many-to-one, to form a
+uni-directional association object relationship from parent, to association, to
+child. For a bi-directional relationship, four :func:`_orm.relationship`
+constructs are used to link the mapped association class to both parent and
+child in both directions.
+
+The example below illustrates a new class ``Association`` which maps
+to the :class:`.Table` named ``association``; this table now includes
+an additional column called ``extra_data``, which is a string value that
is stored along with each association between ``Parent`` and
-``Child``::
+``Child``. By mapping the table to an explicit class, rudimental access
+from ``Parent`` to ``Child`` makes explicit use of ``Association``::
+
+ from typing import Optional
class Association(Base):
__tablename__ = "association"
- left_id = Column(ForeignKey("left.id"), primary_key=True)
- right_id = Column(ForeignKey("right.id"), primary_key=True)
- extra_data = Column(String(50))
- child = relationship("Child")
+ left_id: Mapped[int] = mapped_column(ForeignKey("left.id"), primary_key=True)
+ right_id: Mapped[int] = mapped_column(ForeignKey("right.id"), primary_key=True)
+ extra_data: Mapped[Optional[str]]
+ child: Mapped["Child"] = relationship()
class Parent(Base):
__tablename__ = "left"
- id = Column(Integer, primary_key=True)
- children = relationship("Association")
+ id: Mapped[int] = mapped_column(primary_key=True)
+ children: Mapped[list["Association"]] = relationship()
class Child(Base):
__tablename__ = "right"
- id = Column(Integer, primary_key=True)
+ id: Mapped[int] = mapped_column(primary_key=True)
+
+To illustrate the bi-directional version, we add two more :func:`_orm.relationship`
+constructs, linked to the existing ones using :paramref:`_orm.relationship.back_populates`::
-As always, the bidirectional version makes use of :paramref:`_orm.relationship.back_populates`
-or :paramref:`_orm.relationship.backref`::
+ from typing import Optional
class Association(Base):
__tablename__ = "association"
- left_id = Column(ForeignKey("left.id"), primary_key=True)
- right_id = Column(ForeignKey("right.id"), primary_key=True)
- extra_data = Column(String(50))
- child = relationship("Child", back_populates="parents")
- parent = relationship("Parent", back_populates="children")
+ left_id: Mapped[int] = mapped_column(ForeignKey("left.id"), primary_key=True)
+ right_id: Mapped[int] = mapped_column(ForeignKey("right.id"), primary_key=True)
+ extra_data: Mapped[Optional[str]]
+ child: Mapped["Child"] = relationship(back_populates="parents")
+ parent: Mapped["Parent"] = relationship(back_populates="children")
class Parent(Base):
__tablename__ = "left"
- id = Column(Integer, primary_key=True)
- children = relationship("Association", back_populates="parent")
+ id: Mapped[int] = mapped_column(primary_key=True)
+ children: Mapped[list["Association"]] = relationship(back_populates="parent")
class Child(Base):
__tablename__ = "right"
- id = Column(Integer, primary_key=True)
- parents = relationship("Association", back_populates="child")
+ id: Mapped[int] = mapped_column(primary_key=True)
+ parents: Mapped[list["Association"]] = relationship(back_populates="child")
+
+Schemes that use :paramref:`_orm.relationship.backref` are possible as well,
+where there would be two explicit :func:`_orm.relationship` constructs, each
+of which would then include :paramref:`_orm.relationship.backref`
+parameters that imply the production of two more
+:func:`_orm.relationship` constructs.
Working with the association pattern in its direct form requires that child
objects are associated with an association instance before being appended to
access two "hops" with a single access, one "hop" to the
associated object, and a second to a target attribute.
+.. seealso::
+
+ :ref:`associationproxy_toplevel` - allows direct "many to many" style
+ access between parent and child for a three-class association object mapping.
+
.. warning::
- The association object pattern **does not coordinate changes with a
- separate relationship that maps the association table as "secondary"**.
+ Avoid mixing the association object pattern with the :ref:`many-to-many <relationships_many_to_many>`
+ pattern directly, as this produces conditions where data may be read
+ and written in an inconsistent fashion without special steps;
+ the :ref:`association proxy <associationproxy_toplevel>` is typically
+ used to provide more succinct access. For more detailed background
+ on the caveats introduced by this combination, see the next section
+ :ref:`association_pattern_w_m2m`.
+
+.. _association_pattern_w_m2m:
- Below, changes made to ``Parent.children`` will not be coordinated
- with changes made to ``Parent.child_associations`` or
- ``Child.parent_associations`` in Python; while all of these relationships will continue
- to function normally by themselves, changes on one will not show up in another
- until the :class:`.Session` is expired, which normally occurs automatically
- after :meth:`.Session.commit`::
+Combining Association Object with Many-to-Many Access Patterns
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- class Association(Base):
- __tablename__ = "association"
+As mentioned in the previous section, the association object pattern does not
+automatically integrate with usage of the many-to-many pattern against the same
+tables/columns at the same time. From this it follows that read operations
+may return conflicting data and write operations may also attempt to flush
+conflicting changes, causing either integrity errors or unexpected
+inserts or deletes.
- left_id = Column(ForeignKey("left.id"), primary_key=True)
- right_id = Column(ForeignKey("right.id"), primary_key=True)
- extra_data = Column(String(50))
+To illustrate, the example below configures a bidirectional many-to-many relationship
+between ``Parent`` and ``Child`` via ``Parent.children`` and ``Child.parents``.
+At the same time, an association object relationship is also configured,
+between ``Parent.child_associations -> Association.child``
+and ``Child.parent_associations -> Association.parent``::
- child = relationship("Child", backref="parent_associations")
- parent = relationship("Parent", backref="child_associations")
+ from typing import Optional
+ class Association(Base):
+ __tablename__ = "association"
- class Parent(Base):
- __tablename__ = "left"
- id = Column(Integer, primary_key=True)
+ left_id: Mapped[int] = mapped_column(ForeignKey("left.id"), primary_key=True)
+ right_id: Mapped[int] = mapped_column(ForeignKey("right.id"), primary_key=True)
+ extra_data: Mapped[Optional[str]]
+
+ # association between Assocation -> Child
+ child: Mapped["Child"] = relationship(back_populates="parent_associations")
+
+ # association between Assocation -> Parent
+ parent: Mapped["Parent"] = relationship(back_populates="child_associations")
+
+ class Parent(Base):
+ __tablename__ = "left"
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+
+ # many-to-many relationship to Child, bypassing the `Association` class
+ children: Mapped[list["Child"]] = relationship(secondary="association", back_populates="parents")
+
+ # association between Parent -> Association -> Child
+ child_associations: Mapped[list["Association"]] = relationship(back_populates="parent")
+
+ class Child(Base):
+ __tablename__ = "right"
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+
+ # many-to-many relationship to Parent, bypassing the `Association` class
+ parents: Mapped[list["Parent"]] = relationship(secondary="association", back_populates="children")
+
+ # association between Child -> Association -> Parent
+ parent_associations: Mapped[list["Association"]] = relationship(back_populates="child")
+
+When using this ORM model to make changes, changes made to
+``Parent.children`` will not be coordinated with changes made to
+``Parent.child_associations`` or ``Child.parent_associations`` in Python;
+while all of these relationships will continue to function normally by
+themselves, changes on one will not show up in another until the
+:class:`.Session` is expired, which normally occurs automatically after
+:meth:`.Session.commit`.
+
+Additionally, if conflicting changes are made,
+such as adding a new ``Association`` object while also appending the same
+related ``Child`` to ``Parent.children``, this will raise integrity
+errors when the unit of work flush process proceeds, as in the
+example below::
+
+ p1 = Parent()
+ c1 = Child()
+ p1.children.append(c1)
+
+ # redundant, will cause a duplicate INSERT on Association
+ p1.child_associations.append(Association(child=c1))
+
+Appending ``Child`` to ``Parent.children`` directly also implies the
+creation of rows in the ``association`` table without indicating any
+value for the ``association.extra_data`` column, which will receive
+``NULL`` for its value.
+
+It's fine to use a mapping like the above if you know what you're doing; there
+may be good reason to use many-to-many relationships in the case where use
+of the "association object" pattern is infrequent, which is that it's easier to
+load relationships along a single many-to-many relationship, which can also
+optimize slightly better how the "secondary" table is used in SQL statements,
+compared to how two separate relationships to an explicit association class is
+used. It's at least a good idea to apply the
+:paramref:`_orm.relationship.viewonly` parameter
+to the "secondary" relationship to avoid the issue of conflicting
+changes occurring, as well as preventing ``NULL`` being written to the
+additional association columns, as below::
+
+ class Parent(Base):
+ __tablename__ = "left"
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+
+ # many-to-many relationship to Child, bypassing the `Association` class
+ children: Mapped[list["Child"]] = relationship(
+ secondary="association", back_populates="parents", viewonly=True
+ )
- children = relationship("Child", secondary="association")
+ # association between Parent -> Association -> Child
+ child_associations: Mapped[list["Association"]] = relationship(back_populates="parent")
+ class Child(Base):
+ __tablename__ = "right"
- class Child(Base):
- __tablename__ = "right"
- id = Column(Integer, primary_key=True)
+ id: Mapped[int] = mapped_column(primary_key=True)
- Additionally, just as changes to one relationship aren't reflected in the
- others automatically, writing the same data to both relationships will cause
- conflicting INSERT or DELETE statements as well, such as below where we
- establish the same relationship between a ``Parent`` and ``Child`` object
- twice::
+ # many-to-many relationship to Parent, bypassing the `Association` class
+ parents: Mapped[list["Parent"]] = relationship(
+ secondary="association", back_populates="children", viewonly=True
+ )
- p1 = Parent()
- c1 = Child()
- p1.children.append(c1)
+ # association between Child -> Association -> Parent
+ parent_associations: Mapped[list["Association"]] = relationship(back_populates="child")
+
+The above mapping will not write any changes to ``Parent.children`` or
+``Child.parents`` to the database, preventing conflicting writes. However, reads
+of ``Parent.children`` or ``Child.parents`` will not necessarily match the data
+that's read from ``Parent.child_associations`` or ``Child.parent_associations``,
+if changes are being made to these collections within the same transaction
+or :class:`.Session` as where the viewonly collections are being read. If
+use of the association object relationships is infrequent and is carefully
+organized against code that accesses the many-to-many collections to avoid
+stale reads (in extreme cases, making direct use of :meth:`_orm.Session.expire`
+to cause collections to be refreshed within the current transaction), the pattern may be feasible.
+
+A popular alternative to the above pattern is one where the direct many-to-many
+``Parent.children`` and ``Child.parents`` relationships are replaced with
+an extension that will transparently proxy through the ``Association``
+class, while keeping everything consistent from the ORM's point of
+view. This extension is known as the :ref:`Association Proxy <associationproxy_toplevel>`.
- # redundant, will cause a duplicate INSERT on Association
- p1.child_associations.append(Association(child=c1))
+.. seealso::
- It's fine to use a mapping like the above if you know what
- you're doing, though it may be a good idea to apply the ``viewonly=True`` parameter
- to the "secondary" relationship to avoid the issue of redundant changes
- being logged. However, to get a foolproof pattern that allows a simple
- two-object ``Parent->Child`` relationship while still using the association
- object pattern, use the association proxy extension
- as documented at :ref:`associationproxy_toplevel`.
+ :ref:`associationproxy_toplevel` - allows direct "many to many" style
+ access between parent and child for a three-class association object mapping.
.. _orm_declarative_relationship_eval:
Late-Evaluation of Relationship Arguments
------------------------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Many of the examples in the preceding sections illustrate mappings
+Most of the examples in the preceding sections illustrate mappings
where the various :func:`_orm.relationship` constructs refer to their target
-classes using a string name, rather than the class itself::
+classes using a string name, rather than the class itself, such as when
+using :class:`_orm.Mapped`, a forward reference is generated that exists
+at runtime only as a string::
class Parent(Base):
# ...
- children = relationship("Child", back_populates="parent")
+ children: Mapped[list["Child"]] = relationship(back_populates="parent")
class Child(Base):
# ...
- parent = relationship("Parent", back_populates="children")
+ parent: Mapped["Parent"] = relationship(back_populates="children")
+
+Similarly, when using non-annotated forms such as non-annotated Declarative
+or Imperative mappings, a string name is also supported directly by
+the :func:`_orm.relationship` construct::
+
+ registry.map_imperatively(
+ Parent, parent_table, properties={
+ "children": relationship("Child", back_populates="parent")
+ }
+ )
+
+ registry.map_imperatively(
+ Child, child_table, properties={
+ "parent": relationship("Parent", back_populates="children")
+ }
+ )
These string names are resolved into classes in the mapper resolution stage,
which is an internal process that occurs typically after all mappings have
class Parent(Base):
# ...
- children = relationship(
- "Child",
+ children: Mapped[list["Child"]] = relationship(
order_by="desc(Child.email_address)",
primaryjoin="Parent.id == Child.parent_id",
)
class Parent(Base):
# ...
- children = relationship(
+ children: Mapped[list["myapp.mymodel.Child"]] = relationship(
+ order_by="desc(myapp.mymodel.Child.email_address)",
+ primaryjoin="myapp.mymodel.Parent.id == myapp.mymodel.Child.parent_id",
+ )
+
+In an example like the above, the string passed to :class:`_orm.Mapped`
+can be disambiguated from a specific class argument by passing the class
+location string directly to :paramref:`_orm.relationship.argument` as well.
+Below illustrates a typing-only import for ``Child``, combined with a
+runtime specifier for the target class that will search for the correct
+name within the :class:`_orm.registry`::
+
+ import typing
+
+ if typing.TYPE_CHECKING:
+ from myapp.mymodel import Child
+
+
+ class Parent(Base):
+ # ...
+
+ children: Mapped[list["Child"]] = relationship(
"myapp.mymodel.Child",
order_by="desc(myapp.mymodel.Child.email_address)",
primaryjoin="myapp.mymodel.Parent.id == myapp.mymodel.Child.parent_id",
)
+
The qualified path can be any partial path that removes ambiguity between
the names. For example, to disambiguate between
``myapp.model1.Child`` and ``myapp.model2.Child``,
class Parent(Base):
# ...
- children = relationship(
+ children: Mapped[list["Child"]] = relationship(
"model1.Child",
order_by="desc(mymodel1.Child.email_address)",
primaryjoin="Parent.id == model1.Child.parent_id",
)
The :func:`_orm.relationship` construct also accepts Python functions or
-lambdas as input for these arguments. This has the advantage of providing
-more compile-time safety and better support for IDEs and :pep:`484` scenarios.
+lambdas as input for these arguments. A Python functional approach might look
+like the following::
-A Python functional approach might look like the following::
+ import typing
from sqlalchemy import desc
+ if typing.TYPE_CHECKING:
+ from myapplication import Child
+
def _resolve_child_model():
from myapplication import Child
class Parent(Base):
# ...
- children = relationship(
+ children: Mapped[list["Child"]] = relationship(
_resolve_child_model(),
order_by=lambda: desc(_resolve_child_model().email_address),
primaryjoin=lambda: Parent.id == _resolve_child_model().parent_id,
* :paramref:`_orm.relationship._user_defined_foreign_keys`
-.. versionchanged:: 1.3.16
-
- Prior to SQLAlchemy 1.3.16, the main :paramref:`_orm.relationship.argument`
- to :func:`_orm.relationship` was also evaluated through ``eval()`` As of
- 1.3.16 the string name is resolved from the class resolver directly without
- supporting custom Python expressions.
-
.. warning::
As stated previously, the above parameters to :func:`_orm.relationship`
are **evaluated as Python code expressions using eval(). DO NOT PASS
UNTRUSTED INPUT TO THESE ARGUMENTS.**
+Adding Relationships to Mapped Classes After Declaration
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
It should also be noted that in a similar way as described at
:ref:`orm_declarative_table_adding_columns`, any :class:`_orm.MapperProperty`
-construct can be added to a declarative base mapping at any time. If
+construct can be added to a declarative base mapping at any time
+(noting that annotated forms are not supported in this context). If
we wanted to implement this :func:`_orm.relationship` after the ``Address``
class were available, we could also apply it afterwards::
# declarative base class will intercept this and map the relationship.
Parent.children = relationship(Child, primaryjoin=Child.parent_id == Parent.id)
-.. note:: assignment of mapped properties to a declaratively mapped class will only
- function correctly if the "declarative base" class is used, which also
- provides for a metaclass-driven ``__setattr__()`` method which will
- intercept these operations. It will **not** work if the declarative
- decorator provided by :meth:`_orm.registry.mapped` is used, nor will it
- work for an imperatively mapped class mapped by
- :meth:`_orm.registry.map_imperatively`.
+As is the case for ORM mapped columns, there's no capability for
+the :class:`_orm.Mapped` annotation type to take part in this operation;
+therefore, the related class must be specified directly within the
+:func:`_orm.relationship` construct, either as the class itself, the string
+name of the class, or a callable function that returns a reference to
+the target class.
+
+.. note:: As is the case for ORM mapped columns, assignment of mapped
+ properties to an already mapped class will only
+ function correctly if the "declarative base" class is used, meaning
+ the user-defined subclass of :class:`_orm.DeclarativeBase` or the
+ dynamically generated class returned by :func:`_orm.declarative_base`
+ or :meth:`_orm.registry.generate_base`. This "base" class includes
+ a Python metaclass which implements a special ``__setattr__()`` method
+ that intercepts these operations.
+
+ Runtime assignment of class-mapped attributes to a mapped class will **not** work
+ if the class is mapped using decorators like :meth:`_orm.registry.mapped`
+ or imperative functions like :meth:`_orm.registry.map_imperatively`.
.. _orm_declarative_relationship_secondary_eval:
-Late-Evaluation for a many-to-many relationship
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Many-to-many relationships include a reference to an additional, typically non-mapped
-:class:`_schema.Table` object that is typically present in the :class:`_schema.MetaData`
-collection referred towards by the :class:`_orm.registry`. The late-evaluation
-system also includes support for having this attribute be specified as a
-string argument which will be resolved from this :class:`_schema.MetaData`
-collection. Below we specify an association table ``keyword_author``,
-sharing the :class:`_schema.MetaData` collection associated with our
-declarative base and its :class:`_orm.registry`. We can then refer to this
-:class:`_schema.Table` by name in the :paramref:`_orm.relationship.secondary`
-parameter::
-
- keyword_author = Table(
- "keyword_author",
- Base.metadata,
- Column("author_id", Integer, ForeignKey("authors.id")),
- Column("keyword_id", Integer, ForeignKey("keywords.id")),
- )
+Using a late-evaluated form for the "secondary" argument of many-to-many
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Many-to-many relationships make use of the
+:paramref:`_orm.relationship.secondary` parameter, which ordinarily
+indicates a reference to a typically non-mapped :class:`_schema.Table`
+object or other Core selectable object. Late evaluation
+using either a lambda callable or string name is supported, where string
+resolution works by evaluation of given Python expression which links
+identifier names to same-named :class:`_schema.Table` objects that
+are present in the same
+:class:`_schema.MetaData` collection referred towards by the current
+:class:`_orm.registry`.
+
+For the example given at :ref:`relationships_many_to_many`, if we assumed
+that the ``association_table`` :class:`.Table` object would be defined at a point later on in the
+module than the mapped class itself, we may write the :func:`_orm.relationship`
+using a lambda as::
+
+ class Parent(Base):
+ __tablename__ = "left"
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+ children: Mapped[list["Child"]] = relationship(
+ "Child",
+ secondary=lambda: association_table
+ )
+Or to illustrate locating the same :class:`.Table` object by name,
+the name of the :class:`.Table` is used as the argument.
+From a Python perspective, this is a Python expression evaluated as a variable
+named "association" that is resolved against the table names within
+the :class:`.MetaData` collection::
- class Author(Base):
- __tablename__ = "authors"
- id = Column(Integer, primary_key=True)
- keywords = relationship("Keyword", secondary="keyword_author")
+ class Parent(Base):
+ __tablename__ = "left"
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+ children: Mapped[list["Child"]] = relationship(secondary="association")
+
+
+.. warning:: When passed as a string,
+ :paramref:`_orm.relationship.secondary` argument is interpreted using Python's
+ ``eval()`` function, even though it's typically the name of a table.
+ **DO NOT PASS UNTRUSTED INPUT TO THIS STRING**.
-For additional detail on many-to-many relationships see the section
-:ref:`relationships_many_to_many`.
class Parent(Base):
__tablename__ = "left"
- id = Column(Integer, primary_key=True)
+ id = mapped_column(Integer, primary_key=True)
children = relationship(
"Child",
secondary=association_table,
class Child(Base):
__tablename__ = "right"
- id = Column(Integer, primary_key=True)
+ id = mapped_column(Integer, primary_key=True)
parents = relationship(
"Parent",
secondary=association_table,
class Parent(Base):
__tablename__ = "parent"
- id = Column(Integer, primary_key=True)
+ id = mapped_column(Integer, primary_key=True)
children = relationship(
"Child",
back_populates="parent",
class Child(Base):
__tablename__ = "child"
- id = Column(Integer, primary_key=True)
- parent_id = Column(Integer, ForeignKey("parent.id", ondelete="CASCADE"))
+ id = mapped_column(Integer, primary_key=True)
+ parent_id = mapped_column(Integer, ForeignKey("parent.id", ondelete="CASCADE"))
parent = relationship("Parent", back_populates="children")
The behavior of the above configuration when a parent row is deleted
class Parent(Base):
__tablename__ = "left"
- id = Column(Integer, primary_key=True)
+ id = mapped_column(Integer, primary_key=True)
children = relationship(
"Child",
secondary=association_table,
class Child(Base):
__tablename__ = "right"
- id = Column(Integer, primary_key=True)
+ id = mapped_column(Integer, primary_key=True)
parents = relationship(
"Parent",
secondary=association_table,
=============================
Mapping a one-to-many or many-to-many relationship results in a collection of
-values accessible through an attribute on the parent instance. By default,
-this collection is a ``list``::
+values accessible through an attribute on the parent instance. The two
+common collection types for these are ``list`` and ``set``, which in
+:ref:`Declarative <orm_declarative_styles_toplevel>` mappings that use
+:class:`_orm.Mapped` is established by using the collection type within
+the :class:`_orm.Mapped` container, as demonstrated in the ``Parent.children`` collection
+below where ``list`` is used::
+
+ from sqlalchemy import ForeignKey
+
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import mapped_column
+ from sqlalchemy.orm import relationship
+
+ class Base(DeclarativeBase):
+ pass
class Parent(Base):
__tablename__ = "parent"
- parent_id = Column(Integer, primary_key=True)
- children = relationship(Child)
+ parent_id: Mapped[int] = mapped_column(primary_key=True)
+
+ # use a list
+ children: Mapped[list["Child"]] = relationship()
+
+ class Child(Base):
+ __tablename__ = "child"
+ child_id: Mapped[int] = mapped_column(primary_key=True)
+ parent_id: Mapped[int] = mapped_column(ForeignKey("parent.id"))
- parent = Parent()
- parent.children.append(Child())
- print(parent.children[0])
+Or for a ``set``, illustrated in the same
+``Parent.children`` collection::
-Collections are not limited to lists. Sets, mutable sequences and almost any
-other Python object that can act as a container can be used in place of the
-default list, by specifying the :paramref:`_orm.relationship.collection_class` option on
-:func:`~sqlalchemy.orm.relationship`::
+ from sqlalchemy import ForeignKey
+
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import mapped_column
+ from sqlalchemy.orm import relationship
+
+ class Base(DeclarativeBase):
+ pass
class Parent(Base):
__tablename__ = "parent"
- parent_id = Column(Integer, primary_key=True)
+
+ parent_id: Mapped[int] = mapped_column(primary_key=True)
# use a set
- children = relationship(Child, collection_class=set)
+ children: Mapped[set["Child"]] = relationship()
+
+ class Child(Base):
+ __tablename__ = "child"
+
+ child_id: Mapped[int] = mapped_column(primary_key=True)
+ parent_id: Mapped[int] = mapped_column(ForeignKey("parent.id"))
+
+.. note:: If using Python 3.7 or 3.8, annotations for collections need
+ to use ``typing.List`` or ``typing.Set``, e.g. ``Mapped[list["Child"]]`` or
+ ``Mapped[set["Child"]]``; the ``list`` and ``set`` Python built-ins
+ don't yet support generic annotation in these Python versions, such as::
+
+ from typing import List
+
+
+ class Parent(Base):
+ __tablename__ = "parent"
+
+ parent_id: Mapped[int] = mapped_column(primary_key=True)
+
+ # use a List, Python 3.8 and earlier
+ children: Mapped[List["Child"]] = relationship()
+
+
+When using mappings without the :class:`_orm.Mapped` annotation, such as when
+using :ref:`imperative mappings <orm_imperative_mapping>` or untyped
+Python code, as well as in a few special cases, the collection class for a
+:func:`_orm.relationship` can always be specified directly using the
+:paramref:`_orm.relationship.collection_class` parameter::
+
+ # non-annotated mapping
+
+ class Parent(Base):
+ __tablename__ = "parent"
+ parent_id = mapped_column(Integer, primary_key=True)
- parent = Parent()
- child = Child()
- parent.children.add(child)
- assert child in parent.children
+ children = relationship("Child", collection_class=set)
+
+ class Child(Base):
+ __tablename__ = "child"
+
+ child_id = mapped_column(Integer, primary_key=True)
+ parent_id = mapped_column(ForeignKey("parent.id"))
+
+In the absence of :paramref:`_orm.relationship.collection_class`
+or :class:`_orm.Mapped`, the default collection type is ``list``.
+
+Beyond ``list`` and ``set`` builtins, there is also support for two varities of
+dictionary, described below at :ref:`orm_dictionary_collection`. There is also
+support for any arbitrary mutable sequence type can be set up as the target
+collection, with some additional configuration steps; this is described in the
+section :ref:`orm_custom_collection`.
+
+
+.. _orm_dictionary_collection:
Dictionary Collections
----------------------
:func:`.attribute_mapped_collection` function is by far the most common way
to achieve a simple dictionary collection. It produces a dictionary class that will apply a particular attribute
of the mapped class as a key. Below we map an ``Item`` class containing
-a dictionary of ``Note`` items keyed to the ``Note.keyword`` attribute::
+a dictionary of ``Note`` items keyed to the ``Note.keyword`` attribute.
+When using :func:`.attribute_mapped_collection`, the :class:`_orm.Mapped`
+annotation may be typed using the :class:`_orm.MappedCollection`
+type, however the :paramref:`_orm.relationship.collection_class` parameter
+is required in this case so that the :func:`.attribute_mapped_collection`
+may be appropriately parametrized::
- from sqlalchemy import Column, ForeignKey, Integer, String
- from sqlalchemy.orm import declarative_base, relationship
- from sqlalchemy.orm.collections import attribute_mapped_collection
+ from typing import Optional
- Base = declarative_base()
+ from sqlalchemy import ForeignKey
+ from sqlalchemy.orm import attribute_mapped_collection
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import mapped_column
+ from sqlalchemy.orm import relationship
+ from sqlalchemy.orm import MappedCollection
+
+
+ class Base(DeclarativeBase):
+ pass
class Item(Base):
__tablename__ = "item"
- id = Column(Integer, primary_key=True)
- notes = relationship(
- "Note",
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+
+ notes: Mapped[MappedCollection[str, "Note"]] = relationship(
collection_class=attribute_mapped_collection("keyword"),
cascade="all, delete-orphan",
)
class Note(Base):
__tablename__ = "note"
- id = Column(Integer, primary_key=True)
- item_id = Column(Integer, ForeignKey("item.id"), nullable=False)
- keyword = Column(String)
- text = Column(String)
- def __init__(self, keyword, text):
+ id: Mapped[int] = mapped_column(primary_key=True)
+ item_id: Mapped[int] = mapped_column(ForeignKey("item.id"))
+ keyword: Mapped[str]
+ text: Mapped[Optional[str]]
+
+ def __init__(self, keyword: str, text: str):
self.keyword = keyword
self.text = text
class Item(Base):
__tablename__ = "item"
- id = Column(Integer, primary_key=True)
- notes = relationship(
- "Note",
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+
+ notes: Mapped[MappedCollection[str, "Note"]] = relationship(
collection_class=attribute_mapped_collection("note_key"),
- backref="item",
+ back_populates="item",
cascade="all, delete-orphan",
)
-
class Note(Base):
__tablename__ = "note"
- id = Column(Integer, primary_key=True)
- item_id = Column(Integer, ForeignKey("item.id"), nullable=False)
- keyword = Column(String)
- text = Column(String)
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+ item_id: Mapped[int] = mapped_column(ForeignKey("item.id"))
+ keyword: Mapped[str]
+ text: Mapped[str]
+
+ item: Mapped["Item"] = relationship()
@property
def note_key(self):
return (self.keyword, self.text[0:10])
- def __init__(self, keyword, text):
+ def __init__(self, keyword: str, text: str):
self.keyword = keyword
self.text = text
-Above we added a ``Note.item`` backref. Assigning to this reverse relationship, the ``Note``
+Above we added a ``Note.item`` relationship, with a bi-directional
+:paramref:`_orm.relationship.back_populates` configuration.
+Assigning to this reverse relationship, the ``Note``
is added to the ``Item.notes`` dictionary and the key is generated for us automatically::
>>> item = Item()
which is almost like :func:`.attribute_mapped_collection` except given the :class:`_schema.Column`
object directly::
- from sqlalchemy.orm.collections import column_mapped_collection
+ from sqlalchemy.orm import column_mapped_collection
class Item(Base):
__tablename__ = "item"
- id = Column(Integer, primary_key=True)
- notes = relationship(
- "Note",
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+
+ notes: Mapped[MappedCollection[str, "Note"]] = relationship(
collection_class=column_mapped_collection(Note.__table__.c.keyword),
cascade="all, delete-orphan",
)
Note that it's usually easier to use :func:`.attribute_mapped_collection` along
with a ``@property`` as mentioned earlier::
- from sqlalchemy.orm.collections import mapped_collection
+ from sqlalchemy.orm import mapped_collection
class Item(Base):
__tablename__ = "item"
- id = Column(Integer, primary_key=True)
- notes = relationship(
- "Note",
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+
+ notes: Mapped[MappedCollection[str, "Note"]] = relationship(
collection_class=mapped_collection(lambda note: note.text[0:10]),
cascade="all, delete-orphan",
)
class A(Base):
__tablename__ = "a"
- id = Column(Integer, primary_key=True)
- bs = relationship(
- "B",
+ id: Mapped[int] = mapped_column(primary_key=True)
+
+ bs: Mapped[MappedCollection[str, "B"]] = relationship(
collection_class=attribute_mapped_collection("data"),
back_populates="a",
)
class B(Base):
__tablename__ = "b"
- id = Column(Integer, primary_key=True)
- a_id = Column(ForeignKey("a.id"))
- data = Column(String)
- a = relationship("A", back_populates="bs")
+ id: Mapped[int] = mapped_column(primary_key=True)
+ a_id: Mapped[int] = mapped_column(ForeignKey("a.id"))
+ data: Mapped[str]
+
+ a: Mapped["A"] = relationship(back_populates="bs")
Above, if we create a ``B()`` that refers to a specific ``A()``, the back
populates will then add the ``B()`` to the ``A.bs`` collection, however
.. autofunction:: mapped_collection
+.. _orm_custom_collection:
+
Custom Collection Implementations
=================================
methods in the basic dictionary interface for SQLAlchemy to use by default.
Iteration will go through ``itervalues()`` unless otherwise decorated.
-.. autoclass:: sqlalchemy.orm.collections.MappedCollection
+.. autoclass:: sqlalchemy.orm.MappedCollection
:members:
Instrumentation and Custom Types
Composite Column Types
======================
-.. note::
-
- This documentation is not yet updated to illustrate the new
- typing-annotation syntax or direct support for dataclasses.
-
-Sets of columns can be associated with a single user-defined datatype. The ORM
+Sets of columns can be associated with a single user-defined datatype,
+which in modern use is normally a Python dataclass_. The ORM
provides a single attribute which represents the group of columns using the
class you provide.
-A simple example represents pairs of columns as a ``Point`` object.
-``Point`` represents such a pair as ``.x`` and ``.y``::
-
- class Point:
- def __init__(self, x, y):
- self.x = x
- self.y = y
+A simple example represents pairs of :class:`_types.Integer` columns as a
+``Point`` object, with attributes ``.x`` and ``.y``. Using a
+dataclass, these attributes are defined with the corresponding ``int``
+Python type::
- def __composite_values__(self):
- return self.x, self.y
+ import dataclasses
- def __repr__(self):
- return f"Point(x={self.x!r}, y={self.y!r})"
+ @dataclasses.dataclass
+ class Point:
+ x: int
+ y: int
- def __eq__(self, other):
- return (
- isinstance(other, Point)
- and other.x == self.x
- and other.y == self.y
- )
+Non-dataclass forms are also accepted, but require additional methods
+to be implemented. For an example using a non-dataclass class, see the section
+:ref:`composite_legacy_no_dataclass`.
- def __ne__(self, other):
- return not self.__eq__(other)
-
-The requirements for the custom datatype class are that it have a constructor
-which accepts positional arguments corresponding to its column format, and
-also provides a method ``__composite_values__()`` which returns the state of
-the object as a list or tuple, in order of its column-based attributes. It
-also should supply adequate ``__eq__()`` and ``__ne__()`` methods which test
-the equality of two instances.
+.. versionadded:: 2.0 The :func:`_orm.composite` construct fully supports
+ Python dataclasses including the ability to derive mapped column datatypes
+ from the composite class.
We will create a mapping to a table ``vertices``, which represents two points
-as ``x1/y1`` and ``x2/y2``. These are created normally as :class:`_schema.Column`
-objects. Then, the :func:`.composite` function is used to assign new
-attributes that will represent sets of columns via the ``Point`` class::
+as ``x1/y1`` and ``x2/y2``. The ``Point`` class is associated with
+the mapped columns using the :func:`_orm.composite` construct.
+
+The example below illustrates the most modern form of :func:`_orm.composite` as
+used with a fully
+:ref:`Annotated Declarative Table <orm_declarative_mapped_column>`
+configuration. :func:`_orm.mapped_column` constructs representing each column
+are passed directly to :func:`_orm.composite`, indicating zero or more aspects
+of the columns to be generated, in this case the names; the
+:func:`_orm.composite` construct derives the column types (in this case
+``int``, corresponding to :class:`_types.Integer`) from the dataclass directly::
- from sqlalchemy import Column, Integer
- from sqlalchemy.orm import composite, declarative_base
+ from sqlalchemy.orm import DeclarativeBase, Mapped
+ from sqlalchemy.orm import composite, mapped_column
- Base = declarative_base()
+ class Base(DeclarativeBase):
+ pass
class Vertex(Base):
__tablename__ = "vertices"
- id = Column(Integer, primary_key=True)
- x1 = Column(Integer)
- y1 = Column(Integer)
- x2 = Column(Integer)
- y2 = Column(Integer)
+ id: Mapped[int] = mapped_column(primary_key=True)
- start = composite(Point, x1, y1)
- end = composite(Point, x2, y2)
+ start: Mapped[Point] = composite(mapped_column("x1"), mapped_column("y1"))
+ end: Mapped[Point] = composite(mapped_column("x2"), mapped_column("y2"))
-A classical mapping above would define each :func:`.composite`
-against the existing table::
+ def __repr__(self):
+ return f"Vertex(start={self.start}, end={self.end})"
- mapper_registry.map_imperatively(
- Vertex,
- vertices_table,
- properties={
- "start": composite(Point, vertices_table.c.x1, vertices_table.c.y1),
- "end": composite(Point, vertices_table.c.x2, vertices_table.c.y2),
- },
+The above mapping would correspond to a CREATE TABLE statement as::
+
+ >>> from sqlalchemy.schema import CreateTable
+ {sql}>>> print(CreateTable(Vertex.__table__))
+
+ CREATE TABLE vertices (
+ id INTEGER NOT NULL,
+ x1 INTEGER NOT NULL,
+ y1 INTEGER NOT NULL,
+ x2 INTEGER NOT NULL,
+ y2 INTEGER NOT NULL,
+ PRIMARY KEY (id)
)
-We can now persist and use ``Vertex`` instances, as well as query for them,
-using the ``.start`` and ``.end`` attributes against ad-hoc ``Point`` instances:
-.. sourcecode:: python+sql
+Working with Mapped Composite Column Types
+-------------------------------------------
+
+With a mapping as illustrated in the top section, we can work with the
+``Vertex`` class, where the ``.start`` and ``.end`` attributes will
+transparently refer to the columns referred towards by the ``Point`` class, as
+well as with instances of the ``Vertex`` class, where the ``.start`` and
+``.end`` attributes will refer to instances of the ``Point`` class. The ``x1``,
+``y1``, ``x2``, and ``y2`` columns are handled transparently:
+
+* **Persisting Point objects**
+
+ We can create a ``Vertex`` object, assign ``Point`` objects as members,
+ and they will be persisted as expected:
+
+ .. sourcecode:: python+sql
>>> v = Vertex(start=Point(3, 4), end=Point(5, 6))
>>> session.add(v)
- >>> q = select(Vertex).filter(Vertex.start == Point(3, 4))
- {sql}>>> print(session.scalars(q).first().start)
+ {sql}>>> session.commit()
BEGIN (implicit)
INSERT INTO vertices (x1, y1, x2, y2) VALUES (?, ?, ?, ?)
[generated in ...] (3, 4, 5, 6)
+ COMMIT
+
+* **Selecting Point objects as columns**
+
+ :func:`_orm.composite` will allow the ``Vertex.start`` and ``Vertex.end``
+ attributes to behave like a single SQL expression to as much an extent
+ as possible when using the ORM :class:`_orm.Session` (including the legacy
+ :class:`_orm.Query` object) to select ``Point`` objects:
+
+ .. sourcecode:: python+sql
+
+ >>> stmt = select(Vertex.start, Vertex.end)
+ {sql}>>> session.execute(stmt).all()
+ SELECT vertices.x1, vertices.y1, vertices.x2, vertices.y2
+ FROM vertices
+ [...] ()
+ {stop}[(Point(x=3, y=4), Point(x=5, y=6))]
+
+* **Comparing Point objects in SQL expressions**
+
+ The ``Vertex.start`` and ``Vertex.end`` attributes may be used in
+ WHERE criteria and similar, using ad-hoc ``Point`` objects for comparisons:
+
+ .. sourcecode:: python+sql
+
+ >>> stmt = select(Vertex).where(Vertex.start == Point(3, 4)).where(Vertex.end < Point(7, 8))
+ {sql}>>> session.scalars(stmt).all()
SELECT vertices.id, vertices.x1, vertices.y1, vertices.x2, vertices.y2
FROM vertices
- WHERE vertices.x1 = ? AND vertices.y1 = ?
- [generated in ...] (3, 4)
- {stop}Point(x=3, y=4)
+ WHERE vertices.x1 = ? AND vertices.y1 = ? AND vertices.x2 < ? AND vertices.y2 < ?
+ [...] (3, 4, 7, 8)
+ {stop}[Vertex(Point(x=3, y=4), Point(x=5, y=6))]
-.. autofunction:: composite
+ .. versionadded:: 2.0 :func:`_orm.composite` constructs now support
+ "ordering" comparisons such as ``<``, ``>=``, and similar, in addition
+ to the already-present support for ``==``, ``!=``.
+
+ .. tip:: The "ordering" comparison above using the "less than" operator (``<``)
+ as well as the "equality" comparison using ``==``, when used to generate
+ SQL expressions, are implemented by the :class:`_orm.Composite.Comparator`
+ class, and don't make use of the comparison methods on the composite class
+ itself, e.g. the ``__lt__()`` or ``__eq__()`` methods. From this it
+ follows that the ``Point`` dataclass above also need not implement the
+ dataclasses ``order=True`` parameter for the above SQL operations to work.
+ The section :ref:`composite_operations` contains background on how
+ to customize the comparison operations.
+
+* **Updating Point objects on Vertex Instances**
+
+ By default, the ``Point`` object **must be replaced by a new object** for
+ changes to be detected:
+
+ .. sourcecode:: python+sql
+
+ {sql}>>> v1 = session.scalars(select(Vertex)).one()
+ SELECT vertices.id, vertices.x1, vertices.y1, vertices.x2, vertices.y2
+ FROM vertices
+ [...] ()
+ {stop}
+
+ v1.end = Point(x=10, y=14)
+ {sql}session.commit()
+ UPDATE vertices SET x2=?, y2=? WHERE vertices.id = ?
+ [...] (10, 14, 1)
+ COMMIT
+
+ In order to allow in place changes on the composite object, the
+ :ref:`mutable_toplevel` extension must be used. See the section
+ :ref:`mutable_composites` for examples.
+
+
+
+.. _orm_composite_other_forms:
+
+Other mapping forms for composites
+----------------------------------
+
+The :func:`_orm.composite` construct may be passed the relevant columns
+using a :func:`_orm.mapped_column` construct, a :class:`_schema.Column`,
+or the string name of an existing mapped column. The following examples
+illustrate an equvalent mapping as that of the main section above.
+
+* Map columns directly, then pass to composite
+
+ Here we pass the existing :func:`_orm.mapped_column` instances to the
+ :func:`_orm.composite` construct, as in the non-annotated example below
+ where we also pass the ``Point`` class as the first argument to
+ :func:`_orm.composite`::
+
+ from sqlalchemy import Integer
+ from sqlalchemy.orm import mapped_column, composite
+
+ class Vertex(Base):
+ __tablename__ = "vertices"
+
+ id = mapped_column(Integer, primary_key=True)
+ x1 = mapped_column(Integer)
+ y1 = mapped_column(Integer)
+ x2 = mapped_column(Integer)
+ y2 = mapped_column(Integer)
+
+ start = composite(Point, x1, y1)
+ end = composite(Point, x2, y2)
+
+* Map columns directly, pass attribute names to composite
+
+ We can write the same example above using more annotated forms where we have
+ the option to pass attribute names to :func:`_orm.composite` instead of
+ full column constructs::
+
+ from sqlalchemy.orm import mapped_column, composite, Mapped
+
+ class Vertex(Base):
+ __tablename__ = "vertices"
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+ x1: Mapped[int]
+ y1: Mapped[int]
+ x2: Mapped[int]
+ y2: Mapped[int]
+
+ start: Mapped[Point] = composite("x1", "y1")
+ end: Mapped[Point] = composite("x2", "y2")
+
+* Imperative mapping and imperative table
+
+ When using :ref:`imperative table <orm_imperative_table_configuration>` or
+ fully :ref:`imperative <orm_imperative_mapping>` mappings, we have access
+ to :class:`_schema.Column` objects directly. These may be passed to
+ :func:`_orm.composite` as well, as in the imperative example below::
+
+ mapper_registry.map_imperatively(
+ Vertex,
+ vertices_table,
+ properties={
+ "start": composite(Point, vertices_table.c.x1, vertices_table.c.y1),
+ "end": composite(Point, vertices_table.c.x2, vertices_table.c.y2),
+ },
+ )
+
+
+.. _composite_legacy_no_dataclass:
+
+Using Legacy Non-Dataclasses
+----------------------------
+
+
+If not using a dataclass, the requirements for the custom datatype class are
+that it have a constructor
+which accepts positional arguments corresponding to its column format, and
+also provides a method ``__composite_values__()`` which returns the state of
+the object as a list or tuple, in order of its column-based attributes. It
+also should supply adequate ``__eq__()`` and ``__ne__()`` methods which test
+the equality of two instances.
+
+To illustrate the equivalent ``Point`` class from the main section
+not using a dataclass::
+
+ class Point:
+ def __init__(self, x, y):
+ self.x = x
+ self.y = y
+
+ def __composite_values__(self):
+ return self.x, self.y
+
+ def __repr__(self):
+ return f"Point(x={self.x!r}, y={self.y!r})"
+
+ def __eq__(self, other):
+ return (
+ isinstance(other, Point)
+ and other.x == self.x
+ and other.y == self.y
+ )
+
+ def __ne__(self, other):
+ return not self.__eq__(other)
+
+Usage with :func:`_orm.composite` then proceeds where the columns to be
+associated with the ``Point`` class must also be declared with explicit
+types, using one of the forms at :ref:`orm_composite_other_forms`.
Tracking In-Place Mutations on Composites
Below we illustrate the "greater than" operator, implementing
the same expression that the base "greater than" does::
- from sqlalchemy import sql
- from sqlalchemy.orm.properties import CompositeProperty
+ import dataclasses
+
+ from sqlalchemy.orm import composite
+ from sqlalchemy.orm import CompositeProperty
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import mapped_column
+ from sqlalchemy.sql import and_
+
+ @dataclasses.dataclass
+ class Point:
+ x: int
+ y: int
class PointComparator(CompositeProperty.Comparator):
def __gt__(self, other):
"""redefine the 'greater than' operation"""
- return sql.and_(
+ return and_(
*[
a > b
for a, b in zip(
self.__clause_element__().clauses,
- other.__composite_values__(),
+ dataclasses.astuple(other),
)
]
)
+ class Base(DeclarativeBase):
+ pass
class Vertex(Base):
- ___tablename__ = "vertices"
+ __tablename__ = "vertices"
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+
+ start: Mapped[Point] = composite(
+ mapped_column("x1"),
+ mapped_column("y1"),
+ comparator_factory=PointComparator
+ )
+ end: Mapped[Point] = composite(
+ mapped_column("x2"),
+ mapped_column("y2"),
+ comparator_factory=PointComparator
+ )
+
+Since ``Point`` is a dataclass, we may make use of
+``dataclasses.astuple()`` to get a tuple form of ``Point`` instances.
+
+The custom comparator then returns the appropriate SQL expression::
- id = Column(Integer, primary_key=True)
- x1 = Column(Integer)
- y1 = Column(Integer)
- x2 = Column(Integer)
- y2 = Column(Integer)
+ >>> print(Vertex.start > Point(5, 6))
+ vertices.x1 > :x1_1 AND vertices.y1 > :y1_1
- start = composite(Point, x1, y1, comparator_factory=PointComparator)
- end = composite(Point, x2, y2, comparator_factory=PointComparator)
Nesting Composites
-------------------
Composite objects can be defined to work in simple nested schemes, by
redefining behaviors within the composite class to work as desired, then
mapping the composite class to the full length of individual columns normally.
-Typically, it is convenient to define separate constructors for user-defined
-use and generate-from-row use. Below we reorganize the ``Vertex`` class to
-itself be a composite object, which is then mapped to a class ``HasVertex``::
+This requires that additional methods to move between the "nested" and
+"flat" forms are defined.
+
+Below we reorganize the ``Vertex`` class to itself be a composite object which
+refers to ``Point`` objects. ``Vertex`` and ``Point`` can be dataclasses,
+however we will add a custom construction method to ``Vertex`` that can be used
+to create new ``Vertex`` objects given four column values, which will will
+arbitrarily name ``_generate()`` and define as a classmethod so that we can
+make new ``Vertex`` objects by passing values to the ``Vertex._generate()``
+method.
+
+We will also implement the ``__composite_values__()`` method, which is a fixed
+name recognized by the :func:`_orm.composite` construct (introduced previously
+at :ref:`composite_legacy_no_dataclass`) that indicates a standard way of
+receiving the object as a flat tuple of column values, which in this case will
+supersede the usual dataclass-oriented methodology.
+
+With our custom ``_generate()`` constructor and
+``__composite_values__()`` serializer method, we can now move between
+a flat tuple of columns and ``Vertex`` objects that contain ``Point``
+instances. The ``Vertex._generate`` method is passed as the
+first argument to the :func:`_orm.composite` construct as the source of new
+``Vertex`` instances, and the ``__composite_values__()`` method will be
+used implicitly by :func:`_orm.composite`.
+
+For the purposes of the example, the ``Vertex`` composite is then mapped to a
+class called ``HasVertex``, which is where the :class:`.Table` containing the
+four source columns ultimately resides::
+
+ import dataclasses
from sqlalchemy.orm import composite
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import mapped_column
-
+ @dataclasses.dataclass
class Point:
- def __init__(self, x, y):
- self.x = x
- self.y = y
-
- def __composite_values__(self):
- return self.x, self.y
-
- def __repr__(self):
- return f"Point(x={self.x!r}, y={self.y!r})"
-
- def __eq__(self, other):
- return (
- isinstance(other, Point)
- and other.x == self.x
- and other.y == self.y
- )
-
- def __ne__(self, other):
- return not self.__eq__(other)
+ x: int
+ y: int
+ @dataclasses.dataclass
class Vertex:
- def __init__(self, start, end):
- self.start = start
- self.end = end
+ start: Point
+ end: Point
@classmethod
def _generate(self, x1, y1, x2, y2):
return Vertex(Point(x1, y1), Point(x2, y2))
def __composite_values__(self):
+ """generate a row from a Vertex"""
return (
- self.start.__composite_values__()
- + self.end.__composite_values__()
+ dataclasses.astuple(self.start) + dataclasses.astuple(self.end)
)
+ class Base(DeclarativeBase):
+ pass
+
class HasVertex(Base):
__tablename__ = "has_vertex"
- id = Column(Integer, primary_key=True)
- x1 = Column(Integer)
- y1 = Column(Integer)
- x2 = Column(Integer)
- y2 = Column(Integer)
+ id: Mapped[int] = mapped_column(primary_key=True)
+ x1: Mapped[int]
+ y1: Mapped[int]
+ x2: Mapped[int]
+ y2: Mapped[int]
- vertex = composite(Vertex._generate, x1, y1, x2, y2)
+ vertex: Mapped[Vertex] = composite(Vertex._generate, "x1", "y1", "x2", "y2")
-We can then use the above mapping as::
+The above mapping can then be used in terms of ``HasVertex``, ``Vertex``, and
+``Point``::
hv = HasVertex(vertex=Vertex(Point(1, 2), Point(3, 4)))
- s.add(hv)
- s.commit()
+ session.add(hv)
+ session.commit()
- hv = s.scalars(
- select(HasVertex).filter(
- HasVertex.vertex == Vertex(Point(1, 2), Point(3, 4))
- )
- ).first()
+ stmt = select(HasVertex).where(
+ HasVertex.vertex == Vertex(Point(1, 2), Point(3, 4))
+ )
+
+ hv = session.scalars(stmt).first()
print(hv.vertex.start)
print(hv.vertex.end)
+.. _dataclass: https://docs.python.org/3/library/dataclasses.html
+
+Composite API
+-------------
+
+.. autofunction:: composite
+
Integration with dataclasses and attrs
======================================
-SQLAlchemy 1.4 has limited support for ORM mappings that are established
-against classes that have already been pre-instrumented using either Python's
-built-in dataclasses_ library or the attrs_ third party integration library.
+SQLAlchemy as of version 2.0 features "native dataclass" integration where
+an :ref:`Annotated Declarative Table <orm_declarative_mapped_column>`
+mapping may be turned into a Python dataclass_ by adding a single mixin
+or decorator to mapped classes.
+
+.. versionadded:: 2.0 Integrated dataclass creation with ORM Declarative classes
+
+There are also patterns available that allow existing dataclasses to be
+mapped, as well as to map classes instrumented by the
+attrs_ third party integration library.
+
+.. _orm_declarative_native_dataclasses:
+
+Declarative Dataclass Mapping
+-------------------------------
+
+SQLAlchemy :ref:`Annotated Declarative Table <orm_declarative_mapped_column>`
+mappings may be augmented with an additional
+mixin class or decorator directive, which will add an additional step to
+the Declarative process after the mapping is complete that will convert
+the mapped class **in-place** into a Python dataclass_, before completing
+the mapping process which applies ORM-specific :term:`instrumentation`
+to the class. The most prominent behavioral addition this provides is
+generation of an ``__init__()`` method with fine-grained control over
+positional and keyword arguments with or without defaults, as well as
+generation of methods like ``__repr__()`` and ``__eq__()``.
+
+From a :pep:`484` typing perspective, the class is recognized
+as having Dataclass-specific behaviors, most notably by taking advantage of :pep:`681`
+"Dataclass Transforms", which allows typing tools to consider the class
+as though it were explicitly decorated using the ``@dataclasses.dataclass``
+decorator.
+
+.. note:: Support for :pep:`681` in typing tools as of **July 3, 2022** is
+ limited and is currently known to be supported by Pyright_, but not yet
+ Mypy_. When :pep:`681` is not supported, typing tools will see the
+ ``__init__()`` constructor provided by the :class:`_orm.DeclarativeBase`
+ superclass, if used, else will see the constructor as untyped.
+
+Dataclass conversion may be added to any Declarative class either by adding the
+:class:`_orm.MappedAsDataclass` mixin to a :class:`_orm.DeclarativeBase` class
+hierarchy, or for decorator mapping by using the
+:meth:`_orm.registry.mapped_as_dataclass` class decorator.
+
+The :class:`_orm.MappedAsDataclass` mixin may be applied either
+to the Declarative ``Base`` class or any superclass, as in the example
+below::
+
+
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import mapped_column
+ from sqlalchemy.orm import MappedAsDataclass
+
+
+ class Base(MappedAsDataclass, DeclarativeBase):
+ """subclasses will be converted to dataclasses"""
+
+ class User(Base):
+ __tablename__ = "user_account"
+
+ id: Mapped[int] = mapped_column(init=False, primary_key=True)
+ name: Mapped[str]
+
+Or may be applied directly to classes that extend from the Declarative base::
+
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import mapped_column
+ from sqlalchemy.orm import MappedAsDataclass
+
+
+ class Base(DeclarativeBase):
+ pass
+
+ class User(MappedAsDataclass, Base):
+ """User class will be converted to a dataclass"""
+
+ __tablename__ = "user_account"
+
+ id: Mapped[int] = mapped_column(init=False, primary_key=True)
+ name: Mapped[str]
+
+When using the decorator form, only the :meth:`_orm.registry.mapped_as_dataclass`
+decorator is supported::
+
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import mapped_column
+ from sqlalchemy.orm import registry
+
+
+ reg = registry()
+
+ @reg.mapped_as_dataclass
+ class User:
+ __tablename__ = "user_account"
+
+ id: Mapped[int] = mapped_column(init=False, primary_key=True)
+ name: Mapped[str]
+
+Class level feature configuration
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Support for dataclasses features is partial. Currently **supported** are
+the ``init``, ``repr``, ``eq``, ``order`` and ``unsafe_hash`` features.
+Currently **not supported** are the ``frozen``, ``slots``, ``match_args``,
+and ``kw_only`` features.
+
+When using the mixin class form with :class:`_orm.MappedAsDataclass`,
+class configuration arguments are passed as class-level parameters::
+
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import mapped_column
+ from sqlalchemy.orm import MappedAsDataclass
+
+
+ class Base(DeclarativeBase):
+ pass
+
+ class User(MappedAsDataclass, Base, repr=False, unsafe_hash=True):
+ """User class will be converted to a dataclass"""
+
+ __tablename__ = "user_account"
+
+ id: Mapped[int] = mapped_column(init=False, primary_key=True)
+ name: Mapped[str]
+
+When using the decorator form with :meth:`_orm.registry.mapped_as_dataclass`,
+class configuration arguments are passed to the decorator directly::
+
+ from sqlalchemy.orm import registry
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import mapped_column
+
+
+ reg = registry()
+
+
+ @reg.mapped_as_dataclass(unsafe_hash=True)
+ class User:
+ """User class will be converted to a dataclass"""
+
+ __tablename__ = "user_account"
+
+ id: Mapped[int] = mapped_column(init=False, primary_key=True)
+ name: Mapped[str]
+
+For background on dataclass class options, see the dataclasses_ documentation
+at `@dataclasses.dataclass <https://docs.python.org/3/library/dataclasses.html#dataclasses.dataclass>`_.
+
+Attribute Configuration
+^^^^^^^^^^^^^^^^^^^^^^^
+
+SQLAlchemy native dataclasses differ from normal dataclasses in that
+attributes to be mapped are described using the :class:`_orm.Mapped`
+generic annotation container in all cases. Mappings follow the same
+forms as those documented at :ref:`orm_declarative_table`, and all
+features of :func:`_orm.mapped_column` and :class:`_orm.Mapped` are supported.
+
+Additionally, ORM attribute configuration constructs including
+:func:`_orm.mapped_column`, :func:`_orm.relationship` and :func:`_orm.composite`
+support **per-attribute field options**, including ``init``, ``default``,
+``default_factory`` and ``repr``. The names of these arguments is fixed
+as specified in :pep:`681`. Functionality is equivalent to dataclasses:
+
+* ``init``, as in :paramref:`_orm.mapped_column.init`,
+ :paramref:`_orm.relationship.init`, if False indicates the field should
+ not be part of the ``__init__()`` method
+* ``default``, as in :paramref:`_orm.mapped_column.default`,
+ :paramref:`_orm.relationship.default`
+ indicates a default value for the field as given as a keyword argument
+ in the ``__init__()`` method.
+* ``default_factory``, as in :paramref:`_orm.mapped_column.default_factory`,
+ :paramref:`_orm.relationship.default_factory`, indicates a callable function
+ that will be invoked to generate a new default value for a parameter
+ if not passed explicitly to the ``__init__()`` method.
+* ``repr`` True by default, indicates the field should be part of the generated
+ ``__repr__()`` method
+
+
+Another key difference from dataclasses is that default values for attributes
+**must** be configured using the ``default`` parameter of the ORM construct,
+such as ``mapped_column(default=None)``. A syntax that resembles dataclass
+syntax which accepts simple Python values as defaults without using
+``@dataclases.field()`` is not supported.
+
+As an example using :func:`_orm.mapped_column`, the mapping below will
+produce an ``__init__()`` method that accepts only the fields ``name`` and
+``fullname``, where ``name`` is required and may be passed positionally,
+and ``fullname`` is optional. The ``id`` field, which we expect to be
+database-generated, is not part of the constructor at all::
+
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import mapped_column
+ from sqlalchemy.orm import registry
+
+ reg = registry()
+
+ @reg.mapped_as_dataclass
+ class User:
+ __tablename__ = "user_account"
+
+ id: Mapped[int] = mapped_column(init=False, primary_key=True)
+ name: Mapped[str]
+ fullname: Mapped[str] = mapped_column(default=None)
+
+ # 'fullname' is optional keyword argument
+ u1 = User('name')
+
+Column Defaults
+~~~~~~~~~~~~~~~
+
+In order to accommodate the name overlap of the ``default`` argument with
+the existing :paramref:`_schema.Column.default` parameter of the :class:`_schema.Column`
+construct, the :func:`_orm.mapped_column` construct disambiguates the two
+names by adding a new parameter :paramref:`_orm.mapped_column.insert_default`,
+which will be populated directly into the
+:paramref:`_schema.Column.default` parameter of :class:`_schema.Column`,
+independently of what may be set on
+:paramref:`_orm.mapped_column.default`, which is always used for the
+dataclasses configuration. For example, to configure a datetime column with
+a :paramref:`_schema.Column.default` set to the ``func.utc_timestamp()`` SQL function,
+but where the parameter is optional in the constructor::
+
+ from datetime import datetime
+
+ from sqlalchemy import func
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import mapped_column
+ from sqlalchemy.orm import registry
+
+ reg = registry()
+
+ @reg.mapped_as_dataclass
+ class User:
+ __tablename__ = "user_account"
+
+ id: Mapped[int] = mapped_column(init=False, primary_key=True)
+ created_at: Mapped[datetime] = mapped_column(
+ insert_default=func.utc_timestamp(),
+ default=None
+ )
+
+With the above mapping, an ``INSERT`` for a new ``User`` object where no
+parameter for ``created_at`` were passed proceeds as:
+
+.. sourcecode:: pycon+sql
+
+ >>> with Session(e) as session:
+ ... session.add(User())
+ {sql}... session.commit()
+ BEGIN (implicit)
+ INSERT INTO user_account (created_at) VALUES (utc_timestamp())
+ [generated in 0.00010s] ()
+ COMMIT
+
+
+
+Integration with Annotated
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The approach introduced at :ref:`orm_declarative_mapped_column_pep593` illustrates
+how to use :pep:`593` ``Annotated`` objects to package whole
+:func:`_orm.mapped_column` constructs for re-use. This feature is supported
+with the dataclasses feature. One aspect of the feature however requires
+a workaround when working with typing tools, which is that the
+:pep:`681`-specific arguments ``init``, ``default``, ``repr``, and ``default_factory``
+**must** be on the right hand side, packaged into an explicit :func:`_orm.mapped_column`
+construct, in order for the typing tool to interpret the attribute correctly.
+As an example, the approach below will work perfectly fine at runtime,
+however typing tools will consider the ``User()`` construction to be
+invalid, as they do not see the ``init=False`` parameter present::
+
+ from typing import Annotated
+
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import mapped_column
+ from sqlalchemy.orm import registry
+
+ # typing tools will ignore init=False here
+ intpk = Annotated[int, mapped_column(init=False, primary_key=True)]
+
+ reg = registry()
+
+ @reg.mapped_as_dataclass
+ class User:
+ __tablename__ = "user_account"
+ id: Mapped[intpk]
+
+ # typing error: Argument missing for parameter "id"
+ u1 = User()
+
+Instead, :func:`_orm.mapped_column` must be present on the right side
+as well with an explicit setting for :paramref:`_orm.mapped_column.init`;
+the other arguments can remain within the ``Annotated`` construct::
+
+ from typing import Annotated
+
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import mapped_column
+ from sqlalchemy.orm import registry
+
+ intpk = Annotated[int, mapped_column(primary_key=True)]
+
+ reg = registry()
+
+ @reg.mapped_as_dataclass
+ class User:
+ __tablename__ = "user_account"
+
+ # init=False and other pep-681 arguments must be inline
+ id: Mapped[intpk] = mapped_column(init=False)
+
+
+ u1 = User()
+
+Relationship Configuration
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The :class:`_orm.Mapped` annotation in combination with
+:func:`_orm.relationship` is used in the same way as described at
+:ref:`relationship_patterns`. When specifying a collection-based
+:func:`_orm.relationship` as an optional keyword argument, the
+:paramref:`_orm.relationship.default_factory` parameter must be passed and it
+must refer to the collection class that's to be used. Many-to-one and
+scalar object references may make use of
+:paramref:`_orm.relationship.default` if the default value is to be ``None``::
+
+ from typing import List
+
+ from sqlalchemy import ForeignKey
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import mapped_column
+ from sqlalchemy.orm import registry
+ from sqlalchemy.orm import relationship
+
+ reg = registry()
+
+ @reg.mapped_as_dataclass
+ class Parent:
+ __tablename__ = "parent"
+ id: Mapped[int] = mapped_column(primary_key=True)
+ children: Mapped[List["Child"]] = relationship(default_factory=list, back_populates='parent')
+
+
+ @reg.mapped_as_dataclass
+ class Child:
+ __tablename__ = "child"
+ id: Mapped[int] = mapped_column(primary_key=True)
+ parent_id: Mapped[int] = mapped_column(ForeignKey("parent.id"))
+ parent: Mapped["Parent"] = relationship(default=None)
+
+The above mapping will generate an empty list for ``Parent.children`` when a
+new ``Parent()`` object is constructed without passing ``children``, and
+similarly a ``None`` value for ``Child.parent`` when a new ``Child()`` object
+is constructed without passsing ``parent``.
+
+While the :paramref:`_orm.relationship.default_factory` can be automatically
+derived from the given collection class of the :func:`_orm.relationship`
+itself, this would break compatibility with dataclasses, as the presence
+of :paramref:`_orm.relationship.default_factory` or
+:paramref:`_orm.relationship.default` is what determines if the parameter is
+to be required or optional when rendered into the ``__init__()`` method.
+
-.. tip:: SQLAlchemy 2.0 will include a new dataclass integration feature which
- allows for a particular class to be mapped and converted into a Python
- dataclass simultaneously, with full support for SQLAlchemy's declarative
- syntax. Within the scope of the 1.4 release, the ``@dataclass`` decorator
- is used separately as documented in this section.
.. _orm_declarative_dataclasses:
Applying ORM Mappings to an existing dataclass
----------------------------------------------
-The dataclasses_ module, added in Python 3.7, provides a ``@dataclass`` class
-decorator to automatically generate boilerplate definitions of common object
-methods including ``__init__()``, ``__repr()__``, and other methods. SQLAlchemy
-supports the application of ORM mappings to a class after it has been processed
-with the ``@dataclass`` decorator, by using either the
+SQLAlchemy's :ref:`native dataclass <orm_declarative_native_dataclasses>`
+support builds upon the previous version of the feature first introduced in
+SQLAlchemy 1.4, which supports the application of ORM mappings to a class after
+it has been processed with the ``@dataclass`` decorator, by using either the
:meth:`_orm.registry.mapped` class decorator, or the
:meth:`_orm.registry.map_imperatively` method to apply ORM mappings to the
-class using Imperative.
+class using Imperative. This approach is still viable for applications that are
+using partially or fully imperative mapping forms with dataclasses.
+
+For fully Declarative mapping combined with dataclasses, the
+:ref:`orm_declarative_native_dataclasses` approach should be preferred.
.. versionadded:: 1.4 Added support for direct mapping of Python dataclasses
Mapping dataclasses using Declarative Mapping
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+.. deprecated:: 2.0 This approach to Declarative mapping with
+ dataclasses should be considered as legacy. It will remain supported
+ however is unlikely to offer any advantages against the new
+ approach detailed at :ref:`orm_declarative_native_dataclasses`.
+
The fully declarative approach requires that :class:`_schema.Column` objects
are declared as class attributes, which when using dataclasses would conflict
with the dataclass-level attributes. An approach to combine these together
__sa_dataclass_metadata_key__ = "sa"
id: int = field(
- init=False, metadata={"sa": Column(Integer, primary_key=True)}
+ init=False, metadata={"sa": mapped_column(Integer, primary_key=True)}
)
- name: str = field(default=None, metadata={"sa": Column(String(50))})
- fullname: str = field(default=None, metadata={"sa": Column(String(50))})
- nickname: str = field(default=None, metadata={"sa": Column(String(12))})
+ name: str = field(default=None, metadata={"sa": mapped_column(String(50))})
+ fullname: str = field(default=None, metadata={"sa": mapped_column(String(50))})
+ nickname: str = field(default=None, metadata={"sa": mapped_column(String(12))})
addresses: List[Address] = field(
default_factory=list, metadata={"sa": relationship("Address")}
)
__tablename__ = "address"
__sa_dataclass_metadata_key__ = "sa"
id: int = field(
- init=False, metadata={"sa": Column(Integer, primary_key=True)}
+ init=False, metadata={"sa": mapped_column(Integer, primary_key=True)}
)
user_id: int = field(
- init=False, metadata={"sa": Column(ForeignKey("user.id"))}
+ init=False, metadata={"sa": mapped_column(ForeignKey("user.id"))}
)
email_address: str = field(
- default=None, metadata={"sa": Column(String(50))}
+ default=None, metadata={"sa": mapped_column(String(50))}
)
+.. _orm_declarative_dataclasses_mixin:
+
+Using Declarative Mixins with Dataclasses
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In the section :ref:`orm_mixins_toplevel`, Declarative Mixin classes
+are introduced. One requirement of declarative mixins is that certain
+constructs that can't be easily duplicated must be given as callables,
+using the :class:`_orm.declared_attr` decorator, such as in the
+example at :ref:`orm_declarative_mixins_relationships`::
+
+ class RefTargetMixin:
+ @declared_attr
+ def target_id(cls):
+ return mapped_column("target_id", ForeignKey("target.id"))
+
+ @declared_attr
+ def target(cls):
+ return relationship("Target")
+
+This form is supported within the Dataclasses ``field()`` object by using
+a lambda to indicate the SQLAlchemy construct inside the ``field()``.
+Using :func:`_orm.declared_attr` to surround the lambda is optional.
+If we wanted to produce our ``User`` class above where the ORM fields
+came from a mixin that is itself a dataclass, the form would be::
+
+ @dataclass
+ class UserMixin:
+ __tablename__ = "user"
+
+ __sa_dataclass_metadata_key__ = "sa"
+
+ id: int = field(
+ init=False, metadata={"sa": mapped_column(Integer, primary_key=True)}
+ )
+
+ addresses: List[Address] = field(
+ default_factory=list, metadata={"sa": lambda: relationship("Address")}
+ )
+
+
+ @dataclass
+ class AddressMixin:
+ __tablename__ = "address"
+ __sa_dataclass_metadata_key__ = "sa"
+ id: int = field(
+ init=False, metadata={"sa": mapped_column(Integer, primary_key=True)}
+ )
+ user_id: int = field(
+ init=False, metadata={"sa": lambda: mapped_column(ForeignKey("user.id"))}
+ )
+ email_address: str = field(
+ default=None, metadata={"sa": mapped_column(String(50))}
+ )
+
+
+ @mapper_registry.mapped
+ class User(UserMixin):
+ pass
+
+
+ @mapper_registry.mapped
+ class Address(AddressMixin):
+ pass
+
+.. versionadded:: 1.4.2 Added support for "declared attr" style mixin attributes,
+ namely :func:`_orm.relationship` constructs as well as :class:`_schema.Column`
+ objects with foreign key declarations, to be used within "Dataclasses
+ with Declarative Table" style mappings.
+
+
+
.. _orm_imperative_dataclasses:
Mapping dataclasses using Imperative Mapping
mapper_registry.map_imperatively(Address, address)
-.. _orm_declarative_dataclasses_mixin:
-
-Using Declarative Mixins with Dataclasses
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-In the section :ref:`orm_mixins_toplevel`, Declarative Mixin classes
-are introduced. One requirement of declarative mixins is that certain
-constructs that can't be easily duplicated must be given as callables,
-using the :class:`_orm.declared_attr` decorator, such as in the
-example at :ref:`orm_declarative_mixins_relationships`::
-
- class RefTargetMixin:
- @declared_attr
- def target_id(cls):
- return Column("target_id", ForeignKey("target.id"))
-
- @declared_attr
- def target(cls):
- return relationship("Target")
-
-This form is supported within the Dataclasses ``field()`` object by using
-a lambda to indicate the SQLAlchemy construct inside the ``field()``.
-Using :func:`_orm.declared_attr` to surround the lambda is optional.
-If we wanted to produce our ``User`` class above where the ORM fields
-came from a mixin that is itself a dataclass, the form would be::
-
- @dataclass
- class UserMixin:
- __tablename__ = "user"
-
- __sa_dataclass_metadata_key__ = "sa"
-
- id: int = field(
- init=False, metadata={"sa": Column(Integer, primary_key=True)}
- )
-
- addresses: List[Address] = field(
- default_factory=list, metadata={"sa": lambda: relationship("Address")}
- )
-
-
- @dataclass
- class AddressMixin:
- __tablename__ = "address"
- __sa_dataclass_metadata_key__ = "sa"
- id: int = field(
- init=False, metadata={"sa": Column(Integer, primary_key=True)}
- )
- user_id: int = field(
- init=False, metadata={"sa": lambda: Column(ForeignKey("user.id"))}
- )
- email_address: str = field(
- default=None, metadata={"sa": Column(String(50))}
- )
-
-
- @mapper_registry.mapped
- class User(UserMixin):
- pass
-
-
- @mapper_registry.mapped
- class Address(AddressMixin):
- pass
-
-.. versionadded:: 1.4.2 Added support for "declared attr" style mixin attributes,
- namely :func:`_orm.relationship` constructs as well as :class:`_schema.Column`
- objects with foreign key declarations, to be used within "Dataclasses
- with Declarative Table" style mappings.
-
-
.. _orm_declarative_attrs_imperative_table:
+.. _dataclass: https://docs.python.org/3/library/dataclasses.html
.. _dataclasses: https://docs.python.org/3/library/dataclasses.html
.. _attrs: https://pypi.org/project/attrs/
+.. _mypy: https://mypy.readthedocs.io/en/stable/
+.. _pyright: https://github.com/microsoft/pyright
--------------------------------------------
The examples given at :ref:`orm_declarative_table_config_toplevel`
-illustrate mappings against table-bound columns;
-the mapping of an individual column to an ORM class attribute is represented
-internally by the :class:`_orm.ColumnProperty` construct. There are many
-other varieties of mapper properties, the most common being the
+illustrate mappings against table-bound columns, using the :func:`_orm.mapped_column`
+construct. There are several other varieties of ORM mapped constructs
+that may be configured besides table-bound columns, the most common being the
:func:`_orm.relationship` construct. Other kinds of properties include
-synonyms to columns which are defined using the :func:`_orm.synonym`
-construct, SQL expressions that are defined using the :func:`_orm.column_property`
-construct, and deferred columns and SQL expressions which load only when
-accessed, defined using the :func:`_orm.deferred` construct.
+SQL expressions that are defined using the :func:`_orm.column_property`
+construct and multiple-column mappings using the :func:`_orm.composite`
+construct.
While an :ref:`imperative mapping <orm_imperative_mapping>` makes use of
the :ref:`properties <orm_mapping_properties>` dictionary to establish
:class:`_schema.Table` object.
Working with the example mapping of ``User`` and ``Address``, we may illustrate
-a declarative table mapping that includes not just :class:`_schema.Column`
+a declarative table mapping that includes not just :func:`_orm.mapped_column`
objects but also relationships and SQL expressions::
- # mapping attributes using declarative with declarative table
- # i.e. __tablename__
+ from typing import List
+ from typing import Optional
+
+ from sqlalchemy import Column
+ from sqlalchemy import ForeignKey
+ from sqlalchemy import String
+ from sqlalchemy import Text
+ from sqlalchemy.orm import column_property
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import mapped_column
+ from sqlalchemy.orm import relationship
- from sqlalchemy import Column, ForeignKey, Integer, String, Text
- from sqlalchemy.orm import (
- column_property,
- declarative_base,
- deferred,
- relationship,
- )
- Base = declarative_base()
+ class Base(DeclarativeBase):
+ pass
class User(Base):
__tablename__ = "user"
- id = Column(Integer, primary_key=True)
- name = Column(String)
- firstname = Column(String(50))
- lastname = Column(String(50))
-
- fullname = column_property(firstname + " " + lastname)
+ id: Mapped[int] = mapped_column(primary_key=True)
+ name: Mapped[str]
+ firstname: Mapped[str] = mapped_column(String(50))
+ lastname: Mapped[str] = mapped_column(String(50))
+ fullname: Mapped[str] = column_property(firstname + " " + lastname)
- addresses = relationship("Address", back_populates="user")
+ addresses: Mapped[List["Address"]] = relationship(back_populates="user")
class Address(Base):
__tablename__ = "address"
- id = Column(Integer, primary_key=True)
- user_id = Column(ForeignKey("user.id"))
- email_address = Column(String)
- address_statistics = deferred(Column(Text))
+ id: Mapped[int] = mapped_column(primary_key=True)
+ user_id: Mapped[int] = mapped_column(ForeignKey("user.id"))
+ email_address: Mapped[str]
+ address_statistics: Mapped[Optional[str]] = mapped_column(
+ Text, deferred=True
+ )
- user = relationship("User", back_populates="addresses")
+ user: Mapped["User"] = relationship(back_populates="addresses")
The above declarative table mapping features two tables, each with a
:func:`_orm.relationship` referring to the other, as well as a simple
SQL expression mapped by :func:`_orm.column_property`, and an additional
-:class:`_schema.Column` that will be loaded on a "deferred" basis as defined
-by the :func:`_orm.deferred` construct. More documentation
+:func:`_orm.mapped_column` that indicates loading should be on a
+"deferred" basis as defined
+by the :paramref:`_orm.mapped_column.deferred` keyword. More documentation
on these particular concepts may be found at :ref:`relationship_patterns`,
:ref:`mapper_column_property_sql_expressions`, and :ref:`deferred`.
# i.e. __table__
from sqlalchemy import Column, ForeignKey, Integer, String, Table, Text
- from sqlalchemy.orm import (
- column_property,
- declarative_base,
- deferred,
- relationship,
- )
+ from sqlalchemy.orm import column_property
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import deferred
+ from sqlalchemy.orm import relationship
- Base = declarative_base()
+ class Base(DeclarativeBase):
+ pass
class User(Base):
class GroupUsers(Base):
__tablename__ = 'group_users'
- user_id = Column(String(40))
- group_id = Column(String(40))
+ user_id = mapped_column(String(40))
+ group_id = mapped_column(String(40))
__mapper_args__ = {
"primary_key": [user_id, group_id]
class Widget(Base):
__tablename__ = "widgets"
- id = Column(Integer, primary_key=True)
- timestamp = Column(DateTime, nullable=False)
+ id = mapped_column(Integer, primary_key=True)
+ timestamp = mapped_column(DateTime, nullable=False)
__mapper_args__ = {
"version_id_col": timestamp,
class Person(Base):
__tablename__ = "person"
- person_id = Column(Integer, primary_key=True)
- type = Column(String, nullable=False)
+ person_id = mapped_column(Integer, primary_key=True)
+ type = mapped_column(String, nullable=False)
__mapper_args__ = dict(
polymorphic_on=type,
from sqlalchemy import Integer
from sqlalchemy import select
from sqlalchemy import String
- from sqlalchemy.orm import declarative_base
+ from sqlalchemy.orm import DeclarativeBase
from sqlalchemy.orm import declared_attr
]
}
- Base = declarative_base()
+ class Base(DeclarativeBase):
+ pass
class SomeClass(ExcludeColsWFlag, Base):
__tablename__ = 'some_table'
- id = Column(Integer, primary_key=True)
- data = Column(String)
- not_needed = Column(String, info={"exclude": True})
+ id = mapped_column(Integer, primary_key=True)
+ data = mapped_column(String)
+ not_needed = mapped_column(String, info={"exclude": True})
Above, the ``ExcludeColsWFlag`` mixin provides a per-class ``__mapper_args__``
The :class:`_schema.MetaData` collection normally used to assign a new
:class:`_schema.Table` is the :attr:`_orm.registry.metadata` attribute
associated with the :class:`_orm.registry` object in use. When using a
-declarative base class such as that generated by :func:`_orm.declarative_base`
-as well as :meth:`_orm.registry.generate_base`, this :class:`_schema.MetaData`
-is also normally present also as an attribute named ``.metadata`` that's
-directly on the base class, and thus also on the mapped class via
-inheritance. Declarative uses this attribute, when present, in order to
-determine the target :class:`_schema.MetaData` collection, or if not
+declarative base class such as that produced by the
+:class:`_orm.DeclarativeBase` superclass, as well as legacy functions such as
+:func:`_orm.declarative_base` and :meth:`_orm.registry.generate_base`, this
+:class:`_schema.MetaData` is also normally present as an attribute named
+``.metadata`` that's directly on the base class, and thus also on the mapped
+class via inheritance. Declarative uses this attribute, when present, in order
+to determine the target :class:`_schema.MetaData` collection, or if not
present, uses the :class:`_schema.MetaData` associated directly with the
:class:`_orm.registry`.
class ClassOne:
__tablename__ = "t1" # will use reg.metadata
- id = Column(Integer, primary_key=True)
+ id = mapped_column(Integer, primary_key=True)
@reg.mapped
class ClassTwo(BaseOne):
__tablename__ = "t1" # will use BaseOne.metadata
- id = Column(Integer, primary_key=True)
+ id = mapped_column(Integer, primary_key=True)
@reg.mapped
class ClassThree(BaseTwo):
__tablename__ = "t1" # will use BaseTwo.metadata
- id = Column(Integer, primary_key=True)
-
-.. versionchanged:: 1.4.3 The :meth:`_orm.registry.mapped` decorator will
- honor an attribute named ``.metadata`` on the class as an alternate
- :class:`_schema.MetaData` collection to be used in place of the
- :class:`_schema.MetaData` that's on the :class:`_orm.registry` itself.
- This matches the behavior of the base class returned by the
- :meth:`_orm.registry.generate_base` and :meth:`_orm.declarative_base`
- method/function. Note this feature was broken due to a regression in
- 1.4.0, 1.4.1 and 1.4.2, even when using :func:`_orm.declarative_base`;
- 1.4.3 is needed to restore the behavior.
+ id = mapped_column(Integer, primary_key=True)
.. seealso::
One possible use of ``__abstract__`` is to use a distinct
:class:`_schema.MetaData` for different bases::
- Base = declarative_base()
+ class Base(DeclarativeBase):
+ pass
class DefaultBase(Base):
class Person(AutoTable, Base):
- id = Column(Integer, primary_key=True)
+ id = mapped_column(Integer, primary_key=True)
class Employee(Person):
- employee_name = Column(String)
+ employee_name = mapped_column(String)
The above ``Employee`` class would be mapped as single-table inheritance
against ``Person``; the ``employee_name`` column would be added as a member
========================================
A common need when mapping classes using the :ref:`Declarative
-<orm_declarative_mapping>` style is to share some functionality, such as a set
-of common columns, some common table options, or other mapped properties,
-across many classes. The standard Python idioms for this is to have the
-classes inherit from a superclass which includes these common features.
-
-When using declarative mappings, this idiom is allowed via the
-usage of mixin classes, as well as via augmenting the declarative base
-produced by either the :meth:`_orm.registry.generate_base` method
-or :func:`_orm.declarative_base` functions.
-
-When using mixins or abstract base classes with Declarative, a decorator
-known as :func:`_orm.declared_attr` is frequently used. This decorator
-allows the creation of class methods that produce a parameter or ORM construct that will be
-part of a declarative mapping. Generating constructs using a callable
-allows for Declarative to get a new copy of a particular kind of object
-each time it calls upon the mixin or abstract base on behalf of a new
-class that's being mapped.
+<orm_declarative_mapping>` style is to share common functionality, such as
+particular columns, table or mapper options, naming schemes, or other mapped
+properties, across many classes. When using declarative mappings, this idiom
+is supported via the use of :term:`mixin classes`, as well as via augmenting the declarative base
+class itself.
+
+.. tip:: In addition to mixin classes, common column options may also be
+ shared among many classes using :pep:`593` ``Annotated`` types; see
+ :ref:`orm_declarative_mapped_column_type_map_pep593` and
+ :ref:`orm_declarative_mapped_column_pep593` for background on these
+ SQLAlchemy 2.0 features.
An example of some commonly mixed-in idioms is below::
- from sqlalchemy.orm import declarative_mixin, declared_attr
+ from sqlalchemy import ForeignKey
+ from sqlalchemy.orm import declared_attr
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import mapped_column
+ from sqlalchemy.orm import relationship
+ class Base(DeclarativeBase):
+ pass
- @declarative_mixin
- class MyMixin:
- @declared_attr
- def __tablename__(cls):
+ class CommonMixin:
+ """define a series of common elements that may be applied to mapped
+ classes using this class as a mixin class."""
+
+ @declared_attr.directive
+ def __tablename__(cls) -> str:
return cls.__name__.lower()
__table_args__ = {"mysql_engine": "InnoDB"}
- __mapper_args__ = {"always_refresh": True}
+ __mapper_args__ = {"eager_defaults": True}
- id = Column(Integer, primary_key=True)
+ id: Mapped[int] = mapped_column(primary_key=True)
+ class HasLogRecord:
+ """mark classes that have a many-to-one relationship to the
+ ``LogRecord`` class."""
- class MyModel(MyMixin, Base):
- name = Column(String(1000))
-
-Where above, the class ``MyModel`` will contain an "id" column
-as the primary key, a ``__tablename__`` attribute that derives
-from the name of the class itself, as well as ``__table_args__``
-and ``__mapper_args__`` defined by the ``MyMixin`` mixin class. The
-:func:`_orm.declared_attr` decorator applied to a class method called
-``def __tablename__(cls):`` has the effect of turning the method into a class
-method while also indicating to Declarative that this attribute is significant
-within the mapping.
-
-.. tip::
-
- The use of the :func:`_orm.declarative_mixin` class decorator marks a
- particular class as providing the service of providing SQLAlchemy declarative
- assignments as a mixin for other classes. This decorator is currently only
- necessary to provide a hint to the :ref:`Mypy plugin <mypy_toplevel>` that
- this class should be handled as part of declarative mappings.
-
-There's no fixed convention over whether ``MyMixin`` precedes
-``Base`` or not. Normal Python method resolution rules apply, and
+ log_record_id: Mapped[int] = mapped_column(ForeignKey("logrecord.id"))
+
+ @declared_attr
+ def log_record(self) -> Mapped["LogRecord"]:
+ return relationship("LogRecord")
+
+ class LogRecord(CommonMixin, Base):
+ log_info: Mapped[str]
+
+ class MyModel(CommonMixin, HasLogRecord, Base):
+ name: Mapped[str]
+
+The above example illustrates a class ``MyModel`` which includes two mixins
+``CommonMixin`` and ``HasLogRecord`` in its bases, as well as a supplementary
+class ``LogRecord`` which also includes ``CommonMixin``, demonstrating a
+variety of constructs that are supported on mixins and base classes, including:
+
+* columns declared using :func:`_orm.mapped_column`, :class:`_orm.Mapped`
+ or :class:`_schema.Column` are copied from mixins or base classes onto
+ the target class to be mapped; above this is illustrated via the
+ column attributes ``CommonMixin.id`` and ``HasLogRecord.log_record_id``.
+* Declarative directives such as ``__table_args__`` and ``__mapper_args__``
+ can be assigned to a mixin or base class, where they will take effect
+ automatically for any classes which inherit from the mixin or base.
+ The above example illustrates this using
+ the ``__table_args__`` and ``__mapper_args__`` attributes.
+* All Declarative directives, including all of ``__tablename__``, ``__table__``,
+ ``__table_args__`` and ``__mapper_args__``, may be implemented using
+ user-defined class methods, which are decorated with the
+ :class:`_orm.declared_attr` decorator (specifically the
+ :attr:`_orm.declared_attr.directive` sub-member, more on that in a moment).
+ Above, this is illustrated using a ``def __tablename__(cls)`` classmethod that
+ generates a :class:`.Table` name dynamically; when applied to the
+ ``MyModel`` class, the table name will be generated as ``"mymodel"``, and
+ when applied to the ``LogRecord`` class, the table name will be generated
+ as ``"logrecord"``.
+* Other ORM properties such as :func:`_orm.relationship` can be generated
+ on the target class to be mapped using user-defined class methods also
+ decorated with the :class:`_orm.declared_attr` decorator. Above, this is
+ illustrated by generating a many-to-one :func:`_orm.relationship` to a mapped
+ object called ``LogRecord``.
+
+The features above may all be demonstrated using a :func:`_sql.select`
+example::
+
+ >>> from sqlalchemy import select
+ >>> print(select(MyModel).join(MyModel.log_record))
+ SELECT mymodel.name, mymodel.id, mymodel.log_record_id
+ FROM mymodel JOIN logrecord ON logrecord.id = mymodel.log_record_id
+
+.. tip:: The examples of :class:`_orm.declared_attr` will attempt to illustrate
+ the correct :pep:`484` annotations for each method example. The use of annotations with
+ :class:`_orm.declared_attr` functions are **completely optional**, and
+ are not
+ consumed by Declarative; however, these annotations are required in order
+ to pass Mypy ``--strict`` type checking.
+
+ Additionally, the :attr:`_orm.declared_attr.directive` sub-member
+ illustrated above is optional as well, and is only significant for
+ :pep:`484` typing tools, as it adjusts for the expected return type when
+ creating methods to override Declarative directives such as
+ ``__tablename__``, ``__mapper_args__`` and ``__table_args__``.
+
+ .. versionadded:: 2.0 As part of :pep:`484` typing support for the
+ SQLAlchemy ORM, added the :attr:`_orm.declared_attr.directive` to
+ :class:`_orm.declared_attr` to distinguish between :class:`_orm.Mapped`
+ attributes and Declarative configurational attributes
+
+There's no fixed convention for the order of mixins and base classes.
+Normal Python method resolution rules apply, and
the above example would work just as well with::
- class MyModel(Base, MyMixin):
- name = Column(String(1000))
+ class MyModel(Base, HasLogRecord, CommonMixin):
+ name: Mapped[str] = mapped_column()
+
+This works because ``Base`` here doesn't define any of the variables that
+``CommonMixin`` or ``HasLogRecord`` defines, i.e. ``__tablename__``,
+``__table_args__``, ``id``, etc. If the ``Base`` did define an attribute of the
+same name, the class placed first in the inherits list would determine which
+attribute is used on the newly defined class.
+
+.. tip:: While the above example is using
+ :ref:`Annotated Declarative Table <orm_declarative_mapped_column>` form
+ based on the :class:`_orm.Mapped` annotation class, mixin classes also work
+ perfectly well with non-annotated and legacy Declarative forms, such as when
+ using :class:`_schema.Column` directly instead of
+ :func:`_orm.mapped_column`.
+
+.. versionchanged:: 2.0 For users coming from the 1.4 series of SQLAlchemy
+ who may have been using the :ref:`mypy plugin <mypy_toplevel>`, the
+ :func:`_orm.declarative_mixin` class decorator is no longer needed
+ to mark declarative mixins, assuming the mypy plugin is no longer in use.
-This works because ``Base`` here doesn't define any of the
-variables that ``MyMixin`` defines, i.e. ``__tablename__``,
-``__table_args__``, ``id``, etc. If the ``Base`` did define
-an attribute of the same name, the class placed first in the
-inherits list would determine which attribute is used on the
-newly defined class.
Augmenting the Base
~~~~~~~~~~~~~~~~~~~
In addition to using a pure mixin, most of the techniques in this
-section can also be applied to the base class itself, for patterns that
-should apply to all classes derived from a particular base. This is achieved
-using the ``cls`` argument of the :func:`_orm.declarative_base` function::
+section can also be applied to the base class directly, for patterns that
+should apply to all classes derived from a particular base. The example
+below illustrates some of the the previous section's example in terms of the
+``Base`` class::
- from sqlalchemy.orm import declarative_base, declared_attr
+ from sqlalchemy import ForeignKey
+ from sqlalchemy.orm import declared_attr
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import mapped_column
+ from sqlalchemy.orm import relationship
+ class Base(DeclarativeBase):
+ """define a series of common elements that may be applied to mapped
+ classes using this class as a base class."""
+
+ @declared_attr.directive
+ def __tablename__(cls) -> str:
+ return cls.__name__.lower()
+
+ __table_args__ = {"mysql_engine": "InnoDB"}
+ __mapper_args__ = {"eager_defaults": True}
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+
+ class HasLogRecord:
+ """mark classes that have a many-to-one relationship to the
+ ``LogRecord`` class."""
+
+ log_record_id: Mapped[int] = mapped_column(ForeignKey("logrecord.id"))
- class Base:
@declared_attr
+ def log_record(self) -> Mapped["LogRecord"]:
+ return relationship("LogRecord")
+
+ class LogRecord(Base):
+ log_info: Mapped[str]
+
+ class MyModel(HasLogRecord, Base):
+ name: Mapped[str]
+
+Where above, ``MyModel`` as well as ``LogRecord``, in deriving from
+``Base``, will both have their table name derived from their class name,
+a primary key column named ``id``, as well as the above table and mapper
+arguments defined by ``Base.__table_args__`` and ``Base.__mapper_args__``.
+
+When using legacy :func:`_orm.declarative_base` or :meth:`_orm.registry.generate_base`,
+the :paramref:`_orm.declarative_base.cls` parameter may be used as follows
+to generate an equivalent effect, as illustrated in the non-annotated
+example below::
+
+ # legacy declarative_base() use
+
+ from sqlalchemy import Integer, String
+ from sqlalchemy import ForeignKey
+ from sqlalchemy.orm import declared_attr
+ from sqlalchemy.orm import declarative_base
+ from sqlalchemy.orm import mapped_column
+ from sqlalchemy.orm import relationship
+
+ class Base:
+ """define a series of common elements that may be applied to mapped
+ classes using this class as a base class."""
+
+ @declared_attr.directive
def __tablename__(cls):
return cls.__name__.lower()
__table_args__ = {"mysql_engine": "InnoDB"}
+ __mapper_args__ = {"eager_defaults": True}
- id = Column(Integer, primary_key=True)
-
+ id = mapped_column(Integer, primary_key=True)
Base = declarative_base(cls=Base)
+ class HasLogRecord:
+ """mark classes that have a many-to-one relationship to the
+ ``LogRecord`` class."""
+
+ log_record_id = mapped_column(ForeignKey("logrecord.id"))
- class MyModel(Base):
- name = Column(String(1000))
+ @declared_attr
+ def log_record(self):
+ return relationship("LogRecord")
+
+ class LogRecord(Base):
+ log_info = mapped_column(String)
-Where above, ``MyModel`` and all other classes that derive from ``Base`` will
-have a table name derived from the class name, an ``id`` primary key column,
-as well as the "InnoDB" engine for MySQL.
+ class MyModel(HasLogRecord, Base):
+ name = mapped_column(String)
Mixing in Columns
~~~~~~~~~~~~~~~~~
-The most basic way to specify a column on a mixin is by simple
-declaration::
+Columns can be indicated in mixins assuming the
+:ref:`Declarative table <orm_declarative_table>` style of configuration
+is in use (as opposed to
+:ref:`imperative table <orm_imperative_table_configuration>` configuration),
+so that columns declared on the mixin can then be copied to be
+part of the :class:`_schema.Table` that the Declarative process generates.
+All three of the :func:`_orm.mapped_column`, :class:`_orm.Mapped`,
+and :class:`_schema.Column` constructs may be declared inline in a
+declarative mixin::
- @declarative_mixin
class TimestampMixin:
- created_at = Column(DateTime, default=func.now())
+ created_at: Mapped[datetime] = mapped_column(default=func.now())
+ updated_at: Mapped[datetime]
class MyModel(TimestampMixin, Base):
__tablename__ = "test"
- id = Column(Integer, primary_key=True)
- name = Column(String(1000))
+ id: Mapped[int] = mapped_column(primary_key=True)
+ name: Mapped[str]
Where above, all declarative classes that include ``TimestampMixin``
-will also have a column ``created_at`` that applies a timestamp to
-all row insertions.
-
-Those familiar with the SQLAlchemy expression language know that
-the object identity of clause elements defines their role in a schema.
-Two ``Table`` objects ``a`` and ``b`` may both have a column called
-``id``, but the way these are differentiated is that ``a.c.id``
-and ``b.c.id`` are two distinct Python objects, referencing their
-parent tables ``a`` and ``b`` respectively.
-
-In the case of the mixin column, it seems that only one
-:class:`_schema.Column` object is explicitly created, yet the ultimate
-``created_at`` column above must exist as a distinct Python object
-for each separate destination class. To accomplish this, the declarative
-extension creates a **copy** of each :class:`_schema.Column` object encountered on
-a class that is detected as a mixin.
-
-This copy mechanism is limited to :class:`_schema.Column` and
-:class:`_orm.MappedColumn` constructs. For :class:`_schema.Column` and
-:class:`_orm.MappedColumn` constructs that contain references to
-:class:`_schema.ForeignKey` constructs, the copy mechanism is limited to
-foreign key references to remote tables only.
-
-.. versionchanged:: 2.0 The declarative API can now accommodate
- :class:`_schema.Column` objects which refer to :class:`_schema.ForeignKey`
- constraints to remote tables without the need to use the
- :class:`_orm.declared_attr` function decorator.
-
-For the variety of mapper-level constructs that require destination-explicit
-context, including self-referential foreign keys and constructs like
-:func:`_orm.deferred`, :func:`_orm.relationship`, etc, the
-:class:`_orm.declared_attr` decorator is provided so that patterns common to
-many classes can be defined as callables::
-
- from sqlalchemy.orm import declared_attr
-
-
- @declarative_mixin
- class HasRelatedDataMixin:
- @declared_attr
- def related_data(cls):
- return deferred(Column(Text()))
+in their class bases will automatically include a column ``created_at``
+that applies a timestamp to all row insertions, as well as an ``updated_at``
+column, which does not include a default for the purposes of the example
+(if it did, we would use the :paramref:`_schema.Column.onupdate` parameter
+which is accepted by :func:`_orm.mapped_column`). These column constructs
+are always **copied from the originating mixin or base class**, so that the
+same mixin/base class may be applied to any number of target classes
+which will each have their own column constructs.
+All Declarative column forms are supported by mixins, including:
- class User(HasRelatedDataMixin, Base):
- __tablename__ = "user"
- id = Column(Integer, primary_key=True)
+* **Annotated attributes** - with or without :func:`_orm.mapped_column` present::
-Where above, the ``related_data`` class-level callable is executed at the
-point at which the ``User`` class is constructed, and the declarative
-extension can use the resulting :func`_orm.deferred` object as returned by
-the method without the need to copy it.
-
-For a self-referential foreign key on a mixin, the referenced
-:class:`_schema.Column` object may be referenced in terms of the class directly
-within the :class:`_orm.declared_attr`::
+ class TimestampMixin:
+ created_at: Mapped[datetime] = mapped_column(default=func.now())
+ updated_at: Mapped[datetime]
- class SelfReferentialMixin:
- id = Column(Integer, primary_key=True)
+* **mapped_column** - with or without :class:`_orm.Mapped` present::
- @declared_attr
- def parent_id(cls):
- return Column(Integer, ForeignKey(cls.id))
+ class TimestampMixin:
+ created_at = mapped_column(default=func.now())
+ updated_at: Mapped[datetime] = mapped_column()
+* **Column** - legacy Declarative form::
- class A(SelfReferentialMixin, Base):
- __tablename__ = "a"
+ class TimestampMixin:
+ created_at = Column(DateTime, default=func.now())
+ updated_at = Column(DateTime)
+In each of the above forms, Declarative handles the column-based attributes
+on the mixin class by creating a **copy** of the construct, which is then
+applied to the target class.
- class B(SelfReferentialMixin, Base):
- __tablename__ = "b"
+.. versionchanged:: 2.0 The declarative API can now accommodate
+ :class:`_schema.Column` objects as well as :func:`_orm.mapped_column`
+ constructs of any form when using mixins without the need to use
+ :func:`_orm.declared_attr`. Previous limitations which prevented columns
+ with :class:`_schema.ForeignKey` elements from being used directly
+ in mixins have been removed.
-Above, both classes ``A`` and ``B`` will contain columns ``id`` and
-``parent_id``, where ``parent_id`` refers to the ``id`` column local to the
-corresponding table ('a' or 'b').
.. _orm_declarative_mixins_relationships:
relationship so that two classes ``Foo`` and ``Bar`` can both be configured to
reference a common target class via many-to-one::
- @declarative_mixin
+ from sqlalchemy import ForeignKey
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import declared_attr
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import mapped_column
+ from sqlalchemy.orm import relationship
+
+ class Base(DeclarativeBase):
+ pass
+
class RefTargetMixin:
- target_id = Column("target_id", ForeignKey("target.id"))
+ target_id: Mapped[int] = mapped_column(ForeignKey("target.id"))
@declared_attr
- def target(cls):
+ def target(cls) -> Mapped["Target"]:
return relationship("Target")
-
class Foo(RefTargetMixin, Base):
__tablename__ = "foo"
- id = Column(Integer, primary_key=True)
+ id: Mapped[int] = mapped_column(primary_key=True)
class Bar(RefTargetMixin, Base):
__tablename__ = "bar"
- id = Column(Integer, primary_key=True)
+ id: Mapped[int] = mapped_column(primary_key=True)
class Target(Base):
__tablename__ = "target"
- id = Column(Integer, primary_key=True)
-
-
-Using Advanced Relationship Arguments (e.g. ``primaryjoin``, etc.)
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-:func:`~sqlalchemy.orm.relationship` definitions which require explicit
-primaryjoin, order_by etc. expressions should in all but the most
-simplistic cases use **late bound** forms
-for these arguments, meaning, using either the string form or a function/lambda.
-The reason for this is that the related :class:`_schema.Column` objects which are to
-be configured using ``@declared_attr`` are not available to another
-``@declared_attr`` attribute; while the methods will work and return new
-:class:`_schema.Column` objects, those are not the :class:`_schema.Column` objects that
-Declarative will be using as it calls the methods on its own, thus using
-*different* :class:`_schema.Column` objects.
-
-The canonical example is the primaryjoin condition that depends upon
-another mixed-in column::
+ id: Mapped[int] = mapped_column(primary_key=True)
+
+With the above mapping, each of ``Foo`` and ``Bar`` contain a relationship
+to ``Target`` accessed along the ``.target`` attribute::
+
+ >>> from sqlalchemy import select
+ >>> print(select(Foo).join(Foo.target))
+ SELECT foo.id, foo.target_id
+ FROM foo JOIN target ON target.id = foo.target_id
+ >>> print(select(Bar).join(Bar.target))
+ SELECT bar.id, bar.target_id
+ FROM bar JOIN target ON target.id = bar.target_id
+
+Special arguments such as :paramref:`_orm.relationship.primaryjoin` may also
+be used within mixed-in classmethods, which often need to refer to the class
+that's being mapped. For schemes that need to refer to locally mapped columns, in
+ordinary cases these columns are made available by Declarative as attributes
+on the mapped class which is passed as the ``cls`` argument to the
+decorated classmethod. Using this feature, we could for
+example rewrite the ``RefTargetMixin.target`` method using an
+explicit primaryjoin which refers to pending mapped columns on both
+``Target`` and ``cls``::
- @declarative_mixin
- class RefTargetMixin:
- @declared_attr
- def target_id(cls):
- return Column("target_id", ForeignKey("target.id"))
-
- @declared_attr
- def target(cls):
- return relationship(
- Target,
- primaryjoin=Target.id == cls.target_id, # this is *incorrect*
- )
-
-Mapping a class using the above mixin, we will get an error like::
-
- sqlalchemy.exc.InvalidRequestError: this ForeignKey's parent column is not
- yet associated with a Table.
-
-This is because the ``target_id`` :class:`_schema.Column` we've called upon in our
-``target()`` method is not the same :class:`_schema.Column` that declarative is
-actually going to map to our table.
-
-The condition above is resolved using a lambda::
+ class Target(Base):
+ __tablename__ = "target"
+ id: Mapped[int] = mapped_column(primary_key=True)
- @declarative_mixin
class RefTargetMixin:
- @declared_attr
- def target_id(cls):
- return Column('target_id', ForeignKey('target.id'))
+ target_id: Mapped[int] = mapped_column(ForeignKey("target.id"))
@declared_attr
- def target(cls):
- return relationship(Target,
- primaryjoin=lambda: Target.id==cls.target_id
- )
+ def target(cls) -> Mapped["Target"]:
+ # illustrates explicit 'primaryjoin' argument
+ return relationship("Target", primaryjoin=Target.id == cls.target_id)
+
+.. _orm_declarative_mixins_mapperproperty:
+
+Mixing in :func:`_orm.column_property` and other :class:`_orm.MapperProperty` classes
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Like :func:`_orm.relationship`, other
+:class:`_orm.MapperProperty` subclasses such as
+:func:`_orm.column_property` also need to have class-local copies generated
+when used by mixins, so are also declared within functions that are
+decorated by :class:`_orm.declared_attr`. Within the function,
+other ordinary mapped columns that were declared with :func:`_orm.mapped_column`,
+:class:`_orm.Mapped`, or :class:`_schema.Column` will be made available from the ``cls`` argument
+so that they may be used to compose new attributes, as in the example below which adds two
+columns together::
+
+ from sqlalchemy.orm import column_property
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import declared_attr
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import mapped_column
-or alternatively, the string form (which ultimately generates a lambda)::
+ class Base(DeclarativeBase):
+ pass
- @declarative_mixin
- class RefTargetMixin:
- @declared_attr
- def target_id(cls):
- return Column("target_id", ForeignKey("target.id"))
+ class SomethingMixin:
+ x: Mapped[int]
+ y: Mapped[int]
@declared_attr
- def target(cls):
- return relationship(
- Target, primaryjoin=f"Target.id=={cls.__name__}.target_id"
- )
-
-.. seealso::
+ def x_plus_y(cls) -> Mapped[int]:
+ return column_property(cls.x + cls.y)
- :ref:`orm_declarative_relationship_eval`
+ class Something(SomethingMixin, Base):
+ __tablename__ = "something"
-Mixing in deferred(), column_property(), and other MapperProperty classes
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ id: Mapped[int] = mapped_column(primary_key=True)
-Like :func:`~sqlalchemy.orm.relationship`, all
-:class:`~sqlalchemy.orm.interfaces.MapperProperty` subclasses such as
-:func:`~sqlalchemy.orm.deferred`, :func:`~sqlalchemy.orm.column_property`,
-etc. ultimately involve references to columns, and therefore, when
-used with declarative mixins, have the :class:`_orm.declared_attr`
-requirement so that no reliance on copying is needed::
+Above, we may make use of ``Something.x_plus_y`` in a statement where
+it produces the full expression::
- @declarative_mixin
- class SomethingMixin:
- @declared_attr
- def dprop(cls):
- return deferred(Column(Integer))
+ >>> from sqlalchemy import select
+ >>> print(select(Something.x_plus_y))
+ SELECT something.x + something.y AS anon_1
+ FROM something
+.. tip:: The :class:`_orm.declared_attr` decorator causes the decorated callable
+ to behave exactly as a classmethod. However, typing tools like Pylance_
+ may not be able to recognize this, which can sometimes cause it to complain
+ about access to the ``cls`` variable inside the body of the function. To
+ resolve this issue when it occurs, the ``@classmethod`` decorator may be
+ combined directly with :class:`_orm.declared_attr` as::
- class Something(SomethingMixin, Base):
- __tablename__ = "something"
-The :func:`.column_property` or other construct may refer
-to other columns from the mixin. These are copied ahead of time before
-the :class:`_orm.declared_attr` is invoked::
+ class SomethingMixin:
+ x: Mapped[int]
+ y: Mapped[int]
- @declarative_mixin
- class SomethingMixin:
- x = Column(Integer)
- y = Column(Integer)
+ @declared_attr
+ @classmethod
+ def x_plus_y(cls) -> Mapped[int]:
+ return column_property(cls.x + cls.y)
- @declared_attr
- def x_plus_y(cls):
- return column_property(cls.x + cls.y)
+ .. versionadded:: 2.0 - :class:`_orm.declared_attr` can accommodate a
+ function decorated with ``@classmethod`` to help with :pep:`484`
+ integration where needed.
-.. versionchanged:: 1.0.0 mixin columns are copied to the final mapped class
- so that :class:`_orm.declared_attr` methods can access the actual column
- that will be mapped.
.. _decl_mixin_inheritance:
-Controlling table inheritance with mixins
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-The ``__tablename__`` attribute may be used to provide a function that
-will determine the name of the table used for each class in an inheritance
-hierarchy, as well as whether a class has its own distinct table.
-
-This is achieved using the :class:`_orm.declared_attr` indicator in conjunction
-with a method named ``__tablename__()``. Declarative will always
-invoke :class:`_orm.declared_attr` for the special names
-``__tablename__``, ``__mapper_args__`` and ``__table_args__``
-function **for each mapped class in the hierarchy, except if overridden
-in a subclass**. The function therefore
-needs to expect to receive each class individually and to provide the
-correct answer for each.
-
-For example, to create a mixin that gives every class a simple table
-name based on class name::
+Using Mixins and Base Classes with Mapped Inheritance Patterns
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+When dealing with mapper inheritance patterns as documented at
+:ref:`inheritance_toplevel`, some additional capabilities are present
+when using :class:`_orm.declared_attr` either with mixin classes, or when
+augmenting both mapped and un-mapped superclasses in a class hierarchy.
+
+When defining functions decorated by :class:`_orm.declared_attr` on mixins or
+base classes to be interpreted by subclasses in a mapped inheritance hierarchy,
+there is an important distinction
+made between functions that generate the special names used by Declarative such
+as ``__tablename__``, ``__mapper_args__`` vs. those that may generate ordinary
+mapped attributes such as :func:`_orm.mapped_column` and
+:func:`_orm.relationship`. Functions that define **Declarative directives** are
+**invoked for each subclass in a hierarchy**, whereas functions that
+generate **mapped attributes** are **invoked only for the first mapped
+superclass in a hierarchy**.
+
+The rationale for this difference in behavior is based on the fact that
+mapped properties are already inheritable by classes, such as a particular
+column on a superclass' mapped table should not be duplicated to that of a
+subclass as well, whereas elements that are specific to a particular
+class or its mapped table are not inheritable, such as the name of the
+table that is locally mapped.
+
+The difference in behavior between these two use cases is demonstrated
+in the following two sections.
+
+Using :func:`_orm.declared_attr` with inheriting :class:`.Table` and :class:`.Mapper` arguments
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+A common recipe with mixins is to create a ``def __tablename__(cls)``
+function that generates a name for the mapped :class:`.Table` dynamically.
+
+This recipe can be used to generate table names for an inheriting mapper
+hierarchy as in the example below which creates a mixin that gives every class a simple table
+name based on class name. The recipe is illustrated below where a table name
+is generated for the ``Person`` mapped class and the ``Engineer`` subclass
+of ``Person``, but not for the ``Manager`` subclass of ``Person``::
+
+ from typing import Optional
+
+ from sqlalchemy import ForeignKey
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import declared_attr
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import mapped_column
- from sqlalchemy.orm import declarative_mixin, declared_attr
+ class Base(DeclarativeBase):
+ pass
- @declarative_mixin
class Tablename:
- @declared_attr
- def __tablename__(cls):
+ @declared_attr.directive
+ def __tablename__(cls) -> Optional[str]:
return cls.__name__.lower()
class Person(Tablename, Base):
- id = Column(Integer, primary_key=True)
- discriminator = Column("type", String(50))
- __mapper_args__ = {"polymorphic_on": discriminator}
+ id: Mapped[int] = mapped_column(primary_key=True)
+ discriminator: Mapped[str]
+ __mapper_args__ = {"polymorphic_on": "discriminator"}
class Engineer(Person):
- __tablename__ = None
+ id: Mapped[int] = mapped_column(ForeignKey('person.id'), primary_key=True)
+
+ primary_language: Mapped[str]
+
__mapper_args__ = {"polymorphic_identity": "engineer"}
- primary_language = Column(String(50))
-Alternatively, we can modify our ``__tablename__`` function to return
-``None`` for subclasses, using :func:`.has_inherited_table`. This has
-the effect of those subclasses being mapped with single table inheritance
-against the parent::
- from sqlalchemy.orm import (
- declarative_mixin,
- declared_attr,
- has_inherited_table,
- )
+ class Manager(Person):
+ @declared_attr.directive
+ def __tablename__(cls) -> Optional[str]:
+ """override __tablename__ so that Manager is single-inheritance to Person"""
+
+ return None
+
+ __mapper_args__ = {"polymorphic_identity": "manager"}
+
+In the above example, both the ``Person`` base class as well as the
+``Engineer`` class, being subclasses of the ``Tablename`` mixin class which
+generates new table names, will have a generated ``__tablename__``
+attribute, which to
+Declarative indicates that each class should have its own :class:`.Table`
+generated to which it will be mapped. For the ``Engineer`` subclass, the style of inheritance
+applied is :ref:`joined table inheritance <joined_inheritance>`, as it
+will be mapped to a table ``engineer`` that joins to the base ``person``
+table. Any other subclasses that inherit from ``Person`` will also have
+this style of inheritance applied by default (and within this particular example, would need to
+each specify a primary key column; more on that in the next section).
+
+By contrast, the ``Manager`` subclass of ``Person`` **overrides** the
+``__tablename__`` classmethod to return ``None``. This indicates to
+Declarative that this class should **not** have a :class:`.Table` generated,
+and will instead make use exclusively of the base :class:`.Table` to which
+``Person`` is mapped. For the ``Manager`` subclass, the style of inheritance
+applied is :ref:`single table inheritance <single_inheritance>`.
+
+The example above illustrates that Declarative directives like
+``__tablename__`` are necessarily **applied to each subclass** individually,
+as each mapped class needs to state which :class:`.Table` it will be mapped
+towards, or if it will map itself to the inheriting superclass' :class:`.Table`.
+
+If we instead wanted to **reverse** the default table scheme illustrated
+above, so that
+single table inheritance were the default and joined table inheritance
+could be defined only when a ``__tablename__`` directive were supplied to
+override it, we can make use of
+Declarative helpers within the top-most ``__tablename__()`` method, in this
+case a helper called :func:`.has_inherited_table`. This function will
+return ``True`` if a superclass is already mapped to a :class:`.Table`.
+We may use this helper within the base-most ``__tablename__()`` classmethod
+so that we may **conditionally** return ``None`` for the table name,
+if a table is already present, thus indicating single-table inheritance
+for inheriting subclasses by default::
+
+ from sqlalchemy import ForeignKey
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import declared_attr
+ from sqlalchemy.orm import has_inherited_table
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import mapped_column
+
+ class Base(DeclarativeBase):
+ pass
- @declarative_mixin
class Tablename:
- @declared_attr
+ @declared_attr.directive
def __tablename__(cls):
if has_inherited_table(cls):
return None
class Person(Tablename, Base):
- id = Column(Integer, primary_key=True)
- discriminator = Column("type", String(50))
- __mapper_args__ = {"polymorphic_on": discriminator}
+ id: Mapped[int] = mapped_column(primary_key=True)
+ discriminator: Mapped[str]
+ __mapper_args__ = {"polymorphic_on": "discriminator"}
class Engineer(Person):
- primary_language = Column(String(50))
+ @declared_attr.directive
+ def __tablename__(cls):
+ """override __tablename__ so that Engineer is joined-inheritance to Person"""
+
+ return cls.__name__.lower()
+
+ id: Mapped[int] = mapped_column(ForeignKey('person.id'), primary_key=True)
+
+ primary_language: Mapped[str]
+
__mapper_args__ = {"polymorphic_identity": "engineer"}
+
+ class Manager(Person):
+
+ __mapper_args__ = {"polymorphic_identity": "manager"}
+
+
.. _mixin_inheritance_columns:
-Mixing in Columns in Inheritance Scenarios
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Using :func:`_orm.declared_attr` to generate table-specific inheriting columns
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
In contrast to how ``__tablename__`` and other special names are handled when
used with :class:`_orm.declared_attr`, when we mix in columns and properties (e.g.
relationships, column properties, etc.), the function is
-invoked for the **base class only** in the hierarchy. Below, only the
+invoked for the **base class only** in the hierarchy, unless the
+:class:`_orm.declared_attr` directive is used in combination with the
+:attr:`_orm.declared_attr.cascading` sub-directive. Below, only the
``Person`` class will receive a column
called ``id``; the mapping will fail on ``Engineer``, which is not given
a primary key::
- @declarative_mixin
class HasId:
- @declared_attr
- def id(cls):
- return Column("id", Integer, primary_key=True)
-
+ id: Mapped[int] = mapped_column(primary_key=True)
class Person(HasId, Base):
__tablename__ = "person"
- discriminator = Column("type", String(50))
- __mapper_args__ = {"polymorphic_on": discriminator}
+ discriminator: Mapped[str]
+ __mapper_args__ = {"polymorphic_on": "discriminator"}
+ # this mapping will fail, as there's no primary key
class Engineer(Person):
__tablename__ = "engineer"
- primary_language = Column(String(50))
+
+ primary_language: Mapped[str]
__mapper_args__ = {"polymorphic_identity": "engineer"}
It is usually the case in joined-table inheritance that we want distinctly
function should be invoked **for each class in the hierarchy**, in *almost*
(see warning below) the same way as it does for ``__tablename__``::
- @declarative_mixin
class HasIdMixin:
@declared_attr.cascading
- def id(cls):
+ def id(cls) -> Mapped[int]:
if has_inherited_table(cls):
- return Column(ForeignKey("person.id"), primary_key=True)
+ return mapped_column(ForeignKey("person.id"), primary_key=True)
else:
- return Column(Integer, primary_key=True)
+ return mapped_column(Integer, primary_key=True)
class Person(HasIdMixin, Base):
__tablename__ = "person"
- discriminator = Column("type", String(50))
- __mapper_args__ = {"polymorphic_on": discriminator}
+
+ discriminator: Mapped[str]
+ __mapper_args__ = {"polymorphic_on": "discriminator"}
class Engineer(Person):
__tablename__ = "engineer"
- primary_language = Column(String(50))
+
+ primary_language: Mapped[str]
__mapper_args__ = {"polymorphic_identity": "engineer"}
.. warning::
**not** allow for a subclass to override the attribute with a different
function or value. This is a current limitation in the mechanics of
how ``@declared_attr`` is resolved, and a warning is emitted if
- this condition is detected. This limitation does **not**
- exist for the special attribute names such as ``__tablename__``, which
+ this condition is detected. This limitation only applies to
+ ORM mapped columns, relationships, and other :class:`.MapperProperty`
+ styles of attribute. It does **not** apply to Declarative directives
+ such as ``__tablename__``, ``__mapper_args__``, etc., which
resolve in a different way internally than that of
:attr:`.declared_attr.cascading`.
-.. versionadded:: 1.0.0 added :attr:`.declared_attr.cascading`.
-
Combining Table/Mapper Arguments from Multiple Mixins
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
from sqlalchemy.orm import declarative_mixin, declared_attr
- @declarative_mixin
class MySQLSettings:
__table_args__ = {"mysql_engine": "InnoDB"}
- @declarative_mixin
class MyOtherMixin:
__table_args__ = {"info": "foo"}
args.update(MyOtherMixin.__table_args__)
return args
- id = Column(Integer, primary_key=True)
+ id = mapped_column(Integer, primary_key=True)
Creating Indexes with Mixins
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
tables derived from a mixin, use the "inline" form of :class:`.Index` and
establish it as part of ``__table_args__``::
- @declarative_mixin
class MyMixin:
- a = Column(Integer)
- b = Column(Integer)
+ a = mapped_column(Integer)
+ b = mapped_column(Integer)
@declared_attr
def __table_args__(cls):
class MyModel(MyMixin, Base):
__tablename__ = "atable"
- c = Column(Integer, primary_key=True)
+ c = mapped_column(Integer, primary_key=True)
+
+.. _Pylance: https://github.com/microsoft/pylance-release
mapper configuration.
+.. _orm_explicit_declarative_base:
+
.. _orm_declarative_generated_base_class:
-Using a Generated Base Class
-----------------------------
+Using a Declarative Base Class
+-------------------------------
-The most common approach is to generate a "base" class using the
-:func:`_orm.declarative_base` function::
+The most common approach is to generate a "Declarative Base" class by
+subclassing the :class:`_orm.DeclarativeBase` superclass::
- from sqlalchemy.orm import declarative_base
+ from sqlalchemy.orm import DeclarativeBase
# declarative base class
- Base = declarative_base()
+ class Base(DeclarativeBase):
+ pass
+The Declarative Base class may also be created given an existing
+:class:`_orm.registry` by assigning it as a class variable named
+``registry``::
-The declarative base class may also be created from an existing
-:class:`_orm.registry`, by using the :meth:`_orm.registry.generate_base`
-method::
-
+ from sqlalchemy.orm import DeclarativeBase
from sqlalchemy.orm import registry
reg = registry()
+
# declarative base class
- Base = reg.generate_base()
+ class Base(DeclarativeBase):
+ registry = reg
+
+.. versionchanged:: 2.0 The :class:`_orm.DeclarativeBase` superclass supersedes
+ the use of the :func:`_orm.declarative_base` function and
+ :meth:`_orm.registry.generate_base` methods; the superclass approach
+ integrates with :pep:`484` tools without the use of plugins.
+ See :ref:`whatsnew_20_orm_declarative_typing` for migration notes.
With the declarative base class, new mapped classes are declared as subclasses
of the base::
- from sqlalchemy import Column, ForeignKey, Integer, String
- from sqlalchemy.orm import declarative_base
+ from datetime import datetime
+ from typing import List
+ from typing import Optional
- # declarative base class
- Base = declarative_base()
+ from sqlalchemy import Integer, String
+ from sqlalchemy import func
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import mapped_column
+ class Base(DeclarativeBase):
+ pass
- # an example mapping using the base
class User(Base):
__tablename__ = "user"
- id = Column(Integer, primary_key=True)
- name = Column(String)
- fullname = Column(String)
- nickname = Column(String)
-
-Above, the :func:`_orm.declarative_base` function returns a new base class from
-which new classes to be mapped may inherit from, as above a new mapped
-class ``User`` is constructed.
-
-For each subclass constructed, the body of the class then follows the
-declarative mapping approach which defines both a :class:`_schema.Table`
-as well as a :class:`_orm.Mapper` object behind the scenes which comprise
-a full mapping.
-
-.. seealso::
-
- :ref:`orm_declarative_table_config_toplevel`
-
- :ref:`orm_declarative_mapper_config_toplevel`
+ id = mapped_column(Integer, primary_key=True)
+ name: Mapped[str]
+ fullname: Mapped[Optional[str]]
+ nickname: Mapped[Optional[str]] = mapped_column(String(64))
+ create_date: Mapped[datetime] = mapped_column(insert_default=func.now())
+ addresses: Mapped[List["Address"]] = relationship(back_populates="user")
-.. _orm_explicit_declarative_base:
-
-Creating an Explicit Base Non-Dynamically (for use with mypy, similar)
-----------------------------------------------------------------------
-
-SQLAlchemy includes a :ref:`Mypy plugin <mypy_toplevel>` that automatically
-accommodates for the dynamically generated ``Base`` class
-delivered by SQLAlchemy functions like :func:`_orm.declarative_base`.
-
-When this plugin is not in use, or when using other :pep:`484` tools which
-may not know how to interpret this class, the declarative base class may
-be produced in a fully explicit fashion using the
-:class:`_orm.DeclarativeMeta` directly as follows::
-
- from sqlalchemy.orm import registry
- from sqlalchemy.orm.decl_api import DeclarativeMeta
-
- mapper_registry = registry()
+ class Address(Base):
+ __tablename__ = "address"
- class Base(metaclass=DeclarativeMeta):
- __abstract__ = True
+ id = mapped_column(Integer, primary_key=True)
+ user_id = mapped_column(ForeignKey("user.id"))
+ email_address: Mapped[str]
- registry = mapper_registry
- metadata = mapper_registry.metadata
+ user: Mapped["User"] = relationship(back_populates="addresses")
- __init__ = mapper_registry.constructor
+Above, the ``Base`` class serves as a base for new classes that are to be
+mapped, as above new mapped classes ``User`` and ``Address`` are constructed.
-The above ``Base`` is equivalent to one created using the
-:meth:`_orm.registry.generate_base` method and will be fully understood by
-type analysis tools without the use of plugins.
+For each subclass constructed, the body of the class then follows the
+declarative mapping approach which defines both a :class:`_schema.Table` as
+well as a :class:`_orm.Mapper` object behind the scenes which comprise a full
+mapping.
.. seealso::
- :ref:`mypy_toplevel` - background on the Mypy plugin which applies the
- above structure automatically when running Mypy.
+ :ref:`orm_declarative_table_config_toplevel` - describes how to specify
+ the components of the mapped :class:`_schema.Table` to be generated,
+ including notes and options on the use of the :func:`_orm.mapped_column`
+ construct and how it interacts with the :class:`_orm.Mapped` annotation
+ type
+
+ :ref:`orm_declarative_mapper_config_toplevel` - describes all other
+ aspects of ORM mapper configuration within Declarative including
+ :func:`_orm.relationship` configuration, SQL expressions and
+ :class:`_orm.Mapper` parameters
.. _orm_declarative_decorator:
similar to that of a "classical" mapping, or more succinctly by using
a decorator. The :meth:`_orm.registry.mapped` function is a class decorator
that can be applied to any Python class with no hierarchy in place. The
-Python class otherwise is configured in declarative style normally::
+Python class otherwise is configured in declarative style normally.
+
+The example below sets up the identical mapping as seen in the
+previous section, using the :meth:`_orm.registry.mapped`
+decorator rather than using the :class:`_orm.DeclarativeBase` superclass::
- from sqlalchemy import Column, ForeignKey, Integer, String, Text
+ from sqlalchemy import mapped_column, ForeignKey, Integer, String, Text
from sqlalchemy.orm import registry, relationship
mapper_registry = registry()
class User:
__tablename__ = "user"
- id = Column(Integer, primary_key=True)
- name = Column(String)
+ id = mapped_column(Integer, primary_key=True)
+ name: Mapped[str]
+ fullname: Mapped[Optional[str]]
+ nickname: Mapped[Optional[str]] = mapped_column(String(64))
+ create_date: Mapped[datetime] = mapped_column(insert_default=func.now())
- addresses = relationship("Address", back_populates="user")
+ addresses: Mapped[List["Address"]] = relationship(back_populates="user")
@mapper_registry.mapped
class Address:
__tablename__ = "address"
- id = Column(Integer, primary_key=True)
- user_id = Column(ForeignKey("user.id"))
- email_address = Column(String)
+ id = mapped_column(Integer, primary_key=True)
+ user_id = mapped_column(ForeignKey("user.id"))
+ email_address: Mapped[str]
- user = relationship("User", back_populates="addresses")
+ user: Mapped["User"] = relationship(back_populates="addresses")
-Above, the same :class:`_orm.registry` that we'd use to generate a declarative
-base class via its :meth:`_orm.registry.generate_base` method may also apply
-a declarative-style mapping to a class without using a base. When using
-the above style, the mapping of a particular class will **only** proceed
-if the decorator is applied to that class directly. For inheritance
-mappings, the decorator should be applied to each subclass::
+When using the above style, the mapping of a particular class will **only**
+proceed if the decorator is applied to that class directly. For inheritance
+mappings (described in detail at :ref:`inheritance_toplevel`), the decorator
+should be applied to each subclass that is to be mapped::
from sqlalchemy.orm import registry
class Person:
__tablename__ = "person"
- person_id = Column(Integer, primary_key=True)
- type = Column(String, nullable=False)
+ person_id = mapped_column(Integer, primary_key=True)
+ type = mapped_column(String, nullable=False)
__mapper_args__ = {
"polymorphic_on": type,
class Employee(Person):
__tablename__ = "employee"
- person_id = Column(ForeignKey("person.person_id"), primary_key=True)
+ person_id = mapped_column(ForeignKey("person.person_id"), primary_key=True)
__mapper_args__ = {
"polymorphic_identity": "employee",
}
-Both the "declarative table" and "imperative table" styles of declarative
-mapping may be used with the above mapping style.
+Both the :ref:`declarative table <orm_declarative_table>` and
+:ref:`imperative table <orm_imperative_table_configuration>`
+table configuration styles may be used with either the Declarative Base
+or decorator styles of Declarative mapping.
+
+The decorator form of mapping is useful when combining a
+SQLAlchemy declarative mapping with other class instrumentation systems
+such as dataclasses_ and attrs_, though note that SQLAlchemy 2.0 now features
+dataclasses integration with Declarative Base classes as well.
-The decorator form of mapping is particularly useful when combining a
-SQLAlchemy declarative mapping with other forms of class declaration, notably
-the Python ``dataclasses`` module. See the next section.
+.. _dataclass: https://docs.python.org/3/library/dataclasses.html
+.. _dataclasses: https://docs.python.org/3/library/dataclasses.html
+.. _attrs: https://pypi.org/project/attrs/
The following examples assume a declarative base class as::
- from sqlalchemy.orm import declarative_base
+ from sqlalchemy.orm import DeclarativeBase
+
+ class Base(DeclarativeBase):
+ pass
- Base = declarative_base()
All of the examples that follow illustrate a class inheriting from the above
``Base``. The decorator style introduced at :ref:`orm_declarative_decorator`
-is fully supported with all the following examples as well.
+is fully supported with all the following examples as well, as are legacy
+forms of Declarative Base including base classes generated by
+:func:`_orm.declarative_base`.
+
.. _orm_declarative_table:
-Declarative Table
------------------
+Declarative Table with ``mapped_column()``
+------------------------------------------
-With the declarative base class, the typical form of mapping includes an
-attribute ``__tablename__`` that indicates the name of a :class:`_schema.Table`
-that should be generated along with the mapping::
+When using Declarative, the body of the class to be mapped in most cases
+includes an attribute ``__tablename__`` that indicates the string name of a
+:class:`_schema.Table` that should be generated along with the mapping. The
+:func:`_orm.mapped_column` construct, which features additional ORM-specific
+configuration capabilities not present in the plain :class:`_schema.Column`
+class, is then used within the class body to indicate columns in the table. The
+example below illustrates the most basic use of this construct within a
+Declarative mapping::
- from sqlalchemy import Column, ForeignKey, Integer, String
- from sqlalchemy.orm import declarative_base
- Base = declarative_base()
+ from sqlalchemy import Integer, String
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import mapped_column
+
+ class Base(DeclarativeBase):
+ pass
class User(Base):
__tablename__ = "user"
- id = Column(Integer, primary_key=True)
- name = Column(String)
- fullname = Column(String)
- nickname = Column(String)
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(50), nullable=False)
+ fullname = mapped_column(String)
+ nickname = mapped_column(String(30))
-Above, :class:`_schema.Column` objects are placed inline with the class
-definition. The declarative mapping process will generate a new
+Above, :func:`_orm.mapped_column` constructs are placed inline within the class
+definition as class level attributes. At the point at which the class is
+declared, the Declarative mapping process will generate a new
:class:`_schema.Table` object against the :class:`_schema.MetaData` collection
-associated with the declarative base, and each specified
-:class:`_schema.Column` object will become part of the :attr:`.schema.Table.columns`
-collection of this :class:`_schema.Table` object. The :class:`_schema.Column`
-objects can omit their "name" field, which is usually the first positional
-argument to the :class:`_schema.Column` constructor; the declarative system
-will assign the key associated with each :class:`_schema.Column` as the name,
-to produce a :class:`_schema.Table` that is equivalent to::
+associated with the Declarative ``Base``; each instance of
+:func:`_orm.mapped_column` will then be used to generate a
+:class:`_schema.Column` object during this process, which will become part of
+the :attr:`.schema.Table.columns` collection of this :class:`_schema.Table`
+object.
+
+In the above example, Declarative will build a :class:`_schema.Table`
+construct that is equivalent to the following::
# equivalent Table object produced
user_table = Table(
"user",
Base.metadata,
Column("id", Integer, primary_key=True),
- Column("name", String),
- Column("fullname", String),
- Column("nickname", String),
+ Column("name", String(50)),
+ Column("fullname", String()),
+ Column("nickname", String(30)),
)
+When the ``User`` class above is mapped, this :class:`_schema.Table` object
+can be accessed directly via the ``__table__`` attribute; this is described
+further at :ref:`orm_declarative_metadata`.
+
+.. sidebar:: ``mapped_column()`` supersedes the use of ``Column()``
+
+ Users of 1.x SQLAlchemy will note the use of the :func:`_orm.mapped_column`
+ construct, which is new as of the SQLAlchemy 2.0 series. This ORM-specific
+ construct is intended first and foremost to be a drop-in replacement for
+ the use of :class:`_schema.Column` within Declarative mappings only, adding
+ new ORM-specific convenience features such as the ability to establish
+ :paramref:`_orm.mapped_column.deferred` within the construct, and most
+ importantly to indicate to typing tools such as Mypy_ and Pylance_ an
+ accurate representation of how the attribute will behave at runtime at
+ both the class level as well as the instance level. As will be seen in
+ the following sections, it's also at the forefront of a new
+ annotation-driven configuration style introduced in SQLAlchemy 2.0.
+
+ Users of legacy code should be aware that the :class:`_schema.Column` form
+ will always work in Declarative in the same way it always has. The different
+ forms of attribute mapping may also be mixed within a single mapping on an
+ attribute by attribute basis, so migration to the new form can be at
+ any pace. See the section :ref:`whatsnew_20_orm_declarative_typing` for
+ a step by step guide to migrating a Declarative model to the new form.
+
+
+The :func:`_orm.mapped_column` construct accepts all arguments that are
+accepted by the :class:`_schema.Column` construct, as well as additional
+ORM-specific arguments. The :paramref:`_orm.mapped_column.__name` field,
+indicating the name of the database column, is typically omitted, as the
+Declarative process will make use of the attribute name given to the construct
+and assign this as the name of the column (in the above example, this refers to
+the names ``id``, ``name``, ``fullname``, ``nickname``). Assigning an alternate
+:paramref:`_orm.mapped_column.__name` is valid as well, where the resulting
+:class:`_schema.Column` will use the given name in SQL and DDL statements,
+while the ``User`` mapped class will continue to allow access to the attribute
+using the attribute name given, independent of the name given to the column
+itself (more on this at :ref:`mapper_column_distinct_names`).
+
+.. tip::
+
+ The :func:`_orm.mapped_column` construct is **only valid within a
+ Declarative class mapping**. When constructing a :class:`_schema.Table`
+ object using Core as well as when using
+ :ref:`imperative table <orm_imperative_table_configuration>` configuration,
+ the :class:`_schema.Column` construct is still required in order to
+ indicate the presence of a database column.
+
.. seealso::
:ref:`mapping_columns_toplevel` - contains additional notes on affecting
how :class:`_orm.Mapper` interprets incoming :class:`.Column` objects.
+.. _orm_declarative_mapped_column:
+
+Using Annotated Declarative Table (Type Annotated Forms for ``mapped_column()``)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The :func:`_orm.mapped_column` construct is capable of deriving its column-configuration
+information from :pep:`484` type annotations associated with the attribute
+as declared in the Declarative mapped class. These type annotations,
+if used, **must**
+be present within a special SQLAlchemy type called :class:`_orm.Mapped`, which
+is a generic_ type that then indicates a specific Python type within it.
+
+Below illustrates the mapping from the previous section, adding the use of
+:class:`_orm.Mapped`::
+
+ from typing import Optional
+
+ from sqlalchemy import String
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import mapped_column
+
+ class Base(DeclarativeBase):
+ pass
+
+
+ class User(Base):
+ __tablename__ = "user"
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+ name: Mapped[str] = mapped_column(String(50))
+ fullname: Mapped[Optional[str]]
+ nickname: Mapped[Optional[str]] = mapped_column(String(30))
+
+Above, when Declarative processes each class attribute, each
+:func:`_orm.mapped_column` will derive additional arguments from the
+corresponding :class:`_orm.Mapped` type annotation on the left side, if
+present. Additionally, Declarative will generate an empty
+:func:`_orm.mapped_column` directive implicitly, whenever a
+:class:`_orm.Mapped` type annotation is encountered that does not have
+a value assigned to the attribute (this form is inspired by the similar
+style used in Python dataclasses_); this :func:`_orm.mapped_column` construct
+proceeds to derive its configuration from the :class:`_orm.Mapped`
+annotation present.
+
+.. _orm_declarative_mapped_column_nullability:
+
+``mapped_column()`` derives the datatype and nullability from the ``Mapped`` annotation
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The two qualities that :func:`_orm.mapped_column` derives from the
+:class:`_orm.Mapped` annotation are:
+
+* **datatype** - the Python type given inside :class:`_orm.Mapped`, as contained
+ within the ``typing.Optional`` construct if present, is associated with a
+ :class:`_sqltypes.TypeEngine` subclass such as :class:`.Integer`, :class:`.String`,
+ :class:`.DateTime`, or :class:`.Uuid`, to name a few common types.
+
+ The datatype is determined based on a dictionary of Python type to
+ SQLAlchemy datatype. This dictionary is completely customizable,
+ as detailed in the next section :ref:`orm_declarative_mapped_column_type_map`.
+ The default type map is implemented as in the code example below::
+
+ from typing import Any
+ from typing import Dict
+ from typing import Type
+
+ import datetime
+ import decimal
+ import uuid
+
+ from sqlalchemy import types
+
+ # default type mapping, deriving the type for mapped_column()
+ # from a Mapped[] annotation
+ type_map: Dict[Type[Any], TypeEngine[Any]] = {
+ bool: types.Boolean(),
+ bytes: types.LargeBinary(),
+ datetime.date: types.Date(),
+ datetime.datetime: types.DateTime(),
+ datetime.time: types.Time(),
+ datetime.timedelta: types.Interval(),
+ decimal.Decimal: types.Numeric(),
+ float: types.Float(),
+ int: types.Integer(),
+ str: types.String(),
+ uuid.UUID: types.Uuid(),
+ }
+
+ If the :func:`_orm.mapped_column` construct indicates an explicit type
+ as passed to the :paramref:`_orm.mapped_column.__type` argument, then
+ the given Python type is disregarded.
+
+* **nullability** - The :func:`_orm.mapped_column` construct will indicate
+ its :class:`_schema.Column` as ``NULL`` or ``NOT NULL`` first and foremost by
+ the presence of the :paramref:`_orm.mapped_column.nullable` parameter, passed
+ either as ``True`` or ``False``. Additionally , if the
+ :paramref:`_orm.mapped_column.primary_key` parameter is present and set to
+ ``True``, that will also imply that the column should be ``NOT NULL``.
+
+ In the absence of **both** of these parameters, the presence of
+ ``typing.Optional[]`` within the :class:`_orm.Mapped` type annotation will be
+ used to determine nullability, where ``typing.Optional[]`` means ``NULL``,
+ and the absense of ``typing.Optional[]`` means ``NOT NULL``. If there is no
+ ``Mapped[]`` annotation present at all, and there is no
+ :paramref:`_orm.mapped_column.nullable` or
+ :paramref:`_orm.mapped_column.primary_key` parameter, then SQLAlchemy's usual
+ default for :class:`_schema.Column` of ``NULL`` is used.
+
+ In the example below, the ``id`` and ``data`` columns will be ``NOT NULL``,
+ and the ``additional_info`` column will be ``NULL``::
+
+ from typing import Optional
+
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import mapped_column
+
+ class Base(DeclarativeBase):
+ pass
+
+ class SomeClass(Base):
+ __tablename__ = 'some_table'
+
+ # primary_key=True, therefore will be NOT NULL
+ id: Mapped[int] = mapped_column(primary_key=True)
+
+ # not Optional[], therefore will be NOT NULL
+ data: Mapped[str]
+
+ # Optional[], therefore will be NULL
+ additional_info: Mapped[Optional[str]]
+
+ It is also perfectly valid to have a :func:`_orm.mapped_column` whose
+ nullability is **different** from what would be implied by the annotation.
+ For example, an ORM mapped attribute may be annotated as allowing ``None``
+ within Python code that works with the object as it is first being created
+ and populated, however the value will ultimately be written to a database
+ column that is ``NOT NULL``. The :paramref:`_orm.mapped_column.nullable`
+ parameter, when present, will always take precedence::
+
+ class SomeClass(Base):
+ # ...
+
+ # will be String() NOT NULL, but can be None in Python
+ data: Mapped[Optional[str]] = mapped_column(nullable=False)
+
+ Similarly, a non-None attribute that's written to a database column that
+ for whatever reason needs to be NULL at the schema level,
+ :paramref:`_orm.mapped_column.nullable` may be set to ``True``::
+
+ class SomeClass(Base):
+ # ...
+
+ # will be String() NULL, but type checker will not expect
+ # the attribute to be None
+ data: Mapped[str] = mapped_column(nullable=True)
+
+.. _orm_declarative_mapped_column_type_map:
+
+Customizing the Type Map
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+The mapping of Python types to SQLAlchemy :class:`_types.TypeEngine` types
+described in the previous section defaults to a hardcoded dictionary
+present in the ``sqlalchemy.sql.sqltypes`` module. However, the :class:`_orm.registry`
+object that coordinates the Declarative mapping process will first consult
+a local, user defined dictionary of types which may be passed
+as the :paramref:`_orm.registry.type_annotation_map` parameter when
+constructing the :class:`_orm.registry`, which may be associated with
+the :class:`_orm.DeclarativeBase` superclass when first used.
+
+As an example, if we wish to make use of the :class:`_sqltypes.BIGINT` datatype for
+``int``, the :class:`_sqltypes.TIMESTAMP` datatype with ``timezone=True`` for
+``datetime.datetime``, and then only on Microsoft SQL Server we'd like to use
+:class:`_sqltypes.NVARCHAR` datatype when Python ``str`` is used,
+the registry and Declarative base could be configured as::
+
+ import datetime
+
+ from sqlalchemy import BIGINT, Integer, NVARCHAR, String, TIMESTAMP
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import Mapped, mapped_column, registry
+
+ class Base(DeclarativeBase):
+ registry = registry(type_annotation_map={
+ int: BIGINT,
+ datetime.datetime: TIMESTAMP(timezone=True),
+ str: String().with_variant(NVARCHAR, "mssql"),
+ })
+
+
+ class SomeClass(Base):
+ __tablename__ = 'some_table'
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+ date: Mapped[datetime.datetime]
+ status: Mapped[str]
+
+Below illustrates the CREATE TABLE statement generated for the above mapping,
+first on the Microsoft SQL Server backend, illustrating the ``NVARCHAR`` datatype::
+
+ >>> from sqlalchemy.schema import CreateTable
+ >>> from sqlalchemy.dialects import mssql, postgresql
+ >>> print(CreateTable(SomeClass.__table__).compile(dialect=mssql.dialect()))
+
+ CREATE TABLE some_table (
+ id BIGINT NOT NULL IDENTITY,
+ date TIMESTAMP NOT NULL,
+ status NVARCHAR(max) NOT NULL,
+ PRIMARY KEY (id)
+ )
+
+Then on the PostgreSQL backend, illustrating ``TIMESTAMP WITH TIME ZONE``::
+
+ >>> print(CreateTable(SomeClass.__table__).compile(dialect=postgresql.dialect()))
+
+ CREATE TABLE some_table (
+ id BIGSERIAL NOT NULL,
+ date TIMESTAMP WITH TIME ZONE NOT NULL,
+ status VARCHAR NOT NULL,
+ PRIMARY KEY (id)
+ )
+
+By making use of methods such as :meth:`.TypeEngine.with_variant`, we're able
+to build up a type map that's customized to what we need for different backends,
+while still being able to use succinct annotation-only :func:`_orm.mapped_column`
+configurations. There are two more levels of Python-type configurability
+available beyond this, described in the next two sections.
+
+
+.. _orm_declarative_mapped_column_type_map_pep593:
+
+Mapping Multiple Type Configurations to Python Types
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+As individual Python types may be associated with :class:`_types.TypeEngine`
+configurations of any variety by using the :paramref:`_orm.registry.type_annotation_map`
+parameter, an additional
+capability is the ability to associate a single Python type with different
+variants of a SQL type based on additional type qualifiers. One typical
+example of this is mapping the Python ``str`` datatype to ``VARCHAR``
+SQL types of different lengths. Another is mapping different varieties of
+``decimal.Decimal`` to differently sized ``NUMERIC`` columns.
+
+Python's typing system provides a great way to add additional metadata to a
+Python type which is by using the :pep:`593` ``Annotated`` generic type, which
+allows additional information to be bundled along with a Python type. The
+:func:`_orm.mapped_column` construct will correctly interpret an ``Annotated``
+object by identity when resolving it in the
+:paramref:`_orm.registry.type_annotation_map`, as in the example below where we
+declare two variants of :class:`.String` and :class:`.Numeric`::
+
+ from decimal import Decimal
+
+ from typing_extensions import Annotated
+
+ from sqlalchemy import Numeric
+ from sqlalchemy import String
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import mapped_column
+ from sqlalchemy.orm import registry
+
+ str_30 = Annotated[str, 30]
+ str_50 = Annotated[str, 50]
+ num_12_4 = Annotated[Decimal, 12]
+ num_6_2 = Annotated[Decimal, 6]
+
+ class Base(DeclarativeBase):
+ registry = registry(type_annotation_map={
+ str_30: String(30),
+ str_50: String(50),
+ num_12_4: Numeric(12, 4),
+ num_6_2: Numeric(6, 2)
+ })
+
+The Python type passed to the ``Annotated`` container, in the above example the
+``str`` and ``Decimal`` types, is important only for the benefit of typing
+tools; as far as the :func:`_orm.mapped_column` construct is concerned, it will only need
+perform a lookup of each type object in the
+:paramref:`_orm.registry.type_annotation_map` dictionary without actually
+looking inside of the ``Annotated`` object, at least in this particular
+context. Similarly, the arguments passed to ``Annotated`` beyond the underlying
+Python type itself are also not important, it's only that at least one argument
+must be present for the ``Annotated`` construct to be valid. We can then use
+these augmented types directly in our mapping where they will be matched to the
+more specific type constructions, as in the following example::
+
+ class SomeClass(Base):
+ __tablename__ = 'some_table'
+
+ short_name: Mapped[str_30] = mapped_column(primary_key=True)
+ long_name: Mapped[str_50]
+ num_value: Mapped[num_12_4]
+ short_num_value: Mapped[num_6_2]
+
+a CREATE TABLE for the above mapping will illustrate the different variants
+of ``VARCHAR`` and ``NUMERIC`` we've configured, and looks like::
+
+ >>> from sqlalchemy.schema import CreateTable
+ >>> print(CreateTable(SomeClass.__table__))
+ CREATE TABLE some_table (
+ short_name VARCHAR(30) NOT NULL,
+ long_name VARCHAR(50) NOT NULL,
+ num_value NUMERIC(12, 4) NOT NULL,
+ short_num_value NUMERIC(6, 2) NOT NULL,
+ PRIMARY KEY (short_name)
+ )
+
+While variety in linking ``Annotated`` types to different SQL types grants
+us a wide degree of flexibility, the next section illustrates a second
+way in which ``Annotated`` may be used with Declarative that is even
+more open ended.
+
+.. _orm_declarative_mapped_column_pep593:
+
+Mapping Whole Column Declarations to Python Types
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The previous section illustrated using :pep:`593` ``Annotated`` type
+instances as keys within the :paramref:`_orm.registry.type_annotation_map`
+dictionary. In this form, the :func:`_orm.mapped_column` construct does not
+actually look inside the ``Annotated`` object itself, it's instead
+used only as a dictionary key. However, Declarative also has the ability to extract
+an entire pre-established :func:`_orm.mapped_column` construct from
+an ``Annotated`` object directly. Using this form, we can define not only
+different varieties of SQL datatypes linked to Python types without using
+the :paramref:`_orm.registry.type_annotation_map` dictionary, we can also
+set up any number of arguments such as nullability, column defaults,
+and constraints in a reusable fashion.
+
+A set of ORM models will usually have some kind of primary
+key style that is common to all mapped classes. There also may be
+common column configurations such as timestamps with defaults and other fields of
+pre-established sizes and configurations. We can compose these configurations
+into :func:`_orm.mapped_column` instances that we then bundle directly into
+instances of ``Annotated``, which are then re-used in any number of class
+declarations. Declarative will unpack an ``Annotated`` object
+when provided in this manner, skipping over any other directives that don't
+apply to SQLAlchemy and searching only for SQLAlchemy ORM constructs.
+
+The example below illustrates a variety of pre-configured field types used
+in this way, where we define ``intpk`` that represents an :class:`.Integer` primary
+key column, ``timestamp`` that represents a :class:`.DateTime` type
+which will use ``CURRENT_TIMESTAMP`` as a DDL level column default,
+and ``required_name`` which is a :class:`.String` of length 30 that's
+``NOT NULL``::
+
+ import datetime
+
+ from typing_extensions import Annotated
+
+ from sqlalchemy import func
+ from sqlalchemy import String
+ from sqlalchemy.orm import mapped_column
+
+
+ intpk = Annotated[int, mapped_column(primary_key=True)]
+ timestamp = Annotated[
+ datetime.datetime,
+ mapped_column(nullable=False, server_default=func.CURRENT_TIMESTAMP()),
+ ]
+ required_name = Annotated[str, mapped_column(String(30), nullable=False)]
+
+The above ``Annotated`` objects can then be used directly within
+:class:`_orm.Mapped`, where the pre-configured :func:`_orm.mapped_column`
+constructs will be extracted and copied to a new instance that will be
+specific to each attribute::
+
+ class Base(DeclarativeBase):
+ pass
+
+
+ class SomeClass(Base):
+ __tablename__ = "some_table"
+
+ id: Mapped[intpk]
+ name: Mapped[required_name]
+ created_at: Mapped[timestamp]
+
+``CREATE TABLE`` for our above mapping looks like::
+
+ >>> from sqlalchemy.schema import CreateTable
+ >>> print(CreateTable(SomeClass.__table__))
+ CREATE TABLE some_table (
+ id INTEGER NOT NULL,
+ name VARCHAR(30) NOT NULL,
+ created_at DATETIME DEFAULT CURRENT_TIMESTAMP NOT NULL,
+ PRIMARY KEY (id)
+ )
+
+When using ``Annotated`` types in this way, the configuration of the type
+may also be affected on a per-attribute basis. For the types in the above
+example that feature explcit use of :paramref:`_orm.mapped_column.nullable`,
+we can apply the ``Optional[]`` generic modifier to any of our types so that
+the field is optional or not at the Python level, which will be independent
+of the ``NULL`` / ``NOT NULL`` setting that takes place in the database::
+
+ from typing_extensions import Annotated
+
+ import datetime
+ from typing import Optional
+
+ from sqlalchemy.orm import DeclarativeBase
+
+ timestamp = Annotated[
+ datetime.datetime,
+ mapped_column(nullable=False),
+ ]
+
+ class Base(DeclarativeBase):
+ pass
+
+ class SomeClass(Base):
+
+ # ...
+
+ # pep-484 type will be Optional, but column will be
+ # NOT NULL
+ created_at: Mapped[Optional[timestamp]]
+
+The :func:`_orm.mapped_column` construct is also reconciled with an explicitly
+passed :func:`_orm.mapped_column` construct, whose arguments will take precedence
+over those of the ``Annotated`` construct. Below we add a :class:`.ForeignKey`
+constraint to our integer primary key and also use an alternate server
+default for the ``created_at`` column::
+
+ import datetime
+
+ from typing_extensions import Annotated
+
+ from sqlalchemy import ForeignKey
+ from sqlalchemy import func
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import mapped_column
+ from sqlalchemy.schema import CreateTable
+
+ intpk = Annotated[int, mapped_column(primary_key=True)]
+ timestamp = Annotated[
+ datetime.datetime,
+ mapped_column(nullable=False, server_default=func.CURRENT_TIMESTAMP()),
+ ]
+
+ class Base(DeclarativeBase):
+ pass
+
+ class Parent(Base):
+ __tablename__ = 'parent'
+
+ id: Mapped[intpk]
+
+ class SomeClass(Base):
+ __tablename__ = 'some_table'
+
+ # add ForeignKey to mapped_column(Integer, primary_key=True)
+ id: Mapped[intpk] = mapped_column(ForeignKey('parent.id'))
+
+ # change server default from CURRENT_TIMESTAMP to UTC_TIMESTAMP
+ created_at: Mapped[timestamp] = mapped_column(server_default=func.UTC_TIMESTAMP())
+
+The CREATE TABLE statement illustrates these per-attribute settings,
+adding a ``FOREIGN KEY`` constraint as well as substituting
+``UTC_TIMESTAMP`` for ``CURRENT_TIMESTAMP``::
+
+ >>> from sqlalchemy.schema import CreateTable
+ >>> print(CreateTable(SomeClass.__table__))
+ CREATE TABLE some_table (
+ id INTEGER NOT NULL,
+ created_at DATETIME DEFAULT UTC_TIMESTAMP() NOT NULL,
+ PRIMARY KEY (id),
+ FOREIGN KEY(id) REFERENCES parent (id)
+ )
+
+.. note:: The above feature of :func:`_orm.mapped_column` can in theory
+ work for other constructs as well such as :func:`_orm.relationship` and
+ :func:`_orm.composite`. At the moment, these other use cases are not
+ implemented and raise a ``NotImplementedError``, but may be implemented
+ in future releases.
+
+Dataclass features in ``mapped_column()``
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The :func:`_orm.mapped_column` construct integrates with SQLAlchemy's
+"native dataclasses" feature, discussed at
+:ref:`orm_declarative_native_dataclasses`. See that section for current
+background on additional directives supported by :func:`_orm.mapped_column`.
+
+
+
.. _orm_declarative_metadata:
Accessing Table and Metadata
A class may also specify the ``__table_args__`` declarative attribute,
as well as the ``__tablename__`` attribute, in a dynamic style using the
-:func:`_orm.declared_attr` method decorator. See the section
-:ref:`declarative_mixins` for examples on how this is often used.
+:func:`_orm.declared_attr` method decorator. See
+:ref:`orm_mixins_toplevel` for background.
.. _orm_declarative_table_schema_name:
tables, this option is passed like any other to the ``__table_args__``
dictionary::
+ from sqlalchemy.orm import DeclarativeBase
+
+ class Base(DeclarativeBase):
+ pass
class MyClass(Base):
__tablename__ = "sometable"
The schema name can also be applied to all :class:`_schema.Table` objects
globally by using the :paramref:`_schema.MetaData.schema` parameter documented
at :ref:`schema_metadata_schema_name`. The :class:`_schema.MetaData` object
-may be constructed separately and passed either to :func:`_orm.registry`
-or :func:`_orm.declarative_base`::
+may be constructed separately and associated with a :class:`_orm.DeclarativeBase`
+subclass by assigning to the ``metadata`` attribute directly::
from sqlalchemy import MetaData
+ from sqlalchemy.orm import DeclarativeBase
metadata_obj = MetaData(schema="some_schema")
- Base = declarative_base(metadata=metadata_obj)
+ class Base(DeclarativeBase):
+ metadata = metadata_obj
class MyClass(Base):
:ref:`schema_table_schema_name` - in the :ref:`metadata_toplevel` documentation.
+.. _orm_declarative_column_options:
+
+Setting Load and Persistence Options for Declarative Mapped Columns
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The :func:`_orm.mapped_column` construct accepts additional ORM-specific
+arguments that affect how the generated :class:`_schema.Column` is
+mapped, affecting its load and persistence-time behavior. Options
+that are commonly used include:
+
+* **deferred column loading** - The :paramref:`_orm.mapped_column.deferred`
+ boolean establishes the :class:`_schema.Column` using
+ :ref:`deferred column loading <deferred>` by default. In the example
+ below, the ``User.bio`` column will not be loaded by default, but only
+ when accessed::
+
+ class User(Base):
+ __tablename__ = "user"
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+ name: Mapped[str]
+ bio: Mapped[str] = mapped_column(Text, deferred=True)
+
+ .. seealso::
+
+ :ref:`deferred` - full description of deferred column loading
+
+* **active history** - The :paramref:`_orm.mapped_column.active_history`
+ ensures that upon change of value for the attribute, the previous value
+ will have been loaded and made part of the :attr:`.AttributeState.history`
+ collection when inspecting the history of the attribute. This may incur
+ additional SQL statements::
+
+ class User(Base):
+ __tablename__ = "user"
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+ important_identifier: Mapped[str] = mapped_column(active_history=True)
+
+See the docstring for :func:`_orm.mapped_column` for a list of supported
+parameters.
+
+.. seealso::
+
+ :ref:`orm_imperative_table_column_options` - describes using
+ :func:`_orm.column_property` and :func:`_orm.deferred` for use with
+ Imperative Table configuration
+
+.. _mapper_column_distinct_names:
+
+.. _orm_declarative_table_column_naming:
+
+Naming Declarative Mapped Columns Explicitly
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+All of the examples thus far feature the :func:`_orm.mapped_column` construct
+linked to an ORM mapped attribute, where the Python attribute name given
+to the :func:`_orm.mapped_column` is also that of the column as we see in
+CREATE TABLE statements as well as queries. The name for a column as
+expressed in SQL may be indicated by passing the string positional argument
+:paramref:`_orm.mapped_column.__name` as the first positional argument.
+In the example below, the ``User`` class is mapped with alternate names
+given to the columns themselves::
+
+ class User(Base):
+ __tablename__ = 'user'
+
+ id: Mapped[int] = mapped_column('user_id', primary_key=True)
+ name: Mapped[str] = mapped_column('user_name')
+
+Where above ``User.id`` resolves to a column named ``user_id``
+and ``User.name`` resolves to a column named ``user_name``. We
+may write a :func:`_sql.select` statement using our Python attribute names
+and will see the SQL names generated::
+
+ >>> from sqlalchemy import select
+ >>> print(select(User.id, User.name).where(User.name == 'x'))
+ SELECT "user".user_id, "user".user_name
+ FROM "user"
+ WHERE "user".user_name = :user_name_1
+
+
+.. seealso::
+
+ :ref:`orm_imperative_table_column_naming` - applies to Imperative Table
+
.. _orm_declarative_table_adding_columns:
Appending additional columns to an existing Declarative mapped class
For a declarative class that is declared using a declarative base class,
the underlying metaclass :class:`.DeclarativeMeta` includes a ``__setattr__()``
-method that will intercept additional :class:`.Column` objects and
+method that will intercept additional :func:`_orm.mapped_column` or Core
+:class:`.Column` objects and
add them to both the :class:`.Table` using :meth:`.Table.append_column`
as well as to the existing :class:`.Mapper` using :meth:`.Mapper.add_property`::
- MyClass.some_new_column = Column("data", Unicode)
+ MyClass.some_new_column = mapped_column(String)
+
+Using core :class:`_schema.Column`::
+
+ MyClass.some_new_column = Column(String)
+
+All arguments are supported including an alternate name, such as
+``MyClass.some_new_column = mapped_column("some_name", String)``. However,
+the SQL type must be passed to the :func:`_orm.mapped_column` or
+:class:`_schema.Column` object explicitly, as in the above examples where
+the :class:`_sqltypes.String` type is passed. There's no capability for
+the :class:`_orm.Mapped` annotation type to take part in the operation.
Additional :class:`_schema.Column` objects may also be added to a mapping
in the specific circumstance of using single table inheritance, where
no :class:`.Table` of their own. This is illustrated in the section
:ref:`single_inheritance`.
+.. note:: Assignment of mapped
+ properties to an already mapped class will only
+ function correctly if the "declarative base" class is used, meaning
+ the user-defined subclass of :class:`_orm.DeclarativeBase` or the
+ dynamically generated class returned by :func:`_orm.declarative_base`
+ or :meth:`_orm.registry.generate_base`. This "base" class includes
+ a Python metaclass which implements a special ``__setattr__()`` method
+ that intercepts these operations.
+
+ Runtime assignment of class-mapped attributes to a mapped class will **not** work
+ if the class is mapped using decorators like :meth:`_orm.registry.mapped`
+ or imperative functions like :meth:`_orm.registry.map_imperatively`.
+
+
.. _orm_imperative_table_configuration:
Declarative with Imperative Table (a.k.a. Hybrid Declarative)
from sqlalchemy import Column, ForeignKey, Integer, String
- from sqlalchemy.orm import declarative_base
+ from sqlalchemy.orm import DeclarativeBase
- Base = declarative_base()
+ class Base(DeclarativeBase):
+ pass
# construct a Table directly. The Base.metadata collection is
# usually a good choice for MetaData but any MetaData
:ref:`orm_declarative_dataclasses`
+.. _orm_imperative_table_column_naming:
+
+Alternate Attribute Names for Mapping Table Columns
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The section :ref:`orm_declarative_table_column_naming` illustrated how to
+use :func:`_orm.mapped_column` to provide a specific name for the generated
+:class:`_schema.Column` object separate from the attribute name under which
+it is mapped.
+
+When using Imperative Table configuration, we already have
+:class:`_schema.Column` objects present. To map these to alternate names
+we may assign the :class:`_schema.Column` to the desired attributes
+directly::
+
+ user_table = Table(
+ "user",
+ Base.metadata,
+ Column("user_id", Integer, primary_key=True),
+ Column("user_name", String),
+ )
+
+ class User(Base):
+ __table__ = user_table
+
+ id = user_table.c.user_id
+ name = user_table.c.user_name
+
+The ``User`` mapping above will refer to the ``"user_id"`` and ``"user_name"``
+columns via the ``User.id`` and ``User.name`` attributes, in the same
+way as demonstrated at :ref:`orm_declarative_table_column_naming`.
+
+One caveat to the above mapping is that the direct inline link to
+:class:`_schema.Column` will not be typed correctly when using
+:pep:`484` typing tools. A strategy to resolve this is to apply the
+:class:`_schema.Column` objects within the :func:`_orm.column_property`
+function; while the :class:`_orm.Mapper` already generates this property
+object for its internal use automatically, by naming it in the class
+declaration, typing tools will be able to match the attribute to the
+:class:`_orm.Mapped` annotation::
+
+ from sqlalchemy.orm import column_property
+ from sqlalchemy.orm import Mapped
+
+ class User(Base):
+ __table__ = user_table
+
+ id: Mapped[int] = column_property(user_table.c.user_id)
+ name: Mapped[str] = column_property(user_table.c.user_name)
+
+.. seealso::
+
+ :ref:`orm_declarative_table_column_naming` - applies to Declarative Table
+
+.. _column_property_options:
+
+.. _orm_imperative_table_column_options:
+
+Applying Load, Persistence and Mapping Options for Mapped Table Columns
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The section :ref:`orm_declarative_column_options` reviewed how to set load
+and persistence options when using the :func:`_orm.mapped_column` construct
+with Declarative Table configuration. When using Imperative Table configuration,
+we already have existing :class:`_schema.Column` objects that are mapped.
+In order to map these :class:`_schema.Column` objects along with additional
+parameters that are specific to the ORM mapping, we may use the
+:func:`_orm.column_property` and :func:`_orm.deferred` constructs in order to
+associate additional parameters with the column. Options include:
+
+* **deferred column loading** - The :func:`_orm.deferred` function is shorthand
+ for invoking :func:`_orm.column_property` with the
+ :paramref:`_orm.column_property.deferred` parameter set to ``True``;
+ this construct establishes the :class:`_schema.Column` using
+ :ref:`deferred column loading <deferred>` by default. In the example
+ below, the ``User.bio`` column will not be loaded by default, but only
+ when accessed::
+
+ from sqlalchemy.orm import deferred
+
+ user_table = Table(
+ "user",
+ Base.metadata,
+ Column("id", Integer, primary_key=True),
+ Column("name", String),
+ Column("bio", Text),
+ )
+
+ class User(Base):
+ __table__ = user_table
+
+ bio = deferred(user_table.c.bio)
+
+
+ .. seealso::
+
+ :ref:`deferred` - full description of deferred column loading
+
+* **active history** - The :paramref:`_orm.column_property.active_history`
+ ensures that upon change of value for the attribute, the previous value
+ will have been loaded and made part of the :attr:`.AttributeState.history`
+ collection when inspecting the history of the attribute. This may incur
+ additional SQL statements::
+
+ from sqlalchemy.orm import deferred
+
+ user_table = Table(
+ "user",
+ Base.metadata,
+ Column("id", Integer, primary_key=True),
+ Column("important_identifier", String)
+ )
+
+ class User(Base):
+ __table__ = user_table
+
+ important_identifier = column_property(user_table.c.important_identifier, active_history=True)
+
+
+.. seealso::
+
+ The :func:`_orm.column_property` construct is also important for cases
+ where classes are mapped to alternative FROM clauses such as joins and
+ selects. More background on these cases is at:
+
+ * :ref:`maptojoin`
+
+ * :ref:`mapper_sql_expressions`
+
+ For Declarative Table configuration with :func:`_orm.mapped_column`,
+ most options are available directly; see the section
+ :ref:`orm_declarative_column_options` for examples.
+
+
+
.. _orm_declarative_reflected:
Mapping Declaratively with Reflected Tables
introspected from the database, using the reflection process described at
:ref:`metadata_reflection`.
-A very simple way to map a class to a table reflected from the database is to
+A simple way to map a class to a table reflected from the database is to
use a declarative hybrid mapping, passing the
-:paramref:`_schema.Table.autoload_with` parameter to the
+:paramref:`_schema.Table.autoload_with` parameter to the constructor for
:class:`_schema.Table`::
from sqlalchemy import create_engine
from sqlalchemy import Table
- from sqlalchemy.orm import declarative_base
+ from sqlalchemy.orm import DeclarativeBase
engine = create_engine(
"postgresql+psycopg2://user:pass@hostname/my_existing_database"
)
- Base = declarative_base()
+ class Base(DeclarativeBase):
+ pass
class MyClass(Base):
__table__ = Table(
autoload_with=engine,
)
-A variant on the above pattern that scales much better is to use the
+A variant on the above pattern that scales for many tables is to use the
:meth:`.MetaData.reflect` method to reflect a full set of :class:`.Table`
objects at once, then refer to them from the :class:`.MetaData`::
from sqlalchemy import create_engine
from sqlalchemy import Table
- from sqlalchemy.orm import declarative_base
+ from sqlalchemy.orm import DeclarativeBase
engine = create_engine(
"postgresql+psycopg2://user:pass@hostname/my_existing_database"
)
- Base = declarative_base()
+ class Base(DeclarativeBase):
+ pass
Base.metadata.reflect(engine)
class MyClass(Base):
__table__ = Base.metadata.tables['mytable']
-.. seealso::
-
- :ref:`mapper_automated_reflection_schemes` - further notes on using
- table reflection with mapped classes
-
-A major downside to the above approach is that the mapped classes cannot
+One caveat to the approach of using ``__table__`` is that the mapped classes cannot
be declared until the tables have been reflected, which requires the database
connectivity source to be present while the application classes are being
declared; it's typical that classes are declared as the modules of an
application are being imported, but database connectivity isn't available
until the application starts running code so that it can consume configuration
information and create an engine. There are currently two approaches
-to working around this.
+to working around this, described in the next two sections.
.. _orm_declarative_reflected_deferred_reflection:
use the ``__tablename__`` attribute::
from sqlalchemy.ext.declarative import DeferredReflection
- from sqlalchemy.orm import declarative_base
-
- Base = declarative_base()
+ from sqlalchemy.orm import DeclarativeBase
+ class Base(DeclarativeBase):
+ pass
class Reflected(DeferredReflection):
__abstract__ = True
class Bar(Reflected, Base):
__tablename__ = "bar"
- foo_id = Column(Integer, ForeignKey("foo.id"))
+ foo_id = mapped_column(Integer, ForeignKey("foo.id"))
Above, we create a mixin class ``Reflected`` that will serve as a base
for classes in our declarative hierarchy that should become mapped when
class naming and relationship naming schemes, automap is oriented towards an
expedient zero-configuration style of working. If an application wishes to have
a fully explicit model that makes use of table reflection, the
-:ref:`orm_declarative_reflected_deferred_reflection` may be preferable.
+:ref:`DeferredReflection <orm_declarative_reflected_deferred_reflection>`
+class may be preferable for its less automated approach.
.. seealso::
:ref:`automap_toplevel`
+
+
+.. _mapper_automated_reflection_schemes:
+
+Automating Column Naming Schemes from Reflected Tables
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+When using any of the previous reflection techniques, we have the option
+to change the naming scheme by which columns are mapped. The
+:class:`_schema.Column` object includes a parameter
+:paramref:`_schema.Column.key` which is a string name that determines
+under what name
+this :class:`_schema.Column` will be present in the :attr:`_schema.Table.c`
+collection, independently of the SQL name of the column. This key is also
+used by :class:`_orm.Mapper` as the attribute name under which the
+:class:`_schema.Column` will be mapped, if not supplied through other
+means such as that illustrated at :ref:`orm_imperative_table_column_naming`.
+
+When working with table reflection, we can intercept the parameters that
+will be used for :class:`_schema.Column` as they are received using
+the :meth:`_events.DDLEvents.column_reflect` event and apply whatever
+changes we need, including the ``.key`` attribute but also things like
+datatypes.
+
+The event hook is most easily
+associated with the :class:`_schema.MetaData` object that's in use
+as illustrated below::
+
+ from sqlalchemy import event
+ from sqlalchemy.orm import DeclarativeBase
+
+ class Base(DeclarativeBase):
+ pass
+
+
+ @event.listens_for(Base.metadata, "column_reflect")
+ def column_reflect(inspector, table, column_info):
+ # set column.key = "attr_<lower_case_name>"
+ column_info['key'] = "attr_%s" % column_info['name'].lower()
+
+With the above event, the reflection of :class:`_schema.Column` objects will be intercepted
+with our event that adds a new ".key" element, such as in a mapping as below::
+
+ class MyClass(Base):
+ __table__ = Table("some_table", Base.metadata,
+ autoload_with=some_engine)
+
+The approach also works with both the :class:`.DeferredReflection` base class
+as well as with the :ref:`automap_toplevel` extension. For automap
+specifically, see the section :ref:`automap_intercepting_columns` for
+background.
+
+.. seealso::
+
+ :ref:`orm_declarative_reflected`
+
+ :meth:`_events.DDLEvents.column_reflect`
+
+ :ref:`automap_intercepting_columns` - in the :ref:`automap_toplevel` documentation
+
+
+.. _mapper_primary_key:
+
+Mapping to an Explicit Set of Primary Key Columns
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The :class:`.Mapper` construct in order to successfully map a table always
+requires that at least one column be identified as the "primary key" for
+that selectable. This is so that when an ORM object is loaded or persisted,
+it can be placed in the :term:`identity map` with an appropriate
+:term:`identity key`.
+
+In those cases where the a reflected table to be mapped does not include
+a primary key constraint, as well as in the general case for
+:ref:`mapping against arbitrary selectables <orm_mapping_arbitrary_subqueries>`
+where primary key columns might not be present, the
+:paramref:`.Mapper.primary_key` parameter is provided so that any set of
+columns may be configured as the "primary key" for the table, as far as
+ORM mapping is concerned.
+
+Given the following example of an Imperative Table
+mapping against an existing :class:`.Table` object where the table does not
+have any declared primary key (as may occur in reflection scenarios), we may
+map such a table as in the following example::
+
+ from sqlalchemy import Column
+ from sqlalchemy import MetaData
+ from sqlalchemy import String
+ from sqlalchemy import Table
+ from sqlalchemy import UniqueConstraint
+ from sqlalchemy.orm import DeclarativeBase
+
+
+ metadata = MetaData()
+ group_users = Table(
+ "group_users",
+ metadata,
+ Column("user_id", String(40), nullable=False),
+ Column("group_id", String(40), nullable=False),
+ UniqueConstraint("user_id", "group_id")
+ )
+
+
+ class Base(DeclarativeBase):
+ pass
+
+
+ class GroupUsers(Base):
+ __table__ = group_users
+ __mapper_args__ = {
+ "primary_key": [group_users.c.user_id, group_users.c.group_id]
+ }
+
+Above, the ``group_users`` table is an association table of some kind
+with string columns ``user_id`` and ``group_id``, but no primary key is set up;
+instead, there is only a :class:`.UniqueConstraint` establishing that the
+two columns represent a unique key. The :class:`.Mapper` does not automatically
+inspect unique constraints for primary keys; instead, we make use of the
+:paramref:`.Mapper.primary_key` parameter, passing a collection of
+``[group_users.c.user_id, group_users.c.group_id]``, indicating that these two
+columns should be used in order to construct the identity key for instances
+of the ``GroupUsers`` class.
+
+
+.. _include_exclude_cols:
+
+Mapping a Subset of Table Columns
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Sometimes table reflection may provide a :class:`_schema.Table` with many
+columns that are not important for our needs and may be safely ignored.
+For such a table that has lots of columns that don't need to be referenced
+in the application, the :paramref:`_orm.Mapper.include_properties`
+or :paramref:`_orm.Mapper.exclude_properties` parameters can indicate
+a subset of columns to be mapped, where other columns from the
+target :class:`_schema.Table` will not be considered by the ORM in any
+way. Example::
+
+ class User(Base):
+ __table__ = user_table
+ __mapper_args__ = {
+ 'include_properties' :['user_id', 'user_name']
+ }
+
+In the above example, the ``User`` class will map to the ``user_table`` table, only including
+the ``user_id`` and ``user_name`` columns - the rest are not referenced.
+
+Similarly::
+
+ class Address(Base):
+ __table__ = address_table
+ __mapper_args__ = {
+ 'exclude_properties' : ["street", "city", "state", "zip"]
+ }
+
+will map the ``Address`` class to the ``address_table`` table, including
+all columns present except ``street``, ``city``, ``state``, and ``zip``.
+
+As indicated in the two examples, columns may be referred towards either
+by string name or by referring to the :class:`_schema.Column` object
+directly. Referring to the object directly may be useful for explicitness as well as to resolve ambiguities when
+mapping to multi-table constructs that might have repeated names::
+
+ class User(Base):
+ __table__ = user_table
+ __mapper_args__ = {
+ 'include_properties' :[user_table.c.user_id, user_table.c.user_name]
+ }
+
+When columns are not included in a mapping, these columns will not be
+referenced in any SELECT statements emitted when executing :func:`_sql.select`
+or legacy :class:`_query.Query` objects, nor will there be any mapped attribute
+on the mapped class which represents the column; assigning an attribute of that
+name will have no effect beyond that of a normal Python attribute assignment.
+
+However, it is important to note that **schema level column defaults WILL
+still be in effect** for those :class:`_schema.Column` objects that include them,
+even though they may be excluded from the ORM mapping.
+
+"Schema level column defaults" refers to the defaults described at
+:ref:`metadata_defaults` including those configured by the
+:paramref:`_schema.Column.default`, :paramref:`_schema.Column.onupdate`,
+:paramref:`_schema.Column.server_default` and
+:paramref:`_schema.Column.server_onupdate` parameters. These constructs
+continue to have normal effects because in the case of
+:paramref:`_schema.Column.default` and :paramref:`_schema.Column.onupdate`, the
+:class:`_schema.Column` object is still present on the underlying
+:class:`_schema.Table`, thus allowing the default functions to take place when
+the ORM emits an INSERT or UPDATE, and in the case of
+:paramref:`_schema.Column.server_default` and
+:paramref:`_schema.Column.server_onupdate`, the relational database itself
+emits these defaults as a server side behavior.
+
+
+
+.. _mypy: https://mypy.readthedocs.io/en/stable/
+
+.. _pylance: https://github.com/microsoft/pylance-release
+
+.. _generic: https://peps.python.org/pep-0484/#generics
+
+.. _dataclasses: https://docs.python.org/3/library/dataclasses.html
(the many-to-many pattern is described at :ref:`relationships_many_to_many`)::
from sqlalchemy import Column, ForeignKey, Integer, String, Table
- from sqlalchemy.orm import declarative_base, relationship
+ from sqlalchemy.orm import DeclarativeBase, relationship
- Base = declarative_base()
+ class Base(DeclarativeBase):
+ pass
class User(Base):
__tablename__ = "user"
- id = Column(Integer, primary_key=True)
- name = Column(String(64))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(64))
kw = relationship("Keyword", secondary=lambda: user_keyword_table)
def __init__(self, name):
class Keyword(Base):
__tablename__ = "keyword"
- id = Column(Integer, primary_key=True)
- keyword = Column("keyword", String(64))
+ id = mapped_column(Integer, primary_key=True)
+ keyword = mapped_column("keyword", String(64))
def __init__(self, keyword):
self.keyword = keyword
class User(Base):
__tablename__ = "user"
- id = Column(Integer, primary_key=True)
- name = Column(String(64))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(64))
kw = relationship("Keyword", secondary=lambda: user_keyword_table)
def __init__(self, name):
from sqlalchemy import Column, ForeignKey, Integer, String
from sqlalchemy.ext.associationproxy import association_proxy
- from sqlalchemy.orm import declarative_base, relationship
+ from sqlalchemy.orm import DeclarativeBase, relationship
- Base = declarative_base()
+ class Base(DeclarativeBase):
+ pass
class User(Base):
__tablename__ = "user"
- id = Column(Integer, primary_key=True)
- name = Column(String(64))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(64))
user_keyword_associations = relationship(
"UserKeywordAssociation",
class UserKeywordAssociation(Base):
__tablename__ = "user_keyword"
- user_id = Column(Integer, ForeignKey("user.id"), primary_key=True)
- keyword_id = Column(Integer, ForeignKey("keyword.id"), primary_key=True)
- special_key = Column(String(50))
+ user_id = mapped_column(Integer, ForeignKey("user.id"), primary_key=True)
+ keyword_id = mapped_column(Integer, ForeignKey("keyword.id"), primary_key=True)
+ special_key = mapped_column(String(50))
user = relationship(User, back_populates="user_keyword_associations")
class Keyword(Base):
__tablename__ = "keyword"
- id = Column(Integer, primary_key=True)
- keyword = Column("keyword", String(64))
+ id = mapped_column(Integer, primary_key=True)
+ keyword = mapped_column("keyword", String(64))
def __init__(self, keyword):
self.keyword = keyword
from sqlalchemy import Column, ForeignKey, Integer, String
from sqlalchemy.ext.associationproxy import association_proxy
- from sqlalchemy.orm import declarative_base, relationship
+ from sqlalchemy.orm import DeclarativeBase, relationship
from sqlalchemy.orm.collections import attribute_mapped_collection
- Base = declarative_base()
+ class Base(DeclarativeBase):
+ pass
class User(Base):
__tablename__ = "user"
- id = Column(Integer, primary_key=True)
- name = Column(String(64))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(64))
# user/user_keyword_associations relationship, mapping
# user_keyword_associations with a dictionary against "special_key" as key.
class UserKeywordAssociation(Base):
__tablename__ = "user_keyword"
- user_id = Column(Integer, ForeignKey("user.id"), primary_key=True)
- keyword_id = Column(Integer, ForeignKey("keyword.id"), primary_key=True)
- special_key = Column(String)
+ user_id = mapped_column(Integer, ForeignKey("user.id"), primary_key=True)
+ keyword_id = mapped_column(Integer, ForeignKey("keyword.id"), primary_key=True)
+ special_key = mapped_column(String)
user = relationship(
User,
class Keyword(Base):
__tablename__ = "keyword"
- id = Column(Integer, primary_key=True)
- keyword = Column("keyword", String(64))
+ id = mapped_column(Integer, primary_key=True)
+ keyword = mapped_column("keyword", String(64))
def __init__(self, keyword):
self.keyword = keyword
from sqlalchemy import Column, ForeignKey, Integer, String
from sqlalchemy.ext.associationproxy import association_proxy
- from sqlalchemy.orm import declarative_base, relationship
+ from sqlalchemy.orm import DeclarativeBase, relationship
from sqlalchemy.orm.collections import attribute_mapped_collection
- Base = declarative_base()
+ class Base(DeclarativeBase):
+ pass
class User(Base):
__tablename__ = "user"
- id = Column(Integer, primary_key=True)
- name = Column(String(64))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(64))
user_keyword_associations = relationship(
"UserKeywordAssociation",
class UserKeywordAssociation(Base):
__tablename__ = "user_keyword"
- user_id = Column(ForeignKey("user.id"), primary_key=True)
- keyword_id = Column(ForeignKey("keyword.id"), primary_key=True)
- special_key = Column(String)
+ user_id = mapped_column(ForeignKey("user.id"), primary_key=True)
+ keyword_id = mapped_column(ForeignKey("keyword.id"), primary_key=True)
+ special_key = mapped_column(String)
user = relationship(
User,
back_populates="user_keyword_associations",
class Keyword(Base):
__tablename__ = "keyword"
- id = Column(Integer, primary_key=True)
- keyword = Column("keyword", String(64))
+ id = mapped_column(Integer, primary_key=True)
+ keyword = mapped_column("keyword", String(64))
def __init__(self, keyword):
self.keyword = keyword
from sqlalchemy import Column, ForeignKey, Integer, String
from sqlalchemy.ext.associationproxy import association_proxy
- from sqlalchemy.orm import declarative_base, relationship
+ from sqlalchemy.orm import DeclarativeBase, relationship
from sqlalchemy.orm.collections import attribute_mapped_collection
- Base = declarative_base()
+ class Base(DeclarativeBase):
+ pass
class User(Base):
class A(Base):
__tablename__ = "test_a"
- id = Column(Integer, primary_key=True)
+ id = mapped_column(Integer, primary_key=True)
ab = relationship("AB", backref="a", uselist=False)
b = association_proxy(
"ab", "b", creator=lambda b: AB(b=b), cascade_scalar_deletes=True
class B(Base):
__tablename__ = "test_b"
- id = Column(Integer, primary_key=True)
+ id = mapped_column(Integer, primary_key=True)
ab = relationship("AB", backref="b", cascade="all, delete-orphan")
class AB(Base):
__tablename__ = "test_ab"
- a_id = Column(Integer, ForeignKey(A.id), primary_key=True)
- b_id = Column(Integer, ForeignKey(B.id), primary_key=True)
+ a_id = mapped_column(Integer, ForeignKey(A.id), primary_key=True)
+ b_id = mapped_column(Integer, ForeignKey(B.id), primary_key=True)
An assignment to ``A.b`` will generate an ``AB`` object::
select,
)
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
- from sqlalchemy.orm import declarative_base, relationship, selectinload
+ from sqlalchemy.orm import DeclarativeBase, relationship, selectinload
- Base = declarative_base()
+ class Base(DeclarativeBase):
+ pass
class A(Base):
__tablename__ = "a"
- id = Column(Integer, primary_key=True)
- data = Column(String)
- create_date = Column(DateTime, server_default=func.now())
+ id = mapped_column(Integer, primary_key=True)
+ data = mapped_column(String)
+ create_date = mapped_column(DateTime, server_default=func.now())
bs = relationship("B")
# required in order to access columns with server defaults
class B(Base):
__tablename__ = "b"
- id = Column(Integer, primary_key=True)
- a_id = Column(ForeignKey("a.id"))
- data = Column(String)
+ id = mapped_column(Integer, primary_key=True)
+ a_id = mapped_column(ForeignKey("a.id"))
+ data = mapped_column(String)
async def async_main():
# ...
# column with a server_default, or SQL expression default
- create_date = Column(DateTime, server_default=func.now())
+ create_date = mapped_column(DateTime, server_default=func.now())
# add this so that it can be accessed
__mapper_args__ = {"eager_defaults": True}
========================================
Support for :pep:`484` typing annotations as well as the
-MyPy_ type checking tool.
+MyPy_ type checking tool when using SQLAlchemy
+:ref:`declarative <orm_declarative_mapper_config_toplevel>` mappings
+that refer to the :class:`_schema.Column` object directly, rather than
+the :func:`_orm.mapped_column` construct introduced in SQLAlchemy 2.0.
.. topic:: SQLAlchemy Mypy Plugin Status Update
- **Updated February 17, 2022**
+ **Updated June 27, 2022**
+
+ For SQLAlchemy 2.0, the Mypy plugin continues to work at the level at which
+ it reached in the SQLAlchemy 1.4 release. However, SQLAlchemy 2.0,
+ when released, will feature an
+ :ref:`all new typing system <whatsnew_20_orm_declarative_typing>`
+ for ORM Declarative models that removes the need for the Mypy plugin and
+ delivers much more consistent behavior with generally superior capabilities.
+ Note that this new capability is **not
+ part of SQLAlchemy 1.4, it is only in SQLAlchemy 2.0, which is not released
+ yet as of June 27, 2022**.
The SQLAlchemy Mypy plugin, while it has technically never left the "alpha"
stage, should **now be considered as legacy, even though it is still
- necessary for full Mypy support when using SQLAlchemy 1.4**. SQLAlchemy
- version 2.0, when released, will include new constructs that will allow for
- construction of declarative mappings in place which will support proper
- typing directly, without the need for plugins. This new feature is **not
- part of SQLAlchemy 1.4, it is only in SQLAlchemy 2.0, which is not released
- yet as of Feb 17, 2022**.
+ necessary for full Mypy support when using SQLAlchemy 1.4**.
The Mypy plugin itself does not solve the issue of supplying correct typing
with other typing tools such as Pylance/Pyright, Pytype, Pycharm, etc, which
End-user code that passes all checks under SQLAlchemy 1.4 with the Mypy
plugin will be able to incrementally migrate to the new structures, once
- that code is running exclusively on SQLAlchemy 2.0. The change consists of
- altering how the :func:`_orm.declarative_base` construct is produced, and
- then the replacement of inline Declarative :class:`_schema.Column`
- structures with a fully cross-compatible ``mapped_column()`` construct. Both
- constructs can coexist on any declaratively mapped class.
+ that code is running exclusively on SQLAlchemy 2.0. See the section
+ :ref:`whatsnew_20_orm_declarative_typing` for background on how this
+ migration may proceed.
Code that is running exclusively on **not-released-yet** SQLAlchemy version
2.0 and has fully migrated to the new declarative constructs will enjoy full
Installation
------------
-TODO: document uninstallation of existing stubs:
-
-* ``sqlalchemy2-stubs``
-* ``sqlalchemy-stubs``
-
-SQLAlchemy 2.0 is expected to be directly typed.
+For **SQLAlchemy 2.0 only**: No stubs should be installed and packages
+like sqlalchemy-stubs_ and sqlalchemy2-stubs_ should be fully uninstalled.
The Mypy_ package itself is a dependency.
-Both packages may be installed using the "mypy" extras hook using pip::
+Mypy may be installed using the "mypy" extras hook using pip::
pip install sqlalchemy[mypy]
[mypy]
plugins = sqlalchemy.ext.mypy.plugin
+.. _sqlalchemy-stubs: https://github.com/dropbox/sqlalchemy-stubs
+
+.. _sqlalchemy2-stubs: https://github.com/sqlalchemy/sqlalchemy2-stubs
+
What the Plugin Does
--------------------
:class:`_orm.Mapped` class is now the base class for the :class:`_orm.InstrumentedAttribute`
class that is used for all ORM mapped attributes.
- In ``sqlalchemy2-stubs``,
:class:`_orm.Mapped` is defined as a generic class against arbitrary Python
types, meaning specific occurrences of :class:`_orm.Mapped` are associated
with a specific Python type, such as ``Mapped[Optional[int]]`` and
generate a constructor that matches the annotations in terms of optional
vs. required attributes.
-.. tip::
-
- In the above examples the :class:`_types.Integer` and
- :class:`_types.String` datatypes are both :class:`_types.TypeEngine`
- subclasses. In ``sqlalchemy2-stubs``, the :class:`_schema.Column` object is
- a `generic <https://www.python.org/dev/peps/pep-0484/#generics>`_ which
- subscribes to the type, e.g. above the column types are
- ``Column[Integer]``, ``Column[String]``, and ``Column[String]``. The
- :class:`_types.Integer` and :class:`_types.String` classes are in turn
- generically subscribed to the Python types they correspond towards, i.e.
- ``Integer(TypeEngine[int])``, ``String(TypeEngine[str])``.
Columns that Don't have an Explicit Type
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
class Employee(Base):
__tablename__ = "employee"
- id = Column(Integer, primary_key=True)
- name = Column(String(50))
- type = Column(String(50))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(50))
+ type = mapped_column(String(50))
__mapper_args__ = {
"polymorphic_identity": "employee",
class Engineer(Employee):
__tablename__ = "engineer"
- id = Column(Integer, ForeignKey("employee.id"), primary_key=True)
- engineer_name = Column(String(30))
+ id = mapped_column(Integer, ForeignKey("employee.id"), primary_key=True)
+ engineer_name = mapped_column(String(30))
__mapper_args__ = {
"polymorphic_identity": "engineer",
class Manager(Employee):
__tablename__ = "manager"
- id = Column(Integer, ForeignKey("employee.id"), primary_key=True)
- manager_name = Column(String(30))
+ id = mapped_column(Integer, ForeignKey("employee.id"), primary_key=True)
+ manager_name = mapped_column(String(30))
__mapper_args__ = {
"polymorphic_identity": "manager",
class Company(Base):
__tablename__ = "company"
- id = Column(Integer, primary_key=True)
- name = Column(String(50))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(50))
employees = relationship("Employee", back_populates="company")
class Employee(Base):
__tablename__ = "employee"
- id = Column(Integer, primary_key=True)
- name = Column(String(50))
- type = Column(String(50))
- company_id = Column(ForeignKey("company.id"))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(50))
+ type = mapped_column(String(50))
+ company_id = mapped_column(ForeignKey("company.id"))
company = relationship("Company", back_populates="employees")
__mapper_args__ = {
class Company(Base):
__tablename__ = "company"
- id = Column(Integer, primary_key=True)
- name = Column(String(50))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(50))
managers = relationship("Manager", back_populates="company")
class Employee(Base):
__tablename__ = "employee"
- id = Column(Integer, primary_key=True)
- name = Column(String(50))
- type = Column(String(50))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(50))
+ type = mapped_column(String(50))
__mapper_args__ = {
"polymorphic_identity": "employee",
class Manager(Employee):
__tablename__ = "manager"
- id = Column(Integer, ForeignKey("employee.id"), primary_key=True)
- manager_name = Column(String(30))
+ id = mapped_column(Integer, ForeignKey("employee.id"), primary_key=True)
+ manager_name = mapped_column(String(30))
- company_id = Column(ForeignKey("company.id"))
+ company_id = mapped_column(ForeignKey("company.id"))
company = relationship("Company", back_populates="managers")
__mapper_args__ = {
class Employee(Base):
__tablename__ = "employee"
- id = Column(Integer, primary_key=True)
- name = Column(String(50))
- type = Column(String(20))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(50))
+ type = mapped_column(String(20))
__mapper_args__ = {
"polymorphic_on": type,
class Manager(Employee):
- manager_data = Column(String(50))
+ manager_data = mapped_column(String(50))
__mapper_args__ = {
"polymorphic_identity": "manager",
class Engineer(Employee):
- engineer_info = Column(String(50))
+ engineer_info = mapped_column(String(50))
__mapper_args__ = {
"polymorphic_identity": "engineer",
class Employee(Base):
__tablename__ = "employee"
- id = Column(Integer, primary_key=True)
- name = Column(String(50))
- type = Column(String(20))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(50))
+ type = mapped_column(String(20))
__mapper_args__ = {
"polymorphic_on": type,
__mapper_args__ = {
"polymorphic_identity": "engineer",
}
- start_date = Column(DateTime)
+ start_date = mapped_column(DateTime)
class Manager(Employee):
__mapper_args__ = {
"polymorphic_identity": "manager",
}
- start_date = Column(DateTime)
+ start_date = mapped_column(DateTime)
Above, the ``start_date`` column declared on both ``Engineer`` and ``Manager``
will result in an error::
class Employee(Base):
__tablename__ = "employee"
- id = Column(Integer, primary_key=True)
- name = Column(String(50))
- type = Column(String(20))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(50))
+ type = mapped_column(String(20))
__mapper_args__ = {
"polymorphic_on": type,
@declared_attr
def start_date(cls):
"Start date column, if not present already."
- return Employee.__table__.c.get("start_date", Column(DateTime))
+ return Employee.__table__.c.get("start_date", mapped_column(DateTime))
class Manager(Employee):
@declared_attr
def start_date(cls):
"Start date column, if not present already."
- return Employee.__table__.c.get("start_date", Column(DateTime))
+ return Employee.__table__.c.get("start_date", mapped_column(DateTime))
Above, when ``Manager`` is mapped, the ``start_date`` column is
already present on the ``Employee`` class; by returning the existing
class Employee(Base):
__tablename__ = "employee"
- id = Column(Integer, primary_key=True)
- name = Column(String(50))
- type = Column(String(20))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(50))
+ type = mapped_column(String(20))
__mapper_args__ = {
"polymorphic_on": type,
class HasStartDate:
@declared_attr
def start_date(cls):
- return cls.__table__.c.get("start_date", Column(DateTime))
+ return cls.__table__.c.get("start_date", mapped_column(DateTime))
class Engineer(HasStartDate, Employee):
class Company(Base):
__tablename__ = "company"
- id = Column(Integer, primary_key=True)
- name = Column(String(50))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(50))
employees = relationship("Employee", back_populates="company")
class Employee(Base):
__tablename__ = "employee"
- id = Column(Integer, primary_key=True)
- name = Column(String(50))
- type = Column(String(50))
- company_id = Column(ForeignKey("company.id"))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(50))
+ type = mapped_column(String(50))
+ company_id = mapped_column(ForeignKey("company.id"))
company = relationship("Company", back_populates="employees")
__mapper_args__ = {
class Manager(Employee):
- manager_data = Column(String(50))
+ manager_data = mapped_column(String(50))
__mapper_args__ = {
"polymorphic_identity": "manager",
class Engineer(Employee):
- engineer_info = Column(String(50))
+ engineer_info = mapped_column(String(50))
__mapper_args__ = {
"polymorphic_identity": "engineer",
class Company(Base):
__tablename__ = "company"
- id = Column(Integer, primary_key=True)
- name = Column(String(50))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(50))
managers = relationship("Manager", back_populates="company")
class Employee(Base):
__tablename__ = "employee"
- id = Column(Integer, primary_key=True)
- name = Column(String(50))
- type = Column(String(50))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(50))
+ type = mapped_column(String(50))
__mapper_args__ = {
"polymorphic_identity": "employee",
class Manager(Employee):
- manager_name = Column(String(30))
+ manager_name = mapped_column(String(30))
- company_id = Column(ForeignKey("company.id"))
+ company_id = mapped_column(ForeignKey("company.id"))
company = relationship("Company", back_populates="managers")
__mapper_args__ = {
class Engineer(Employee):
- engineer_info = Column(String(50))
+ engineer_info = mapped_column(String(50))
__mapper_args__ = {
"polymorphic_identity": "engineer",
class Employee(Base):
__tablename__ = "employee"
- id = Column(Integer, primary_key=True)
- name = Column(String(50))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(50))
class Manager(Employee):
__tablename__ = "manager"
- id = Column(Integer, primary_key=True)
- name = Column(String(50))
- manager_data = Column(String(50))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(50))
+ manager_data = mapped_column(String(50))
__mapper_args__ = {
"concrete": True,
class Engineer(Employee):
__tablename__ = "engineer"
- id = Column(Integer, primary_key=True)
- name = Column(String(50))
- engineer_info = Column(String(50))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(50))
+ engineer_info = mapped_column(String(50))
__mapper_args__ = {
"concrete": True,
class Employee(ConcreteBase, Base):
__tablename__ = "employee"
- id = Column(Integer, primary_key=True)
- name = Column(String(50))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(50))
__mapper_args__ = {
"polymorphic_identity": "employee",
class Manager(Employee):
__tablename__ = "manager"
- id = Column(Integer, primary_key=True)
- name = Column(String(50))
- manager_data = Column(String(40))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(50))
+ manager_data = mapped_column(String(40))
__mapper_args__ = {
"polymorphic_identity": "manager",
class Engineer(Employee):
__tablename__ = "engineer"
- id = Column(Integer, primary_key=True)
- name = Column(String(50))
- engineer_info = Column(String(40))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(50))
+ engineer_info = mapped_column(String(40))
__mapper_args__ = {
"polymorphic_identity": "engineer",
class Manager(Employee):
__tablename__ = "manager"
- id = Column(Integer, primary_key=True)
- name = Column(String(50))
- manager_data = Column(String(40))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(50))
+ manager_data = mapped_column(String(40))
__mapper_args__ = {
"polymorphic_identity": "manager",
class Engineer(Employee):
__tablename__ = "engineer"
- id = Column(Integer, primary_key=True)
- name = Column(String(50))
- engineer_info = Column(String(40))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(50))
+ engineer_info = mapped_column(String(40))
__mapper_args__ = {
"polymorphic_identity": "engineer",
class Manager(Employee):
__tablename__ = "manager"
- id = Column(Integer, primary_key=True)
- name = Column(String(50))
- manager_data = Column(String(40))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(50))
+ manager_data = mapped_column(String(40))
__mapper_args__ = {
"polymorphic_identity": "manager",
class Engineer(Employee):
__tablename__ = "engineer"
- id = Column(Integer, primary_key=True)
- name = Column(String(50))
- engineer_info = Column(String(40))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(50))
+ engineer_info = mapped_column(String(40))
__mapper_args__ = {
"polymorphic_identity": "engineer",
class Company(Base):
__tablename__ = "company"
- id = Column(Integer, primary_key=True)
- name = Column(String(50))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(50))
employees = relationship("Employee")
class Employee(ConcreteBase, Base):
__tablename__ = "employee"
- id = Column(Integer, primary_key=True)
- name = Column(String(50))
- company_id = Column(ForeignKey("company.id"))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(50))
+ company_id = mapped_column(ForeignKey("company.id"))
__mapper_args__ = {
"polymorphic_identity": "employee",
class Manager(Employee):
__tablename__ = "manager"
- id = Column(Integer, primary_key=True)
- name = Column(String(50))
- manager_data = Column(String(40))
- company_id = Column(ForeignKey("company.id"))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(50))
+ manager_data = mapped_column(String(40))
+ company_id = mapped_column(ForeignKey("company.id"))
__mapper_args__ = {
"polymorphic_identity": "manager",
class Engineer(Employee):
__tablename__ = "engineer"
- id = Column(Integer, primary_key=True)
- name = Column(String(50))
- engineer_info = Column(String(40))
- company_id = Column(ForeignKey("company.id"))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(50))
+ engineer_info = mapped_column(String(40))
+ company_id = mapped_column(ForeignKey("company.id"))
__mapper_args__ = {
"polymorphic_identity": "engineer",
class Company(Base):
__tablename__ = "company"
- id = Column(Integer, primary_key=True)
- name = Column(String(50))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(50))
employees = relationship("Employee", back_populates="company")
class Employee(ConcreteBase, Base):
__tablename__ = "employee"
- id = Column(Integer, primary_key=True)
- name = Column(String(50))
- company_id = Column(ForeignKey("company.id"))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(50))
+ company_id = mapped_column(ForeignKey("company.id"))
company = relationship("Company", back_populates="employees")
__mapper_args__ = {
class Manager(Employee):
__tablename__ = "manager"
- id = Column(Integer, primary_key=True)
- name = Column(String(50))
- manager_data = Column(String(40))
- company_id = Column(ForeignKey("company.id"))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(50))
+ manager_data = mapped_column(String(40))
+ company_id = mapped_column(ForeignKey("company.id"))
company = relationship("Company", back_populates="employees")
__mapper_args__ = {
class Engineer(Employee):
__tablename__ = "engineer"
- id = Column(Integer, primary_key=True)
- name = Column(String(50))
- engineer_info = Column(String(40))
- company_id = Column(ForeignKey("company.id"))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(50))
+ engineer_info = mapped_column(String(40))
+ company_id = mapped_column(ForeignKey("company.id"))
company = relationship("Company", back_populates="employees")
__mapper_args__ = {
class Employee(Base):
__tablename__ = 'employee'
- id = Column(Integer, primary_key=True)
- name = Column(String(50))
- type = Column(String(50))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(50))
+ type = mapped_column(String(50))
__mapper_args__ = {
'polymorphic_identity':'employee',
class Engineer(Employee):
__tablename__ = 'engineer'
- id = Column(Integer, ForeignKey('employee.id'), primary_key=True)
- engineer_info = Column(String(50))
+ id = mapped_column(Integer, ForeignKey('employee.id'), primary_key=True)
+ engineer_info = mapped_column(String(50))
__mapper_args__ = {
'polymorphic_identity':'engineer',
'polymorphic_load': 'inline'
class Manager(Employee):
__tablename__ = 'manager'
- id = Column(Integer, ForeignKey('employee.id'), primary_key=True)
- manager_data = Column(String(50))
+ id = mapped_column(Integer, ForeignKey('employee.id'), primary_key=True)
+ manager_data = mapped_column(String(50))
__mapper_args__ = {
'polymorphic_identity':'manager',
'polymorphic_load': 'inline'
class Employee(Base):
__tablename__ = 'employee'
- id = Column(Integer, primary_key=True)
- name = Column(String(50))
- type = Column(String(50))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(50))
+ type = mapped_column(String(50))
__mapper_args__ = {
'polymorphic_identity': 'employee',
class Engineer(Employee):
__tablename__ = 'engineer'
- id = Column(Integer, ForeignKey('employee.id'), primary_key=True)
- engineer_name = Column(String(30))
+ id = mapped_column(Integer, ForeignKey('employee.id'), primary_key=True)
+ engineer_name = mapped_column(String(30))
__mapper_args__ = {
'polymorphic_load': 'selectin',
class Manager(Employee):
__tablename__ = 'manager'
- id = Column(Integer, ForeignKey('employee.id'), primary_key=True)
- manager_name = Column(String(30))
+ id = mapped_column(Integer, ForeignKey('employee.id'), primary_key=True)
+ manager_name = mapped_column(String(30))
__mapper_args__ = {
'polymorphic_load': 'selectin',
class Manager(Employee):
__tablename__ = 'manager'
- id = Column(Integer, ForeignKey('employee.id'), primary_key=True)
- manager_name = Column(String(30))
+ id = mapped_column(Integer, ForeignKey('employee.id'), primary_key=True)
+ manager_name = mapped_column(String(30))
__mapper_args__ = {
'polymorphic_load': 'selectin',
}
class VicePresident(Manager):
- vp_info = Column(String(30))
+ vp_info = mapped_column(String(30))
__mapper_args__ = {
"polymorphic_load": "inline",
class Company(Base):
__tablename__ = 'company'
- id = Column(Integer, primary_key=True)
- name = Column(String(50))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(50))
employees = relationship("Employee",
backref='company')
class Employee(Base):
__tablename__ = 'employee'
- id = Column(Integer, primary_key=True)
- type = Column(String(20))
- company_id = Column(Integer, ForeignKey('company.id'))
+ id = mapped_column(Integer, primary_key=True)
+ type = mapped_column(String(20))
+ company_id = mapped_column(Integer, ForeignKey('company.id'))
__mapper_args__ = {
'polymorphic_on':type,
'polymorphic_identity':'employee',
class Engineer(Employee):
__tablename__ = 'engineer'
- id = Column(Integer, ForeignKey('employee.id'), primary_key=True)
- engineer_info = Column(String(50))
+ id = mapped_column(Integer, ForeignKey('employee.id'), primary_key=True)
+ engineer_info = mapped_column(String(50))
__mapper_args__ = {'polymorphic_identity':'engineer'}
class Manager(Employee):
__tablename__ = 'manager'
- id = Column(Integer, ForeignKey('employee.id'), primary_key=True)
- manager_data = Column(String(50))
+ id = mapped_column(Integer, ForeignKey('employee.id'), primary_key=True)
+ manager_data = mapped_column(String(50))
__mapper_args__ = {'polymorphic_identity':'manager'}
When querying from ``Company`` onto the ``Employee`` relationship, the
:class:`_schema.Column` specified::
class Manager(Employee):
- manager_data = Column(String(50))
+ manager_data = mapped_column(String(50))
__mapper_args__ = {
'polymorphic_identity':'manager'
class::
from sqlalchemy import Integer, ForeignKey, String, Column
- from sqlalchemy.ext.declarative import declarative_base
+ from sqlalchemy.orm import DeclarativeBase
from sqlalchemy.orm import relationship
- Base = declarative_base()
+ class Base(DeclarativeBase):
+ pass
class Customer(Base):
__tablename__ = 'customer'
- id = Column(Integer, primary_key=True)
- name = Column(String)
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String)
- billing_address_id = Column(Integer, ForeignKey("address.id"))
- shipping_address_id = Column(Integer, ForeignKey("address.id"))
+ billing_address_id = mapped_column(Integer, ForeignKey("address.id"))
+ shipping_address_id = mapped_column(Integer, ForeignKey("address.id"))
billing_address = relationship("Address")
shipping_address = relationship("Address")
class Address(Base):
__tablename__ = 'address'
- id = Column(Integer, primary_key=True)
- street = Column(String)
- city = Column(String)
- state = Column(String)
- zip = Column(String)
+ id = mapped_column(Integer, primary_key=True)
+ street = mapped_column(String)
+ city = mapped_column(String)
+ state = mapped_column(String)
+ zip = mapped_column(String)
The above mapping, when we attempt to use it, will produce the error::
class Customer(Base):
__tablename__ = 'customer'
- id = Column(Integer, primary_key=True)
- name = Column(String)
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String)
- billing_address_id = Column(Integer, ForeignKey("address.id"))
- shipping_address_id = Column(Integer, ForeignKey("address.id"))
+ billing_address_id = mapped_column(Integer, ForeignKey("address.id"))
+ shipping_address_id = mapped_column(Integer, ForeignKey("address.id"))
billing_address = relationship("Address", foreign_keys=[billing_address_id])
shipping_address = relationship("Address", foreign_keys=[shipping_address_id])
load those ``Address`` objects which specify a city of "Boston"::
from sqlalchemy import Integer, ForeignKey, String, Column
- from sqlalchemy.ext.declarative import declarative_base
+ from sqlalchemy.orm import DeclarativeBase
from sqlalchemy.orm import relationship
- Base = declarative_base()
+ class Base(DeclarativeBase):
+ pass
class User(Base):
__tablename__ = 'user'
- id = Column(Integer, primary_key=True)
- name = Column(String)
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String)
boston_addresses = relationship("Address",
primaryjoin="and_(User.id==Address.user_id, "
"Address.city=='Boston')")
class Address(Base):
__tablename__ = 'address'
- id = Column(Integer, primary_key=True)
- user_id = Column(Integer, ForeignKey('user.id'))
+ id = mapped_column(Integer, primary_key=True)
+ user_id = mapped_column(Integer, ForeignKey('user.id'))
- street = Column(String)
- city = Column(String)
- state = Column(String)
- zip = Column(String)
+ street = mapped_column(String)
+ city = mapped_column(String)
+ state = mapped_column(String)
+ zip = mapped_column(String)
Within this string SQL expression, we made use of the :func:`.and_` conjunction
construct to establish two distinct predicates for the join condition - joining
from sqlalchemy.orm import relationship
from sqlalchemy.dialects.postgresql import INET
- from sqlalchemy.ext.declarative import declarative_base
+ from sqlalchemy.orm import DeclarativeBase
- Base = declarative_base()
+ class Base(DeclarativeBase):
+ pass
class HostEntry(Base):
__tablename__ = 'host_entry'
- id = Column(Integer, primary_key=True)
- ip_address = Column(INET)
- content = Column(String(50))
+ id = mapped_column(Integer, primary_key=True)
+ ip_address = mapped_column(INET)
+ content = mapped_column(String(50))
# relationship() using explicit foreign_keys, remote_side
parent_host = relationship("HostEntry",
class HostEntry(Base):
__tablename__ = 'host_entry'
- id = Column(Integer, primary_key=True)
- ip_address = Column(INET)
- content = Column(String(50))
+ id = mapped_column(Integer, primary_key=True)
+ ip_address = mapped_column(INET)
+ content = mapped_column(String(50))
# relationship() using explicit foreign() and remote() annotations
# in lieu of separate arguments
class IPA(Base):
__tablename__ = 'ip_address'
- id = Column(Integer, primary_key=True)
- v4address = Column(INET)
+ id = mapped_column(Integer, primary_key=True)
+ v4address = mapped_column(INET)
network = relationship("Network",
primaryjoin="IPA.v4address.bool_op('<<')"
class Network(Base):
__tablename__ = 'network'
- id = Column(Integer, primary_key=True)
- v4representation = Column(CIDR)
+ id = mapped_column(Integer, primary_key=True)
+ v4representation = mapped_column(CIDR)
Above, a query such as::
class Polygon(Base):
__tablename__ = "polygon"
- id = Column(Integer, primary_key=True)
- geom = Column(Geometry("POLYGON", srid=4326))
+ id = mapped_column(Integer, primary_key=True)
+ geom = mapped_column(Geometry("POLYGON", srid=4326))
points = relationship(
"Point",
primaryjoin="func.ST_Contains(foreign(Polygon.geom), Point.geom).as_comparison(1, 2)",
class Point(Base):
__tablename__ = "point"
- id = Column(Integer, primary_key=True)
- geom = Column(Geometry("POINT", srid=4326))
+ id = mapped_column(Integer, primary_key=True)
+ geom = mapped_column(Geometry("POINT", srid=4326))
Above, the :meth:`.FunctionElement.as_comparison` indicates that the
``func.ST_Contains()`` SQL function is comparing the ``Polygon.geom`` and
class Magazine(Base):
__tablename__ = 'magazine'
- id = Column(Integer, primary_key=True)
+ id = mapped_column(Integer, primary_key=True)
class Article(Base):
__tablename__ = 'article'
- article_id = Column(Integer)
- magazine_id = Column(ForeignKey('magazine.id'))
- writer_id = Column()
+ article_id = mapped_column(Integer)
+ magazine_id = mapped_column(ForeignKey('magazine.id'))
+ writer_id = mapped_column()
magazine = relationship("Magazine")
writer = relationship("Writer")
class Writer(Base):
__tablename__ = 'writer'
- id = Column(Integer, primary_key=True)
- magazine_id = Column(ForeignKey('magazine.id'), primary_key=True)
+ id = mapped_column(Integer, primary_key=True)
+ magazine_id = mapped_column(ForeignKey('magazine.id'), primary_key=True)
magazine = relationship("Magazine")
When the above mapping is configured, we will see this warning emitted::
class Element(Base):
__tablename__ = 'element'
- path = Column(String, primary_key=True)
+ path = mapped_column(String, primary_key=True)
descendants = relationship('Element',
primaryjoin=
is when establishing a many-to-many relationship from a class to itself, as shown below::
from sqlalchemy import Integer, ForeignKey, String, Column, Table
- from sqlalchemy.ext.declarative import declarative_base
+ from sqlalchemy.orm import DeclarativeBase
from sqlalchemy.orm import relationship
- Base = declarative_base()
+ class Base(DeclarativeBase):
+ pass
node_to_node = Table("node_to_node", Base.metadata,
Column("left_node_id", Integer, ForeignKey("node.id"), primary_key=True),
class Node(Base):
__tablename__ = 'node'
- id = Column(Integer, primary_key=True)
- label = Column(String)
+ id = mapped_column(Integer, primary_key=True)
+ label = mapped_column(String)
right_nodes = relationship("Node",
secondary=node_to_node,
primaryjoin=id==node_to_node.c.left_node_id,
class Node(Base):
__tablename__ = 'node'
- id = Column(Integer, primary_key=True)
- label = Column(String)
+ id = mapped_column(Integer, primary_key=True)
+ label = mapped_column(String)
right_nodes = relationship("Node",
secondary="node_to_node",
primaryjoin="Node.id==node_to_node.c.left_node_id",
class A(Base):
__tablename__ = 'a'
- id = Column(Integer, primary_key=True)
- b_id = Column(ForeignKey('b.id'))
+ id = mapped_column(Integer, primary_key=True)
+ b_id = mapped_column(ForeignKey('b.id'))
d = relationship("D",
secondary="join(B, D, B.d_id == D.id)."
class B(Base):
__tablename__ = 'b'
- id = Column(Integer, primary_key=True)
- d_id = Column(ForeignKey('d.id'))
+ id = mapped_column(Integer, primary_key=True)
+ d_id = mapped_column(ForeignKey('d.id'))
class C(Base):
__tablename__ = 'c'
- id = Column(Integer, primary_key=True)
- a_id = Column(ForeignKey('a.id'))
- d_id = Column(ForeignKey('d.id'))
+ id = mapped_column(Integer, primary_key=True)
+ a_id = mapped_column(ForeignKey('a.id'))
+ d_id = mapped_column(ForeignKey('d.id'))
class D(Base):
__tablename__ = 'd'
- id = Column(Integer, primary_key=True)
+ id = mapped_column(Integer, primary_key=True)
In the above example, we provide all three of :paramref:`_orm.relationship.secondary`,
:paramref:`_orm.relationship.primaryjoin`, and :paramref:`_orm.relationship.secondaryjoin`,
class A(Base):
__tablename__ = 'a'
- id = Column(Integer, primary_key=True)
- b_id = Column(ForeignKey('b.id'))
+ id = mapped_column(Integer, primary_key=True)
+ b_id = mapped_column(ForeignKey('b.id'))
class B(Base):
__tablename__ = 'b'
- id = Column(Integer, primary_key=True)
+ id = mapped_column(Integer, primary_key=True)
class C(Base):
__tablename__ = 'c'
- id = Column(Integer, primary_key=True)
- a_id = Column(ForeignKey('a.id'))
+ id = mapped_column(Integer, primary_key=True)
+ a_id = mapped_column(ForeignKey('a.id'))
- some_c_value = Column(String)
+ some_c_value = mapped_column(String)
class D(Base):
__tablename__ = 'd'
- id = Column(Integer, primary_key=True)
- c_id = Column(ForeignKey('c.id'))
- b_id = Column(ForeignKey('b.id'))
+ id = mapped_column(Integer, primary_key=True)
+ c_id = mapped_column(ForeignKey('c.id'))
+ b_id = mapped_column(ForeignKey('b.id'))
- some_d_value = Column(String)
+ some_d_value = mapped_column(String)
# 1. set up the join() as a variable, so we can refer
# to it in the mapping multiple times.
class A(Base):
__tablename__ = 'a'
- id = Column(Integer, primary_key=True)
+ id = mapped_column(Integer, primary_key=True)
class B(Base):
__tablename__ = 'b'
- id = Column(Integer, primary_key=True)
- a_id = Column(ForeignKey("a.id"))
+ id = mapped_column(Integer, primary_key=True)
+ a_id = mapped_column(ForeignKey("a.id"))
partition = select(
B,
class User(Base):
__tablename__ = 'user'
- id = Column(Integer, primary_key=True)
+ id = mapped_column(Integer, primary_key=True)
@property
def addresses(self):
Deferred column loading allows particular columns of a table be loaded only
upon direct access, instead of when the entity is queried using
-:class:`_sql.Select`. This feature is useful when one wants to avoid
+:class:`_sql.Select` or :class:`_orm.Query`. This feature is useful when one wants to avoid
loading a large text or binary field into memory when it's not needed.
-Individual columns can be lazy loaded by themselves or placed into groups that
-lazy-load together, using the :func:`_orm.deferred` function to
-mark them as "deferred". In the example below, we define a mapping that will load each of
+
+Configuring Deferred Loading at Mapper Configuration Time
+---------------------------------------------------------
+
+First introduced at :ref:`orm_declarative_column_options` and
+:ref:`orm_imperative_table_column_options`, the
+:paramref:`_orm.mapped_column.deferred` parameter of :func:`_orm.mapped_column`,
+as well as the :func:`_orm.deferred` ORM function may be used to indicate mapped
+columns as "deferred" at mapper configuration time. With this configuration,
+the target columns will not be loaded in SELECT statements by default, and
+will instead only be loaded "lazily" when their corresponding attribute is
+accessed on a mapped instance. Deferral can be configured for individual
+columns or groups of columns that will load together when any of them
+are accessed.
+
+In the example below, using :ref:`Declarative Table <orm_declarative_table>`
+configuration, we define a mapping that will load each of
``.excerpt`` and ``.photo`` in separate, individual-row SELECT statements when each
attribute is first referenced on the individual object instance::
- from sqlalchemy.orm import deferred
- from sqlalchemy import Integer, String, Text, Binary, Column
+ from sqlalchemy import Text
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import mapped_column
+
+ class Base(DeclarativeBase):
+ pass
class Book(Base):
__tablename__ = 'book'
- book_id = Column(Integer, primary_key=True)
- title = Column(String(200), nullable=False)
- summary = Column(String(2000))
- excerpt = deferred(Column(Text))
- photo = deferred(Column(Binary))
+ book_id: Mapped[int] = mapped_column(primary_key=True)
+ title: Mapped[str]
+ summary: Mapped[str]
+ excerpt: Mapped[str] = mapped_column(Text, deferred=True)
+ photo: Mapped[bytes] = mapped_column(deferred=True)
-Classical mappings as always place the usage of :func:`_orm.deferred` in the
-``properties`` dictionary against the table-bound :class:`_schema.Column`::
+A :func:`_sql.select` construct for the above mapping will not include
+``excerpt`` and ``photo`` by default::
+
+ >>> from sqlalchemy import select
+ >>> print(select(Book))
+ SELECT book.book_id, book.title, book.summary
+ FROM book
+
+When an object of type ``Book`` is loaded by the ORM, accessing the
+``.excerpt`` or ``.photo`` attributes will instead :term:`lazy load` the
+data from each column using a new SQL statement.
+
+When using :ref:`Imperative Table <orm_imperative_table_configuration>`
+or fully :ref:`Imperative <orm_imperative_mapping>` configuration, the
+:func:`_orm.deferred` construct should be used instead, passing the
+target :class:`_schema.Column` object to be mapped as the argument::
+
+ from sqlalchemy import Column, Integer, LargeBinary, String, Table, Text
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import deferred
+
+
+ class Base(DeclarativeBase):
+ pass
+
+
+ book = Table(
+ "book",
+ Base.metadata,
+ Column("book_id", Integer, primary_key=True),
+ Column("title", String),
+ Column("summary", String),
+ Column("excerpt", Text),
+ Column("photo", LargeBinary),
+ )
+
+
+ class Book(Base):
+ __table__ = book
+
+ excerpt = deferred(book.c.excerpt)
+ photo = deferred(book.c.photo)
- mapper_registry.map_imperatively(Book, book_table, properties={
- 'photo':deferred(book_table.c.photo)
- })
Deferred columns can be associated with a "group" name, so that they load
-together when any of them are first accessed. The example below defines a
-mapping with a ``photos`` deferred group. When one ``.photo`` is accessed, all three
-photos will be loaded in one SELECT statement. The ``.excerpt`` will be loaded
-separately when it is accessed::
+together when any of them are first accessed. When using
+:func:`_orm.mapped_column`, this group name may be specified using the
+:paramref:`_orm.mapped_column.deferred_group` parameter, which implies
+:paramref:`_orm.mapped_column.deferred` if that parameter is not already
+set. When using :func:`_orm.deferred`, the :paramref:`_orm.deferred.group`
+parameter may be used.
+
+The example below defines a mapping with a ``photos`` deferred group. When
+an attribute within the group ``.photo1``, ``.photo2``, ``.photo3``
+is accessed on an instance of ``Book``, all three columns will be loaded in one SELECT
+statement. The ``.excerpt`` column however will only be loaded when it
+is directly accessed::
+
+ from sqlalchemy import Text
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import mapped_column
+
+ class Base(DeclarativeBase):
+ pass
class Book(Base):
__tablename__ = 'book'
- book_id = Column(Integer, primary_key=True)
- title = Column(String(200), nullable=False)
- summary = Column(String(2000))
- excerpt = deferred(Column(Text))
- photo1 = deferred(Column(Binary), group='photos')
- photo2 = deferred(Column(Binary), group='photos')
- photo3 = deferred(Column(Binary), group='photos')
+ book_id: Mapped[int] = mapped_column(primary_key=True)
+ title: Mapped[str]
+ summary: Mapped[str]
+ excerpt: Mapped[str] = mapped_column(Text, deferred=True)
+ photo1: Mapped[bytes] = mapped_column(deferred_group="photos")
+ photo2: Mapped[bytes] = mapped_column(deferred_group="photos")
+ photo3: Mapped[bytes] = mapped_column(deferred_group="photos")
+
.. _deferred_options:
Deferred Column Loader Query Options
------------------------------------
+At query time, the :func:`_orm.defer`, :func:`_orm.undefer` and
+:func:`_orm.undefer_group` loader options may be used to further control the
+"deferral behavior" of mapped columns.
Columns can be marked as "deferred" or reset to "undeferred" at query time
using options which are passed to the :meth:`_sql.Select.options` method; the most
from sqlalchemy import select
stmt = select(Book)
- stmt = stmt.options(defer('summary'), undefer('excerpt'))
- session.scalars(stmt).all()
+ stmt = stmt.options(defer(Book.summary), undefer(Book.excerpt))
+ book_objs = session.scalars(stmt).all()
Above, the "summary" column will not load until accessed, and the "excerpt"
stmt = select(Book)
stmt = stmt.options(undefer_group('photos'))
- session.scalars(stmt).all()
+ book_objs = session.scalars(stmt).all()
.. _deferred_loading_w_multiple:
Deferred Loading across Multiple Entities
-----------------------------------------
-To specify column deferral for a :class:`_sql.Select` that loads multiple types of
-entities at once, the deferral options may be specified more explicitly using
-class-bound attributes, rather than string names::
+Column deferral may also be used for a statement that loads multiple types of
+entities at once, by referring to the appropriate class bound attribute
+within the :func:`_orm.defer` function. Suppose ``Book`` has a
+relationship ``Book.author`` to a related class ``Author``, we could write
+a query as follows which will defer the ``Author.bio`` column::
from sqlalchemy.orm import defer
from sqlalchemy import select
stmt = select(Book, Author).join(Book.author)
stmt = stmt.options(defer(Author.bio))
+ book_author_objs = session.execute(stmt).all()
+
Column deferral options may also indicate that they take place along various
relationship paths, which are themselves often :ref:`eagerly loaded
joinedload(Author.books).load_only(Book.summary, Book.excerpt)
)
+ author_objs = session.scalars(stmt).all()
+
Option structures as above can also be organized in more complex ways, such
as hierarchically using the :meth:`_orm.Load.options`
method, which allows multiple sub-options to be chained to a common parent
-option at once. Any mixture of string names and class-bound attribute objects
-may be used::
+option at once. The example below illustrates a more complex structure::
from sqlalchemy.orm import defer
from sqlalchemy.orm import joinedload
)
)
)
+ author_objs = session.scalars(stmt).all()
-
-.. versionadded:: 1.3.6 Added :meth:`_orm.Load.options` to allow easier
- construction of hierarchies of loader options.
-
Another way to apply options to a path is to use the :func:`_orm.defaultload`
function. This function is used to indicate a particular path within a loader
option structure without actually setting any options at that level, so that further
defaultload(Author.book).defaultload(Book.citations).defer(Citation.fulltext)
)
+ author_objs = session.scalars(stmt).all()
+
.. seealso::
:ref:`relationship_loader_options` - targeted towards relationship loading
from sqlalchemy.orm import undefer
from sqlalchemy import select
- select(Book).options(
- defer('*'), undefer("summary"), undefer("excerpt"))
+ stmt = select(Book).options(
+ defer('*'), undefer(Book.summary), undefer(Book.excerpt))
+
+ book_objs = session.scalars(stmt).all()
Above, the :func:`.defer` option is applied using a wildcard to all column
attributes on the ``Book`` class. Then, the :func:`.undefer` option is used
from sqlalchemy.orm import load_only
from sqlalchemy import select
- select(Book).options(load_only(Book.summary, Book.excerpt))
+ stmt = select(Book).options(load_only(Book.summary, Book.excerpt))
+
+ book_objs = session.scalars(stmt).all()
Wildcard and Exclusionary Options with Multiple-Entity Queries
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Wildcard options and exclusionary options such as :func:`.load_only` may
-only be applied to a single entity at a time within a :class:`_query.Query`. To
-suit the less common case where a :class:`_query.Query` is returning multiple
+only be applied to a single entity at a time within a statement.
+To suit the less common case where a statement is returning multiple
primary entities at once, a special calling style may be required in order
-to apply a wildcard or exclusionary option, which is to use the
+to apply a wildcard or exclusionary option to a specific entity, which is to use the
:class:`_orm.Load` object to indicate the starting entity for a deferral option.
-Such as, if we were loading ``Book`` and ``Author`` at once, the :class:`_query.Query`
+Such as, if we were loading ``Book`` and ``Author`` at once, the ORM
will raise an informative error if we try to apply :func:`.load_only` to
-both at once. Using :class:`_orm.Load` looks like::
+both at once. Instead, we may use :class:`_orm.Load` to apply the option
+to either or both of ``Book`` and ``Author`` individually::
from sqlalchemy.orm import Load
- query = session.query(Book, Author).join(Book.author)
- query = query.options(
+ stmt = select(Book, Author).join(Book.author)
+ stmt = stmt.options(
Load(Book).load_only(Book.summary, Book.excerpt)
)
+ book_author_objs = session.execute(stmt).all()
Above, :class:`_orm.Load` is used in conjunction with the exclusionary option
:func:`.load_only` so that the deferral of all other columns only takes
place for the ``Book`` class and not the ``Author`` class. Again,
-the :class:`_query.Query` object should raise an informative error message when
+the ORM should raise an informative error message when
the above calling style is actually required that describes those cases
where explicit use of :class:`_orm.Load` is needed.
support the concept of "raiseload", which is a loader strategy that will raise
:class:`.InvalidRequestError` if the attribute is accessed such that it would
need to emit a SQL query in order to be loaded. This behavior is the
-column-based equivalent of the :func:`.raiseload` feature for relationship
+column-based equivalent of the :func:`_orm.raiseload` feature for relationship
loading, discussed at :ref:`prevent_lazy_with_raiseload`. Using the
-:paramref:`.orm.defer.raiseload` parameter on the :func:`.defer` option,
+:paramref:`_orm.defer.raiseload` parameter on the :func:`_orm.defer` option,
an exception is raised if the attribute is accessed::
- book = session.scalars(select(Book).options(defer(Book.summary, raiseload=True)).limit(1)).first()
+ book = session.scalar(
+ select(Book).options(defer(Book.summary, raiseload=True)).limit(1)
+ )
# would raise an exception
book.summary
Deferred "raiseload" can be configured at the mapper level via
-:paramref:`.orm.deferred.raiseload` on :func:`.deferred`, so that an explicit
-:func:`.undefer` is required in order for the attribute to be usable::
+:paramref:`.orm.deferred.raiseload` on either :func:`_orm.mapped_column`
+or in :func:`.deferred`, so that an explicit
+:func:`.undefer` is required in order for the attribute to be usable.
+Below is a :ref:`Declarative table <orm_declarative_table>` configuration example::
+
+
+ from sqlalchemy import Text
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import mapped_column
+ class Base(DeclarativeBase):
+ pass
class Book(Base):
__tablename__ = 'book'
- book_id = Column(Integer, primary_key=True)
- title = Column(String(200), nullable=False)
- summary = deferred(Column(String(2000)), raiseload=True)
- excerpt = deferred(Column(Text), raiseload=True)
+ book_id: Mapped[int] = mapped_column(primary_key=True)
+ title: Mapped[str]
+ summary: Mapped[str] = mapped_column(raiseload=True)
+ excerpt: Mapped[str] = mapped_column(Text, raiseload=True)
+
+Alternatively, the example below illustrates the same mapping using a
+:ref:`Imperative table <orm_imperative_table_configuration>` configuration::
+
+ from sqlalchemy import Column, Integer, LargeBinary, String, Table, Text
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import deferred
+
+
+ class Base(DeclarativeBase):
+ pass
+
+
+ book = Table(
+ "book",
+ Base.metadata,
+ Column("book_id", Integer, primary_key=True),
+ Column("title", String),
+ Column("summary", String),
+ Column("excerpt", Text),
+ )
+
+
+ class Book(Base):
+ __table__ = book
+
+ summary = deferred(book.c.summary, raiseload=True)
+ excerpt = deferred(book.c.excerpt, raiseload=True)
- book_w_excerpt = session.scalars(select(Book).options(undefer(Book.excerpt)).limit(1)).first()
+With both mappings, if we wish to have either or both of ``.excerpt``
+or ``.summary`` available on an object when loaded, we make use of the
+:func:`_orm.undefer` loader option::
+ book_w_excerpt = session.scalars(
+ select(Book).options(undefer(Book.excerpt)).where(Book.id == 12)
+ ).first()
+The :func:`_orm.undefer` option will populate the ``.excerpt`` attribute
+above, even if the ``Book`` object were already loaded, assuming the
+``.excerpt`` field was not populated by some other means previously.
Column Deferral API
class Parent(Base):
__tablename__ = 'parent'
- id = Column(Integer, primary_key=True)
+ id = mapped_column(Integer, primary_key=True)
children = relationship("Child", lazy='joined')
Above, whenever a collection of ``Parent`` objects are loaded, each
class Address(Base):
# ...
- user_id = Column(ForeignKey('users.id'), nullable=False)
+ user_id = mapped_column(ForeignKey('users.id'), nullable=False)
user = relationship(User, lazy="joined", innerjoin=True)
At the query option level, via the :paramref:`_orm.joinedload.innerjoin` flag::
class User(Base):
__tablename__ = 'user'
- id = Column(Integer, primary_key=True)
+ id = mapped_column(Integer, primary_key=True)
addresses = relationship("Address")
class Address(Base):
from sqlalchemy import Integer, ForeignKey, Column
from sqlalchemy.orm import relationship, backref
- from sqlalchemy.ext.declarative import declarative_base
+ from sqlalchemy.orm import DeclarativeBase
- Base = declarative_base()
+ class Base(DeclarativeBase):
+ pass
class A(Base):
__tablename__ = 'a'
- id = Column(Integer, primary_key=True)
- b_id = Column(ForeignKey('b.id'))
+ id = mapped_column(Integer, primary_key=True)
+ b_id = mapped_column(ForeignKey('b.id'))
b = relationship(
"B",
backref=backref("a", uselist=False),
class B(Base):
__tablename__ = 'b'
- id = Column(Integer, primary_key=True)
+ id = mapped_column(Integer, primary_key=True)
If we query for an ``A`` row, and then ask it for ``a.b.a``, we will get
Changing Attribute Behavior
===========================
+This section will discuss features and techniques used to modify the
+behavior of ORM mapped attributes, including those mapped with
+:func:`_orm.mapped_column`, :func:`_orm.relationship`, and others.
+
.. _simple_validators:
Simple Validators
class EmailAddress(Base):
__tablename__ = 'address'
- id = Column(Integer, primary_key=True)
- email = Column(String)
+ id = mapped_column(Integer, primary_key=True)
+ email = mapped_column(String)
@validates('email')
def validate_email(self, key, address):
raise ValueError("failed simple email validation")
return address
-.. versionchanged:: 1.0.0 - validators are no longer triggered within
- the flush process when the newly fetched values for primary key
- columns as well as some python- or server-side defaults are fetched.
- Prior to 1.0, validators may be triggered in those cases as well.
-
Validators also receive collection append events, when items are added to a
collection::
class EmailAddress(Base):
__tablename__ = 'email_address'
- id = Column(Integer, primary_key=True)
+ id = mapped_column(Integer, primary_key=True)
# name the attribute with an underscore,
# different from the column name
- _email = Column("email", String)
+ _email = mapped_column("email", String)
# then create an ".email" attribute
# to get/set "._email"
class EmailAddress(Base):
__tablename__ = 'email_address'
- id = Column(Integer, primary_key=True)
+ id = mapped_column(Integer, primary_key=True)
- _email = Column("email", String)
+ _email = mapped_column("email", String)
@hybrid_property
def email(self):
class EmailAddress(Base):
__tablename__ = 'email_address'
- id = Column(Integer, primary_key=True)
+ id = mapped_column(Integer, primary_key=True)
- _email = Column("email", String)
+ _email = mapped_column("email", String)
@hybrid_property
def email(self):
attribute available by an additional name::
from sqlalchemy.orm import synonym
-
+
class MyClass(Base):
__tablename__ = 'my_table'
- id = Column(Integer, primary_key=True)
- job_status = Column(String(50))
+ id = mapped_column(Integer, primary_key=True)
+ job_status = mapped_column(String(50))
status = synonym("job_status")
class MyClass(Base):
__tablename__ = 'my_table'
- id = Column(Integer, primary_key=True)
- status = Column(String(50))
+ id = mapped_column(Integer, primary_key=True)
+ status = mapped_column(String(50))
@property
def job_status(self):
class MyClass(Base):
__tablename__ = 'my_table'
- id = Column(Integer, primary_key=True)
- status = Column(String(50))
+ id = mapped_column(Integer, primary_key=True)
+ status = mapped_column(String(50))
@synonym_for("status")
@property
class User(Base):
__tablename__ = 'user'
- id = Column(Integer, primary_key=True)
- firstname = Column(String(50))
- lastname = Column(String(50))
+ id = mapped_column(Integer, primary_key=True)
+ firstname = mapped_column(String(50))
+ lastname = mapped_column(String(50))
@hybrid_property
def fullname(self):
class User(Base):
__tablename__ = 'user'
- id = Column(Integer, primary_key=True)
- firstname = Column(String(50))
- lastname = Column(String(50))
+ id = mapped_column(Integer, primary_key=True)
+ firstname = mapped_column(String(50))
+ lastname = mapped_column(String(50))
@hybrid_property
def fullname(self):
class User(Base):
__tablename__ = 'user'
- id = Column(Integer, primary_key=True)
- firstname = Column(String(50))
- lastname = Column(String(50))
+ id = mapped_column(Integer, primary_key=True)
+ firstname = mapped_column(String(50))
+ lastname = mapped_column(String(50))
fullname = column_property(firstname + " " + lastname)
Correlated subqueries may be used as well. Below we use the
from sqlalchemy import select, func
from sqlalchemy import Column, Integer, String, ForeignKey
- from sqlalchemy.ext.declarative import declarative_base
+ from sqlalchemy.orm import DeclarativeBase
- Base = declarative_base()
+ class Base(DeclarativeBase):
+ pass
class Address(Base):
__tablename__ = 'address'
- id = Column(Integer, primary_key=True)
- user_id = Column(Integer, ForeignKey('user.id'))
+ id = mapped_column(Integer, primary_key=True)
+ user_id = mapped_column(Integer, ForeignKey('user.id'))
class User(Base):
__tablename__ = 'user'
- id = Column(Integer, primary_key=True)
+ id = mapped_column(Integer, primary_key=True)
address_count = column_property(
select(func.count(Address.id)).
where(Address.user_id==id).
of joins between ``User`` and ``Address`` tables where SELECT statements against
``Address`` are nested.
+For a :func:`.column_property` that refers to columns linked from a
+many-to-many relationship, use :func:`.and_` to join the fields of the
+association table to both tables in a relationship::
+
+ from sqlalchemy import and_
+
+ class Author(Base):
+ # ...
+
+ book_count = column_property(
+ select(func.count(books.c.id)
+ ).where(
+ and_(
+ book_authors.c.author_id==authors.c.id,
+ book_authors.c.book_id==books.c.id
+ )
+ ).scalar_subquery()
+ )
+
+
+Adding column_property() to an existing Declarative mapped class
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
If import issues prevent the :func:`.column_property` from being defined
inline with the class, it can be assigned to the class after both
-are configured. When using mappings that make use of a :func:`_orm.declarative_base`
-base class, this attribute assignment has the effect of calling :meth:`_orm.Mapper.add_property`
+are configured. When using mappings that make use of a Declarative
+base class (i.e. produced by the :class:`_orm.DeclarativeBase` superclass
+or legacy functions such as :func:`_orm.declarative_base`),
+this attribute assignment has the effect of calling :meth:`_orm.Mapper.add_property`
to add an additional property after the fact::
# only works if a declarative base class is in use
scalar_subquery()
)
-When using mapping styles that don't use :func:`_orm.declarative_base`,
+When using mapping styles that don't use Declarative base classes
such as the :meth:`_orm.registry.mapped` decorator, the :meth:`_orm.Mapper.add_property`
method may be invoked explicitly on the underlying :class:`_orm.Mapper` object,
which can be obtained using :func:`_sa.inspect`::
)
)
-For a :func:`.column_property` that refers to columns linked from a
-many-to-many relationship, use :func:`.and_` to join the fields of the
-association table to both tables in a relationship::
+.. seealso::
- from sqlalchemy import and_
+ :ref:`orm_declarative_table_adding_columns`
- class Author(Base):
- # ...
-
- book_count = column_property(
- select(func.count(books.c.id)
- ).where(
- and_(
- book_authors.c.author_id==authors.c.id,
- book_authors.c.book_id==books.c.id
- )
- ).scalar_subquery()
- )
.. _mapper_column_property_sql_expressions_composed:
class File(Base):
__tablename__ = 'file'
- id = Column(Integer, primary_key=True)
- name = Column(String(64))
- extension = Column(String(8))
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(64))
+ extension = mapped_column(String(8))
filename = column_property(name + '.' + extension)
path = column_property('C:/' + filename.expression)
stmt = select(File.path).where(File.filename == 'foo.txt')
+.. autofunction:: column_property
+
Using a plain descriptor
------------------------
class User(Base):
__tablename__ = 'user'
- id = Column(Integer, primary_key=True)
- firstname = Column(String(50))
- lastname = Column(String(50))
+ id = mapped_column(Integer, primary_key=True)
+ firstname = mapped_column(String(50))
+ lastname = mapped_column(String(50))
@property
def address_count(self):
class A(Base):
__tablename__ = 'a'
- id = Column(Integer, primary_key=True)
- x = Column(Integer)
- y = Column(Integer)
+ id = mapped_column(Integer, primary_key=True)
+ x = mapped_column(Integer)
+ y = mapped_column(Integer)
expr = query_expression()
.. toctree::
- :maxdepth: 3
+ :maxdepth: 4
mapping_styles
declarative_mapping
dataclasses
- scalar_mapping
+ mapped_sql_expr
+ mapped_attributes
+ composites
inheritance
nonstandard_mappings
versioning
mapping_api
+
+.. toctree::
+ :hidden:
+
+ scalar_mapping
\ No newline at end of file
.. autofunction:: mapped_column
.. autoclass:: declared_attr
- :members:
+
+ .. attribute:: cascading
+
+ Mark a :class:`.declared_attr` as cascading.
+
+ This is a special-use modifier which indicates that a column
+ or MapperProperty-based declared attribute should be configured
+ distinctly per mapped subclass, within a mapped-inheritance scenario.
+
+ .. warning::
+
+ The :attr:`.declared_attr.cascading` modifier has several
+ limitations:
+
+ * The flag **only** applies to the use of :class:`.declared_attr`
+ on declarative mixin classes and ``__abstract__`` classes; it
+ currently has no effect when used on a mapped class directly.
+
+ * The flag **only** applies to normally-named attributes, e.g.
+ not any special underscore attributes such as ``__tablename__``.
+ On these attributes it has **no** effect.
+
+ * The flag currently **does not allow further overrides** down
+ the class hierarchy; if a subclass tries to override the
+ attribute, a warning is emitted and the overridden attribute
+ is skipped. This is a limitation that it is hoped will be
+ resolved at some point.
+
+ Below, both MyClass as well as MySubClass will have a distinct
+ ``id`` Column object established::
+
+ class HasIdMixin:
+ @declared_attr.cascading
+ def id(cls):
+ if has_inherited_table(cls):
+ return Column(
+ ForeignKey('myclass.id'), primary_key=True
+ )
+ else:
+ return Column(Integer, primary_key=True)
+
+ class MyClass(HasIdMixin, Base):
+ __tablename__ = 'myclass'
+ # ...
+
+ class MySubClass(MyClass):
+ ""
+ # ...
+
+ The behavior of the above configuration is that ``MySubClass``
+ will refer to both its own ``id`` column as well as that of
+ ``MyClass`` underneath the attribute named ``some_id``.
+
+ .. seealso::
+
+ :ref:`declarative_inheritance`
+
+ :ref:`mixin_inheritance_columns`
+
+ .. attribute:: directive
+
+ Mark a :class:`.declared_attr` as decorating a Declarative
+ directive such as ``__tablename__`` or ``__mapper_args__``.
+
+ The purpose of :attr:`.declared_attr.directive` is strictly to
+ support :pep:`484` typing tools, by allowing the decorated function
+ to have a return type that is **not** using the :class:`_orm.Mapped`
+ generic class, as would normally be the case when :class:`.declared_attr`
+ is used for columns and mapped properties. At
+ runtime, the :attr:`.declared_attr.directive` returns the
+ :class:`.declared_attr` class unmodified.
+
+ E.g.::
+
+ class CreateTableName:
+ @declared_attr.directive
+ def __tablename__(cls) -> str:
+ return cls.__name__.lower()
+
+ .. versionadded:: 2.0
+
+ .. seealso::
+
+ :ref:`orm_mixins_toplevel`
+
+ :class:`_orm.declared_attr`
+
+
.. autoclass:: DeclarativeBaseNoMeta
:members:
.. autoclass:: Mapper
:members:
+
+.. autoclass:: MappedAsDataclass
+ :members:
.. _mapping_columns_toplevel:
-.. currentmodule:: sqlalchemy.orm
-
Mapping Table Columns
=====================
-Introductory background on mapping to columns falls under the subject of
-:class:`.Table` configuration; the general form falls under one of three
-forms:
-
-* :ref:`orm_declarative_table` - :class:`.Column` objects are associated with a
- :class:`.Table` as well as with an ORM mapping in one step by declaring
- them inline as class attributes.
-* :ref:`orm_imperative_table_configuration` - :class:`.Column` objects are
- associated directly with their :class:`.Table` object, as detailed at
- :ref:`metadata_describing_toplevel`; the columns are then mapped by
- the Declarative process by associating the :class:`.Table` with the
- class to be mapped via the ``__table__`` attribute.
-* :ref:`orm_imperative_mapping` - like "Imperative Table", :class:`.Column`
- objects are associated directly with their :class:`.Table` object; the
- columns are then mapped by the Imperative process using
- :meth:`_orm.registry.map_imperatively`.
-
-In all of the above cases, the :class:`_orm.Mapper` constructor is ultimately
-invoked with a completed :class:`.Table` object passed as the selectable unit
-to be mapped. The behavior of :class:`_orm.Mapper` then is to assemble all the
-columns in the mapped :class:`_schema.Table` into mapped object attributes,
-each of which are named according to the name of the column itself
-(specifically, the ``key`` attribute of :class:`_schema.Column`). This behavior
-can be modified in several ways.
-
-.. _mapper_column_distinct_names:
-
-Naming Columns Distinctly from Attribute Names
-----------------------------------------------
-
-A mapping by default shares the same name for a
-:class:`_schema.Column` as that of the mapped attribute - specifically
-it matches the :attr:`_schema.Column.key` attribute on :class:`_schema.Column`, which
-by default is the same as the :attr:`_schema.Column.name`.
-
-The name assigned to the Python attribute which maps to
-:class:`_schema.Column` can be different from either
-:attr:`_schema.Column.name` or :attr:`_schema.Column.key` just by assigning
-it that way, as we illustrate here in a Declarative mapping::
-
- class User(Base):
- __tablename__ = 'user'
- id = Column('user_id', Integer, primary_key=True)
- name = Column('user_name', String(50))
-
-Where above ``User.id`` resolves to a column named ``user_id``
-and ``User.name`` resolves to a column named ``user_name``.
-
-When mapping to an existing table, the :class:`_schema.Column` object
-can be referenced directly::
-
- class User(Base):
- __table__ = user_table
- id = user_table.c.user_id
- name = user_table.c.user_name
-
-The corresponding technique for an :term:`imperative` mapping is
-to place the desired key in the :paramref:`_orm.Mapper.properties`
-dictionary with the desired key::
-
- mapper_registry.map_imperatively(User, user_table, properties={
- 'id': user_table.c.user_id,
- 'name': user_table.c.user_name,
- })
-
-
-.. _mapper_automated_reflection_schemes:
-
-Automating Column Naming Schemes from Reflected Tables
-------------------------------------------------------
-
-In the previous section :ref:`mapper_column_distinct_names`, we showed how
-a :class:`_schema.Column` explicitly mapped to a class can have a different attribute
-name than the column. But what if we aren't listing out :class:`_schema.Column`
-objects explicitly, and instead are automating the production of :class:`_schema.Table`
-objects using reflection (i.e. as described in :ref:`metadata_reflection_toplevel`)?
-In this case we can make use of the :meth:`_events.DDLEvents.column_reflect` event
-to intercept the production of :class:`_schema.Column` objects and provide them
-with the :attr:`_schema.Column.key` of our choice. The event is most easily
-associated with the :class:`_schema.MetaData` object that's in use,
-such as below we use the one linked to the :class:`_orm.declarative_base`
-instance::
-
- @event.listens_for(Base.metadata, "column_reflect")
- def column_reflect(inspector, table, column_info):
- # set column.key = "attr_<lower_case_name>"
- column_info['key'] = "attr_%s" % column_info['name'].lower()
-
-With the above event, the reflection of :class:`_schema.Column` objects will be intercepted
-with our event that adds a new ".key" element, such as in a mapping as below::
-
- class MyClass(Base):
- __table__ = Table("some_table", Base.metadata,
- autoload_with=some_engine)
-
-The approach also works with both the :class:`.DeferredReflection` base class
-as well as with the :ref:`automap_toplevel` extension. For automap
-specifically, see the section :ref:`automap_intercepting_columns` for
-background.
-
-.. seealso::
-
- :ref:`orm_declarative_reflected`
-
- :meth:`_events.DDLEvents.column_reflect`
-
- :ref:`automap_intercepting_columns` - in the :ref:`automap_toplevel` documentation
-
-
-.. _column_property_options:
-
-Using column_property for column level options
-----------------------------------------------
-
-Options can be specified when mapping a :class:`_schema.Column` using the
-:func:`.column_property` function. This function
-explicitly creates the :class:`.ColumnProperty` used by the
-:class:`_orm.Mapper` to keep track of the :class:`_schema.Column`; normally, the
-:class:`_orm.Mapper` creates this automatically. Using :func:`.column_property`,
-we can pass additional arguments about how we'd like the :class:`_schema.Column`
-to be mapped. Below, we pass an option ``active_history``,
-which specifies that a change to this column's value should
-result in the former value being loaded first::
-
- from sqlalchemy.orm import column_property
-
- class User(Base):
- __tablename__ = 'user'
-
- id = Column(Integer, primary_key=True)
- name = column_property(Column(String(50)), active_history=True)
-
-:func:`.column_property` is also used to map a single attribute to
-multiple columns. This use case arises when mapping to a :func:`_expression.join`
-which has attributes which are equated to each other::
-
- class User(Base):
- __table__ = user.join(address)
-
- # assign "user.id", "address.user_id" to the
- # "id" attribute
- id = column_property(user_table.c.id, address_table.c.user_id)
-
-For more examples featuring this usage, see :ref:`maptojoin`.
-
-Another place where :func:`.column_property` is needed is to specify SQL expressions as
-mapped attributes, such as below where we create an attribute ``fullname``
-that is the string concatenation of the ``firstname`` and ``lastname``
-columns::
-
- class User(Base):
- __tablename__ = 'user'
- id = Column(Integer, primary_key=True)
- firstname = Column(String(50))
- lastname = Column(String(50))
- fullname = column_property(firstname + " " + lastname)
-
-See examples of this usage at :ref:`mapper_sql_expressions`.
-
-.. autofunction:: column_property
-
-.. _mapper_primary_key:
-
-Mapping to an Explicit Set of Primary Key Columns
--------------------------------------------------
-
-The :class:`.Mapper` construct in order to successfully map a table always
-requires that at least one column be identified as the "primary key" for
-that selectable. This is so that when an ORM object is loaded or persisted,
-it can be placed in the :term:`identity map` with an appropriate
-:term:`identity key`.
-
-To support this use case, all :class:`.FromClause` objects (where
-:class:`.FromClause` is the common base for objects such as :class:`.Table`,
-:class:`.Join`, :class:`.Subquery`, etc.) have an attribute
-:attr:`.FromClause.primary_key` which returns a collection of those
-:class:`.Column` objects that indicate they are part of a "primary key",
-which is derived from each :class:`.Column` object being a member of a
-:class:`.PrimaryKeyConstraint` collection that's associated with the
-:class:`.Table` from which they ultimately derive.
-
-In those cases where the selectable being mapped does not include columns
-that are explicitly part of the primary key constraint on their parent table,
-a user-defined set of primary key columns must be defined. The
-:paramref:`.Mapper.primary_key` parameter is used for this purpose.
-
-Given the following example of a :ref:`Imperative Table <orm_imperative_table_configuration>`
-mapping against an existing :class:`.Table` object, as would occur in a scenario
-such as when the :class:`.Table` were :term:`reflected` from an existing
-database, where the table does not have any declared primary key, we may
-map such a table as in the following example::
-
- from sqlalchemy import Column
- from sqlalchemy import MetaData
- from sqlalchemy import String
- from sqlalchemy import Table
- from sqlalchemy import UniqueConstraint
- from sqlalchemy.orm import declarative_base
-
-
- metadata = MetaData()
- group_users = Table(
- "group_users",
- metadata,
- Column("user_id", String(40), nullable=False),
- Column("group_id", String(40), nullable=False),
- UniqueConstraint("user_id", "group_id")
- )
-
-
- Base = declarative_base()
-
-
- class GroupUsers(Base):
- __table__ = group_users
- __mapper_args__ = {
- "primary_key": [group_users.c.user_id, group_users.c.group_id]
- }
-
-Above, the ``group_users`` table is an association table of some kind
-with string columns ``user_id`` and ``group_id``, but no primary key is set up;
-instead, there is only a :class:`.UniqueConstraint` establishing that the
-two columns represent a unique key. The :class:`.Mapper` does not automatically
-inspect unique constraints for primary keys; instead, we make use of the
-:paramref:`.Mapper.primary_key` parameter, passing a collection of
-``[group_users.c.user_id, group_users.c.group_id]``, indicating that these two
-columns should be used in order to construct the identity key for instances
-of the ``GroupUsers`` class.
-
-
-.. _include_exclude_cols:
-
-Mapping a Subset of Table Columns
----------------------------------
-
-Sometimes, a :class:`_schema.Table` object was made available using the
-reflection process described at :ref:`metadata_reflection` to load
-the table's structure from the database.
-For such a table that has lots of columns that don't need to be referenced
-in the application, the ``include_properties`` or ``exclude_properties``
-arguments can specify that only a subset of columns should be mapped.
-For example::
-
- class User(Base):
- __table__ = user_table
- __mapper_args__ = {
- 'include_properties' :['user_id', 'user_name']
- }
-
-...will map the ``User`` class to the ``user_table`` table, only including
-the ``user_id`` and ``user_name`` columns - the rest are not referenced.
-Similarly::
-
- class Address(Base):
- __table__ = address_table
- __mapper_args__ = {
- 'exclude_properties' : ['street', 'city', 'state', 'zip']
- }
-
-...will map the ``Address`` class to the ``address_table`` table, including
-all columns present except ``street``, ``city``, ``state``, and ``zip``.
-
-When this mapping is used, the columns that are not included will not be
-referenced in any SELECT statements emitted by :class:`_query.Query`, nor will there
-be any mapped attribute on the mapped class which represents the column;
-assigning an attribute of that name will have no effect beyond that of
-a normal Python attribute assignment.
-
-In some cases, multiple columns may have the same name, such as when
-mapping to a join of two or more tables that share some column name.
-``include_properties`` and ``exclude_properties`` can also accommodate
-:class:`_schema.Column` objects to more accurately describe which columns
-should be included or excluded::
-
- class UserAddress(Base):
- __table__ = user_table.join(addresses_table)
- __mapper_args__ = {
- 'exclude_properties' :[address_table.c.id],
- 'primary_key' : [user_table.c.id]
- }
-
-.. note::
-
- insert and update defaults configured on individual :class:`_schema.Column`
- objects, i.e. those described at :ref:`metadata_defaults` including those
- configured by the :paramref:`_schema.Column.default`,
- :paramref:`_schema.Column.onupdate`, :paramref:`_schema.Column.server_default` and
- :paramref:`_schema.Column.server_onupdate` parameters, will continue to function
- normally even if those :class:`_schema.Column` objects are not mapped. This is
- because in the case of :paramref:`_schema.Column.default` and
- :paramref:`_schema.Column.onupdate`, the :class:`_schema.Column` object is still present
- on the underlying :class:`_schema.Table`, thus allowing the default functions to
- take place when the ORM emits an INSERT or UPDATE, and in the case of
- :paramref:`_schema.Column.server_default` and :paramref:`_schema.Column.server_onupdate`,
- the relational database itself emits these defaults as a server side
- behavior.
+This section has been integrated into the
+:ref:`orm_declarative_table_config_toplevel` Declarative section.
:ref:`unified_tutorial`, where ORM configuration is first introduced at
:ref:`tutorial_orm_table_metadata`.
+.. _orm_mapping_styles:
ORM Mapping Styles
==================
Declarative Mapping
-------------------
-The **Declarative Mapping** is the typical way that
-mappings are constructed in modern SQLAlchemy. The most common pattern
-is to first construct a base class using the :func:`_orm.declarative_base`
-function, which will apply the declarative mapping process to all subclasses
-that derive from it. Below features a declarative base which is then
-used in a declarative table mapping::
+The **Declarative Mapping** is the typical way that mappings are constructed in
+modern SQLAlchemy. The most common pattern is to first construct a base class
+using the :class:`_orm.DeclarativeBase` superclass. The resulting base class,
+when subclassed will apply the declarative mapping process to all subclasses
+that derive from it, relative to a particular :class:`_orm.registry` that
+is local to the new base by default. The example below illustrates
+the use of a declarative base which is then used in a declarative table mapping::
- from sqlalchemy import Column, Integer, String, ForeignKey
- from sqlalchemy.orm import declarative_base
+ from sqlalchemy import Integer, String, ForeignKey
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import mapped_column
# declarative base class
- Base = declarative_base()
+ class Base(DeclarativeBase):
+ pass
# an example mapping using the base
class User(Base):
__tablename__ = 'user'
- id = Column(Integer, primary_key=True)
- name = Column(String)
- fullname = Column(String)
- nickname = Column(String)
+ id: Mapped[int] = mapped_column(primary_key=True)
+ name: Mapped[str]
+ fullname: Mapped[str] = mapped_column(String(30))
+ nickname: Mapped[Optional[str]]
-Above, the :func:`_orm.declarative_base` callable returns a new base class from
+Above, the :class:`_orm.DeclarativeBase` class is used to generate a new
+base class (within SQLAlchemy's documentation it's typically referred towards
+as ``Base``, however can have any desired name) from
which new classes to be mapped may inherit from, as above a new mapped
class ``User`` is constructed.
-The base class refers to a :class:`_orm.registry` object that maintains a
-collection of related mapped classes. The :func:`_orm.declarative_base`
-function is in fact shorthand for first creating the registry with the
-:class:`_orm.registry` constructor, and then generating a base class using the
-:meth:`_orm.registry.generate_base` method::
-
- from sqlalchemy.orm import registry
+.. versionchanged:: 2.0 The :class:`_orm.DeclarativeBase` superclass supersedes
+ the use of the :func:`_orm.declarative_base` function and
+ :meth:`_orm.registry.generate_base` methods; the superclass approach
+ integrates with :pep:`484` tools without the use of plugins.
+ See :ref:`whatsnew_20_orm_declarative_typing` for migration notes.
- # equivalent to Base = declarative_base()
-
- mapper_registry = registry()
- Base = mapper_registry.generate_base()
+The base class refers to a :class:`_orm.registry` object that maintains a
+collection of related mapped classes. as well as to a :class:`_schema.MetaData`
+object that retains a collection of :class:`_schema.Table` objects to which
+the classes are mapped.
The major Declarative mapping styles are further detailed in the following
sections:
* :ref:`orm_declarative_generated_base_class` - declarative mapping using a
- base class generated by the :class:`_orm.registry` object.
+ base class.
* :ref:`orm_declarative_decorator` - declarative mapping using a decorator,
rather than a base class.
Within the scope of a Declarative mapped class, there are also two varieties
of how the :class:`_schema.Table` metadata may be declared. These include:
-* :ref:`orm_declarative_table` - individual :class:`_schema.Column` definitions
- are combined with a table name and additional arguments, where the Declarative
- mapping process will construct a :class:`_schema.Table` object to be mapped.
+* :ref:`orm_declarative_table` - table columns are declared inline
+ within the mapped class using the :func:`_orm.mapped_column` directive
+ (or in legacy form, using the :class:`_schema.Column` object directly).
+ The :func:`_orm.mapped_column` directive may also be optionally combined with
+ type annotations using the :class:`_orm.Mapped` class which can provide
+ some details about the mapped columns directly. The column
+ directives, in combination with the ``__tablename__`` and optional
+ ``__table_args__`` class level directives will allow the
+ Declarative mapping process to construct a :class:`_schema.Table` object to
+ be mapped.
* :ref:`orm_imperative_table_configuration` - Instead of specifying table name
and attributes separately, an explicitly constructed :class:`_schema.Table` object
is associated with a class that is otherwise mapped declaratively. This
- style of mapping is a hybrid of "declarative" and "imperative" mapping.
+ style of mapping is a hybrid of "declarative" and "imperative" mapping,
+ and applies to techniques such as mapping classes to :term:`reflected`
+ :class:`_schema.Table` objects, as well as mapping classes to existing
+ Core constructs such as joins and subqueries.
+
Documentation for Declarative mapping continues at :ref:`declarative_config_toplevel`.
mapped class using the :meth:`_orm.registry.map_imperatively` method,
where the target class does not include any declarative class attributes.
+.. tip:: The imperative mapping form is a lesser-used form of mapping that
+ originates from the very first releases of SQLAlchemy in 2006. It's
+ essentially a means of bypassing the Declarative system to provide a
+ more "barebones" system of mapping, and does not offer modern features
+ such as :pep:`484` support. As such, most documentation examples
+ use Declarative forms, and it's recommended that new users start
+ with :ref:`Declarative Table <orm_declarative_table_config_toplevel>`
+ configuration.
+
.. versionchanged:: 2.0 The :meth:`_orm.registry.map_imperatively` method
is now used to create classical mappings. The ``sqlalchemy.orm.mapper()``
standalone function is effectively removed.
In "classical" form, the table metadata is created separately with the
:class:`_schema.Table` construct, then associated with the ``User`` class via
-the :meth:`_orm.registry.map_imperatively` method::
+the :meth:`_orm.registry.map_imperatively` method, after establishing
+a :class:`_orm.registry` instance. Normally, a single instance of
+:class:`_orm.registry`
+shared for all mapped classes that are related to each other::
from sqlalchemy import Table, Column, Integer, String, ForeignKey
from sqlalchemy.orm import registry
mapper_registry.map_imperatively(Address, address)
-When using classical mappings, classes must be provided directly without the benefit
-of the "string lookup" system provided by Declarative. SQL expressions are typically
-specified in terms of the :class:`_schema.Table` objects, i.e. ``address.c.id`` above
-for the ``Address`` relationship, and not ``Address.id``, as ``Address`` may not
-yet be linked to table metadata, nor can we specify a string here.
-
-Some examples in the documentation still use the classical approach, but note that
-the classical as well as Declarative approaches are **fully interchangeable**. Both
-systems ultimately create the same configuration, consisting of a :class:`_schema.Table`,
-user-defined class, linked together with a :class:`_orm.Mapper` object. When we talk about
-"the behavior of :class:`_orm.Mapper`", this includes when using the Declarative system
-as well - it's still used, just behind the scenes.
+Note that classes which are mapped with the Imperative approach are **fully
+interchangeable** with those mapped with the Declarative approach. Both systems
+ultimately create the same configuration, consisting of a
+:class:`_schema.Table`, user-defined class, linked together with a
+:class:`_orm.Mapper` object. When we talk about "the behavior of
+:class:`_orm.Mapper`", this includes when using the Declarative system as well
+- it's still used, just behind the scenes.
.. _orm_mapper_configuration_overview:
==================================
With all mapping forms, the mapping of the class can be configured in many ways
-by passing construction arguments that become part of the :class:`_orm.Mapper`
-object. The construct which ultimately receives these arguments is the
-constructor to the :class:`_orm.Mapper` class, and the arguments are delivered
-to it originating from one of the front-facing mapping functions defined on the
-:class:`_orm.registry` object.
+by passing construction arguments that ultimately become part of the :class:`_orm.Mapper`
+object via its constructor. The parameters that are delivered to
+:class:`_orm.Mapper` originate from the given mapping form, including
+parameters passed to :meth:`_orm.registry.map_imperatively` for an Imperative
+mapping, or when using the Declarative system, from a combination
+of the table columns, SQL expressions and
+relationships being mapped along with that of attributes such as
+:ref:`__mapper_args__ <orm_declarative_mapper_options>`.
There are four general classes of configuration information that the
:class:`_orm.Mapper` class looks for:
the section :ref:`orm_declarative_properties` for notes on this process.
When mapping with the :ref:`imperative <orm_imperative_mapping>` style, the
-properties dictionary is passed directly as the ``properties`` argument
+properties dictionary is passed directly as the
+``properties`` parameter
to :meth:`_orm.registry.map_imperatively`, which will pass it along to the
:paramref:`_orm.Mapper.properties` parameter.
a convenient keyword constructor that will accept as optional keyword arguments
all the attributes that are named. E.g.::
- from sqlalchemy.orm import declarative_base
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import mapped_column
- Base = declarative_base()
+ class Base(DeclarativeBase):
+ pass
class User(Base):
__tablename__ = 'user'
- id = Column(...)
- name = Column(...)
- fullname = Column(...)
+ id: Mapped[int] = mapped_column(primary_key=True)
+ name: Mapped[str]
+ fullname: Mapped[str]
An object of type ``User`` above will have a constructor which allows
``User`` objects to be created as::
u1 = User(name='some name', fullname='some fullname')
-The above constructor may be customized by passing a Python callable to
-the :paramref:`_orm.registry.constructor` parameter which provides the
-desired default ``__init__()`` behavior.
+.. tip::
+
+ The :ref:`orm_declarative_native_dataclasses` feature provides an alternate
+ means of generating a default ``__init__()`` method by using
+ Python dataclasses, and allows for a highly configurable constructor
+ form.
+
+A class that includes an explicit ``__init__()`` method will maintain
+that method, and no default constructor will be applied.
+
+To change the default constructor used, a user-defined Python callable may be
+provided to the :paramref:`_orm.registry.constructor` parameter which will be
+used as the default constructor.
The constructor also applies to imperative mappings::
from sqlalchemy import Table, Column, Integer, \
String, MetaData, join, ForeignKey
- from sqlalchemy.ext.declarative import declarative_base
+ from sqlalchemy.orm import DeclarativeBase
from sqlalchemy.orm import column_property
metadata_obj = MetaData()
# columns.
user_address_join = join(user_table, address_table)
- Base = declarative_base()
+ class Base(DeclarativeBase):
+ metadata = metadata_obj
# map to it
class AddressUser(Base):
# ...
- value = Column(Integer)
+ value = mapped_column(Integer)
someobject = session.get(SomeClass, 5)
class Foo(Base):
__tablename__ = 'foo'
- pk = Column(Integer, primary_key=True)
- bar = Column(Integer)
+ pk = mapped_column(Integer, primary_key=True)
+ bar = mapped_column(Integer)
e = create_engine("postgresql+psycopg2://scott:tiger@localhost/test", echo=True)
Base.metadata.create_all(e)
class MyObject(Base):
__tablename__ = 'my_table'
- id = Column(Integer, primary_key=True)
- data = Column(String(50), nullable=True)
+ id = mapped_column(Integer, primary_key=True)
+ data = mapped_column(String(50), nullable=True)
obj = MyObject(id=1)
session.add(obj)
class MyObject(Base):
__tablename__ = 'my_table'
- id = Column(Integer, primary_key=True)
- data = Column(String(50), nullable=True, server_default="default")
+ id = mapped_column(Integer, primary_key=True)
+ data = mapped_column(String(50), nullable=True, server_default="default")
obj = MyObject(id=1)
session.add(obj)
class MyObject(Base):
__tablename__ = 'my_table'
- id = Column(Integer, primary_key=True)
- data = Column(String(50), nullable=True, server_default="default")
+ id = mapped_column(Integer, primary_key=True)
+ data = mapped_column(String(50), nullable=True, server_default="default")
obj = MyObject(id=1, data=None)
session.add(obj)
class MyObject(Base):
__tablename__ = 'my_table'
- id = Column(Integer, primary_key=True)
- data = Column(
+ id = mapped_column(Integer, primary_key=True)
+ data = mapped_column(
String(50).evaluates_none(), # indicate that None should always be passed
nullable=True, server_default="default")
class MyModel(Base):
__tablename__ = 'my_table'
- id = Column(Integer, primary_key=True)
- timestamp = Column(DateTime(), server_default=func.now())
+ id = mapped_column(Integer, primary_key=True)
+ timestamp = mapped_column(DateTime(), server_default=func.now())
# assume a database trigger populates a value into this column
# during INSERT
- special_identifier = Column(String(50), server_default=FetchedValue())
+ special_identifier = mapped_column(String(50), server_default=FetchedValue())
__mapper_args__ = {"eager_defaults": True}
class MyModel(Base):
__tablename__ = 'my_table'
- id = Column(Integer, primary_key=True)
- timestamp = Column(DateTime(), server_default=func.now())
+ id = mapped_column(Integer, primary_key=True)
+ timestamp = mapped_column(DateTime(), server_default=func.now())
# assume a database trigger populates a value into this column
# during INSERT
- special_identifier = Column(String(50), server_default=FetchedValue())
+ special_identifier = mapped_column(String(50), server_default=FetchedValue())
After a record with the above mapping is INSERTed, the "timestamp" and
"special_identifier" columns will remain empty, and will be fetched via
class MyOracleModel(Base):
__tablename__ = 'my_table'
- id = Column(Integer, Sequence("my_sequence"), primary_key=True)
- data = Column(String(50))
+ id = mapped_column(Integer, Sequence("my_sequence"), primary_key=True)
+ data = mapped_column(String(50))
The INSERT for a model as above on Oracle looks like:
class MyModel(Base):
__tablename__ = 'my_table'
- timestamp = Column(TIMESTAMP(), server_default=FetchedValue(), primary_key=True)
+ timestamp = mapped_column(TIMESTAMP(), server_default=FetchedValue(), primary_key=True)
An INSERT for the above table on SQL Server looks like:
class MyModel(Base):
__tablename__ = 'my_table'
- timestamp = Column(DateTime(), default=func.now(), primary_key=True)
+ timestamp = mapped_column(DateTime(), default=func.now(), primary_key=True)
Where above, we select the "NOW()" function to deliver a datetime value
to the column. The SQL generated by the above is:
class MyModel(Base):
__tablename__ = 'my_table'
- timestamp = Column(
+ timestamp = mapped_column(
TIMESTAMP(),
default=cast(func.now(), Binary),
primary_key=True)
class MyModel(Base):
__tablename__ = 'my_table'
- timestamp = Column(
+ timestamp = mapped_column(
DateTime,
default=func.datetime('now', 'localtime', type_=DateTime),
primary_key=True)
class MyModel(Base):
__tablename__ = 'my_table'
- id = Column(Integer, primary_key=True)
+ id = mapped_column(Integer, primary_key=True)
- created = Column(DateTime(), default=func.now(), server_default=FetchedValue())
- updated = Column(DateTime(), onupdate=func.now(), server_default=FetchedValue(), server_onupdate=FetchedValue())
+ created = mapped_column(DateTime(), default=func.now(), server_default=FetchedValue())
+ updated = mapped_column(DateTime(), onupdate=func.now(), server_default=FetchedValue(), server_onupdate=FetchedValue())
__mapper_args__ = {"eager_defaults": True}
Supposing two declarative bases are representing two different database
connections::
- BaseA = declarative_base()
+ from sqlalchemy.orm import DeclarativeBase
+ from sqlalchemy.orm import Session
- BaseB = declarative_base()
+ class BaseA(DeclarativeBase):
+ pass
+
+ class BaseB(DeclarativeBase):
+ pass
class User(BaseA):
# ...
>>> metadata_obj.create_all(engine)
BEGIN (implicit)
...
- >>> from sqlalchemy.orm import declarative_base
- >>> Base = declarative_base()
+ >>> from sqlalchemy.orm import DeclarativeBase
+ >>> class Base(DeclarativeBase):
+ ... pass
>>> from sqlalchemy.orm import relationship
>>> class User(Base):
... __table__ = user_table
Python object model, as well as :term:`database metadata` that describes
real SQL tables that exist, or will exist, in a particular database::
- >>> from typing import List
>>> from typing import Optional
>>> from sqlalchemy import ForeignKey
>>> from sqlalchemy import String
... name: Mapped[str] = mapped_column(String(30))
... fullname: Mapped[Optional[str]]
...
- ... addresses: Mapped[List["Address"]] = relationship(
+ ... addresses: Mapped[list["Address"]] = relationship(
... back_populates="user", cascade="all, delete-orphan"
... )
...
- ... def __repr__(self):
+ ... def __repr__(self) -> str:
... return f"User(id={self.id!r}, name={self.name!r}, fullname={self.fullname!r})"
>>> class Address(Base):
...
... user: Mapped["User"] = relationship(back_populates="addresses")
...
- ... def __repr__(self):
+ ... def __repr__(self) -> str:
... return f"Address(id={self.id!r}, email_address={self.email_address!r})"
Above, new ORM mapped classes are declared as part of a common base, which is
which is used here to illustrate usage of the classes, there is also an
option for methods such as ``__repr__()``, ``__eq__()`` and others to be
generated automatically using Python dataclasses. More on dataclass mapping
-at: (the new 2.0 style dataclass documentation is TODO! but it will be great
-:) )
+at :ref:`orm_declarative_native_dataclasses`.
More on table metadata and an intro to ORM declared mapping is in the
Tutorial at :ref:`tutorial_working_with_metadata`.
a complete example, including two :class:`_schema.ForeignKey` constructs::
from sqlalchemy import Integer, ForeignKey, Column
- from sqlalchemy.ext.declarative import declarative_base
+ from sqlalchemy.orm import DeclarativeBase
from sqlalchemy.orm import relationship
- Base = declarative_base()
+ class Base(DeclarativeBase):
+ pass
class Entry(Base):
__tablename__ = 'entry'
- entry_id = Column(Integer, primary_key=True)
- widget_id = Column(Integer, ForeignKey('widget.widget_id'))
- name = Column(String(50))
+ entry_id = mapped_column(Integer, primary_key=True)
+ widget_id = mapped_column(Integer, ForeignKey('widget.widget_id'))
+ name = mapped_column(String(50))
class Widget(Base):
__tablename__ = 'widget'
- widget_id = Column(Integer, primary_key=True)
- favorite_entry_id = Column(Integer,
+ widget_id = mapped_column(Integer, primary_key=True)
+ favorite_entry_id = mapped_column(Integer,
ForeignKey('entry.entry_id',
name="fk_favorite_entry"))
- name = Column(String(50))
+ name = mapped_column(String(50))
entries = relationship(Entry, primaryjoin=
widget_id==Entry.widget_id)
from sqlalchemy import Integer, ForeignKey, String, \
Column, UniqueConstraint, ForeignKeyConstraint
- from sqlalchemy.ext.declarative import declarative_base
+ from sqlalchemy.orm import DeclarativeBase
from sqlalchemy.orm import relationship
- Base = declarative_base()
+ class Base(DeclarativeBase):
+ pass
class Entry(Base):
__tablename__ = 'entry'
- entry_id = Column(Integer, primary_key=True)
- widget_id = Column(Integer, ForeignKey('widget.widget_id'))
- name = Column(String(50))
+ entry_id = mapped_column(Integer, primary_key=True)
+ widget_id = mapped_column(Integer, ForeignKey('widget.widget_id'))
+ name = mapped_column(String(50))
__table_args__ = (
UniqueConstraint("entry_id", "widget_id"),
)
class Widget(Base):
__tablename__ = 'widget'
- widget_id = Column(Integer, autoincrement='ignore_fk', primary_key=True)
- favorite_entry_id = Column(Integer)
+ widget_id = mapped_column(Integer, autoincrement='ignore_fk', primary_key=True)
+ favorite_entry_id = mapped_column(Integer)
- name = Column(String(50))
+ name = mapped_column(String(50))
__table_args__ = (
ForeignKeyConstraint(
__tablename__ = 'user'
__table_args__ = {'mysql_engine': 'InnoDB'}
- username = Column(String(50), primary_key=True)
- fullname = Column(String(100))
+ username = mapped_column(String(50), primary_key=True)
+ fullname = mapped_column(String(100))
addresses = relationship("Address")
__tablename__ = 'address'
__table_args__ = {'mysql_engine': 'InnoDB'}
- email = Column(String(50), primary_key=True)
- username = Column(String(50),
+ email = mapped_column(String(50), primary_key=True)
+ username = mapped_column(String(50),
ForeignKey('user.username', onupdate="cascade")
)
class User(Base):
__tablename__ = 'user'
- username = Column(String(50), primary_key=True)
- fullname = Column(String(100))
+ username = mapped_column(String(50), primary_key=True)
+ fullname = mapped_column(String(100))
# passive_updates=False *only* needed if the database
# does not implement ON UPDATE CASCADE
class Address(Base):
__tablename__ = 'address'
- email = Column(String(50), primary_key=True)
- username = Column(String(50), ForeignKey('user.username'))
+ email = mapped_column(String(50), primary_key=True)
+ username = mapped_column(String(50), ForeignKey('user.username'))
Key limitations of ``passive_updates=False`` include:
-.. currentmodule:: sqlalchemy.orm
-
===============================
-Mapping Columns and Expressions
+Mapping SQL Expressions
===============================
-The following sections discuss how table columns and SQL expressions are
-mapped to individual object attributes.
+This page has been merged into the
+:ref:`mapper_config_toplevel` index.
+
.. toctree::
- :maxdepth: 3
+ :hidden:
mapping_columns
- mapped_sql_expr
- mapped_attributes
- composites
class Node(Base):
__tablename__ = 'node'
- id = Column(Integer, primary_key=True)
- parent_id = Column(Integer, ForeignKey('node.id'))
- data = Column(String(50))
+ id = mapped_column(Integer, primary_key=True)
+ parent_id = mapped_column(Integer, ForeignKey('node.id'))
+ data = mapped_column(String(50))
children = relationship("Node")
With this structure, a graph such as the following::
class Node(Base):
__tablename__ = 'node'
- id = Column(Integer, primary_key=True)
- parent_id = Column(Integer, ForeignKey('node.id'))
- data = Column(String(50))
+ id = mapped_column(Integer, primary_key=True)
+ parent_id = mapped_column(Integer, ForeignKey('node.id'))
+ data = mapped_column(String(50))
parent = relationship("Node", remote_side=[id])
Where above, the ``id`` column is applied as the :paramref:`_orm.relationship.remote_side`
class Node(Base):
__tablename__ = 'node'
- id = Column(Integer, primary_key=True)
- parent_id = Column(Integer, ForeignKey('node.id'))
- data = Column(String(50))
+ id = mapped_column(Integer, primary_key=True)
+ parent_id = mapped_column(Integer, ForeignKey('node.id'))
+ data = mapped_column(String(50))
children = relationship("Node",
backref=backref('parent', remote_side=[id])
)
['folder.account_id', 'folder.folder_id']),
)
- account_id = Column(Integer, primary_key=True)
- folder_id = Column(Integer, primary_key=True)
- parent_id = Column(Integer)
- name = Column(String)
+ account_id = mapped_column(Integer, primary_key=True)
+ folder_id = mapped_column(Integer, primary_key=True)
+ parent_id = mapped_column(Integer)
+ name = mapped_column(String)
parent_folder = relationship("Folder",
backref="child_folders",
class Node(Base):
__tablename__ = 'node'
- id = Column(Integer, primary_key=True)
- parent_id = Column(Integer, ForeignKey('node.id'))
- data = Column(String(50))
+ id = mapped_column(Integer, primary_key=True)
+ parent_id = mapped_column(Integer, ForeignKey('node.id'))
+ data = mapped_column(String(50))
children = relationship("Node",
lazy="joined",
join_depth=2)
import datetime
class HasTimestamp:
- timestamp = Column(DateTime, default=datetime.datetime.now)
+ timestamp = mapped_column(DateTime, default=datetime.datetime.now)
class SomeEntity(HasTimestamp, Base):
__tablename__ = "some_entity"
- id = Column(Integer, primary_key=True)
+ id = mapped_column(Integer, primary_key=True)
class SomeOtherEntity(HasTimestamp, Base):
__tablename__ = "some_entity"
- id = Column(Integer, primary_key=True)
+ id = mapped_column(Integer, primary_key=True)
The above classes ``SomeEntity`` and ``SomeOtherEntity`` will each have a column
event is applied to a specific class or superclass. For example, to
intercept all new objects for a particular declarative base::
- from sqlalchemy.ext.declarative import declarative_base
+ from sqlalchemy.orm import DeclarativeBase
from sqlalchemy import event
- Base = declarative_base()
+ class Base(DeclarativeBase):
+ pass
@event.listens_for(Base, "init", propagate=True)
def intercept_init(instance, args, kwargs):
class User(Base):
__tablename__ = 'user'
- id = Column(Integer, primary_key=True)
- name = Column(String(50), nullable=False)
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(50), nullable=False)
addresses = relationship("Address", backref="user")
class Address(Base):
__tablename__ = 'address'
- id = Column(Integer, primary_key=True)
- email_address = Column(String(50), nullable=False)
- user_id = Column(Integer, ForeignKey('user.id'), nullable=False)
+ id = mapped_column(Integer, primary_key=True)
+ email_address = mapped_column(String(50), nullable=False)
+ user_id = mapped_column(Integer, ForeignKey('user.id'), nullable=False)
Assume a ``User`` object with one ``Address``, already persistent::
class User(Base):
__tablename__ = 'user'
- id = Column(Integer, primary_key=True)
- version_id = Column(Integer, nullable=False)
- name = Column(String(50), nullable=False)
+ id = mapped_column(Integer, primary_key=True)
+ version_id = mapped_column(Integer, nullable=False)
+ name = mapped_column(String(50), nullable=False)
__mapper_args__ = {
"version_id_col": version_id
class User(Base):
__tablename__ = 'user'
- id = Column(Integer, primary_key=True)
- version_uuid = Column(String(32), nullable=False)
- name = Column(String(50), nullable=False)
+ id = mapped_column(Integer, primary_key=True)
+ version_uuid = mapped_column(String(32), nullable=False)
+ name = mapped_column(String(50), nullable=False)
__mapper_args__ = {
'version_id_col':version_uuid,
class User(Base):
__tablename__ = 'user'
- id = Column(Integer, primary_key=True)
- name = Column(String(50), nullable=False)
- xmin = Column("xmin", String, system=True, server_default=FetchedValue())
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(50), nullable=False)
+ xmin = mapped_column("xmin", String, system=True, server_default=FetchedValue())
__mapper_args__ = {
'version_id_col': xmin,
class User(Base):
__tablename__ = 'user'
- id = Column(Integer, primary_key=True)
- version_uuid = Column(String(32), nullable=False)
- name = Column(String(50), nullable=False)
+ id = mapped_column(Integer, primary_key=True)
+ version_uuid = mapped_column(String(32), nullable=False)
+ name = mapped_column(String(50), nullable=False)
__mapper_args__ = {
'version_id_col':version_uuid,
to declare our user-defined classes and :class:`_schema.Table` metadata
at once.
-Setting up the Registry
-^^^^^^^^^^^^^^^^^^^^^^^
+Establishing a Declarative Base
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
When using the ORM, the :class:`_schema.MetaData` collection remains present,
-however it itself is contained within an ORM-only object known as the
-:class:`_orm.registry`. We create a :class:`_orm.registry` by constructing
-it::
-
- >>> from sqlalchemy.orm import registry
- >>> mapper_registry = registry()
-
-The above :class:`_orm.registry`, when constructed, automatically includes
-a :class:`_schema.MetaData` object that will store a collection of
-:class:`_schema.Table` objects::
-
- >>> mapper_registry.metadata
+however it itself is associated with an ORM-only construct commonly referred
+towards as the **Declarative Base**. The most expedient way to acquire
+a new Declarative Base is to create a new class that subclasses the
+SQLAlchemy :class:`_orm.DeclarativeBase` class::
+
+ >>> from sqlalchemy.orm import DeclarativeBase
+ >>> class Base(DeclarativeBase):
+ ... pass
+
+Above, the ``Base`` class is what we'll refer towards as the Declarative Base.
+When we make new classes that are subclasses of ``Base``, combined with
+appropriate class-level directives, they will each be established as part of an
+object relational mapping against a particular database table (or tables, in
+advanced usages).
+
+The Declarative Base, when declared as a new class, refers to a
+:class:`_schema.MetaData` collection that is
+created for us automatically (options exist to use our own :class:`.MetaData`
+object as well); this :class:`.MetaData` is accessible via the ``.metadata``
+class-level attribute. As we create new mapped classes, they each will reference a
+:class:`.Table` within this :class:`.MetaData` collection::
+
+ >>> Base.metadata
MetaData()
-Instead of declaring :class:`_schema.Table` objects directly, we will now
-declare them indirectly through directives applied to our mapped classes. In
-the most common approach, each mapped class descends from a common base class
-known as the **declarative base**. We get a new declarative base from the
-:class:`_orm.registry` using the :meth:`_orm.registry.generate_base` method::
-
- >>> Base = mapper_registry.generate_base()
-
-.. tip::
-
- The steps of creating the :class:`_orm.registry` and "declarative base"
- classes can be combined into one step using the historically familiar
- :func:`_orm.declarative_base` function::
-
- from sqlalchemy.orm import declarative_base
- Base = declarative_base()
-
- ..
+The Declarative Base also refers to a collection called :class:`_orm.registry`, which
+is the central "mapper configuration" unit in the SQLAlchemy ORM. While
+seldom accessed directly, this object is central to the mapper configuration
+process, as a set of ORM mapped classes will coordinate with each other via
+this registry. As was the case with :class:`.MetaData`, our Declarative
+Base also created a :class:`_orm.registry` for us (again with options to
+pass our own :class:`_orm.registry`), which we can access
+via the ``.registry`` class variable::
+
+ >>> Base.registry
+ <sqlalchemy.orm.decl_api.registry object at 0x...>
+
+:class:`_orm.registry` also provides other mapper configurational patterns,
+including different ways to acquire a Declarative Base object, as well as class
+decorators and class-processing functions which allow user-defined classes to
+be mapped without using any particular base class. Therefore, keep in mind that
+all the ORM patterns here that use "declarative base" can just as easily
+use other patterns based on class decorators or configurational functions.
.. _tutorial_declaring_mapped_classes:
Declaring Mapped Classes
^^^^^^^^^^^^^^^^^^^^^^^^
-The ``Base`` object above is a Python class which will serve as the base class
-for the ORM mapped classes we declare. We can now define ORM mapped classes
-for the ``user`` and ``address`` table in terms of new classes ``User`` and
-``Address``::
-
+With the ``Base`` class established, we can now define ORM mapped classes
+for the ``user_account`` and ``address`` tables in terms of new classes ``User`` and
+``Address``. We illustrate below the most modern form of Declarative, which
+is driven from :pep:`484` type annotations using a special type
+:class:`.Mapped`, which indicates attributes to be mapped as particular
+types::
+
+ >>> from typing import List
+ >>> from typing import Optional
+ >>> from sqlalchemy.orm import Mapped
+ >>> from sqlalchemy.orm import mapped_column
>>> from sqlalchemy.orm import relationship
+
>>> class User(Base):
... __tablename__ = 'user_account'
...
- ... id = Column(Integer, primary_key=True)
- ... name = Column(String(30))
- ... fullname = Column(String)
+ ... id: Mapped[int] = mapped_column(primary_key=True)
+ ... name: Mapped[str] = mapped_column(String(30))
+ ... fullname: Mapped[Optional[str]]
...
- ... addresses = relationship("Address", back_populates="user")
+ ... addresses: Mapped[List["Address"]] = relationship(back_populates="user")
...
- ... def __repr__(self):
+ ... def __repr__(self) -> str:
... return f"User(id={self.id!r}, name={self.name!r}, fullname={self.fullname!r})"
>>> class Address(Base):
... __tablename__ = 'address'
...
- ... id = Column(Integer, primary_key=True)
- ... email_address = Column(String, nullable=False)
- ... user_id = Column(Integer, ForeignKey('user_account.id'))
+ ... id: Mapped[int] = mapped_column(primary_key=True)
+ ... email_address: Mapped[str]
+ ... user_id = mapped_column(ForeignKey('user_account.id'))
...
- ... user = relationship("User", back_populates="addresses")
+ ... user: Mapped[User] = relationship(back_populates="addresses")
...
- ... def __repr__(self):
+ ... def __repr__(self) -> str:
... return f"Address(id={self.id!r}, email_address={self.email_address!r})"
-The above two classes are now our mapped classes, and are available for use in
-ORM persistence and query operations, which will be described later. But they
-also include :class:`_schema.Table` objects that were generated as part of the
-declarative mapping process, and are equivalent to the ones that we declared
-directly in the previous Core section. We can see these
-:class:`_schema.Table` objects from a declarative mapped class using the
-``.__table__`` attribute::
-
- >>> User.__table__
- Table('user_account', MetaData(),
- Column('id', Integer(), table=<user_account>, primary_key=True, nullable=False),
- Column('name', String(length=30), table=<user_account>),
- Column('fullname', String(), table=<user_account>), schema=None)
-
-This :class:`_schema.Table` object was generated from the declarative process
-based on the ``.__tablename__`` attribute defined on each of our classes,
-as well as through the use of :class:`_schema.Column` objects assigned
-to class-level attributes within the classes. These :class:`_schema.Column`
-objects can usually be declared without an explicit "name" field inside
-the constructor, as the Declarative process will name them automatically
-based on the attribute name that was used.
-
-.. seealso::
-
- :ref:`orm_declarative_mapping` - overview of Declarative class mapping
-
-
-Other Mapped Class Details
-^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-For a few quick explanations for the classes above, note the following
-attributes:
-
-* **the classes have an automatically generated __init__() method** - both classes by default
- receive an ``__init__()`` method that allows for parameterized construction
- of the objects. We are free to provide our own ``__init__()`` method as well.
- The ``__init__()`` allows us to create instances of ``User`` and ``Address``
- passing attribute names, most of which above are linked directly to
- :class:`_schema.Column` objects, as parameter names::
+The two classes above, ``User`` and ``Address``, are now referred towards
+as **ORM Mapped Classes**, and are available for use in
+ORM persistence and query operations, which will be described later. Details
+about these classes include:
+
+* Each class refers to a :class:`_schema.Table` object that was generated as
+ part of the declarative mapping process, and are equivalent
+ in structure to the :class:`_schema.Table` objects we constructed
+ directly in the previous Core section. This :class:`._schema.Table`
+ is available from an attribute added to the class called ``__table__``.
+* To indicate columns in the :class:`_schema.Table`, we use the
+ :func:`_orm.mapped_column` construct, in combination with
+ typing annotations based on the :class:`_orm.Mapped` type.
+* For columns with simple datatypes and no other options, we can indicate a
+ :class:`_orm.Mapped` type annotation alone, using simple Python types like
+ ``int`` and ``str`` to mean :class:`.Integer` and :class:`.String`.
+ Customization of how Python types are interpreted within the Declarative
+ mapping process is very open ended; see the section
+ :ref:`orm_declarative_mapped_column` for background.
+* A column can be declared as "nullable" or "not null" based on the
+ presence of the ``Optional[]`` type annotation; alternatively, the
+ :paramref:`_orm.mapped_column.nullable` parameter may be used instead.
+* Usage of explicit typing annotations is **completely
+ optional**. We can also use :func:`_orm.mapped_column` without annotations.
+ When using this form, we would use more explicit type objects like
+ :class:`.Integer` and :class:`.String` as well as ``nullable=False``
+ as needed within each :func:`_orm.mapped_column` construct.
+* Two additional attributes, ``User.addresses`` and ``Address.user``, define
+ a different kind of attribute called :func:`_orm.relationship`, which
+ features similar annotation-aware configuration styles as shown. The
+ :func:`_orm.relationship` construct is discussed more fully at
+ :ref:`tutorial_orm_related_objects`.
+* The classes are automatically given an ``__init__()`` method if we don't
+ declare one of our own. The default form of this method accepts all
+ attribute names as optional keyword arguments::
>>> sandy = User(name="sandy", fullname="Sandy Cheeks")
- More detail on this method is at :ref:`mapped_class_default_constructor`.
+* The ``__repr__()`` methods are added so that we get a readable string output;
+ there's no requirement for these methods to be here.
+
+.. topic:: Where'd the old Declarative go?
+
+ Users of SQLAlchemy 1.4 or previous will note that the above mapping
+ uses a dramatically different form than before; not only does it use
+ :func:`_orm.mapped_column` instead of :class:`.Column` in the Declarative
+ mapping, it also uses Python type annotations to derive column information.
+
+ To provide context for users of the "old" way, Declarative mappings can
+ still be made using :class:`.Column` objects (as well as using the
+ :func:`_orm.declarative_base` function to create the base class) as before,
+ and these forms will continue to be supported with no plans to
+ remove support. The reason these two facilities
+ are superseded by new constructs is first and foremost to integrate
+ smoothly with :pep:`484` tools, including IDEs such as VSCode and type
+ checkers such as Mypy and Pyright, without the need for plugins. Secondly,
+ deriving the declarations from type annotations is part of SQLAlchemy's
+ integration with Python dataclasses, which can now be
+ :ref:`generated natively <orm_declarative_native_dataclasses>` from mappings.
+
+ For users who like the "old" way, but still desire their IDEs to not
+ mistakenly report typing errors for their declarative mappings, the
+ :func:`_orm.mapped_column` construct is a drop-in replacement for
+ :class:`.Column` in an ORM Declarative mapping (note that
+ :func:`_orm.mapped_column` is for ORM Declarative mappings only; it can't
+ be used within a :class:`.Table` construct), and the type annotations are
+ optional. Our mapping above can be written without annotations as::
+
+ class User(Base):
+ __tablename__ = 'user_account'
+
+ id = mapped_column(Integer, primary_key=True)
+ name = mapped_column(String(30), nullable=False)
+ fullname = mapped_column(String)
+
+ addresses = relationship("Address", back_populates="user")
+
+ # ... definition continues
+
+ The above class has an advantage over one that uses :class:`.Column`
+ directly, in that the ``User`` class as well as instances of ``User``
+ will indicate the correct typing information to typing tools, without
+ the use of plugins. :func:`_orm.mapped_column` also allows for additional
+ ORM-specific parameters to configure behaviors such as deferred column loading,
+ which previously needed a separate :func:`_orm.deferred` function to be
+ used with :class:`_schema.Column`.
+
+ There's also an example of converting an old-style Declarative class
+ to the new style, which can be seen at :ref:`whatsnew_20_orm_declarative_typing`
+ in the :ref:`whatsnew_20_toplevel` guide.
- ..
-
-* **we provided a __repr__() method** - this is **fully optional**, and is
- strictly so that our custom classes have a descriptive string representation
- and is not otherwise required::
-
- >>> sandy
- User(id=None, name='sandy', fullname='Sandy Cheeks')
-
- ..
-
- An interesting thing to note above is that the ``id`` attribute automatically
- returns ``None`` when accessed, rather than raising ``AttributeError`` as
- would be the usual Python behavior for missing attributes.
+.. seealso::
-* **we also included a bidirectional relationship** - this is another **fully optional**
- construct, where we made use of an ORM construct called
- :func:`_orm.relationship` on both classes, which indicates to the ORM that
- these ``User`` and ``Address`` classes refer to each other in a :term:`one to
- many` / :term:`many to one` relationship. The use of
- :func:`_orm.relationship` above is so that we may demonstrate its behavior
- later in this tutorial; it is **not required** in order to define the
- :class:`_schema.Table` structure.
+ :ref:`orm_mapping_styles` - full background on different ORM configurational
+ styles.
+ :ref:`orm_declarative_mapping` - overview of Declarative class mapping
-Emitting DDL to the database
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+ :ref:`orm_declarative_table` - detail on how to use
+ :func:`_orm.mapped_column` and :class:`_orm.Mapped` to define the columns
+ within a :class:`_schema.Table` to be mapped when using Declarative.
-This section is named the same as the section :ref:`tutorial_emitting_ddl`
-discussed in terms of Core. This is because emitting DDL with our
-ORM mapped classes is not any different. If we wanted to emit DDL
-for the :class:`_schema.Table` objects we've created as part of
-our declaratively mapped classes, we still can use
-:meth:`_schema.MetaData.create_all` as before.
-In our case, we have already generated the ``user`` and ``address`` tables
-in our SQLite database. If we had not done so already, we would be free to
-make use of the :class:`_schema.MetaData` associated with our
-:class:`_orm.registry` and ORM declarative base class in order to do so,
-using :meth:`_schema.MetaData.create_all`::
+Emitting DDL to the database from an ORM mapping
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- # emit CREATE statements given ORM registry
- mapper_registry.metadata.create_all(engine)
+As our ORM mapped classes refer to :class:`_schema.Table` objects contained
+within a :class:`_schema.MetaData` collection, emitting DDL given the
+Declarative Base uses the same process as that described previously at
+:ref:`tutorial_emitting_ddl`. In our case, we have already generated the
+``user`` and ``address`` tables in our SQLite database. If we had not done so
+already, we would be free to make use of the :class:`_schema.MetaData`
+associated with our ORM Declarative Base class in order to do so, by accessing
+the collection from the ``Base.metadata`` attribute and then using
+:meth:`_schema.MetaData.create_all` as before::
- # the identical MetaData object is also present on the
- # declarative base
Base.metadata.create_all(engine)
at :ref:`tutorial_declaring_mapped_classes`, we may also make
use of the :class:`_schema.Table` objects we created directly in the section
:ref:`tutorial_core_metadata` in conjunction with
-declarative mapped classes from a :func:`_orm.declarative_base` generated base
+declarative mapped classes from a Declarative Base
class.
-This form is called :ref:`hybrid table <orm_imperative_table_configuration>`,
-and it consists of assigning to the ``.__table__`` attribute directly, rather
-than having the declarative process generate it::
+This form is called
+:ref:`Declarative with Imperative Table <orm_imperative_table_configuration>`,
+and it consists of assigning a :class:`_schema.Table` object to the
+``.__table__`` attribute directly, rather than having the declarative process
+generate it from the ``.__tablename__`` attribute with separate
+:class:`_orm.Mapped` and/or :func:`_orm.mapped_column` directives. This is
+illustrated below by using our pre-existing ``user_table`` and
+``address_table`` :class:`_schema.Table` objects to map them to new classes
+(**note to readers running code**: these examples are for illustration only
+and should not be run)::
- mapper_registry = registry()
- Base = mapper_registry.generate_base()
+ class Base(DeclarativeBase):
+ pass
class User(Base):
__table__ = user_table
- addresses = relationship("Address", back_populates="user")
+ addresses: Mapped[List["Address"]] = relationship(back_populates="user")
def __repr__(self):
return f"User({self.name!r}, {self.fullname!r})"
class Address(Base):
__table__ = address_table
- user = relationship("User", back_populates="addresses")
+ user: Mapped["User"] = relationship(back_populates="addresses")
def __repr__(self):
return f"Address({self.email_address!r})"
-.. note:: The above example is an **alternative form** to the mapping that's
- first illustrated previously at :ref:`tutorial_declaring_mapped_classes`.
- This example is for illustrative purposes only, and is not part of this
- tutorial's "doctest" steps, and as such does not need to be run for readers
- who are executing code examples. The mapping here and the one at
- :ref:`tutorial_declaring_mapped_classes` produce equivalent mappings, but in
- general one would use only **one** of these two forms for particular mapped
- class.
-
-The above two classes are equivalent to those which we declared in the
-previous mapping example.
+The above two classes, ``User`` and ``Address``, are equivalent to those which
+we declared in the previous mapping example using ``__tablename__`` and
+:func:`_orm.mapped_column`.
The traditional "declarative base" approach using ``__tablename__`` to
automatically generate :class:`_schema.Table` objects remains the most popular
to itself, the latter of which is called a **self-referential** relationship.
To describe the basic idea of :func:`_orm.relationship`, first we'll review
-the mapping in short form, omitting the :class:`_schema.Column` mappings
+the mapping in short form, omitting the :func:`_orm.mapped_column` mappings
and other directives:
.. sourcecode:: python
+
+ from sqlalchemy.orm import Mapped
from sqlalchemy.orm import relationship
+
class User(Base):
__tablename__ = 'user_account'
- # ... Column mappings
+ # ... mapped_column() mappings
- addresses = relationship("Address", back_populates="user")
+ addresses: Mapped[list["Address"]] = relationship(back_populates="user")
class Address(Base):
__tablename__ = 'address'
- # ... Column mappings
+ # ... mapped_column() mappings
- user = relationship("User", back_populates="addresses")
+ user: Mapped["User"] = relationship(back_populates="addresses")
Above, the ``User`` class now has an attribute ``User.addresses`` and the
``Address`` class has an attribute ``Address.user``. The
-:func:`_orm.relationship` construct will be used to inspect the table
+:func:`_orm.relationship` construct, in conjunction with the
+:class:`_orm.Mapped` construct to indicate typing behavior, will be used to inspect the table
relationships between the :class:`_schema.Table` objects that are mapped to the
``User`` and ``Address`` classes. As the :class:`_schema.Table` object
representing the
.. sourcecode:: python
+ from sqlalchemy.orm import Mapped
from sqlalchemy.orm import relationship
+
class User(Base):
__tablename__ = 'user_account'
- addresses = relationship("Address", back_populates="user", lazy="selectin")
+ addresses: Mapped[list["Address"]] = relationship(back_populates="user", lazy="selectin")
Each loader strategy object adds some kind of information to the statement that
will be used later by the :class:`_orm.Session` when it is deciding how various
.. sourcecode:: python
+ from sqlalchemy.orm import Mapped
+ from sqlalchemy.orm import relationship
+
class User(Base):
__tablename__ = 'user_account'
- # ... Column mappings
+ # ... mapped_column() mappings
- addresses = relationship("Address", back_populates="user", lazy="raise_on_sql")
+ addresses: Mapped[list["Address"]] = relationship(back_populates="user", lazy="raise_on_sql")
class Address(Base):
__tablename__ = 'address'
- # ... Column mappings
+ # ... mapped_column() mappings
- user = relationship("User", back_populates="addresses", lazy="raise_on_sql")
+ user: Mapped["User"] = relationship(back_populates="addresses", lazy="raise_on_sql")
Using such a mapping, the application is blocked from lazy loading,
As is the case with :class:`.Mutable`, the user-defined composite class
subclasses :class:`.MutableComposite` as a mixin, and detects and delivers
change events to its parents via the :meth:`.MutableComposite.changed` method.
-In the case of a composite class, the detection is usually via the usage of
-Python descriptors (i.e. ``@property``), or alternatively via the special
-Python method ``__setattr__()``. Below we expand upon the ``Point`` class
-introduced in :ref:`mapper_composite` to subclass :class:`.MutableComposite`
-and to also route attribute set events via ``__setattr__`` to the
-:meth:`.MutableComposite.changed` method::
+In the case of a composite class, the detection is usually via the usage of the
+special Python method ``__setattr__()``. In the example below, we expand upon the ``Point``
+class introduced in :ref:`mapper_composite` to include
+:class:`.MutableComposite` in its bases and to route attribute set events via
+``__setattr__`` to the :meth:`.MutableComposite.changed` method::
+ import dataclasses
from sqlalchemy.ext.mutable import MutableComposite
+ @dataclasses.dataclass
class Point(MutableComposite):
- def __init__(self, x, y):
- self.x = x
- self.y = y
+ x: int
+ y: int
def __setattr__(self, key, value):
"Intercept set events"
# alert all parents to the change
self.changed()
- def __composite_values__(self):
- return self.x, self.y
-
- def __eq__(self, other):
- return isinstance(other, Point) and \
- other.x == self.x and \
- other.y == self.y
-
- def __ne__(self, other):
- return not self.__eq__(other)
The :class:`.MutableComposite` class makes use of class mapping events to
automatically establish listeners for any usage of :func:`_orm.composite` that
class, listeners are established which will route change events from ``Point``
objects to each of the ``Vertex.start`` and ``Vertex.end`` attributes::
- from sqlalchemy.orm import composite, mapper
- from sqlalchemy import Table, Column
-
- vertices = Table('vertices', metadata,
- Column('id', Integer, primary_key=True),
- Column('x1', Integer),
- Column('y1', Integer),
- Column('x2', Integer),
- Column('y2', Integer),
- )
+ from sqlalchemy.orm import DeclarativeBase, Mapped
+ from sqlalchemy.orm import composite, mapped_column
- class Vertex:
+ class Base(DeclarativeBase):
pass
- mapper(Vertex, vertices, properties={
- 'start': composite(Point, vertices.c.x1, vertices.c.y1),
- 'end': composite(Point, vertices.c.x2, vertices.c.y2)
- })
+
+ class Vertex(Base):
+ __tablename__ = "vertices"
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+
+ start: Mapped[Point] = composite(mapped_column("x1"), mapped_column("y1"))
+ end: Mapped[Point] = composite(mapped_column("x2"), mapped_column("y2"))
+
+ def __repr__(self):
+ return f"Vertex(start={self.start}, end={self.end})"
Any in-place changes to the ``Vertex.start`` or ``Vertex.end`` members
-will flag the attribute as "dirty" on the parent object::
+will flag the attribute as "dirty" on the parent object:
- >>> from sqlalchemy.orm import Session
+.. sourcecode:: python+sql
- >>> sess = Session()
+ >>> from sqlalchemy.orm import Session
+ >>> sess = Session(engine)
>>> v1 = Vertex(start=Point(3, 4), end=Point(12, 15))
>>> sess.add(v1)
- >>> sess.commit()
+ {sql}>>> sess.flush()
+ BEGIN (implicit)
+ INSERT INTO vertices (x1, y1, x2, y2) VALUES (?, ?, ?, ?)
+ [...] (3, 4, 12, 15)
- >>> v1.end.x = 8
+ {stop}>>> v1.end.x = 8
>>> assert v1 in sess.dirty
True
+ {sql}>>> sess.commit()
+ UPDATE vertices SET x2=? WHERE vertices.id = ?
+ [...] (8, 1)
+ COMMIT
Coercing Mutable Composites
---------------------------
to using a :func:`.validates` validation routine for all attributes which
make use of the custom composite type::
+ @dataclasses.dataclass
class Point(MutableComposite):
# other Point methods
# ...
Below we define both a ``__getstate__`` and a ``__setstate__`` that package up
the minimal form of our ``Point`` class::
+ @dataclasses.dataclass
class Point(MutableComposite):
# ...
pickling process of the parent's object-relational state so that the
:meth:`MutableBase._parents` collection is restored to all ``Point`` objects.
-"""
+""" # noqa: E501
+
from collections import defaultdict
import weakref
from .interfaces import UserDefinedOption as UserDefinedOption
from .loading import merge_frozen_result as merge_frozen_result
from .loading import merge_result as merge_result
-from .mapped_collection import attribute_mapped_collection
-from .mapped_collection import column_mapped_collection
-from .mapped_collection import mapped_collection
-from .mapped_collection import MappedCollection
+from .mapped_collection import (
+ attribute_mapped_collection as attribute_mapped_collection,
+)
+from .mapped_collection import (
+ column_mapped_collection as column_mapped_collection,
+)
+from .mapped_collection import mapped_collection as mapped_collection
+from .mapped_collection import MappedCollection as MappedCollection
from .mapper import configure_mappers as configure_mappers
from .mapper import Mapper as Mapper
from .mapper import reconstructor as reconstructor
Union[bool, Literal[SchemaConst.NULL_UNSPECIFIED]]
] = SchemaConst.NULL_UNSPECIFIED,
primary_key: Optional[bool] = False,
- deferred: bool = False,
+ deferred: Union[_NoArg, bool] = _NoArg.NO_ARG,
+ deferred_group: Optional[str] = None,
+ deferred_raiseload: bool = False,
name: Optional[str] = None,
type_: Optional[_TypeEngineArgument[Any]] = None,
autoincrement: Union[bool, Literal["auto", "ignore_fk"]] = "auto",
comment: Optional[str] = None,
**dialect_kwargs: Any,
) -> MappedColumn[Any]:
- r"""construct a new ORM-mapped :class:`_schema.Column` construct.
+ r"""declare a new ORM-mapped :class:`_schema.Column` construct
+ for use within :ref:`Declarative Table <orm_declarative_table>`
+ configuration.
The :func:`_orm.mapped_column` function provides an ORM-aware and
Python-typing-compatible construct which is used with
:ref:`declarative <orm_declarative_mapping>` mappings to indicate an
attribute that's mapped to a Core :class:`_schema.Column` object. It
provides the equivalent feature as mapping an attribute to a
- :class:`_schema.Column` object directly when using declarative.
+ :class:`_schema.Column` object directly when using Declarative,
+ specifically when using :ref:`Declarative Table <orm_declarative_table>`
+ configuration.
.. versionadded:: 2.0
:func:`_orm.mapped_column` is normally used with explicit typing along with
- the :class:`_orm.Mapped` mapped attribute type, where it can derive the SQL
- type and nullability for the column automatically, such as::
-
- from typing import Optional
-
- from sqlalchemy.orm import Mapped
- from sqlalchemy.orm import mapped_column
-
- class User(Base):
- __tablename__ = 'user'
-
- id: Mapped[int] = mapped_column(primary_key=True)
- name: Mapped[str] = mapped_column()
- options: Mapped[Optional[str]] = mapped_column()
-
- In the above example, the ``int`` and ``str`` types are inferred by the
- Declarative mapping system to indicate use of the :class:`_types.Integer`
- and :class:`_types.String` datatypes, and the presence of ``Optional`` or
- not indicates whether or not each non-primary-key column is to be
- ``nullable=True`` or ``nullable=False``.
-
- The above example, when interpreted within a Declarative class, will result
- in a table named ``"user"`` which is equivalent to the following::
-
- from sqlalchemy import Integer
- from sqlalchemy import String
- from sqlalchemy import Table
-
- Table(
- 'user',
- Base.metadata,
- Column("id", Integer, primary_key=True),
- Column("name", String, nullable=False),
- Column("options", String, nullable=True),
- )
-
- The :func:`_orm.mapped_column` construct accepts the same arguments as
- that of :class:`_schema.Column` directly, including optional "name"
- and "type" fields, so the above mapping can be stated more explicitly
- as::
+ the :class:`_orm.Mapped` annotation type, where it can derive the SQL
+ type and nullability for the column based on what's present within the
+ :class:`_orm.Mapped` annotation. It also may be used without annotations
+ as a drop-in replacement for how :class:`_schema.Column` is used in
+ Declarative mappings in SQLAlchemy 1.x style.
- from typing import Optional
+ For usage examples of :func:`_orm.mapped_column`, see the documentation
+ at :ref:`orm_declarative_table`.
- from sqlalchemy import Integer
- from sqlalchemy import String
- from sqlalchemy.orm import Mapped
- from sqlalchemy.orm import mapped_column
-
- class User(Base):
- __tablename__ = 'user'
+ .. seealso::
- id: Mapped[int] = mapped_column("id", Integer, primary_key=True)
- name: Mapped[str] = mapped_column("name", String, nullable=False)
- options: Mapped[Optional[str]] = mapped_column(
- "name", String, nullable=True
- )
+ :ref:`orm_declarative_table` - complete documentation
- Arguments passed to :func:`_orm.mapped_column` always supersede those which
- would be derived from the type annotation and/or attribute name. To state
- the above mapping with more specific datatypes for ``id`` and ``options``,
- and a different column name for ``name``, looks like::
-
- from sqlalchemy import BigInteger
-
- class User(Base):
- __tablename__ = 'user'
-
- id: Mapped[int] = mapped_column("id", BigInteger, primary_key=True)
- name: Mapped[str] = mapped_column("user_name")
- options: Mapped[Optional[str]] = mapped_column(String(50))
-
- Where again, datatypes and nullable parameters that can be automatically
- derived may be omitted.
-
- The datatypes passed to :class:`_orm.Mapped` are mapped to SQL
- :class:`_types.TypeEngine` types with the following default mapping::
-
- _type_map = {
- int: Integer(),
- float: Float(),
- bool: Boolean(),
- decimal.Decimal: Numeric(),
- dt.date: Date(),
- dt.datetime: DateTime(),
- dt.time: Time(),
- dt.timedelta: Interval(),
- util.NoneType: NULLTYPE,
- bytes: LargeBinary(),
- str: String(),
- }
-
- The above mapping may be expanded to include any combination of Python
- datatypes to SQL types by using the
- :paramref:`_orm.registry.type_annotation_map` parameter to
- :class:`_orm.registry`, or as the attribute ``type_annotation_map`` upon
- the :class:`_orm.DeclarativeBase` base class.
-
- Finally, :func:`_orm.mapped_column` is implicitly used by the Declarative
- mapping system for any :class:`_orm.Mapped` annotation that has no
- attribute value set up. This is much in the way that Python dataclasses
- allow the ``field()`` construct to be optional, only needed when additional
- parameters should be associated with the field. Using this functionality,
- our original mapping can be stated even more succinctly as::
-
- from typing import Optional
-
- from sqlalchemy.orm import Mapped
- from sqlalchemy.orm import mapped_column
-
- class User(Base):
- __tablename__ = 'user'
-
- id: Mapped[int] = mapped_column(primary_key=True)
- name: Mapped[str]
- options: Mapped[Optional[str]]
-
- Above, the ``name`` and ``options`` columns will be evaluated as
- ``Column("name", String, nullable=False)`` and
- ``Column("options", String, nullable=True)``, respectively.
+ :ref:`whatsnew_20_orm_declarative_typing` - migration notes for
+ Declarative mappings using 1.x style mappings
:param __name: String name to give to the :class:`_schema.Column`. This
is an optional, positional only argument that if present must be the
ORM declarative process, and is not part of the :class:`_schema.Column`
itself; instead, it indicates that this column should be "deferred" for
loading as though mapped by :func:`_orm.deferred`.
- :param default: This keyword argument, if present, is passed along to the
- :class:`_schema.Column` constructor as the value of the
- :paramref:`_schema.Column.default` parameter. However, as
- :paramref:`_orm.mapped_column.default` is also consumed as a dataclasses
- directive, the :paramref:`_orm.mapped_column.insert_default` parameter
- should be used instead in a dataclasses context.
+
+ .. seealso::
+
+ :ref:`deferred`
+
+ :param deferred_group: Implies :paramref:`_orm.mapped_column.deferred`
+ to ``True``, and set the :paramref:`_orm.deferred.group` parameter.
+ :param deferred_raiseload: Implies :paramref:`_orm.mapped_column.deferred`
+ to ``True``, and set the :paramref:`_orm.deferred.raiseload` parameter.
+
+ :param default: Passed directly to the
+ :paramref:`_schema.Column.default` parameter if the
+ :paramref:`_orm.mapped_column.insert_default` parameter is not present.
+ Additionally, when used with :ref:`orm_declarative_native_dataclasses`,
+ indicates a default Python value that should be applied to the keyword
+ constructor within the generated ``__init__()`` method.
+
+ Note that in the case of dataclass generation when
+ :paramref:`_orm.mapped_column.insert_default` is not present, this means
+ the :paramref:`_orm.mapped_column.default` value is used in **two**
+ places, both the ``__init__()`` method as well as the
+ :paramref:`_schema.Column.default` parameter. While this behavior may
+ change in a future release, for the moment this tends to "work out"; a
+ default of ``None`` will mean that the :class:`_schema.Column` gets no
+ default generator, whereas a default that refers to a non-``None`` Python
+ or SQL expression value will be assigned up front on the object when
+ ``__init__()`` is called, which is the same value that the Core
+ :class:`_sql.Insert` construct would use in any case, leading to the same
+ end result.
+
:param insert_default: Passed directly to the
:paramref:`_schema.Column.default` parameter; will supersede the value
of :paramref:`_orm.mapped_column.default` when present, however
:paramref:`_orm.mapped_column.default` will always apply to the
constructor default for a dataclasses mapping.
+
+ :param init: Specific to :ref:`orm_declarative_native_dataclasses`,
+ specifies if the mapped attribute should be part of the ``__init__()``
+ method as generated by the dataclass process.
+ :param repr: Specific to :ref:`orm_declarative_native_dataclasses`,
+ specifies if the mapped attribute should be part of the ``__repr__()``
+ method as generated by the dataclass process.
+ :param default_factory: Specific to
+ :ref:`orm_declarative_native_dataclasses`,
+ specifies a default-value generation function that will take place
+ as part of the ``__init__()``
+ method as generated by the dataclass process.
+
:param \**kw: All remaining keyword argments are passed through to the
constructor for the :class:`_schema.Column`.
comment=comment,
system=system,
deferred=deferred,
+ deferred_group=deferred_group,
+ deferred_raiseload=deferred_raiseload,
**dialect_kwargs,
)
:param info: Optional data dictionary which will be populated into the
:attr:`.MapperProperty.info` attribute of this object.
+ :param init: Specific to :ref:`orm_declarative_native_dataclasses`,
+ specifies if the mapped attribute should be part of the ``__init__()``
+ method as generated by the dataclass process.
+ :param repr: Specific to :ref:`orm_declarative_native_dataclasses`,
+ specifies if the mapped attribute should be part of the ``__repr__()``
+ method as generated by the dataclass process.
+ :param default_factory: Specific to
+ :ref:`orm_declarative_native_dataclasses`,
+ specifies a default-value generation function that will take place
+ as part of the ``__init__()``
+ method as generated by the dataclass process.
+
"""
if __kw:
raise _no_kw()
"""Provide a relationship between two mapped classes.
This corresponds to a parent-child or associative table relationship.
- The constructed class is an instance of
- :class:`.Relationship`.
-
- A typical :func:`_orm.relationship`, used in a classical mapping::
-
- mapper(Parent, properties={
- 'children': relationship(Child)
- })
-
- Some arguments accepted by :func:`_orm.relationship`
- optionally accept a
- callable function, which when called produces the desired value.
- The callable is invoked by the parent :class:`_orm.Mapper` at "mapper
- initialization" time, which happens only when mappers are first used,
- and is assumed to be after all mappings have been constructed. This
- can be used to resolve order-of-declaration and other dependency
- issues, such as if ``Child`` is declared below ``Parent`` in the same
- file::
-
- mapper(Parent, properties={
- "children":relationship(lambda: Child,
- order_by=lambda: Child.id)
- })
-
- When using the :ref:`declarative_toplevel` extension, the Declarative
- initializer allows string arguments to be passed to
- :func:`_orm.relationship`. These string arguments are converted into
- callables that evaluate the string as Python code, using the
- Declarative class-registry as a namespace. This allows the lookup of
- related classes to be automatic via their string name, and removes the
- need for related classes to be imported into the local module space
- before the dependent classes have been declared. It is still required
- that the modules in which these related classes appear are imported
- anywhere in the application at some point before the related mappings
- are actually used, else a lookup error will be raised when the
- :func:`_orm.relationship`
- attempts to resolve the string reference to the
- related class. An example of a string- resolved class is as
- follows::
-
- from sqlalchemy.ext.declarative import declarative_base
-
- Base = declarative_base()
-
- class Parent(Base):
- __tablename__ = 'parent'
- id = Column(Integer, primary_key=True)
- children = relationship("Child", order_by="Child.id")
+ The constructed class is an instance of :class:`.Relationship`.
.. seealso::
- :ref:`relationship_config_toplevel` - Full introductory and
- reference documentation for :func:`_orm.relationship`.
+ :ref:`tutorial_orm_related_objects` - tutorial introduction
+ to :func:`_orm.relationship` in the :ref:`unified_tutorial`
- :ref:`tutorial_orm_related_objects` - ORM tutorial introduction.
+ :ref:`relationship_config_toplevel` - narrative documentation
:param argument:
- A mapped class, or actual :class:`_orm.Mapper` instance,
- representing
- the target of the relationship.
+ This parameter refers to the class that is to be related. It
+ accepts several forms, including a direct reference to the target
+ class itself, the :class:`_orm.Mapper` instance for the target class,
+ a Python callable / lambda that will return a reference to the
+ class or :class:`_orm.Mapper` when called, and finally a string
+ name for the class, which will be resolved from the
+ :class:`_orm.registry` in use in order to locate the class, e.g.::
- :paramref:`_orm.relationship.argument`
- may also be passed as a callable
- function which is evaluated at mapper initialization time, and may
- be passed as a string name when using Declarative.
+ class SomeClass(Base):
+ # ...
- .. warning:: Prior to SQLAlchemy 1.3.16, this value is interpreted
- using Python's ``eval()`` function.
- **DO NOT PASS UNTRUSTED INPUT TO THIS STRING**.
- See :ref:`declarative_relationship_eval` for details on
- declarative evaluation of :func:`_orm.relationship` arguments.
+ related = relationship("RelatedClass")
+
+ The :paramref:`_orm.relationship.argument` may also be omitted from the
+ :func:`_orm.relationship` construct entirely, and instead placed inside
+ a :class:`_orm.Mapped` annotation on the left side, which should
+ include a Python collection type if the relationship is expected
+ to be a collection, such as::
+
+ class SomeClass(Base):
+ # ...
- .. versionchanged 1.3.16::
+ related_items: Mapped[List["RelatedItem"]] = relationship()
- The string evaluation of the main "argument" no longer accepts an
- open ended Python expression, instead only accepting a string
- class name or dotted package-qualified name.
+ Or for a many-to-one or one-to-one relationship::
+
+ class SomeClass(Base):
+ # ...
+
+ related_item: Mapped["RelatedItem"] = relationship()
.. seealso::
- :ref:`declarative_configuring_relationships` - further detail
+ :ref:`orm_declarative_properties` - further detail
on relationship configuration when using Declarative.
:param secondary:
A boolean that indicates if this property should be loaded as a
list or a scalar. In most cases, this value is determined
automatically by :func:`_orm.relationship` at mapper configuration
- time, based on the type and direction
+ time. When using explicit :class:`_orm.Mapped` annotations,
+ :paramref:`_orm.relationship.uselist` may be derived from the
+ whether or not the annotation within :class:`_orm.Mapped` contains
+ a collection class.
+ Otherwise, :paramref:`_orm.relationship.uselist` may be derived from
+ the type and direction
of the relationship - one to many forms a list, many to one
forms a scalar, many to many is a list. If a scalar is desired
where normally a list would be present, such as a bi-directional
- one-to-one relationship, set :paramref:`_orm.relationship.uselist`
- to
- False.
+ one-to-one relationship, use an appropriate :class:`_orm.Mapped`
+ annotation or set :paramref:`_orm.relationship.uselist` to False.
The :paramref:`_orm.relationship.uselist`
flag is also available on an
.. seealso::
:ref:`relationships_one_to_one` - Introduction to the "one to
- one" relationship pattern, which is typically when the
- :paramref:`_orm.relationship.uselist` flag is needed.
+ one" relationship pattern, which is typically when an alternate
+ setting for :paramref:`_orm.relationship.uselist` is involved.
:param viewonly=False:
When set to ``True``, the relationship is used only for loading
.. versionadded:: 1.3
+ :param init: Specific to :ref:`orm_declarative_native_dataclasses`,
+ specifies if the mapped attribute should be part of the ``__init__()``
+ method as generated by the dataclass process.
+ :param repr: Specific to :ref:`orm_declarative_native_dataclasses`,
+ specifies if the mapped attribute should be part of the ``__repr__()``
+ method as generated by the dataclass process.
+ :param default_factory: Specific to
+ :ref:`orm_declarative_native_dataclasses`,
+ specifies a default-value generation function that will take place
+ as part of the ``__init__()``
+ method as generated by the dataclass process.
+
+
"""
return Relationship(
r"""Indicate a column-based mapped attribute that by default will
not load unless accessed.
+ When using :func:`_orm.mapped_column`, the same functionality as
+ that of :func:`_orm.deferred` construct is provided by using the
+ :paramref:`_orm.mapped_column.deferred` parameter.
+
:param \*columns: columns to be mapped. This is typically a single
:class:`_schema.Column` object,
however a collection is supported in order
.. versionadded:: 1.4
- .. seealso::
-
- :ref:`deferred_raiseload`
Additional arguments are the same as that of :func:`_orm.column_property`.
:ref:`deferred`
+ :ref:`deferred_raiseload`
+
"""
return ColumnProperty(
column,
checkers such as pylance and mypy so that ORM-mapped attributes
are correctly typed.
+ The most prominent use of :class:`_orm.Mapped` is in
+ the :ref:`Declarative Mapping <orm_explicit_declarative_base>` form
+ of :class:`_orm.Mapper` configuration, where used explicitly it drives
+ the configuration of ORM attributes such as :func:`_orm.mapped_class`
+ and :func:`_orm.relationship`.
+
+ .. seealso::
+
+ :ref:`orm_explicit_declarative_base`
+
+ :ref:`orm_declarative_table`
+
.. tip::
The :class:`_orm.Mapped` class represents attributes that are handled
from typing import ClassVar
from typing import Dict
from typing import FrozenSet
+from typing import Generic
from typing import Iterator
from typing import Mapping
from typing import Optional
from .interfaces import MapperProperty
from .state import InstanceState # noqa
from ..sql._typing import _TypeEngineArgument
+
_T = TypeVar("_T", bound=Any)
+_TT = TypeVar("_TT", bound=Any)
+
# it's not clear how to have Annotated, Union objects etc. as keys here
# from a typing perspective so just leave it open ended for now
_TypeAnnotationMapType = Mapping[Any, "_TypeEngineArgument[Any]"]
return decorate
-class declared_attr(interfaces._MappedAttribute[_T]):
- """Mark a class-level method as representing the definition of
- a mapped property or special declarative member name.
-
- :class:`_orm.declared_attr` is typically applied as a decorator to a class
- level method, turning the attribute into a scalar-like property that can be
- invoked from the uninstantiated class. The Declarative mapping process
- looks for these :class:`_orm.declared_attr` callables as it scans classes,
- and assumes any attribute marked with :class:`_orm.declared_attr` will be a
- callable that will produce an object specific to the Declarative mapping or
- table configuration.
-
- :class:`_orm.declared_attr` is usually applicable to mixins, to define
- relationships that are to be applied to different implementors of the
- class. It is also used to define :class:`_schema.Column` objects that
- include the :class:`_schema.ForeignKey` construct, as these cannot be
- easily reused across different mappings. The example below illustrates
- both::
-
- class ProvidesUser:
- "A mixin that adds a 'user' relationship to classes."
-
- @declared_attr
- def user_id(self):
- return Column(ForeignKey("user_account.id"))
-
- @declared_attr
- def user(self):
- return relationship("User")
-
- :class:`_orm.declared_attr` can also be applied to mapped classes, such as
- to provide a "polymorphic" scheme for inheritance::
-
- class Employee(Base):
- id = Column(Integer, primary_key=True)
- type = Column(String(50), nullable=False)
-
- @declared_attr
- def __tablename__(cls):
- return cls.__name__.lower()
-
- @declared_attr
- def __mapper_args__(cls):
- if cls.__name__ == 'Employee':
- return {
- "polymorphic_on":cls.type,
- "polymorphic_identity":"Employee"
- }
- else:
- return {"polymorphic_identity":cls.__name__}
-
- To use :class:`_orm.declared_attr` inside of a Python dataclass
- as discussed at :ref:`orm_declarative_dataclasses_declarative_table`,
- it may be placed directly inside the field metadata using a lambda::
-
- @dataclass
- class AddressMixin:
- __sa_dataclass_metadata_key__ = "sa"
-
- user_id: int = field(
- init=False, metadata={"sa": declared_attr(lambda: Column(ForeignKey("user.id")))}
- )
- user: User = field(
- init=False, metadata={"sa": declared_attr(lambda: relationship(User))}
- )
-
- :class:`_orm.declared_attr` also may be omitted from this form using a
- lambda directly, as in::
-
- user: User = field(
- init=False, metadata={"sa": lambda: relationship(User)}
- )
-
- .. seealso::
-
- :ref:`orm_mixins_toplevel` - illustrates how to use Declarative Mixins
- which is the primary use case for :class:`_orm.declared_attr`
-
- :ref:`orm_declarative_dataclasses_mixin` - illustrates special forms
- for use with Python dataclasses
-
- """ # noqa: E501
-
- if typing.TYPE_CHECKING:
-
- def __set__(self, instance: Any, value: Any) -> None:
- ...
-
- def __delete__(self, instance: Any) -> None:
- ...
-
+class _declared_attr_common:
def __init__(
self,
- fn: _DeclaredAttrDecorated[_T],
+ fn: Callable[..., Any],
cascading: bool = False,
):
# suppport
def _collect_return_annotation(self) -> Optional[Type[Any]]:
return util.get_annotations(self.fget).get("return")
- # this is the Mapped[] API where at class descriptor get time we want
- # the type checker to see InstrumentedAttribute[_T]. However the
- # callable function prior to mapping in fact calls the given
- # declarative function that does not return InstrumentedAttribute
- @overload
- def __get__(self, instance: None, owner: Any) -> InstrumentedAttribute[_T]:
- ...
-
- @overload
- def __get__(self, instance: object, owner: Any) -> _T:
- ...
-
- def __get__(
- self, instance: Optional[object], owner: Any
- ) -> Union[InstrumentedAttribute[_T], _T]:
+ def __get__(self, instance: Optional[object], owner: Any) -> Any:
# the declared_attr needs to make use of a cache that exists
# for the span of the declarative scan_attributes() phase.
# to achieve this we look at the class manager that's configured.
reg[self] = obj = self.fget(cls)
return obj # type: ignore
- @hybridmethod
- def _stateful(cls, **kw: Any) -> _stateful_declared_attr[_T]:
- return _stateful_declared_attr(**kw)
- @hybridproperty
- def cascading(cls) -> _stateful_declared_attr[_T]:
- """Mark a :class:`.declared_attr` as cascading.
-
- This is a special-use modifier which indicates that a column
- or MapperProperty-based declared attribute should be configured
- distinctly per mapped subclass, within a mapped-inheritance scenario.
-
- .. warning::
-
- The :attr:`.declared_attr.cascading` modifier has several
- limitations:
-
- * The flag **only** applies to the use of :class:`.declared_attr`
- on declarative mixin classes and ``__abstract__`` classes; it
- currently has no effect when used on a mapped class directly.
-
- * The flag **only** applies to normally-named attributes, e.g.
- not any special underscore attributes such as ``__tablename__``.
- On these attributes it has **no** effect.
-
- * The flag currently **does not allow further overrides** down
- the class hierarchy; if a subclass tries to override the
- attribute, a warning is emitted and the overridden attribute
- is skipped. This is a limitation that it is hoped will be
- resolved at some point.
-
- Below, both MyClass as well as MySubClass will have a distinct
- ``id`` Column object established::
-
- class HasIdMixin:
- @declared_attr.cascading
- def id(cls):
- if has_inherited_table(cls):
- return Column(
- ForeignKey('myclass.id'), primary_key=True
- )
- else:
- return Column(Integer, primary_key=True)
-
- class MyClass(HasIdMixin, Base):
- __tablename__ = 'myclass'
- # ...
+class _declared_directive(_declared_attr_common, Generic[_T]):
+ # see mapping_api.rst for docstring
- class MySubClass(MyClass):
- ""
- # ...
+ if typing.TYPE_CHECKING:
- The behavior of the above configuration is that ``MySubClass``
- will refer to both its own ``id`` column as well as that of
- ``MyClass`` underneath the attribute named ``some_id``.
+ def __init__(
+ self,
+ fn: Callable[..., _T],
+ cascading: bool = False,
+ ):
+ ...
- .. seealso::
+ def __get__(self, instance: Optional[object], owner: Any) -> _T:
+ ...
+
+ def __set__(self, instance: Any, value: Any) -> None:
+ ...
- :ref:`declarative_inheritance`
+ def __delete__(self, instance: Any) -> None:
+ ...
- :ref:`mixin_inheritance_columns`
+ def __call__(self, fn: Callable[..., _TT]) -> _declared_directive[_TT]:
+ # extensive fooling of mypy underway...
+ ...
- """
+class declared_attr(interfaces._MappedAttribute[_T], _declared_attr_common):
+ """Mark a class-level method as representing the definition of
+ a mapped property or Declarative directive.
+
+ :class:`_orm.declared_attr` is typically applied as a decorator to a class
+ level method, turning the attribute into a scalar-like property that can be
+ invoked from the uninstantiated class. The Declarative mapping process
+ looks for these :class:`_orm.declared_attr` callables as it scans classes,
+ and assumes any attribute marked with :class:`_orm.declared_attr` will be a
+ callable that will produce an object specific to the Declarative mapping or
+ table configuration.
+
+ :class:`_orm.declared_attr` is usually applicable to
+ :ref:`mixins <orm_mixins_toplevel>`, to define relationships that are to be
+ applied to different implementors of the class. It may also be used to
+ define dynamically generated column expressions and other Declarative
+ attributes.
+
+ Example::
+
+ class ProvidesUserMixin:
+ "A mixin that adds a 'user' relationship to classes."
+
+ user_id: Mapped[int] = mapped_column(ForeignKey("user_table.id"))
+
+ @declared_attr
+ def user(cls) -> Mapped["User"]:
+ return relationship("User")
+
+ When used with Declarative directives such as ``__tablename__``, the
+ :meth:`_orm.declared_attr.directive` modifier may be used which indicates
+ to :pep:`484` typing tools that the given method is not dealing with
+ :class:`_orm.Mapped` attributes::
+
+ class CreateTableName:
+ @declared_attr.directive
+ def __tablename__(cls) -> str:
+ return cls.__name__.lower()
+
+ :class:`_orm.declared_attr` can also be applied directly to mapped
+ classes, to allow for attributes that dynamically configure themselves
+ on subclasses when using mapped inheritance schemes. Below
+ illustrates :class:`_orm.declared_attr` to create a dynamic scheme
+ for generating the :paramref:`_orm.Mapper.polymorphic_identity` parameter
+ for subclasses::
+
+ class Employee(Base):
+ __tablename__ = 'employee'
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+ type: Mapped[str] = mapped_column(String(50))
+
+ @declared_attr.directive
+ def __mapper_args__(cls) -> dict[str, Any]:
+ if cls.__name__ == 'Employee':
+ return {
+ "polymorphic_on":cls.type,
+ "polymorphic_identity":"Employee"
+ }
+ else:
+ return {"polymorphic_identity":cls.__name__}
+
+ class Engineer(Employee):
+ pass
+
+ :class:`_orm.declared_attr` supports decorating functions that are
+ explicitly decorated with ``@classmethod``. This is never necessary from a
+ runtime perspective, however may be needed in order to support :pep:`484`
+ typing tools that don't otherwise recognize the decorated function as
+ having class-level behaviors for the ``cls`` parameter::
+
+ class SomethingMixin:
+ x: Mapped[int]
+ y: Mapped[int]
+
+ @declared_attr
+ @classmethod
+ def x_plus_y(cls) -> Mapped[int]:
+ return column_property(cls.x + cls.y)
+
+ .. versionadded:: 2.0 - :class:`_orm.declared_attr` can accommodate a
+ function decorated with ``@classmethod`` to help with :pep:`484`
+ integration where needed.
+
+
+ .. seealso::
+
+ :ref:`orm_mixins_toplevel` - Declarative Mixin documentation with
+ background on use patterns for :class:`_orm.declared_attr`.
+
+ """ # noqa: E501
+
+ if typing.TYPE_CHECKING:
+
+ def __init__(
+ self,
+ fn: _DeclaredAttrDecorated[_T],
+ cascading: bool = False,
+ ):
+ ...
+
+ def __set__(self, instance: Any, value: Any) -> None:
+ ...
+
+ def __delete__(self, instance: Any) -> None:
+ ...
+
+ # this is the Mapped[] API where at class descriptor get time we want
+ # the type checker to see InstrumentedAttribute[_T]. However the
+ # callable function prior to mapping in fact calls the given
+ # declarative function that does not return InstrumentedAttribute
+ @overload
+ def __get__(
+ self, instance: None, owner: Any
+ ) -> InstrumentedAttribute[_T]:
+ ...
+
+ @overload
+ def __get__(self, instance: object, owner: Any) -> _T:
+ ...
+
+ def __get__(
+ self, instance: Optional[object], owner: Any
+ ) -> Union[InstrumentedAttribute[_T], _T]:
+ ...
+
+ @hybridmethod
+ def _stateful(cls, **kw: Any) -> _stateful_declared_attr[_T]:
+ return _stateful_declared_attr(**kw)
+
+ @hybridproperty
+ def directive(cls) -> _declared_directive[Any]:
+ # see mapping_api.rst for docstring
+ return _declared_directive # type: ignore
+
+ @hybridproperty
+ def cascading(cls) -> _stateful_declared_attr[_T]:
+ # see mapping_api.rst for docstring
return cls._stateful(cascading=True)
cls.metadata = cls.registry.metadata # type: ignore
-class DeclarativeBaseNoMeta(inspection.Inspectable[Mapper[Any]]):
- """Same as :class:`_orm.DeclarativeBase`, but does not use a metaclass
- to intercept new attributes.
-
- The :class:`_orm.DeclarativeBaseNoMeta` base may be used when use of
- custom metaclasses is desirable.
-
- .. versionadded:: 2.0
-
-
- """
-
- registry: ClassVar[_RegistryType]
- _sa_registry: ClassVar[_RegistryType]
- metadata: ClassVar[MetaData]
- __mapper__: ClassVar[Mapper[Any]]
- __table__: Optional[FromClause]
-
- if typing.TYPE_CHECKING:
-
- def __init__(self, **kw: Any):
- ...
-
- def __init_subclass__(cls) -> None:
- if DeclarativeBaseNoMeta in cls.__bases__:
- _setup_declarative_base(cls)
- else:
- cls._sa_registry.map_declaratively(cls)
-
-
class MappedAsDataclass(metaclass=DCTransformDeclarative):
"""Mixin class to indicate when mapping this class, also convert it to be
a dataclass.
.. seealso::
- :meth:`_orm.registry.mapped_as_dataclass`
+ :ref:`orm_declarative_native_dataclasses` - complete background
+ on SQLAlchemy native dataclass mapping
.. versionadded:: 2.0
+
"""
def __init_subclass__(
):
"""Base class used for declarative class definitions.
-
The :class:`_orm.DeclarativeBase` allows for the creation of new
declarative bases in such a way that is compatible with type checkers::
:paramref:`_orm.registry.type_annotation_map`.
:param registry: supply a pre-existing :class:`_orm.registry` directly.
- .. versionadded:: 2.0
+ .. versionadded:: 2.0 Added :class:`.DeclarativeBase`, so that declarative
+ base classes may be constructed in such a way that is also recognized
+ by :pep:`484` type checkers. As a result, :class:`.DeclarativeBase`
+ and other subclassing-oriented APIs should be seen as
+ superseding previous "class returned by a function" APIs, namely
+ :func:`_orm.declarative_base` and :meth:`_orm.registry.generate_base`,
+ where the base class returned cannot be recognized by type checkers
+ without using plugins.
"""
_sa_registry: ClassVar[_RegistryType]
metadata: ClassVar[MetaData]
+ __name__: ClassVar[str]
__mapper__: ClassVar[Mapper[Any]]
__table__: ClassVar[Optional[FromClause]]
- __tablename__: ClassVar[Optional[str]]
+ __tablename__: ClassVar[Any]
def __init__(self, **kw: Any):
...
_as_declarative(cls._sa_registry, cls, cls.__dict__)
+class DeclarativeBaseNoMeta(inspection.Inspectable[Mapper[Any]]):
+ """Same as :class:`_orm.DeclarativeBase`, but does not use a metaclass
+ to intercept new attributes.
+
+ The :class:`_orm.DeclarativeBaseNoMeta` base may be used when use of
+ custom metaclasses is desirable.
+
+ .. versionadded:: 2.0
+
+
+ """
+
+ if typing.TYPE_CHECKING:
+ registry: ClassVar[_RegistryType]
+ _sa_registry: ClassVar[_RegistryType]
+ metadata: ClassVar[MetaData]
+
+ __name__: ClassVar[str]
+ __mapper__: ClassVar[Mapper[Any]]
+ __table__: ClassVar[Optional[FromClause]]
+
+ __tablename__: ClassVar[Any]
+
+ def __init__(self, **kw: Any):
+ ...
+
+ def __init_subclass__(cls) -> None:
+ if DeclarativeBaseNoMeta in cls.__bases__:
+ _setup_declarative_base(cls)
+ else:
+ cls._sa_registry.map_declaratively(cls)
+
+
def add_mapped_attribute(
target: Type[_O], key: str, attr: MapperProperty[Any]
) -> None:
information provided declaratively in the class and any subclasses
of the class.
+ .. versionchanged:: 2.0 Note that the :func:`_orm.declarative_base`
+ function is superseded by the new :class:`_orm.DeclarativeBase` class,
+ which generates a new "base" class using subclassing, rather than
+ return value of a function. This allows an approach that is compatible
+ with :pep:`484` typing tools.
+
The :func:`_orm.declarative_base` function is a shorthand version
of using the :meth:`_orm.registry.generate_base`
method. That is, the following::
to produce column types based on annotations within the
:class:`_orm.Mapped` type.
+
.. versionadded:: 2.0
+ .. seealso::
+
+ :ref:`orm_declarative_mapped_column_type_map`
+
:param metaclass:
Defaults to :class:`.DeclarativeMeta`. A metaclass or __metaclass__
compatible callable to use as the meta type of the generated
.. versionadded:: 2.0
+ .. seealso::
+
+ :ref:`orm_declarative_mapped_column_type_map`
+
+
"""
lcl_metadata = metadata or MetaData()
__init__ = mapper_registry.constructor
+ .. versionchanged:: 2.0 Note that the
+ :meth:`_orm.registry.generate_base` method is superseded by the new
+ :class:`_orm.DeclarativeBase` class, which generates a new "base"
+ class using subclassing, rather than return value of a function.
+ This allows an approach that is compatible with :pep:`484` typing
+ tools.
+
The :meth:`_orm.registry.generate_base` method provides the
implementation for the :func:`_orm.declarative_base` function, which
creates the :class:`_orm.registry` and base class all at once.
.. seealso::
- :meth:`_orm.registry.mapped`
+ :ref:`orm_declarative_native_dataclasses` - complete background
+ on SQLAlchemy native dataclass mapping
+
.. versionadded:: 2.0
class MappedCollection(Dict[_KT, _VT]):
- """A basic dictionary-based collection class.
+ """Base for ORM mapped dictionary classes.
+
+ Extends the ``dict`` type with additional methods needed by SQLAlchemy ORM
+ collection classes. Use of :class:`_orm.MappedCollection` is most directly
+ by using the :func:`.attribute_mapped_collection` or
+ :func:`.column_mapped_collection` class factories.
+ :class:`_orm.MappedCollection` may also serve as the base for user-defined
+ custom dictionary classes.
+
+ .. seealso::
+
+ :ref:`orm_dictionary_collection`
+
+ :ref:`orm_custom_collection`
- Extends dict with the minimal bag semantics that collection
- classes require. ``set`` and ``remove`` are implemented in terms
- of a keying function: any callable that takes an object and
- returns an object for use as a dictionary key.
"""
:param exclude_properties: A list or set of string column names to
be excluded from mapping.
- See :ref:`include_exclude_cols` for an example.
+ .. seealso::
+
+ :ref:`include_exclude_cols`
:param include_properties: An inclusive list or set of string column
names to map.
- See :ref:`include_exclude_cols` for an example.
+ .. seealso::
+
+ :ref:`include_exclude_cols`
:param inherits: A mapped class or the corresponding
:class:`_orm.Mapper`
)
__mapper_args__ = {
- "polymorphic_on":employee_type,
- "polymorphic_identity":"employee"
+ "polymorphic_on": "employee_type",
+ "polymorphic_identity": "employee"
}
When setting ``polymorphic_on`` to reference an
"foreign_keys",
"_has_nullable",
"deferred",
+ "deferred_group",
+ "deferred_raiseload",
"_attribute_options",
"_has_dataclass_arguments",
)
deferred: bool
+ deferred_raiseload: bool
+ deferred_group: Optional[str]
+
column: Column[_T]
foreign_keys: Optional[Set[ForeignKey]]
_attribute_options: _AttributeOptions
kw["default"] = kw.pop("insert_default", None)
- self.deferred = kw.pop("deferred", False)
+ self.deferred_group = kw.pop("deferred_group", None)
+ self.deferred_raiseload = kw.pop("deferred_raiseload", None)
+ self.deferred = kw.pop("deferred", _NoArg.NO_ARG)
+ if self.deferred is _NoArg.NO_ARG:
+ self.deferred = bool(
+ self.deferred_group or self.deferred_raiseload
+ )
+
self.column = cast("Column[_T]", Column(*arg, **kw))
self.foreign_keys = self.column.foreign_keys
self._has_nullable = "nullable" in kw and kw.get("nullable") not in (
return ColumnProperty(
self.column,
deferred=True,
+ group=self.deferred_group,
+ raiseload=self.deferred_raiseload,
attribute_options=self._attribute_options,
)
else:
lambda: util.py38, "Python 3.8 or above required"
)
+ @property
+ def python39(self):
+ return exclusions.only_if(
+ lambda: util.py39, "Python 3.9 or above required"
+ )
+
@property
def cpython(self):
return exclusions.only_if(
class DocTest(fixtures.TestBase):
+ __requires__ = ("python39",)
+
def _setup_logger(self):
rootlogger = logging.getLogger("sqlalchemy.engine.Engine")
class User(HasRelatedDataMixin, Base):
- __tablename__ = "user"
+ @declared_attr.directive
+ def __tablename__(cls) -> str:
+ return "user"
+
+ @declared_attr.directive
+ def __mapper_args__(cls) -> typing.Dict[str, typing.Any]:
+ return {}
+
id = mapped_column(Integer, primary_key=True)
-u1 = User()
+class Foo(Base):
+ __tablename__: typing.ClassVar[str] = "foo"
+ id = mapped_column(Integer, primary_key=True)
-u1.related_data
+u1 = User()
if typing.TYPE_CHECKING:
+ # EXPECTED_TYPE: str
+ reveal_type(User.__tablename__)
+
+ # EXPECTED_TYPE: str
+ reveal_type(Foo.__tablename__)
+
# EXPECTED_TYPE: str
reveal_type(u1.related_data)
async_session = AsyncSession(async_connection)
+# (variable) users1: Sequence[User]
+users1 = session.scalars(select(User)).all()
+
+# (variable) user: User
+user = session.query(User).one()
+
+
+user_iter = iter(session.scalars(select(User)))
+
# EXPECTED_RE_TYPE: sqlalchemy..*AsyncSession\*?
reveal_type(async_session)
def test_can_we_access_the_mixin_straight_special_names(self):
class Mixin:
- @declared_attr
+ @declared_attr.directive
def __table_args__(cls):
return (1, 2, 3)
- @declared_attr
+ @declared_attr.directive
def __arbitrary__(cls):
return (4, 5, 6)
counter = mock.Mock()
class Mixin:
- @declared_attr
+ @declared_attr.directive
def __tablename__(cls):
counter(cls)
return "foo"
class Foo(Mixin, Base):
id = Column(Integer, primary_key=True)
- @declared_attr
+ @declared_attr.directive
def x(cls):
cls.__tablename__
- @declared_attr
+ @declared_attr.directive
def y(cls):
cls.__tablename__
],
)
+ @testing.combinations(True, False, None, "deferred_parameter")
+ def test_group_defer_newstyle(self, deferred_parameter):
+ class Base(DeclarativeBase):
+ pass
+
+ class Order(Base):
+ __tablename__ = "orders"
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+ user_id: Mapped[int]
+ address_id: Mapped[int]
+
+ if deferred_parameter is None:
+ isopen: Mapped[bool] = mapped_column(deferred_group="g1")
+ description: Mapped[str] = mapped_column(deferred_group="g1")
+ else:
+ isopen: Mapped[bool] = mapped_column(
+ deferred=deferred_parameter, deferred_group="g1"
+ )
+ description: Mapped[str] = mapped_column(
+ deferred=deferred_parameter, deferred_group="g1"
+ )
+
+ if deferred_parameter is not False:
+ self.assert_compile(
+ select(Order),
+ "SELECT orders.id, orders.user_id, orders.address_id "
+ "FROM orders",
+ )
+ self.assert_compile(
+ select(Order).options(undefer_group("g1")),
+ "SELECT orders.isopen, orders.description, orders.id, "
+ "orders.user_id, orders.address_id FROM orders",
+ )
+ else:
+ self.assert_compile(
+ select(Order),
+ "SELECT orders.id, orders.user_id, orders.address_id, "
+ "orders.isopen, orders.description FROM orders",
+ )
+ self.assert_compile(
+ select(Order).options(undefer_group("g1")),
+ "SELECT orders.id, orders.user_id, orders.address_id, "
+ "orders.isopen, orders.description FROM orders",
+ )
+
def test_defer_primary_key(self):
"""what happens when we try to defer the primary key?"""
"description",
)
+ def test_raise_on_col_newstyle(self):
+ class Base(DeclarativeBase):
+ pass
+
+ class Order(Base):
+ __tablename__ = "orders"
+
+ id: Mapped[int] = mapped_column(primary_key=True)
+ user_id: Mapped[int]
+ address_id: Mapped[int]
+ isopen: Mapped[bool]
+ description: Mapped[str] = mapped_column(deferred_raiseload=True)
+
+ sess = fixture_session()
+ stmt = sa.select(Order).order_by(Order.id)
+ o1 = (sess.query(Order).from_statement(stmt).all())[0]
+
+ assert_raises_message(
+ sa.exc.InvalidRequestError,
+ "'Order.description' is not available due to raiseload=True",
+ getattr,
+ o1,
+ "description",
+ )
+
def test_locates_col_w_option_rowproc_only(self):
orders, Order = self.tables.orders, self.classes.Order
projects / thirdparty / sqlalchemy / sqlalchemy.git / commitdiff
