From: Mike Bayer Date: Fri, 16 Nov 2018 23:52:42 +0000 (-0500) Subject: - edits for 1.3 migration notes X-Git-Tag: rel_1_3_0b1~2 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=9addf7734298fff8ef790757783efef02afa5d43;p=thirdparty%2Fsqlalchemy%2Fsqlalchemy.git - edits for 1.3 migration notes Change-Id: Id2065053088481df5a703c63bfc88799a9943a5e --- diff --git a/doc/build/changelog/migration_13.rst b/doc/build/changelog/migration_13.rst index 94ea3c856b..eac4118858 100644 --- a/doc/build/changelog/migration_13.rst +++ b/doc/build/changelog/migration_13.rst @@ -84,6 +84,114 @@ users should report a bug, however the change also incldues a flag :ticket:`4340` +.. _change_4359: + +Improvement to the behavior of many-to-one query expressions +------------------------------------------------------------ + +When building a query that compares a many-to-one relationship to an +object value, such as:: + + u1 = session.query(User).get(5) + + query = session.query(Address).filter(Address.user == u1) + +The above expression ``Address.user == u1``, which ultimately compiles to a SQL +expression normally based on the primary key columns of the ``User`` object +like ``"address.user_id = 5"``, uses a deferred callable in order to retrieve +the value ``5`` within the bound expression until as late as possible. This +is to suit both the use case where the ``Address.user == u1`` expression may be +against a ``User`` object that isn't flushed yet which relies upon a server- +generated primary key value, as well as that the expression always returns the +correct result even if the primary key value of ``u1`` has been changed since +the expression was created. + +However, a side effect of this behavior is that if ``u1`` ends up being expired +by the time the expression is evaluated, it results in an additional SELECT +statement, and in the case that ``u1`` was also detached from the +:class:`.Session`, it would raise an error:: + + u1 = session.query(User).get(5) + + query = session.query(Address).filter(Address.user == u1) + + session.expire(u1) + session.expunge(u1) + + query.all() # <-- would raise DetachedInstanceError + +The expiration / expunging of the object can occur implicitly when the +:class:`.Session` is committed and the ``u1`` instance falls out of scope, +as the ``Address.user == u1`` expression does not strongly reference the +object itself, only its :class:`.InstanceState`. + +The fix is to allow the ``Address.user == u1`` expression to evaluate the value +``5`` based on attempting to retrieve or load the value normally at expression +compilation time as it does now, but if the object is detached and has +been expired, it is retrieved from a new mechanism upon the +:class:`.InstanceState` which will memoize the last known value for a +particular attribute on that state when that attribute is expired. This +mechanism is only enabled for a specific attribute / :class:`.InstanceState` +when needed by the expression feature to conserve performance / memory +overhead. + +Originally, simpler approaches such as evaluating the expression immediately +with various arrangements for trying to load the value later if not present +were attempted, however the difficult edge case is that of the value of a +column attribute (typically a natural primary key) that is being changed. In +order to ensure that an expression like ``Address.user == u1`` always returns +the correct answer for the current state of ``u1``, it will return the current +database-persisted value for a persistent object, unexpiring via SELECT query +if necessary, and for a detached object it will return the most recent known +value, regardless of when the object was expired using a new feature within the +:class:`.InstanceState` that tracks the last known value of a column attribute +whenever the attribute is to be expired. + +Modern attribute API features are used to indicate specific error messages when +the value cannot be evaluated, the two cases of which are when the column +attributes have never been set, and when the object was already expired +when the first evaluation was made and is now detached. In all cases, +:class:`.DetachedInstanceError` is no longer raised. + + +:ticket:`4359` + +.. _change_4353: + +Many-to-one replacement won't raise for "raiseload" or detached for "old" object +-------------------------------------------------------------------------------- + +Given the case where a lazy load would proceed on a many-to-one relationship +in order to load the "old" value, if the relationship does not specify +the :paramref:`.relationship.active_history` flag, an assertion will not +be raised for a detached object:: + + a1 = session.query(Address).filter_by(id=5).one() + + session.expunge(a1) + + a1.user = some_user + +Above, when the ``.user`` attribute is replaced on the detached ``a1`` object, +a :class:`.DetachedInstanceError` would be raised as the attribute is attempting +to retrieve the previous value of ``.user`` from the identity map. The change +is that the operation now proceeds without the old value being loaded. + +The same change is also made to the ``lazy="raise"`` loader strategy:: + + class Address(Base): + # ... + + user = relationship("User", ..., lazy="raise") + +Previously, the association of ``a1.user`` would invoke the "raiseload" +exception as a result of the attribute attempting to retrieve the previous +value. This assertion is now skipped in the case of loading the "old" value. + + +:ticket:`4353` + + .. _change_4354: "del" implemented for ORM attributes @@ -131,16 +239,18 @@ and :meth:`.Query.delete` bulk update/delete methods. The ``query_chooser`` callable is consulted when they are called in order to run the update/delete across multiple shards based on given criteria. - :ticket:`4196` -Key Behavioral Changes - ORM -============================= +Association Proxy Improvements +------------------------------- + +While not for any particular reason, the Association Proxy extension +had many improvements this cycle. .. _change_4308: Association proxy has new cascade_scalar_deletes flag ------------------------------------------------------ +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Given a mapping as:: @@ -190,184 +300,10 @@ to ``None``:: :ticket:`4308` -.. _change_4365: - -Query.join() handles ambiguity in deciding the "left" side more explicitly ---------------------------------------------------------------------------- - -Historically, given a query like the following:: - - u_alias = aliased(User) - session.query(User, u_alias).join(Address) - -given the standard tutorial mappings, the query would produce a FROM clause -as: - -.. sourcecode:: sql - - SELECT ... - FROM users AS users_1, users JOIN addresses ON users.id = addresses.user_id - -That is, the JOIN would implcitly be against the first entity that matches. -The new behavior is that an exception requests that this ambiguity be -resolved:: - - sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to - join from, there are multiple FROMS which can join to this entity. - Try adding an explicit ON clause to help resolve the ambiguity. - -The solution is to provide an ON clause, either as an expression:: - - # join to User - session.query(User, u_alias).join(Address, Address.user_id == User.id) - - # join to u_alias - session.query(User, u_alias).join(Address, Address.user_id == u_alias.id) - -Or to use the relationship attribute, if available:: - - # join to User - session.query(User, u_alias).join(Address, User.addresses) - - # join to u_alias - session.query(User, u_alias).join(Address, u_alias.addresses) - -The change includes that a join can now correctly link to a FROM clause that -is not the first element in the list if the join is otherwise non-ambiguous:: - - session.query(func.current_timestamp(), User).join(Address) - -Prior to this enhancement, the above query would raise:: - - sqlalchemy.exc.InvalidRequestError: Don't know how to join from - CURRENT_TIMESTAMP; please use select_from() to establish the - left entity/selectable of this join - -Now the query works fine: - -.. sourcecode:: sql - - SELECT CURRENT_TIMESTAMP AS current_timestamp_1, users.id AS users_id, - users.name AS users_name, users.fullname AS users_fullname, - users.password AS users_password - FROM users JOIN addresses ON users.id = addresses.user_id - -Overall the change is directly towards Python's "explicit is better than -implicit" philosophy. - -:ticket:`4365` - -.. _change_4353: - -Many-to-one replacement won't raise for "raiseload" or detached for "old" object --------------------------------------------------------------------------------- - -Given the case where a lazy load would proceed on a many-to-one relationship -in order to load the "old" value, if the relationship does not specify -the :paramref:`.relationship.active_history` flag, an assertion will not -be raised for a detached object:: - - a1 = session.query(Address).filter_by(id=5).one() - - session.expunge(a1) - - a1.user = some_user - -Above, when the ``.user`` attribute is replaced on the detached ``a1`` object, -a :class:`.DetachedInstanceError` would be raised as the attribute is attempting -to retrieve the previous value of ``.user`` from the identity map. The change -is that the operation now proceeds without the old value being loaded. - -The same change is also made to the ``lazy="raise"`` loader strategy:: - - class Address(Base): - # ... - - user = relationship("User", ..., lazy="raise") - -Previously, the association of ``a1.user`` would invoke the "raiseload" -exception as a result of the attribute attempting to retrieve the previous -value. This assertion is now skipped in the case of loading the "old" value. - - -:ticket:`4353` - -.. _change_4359: - -Improvement to the behavior of many-to-one query expressions ------------------------------------------------------------- - -When building a query that compares a many-to-one relationship to an -object value, such as:: - - u1 = session.query(User).get(5) - - query = session.query(Address).filter(Address.user == u1) - -The above expression ``Address.user == u1``, which ultimately compiles to a SQL -expression normally based on the primary key columns of the ``User`` object -like ``"address.user_id = 5"``, uses a deferred callable in order to retrieve -the value ``5`` within the bound expression until as late as possible. This -is to suit both the use case where the ``Address.user == u1`` expression may be -against a ``User`` object that isn't flushed yet which relies upon a server- -generated primary key value, as well as that the expression always returns the -correct result even if the primary key value of ``u1`` has been changed since -the expression was created. - -However, a side effect of this behavior is that if ``u1`` ends up being expired -by the time the expression is evaluated, it results in an additional SELECT -statement, and in the case that ``u1`` was also detached from the -:class:`.Session`, it would raise an error:: - - u1 = session.query(User).get(5) - - query = session.query(Address).filter(Address.user == u1) - - session.expire(u1) - session.expunge(u1) - - query.all() # <-- would raise DetachedInstanceError - -The expiration / expunging of the object can occur implicitly when the -:class:`.Session` is committed and the ``u1`` instance falls out of scope, -as the ``Address.user == u1`` expression does not strongly reference the -object itself, only its :class:`.InstanceState`. - -The fix is to allow the ``Address.user == u1`` expression to evaluate the value -``5`` based on attempting to retrieve or load the value normally at expression -compilation time as it does now, but if the object is detached and has -been expired, it is retrieved from a new mechanism upon the -:class:`.InstanceState` which will memoize the last known value for a -particular attribute on that state when that attribute is expired. This -mechanism is only enabled for a specific attribute / :class:`.InstanceState` -when needed by the expression feature to conserve performance / memory -overhead. - -Originally, simpler approaches such as evaluating the expression immediately -with various arrangements for trying to load the value later if not present -were attempted, however the difficult edge case is that of the value of a -column attribute (typically a natural primary key) that is being changed. In -order to ensure that an expression like ``Address.user == u1`` always returns -the correct answer for the current state of ``u1``, it will return the current -database-persisted value for a persistent object, unexpiring via SELECT query -if necessary, and for a detached object it will return the most recent known -value, regardless of when the object was expired using a new feature within the -:class:`.InstanceState` that tracks the last known value of a column attribute -whenever the attribute is to be expired. - -Modern attribute API features are used to indicate specific error messages when -the value cannot be evaluated, the two cases of which are when the column -attributes have never been set, and when the object was already expired -when the first evaluation was made and is now detached. In all cases, -:class:`.DetachedInstanceError` is no longer raised. - - -:ticket:`4359` - .. _change_3423: -AssociationProxy stores class-specific state in a separate container --------------------------------------------------------------------- +AssociationProxy stores class-specific state on a per-class basis +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The :class:`.AssociationProxy` object makes lots of decisions based on the parent mapped class it is associated with. While the @@ -429,7 +365,7 @@ specific to the ``User.keywords`` proxy, such as ``target_class``:: .. _change_4351: AssociationProxy now provides standard column operators for a column-oriented target ------------------------------------------------------------------------------------- +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Given an :class:`.AssociationProxy` where the target is a database column, as opposed to an object reference:: @@ -540,6 +476,142 @@ version of the :class:`.AssociationProxyInstance` class. :ticket:`4351` +Association Proxy now Strong References the Parent Object +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The long-standing behavior of the association proxy collection maintaining +only a weak reference to the parent object is reverted; the proxy will now +maintain a strong reference to the parent for as long as the proxy +collection itself is also in memory, eliminating the "stale association +proxy" error. This change is being made on an experimental basis to see if +any use cases arise where it causes side effects. + +As an example, given a mapping with association proxy:: + + class A(Base): + __tablename__ = 'a' + + id = Column(Integer, primary_key=True) + bs = relationship("B") + b_data = association_proxy('bs', 'data') + + + class B(Base): + __tablename__ = 'b' + id = Column(Integer, primary_key=True) + a_id = Column(ForeignKey("a.id")) + data = Column(String) + + + a1 = A(bs=[B(data='b1'), B(data='b2')]) + + b_data = a1.b_data + +Previously, if ``a1`` were deleted out of scope:: + + del a1 + +Trying to iterate the ``b_data`` collection after ``a1`` is deleted from scope +would raise the error ``"stale association proxy, parent object has gone out of +scope"``. This is because the association proxy needs to access the actual +``a1.bs`` collection in order to produce a view, and prior to this change it +maintained only a weak reference to ``a1``. In particular, users would +frequently encounter this error when performing an inline operation +such as:: + + collection = session.query(A).filter_by(id=1).first().b_data + +Above, because the ``A`` object would be garbage collected before the +``b_data`` collection were actually used. + +The change is that the ``b_data`` collection is now maintaining a strong +reference to the ``a1`` object, so that it remains present:: + + assert b_data == ['b1', 'b2'] + +This change introduces the side effect that if an application is passing around +the collection as above, **the parent object won't be garbage collected** until +the collection is also discarded. As always, if ``a1`` is persistent inside a +particular :class:`.Session`, it will remain part of that session's state +until it is garbage collected. + +Note that this change may be revised if it leads to problems. + +:ticket:`4268` + +.. _change_4365: + +Key Behavioral Changes - ORM +============================= + + +Query.join() handles ambiguity in deciding the "left" side more explicitly +--------------------------------------------------------------------------- + +Historically, given a query like the following:: + + u_alias = aliased(User) + session.query(User, u_alias).join(Address) + +given the standard tutorial mappings, the query would produce a FROM clause +as: + +.. sourcecode:: sql + + SELECT ... + FROM users AS users_1, users JOIN addresses ON users.id = addresses.user_id + +That is, the JOIN would implcitly be against the first entity that matches. +The new behavior is that an exception requests that this ambiguity be +resolved:: + + sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to + join from, there are multiple FROMS which can join to this entity. + Try adding an explicit ON clause to help resolve the ambiguity. + +The solution is to provide an ON clause, either as an expression:: + + # join to User + session.query(User, u_alias).join(Address, Address.user_id == User.id) + + # join to u_alias + session.query(User, u_alias).join(Address, Address.user_id == u_alias.id) + +Or to use the relationship attribute, if available:: + + # join to User + session.query(User, u_alias).join(Address, User.addresses) + + # join to u_alias + session.query(User, u_alias).join(Address, u_alias.addresses) + +The change includes that a join can now correctly link to a FROM clause that +is not the first element in the list if the join is otherwise non-ambiguous:: + + session.query(func.current_timestamp(), User).join(Address) + +Prior to this enhancement, the above query would raise:: + + sqlalchemy.exc.InvalidRequestError: Don't know how to join from + CURRENT_TIMESTAMP; please use select_from() to establish the + left entity/selectable of this join + +Now the query works fine: + +.. sourcecode:: sql + + SELECT CURRENT_TIMESTAMP AS current_timestamp_1, users.id AS users_id, + users.name AS users_name, users.fullname AS users_fullname, + users.password AS users_password + FROM users JOIN addresses ON users.id = addresses.user_id + +Overall the change is directly towards Python's "explicit is better than +implicit" philosophy. + +:ticket:`4365` + + + .. _change_4246: @@ -648,69 +720,6 @@ The fix now includes that ``address.user_id`` is left unchanged as per .. _change_4268: -Association Proxy now Strong References the Parent Object -========================================================= - -The long-standing behavior of the association proxy collection maintaining -only a weak reference to the parent object is reverted; the proxy will now -maintain a strong reference to the parent for as long as the proxy -collection itself is also in memory, eliminating the "stale association -proxy" error. This change is being made on an experimental basis to see if -any use cases arise where it causes side effects. - -As an example, given a mapping with association proxy:: - - class A(Base): - __tablename__ = 'a' - - id = Column(Integer, primary_key=True) - bs = relationship("B") - b_data = association_proxy('bs', 'data') - - - class B(Base): - __tablename__ = 'b' - id = Column(Integer, primary_key=True) - a_id = Column(ForeignKey("a.id")) - data = Column(String) - - - a1 = A(bs=[B(data='b1'), B(data='b2')]) - - b_data = a1.b_data - -Previously, if ``a1`` were deleted out of scope:: - - del a1 - -Trying to iterate the ``b_data`` collection after ``a1`` is deleted from scope -would raise the error ``"stale association proxy, parent object has gone out of -scope"``. This is because the association proxy needs to access the actual -``a1.bs`` collection in order to produce a view, and prior to this change it -maintained only a weak reference to ``a1``. In particular, users would -frequently encounter this error when performing an inline operation -such as:: - - collection = session.query(A).filter_by(id=1).first().b_data - -Above, because the ``A`` object would be garbage collected before the -``b_data`` collection were actually used. - -The change is that the ``b_data`` collection is now maintaining a strong -reference to the ``a1`` object, so that it remains present:: - - assert b_data == ['b1', 'b2'] - -This change introduces the side effect that if an application is passing around -the collection as above, **the parent object won't be garbage collected** until -the collection is also discarded. As always, if ``a1`` is persistent inside a -particular :class:`.Session`, it will remain part of that session's state -until it is garbage collected. - -Note that this change may be revised if it leads to problems. - - -:ticket:`4268` New Features and Improvements - Core ====================================