From: Mike Bayer Date: Sun, 23 Apr 2023 16:21:59 +0000 (-0400) Subject: minimum doc build X-Git-Url: http://git.ipfire.org/?a=commitdiff_plain;h=4d74ea61567c781a3521d640136bb9283e0d2e1f;p=thirdparty%2Fsqlalchemy%2Fsqlalchemy.git minimum doc build remove a huge amount of docs so we can run without extensions and send Sphinx a bug report that version 6.x has become unaccepably memory hogging Change-Id: I17b6265187a7901b4714130dd303fccb1d86bb0e --- diff --git a/doc/build/changelog/changelog_01.rst b/doc/build/changelog/changelog_01.rst deleted file mode 100644 index 2122c36f2d..0000000000 --- a/doc/build/changelog/changelog_01.rst +++ /dev/null @@ -1,976 +0,0 @@ - -============= -0.1 Changelog -============= - - -.. changelog:: - :version: 0.1.7 - :released: Fri May 05 2006 - - .. change:: - :tags: - :tickets: - - some fixes to topological sort algorithm - - .. change:: - :tags: - :tickets: - - added DISTINCT ON support to Postgres (just supply distinct=[col1,col2..]) - - .. change:: - :tags: - :tickets: - - added __mod__ (% operator) to sql expressions - - .. change:: - :tags: - :tickets: - - "order_by" mapper property inherited from inheriting mapper - - .. change:: - :tags: - :tickets: - - fix to column type used when mapper UPDATES/DELETEs - - .. change:: - :tags: - :tickets: - - with convert_unicode=True, reflection was failing, has been fixed - - .. change:: - :tags: - :tickets: - - types types types! still weren't working....have to use TypeDecorator again :( - - .. change:: - :tags: - :tickets: - - mysql binary type converts array output to buffer, fixes PickleType - - .. change:: - :tags: - :tickets: - - fixed the attributes.py memory leak once and for all - - .. change:: - :tags: - :tickets: - - unittests are qualified based on the databases that support each one - - .. change:: - :tags: - :tickets: - - fixed bug where column defaults would clobber VALUES clause of insert objects - - .. change:: - :tags: - :tickets: - - fixed bug where table def w/ schema name would force engine connection - - .. change:: - :tags: - :tickets: - - fix for parenthesis to work correctly with subqueries in INSERT/UPDATE - - .. change:: - :tags: - :tickets: - - HistoryArraySet gets extend() method - - .. change:: - :tags: - :tickets: - - fixed lazyload support for other comparison operators besides = - - .. change:: - :tags: - :tickets: - - lazyload fix where two comparisons in the join condition point to the - samem column - - .. change:: - :tags: - :tickets: - - added "construct_new" flag to mapper, will use __new__ to create instances - instead of __init__ (standard in 0.2) - - .. change:: - :tags: - :tickets: - - added selectresults.py to SVN, missed it last time - - .. change:: - :tags: - :tickets: - - tweak to allow a many-to-many relationship from a table to itself via - an association table - - .. change:: - :tags: - :tickets: - - small fix to "translate_row" function used by polymorphic example - - .. change:: - :tags: - :tickets: - - create_engine uses cgi.parse_qsl to read query string (out the window in 0.2) - - .. change:: - :tags: - :tickets: - - tweaks to CAST operator - - .. change:: - :tags: - :tickets: - - fixed function names LOCAL_TIME/LOCAL_TIMESTAMP -> LOCALTIME/LOCALTIMESTAMP - - .. change:: - :tags: - :tickets: - - fixed order of ORDER BY/HAVING in compile - -.. changelog:: - :version: 0.1.6 - :released: Wed Apr 12 2006 - - .. change:: - :tags: - :tickets: - - support for MS-SQL added courtesy Rick Morrison, Runar Petursson - - .. change:: - :tags: - :tickets: - - the latest SQLSoup from J. Ellis - - .. change:: - :tags: - :tickets: - - ActiveMapper has preliminary support for inheritance (Jeff Watkins) - - .. change:: - :tags: - :tickets: - - added a "mods" system which allows pluggable modules that modify/augment - core functionality, using the function "install_mods(\*modnames)". - - .. change:: - :tags: - :tickets: - - added the first "mod", SelectResults, which modifies mapper selects to - return generators that turn ranges into LIMIT/OFFSET queries - (Jonas Borgstr? - - .. change:: - :tags: - :tickets: - - factored out querying capabilities of Mapper into a separate Query object - which is Session-centric. this improves the performance of mapper.using(session) - and makes other things possible. - - .. change:: - :tags: - :tickets: - - objectstore/Session refactored, the official way to save objects is now - via the flush() method. The begin/commit functionality of Session is factored - into LegacySession which is still established as the default behavior, until - the 0.2 series. - - .. change:: - :tags: - :tickets: - - types system is bound to an engine at query compile time, not schema - construction time. this simplifies the types system as well as the ProxyEngine. - - .. change:: - :tags: - :tickets: - - added 'version_id' keyword argument to mapper. this keyword should reference a - Column object with type Integer, preferably non-nullable, which will be used on - the mapped table to track version numbers. this number is incremented on each - save operation and is specified in the UPDATE/DELETE conditions so that it - factors into the returned row count, which results in a ConcurrencyError if the - value received is not the expected count. - - .. change:: - :tags: - :tickets: - - added 'entity_name' keyword argument to mapper. a mapper is now associated - with a class via the class object as well as an optional entity_name parameter, - which is a string defaulting to None. any number of primary mappers can be - created for a class, qualified by the entity name. instances of those classes - will issue all of their load and save operations through their - entity_name-qualified mapper, and maintain separate a identity in the identity - map for an otherwise equivalent object. - - .. change:: - :tags: - :tickets: - - overhaul to the attributes system. code has been clarified, and also fixed to - support proper polymorphic behavior on object attributes. - - .. change:: - :tags: - :tickets: - - added "for_update" flag to Select objects - - .. change:: - :tags: - :tickets: - - some fixes for backrefs - - .. change:: - :tags: - :tickets: - - fix for postgres1 DateTime type - - .. change:: - :tags: - :tickets: - - documentation pages mostly switched over to Markdown syntax - -.. changelog:: - :version: 0.1.5 - :released: Mon Mar 27 2006 - - .. change:: - :tags: - :tickets: - - added SQLSession concept to SQLEngine. this object keeps track of retrieving a - connection from the connection pool as well as an in-progress transaction. - methods push_session() and pop_session() added to SQLEngine which push/pop a new - SQLSession onto the engine, allowing operation upon a second connection "nested" - within the previous one, allowing nested transactions. Other tricks are sure to - come later regarding SQLSession. - - .. change:: - :tags: - :tickets: - - added nest_on argument to objectstore.Session. This is a single SQLEngine or - list of engines for which push_session()/pop_session() will be called each time - this Session becomes the active session (via objectstore.push_session() or - equivalent). This allows a unit of work Session to take advantage of the nested - transaction feature without explicitly calling push_session/pop_session on the - engine. - - .. change:: - :tags: - :tickets: - - factored apart objectstore/unitofwork to separate "Session scoping" from - "uow commit heavy lifting" - - .. change:: - :tags: - :tickets: - - added populate_instance() method to MapperExtension. allows an extension to - modify the population of object attributes. this method can call the - populate_instance() method on another mapper to proxy the attribute population - from one mapper to another; some row translation logic is also built in to help - with this. - - .. change:: - :tags: - :tickets: - - fixed Oracle8-compatibility "use_ansi" flag which converts JOINs to - comparisons with the = and (+) operators, passes basic unittests - - .. change:: - :tags: - :tickets: - - tweaks to Oracle LIMIT/OFFSET support - - .. change:: - :tags: - :tickets: - - Oracle reflection uses ALL_** views instead of USER_** to get larger - list of stuff to reflect from - - .. change:: - :tags: - :tickets: 105 - - fixes to Oracle foreign key reflection - - .. change:: - :tags: - :tickets: - - objectstore.commit(obj1, obj2,...) adds an extra step to seek out private - relations on properties and delete child objects, even though its not a global - commit - - .. change:: - :tags: - :tickets: - - lots and lots of fixes to mappers which use inheritance, strengthened the - concept of relations on a mapper being made towards the "local" table for that - mapper, not the tables it inherits. allows more complex compositional patterns - to work with lazy/eager loading. - - .. change:: - :tags: - :tickets: - - added support for mappers to inherit from others based on the same table, - just specify the same table as that of both parent/child mapper. - - .. change:: - :tags: - :tickets: - - some minor speed improvements to the attributes system with regards to - instantiating and populating new objects. - - .. change:: - :tags: - :tickets: - - fixed MySQL binary unit test - - .. change:: - :tags: - :tickets: - - INSERTs can receive clause elements as VALUES arguments, not just literal - values - - .. change:: - :tags: - :tickets: - - support for calling multi-tokened functions, i.e. schema.mypkg.func() - - .. change:: - :tags: - :tickets: - - added J. Ellis' SQLSoup module to extensions package - - .. change:: - :tags: - :tickets: - - added "polymorphic" examples illustrating methods to load multiple object types - from one mapper, the second of which uses the new populate_instance() method. - small improvements to mapper, UNION construct to help the examples along - - .. change:: - :tags: - :tickets: - - improvements/fixes to session.refresh()/session.expire() (which may have - been called "invalidate" earlier..) - - .. change:: - :tags: - :tickets: - - added session.expunge() which totally removes an object from the current - session - - .. change:: - :tags: - :tickets: - - added \*args, \**kwargs pass-through to engine.transaction(func) allowing easier - creation of transactionalizing decorator functions - - .. change:: - :tags: - :tickets: - - added iterator interface to ResultProxy: "for row in result:..." - - .. change:: - :tags: - :tickets: - - added assertion to tx = session.begin(); tx.rollback(); tx.begin(), i.e. can't - use it after a rollback() - - .. change:: - :tags: - :tickets: - - added date conversion on bind parameter fix to SQLite enabling dates to - work with pysqlite1 - - .. change:: - :tags: - :tickets: 116 - - improvements to subqueries to more intelligently construct their FROM - clauses - - .. change:: - :tags: - :tickets: - - added PickleType to types. - - .. change:: - :tags: - :tickets: - - fixed two bugs with column labels with regards to bind parameters: bind param - keynames they are now generated from a column "label" in all relevant cases to - take advantage of excess-name-length rules, and checks for a peculiar collision - against a column named the same as "tablename_colname" added - - .. change:: - :tags: - :tickets: - - major overhaul to unit of work documentation, other documentation sections. - - .. change:: - :tags: - :tickets: - - fixed attributes bug where if an object is committed, its lazy-loaded list got - blown away if it hadn't been loaded - - .. change:: - :tags: - :tickets: - - added unique_connection() method to engine, connection pool to return a - connection that is not part of the thread-local context or any current - transaction - - .. change:: - :tags: - :tickets: - - added invalidate() function to pooled connection. will remove the connection - from the pool. still need work for engines to auto-reconnect to a stale DB - though. - - .. change:: - :tags: - :tickets: - - added distinct() function to column elements so you can do - func.count(mycol.distinct()) - - .. change:: - :tags: - :tickets: - - added "always_refresh" flag to Mapper, creates a mapper that will always - refresh the attributes of objects it gets/selects from the DB, overwriting any - changes made. - -.. changelog:: - :version: 0.1.4 - :released: Mon Mar 13 2006 - - .. change:: - :tags: - :tickets: - - create_engine() now uses genericized parameters; host/hostname, - db/dbname/database, password/passwd, etc. for all engine connections. makes - engine URIs much more "universal" - - .. change:: - :tags: - :tickets: - - added support for SELECT statements embedded into a column clause, using the - flag "scalar=True" - - .. change:: - :tags: - :tickets: - - another overhaul to EagerLoading when used in conjunction with mappers that - inherit; improvements to eager loads figuring out their aliased queries - correctly, also relations set up against a mapper with inherited mappers will - create joins against the table that is specific to the mapper itself (i.e. and - not any tables that are inherited/are further down the inheritance chain), - this can be overridden by using custom primary/secondary joins. - - .. change:: - :tags: - :tickets: - - added J.Ellis patch to mapper.py so that selectone() throws an exception - if query returns more than one object row, selectfirst() to not throw the - exception. also adds selectfirst_by (synonymous with get_by) and selectone_by - - .. change:: - :tags: - :tickets: - - added onupdate parameter to Column, will exec SQL/python upon an update - statement.Also adds "for_update=True" to all DefaultGenerator subclasses - - .. change:: - :tags: - :tickets: - - added support for Oracle table reflection contributed by Andrija Zaric; - still some bugs to work out regarding composite primary keys/dictionary selection - - .. change:: - :tags: - :tickets: - - checked in an initial Firebird module, awaiting testing. - - .. change:: - :tags: - :tickets: - - added sql.ClauseParameters dictionary object as the result for - compiled.get_params(), does late-typeprocessing of bind parameters so - that the original values are easier to access - - .. change:: - :tags: - :tickets: - - more docs for indexes, column defaults, connection pooling, engine construction - - .. change:: - :tags: - :tickets: - - overhaul to the construction of the types system. uses a simpler inheritance - pattern so that any of the generic types can be easily subclassed, with no need - for TypeDecorator. - - .. change:: - :tags: - :tickets: - - added "convert_unicode=False" parameter to SQLEngine, will cause all String - types to perform unicode encoding/decoding (makes Strings act like Unicodes) - - .. change:: - :tags: - :tickets: - - added 'encoding="utf8"' parameter to engine. the given encoding will be - used for all encode/decode calls within Unicode types as well as Strings - when convert_unicode=True. - - .. change:: - :tags: - :tickets: - - improved support for mapping against UNIONs, added polymorph.py example - to illustrate multi-class mapping against a UNION - - .. change:: - :tags: - :tickets: - - fix to SQLite LIMIT/OFFSET syntax - - .. change:: - :tags: - :tickets: - - fix to Oracle LIMIT syntax - - .. change:: - :tags: - :tickets: - - added backref() function, allows backreferences to have keyword arguments - that will be passed to the backref. - - .. change:: - :tags: - :tickets: - - Sequences and ColumnDefault objects can do execute()/scalar() standalone - - .. change:: - :tags: - :tickets: - - SQL functions (i.e. func.foo()) can do execute()/scalar() standalone - - .. change:: - :tags: - :tickets: - - fix to SQL functions so that the ANSI-standard functions, i.e. current_timestamp - etc., do not specify parenthesis. all other functions do. - - .. change:: - :tags: - :tickets: - - added settattr_clean and append_clean to SmartProperty, which set - attributes without triggering a "dirty" event or any history. used as: - myclass.prop1.setattr_clean(myobject, 'hi') - - .. change:: - :tags: - :tickets: - - improved support to column defaults when used by mappers; mappers will pull - pre-executed defaults from statement's executed bind parameters - (pre-conversion) to populate them into a saved object's attributes; if any - PassiveDefaults have fired off, will instead post-fetch the row from the DB to - populate the object. - - .. change:: - :tags: - :tickets: - - added 'get_session().invalidate(\*obj)' method to objectstore, instances will - refresh() themselves upon the next attribute access. - - .. change:: - :tags: - :tickets: - - improvements to SQL func calls including an "engine" keyword argument so - they can be execute()d or scalar()ed standalone, also added func accessor to - SQLEngine - - .. change:: - :tags: - :tickets: - - fix to MySQL4 custom table engines, i.e. TYPE instead of ENGINE - - .. change:: - :tags: - :tickets: - - slightly enhanced logging, includes timestamps and a somewhat configurable - formatting system, in lieu of a full-blown logging system - - .. change:: - :tags: - :tickets: - - improvements to the ActiveMapper class from the TG gang, including - many-to-many relationships - - .. change:: - :tags: - :tickets: - - added Double and TinyInt support to mysql - -.. changelog:: - :version: 0.1.3 - :released: Thu Mar 02 2006 - - .. change:: - :tags: - :tickets: - - completed "post_update" feature, will add a second update statement before - inserts and after deletes in order to reconcile a relationship without any - dependencies being created; used when persisting two rows that are dependent - on each other - - .. change:: - :tags: - :tickets: - - completed mapper.using(session) function, localized per-object Session - functionality; objects can be declared and manipulated as local to any - user-defined Session - - .. change:: - :tags: - :tickets: - - fix to Oracle "row_number over" clause with multiple tables - - .. change:: - :tags: - :tickets: - - mapper.get() was not selecting multiple-keyed objects if the mapper's table was a join, - such as in an inheritance relationship, this is fixed. - - .. change:: - :tags: - :tickets: - - overhaul to sql/schema packages so that the sql package can run all on its own, - producing selects, inserts, etc. without any engine dependencies. builds upon - new TableClause/ColumnClause lexical objects. Schema's Table/Column objects - are the "physical" subclasses of them. simplifies schema/sql relationship, - extensions (like proxyengine), and speeds overall performance by a large margin. - removes the entire getattr() behavior that plagued 0.1.1. - - .. change:: - :tags: - :tickets: - - refactoring of how the mapper "synchronizes" data between two objects into a - separate module, works better with properties attached to a mapper that has an - additional inheritance relationship to one of the related tables, also the same - methodology used to synchronize parent/child objects now used by mapper to - synchronize between inherited and inheriting mappers. - - .. change:: - :tags: - :tickets: - - made objectstore "check for out-of-identitymap" more aggressive, will perform the - check when object attributes are modified or the object is deleted - - .. change:: - :tags: - :tickets: - - Index object fully implemented, can be constructed standalone, or via - "index" and "unique" arguments on Columns. - - .. change:: - :tags: - :tickets: - - added "convert_unicode" flag to SQLEngine, will treat all String/CHAR types - as Unicode types, with raw-byte/utf-8 translation on the bind parameter and - result set side. - - .. change:: - :tags: - :tickets: - - postgres maintains a list of ANSI functions that must have no parenthesis so - function calls with no arguments work consistently - - .. change:: - :tags: - :tickets: - - tables can be created with no engine specified. this will default their engine - to a module-scoped "default engine" which is a ProxyEngine. this engine can - be connected via the function "global_connect". - - .. change:: - :tags: - :tickets: - - added "refresh(\*obj)" method to objectstore / Session to reload the attributes of - any set of objects from the database unconditionally - -.. changelog:: - :version: 0.1.2 - :released: Fri Feb 24 2006 - - .. change:: - :tags: - :tickets: - - fixed a recursive call in schema that was somehow running 994 times then returning - normally. broke nothing, slowed down everything. thanks to jpellerin for finding this. - -.. changelog:: - :version: 0.1.1 - :released: Thu Feb 23 2006 - - .. change:: - :tags: - :tickets: - - small fix to Function class so that expressions with a func.foo() use the type of the - Function object (i.e. the left side) as the type of the boolean expression, not the - other side which is more of a moving target (changeset 1020). - - .. change:: - :tags: - :tickets: - - creating self-referring mappers with backrefs slightly easier (but still not that easy - - changeset 1019) - - .. change:: - :tags: - :tickets: - - fixes to one-to-one mappings (changeset 1015) - - .. change:: - :tags: - :tickets: - - psycopg1 date/time issue with None fixed (changeset 1005) - - .. change:: - :tags: - :tickets: - - two issues related to postgres, which doesn't want to give you the "lastrowid" - since oids are deprecated: - - * postgres database-side defaults that are on primary key cols *do* execute - explicitly beforehand, even though that's not the idea of a PassiveDefault. this is - because sequences on columns get reflected as PassiveDefaults, but need to be explicitly - executed on a primary key col so we know what we just inserted. - * if you did add a row that has a bunch of database-side defaults on it, - and the PassiveDefault thing was working the old way, i.e. they just execute on - the DB side, the "can't get the row back without an OID" exception that occurred - also will not happen unless someone (usually the ORM) explicitly asks for it. - - .. change:: - :tags: - :tickets: - - fixed a glitch with engine.execute_compiled where it was making a second - ResultProxy that just got thrown away. - - .. change:: - :tags: - :tickets: - - began to implement newer logic in object properties. you can now say - myclass.attr.property, which will give you the PropertyLoader corresponding to that - attribute, i.e. myclass.mapper.props['attr'] - - .. change:: - :tags: - :tickets: - - eager loading has been internally overhauled to use aliases at all times. more - complicated chains of eager loads can now be created without any need for explicit - "use aliases"-type instructions. EagerLoader code is also much simpler now. - - .. change:: - :tags: - :tickets: - - a new somewhat experimental flag "use_update" added to relations, indicates that - this relationship should be handled by a second UPDATE statement, either after a - primary INSERT or before a primary DELETE. handles circular row dependencies. - - .. change:: - :tags: - :tickets: - - added exceptions module, all raised exceptions (except for some - KeyError/AttributeError exceptions) descend from these classes. - - .. change:: - :tags: - :tickets: - - fix to date types with MySQL, returned timedelta converted to datetime.time - - .. change:: - :tags: - :tickets: - - two-phase objectstore.commit operations (i.e. begin/commit) now return a - transactional object (SessionTrans), to more clearly indicate transaction boundaries. - - .. change:: - :tags: - :tickets: - - Index object with create/drop support added to schema - - .. change:: - :tags: - :tickets: - - fix to postgres, where it will explicitly pre-execute a PassiveDefault on a table - if it is a primary key column, pursuant to the ongoing "we can't get inserted rows - back from postgres" issue - - .. change:: - :tags: - :tickets: - - change to information_schema query that gets back postgres table defs, now - uses explicit JOIN keyword, since one user had faster performance with 8.1 - - .. change:: - :tags: - :tickets: - - fix to engine.process_defaults so it works correctly with a table that has - different column name/column keys (changeset 982) - - .. change:: - :tags: - :tickets: - - a column can only be attached to one table - this is now asserted - - .. change:: - :tags: - :tickets: - - postgres time types descend from Time type - - .. change:: - :tags: - :tickets: - - fix to alltests so that it runs types test (now named testtypes) - - .. change:: - :tags: - :tickets: - - fix to Join object so that it correctly exports its foreign keys (cs 973) - - .. change:: - :tags: - :tickets: - - creating relationships against mappers that use inheritance fixed (cs 973) diff --git a/doc/build/changelog/changelog_02.rst b/doc/build/changelog/changelog_02.rst deleted file mode 100644 index 3d40a79a32..0000000000 --- a/doc/build/changelog/changelog_02.rst +++ /dev/null @@ -1,1191 +0,0 @@ - -============= -0.2 Changelog -============= - - -.. changelog:: - :version: 0.2.8 - :released: Tue Sep 05 2006 - - .. change:: - :tags: - :tickets: - - cleanup on connection methods + documentation. custom DBAPI - arguments specified in query string, 'connect_args' argument - to 'create_engine', or custom creation function via 'creator' - function to 'create_engine'. - - .. change:: - :tags: - :tickets: 274 - - added "recycle" argument to Pool, is "pool_recycle" on create_engine, - defaults to 3600 seconds; connections after this age will be closed and - replaced with a new one, to handle db's that automatically close - stale connections - - .. change:: - :tags: - :tickets: 121 - - changed "invalidate" semantics with pooled connection; will - instruct the underlying connection record to reconnect the next - time its called. "invalidate" will also automatically be called - if any error is thrown in the underlying call to connection.cursor(). - this will hopefully allow the connection pool to reconnect to a - database that had been stopped and started without restarting - the connecting application - - .. change:: - :tags: - :tickets: - - eesh ! the tutorial doctest was broken for quite some time. - - .. change:: - :tags: - :tickets: - - add_property() method on mapper does a "compile all mappers" - step in case the given property references a non-compiled mapper - (as it did in the case of the tutorial !) - - .. change:: - :tags: - :tickets: 277 - - check for pg sequence already existing before create - - .. change:: - :tags: - :tickets: - - if a contextual session is established via MapperExtension.get_session - (as it is using the sessioncontext plugin, etc), a lazy load operation - will use that session by default if the parent object is not - persistent with a session already. - - .. change:: - :tags: - :tickets: - - lazy loads will not fire off for an object that does not have a - database identity (why? - see https://www.sqlalchemy.org/trac/wiki/WhyDontForeignKeysLoadData) - - .. change:: - :tags: - :tickets: - - unit-of-work does a better check for "orphaned" objects that are - part of a "delete-orphan" cascade, for certain conditions where the - parent isn't available to cascade from. - - .. change:: - :tags: - :tickets: - - mappers can tell if one of their objects is an "orphan" based - on interactions with the attribute package. this check is based - on a status flag maintained for each relationship - when objects are attached and detached from each other. - - .. change:: - :tags: - :tickets: - - it is now invalid to declare a self-referential relationship with - "delete-orphan" (as the abovementioned check would make them impossible - to save) - - .. change:: - :tags: - :tickets: - - improved the check for objects being part of a session when the - unit of work seeks to flush() them as part of a relationship.. - - .. change:: - :tags: - :tickets: 280 - - statement execution supports using the same BindParam - object more than once in an expression; simplified handling of positional - parameters. nice job by Bill Noon figuring out the basic idea. - - .. change:: - :tags: - :tickets: 60, 71 - - postgres reflection moved to use pg_schema tables, can be overridden - with use_information_schema=True argument to create_engine. - - .. change:: - :tags: - :tickets: 155 - - added case_sensitive argument to MetaData, Table, Column, determines - itself automatically based on if a parent schemaitem has a non-None - setting for the flag, or if not, then whether the identifier name is all lower - case or not. when set to True, quoting is applied to identifiers with mixed or - uppercase identifiers. quoting is also applied automatically in all cases to - identifiers that are known to be reserved words or contain other non-standard - characters. various database dialects can override all of this behavior, but - currently they are all using the default behavior. tested with postgres, mysql, - sqlite, oracle. needs more testing with firebird, ms-sql. part of the ongoing - work with - - .. change:: - :tags: - :tickets: - - unit tests updated to run without any pysqlite installed; pool - test uses a mock DBAPI - - .. change:: - :tags: - :tickets: 281 - - urls support escaped characters in passwords - - .. change:: - :tags: - :tickets: - - added limit/offset to UNION queries (though not yet in oracle) - - .. change:: - :tags: - :tickets: - - added "timezone=True" flag to DateTime and Time types. postgres - so far will convert this to "TIME[STAMP] (WITH|WITHOUT) TIME ZONE", - so that control over timezone presence is more controllable (psycopg2 - returns datetimes with tzinfo's if available, which can create confusion - against datetimes that don't). - - .. change:: - :tags: - :tickets: 287 - - fix to using query.count() with distinct, \**kwargs with SelectResults - count() - - .. change:: - :tags: - :tickets: 289 - - deregister Table from MetaData when autoload fails; - - .. change:: - :tags: - :tickets: 293 - - import of py2.5s sqlite3 - - .. change:: - :tags: - :tickets: 296 - - unicode fix for startswith()/endswith() - -.. changelog:: - :version: 0.2.7 - :released: Sat Aug 12 2006 - - .. change:: - :tags: - :tickets: - - quoting facilities set up so that database-specific quoting can be - turned on for individual table, schema, and column identifiers when - used in all queries/creates/drops. Enabled via "quote=True" in - Table or Column, as well as "quote_schema=True" in Table. Thanks to - Aaron Spike for the excellent efforts. - - .. change:: - :tags: - :tickets: - - assignmapper was setting is_primary=True, causing all sorts of mayhem - by not raising an error when redundant mappers were set up, fixed - - .. change:: - :tags: - :tickets: - - added allow_null_pks option to Mapper, allows rows where some - primary key columns are null (i.e. when mapping to outer joins etc) - - .. change:: - :tags: - :tickets: - - modification to unitofwork to not maintain ordering within the - "new" list or within the UOWTask "objects" list; instead, new objects - are tagged with an ordering identifier as they are registered as new - with the session, and the INSERT statements are then sorted within the - mapper save_obj. the INSERT ordering has basically been pushed all - the way to the end of the flush cycle. that way the various sorts and - organizations occurring within UOWTask (particularly the circular task - sort) don't have to worry about maintaining order (which they weren't anyway) - - .. change:: - :tags: - :tickets: - - fixed reflection of foreign keys to autoload the referenced table - if it was not loaded already - - .. change:: - :tags: - :tickets: 256 - - - pass URL query string arguments to connect() function - - .. change:: - :tags: - :tickets: 257 - - - oracle boolean type - - .. change:: - :tags: - :tickets: - - custom primary/secondary join conditions in a relation *will* be propagated - to backrefs by default. specifying a backref() will override this behavior. - - .. change:: - :tags: - :tickets: - - better check for ambiguous join conditions in sql.Join; propagates to a - better error message in PropertyLoader (i.e. relation()/backref()) for when - the join condition can't be reasonably determined. - - .. change:: - :tags: - :tickets: - - sqlite creates ForeignKeyConstraint objects properly upon table - reflection. - - .. change:: - :tags: - :tickets: 224 - - adjustments to pool stemming from changes made for. - overflow counter should only be decremented if the connection actually - succeeded. added a test script to attempt testing this. - - .. change:: - :tags: - :tickets: - - fixed mysql reflection of default values to be PassiveDefault - - .. change:: - :tags: - :tickets: 263, 264 - - added reflected 'tinyint', 'mediumint' type to MS-SQL. - - .. change:: - :tags: - :tickets: - - SingletonThreadPool has a size and does a cleanup pass, so that - only a given number of thread-local connections stay around (needed - for sqlite applications that dispose of threads en masse) - - .. change:: - :tags: - :tickets: 267, 265 - - fixed small pickle bug(s) with lazy loaders - - .. change:: - :tags: - :tickets: - - fixed possible error in mysql reflection where certain versions - return an array instead of string for SHOW CREATE TABLE call - - .. change:: - :tags: - :tickets: 1770 - - fix to lazy loads when mapping to joins - - .. change:: - :tags: - :tickets: - - all create()/drop() calls have a keyword argument of "connectable". - "engine" is deprecated. - - .. change:: - :tags: - :tickets: - - fixed ms-sql connect() to work with adodbapi - - .. change:: - :tags: - :tickets: - - added "nowait" flag to Select() - - .. change:: - :tags: - :tickets: 271 - - inheritance check uses issubclass() instead of direct __mro__ check - to make sure class A inherits from B, allowing mapper inheritance to more - flexibly correspond to class inheritance - - .. change:: - :tags: - :tickets: 252 - - SelectResults will use a subselect, when calling an aggregate (i.e. - max, min, etc.) on a SelectResults that has an ORDER BY clause - - .. change:: - :tags: - :tickets: 269 - - fixes to types so that database-specific types more easily used; - fixes to mysql text types to work with this methodology - - .. change:: - :tags: - :tickets: - - some fixes to sqlite date type organization - - .. change:: - :tags: - :tickets: 263 - - added MSTinyInteger to MS-SQL - -.. changelog:: - :version: 0.2.6 - :released: Thu Jul 20 2006 - - .. change:: - :tags: - :tickets: 76 - - big overhaul to schema to allow truly composite primary and foreign - key constraints, via new ForeignKeyConstraint and PrimaryKeyConstraint - objects. - Existing methods of primary/foreign key creation have not been changed - but use these new objects behind the scenes. table creation - and reflection is now more table oriented rather than column oriented. - - .. change:: - :tags: - :tickets: - - overhaul to MapperExtension calling scheme, wasn't working very well - previously - - .. change:: - :tags: - :tickets: - - tweaks to ActiveMapper, supports self-referential relationships - - .. change:: - :tags: - :tickets: - - slight rearrangement to objectstore (in activemapper/threadlocal) - so that the SessionContext is referenced by '.context' instead - of subclassed directly. - - .. change:: - :tags: - :tickets: - - activemapper will use threadlocal's objectstore if the mod is - activated when activemapper is imported - - .. change:: - :tags: - :tickets: - - small fix to URL regexp to allow filenames with '@' in them - - .. change:: - :tags: - :tickets: - - fixes to Session expunge/update/etc...needs more cleanup. - - .. change:: - :tags: - :tickets: - - select_table mappers *still* weren't always compiling - - .. change:: - :tags: - :tickets: - - fixed up Boolean datatype - - .. change:: - :tags: - :tickets: - - added count()/count_by() to list of methods proxied by assignmapper; - this also adds them to activemapper - - .. change:: - :tags: - :tickets: - - connection exceptions wrapped in DBAPIError - - .. change:: - :tags: - :tickets: - - ActiveMapper now supports autoloading column definitions from the - database if you supply a __autoload__ = True attribute in your - mapping inner-class. Currently this does not support reflecting - any relationships. - - .. change:: - :tags: - :tickets: - - deferred column load could screw up the connection status in - a flush() under some circumstances, this was fixed - - .. change:: - :tags: - :tickets: - - expunge() was not working with cascade, fixed. - - .. change:: - :tags: - :tickets: - - potential endless loop in cascading operations fixed. - - .. change:: - :tags: - :tickets: - - added "synonym()" function, applied to properties to have a - propname the same as another, for the purposes of overriding props - and allowing the original propname to be accessible in select_by(). - - .. change:: - :tags: - :tickets: - - fix to typing in clause construction which specifically helps - type issues with polymorphic_union (CAST/ColumnClause propagates - its type to proxy columns) - - .. change:: - :tags: - :tickets: - - mapper compilation work ongoing, someday it'll work....moved - around the initialization of MapperProperty objects to be after - all mappers are created to better handle circular compilations. - do_init() method is called on all properties now which are more - aware of their "inherited" status if so. - - .. change:: - :tags: - :tickets: - - eager loads explicitly disallowed on self-referential relationships, or - relationships to an inheriting mapper (which is also self-referential) - - .. change:: - :tags: - :tickets: 244 - - reduced bind param size in query._get to appease the picky oracle - - .. change:: - :tags: - :tickets: 234 - - added 'checkfirst' argument to table.create()/table.drop(), as - well as table.exists() - - .. change:: - :tags: - :tickets: 245 - - some other ongoing fixes to inheritance - - .. change:: - :tags: - :tickets: - - attribute/backref/orphan/history-tracking tweaks as usual... - -.. changelog:: - :version: 0.2.5 - :released: Sat Jul 08 2006 - - .. change:: - :tags: - :tickets: - - fixed endless loop bug in select_by(), if the traversal hit - two mappers that referenced each other - - .. change:: - :tags: - :tickets: - - upgraded all unittests to insert './lib/' into sys.path, - working around new setuptools PYTHONPATH-killing behavior - - .. change:: - :tags: - :tickets: - - further fixes with attributes/dependencies/etc.... - - .. change:: - :tags: - :tickets: - - improved error handling for when DynamicMetaData is not connected - - .. change:: - :tags: - :tickets: - - MS-SQL support largely working (tested with pymssql) - - .. change:: - :tags: - :tickets: - - ordering of UPDATE and DELETE statements within groups is now - in order of primary key values, for more deterministic ordering - - .. change:: - :tags: - :tickets: - - after_insert/delete/update mapper extensions now called per object, - not per-object-per-table - - .. change:: - :tags: - :tickets: - - further fixes/refactorings to mapper compilation - -.. changelog:: - :version: 0.2.4 - :released: Tue Jun 27 2006 - - .. change:: - :tags: - :tickets: - - try/except when the mapper sets init.__name__ on a mapped class, - supports python 2.3 - - .. change:: - :tags: - :tickets: - - fixed bug where threadlocal engine would still autocommit - despite a transaction in progress - - .. change:: - :tags: - :tickets: - - lazy load and deferred load operations require the parent object - to be in a Session to do the operation; whereas before the operation - would just return a blank list or None, it now raises an exception. - - .. change:: - :tags: - :tickets: - - Session.update() is slightly more lenient if the session to which - the given object was formerly attached to was garbage collected; - otherwise still requires you explicitly remove the instance from - the previous Session. - - .. change:: - :tags: - :tickets: - - fixes to mapper compilation, checking for more error conditions - - .. change:: - :tags: - :tickets: - - small fix to eager loading combined with ordering/limit/offset - - .. change:: - :tags: - :tickets: 206 - - utterly remarkable: added a single space between 'CREATE TABLE' - and '(' since *that's how MySQL indicates a non- - reserved word tablename.....* - - .. change:: - :tags: - :tickets: - - more fixes to inheritance, related to many-to-many relations - properly saving - - .. change:: - :tags: - :tickets: - - fixed bug when specifying explicit module to mysql dialect - - .. change:: - :tags: - :tickets: - - when QueuePool times out it raises a TimeoutError instead of - erroneously making another connection - - .. change:: - :tags: - :tickets: - - Queue.Queue usage in pool has been replaced with a locally - modified version (works in py2.3/2.4!) that uses a threading.RLock - for a mutex. this is to fix a reported case where a ConnectionFairy's - __del__() method got called within the Queue's get() method, which - then returns its connection to the Queue via the put() method, - causing a reentrant hang unless threading.RLock is used. - - .. change:: - :tags: - :tickets: - - postgres will not place SERIAL keyword on a primary key column - if it has a foreign key constraint - - .. change:: - :tags: - :tickets: 221 - - cursor() method on ConnectionFairy allows db-specific extension - arguments to be propagated - - .. change:: - :tags: - :tickets: 225 - - lazy load bind params properly propagate column type - - .. change:: - :tags: - :tickets: - - new MySQL types: MSEnum, MSTinyText, MSMediumText, MSLongText, etc. - more support for MS-specific length/precision params in numeric types - patch courtesy Mike Bernson - - .. change:: - :tags: - :tickets: 224 - - some fixes to connection pool invalidate() - -.. changelog:: - :version: 0.2.3 - :released: Sat Jun 17 2006 - - .. change:: - :tags: - :tickets: - - overhaul to mapper compilation to be deferred. this allows mappers - to be constructed in any order, and their relationships to each - other are compiled when the mappers are first used. - - .. change:: - :tags: - :tickets: - - fixed a pretty big speed bottleneck in cascading behavior particularly - when backrefs were in use - - .. change:: - :tags: - :tickets: - - the attribute instrumentation module has been completely rewritten; its - now a large degree simpler and clearer, slightly faster. the "history" - of an attribute is no longer micromanaged with each change and is - instead part of a "CommittedState" object created when the - instance is first loaded. HistoryArraySet is gone, the behavior of - list attributes is now more open ended (i.e. they're not sets anymore). - - .. change:: - :tags: - :tickets: - - py2.4 "set" construct used internally, falls back to sets.Set when - "set" not available/ordering is needed. - - .. change:: - :tags: - :tickets: - - fix to transaction control, so that repeated rollback() calls - don't fail (was failing pretty badly when flush() would raise - an exception in a larger try/except transaction block) - - .. change:: - :tags: - :tickets: 151 - - "foreignkey" argument to relation() can also be a list. fixed - auto-foreignkey detection - - .. change:: - :tags: - :tickets: - - fixed bug where tables with schema names weren't getting indexed in - the MetaData object properly - - .. change:: - :tags: - :tickets: 207 - - fixed bug where Column with redefined "key" property wasn't getting - type conversion happening in the ResultProxy - - .. change:: - :tags: - :tickets: - - fixed 'port' attribute of URL to be an integer if present - - .. change:: - :tags: - :tickets: - - fixed old bug where if a many-to-many table mapped as "secondary" - had extra columns, delete operations didn't work - - .. change:: - :tags: - :tickets: - - bugfixes for mapping against UNION queries - - .. change:: - :tags: - :tickets: - - fixed incorrect exception class thrown when no DB driver present - - .. change:: - :tags: - :tickets: 138 - - added NonExistentTable exception thrown when reflecting a table - that doesn't exist - - .. change:: - :tags: - :tickets: - - small fix to ActiveMapper regarding one-to-one backrefs, other - refactorings - - .. change:: - :tags: - :tickets: - - overridden constructor in mapped classes gets __name__ and - __doc__ from the original class - - .. change:: - :tags: - :tickets: 200 - - fixed small bug in selectresult.py regarding mapper extension - - .. change:: - :tags: - :tickets: - - small tweak to cascade_mappers, not very strongly supported - function at the moment - - .. change:: - :tags: - :tickets: 202 - - some fixes to between(), column.between() to propagate typing - information better - - .. change:: - :tags: - :tickets: 203 - - if an object fails to be constructed, is not added to the - session - - .. change:: - :tags: - :tickets: - - CAST function has been made into its own clause object with - its own compilation function in ansicompiler; allows MySQL - to silently ignore most CAST calls since MySQL - seems to only support the standard CAST syntax with Date types. - MySQL-compatible CAST support for strings, ints, etc. a TODO - -.. changelog:: - :version: 0.2.2 - :released: Mon Jun 05 2006 - - .. change:: - :tags: - :tickets: 190 - - big improvements to polymorphic inheritance behavior, enabling it - to work with adjacency list table structures - - .. change:: - :tags: - :tickets: - - major fixes and refactorings to inheritance relationships overall, - more unit tests - - .. change:: - :tags: - :tickets: - - fixed "echo_pool" flag on create_engine() - - .. change:: - :tags: - :tickets: - - fix to docs, removed incorrect info that close() is unsafe to use - with threadlocal strategy (its totally safe !) - - .. change:: - :tags: - :tickets: 188 - - create_engine() can take URLs as string or unicode - - .. change:: - :tags: - :tickets: - - firebird support partially completed; - thanks to James Ralston and Brad Clements for their efforts. - - .. change:: - :tags: - :tickets: - - Oracle url translation was broken, fixed, will feed host/port/sid - into cx_oracle makedsn() if 'database' field is present, else uses - straight TNS name from the 'host' field - - .. change:: - :tags: - :tickets: - - fix to using unicode criterion for query.get()/query.load() - - .. change:: - :tags: - :tickets: - - count() function on selectables now uses table primary key or - first column instead of "1" for criterion, also uses label "rowcount" - instead of "count". - - .. change:: - :tags: - :tickets: - - got rudimental "mapping to multiple tables" functionality cleaned up, - more correctly documented - - .. change:: - :tags: - :tickets: - - restored global_connect() function, attaches to a DynamicMetaData - instance called "default_metadata". leaving MetaData arg to Table - out will use the default metadata. - - .. change:: - :tags: - :tickets: - - fixes to session cascade behavior, entity_name propagation - - .. change:: - :tags: - :tickets: - - reorganized unittests into subdirectories - - .. change:: - :tags: - :tickets: - - more fixes to threadlocal connection nesting patterns - -.. changelog:: - :version: 0.2.1 - :released: Mon May 29 2006 - - .. change:: - :tags: - :tickets: - - "pool" argument to create_engine() properly propagates - - .. change:: - :tags: - :tickets: - - fixes to URL, raises exception if not parsed, does not pass blank - fields along to the DB connect string (a string such as - user:host@/db was breaking on postgres) - - .. change:: - :tags: - :tickets: - - small fixes to Mapper when it inserts and tries to get - new primary key values back - - .. change:: - :tags: - :tickets: - - rewrote half of TLEngine, the ComposedSQLEngine used with - 'strategy="threadlocal"'. it now properly implements engine.begin()/ - engine.commit(), which nest fully with connection.begin()/trans.commit(). - added about six unittests. - - .. change:: - :tags: - :tickets: - - major "duh" in pool.Pool, forgot to put back the WeakValueDictionary. - unittest which was supposed to check for this was also silently missing - it. fixed unittest to ensure that ConnectionFairy properly falls out - of scope. - - .. change:: - :tags: - :tickets: - - placeholder dispose() method added to SingletonThreadPool, doesn't - do anything yet - - .. change:: - :tags: - :tickets: - - rollback() is automatically called when an exception is raised, - but only if there's no transaction in process (i.e. works more like - autocommit). - - .. change:: - :tags: - :tickets: - - fixed exception raise in sqlite if no sqlite module present - - .. change:: - :tags: - :tickets: - - added extra example detail for association object doc - - .. change:: - :tags: - :tickets: - - Connection adds checks for already being closed - -.. changelog:: - :version: 0.2.0 - :released: Sat May 27 2006 - - .. change:: - :tags: - :tickets: - - overhaul to Engine system so that what was formerly the SQLEngine - is now a ComposedSQLEngine which consists of a variety of components, - including a Dialect, ConnectionProvider, etc. This impacted all the - db modules as well as Session and Mapper. - - .. change:: - :tags: - :tickets: - - create_engine now takes only RFC-1738-style strings: - ``driver://user:password@host:port/database`` - - **update** this format is generally but not exactly RFC-1738, - including that underscores, not dashes or periods, are accepted in the - "scheme" portion. - - .. change:: - :tags: - :tickets: 152 - - total rewrite of connection-scoping methodology, Connection objects - can now execute clause elements directly, added explicit "close" as - well as support throughout Engine/ORM to handle closing properly, - no longer relying upon __del__ internally to return connections - to the pool. - - .. change:: - :tags: - :tickets: - - overhaul to Session interface and scoping. uses hibernate-style - methods, including query(class), save(), save_or_update(), etc. - no threadlocal scope is installed by default. Provides a binding - interface to specific Engines and/or Connections so that underlying - Schema objects do not need to be bound to an Engine. Added a basic - SessionTransaction object that can simplistically aggregate transactions - across multiple engines. - - .. change:: - :tags: - :tickets: - - overhaul to mapper's dependency and "cascade" behavior; dependency logic - factored out of properties.py into a separate module "dependency.py". - "cascade" behavior is now explicitly controllable, proper implementation - of "delete", "delete-orphan", etc. dependency system can now determine at - flush time if a child object has a parent or not so that it makes better - decisions on how that child should be updated in the DB with regards to deletes. - - .. change:: - :tags: - :tickets: - - overhaul to Schema to build upon MetaData object instead of an Engine. - Entire SQL/Schema system can be used with no Engines whatsoever, executed - solely by an explicit Connection object. the "bound" methodology exists via the - BoundMetaData for schema objects. ProxyEngine is generally not needed - anymore and is replaced by DynamicMetaData. - - .. change:: - :tags: - :tickets: 167 - - true polymorphic behavior implemented, fixes - - .. change:: - :tags: - :tickets: 147 - - "oid" system has been totally moved into compile-time behavior; - if they are used in an order_by where they are not available, the order_by - doesn't get compiled, fixes - - .. change:: - :tags: - :tickets: - - overhaul to packaging; "mapping" is now "orm", "objectstore" is now - "session", the old "objectstore" namespace gets loaded in via the - "threadlocal" mod if used - - .. change:: - :tags: - :tickets: - - mods now called in via "import ". extensions favored over - mods as mods are globally-monkeypatching - - .. change:: - :tags: - :tickets: 154 - - fix to add_property so that it propagates properties to inheriting - mappers - - .. change:: - :tags: - :tickets: - - backrefs create themselves against primary mapper of its originating - property, primary/secondary join arguments can be specified to override. - helps their usage with polymorphic mappers - - .. change:: - :tags: - :tickets: 31 - - "table exists" function has been implemented - - .. change:: - :tags: - :tickets: 98 - - "create_all/drop_all" added to MetaData object - - .. change:: - :tags: - :tickets: - - improvements and fixes to topological sort algorithm, as well as more - unit tests - - .. change:: - :tags: - :tickets: - - tutorial page added to docs which also can be run with a custom doctest - runner to ensure its properly working. docs generally overhauled to - deal with new code patterns - - .. change:: - :tags: - :tickets: - - many more fixes, refactorings. - - .. change:: - :tags: - :tickets: - - migration guide is available on the Wiki at - https://www.sqlalchemy.org/trac/wiki/02Migration diff --git a/doc/build/changelog/changelog_03.rst b/doc/build/changelog/changelog_03.rst deleted file mode 100644 index f2ffb81e3d..0000000000 --- a/doc/build/changelog/changelog_03.rst +++ /dev/null @@ -1,2927 +0,0 @@ - -============= -0.3 Changelog -============= - - -.. changelog:: - :version: 0.3.11 - :released: Sun Oct 14 2007 - - .. change:: - :tags: sql - :tickets: - - tweak DISTINCT precedence for clauses like - `func.count(t.c.col.distinct())` - - .. change:: - :tags: sql - :tickets: 719 - - Fixed detection of internal '$' characters in :bind$params - - .. change:: - :tags: sql - :tickets: 768 - - don't assume join criterion consists only of column objects - - .. change:: - :tags: sql - :tickets: 764 - - adjusted operator precedence of NOT to match '==' and others, so that - ~(x==y) produces NOT (x=y), which is compatible with MySQL < 5.0 - (doesn't like "NOT x=y") - - .. change:: - :tags: orm - :tickets: 687 - - added a check for joining from A->B using join(), along two - different m2m tables. this raises an error in 0.3 but is - possible in 0.4 when aliases are used. - - .. change:: - :tags: orm - :tickets: - - fixed small exception throw bug in Session.merge() - - .. change:: - :tags: orm - :tickets: - - fixed bug where mapper, being linked to a join where one table had - no PK columns, would not detect that the joined table had no PK. - - .. change:: - :tags: orm - :tickets: 769 - - fixed bugs in determining proper sync clauses from custom inherit - conditions - - .. change:: - :tags: orm - :tickets: 813 - - backref remove object operation doesn't fail if the other-side - collection doesn't contain the item, supports noload collections - - .. change:: - :tags: engine - :tickets: - - fixed another occasional race condition which could occur - when using pool with threadlocal setting - - .. change:: - :tags: mysql - :tickets: - - fixed specification of YEAR columns when generating schema - - .. change:: - :tags: mssql - :tickets: 679 - - added support for TIME columns (simulated using DATETIME) - - .. change:: - :tags: mssql - :tickets: 721 - - added support for BIGINT, MONEY, SMALLMONEY, UNIQUEIDENTIFIER and - SQL_VARIANT - - .. change:: - :tags: mssql - :tickets: 684 - - index names are now quoted when dropping from reflected tables - - .. change:: - :tags: mssql - :tickets: - - can now specify a DSN for PyODBC, using a URI like mssql:///?dsn=bob - - .. change:: - :tags: postgres - :tickets: - - when reflecting tables from alternate schemas, the "default" placed upon - the primary key, i.e. usually a sequence name, has the "schema" name - unconditionally quoted, so that schema names which need quoting are fine. - its slightly unnecessary for schema names which don't need quoting - but not harmful. - - .. change:: - :tags: sqlite - :tickets: - - passthrough for stringified dates - - .. change:: - :tags: firebird - :tickets: - - supports_sane_rowcount() set to False due to ticket #370 (right way). - - .. change:: - :tags: firebird - :tickets: - - fixed reflection of Column's nullable property. - - .. change:: - :tags: oracle - :tickets: 622, 751 - - removed LONG_STRING, LONG_BINARY from "binary" types, so type objects - don't try to read their values as LOB. - -.. changelog:: - :version: 0.3.10 - :released: Fri Jul 20 2007 - - .. change:: - :tags: general - :tickets: - - a new mutex that was added in 0.3.9 causes the pool_timeout - feature to fail during a race condition; threads would - raise TimeoutError immediately with no delay if many threads - push the pool into overflow at the same time. this issue has been - fixed. - - .. change:: - :tags: sql - :tickets: - - got connection-bound metadata to work with implicit execution - - .. change:: - :tags: sql - :tickets: 667 - - foreign key specs can have any character in their identifiers - - .. change:: - :tags: sql - :tickets: 664 - - added commutativity-awareness to binary clause comparisons to - each other, improves ORM lazy load optimization - - .. change:: - :tags: orm - :tickets: - - cleanup to connection-bound sessions, SessionTransaction - - .. change:: - :tags: postgres - :tickets: 571 - - fixed max identifier length (63) - -.. changelog:: - :version: 0.3.9 - :released: Sun Jul 15 2007 - - .. change:: - :tags: general - :tickets: 607 - - better error message for NoSuchColumnError - - .. change:: - :tags: general - :tickets: 428 - - finally figured out how to get setuptools version in, available - as sqlalchemy.__version__ - - .. change:: - :tags: general - :tickets: - - the various "engine" arguments, such as "engine", "connectable", - "engine_or_url", "bind_to", etc. are all present, but deprecated. - they all get replaced by the single term "bind". you also - set the "bind" of MetaData using - metadata.bind = - - .. change:: - :tags: ext - :tickets: - - iteration over dict association proxies is now dict-like, not - InstrumentedList-like (e.g. over keys instead of values) - - .. change:: - :tags: ext - :tickets: 597 - - association proxies no longer bind tightly to source collections, and are constructed with a thunk instead - - .. change:: - :tags: ext - :tickets: - - added selectone_by() to assignmapper - - .. change:: - :tags: orm - :tickets: - - forwards-compatibility with 0.4: added one(), first(), and - all() to Query. almost all Query functionality from 0.4 is - present in 0.3.9 for forwards-compat purposes. - - .. change:: - :tags: orm - :tickets: - - reset_joinpoint() really really works this time, promise ! lets - you re-join from the root: - query.join(['a', 'b']).filter().reset_joinpoint().\ - join(['a', 'c']).filter().all() - in 0.4 all join() calls start from the "root" - - .. change:: - :tags: orm - :tickets: 613 - - added synchronization to the mapper() construction step, to avoid - thread collisions when pre-existing mappers are compiling in a - different thread - - .. change:: - :tags: orm - :tickets: - - a warning is issued by Mapper when two primary key columns of the - same name are munged into a single attribute. this happens frequently - when mapping to joins (or inheritance). - - .. change:: - :tags: orm - :tickets: 598 - - synonym() properties are fully supported by all Query joining/ - with_parent operations - - .. change:: - :tags: orm - :tickets: - - fixed very stupid bug when deleting items with many-to-many - uselist=False relations - - .. change:: - :tags: orm - :tickets: - - remember all that stuff about polymorphic_union ? for - joined table inheritance ? Funny thing... - You sort of don't need it for joined table inheritance, you - can just string all the tables together via outerjoin(). - The UNION still applies if concrete tables are involved, - though (since nothing to join them on). - - .. change:: - :tags: orm - :tickets: - - small fix to eager loading to better work with eager loads - to polymorphic mappers that are using a straight "outerjoin" - clause - - .. change:: - :tags: sql - :tickets: - - ForeignKey to a table in a schema that's not the default schema - requires the schema to be explicit; i.e. ForeignKey('alt_schema.users.id') - - .. change:: - :tags: sql - :tickets: - - MetaData can now be constructed with an engine or url as the first - argument, just like BoundMetaData - - .. change:: - :tags: sql - :tickets: - - BoundMetaData is now deprecated, and MetaData is a direct substitute. - - .. change:: - :tags: sql - :tickets: - - DynamicMetaData has been renamed to ThreadLocalMetaData. the - DynamicMetaData name is deprecated and is an alias for ThreadLocalMetaData - or a regular MetaData if threadlocal=False - - .. change:: - :tags: sql - :tickets: - - composite primary key is represented as a non-keyed set to allow for - composite keys consisting of cols with the same name; occurs within a - Join. helps inheritance scenarios formulate correct PK. - - .. change:: - :tags: sql - :tickets: 185 - - improved ability to get the "correct" and most minimal set of primary key - columns from a join, equating foreign keys and otherwise equated columns. - this is also mostly to help inheritance scenarios formulate the best - choice of primary key columns. - - .. change:: - :tags: sql - :tickets: - - added 'bind' argument to Sequence.create()/drop(), ColumnDefault.execute() - - .. change:: - :tags: sql - :tickets: 650 - - columns can be overridden in a reflected table with a "key" - attribute different than the column's name, including for primary key - columns - - .. change:: - :tags: sql - :tickets: 657 - - fixed "ambiguous column" result detection, when dupe col names exist - in a result - - .. change:: - :tags: sql - :tickets: - - some enhancements to "column targeting", the ability to match a column - to a "corresponding" column in another selectable. this affects mostly - ORM ability to map to complex joins - - .. change:: - :tags: sql - :tickets: 619 - - MetaData and all SchemaItems are safe to use with pickle. slow - table reflections can be dumped into a pickled file to be reused later. - Just reconnect the engine to the metadata after unpickling. - - .. change:: - :tags: sql - :tickets: - - added a mutex to QueuePool's "overflow" calculation to prevent a race - condition that can bypass max_overflow - - .. change:: - :tags: sql - :tickets: 623 - - fixed grouping of compound selects to give correct results. will break - on sqlite in some cases, but those cases were producing incorrect - results anyway, sqlite doesn't support grouped compound selects - - .. change:: - :tags: sql - :tickets: 620 - - fixed precedence of operators so that parenthesis are correctly applied - - .. change:: - :tags: sql - :tickets: 545 - - calling .in_() (i.e. with no arguments) will return - "CASE WHEN ( IS NULL) THEN NULL ELSE 0 END = 1)", so that - NULL or False is returned in all cases, rather than throwing an error - - .. change:: - :tags: sql - :tickets: - - fixed "where"/"from" criterion of select() to accept a unicode string - in addition to regular string - both convert to text() - - .. change:: - :tags: sql - :tickets: 558 - - added standalone distinct() function in addition to column.distinct() - - .. change:: - :tags: sql - :tickets: - - result.last_inserted_ids() should return a list that is identically - sized to the primary key constraint of the table. values that were - "passively" created and not available via cursor.lastrowid will be None. - - .. change:: - :tags: sql - :tickets: 589 - - long-identifier detection fixed to use > rather than >= for - max ident length - - .. change:: - :tags: sql - :tickets: 593 - - fixed bug where selectable.corresponding_column(selectable.c.col) - would not return selectable.c.col, if the selectable is a join - of a table and another join involving the same table. messed - up ORM decision making - - .. change:: - :tags: sql - :tickets: 595 - - added Interval type to types.py - - .. change:: - :tags: mysql - :tickets: 625 - - fixed catching of some errors that imply a dropped connection - - .. change:: - :tags: mysql - :tickets: 624 - - fixed escaping of the modulo operator - - .. change:: - :tags: mysql - :tickets: 590 - - added 'fields' to reserved words - - .. change:: - :tags: mysql - :tickets: - - various reflection enhancement/fixes - - .. change:: - :tags: oracle - :tickets: 604 - - datetime fixes: got subsecond TIMESTAMP to work, - added OracleDate which supports types.Date with only year/month/day - - .. change:: - :tags: oracle - :tickets: - - added dialect flag "auto_convert_lobs", defaults to True; will cause any - LOB objects detected in a result set to be forced into OracleBinary - so that the LOB is read() automatically, if no typemap was present - (i.e., if a textual execute() was issued). - - .. change:: - :tags: oracle - :tickets: 624 - - mod operator '%' produces MOD - - .. change:: - :tags: oracle - :tickets: 542 - - converts cx_oracle datetime objects to Python datetime.datetime when - Python 2.3 used - - .. change:: - :tags: oracle - :tickets: - - fixed unicode conversion in Oracle TEXT type - - .. change:: - :tags: postgres - :tickets: 624 - - fixed escaping of the modulo operator - - .. change:: - :tags: postgres - :tickets: 570 - - added support for reflection of domains - - .. change:: - :tags: postgres - :tickets: - - types which are missing during reflection resolve to Null type - instead of raising an error - - .. change:: - :tags: postgres - :tickets: - - the fix in "schema" above fixes reflection of foreign keys from an - alt-schema table to a public schema table - - .. change:: - :tags: sqlite - :tickets: - - rearranged dialect initialization so it has time to warn about pysqlite1 - being too old. - - .. change:: - :tags: sqlite - :tickets: - - sqlite better handles datetime/date/time objects mixed and matched - with various Date/Time/DateTime columns - - .. change:: - :tags: sqlite - :tickets: 603 - - string PK column inserts don't get overwritten with OID - - .. change:: - :tags: mssql - :tickets: 634 - - fix port option handling for pyodbc - - .. change:: - :tags: mssql - :tickets: - - now able to reflect start and increment values for identity columns - - .. change:: - :tags: mssql - :tickets: - - preliminary support for using scope_identity() with pyodbc - -.. changelog:: - :version: 0.3.8 - :released: Sat Jun 02 2007 - - .. change:: - :tags: engines - :tickets: - - added detach() to Connection, allows underlying DBAPI connection - to be detached from its pool, closing on dereference/close() - instead of being reused by the pool. - - .. change:: - :tags: engines - :tickets: - - added invalidate() to Connection, immediately invalidates the - Connection and its underlying DBAPI connection. - - .. change:: - :tags: sql - :tickets: - - _Label class overrides compare_self to return its ultimate - object. meaning, if you say someexpr.label('foo') == 5, it - produces the correct "someexpr == 5". - - .. change:: - :tags: sql - :tickets: - - _Label propagates "_hide_froms()" so that scalar selects - behave more properly with regards to FROM clause #574 - - .. change:: - :tags: sql - :tickets: - - fix to long name generation when using oid_column as an order by - (oids used heavily in mapper queries) - - .. change:: - :tags: sql - :tickets: - - significant speed improvement to ResultProxy, pre-caches - TypeEngine dialect implementations and saves on function calls - per column - - .. change:: - :tags: sql - :tickets: - - parenthesis are applied to clauses via a new _Grouping - construct. uses operator precedence to more intelligently apply - parenthesis to clauses, provides cleaner nesting of clauses - (doesn't mutate clauses placed in other clauses, i.e. no 'parens' - flag) - - .. change:: - :tags: sql - :tickets: - - added 'modifier' keyword, works like func. except does not - add parenthesis. e.g. select([modifier.DISTINCT(...)]) etc. - - .. change:: - :tags: sql - :tickets: 578 - - removed "no group by's in a select that's part of a UNION" - restriction - - .. change:: - :tags: orm - :tickets: - - added reset_joinpoint() method to Query, moves the "join point" - back to the starting mapper. 0.4 will change the behavior of - join() to reset the "join point" in all cases so this is an - interim method. for forwards compatibility, ensure joins across - multiple relations are specified using a single join(), i.e. - join(['a', 'b', 'c']). - - .. change:: - :tags: orm - :tickets: - - fixed bug in query.instances() that wouldn't handle more than - on additional mapper or one additional column. - - .. change:: - :tags: orm - :tickets: - - "delete-orphan" no longer implies "delete". ongoing effort to - separate the behavior of these two operations. - - .. change:: - :tags: orm - :tickets: - - many-to-many relationships properly set the type of bind params - for delete operations on the association table - - .. change:: - :tags: orm - :tickets: - - many-to-many relationships check that the number of rows deleted - from the association table by a delete operation matches the - expected results - - .. change:: - :tags: orm - :tickets: - - session.get() and session.load() propagate \**kwargs through to - query - - .. change:: - :tags: orm - :tickets: 577 - - fix to polymorphic query which allows the original - polymorphic_union to be embedded into a correlated subquery - - .. change:: - :tags: orm - :tickets: - - fix to select_by(=) -style joins in - conjunction with many-to-many relationships, bug introduced in - r2556 - - .. change:: - :tags: orm - :tickets: - - the "primary_key" argument to mapper() is propagated to the - "polymorphic" mapper. primary key columns in this list get - normalized to that of the mapper's local table. - - .. change:: - :tags: orm - :tickets: - - restored logging of "lazy loading clause" under - sa.orm.strategies logger, got removed in 0.3.7 - - .. change:: - :tags: orm - :tickets: - - improved support for eagerloading of properties off of mappers - that are mapped to select() statements; i.e. eagerloader is - better at locating the correct selectable with which to attach - its LEFT OUTER JOIN. - - .. change:: - :tags: mysql - :tickets: - - Nearly all MySQL column types are now supported for declaration - and reflection. Added NCHAR, NVARCHAR, VARBINARY, TINYBLOB, - LONGBLOB, YEAR - - .. change:: - :tags: mysql - :tickets: - - The sqltypes.Binary passthrough now always builds a BLOB, - avoiding problems with very old database versions - - .. change:: - :tags: mysql - :tickets: - - support for column-level CHARACTER SET and COLLATE declarations, - as well as ASCII, UNICODE, NATIONAL and BINARY shorthand. - - .. change:: - :tags: firebird - :tickets: - - set max identifier length to 31 - - .. change:: - :tags: firebird - :tickets: - - supports_sane_rowcount() set to False due to ticket #370. - versioned_id_col feature won't work in FB. - - .. change:: - :tags: firebird - :tickets: - - some execution fixes - - .. change:: - :tags: firebird - :tickets: - - new association proxy implementation, implementing complete - proxies to list, dict and set-based relation collections - - .. change:: - :tags: firebird - :tickets: - - added orderinglist, a custom list class that synchronizes an - object attribute with that object's position in the list - - .. change:: - :tags: firebird - :tickets: - - small fix to SelectResultsExt to not bypass itself during - select(). - - .. change:: - :tags: firebird - :tickets: - - added filter(), filter_by() to assignmapper - -.. changelog:: - :version: 0.3.7 - :released: Sun Apr 29 2007 - - .. change:: - :tags: engines - :tickets: - - warnings module used for issuing warnings (instead of logging) - - .. change:: - :tags: engines - :tickets: 480 - - cleanup of DBAPI import strategies across all engines - - .. change:: - :tags: engines - :tickets: - - refactoring of engine internals which reduces complexity, - number of codepaths; places more state inside of ExecutionContext - to allow more dialect control of cursor handling, result sets. - ResultProxy totally refactored and also has two versions of - "buffered" result sets used for different purposes. - - .. change:: - :tags: engines - :tickets: 514 - - server side cursor support fully functional in postgres. - - .. change:: - :tags: engines - :tickets: - - improved framework for auto-invalidation of connections that have - lost their underlying database, via dialect-specific detection - of exceptions corresponding to that database's disconnect - related error messages. Additionally, when a "connection no - longer open" condition is detected, the entire connection pool - is discarded and replaced with a new instance. #516 - - .. change:: - :tags: engines - :tickets: 521 - - the dialects within sqlalchemy.databases become a setuptools - entry points. loading the built-in database dialects works the - same as always, but if none found will fall back to trying - pkg_resources to load an external module - - .. change:: - :tags: engines - :tickets: - - Engine contains a "url" attribute referencing the url.URL object - used by create_engine(). - - .. change:: - :tags: sql - :tickets: - - keys() of result set columns are not lowercased, come back - exactly as they're expressed in cursor.description. note this - causes colnames to be all caps in oracle. - - .. change:: - :tags: sql - :tickets: - - preliminary support for unicode table names, column names and - SQL statements added, for databases which can support them. - Works with sqlite and postgres so far. MySQL *mostly* works - except the has_table() function does not work. Reflection - works too. - - .. change:: - :tags: sql - :tickets: 522 - - the Unicode type is now a direct subclass of String, which now - contains all the "convert_unicode" logic. This helps the variety - of unicode situations that occur in db's such as MS-SQL to be - better handled and allows subclassing of the Unicode datatype. - - .. change:: - :tags: sql - :tickets: - - ClauseElements can be used in in_() clauses now, such as bind - parameters, etc. #476 - - .. change:: - :tags: sql - :tickets: - - reverse operators implemented for `CompareMixin` elements, - allows expressions like "5 + somecolumn" etc. #474 - - .. change:: - :tags: sql - :tickets: - - the "where" criterion of an update() and delete() now correlates - embedded select() statements against the table being updated or - deleted. this works the same as nested select() statement - correlation, and can be disabled via the correlate=False flag on - the embedded select(). - - .. change:: - :tags: sql - :tickets: 512 - - column labels are now generated in the compilation phase, which - means their lengths are dialect-dependent. So on oracle a label - that gets truncated to 30 chars will go out to 63 characters - on postgres. Also, the true labelname is always attached as the - accessor on the parent Selectable so there's no need to be aware - of the "truncated" label names. - - .. change:: - :tags: sql - :tickets: - - column label and bind param "truncation" also generate - deterministic names now, based on their ordering within the - full statement being compiled. this means the same statement - will produce the same string across application restarts and - allowing DB query plan caching to work better. - - .. change:: - :tags: sql - :tickets: 513 - - the "mini" column labels generated when using subqueries, which - are to work around glitchy SQLite behavior that doesn't understand - "foo.id" as equivalent to "id", are now only generated in the case - that those named columns are selected from (part of) - - .. change:: - :tags: sql - :tickets: - - the label() method on ColumnElement will properly propagate the - TypeEngine of the base element out to the label, including a label() - created from a scalar=True select() statement. - - .. change:: - :tags: sql - :tickets: 513 - - MS-SQL better detects when a query is a subquery and knows not to - generate ORDER BY phrases for those - - .. change:: - :tags: sql - :tickets: 505 - - fix for fetchmany() "size" argument being positional in most - dbapis - - .. change:: - :tags: sql - :tickets: - - sending None as an argument to func. will produce - an argument of NULL - - .. change:: - :tags: sql - :tickets: - - query strings in unicode URLs get keys encoded to ascii - for \**kwargs compat - - .. change:: - :tags: sql - :tickets: 523 - - slight tweak to raw execute() change to also support tuples - for positional parameters, not just lists - - .. change:: - :tags: sql - :tickets: - - fix to case() construct to propagate the type of the first - WHEN condition as the return type of the case statement - - .. change:: - :tags: orm - :tickets: - - fixed critical issue when, after options(eagerload()) is used, - the mapper would then always apply query "wrapping" behavior - for all subsequent LIMIT/OFFSET/DISTINCT queries, even if no - eager loading was applied on those subsequent queries. - - .. change:: - :tags: orm - :tickets: 541 - - added query.with_parent(someinstance) method. searches for - target instance using lazy join criterion from parent instance. - takes optional string "property" to isolate the desired relation. - also adds static Query.query_from_parent(instance, property) - version. - - .. change:: - :tags: orm - :tickets: 554 - - improved query.XXX_by(someprop=someinstance) querying to use - similar methodology to with_parent, i.e. using the "lazy" clause - which prevents adding the remote instance's table to the SQL, - thereby making more complex conditions possible - - .. change:: - :tags: orm - :tickets: - - added generative versions of aggregates, i.e. sum(), avg(), etc. - to query. used via query.apply_max(), apply_sum(), etc. - #552 - - .. change:: - :tags: orm - :tickets: - - fix to using distinct() or distinct=True in combination with - join() and similar - - .. change:: - :tags: orm - :tickets: - - corresponding to label/bindparam name generation, eager loaders - generate deterministic names for the aliases they create using - md5 hashes. - - .. change:: - :tags: orm - :tickets: - - improved/fixed custom collection classes when giving it "set"/ - "sets.Set" classes or subclasses (was still looking for append() - methods on them during lazy loads) - - .. change:: - :tags: orm - :tickets: - - restored old "column_property()" ORM function (used to be called - "column()") to force any column expression to be added as a property - on a mapper, particularly those that aren't present in the mapped - selectable. this allows "scalar expressions" of any kind to be - added as relations (though they have issues with eager loads). - - .. change:: - :tags: orm - :tickets: 533 - - fix to many-to-many relationships targeting polymorphic mappers - - .. change:: - :tags: orm - :tickets: 543 - - making progress with session.merge() as well as combining its - usage with entity_name - - .. change:: - :tags: orm - :tickets: - - the usual adjustments to relationships between inheriting mappers, - in this case establishing relation()s to subclass mappers where - the join conditions come from the superclass' table - - .. change:: - :tags: informix - :tickets: - - informix support added ! courtesy James Zhang, who put a ton - of effort in. - - .. change:: - :tags: sqlite - :tickets: - - removed silly behavior where sqlite would reflect UNIQUE indexes - as part of the primary key (?!) - - .. change:: - :tags: oracle - :tickets: - - small fix to allow successive compiles of the same SELECT object - which features LIMIT/OFFSET. oracle dialect needs to modify - the object to have ROW_NUMBER OVER and wasn't performing - the full series of steps on successive compiles. - - .. change:: - :tags: mysql - :tickets: - - support for SSL arguments given as inline within URL query string, - prefixed with "ssl\_", courtesy terjeros@gmail.com. - - .. change:: - :tags: , mysql - :tickets: - - mysql uses "DESCRIBE.", catching exceptions - if table doesn't exist, in order to determine if a table exists. - this supports unicode table names as well as schema names. tested - with MySQL5 but should work with 4.1 series as well. (#557) - - .. change:: - :tags: extensions - :tickets: - - big fix to AssociationProxy so that multiple AssociationProxy - objects can be associated with a single association collection. - - .. change:: - :tags: extensions - :tickets: - - assign_mapper names methods according to their keys (i.e. __name__) - #551 - - .. change:: - :tags: mssql - :tickets: - - pyodbc is now the preferred DB-API for MSSQL, and if no module is - specifically requested, will be loaded first on a module probe. - - .. change:: - :tags: mssql - :tickets: - - The @@SCOPE_IDENTITY is now used instead of @@IDENTITY. This - behavior may be overridden with the engine_connect - "use_scope_identity" keyword parameter, which may also be specified - in the dburi. - -.. changelog:: - :version: 0.3.6 - :released: Fri Mar 23 2007 - - .. change:: - :tags: sql - :tickets: - - bindparam() names are now repeatable! specify two - distinct bindparam()s with the same name in a single statement, - and the key will be shared. proper positional/named args translate - at compile time. for the old behavior of "aliasing" bind parameters - with conflicting names, specify "unique=True" - this option is - still used internally for all the auto-generated (value-based) - bind parameters. - - .. change:: - :tags: sql - :tickets: - - slightly better support for bind params as column clauses, either - via bindparam() or via literal(), i.e. select([literal('foo')]) - - .. change:: - :tags: sql - :tickets: - - MetaData can bind to an engine either via "url" or "engine" kwargs - to constructor, or by using connect() method. BoundMetaData is - identical to MetaData except engine_or_url param is required. - DynamicMetaData is the same and provides thread-local connections be - default. - - .. change:: - :tags: sql - :tickets: - - exists() becomes usable as a standalone selectable, not just in a - WHERE clause, i.e. exists([columns], criterion).select() - - .. change:: - :tags: sql - :tickets: - - correlated subqueries work inside of ORDER BY, GROUP BY - - .. change:: - :tags: sql - :tickets: - - fixed function execution with explicit connections, i.e. - conn.execute(func.dosomething()) - - .. change:: - :tags: sql - :tickets: - - use_labels flag on select() won't auto-create labels for literal text - column elements, since we can make no assumptions about the text. to - create labels for literal columns, you can say "somecol AS - somelabel", or use literal_column("somecol").label("somelabel") - - .. change:: - :tags: sql - :tickets: - - quoting won't occur for literal columns when they are "proxied" into - the column collection for their selectable (is_literal flag is - propagated). literal columns are specified via - literal_column("somestring"). - - .. change:: - :tags: sql - :tickets: - - added "fold_equivalents" boolean argument to Join.select(), which - removes 'duplicate' columns from the resulting column clause that - are known to be equivalent based on the join condition. this is of - great usage when constructing subqueries of joins which Postgres - complains about if duplicate column names are present. - - .. change:: - :tags: sql - :tickets: 503 - - fixed use_alter flag on ForeignKeyConstraint - - .. change:: - :tags: sql - :tickets: 506 - - fixed usage of 2.4-only "reversed" in topological.py - - .. change:: - :tags: sql - :tickets: 501 - - for hackers, refactored the "visitor" system of ClauseElement and - SchemaItem so that the traversal of items is controlled by the - ClauseVisitor itself, using the method visitor.traverse(item). - accept_visitor() methods can still be called directly but will not - do any traversal of child items. ClauseElement/SchemaItem now have a - configurable get_children() method to return the collection of child - elements for each parent object. This allows the full traversal of - items to be clear and unambiguous (as well as loggable), with an - easy method of limiting a traversal (just pass flags which are - picked up by appropriate get_children() methods). - - .. change:: - :tags: sql - :tickets: - - the "else\_" parameter to the case statement now properly works when - set to zero. - - .. change:: - :tags: orm - :tickets: - - the full featureset of the SelectResults extension has been merged - into a new set of methods available off of Query. These methods - all provide "generative" behavior, whereby the Query is copied - and a new one returned with additional criterion added. - The new methods include: - - * filter() - applies select criterion to the query - * filter_by() - applies "by"-style criterion to the query - * avg() - return the avg() function on the given column - * join() - join to a property (or across a list of properties) - * outerjoin() - like join() but uses LEFT OUTER JOIN - * limit()/offset() - apply LIMIT/OFFSET range-based access - which applies limit/offset: session.query(Foo)[3:5] - * distinct() - apply DISTINCT - * list() - evaluate the criterion and return results - - no incompatible changes have been made to Query's API and no methods - have been deprecated. Existing methods like select(), select_by(), - get(), get_by() all execute the query at once and return results - like they always did. join_to()/join_via() are still there although - the generative join()/outerjoin() methods are easier to use. - - .. change:: - :tags: orm - :tickets: - - the return value for multiple mappers used with instances() now - returns a cartesian product of the requested list of mappers, - represented as a list of tuples. this corresponds to the documented - behavior. So that instances match up properly, the "uniquing" is - disabled when this feature is used. - - .. change:: - :tags: orm - :tickets: - - Query has add_entity() and add_column() generative methods. these - will add the given mapper/class or ColumnElement to the query at - compile time, and apply them to the instances() method. the user is - responsible for constructing reasonable join conditions (otherwise - you can get full cartesian products). result set is the list of - tuples, non-uniqued. - - .. change:: - :tags: orm - :tickets: - - strings and columns can also be sent to the \*args of instances() - where those exact result columns will be part of the result tuples. - - .. change:: - :tags: orm - :tickets: - - a full select() construct can be passed to query.select() (which - worked anyway), but also query.selectfirst(), query.selectone() - which will be used as is (i.e. no query is compiled). works - similarly to sending the results to instances(). - - .. change:: - :tags: orm - :tickets: 495 - - eager loading will not "aliasize" "order by" clauses that were - placed in the select statement by something other than the eager - loader itself, to fix possibility of dupe columns as illustrated in. however, this means you have to be more careful with - the columns placed in the "order by" of Query.select(), that you - have explicitly named them in your criterion (i.e. you can't rely on - the eager loader adding them in for you) - - .. change:: - :tags: orm - :tickets: - - added a handy multi-use "identity_key()" method to Session, allowing - the generation of identity keys for primary key values, instances, - and rows, courtesy Daniel Miller - - .. change:: - :tags: orm - :tickets: 249 - - many-to-many table will be properly handled even for operations that - occur on the "backref" side of the operation - - .. change:: - :tags: orm - :tickets: 492 - - added "refresh-expire" cascade. allows refresh() and - expire() calls to propagate along relationships. - - .. change:: - :tags: orm - :tickets: 493 - - more fixes to polymorphic relations, involving proper lazy-clause - generation on many-to-one relationships to polymorphic mappers. also fixes to detection of "direction", more specific - targeting of columns that belong to the polymorphic union vs. those - that don't. - - .. change:: - :tags: orm - :tickets: - - some fixes to relationship calcs when using "viewonly=True" to pull - in other tables into the join condition which aren't parent of the - relationship's parent/child mappings - - .. change:: - :tags: orm - :tickets: - - flush fixes on cyclical-referential relationships that contain - references to other instances outside of the cyclical chain, when - some of the objects in the cycle are not actually part of the flush - - .. change:: - :tags: orm - :tickets: 500 - - put an aggressive check for "flushing object A with a collection of - B's, but you put a C in the collection" error condition - **even if - C is a subclass of B**, unless B's mapper loads polymorphically. - Otherwise, the collection will later load a "B" which should be a - "C" (since its not polymorphic) which breaks in bi-directional - relationships (i.e. C has its A, but A's backref will lazyload it as - a different instance of type "B") This check is going - to bite some of you who do this without issues, so the error message - will also document a flag "enable_typechecks=False" to disable this - checking. But be aware that bi-directional relationships in - particular become fragile without this check. - - .. change:: - :tags: extensions - :tickets: 472 - - options() method on SelectResults now implemented "generatively" - like the rest of the SelectResults methods. But - you're going to just use Query now anyway. - - .. change:: - :tags: extensions - :tickets: - - query() method is added by assignmapper. this helps with - navigating to all the new generative methods on Query. - - .. change:: - :tags: ms-sql - :tickets: - - removed seconds input on DATE column types (probably - should remove the time altogether) - - .. change:: - :tags: ms-sql - :tickets: - - null values in float fields no longer raise errors - - .. change:: - :tags: ms-sql - :tickets: - - LIMIT with OFFSET now raises an error (MS-SQL has no OFFSET support) - - .. change:: - :tags: ms-sql - :tickets: 509 - - added an facility to use the MSSQL type VARCHAR(max) instead of TEXT - for large unsized string fields. Use the new "text_as_varchar" to - turn it on. - - .. change:: - :tags: ms-sql - :tickets: - - ORDER BY clauses without a LIMIT are now stripped in subqueries, as - MS-SQL forbids this usage - - .. change:: - :tags: ms-sql - :tickets: 480 - - cleanup of module importing code; specifiable DB-API module; more - explicit ordering of module preferences. - - .. change:: - :tags: oracle - :tickets: - - got binary working for any size input ! cx_oracle works fine, - it was my fault as BINARY was being passed and not BLOB for - setinputsizes (also unit tests weren't even setting input sizes). - - .. change:: - :tags: oracle - :tickets: - - also fixed CLOB read/write on a separate changeset. - - .. change:: - :tags: oracle - :tickets: - - auto_setinputsizes defaults to True for Oracle, fixed cases where - it improperly propagated bad types. - - .. change:: - :tags: mysql - :tickets: - - added a catchall \**kwargs to MSString, to help reflection of - obscure types (like "varchar() binary" in MS 4.0) - - .. change:: - :tags: mysql - :tickets: - - added explicit MSTimeStamp type which takes effect when using - types.TIMESTAMP. - -.. changelog:: - :version: 0.3.5 - :released: Thu Feb 22 2007 - - .. change:: - :tags: sql - :tickets: - - the value of "case_sensitive" defaults to True now, regardless of the - casing of the identifier, unless specifically set to False. this is - because the object might be label'ed as something else which does - contain mixed case, and propagating "case_sensitive=False" breaks that. - Other fixes to quoting when using labels and "fake" column objects - - .. change:: - :tags: sql - :tickets: - - added a "supports_execution()" method to ClauseElement, so that - individual kinds of clauses can express if they are appropriate for - executing...such as, you can execute a "select", but not a "Table" or a - "Join". - - .. change:: - :tags: sql - :tickets: - - fixed argument passing to straight textual execute() on engine, - connection. can handle \*args or a list instance for positional, \**kwargs - or a dict instance for named args, or a list of list or dicts to invoke - executemany() - - .. change:: - :tags: sql - :tickets: - - small fix to BoundMetaData to accept unicode or string URLs - - .. change:: - :tags: sql - :tickets: 466 - - fixed named PrimaryKeyConstraint generation courtesy - andrija at gmail - - .. change:: - :tags: sql - :tickets: 464 - - fixed generation of CHECK constraints on columns - - .. change:: - :tags: sql - :tickets: - - fixes to tometadata() operation to propagate Constraints at column and - table level - - .. change:: - :tags: oracle - :tickets: 436 - - when returning "rowid" as the ORDER BY column or in use with ROW_NUMBER - OVER, oracle dialect checks the selectable its being applied to and will - switch to table PK if not applicable, i.e. for a UNION. checking for - DISTINCT, GROUP BY (other places that rowid is invalid) still a TODO. - allows polymorphic mappings to function. - - .. change:: - :tags: oracle - :tickets: - - sequences on a non-pk column will properly fire off on INSERT - - .. change:: - :tags: oracle - :tickets: 435 - - added PrefetchingResultProxy support to pre-fetch LOB columns when they - are known to be present, fixes - - .. change:: - :tags: oracle - :tickets: 379 - - implemented reflection of tables based on synonyms, including across - dblinks - - .. change:: - :tags: oracle - :tickets: 363 - - issues a log warning when a related table can't be reflected due to - certain permission errors - - .. change:: - :tags: mysql - :tickets: - - fix to reflection on older DB's that might return array() type for - "show variables like" statements - - .. change:: - :tags: postgres - :tickets: 442 - - better reflection of sequences for alternate-schema Tables - - .. change:: - :tags: postgres - :tickets: - - sequences on a non-pk column will properly fire off on INSERT - - .. change:: - :tags: postgres - :tickets: 460, 444 - - added PGInterval type, PGInet type - - .. change:: - :tags: mssql - :tickets: 419 - - preliminary support for pyodbc (Yay!) - - .. change:: - :tags: mssql - :tickets: 298 - - better support for NVARCHAR types added - - .. change:: - :tags: mssql - :tickets: - - fix for commit logic on pymssql - - .. change:: - :tags: mssql - :tickets: 456 - - fix for query.get() with schema - - .. change:: - :tags: mssql - :tickets: 473 - - fix for non-integer relationships - - .. change:: - :tags: mssql - :tickets: 419 - - DB-API module now selectable at run-time - - .. change:: - :tags: tickets:422, 481, 415, mssql - :tickets: - - now passes many more unit tests - - .. change:: - :tags: mssql - :tickets: 479 - - better unittest compatibility with ANSI functions - - .. change:: - :tags: mssql - :tickets: 415 - - improved support for implicit sequence PK columns with auto-insert - - .. change:: - :tags: mssql - :tickets: 371 - - fix for blank password in adodbapi - - .. change:: - :tags: mssql - :tickets: 481 - - fixes to get unit tests working with pyodbc - - .. change:: - :tags: mssql - :tickets: - - fix to auto_identity_insert on db-url query - - .. change:: - :tags: mssql - :tickets: - - added query_timeout to db-url query params. currently works only for - pymssql - - .. change:: - :tags: mssql - :tickets: - - tested with pymssql 0.8.0 (which is now LGPL) - - .. change:: - :tags: orm, bugs - :tickets: 441, 448, 439 - - another refactoring to relationship calculation. Allows more accurate - ORM behavior with relationships from/to/between mappers, particularly - polymorphic mappers, also their usage with Query, SelectResults. tickets - include,,. - - .. change:: - :tags: orm, bugs - :tickets: - - removed deprecated method of specifying custom collections on classes; - you must now use the "collection_class" option. the old way was - beginning to produce conflicts when people used assign_mapper(), which - now patches an "options" method, in conjunction with a relationship - named "options". (relationships take precedence over monkeypatched - assign_mapper methods). - - .. change:: - :tags: orm, bugs - :tickets: 454 - - extension() query option propagates to Mapper._instance() method so that - all loading-related methods get called - - .. change:: - :tags: orm, bugs - :tickets: - - eager relation to an inheriting mapper won't fail if no rows returned for - the relationship. - - .. change:: - :tags: orm, bugs - :tickets: 486 - - eager relation loading bug fixed for eager relation on multiple - descendant classes - - .. change:: - :tags: orm, bugs - :tickets: 423 - - fix for very large topological sorts, courtesy ants.aasma at gmail - - .. change:: - :tags: orm, bugs - :tickets: - - eager loading is slightly more strict about detecting "self-referential" - relationships, specifically between polymorphic mappers. this results in - an "eager degrade" to lazy loading. - - .. change:: - :tags: orm, bugs - :tickets: 449 - - improved support for complex queries embedded into "where" criterion for - query.select() - - .. change:: - :tags: orm, bugs - :tickets: 485 - - mapper options like eagerload(), lazyload(), deferred(), will work for - "synonym()" relationships - - .. change:: - :tags: orm, bugs - :tickets: 445 - - fixed bug where cascade operations incorrectly included deleted - collection items in the cascade - - .. change:: - :tags: orm, bugs - :tickets: 478 - - fixed relationship deletion error when one-to-many child item is moved - to a new parent in a single unit of work - - .. change:: - :tags: orm, bugs - :tickets: - - fixed relationship deletion error where parent/child with a single - column as PK/FK on the child would raise a "blank out the primary key" - error, if manually deleted or "delete" cascade without "delete-orphan" - was used - - .. change:: - :tags: orm, bugs - :tickets: - - fix to deferred so that load operation doesn't mistakenly occur when only - PK col attributes are set - - .. change:: - :tags: orm, enhancements - :tickets: 385 - - implemented foreign_keys argument to mapper. use in - conjunction with primaryjoin/secondaryjoin arguments to specify/override - foreign keys defined on the Table instance. - - .. change:: - :tags: orm, enhancements - :tickets: - - contains_eager('foo') automatically implies eagerload('foo') - - .. change:: - :tags: orm, enhancements - :tickets: - - added "alias" argument to contains_eager(). use it to specify the string - name or Alias instance of an alias used in the query for the eagerly - loaded child items. easier to use than "decorator" - - .. change:: - :tags: orm, enhancements - :tickets: - - added "contains_alias()" option for result set mapping to an alias of - the mapped table - - .. change:: - :tags: orm, enhancements - :tickets: 468 - - added support for py2.5 "with" statement with SessionTransaction - - .. change:: - :tags: extensions - :tickets: - - added distinct() method to SelectResults. generally should only make a - difference when using count(). - - .. change:: - :tags: extensions - :tickets: 472 - - added options() method to SelectResults, equivalent to query.options() - - .. change:: - :tags: extensions - :tickets: 462 - - added optional __table_opts__ dictionary to ActiveMapper, will send kw - options to Table objects - - .. change:: - :tags: extensions - :tickets: 467 - - added selectfirst(), selectfirst_by() to assign_mapper - -.. changelog:: - :version: 0.3.4 - :released: Tue Jan 23 2007 - - .. change:: - :tags: general - :tickets: - - global "insure"->"ensure" change. in US english "insure" is actually - largely interchangeable with "ensure" (so says the dictionary), so I'm not - completely illiterate, but its definitely sub-optimal to "ensure" which is - non-ambiguous. - - .. change:: - :tags: sql - :tickets: - - added "fetchmany()" support to ResultProxy - - .. change:: - :tags: sql - :tickets: - - added support for column "key" attribute to be usable in - row[]/row. - - .. change:: - :tags: sql - :tickets: - - changed "BooleanExpression" to subclass from "BinaryExpression", so that - boolean expressions can also follow column-clause behaviors (i.e. label(), - etc). - - .. change:: - :tags: sql - :tickets: - - trailing underscores are trimmed from func. calls, such as func.if_() - - .. change:: - :tags: sql - :tickets: - - fix to correlation of subqueries when the column list of the select - statement is constructed with individual calls to append_column(); this - fixes an ORM bug whereby nested select statements were not getting - correlated with the main select generated by the Query object. - - .. change:: - :tags: sql - :tickets: - - another fix to subquery correlation so that a subquery which has only one - FROM element will *not* correlate that single element, since at least one - FROM element is required in a query. - - .. change:: - :tags: sql - :tickets: 414 - - default "timezone" setting is now False. this corresponds to Python's - datetime behavior as well as Postgres' timestamp/time types (which is the - only timezone-sensitive dialect at the moment) - - .. change:: - :tags: sql - :tickets: - - the "op()" function is now treated as an "operation", rather than a - "comparison". the difference is, an operation produces a BinaryExpression - from which further operations can occur whereas comparison produces the - more restrictive BooleanExpression - - .. change:: - :tags: sql - :tickets: - - trying to redefine a reflected primary key column as non-primary key raises - an error - - .. change:: - :tags: sql - :tickets: - - type system slightly modified to support TypeDecorators that can be - overridden by the dialect (ok, that's not very clear, it allows the mssql - tweak below to be possible) - - .. change:: - :tags: mssql - :tickets: - - added an NVarchar type (produces NVARCHAR), also MSUnicode which provides - Unicode-translation for the NVarchar regardless of dialect convert_unicode - setting. - - .. change:: - :tags: postgres - :tickets: 424 - - fix to the initial checkfirst for tables to take current schema into - account - - .. change:: - :tags: postgres - :tickets: - - postgres has an optional "server_side_cursors=True" flag which will utilize - server side cursors. these are appropriate for fetching only partial - results and are necessary for working with very large unbounded result - sets. While we'd like this to be the default behavior, different - environments seem to have different results and the causes have not been - isolated so we are leaving the feature off by default for now. Uses an - apparently undocumented psycopg2 behavior recently discovered on the - psycopg mailing list. - - .. change:: - :tags: postgres - :tickets: - - added "BIGSERIAL" support for postgres table with - PGBigInteger/autoincrement - - .. change:: - :tags: postgres - :tickets: 402 - - fixes to postgres reflection to better handle when schema names are - present; thanks to jason (at) ncsmags.com - - .. change:: - :tags: mysql - :tickets: 420 - - mysql is inconsistent with what kinds of quotes it uses in foreign keys - during a SHOW CREATE TABLE, reflection updated to accommodate for all three - styles - - .. change:: - :tags: mysql - :tickets: 418 - - mysql table create options work on a generic passthru now, i.e. Table(..., - mysql_engine='InnoDB', mysql_collate="latin1_german2_ci", - mysql_auto_increment="5", mysql_...), helps - - .. change:: - :tags: firebird - :tickets: 408 - - order of constraint creation puts primary key first before all other - constraints; required for firebird, not a bad idea for others - - .. change:: - :tags: firebird - :tickets: 409 - - Firebird fix to autoload multifield foreign keys - - .. change:: - :tags: firebird - :tickets: 409 - - Firebird NUMERIC type properly handles a type without precision - - .. change:: - :tags: oracle - :tickets: - - *slight* support for binary, but still need to figure out how to insert - reasonably large values (over 4K). requires auto_setinputsizes=True sent to - create_engine(), rows must be fully fetched individually, etc. - - .. change:: - :tags: orm - :tickets: - - poked the first hole in the can of worms: saying - query.select_by(somerelationname=someinstance) will create the join of the - primary key columns represented by "somerelationname"'s mapper to the - actual primary key in "someinstance". - - .. change:: - :tags: orm - :tickets: - - reworked how relations interact with "polymorphic" mappers, i.e. mappers - that have a select_table as well as polymorphic flags. better determination - of proper join conditions, interaction with user- defined join conditions, - and support for self-referential polymorphic mappers. - - .. change:: - :tags: orm - :tickets: - - related to polymorphic mapping relations, some deeper error checking when - compiling relations, to detect an ambiguous "primaryjoin" in the case that - both sides of the relationship have foreign key references in the primary - join condition. also tightened down conditions used to locate "relation - direction", associating the "foreignkey" of the relationship with the - "primaryjoin" - - .. change:: - :tags: orm - :tickets: - - a little bit of improvement to the concept of a "concrete" inheritance - mapping, though that concept is not well fleshed out yet (added test case - to support concrete mappers on top of a polymorphic base). - - .. change:: - :tags: orm - :tickets: - - fix to "proxy=True" behavior on synonym() - - .. change:: - :tags: orm - :tickets: 427 - - fixed bug where delete-orphan basically didn't work with many-to-many - relationships, backref presence generally hid the symptom - - .. change:: - :tags: orm - :tickets: - - added a mutex to the mapper compilation step. ive been reluctant to add any - kind of threading anything to SA but this is one spot that its really - needed since mappers are typically "global", and while their state does not - change during normal operation, the initial compilation step does modify - internal state significantly, and this step usually occurs not at - module-level initialization time (unless you call compile()) but at - first-request time - - .. change:: - :tags: orm - :tickets: - - basic idea of "session.merge()" actually implemented. needs more testing. - - .. change:: - :tags: orm - :tickets: - - added "compile_mappers()" function as a shortcut to compiling all mappers - - .. change:: - :tags: orm - :tickets: - - fix to MapperExtension create_instance so that entity_name properly - associated with new instance - - .. change:: - :tags: orm - :tickets: - - speed enhancements to ORM object instantiation, eager loading of rows - - .. change:: - :tags: orm - :tickets: 406 - - invalid options sent to 'cascade' string will raise an exception - - .. change:: - :tags: orm - :tickets: 407 - - fixed bug in mapper refresh/expire whereby eager loaders didn't properly - re-populate item lists - - .. change:: - :tags: orm - :tickets: 413 - - fix to post_update to ensure rows are updated even for non insert/delete - scenarios - - .. change:: - :tags: orm - :tickets: 412 - - added an error message if you actually try to modify primary key values on - an entity and then flush it - - .. change:: - :tags: extensions - :tickets: 426 - - added "validate=False" argument to assign_mapper, if True will ensure that - only mapped attributes are named - - .. change:: - :tags: extensions - :tickets: - - assign_mapper gets "options", "instances" functions added (i.e. - MyClass.instances()) - -.. changelog:: - :version: 0.3.3 - :released: Fri Dec 15 2006 - - .. change:: - :tags: - :tickets: - - string-based FROM clauses fixed, i.e. select(..., from_obj=["sometext"]) - - .. change:: - :tags: - :tickets: - - fixes to passive_deletes flag, lazy=None (noload) flag - - .. change:: - :tags: - :tickets: - - added example/docs for dealing with large collections - - .. change:: - :tags: - :tickets: - - added object_session() method to sqlalchemy namespace - - .. change:: - :tags: - :tickets: - - fixed QueuePool bug whereby its better able to reconnect to a database - that was not reachable (thanks to Sébastien Lelong), also fixed dispose() - method - - .. change:: - :tags: - :tickets: 396 - - patch that makes MySQL rowcount work correctly! - - .. change:: - :tags: - :tickets: - - fix to MySQL catch of 2006/2014 errors to properly re-raise OperationalError - exception - -.. changelog:: - :version: 0.3.2 - :released: Sun Dec 10 2006 - - .. change:: - :tags: - :tickets: 387 - - major connection pool bug fixed. fixes MySQL out of sync - errors, will also prevent transactions getting rolled back - accidentally in all DBs - - .. change:: - :tags: - :tickets: - - major speed enhancements vs. 0.3.1, to bring speed - back to 0.2.8 levels - - .. change:: - :tags: - :tickets: - - made conditional dozens of debug log calls that were - time-intensive to generate log messages - - .. change:: - :tags: - :tickets: - - fixed bug in cascade rules whereby the entire object graph - could be unnecessarily cascaded on the save/update cascade - - .. change:: - :tags: - :tickets: - - various speedups in attributes module - - .. change:: - :tags: - :tickets: 388 - - identity map in Session is by default *no longer weak referencing*. - to have it be weak referencing, use create_session(weak_identity_map=True) - fixes - - .. change:: - :tags: - :tickets: - - MySQL detects errors 2006 (server has gone away) and 2014 - (commands out of sync) and invalidates the connection on which it occurred. - - .. change:: - :tags: - :tickets: 307 - - MySQL bool type fix: - - .. change:: - :tags: - :tickets: 382, 349 - - postgres reflection fixes: - - .. change:: - :tags: - :tickets: 247 - - added keywords for EXCEPT, INTERSECT, EXCEPT ALL, INTERSECT ALL - - .. change:: - :tags: - :tickets: 2110 - - assign_mapper in assignmapper extension returns the created mapper - - .. change:: - :tags: - :tickets: - - added label() function to Select class, when scalar=True is used - to create a scalar subquery - i.e. "select x, y, (select max(foo) from table) AS foomax from table" - - .. change:: - :tags: - :tickets: - - added onupdate and ondelete keyword arguments to ForeignKey; propagate - to underlying ForeignKeyConstraint if present. (don't propagate in the - other direction, however) - - .. change:: - :tags: - :tickets: - - fix to session.update() to preserve "dirty" status of incoming object - - .. change:: - :tags: - :tickets: - - sending a selectable to an IN via the in_() function no longer creates - a "union" out of multiple selects; only one selectable to a the in_() function - is allowed now (make a union yourself if union is needed) - - .. change:: - :tags: - :tickets: - - improved support for disabling save-update cascade via cascade="none" etc. - - .. change:: - :tags: - :tickets: - - added "remote_side" argument to relation(), used only with self-referential - mappers to force the direction of the parent/child relationship. replaces - the usage of the "foreignkey" parameter for "switching" the direction. - "foreignkey" argument is deprecated for all uses and will eventually - be replaced by an argument dedicated to ForeignKey specification on mappers. - -.. changelog:: - :version: 0.3.1 - :released: Mon Nov 13 2006 - - .. change:: - :tags: engine/pool - :tickets: - - some new Pool utility classes, updated docs - - .. change:: - :tags: engine/pool - :tickets: - - "use_threadlocal" on Pool defaults to False (same as create_engine) - - .. change:: - :tags: engine/pool - :tickets: - - fixed direct execution of Compiled objects - - .. change:: - :tags: engine/pool - :tickets: - - create_engine() reworked to be strict about incoming \**kwargs. all keyword - arguments must be consumed by one of the dialect, connection pool, and engine - constructors, else a TypeError is thrown which describes the full set of - invalid kwargs in relation to the selected dialect/pool/engine configuration. - - .. change:: - :tags: databases/types - :tickets: - - MySQL catches exception on "describe" and reports as NoSuchTableError - - .. change:: - :tags: databases/types - :tickets: - - further fixes to sqlite booleans, weren't working as defaults - - .. change:: - :tags: databases/types - :tickets: - - fix to postgres sequence quoting when using schemas - - .. change:: - :tags: orm - :tickets: - - the "delete" cascade will load in all child objects, if they were not - loaded already. this can be turned off (i.e. the old behavior) by setting - passive_deletes=True on a relation(). - - .. change:: - :tags: orm - :tickets: - - adjustments to reworked eager query generation to not fail on circular - eager-loaded relationships (like backrefs) - - .. change:: - :tags: orm - :tickets: - - fixed bug where eagerload() (nor lazyload()) option didn't properly - instruct the Query whether or not to use "nesting" when producing a - LIMIT query. - - .. change:: - :tags: orm - :tickets: 360 - - fixed bug in circular dependency sorting at flush time; if object A - contained a cyclical many-to-one relationship to object B, and object B - was just attached to object A, *but* object B itself wasn't changed, - the many-to-one synchronize of B's primary key attribute to A's foreign key - attribute wouldn't occur. - - .. change:: - :tags: orm - :tickets: 325 - - implemented from_obj argument for query.count, improves count function - on selectresults - - .. change:: - :tags: orm - :tickets: - - added an assertion within the "cascade" step of ORM relationships to check - that the class of object attached to a parent object is appropriate - (i.e. if A.items stores B objects, raise an error if a C is appended to A.items) - - .. change:: - :tags: orm - :tickets: - - new extension sqlalchemy.ext.associationproxy, provides transparent - "association object" mappings. new example - examples/association/proxied_association.py illustrates. - - .. change:: - :tags: orm - :tickets: - - improvement to single table inheritance to load full hierarchies beneath - the target class - - .. change:: - :tags: orm - :tickets: 362 - - fix to subtle condition in topological sort where a node could appear twice, - for - - .. change:: - :tags: orm - :tickets: 365 - - additional rework to topological sort, refactoring, for - - .. change:: - :tags: orm - :tickets: - - "delete-orphan" for a certain type can be set on more than one parent class; - the instance is an "orphan" only if its not attached to *any* of those parents - -.. changelog:: - :version: 0.3.0 - :released: Sun Oct 22 2006 - - .. change:: - :tags: general - :tickets: - - logging is now implemented via standard python "logging" module. - "echo" keyword parameters are still functional but set/unset - log levels for their respective classes/instances. all logging - can be controlled directly through the Python API by setting - INFO and DEBUG levels for loggers in the "sqlalchemy" namespace. - class-level logging is under "sqlalchemy..", - instance-level logging under "sqlalchemy...0x..<00-FF>". - Test suite includes "--log-info" and "--log-debug" arguments - which work independently of --verbose/--quiet. Logging added - to orm to allow tracking of mapper configurations, row iteration. - - .. change:: - :tags: general - :tickets: - - the documentation-generation system has been overhauled to be - much simpler in design and more integrated with Markdown - - .. change:: - :tags: sqlite - :tickets: - - sqlite boolean datatype converts False/True to 0/1 by default - - .. change:: - :tags: sqlite - :tickets: 335 - - fixes to Date/Time (SLDate/SLTime) types; works as good as postgres - now - - .. change:: - :tags: ms-sql - :tickets: - - fixes bug 261 (table reflection broken for MS-SQL case-sensitive - databases) - - .. change:: - :tags: ms-sql - :tickets: - - can now specify port for pymssql - - .. change:: - :tags: ms-sql - :tickets: - - introduces new "auto_identity_insert" option for auto-switching - between "SET IDENTITY_INSERT" mode when values specified for IDENTITY columns - - .. change:: - :tags: ms-sql - :tickets: - - now supports multi-column foreign keys - - .. change:: - :tags: ms-sql - :tickets: - - fix to reflecting date/datetime columns - - .. change:: - :tags: ms-sql - :tickets: - - NCHAR and NVARCHAR type support added - - .. change:: - :tags: oracle - :tickets: - - Oracle has experimental support for cx_Oracle.TIMESTAMP, which requires - a setinputsizes() call on the cursor that is now enabled via the - 'auto_setinputsizes' flag to the oracle dialect. - - .. change:: - :tags: firebird - :tickets: - - aliases do not use "AS" - - .. change:: - :tags: firebird - :tickets: - - correctly raises NoSuchTableError when reflecting non-existent table - - .. change:: - :tags: schema - :tickets: - - a fair amount of cleanup to the schema package, removal of ambiguous - methods, methods that are no longer needed. slightly more constrained - usage, greater emphasis on explicitness - - .. change:: - :tags: schema - :tickets: - - the "primary_key" attribute of Table and other selectables becomes - a setlike ColumnCollection object; is ordered but not numerically - indexed. a comparison clause between two pks that are derived from the - same underlying tables (i.e. such as two Alias objects) can be generated - via table1.primary_key==table2.primary_key - - .. change:: - :tags: schema - :tickets: - - ForeignKey(Constraint) supports "use_alter=True", to create/drop a foreign key - via ALTER. this allows circular foreign key relationships to be set up. - - .. change:: - :tags: schema - :tickets: - - append_item() methods removed from Table and Column; preferably - construct Table/Column/related objects inline, but if needed use - append_column(), append_foreign_key(), append_constraint(), etc. - - .. change:: - :tags: schema - :tickets: - - table.create() no longer returns the Table object, instead has no - return value. the usual case is that tables are created via metadata, - which is preferable since it will handle table dependencies. - - .. change:: - :tags: schema - :tickets: - - added UniqueConstraint (goes at Table level), CheckConstraint - (goes at Table or Column level). - - .. change:: - :tags: schema - :tickets: - - index=False/unique=True on Column now creates a UniqueConstraint, - index=True/unique=False creates a plain Index, - index=True/unique=True on Column creates a unique Index. 'index' - and 'unique' keyword arguments to column are now boolean only; for - explicit names and groupings of indexes or unique constraints, use the - UniqueConstraint/Index constructs explicitly. - - .. change:: - :tags: schema - :tickets: - - added autoincrement=True to Column; will disable schema generation - of SERIAL/AUTO_INCREMENT/identity seq for postgres/mysql/mssql if - explicitly set to False - - .. change:: - :tags: schema - :tickets: - - TypeEngine objects now have methods to deal with copying and comparing - values of their specific type. Currently used by the ORM, see below. - - .. change:: - :tags: schema - :tickets: - - fixed condition that occurred during reflection when a primary key - column was explicitly overridden, where the PrimaryKeyConstraint would - get both the reflected and the programmatic column doubled up - - .. change:: - :tags: schema - :tickets: - - the "foreign_key" attribute on Column and ColumnElement in general - is deprecated, in favor of the "foreign_keys" list/set-based attribute, - which takes into account multiple foreign keys on one column. - "foreign_key" will return the first element in the "foreign_keys" list/set - or None if the list is empty. - - .. change:: - :tags: connections/pooling/execution - :tickets: - - connection pool tracks open cursors and automatically closes them - if connection is returned to pool with cursors still opened. Can be - affected by options which cause it to raise an error instead, or to - do nothing. fixes issues with MySQL, others - - .. change:: - :tags: connections/pooling/execution - :tickets: - - fixed bug where Connection wouldn't lose its Transaction - after commit/rollback - - .. change:: - :tags: connections/pooling/execution - :tickets: - - added scalar() method to ComposedSQLEngine, ResultProxy - - .. change:: - :tags: connections/pooling/execution - :tickets: - - ResultProxy will close() the underlying cursor when the ResultProxy - itself is closed. this will auto-close cursors for ResultProxy objects - that have had all their rows fetched (or had scalar() called). - - .. change:: - :tags: connections/pooling/execution - :tickets: - - ResultProxy.fetchall() internally uses DBAPI fetchall() for better efficiency, - added to mapper iteration as well (courtesy Michael Twomey) - - .. change:: - :tags: construction, sql - :tickets: 292 - - changed "for_update" parameter to accept False/True/"nowait" - and "read", the latter two of which are interpreted only by - Oracle and MySQL - - .. change:: - :tags: construction, sql - :tickets: - - added extract() function to sql dialect - (SELECT extract(field FROM expr)) - - .. change:: - :tags: construction, sql - :tickets: - - BooleanExpression includes new "negate" argument to specify - the appropriate negation operator if one is available. - - .. change:: - :tags: construction, sql - :tickets: - - calling a negation on an "IN" or "IS" clause will result in - "NOT IN", "IS NOT" (as opposed to NOT (x IN y)). - - .. change:: - :tags: construction, sql - :tickets: 172 - - Function objects know what to do in a FROM clause now. their - behavior should be the same, except now you can also do things like - select(['*'], from_obj=[func.my_function()]) to get multiple - columns from the result, or even use sql.column() constructs to name the - return columns - - .. change:: - :tags: orm - :tickets: - - attribute tracking modified to be more intelligent about detecting - changes, particularly with mutable types. TypeEngine objects now - take a greater role in defining how to compare two scalar instances, - including the addition of a MutableType mixin which is implemented by - PickleType. unit-of-work now tracks the "dirty" list as an expression - of all persistent objects where the attribute manager detects changes. - The basic issue that's fixed is detecting changes on PickleType - objects, but also generalizes type handling and "modified" object - checking to be more complete and extensible. - - .. change:: - :tags: orm - :tickets: - - a wide refactoring to "attribute loader" and "options" architectures. - ColumnProperty and PropertyLoader define their loading behavior via switchable - "strategies", and MapperOptions no longer use mapper/property copying - in order to function; they are instead propagated via QueryContext - and SelectionContext objects at query/instances time. - All of the internal copying of mappers and properties that was used to handle - inheritance as well as options() has been removed; the structure - of mappers and properties is much simpler than before and is clearly laid out - in the new 'interfaces' module. - - .. change:: - :tags: orm - :tickets: - - related to the mapper/property overhaul, internal refactoring to - mapper instances() method to use a SelectionContext object to track - state during the operation. - SLIGHT API BREAKAGE: the append_result() and populate_instances() - methods on MapperExtension have a slightly different method signature - now as a result of the change; hoping that these methods are not - in widespread use as of yet. - - .. change:: - :tags: orm - :tickets: - - instances() method moved to Query now, backwards-compatible - version remains on Mapper. - - .. change:: - :tags: orm - :tickets: - - added contains_eager() MapperOption, used in conjunction with - instances() to specify properties that should be eagerly loaded - from the result set, using their plain column names by default, or translated - given an custom row-translation function. - - .. change:: - :tags: orm - :tickets: - - more rearrangements of unit-of-work commit scheme to better allow - dependencies within circular flushes to work properly...updated - task traversal/logging implementation - - .. change:: - :tags: orm - :tickets: 321 - - polymorphic mappers (i.e. using inheritance) now produces INSERT - statements in order of tables across all inherited classes - - .. change:: - :tags: orm - :tickets: - - added an automatic "row switch" feature to mapping, which will - detect a pending instance/deleted instance pair with the same - identity key and convert the INSERT/DELETE to a single UPDATE - - .. change:: - :tags: orm - :tickets: - - "association" mappings simplified to take advantage of - automatic "row switch" feature - - .. change:: - :tags: orm - :tickets: 212 - - "custom list classes" is now implemented via the "collection_class" - keyword argument to relation(). the old way still works but is - deprecated - - .. change:: - :tags: orm - :tickets: - - added "viewonly" flag to relation(), allows construction of - relations that have no effect on the flush() process. - - .. change:: - :tags: orm - :tickets: 292 - - added "lockmode" argument to base Query select/get functions, - including "with_lockmode" function to get a Query copy that has - a default locking mode. Will translate "read"/"update" - arguments into a for_update argument on the select side. - - .. change:: - :tags: orm - :tickets: - - implemented "version check" logic in Query/Mapper, used - when version_id_col is in effect and query.with_lockmode() - is used to get() an instance that's already loaded - - .. change:: - :tags: orm - :tickets: 208 - - post_update behavior improved; does a better job at not - updating too many rows, updates only required columns - - .. change:: - :tags: orm - :tickets: 308 - - adjustments to eager loading so that its "eager chain" is - kept separate from the normal mapper setup, thereby - preventing conflicts with lazy loader operation, fixes - - .. change:: - :tags: orm - :tickets: - - fix to deferred group loading - - .. change:: - :tags: orm - :tickets: 346 - - session.flush() won't close a connection it opened - - .. change:: - :tags: orm - :tickets: - - added "batch=True" flag to mapper; if False, save_obj - will fully save one object at a time including calls - to before_XXXX and after_XXXX - - .. change:: - :tags: orm - :tickets: - - added "column_prefix=None" argument to mapper; prepends the - given string (typically '_') to column-based attributes automatically - set up from the mapper's Table - - .. change:: - :tags: orm - :tickets: 315 - - specifying joins in the from_obj argument of query.select() will - replace the main table of the query, if the table is somewhere within - the given from_obj. this makes it possible to produce custom joins and - outerjoins in queries without the main table getting added twice. - - .. change:: - :tags: orm - :tickets: - - eagerloading is adjusted to more thoughtfully attach its LEFT OUTER JOINs - to the given query, looking for custom "FROM" clauses that may have - already been set up. - - .. change:: - :tags: orm - :tickets: - - added join_to and outerjoin_to transformative methods to SelectResults, - to build up join/outerjoin conditions based on property names. also - added select_from to explicitly set from_obj parameter. - - .. change:: - :tags: orm - :tickets: - - removed "is_primary" flag from mapper. diff --git a/doc/build/changelog/changelog_04.rst b/doc/build/changelog/changelog_04.rst deleted file mode 100644 index b0312b0921..0000000000 --- a/doc/build/changelog/changelog_04.rst +++ /dev/null @@ -1,4197 +0,0 @@ - -============= -0.4 Changelog -============= - - -.. changelog:: - :version: 0.4.8 - :released: Sun Oct 12 2008 - - .. change:: - :tags: orm - :tickets: 1039 - - Fixed bug regarding inherit_condition passed - with "A=B" versus "B=A" leading to errors - - .. change:: - :tags: orm - :tickets: - - Changes made to new, dirty and deleted - collections in - SessionExtension.before_flush() will take - effect for that flush. - - .. change:: - :tags: orm - :tickets: - - Added label() method to InstrumentedAttribute - to establish forwards compatibility with 0.5. - - .. change:: - :tags: sql - :tickets: 1074 - - column.in_(someselect) can now be used as - a columns-clause expression without the subquery - bleeding into the FROM clause - - .. change:: - :tags: mysql - :tickets: 1146 - - Added MSMediumInteger type. - - .. change:: - :tags: sqlite - :tickets: 968 - - Supplied a custom strftime() function which - handles dates before 1900. - - .. change:: - :tags: sqlite - :tickets: - - String's (and Unicode's, UnicodeText's, etc.) - convert_unicode logic disabled in the sqlite dialect, - to adjust for pysqlite 2.5.0's new requirement that - only Python unicode objects are accepted; - https://itsystementwicklung.de/pipermail/list-pysqlite/2008-March/000018.html - - .. change:: - :tags: oracle - :tickets: 1155 - - has_sequence() now takes schema name into account - - .. change:: - :tags: oracle - :tickets: 1121 - - added BFILE to the list of reflected types - -.. changelog:: - :version: 0.4.7p1 - :released: Thu Jul 31 2008 - - .. change:: - :tags: orm - :tickets: - - Added "add()" and "add_all()" to scoped_session - methods. Workaround for 0.4.7:: - - from sqlalchemy.orm.scoping import ScopedSession, instrument - - setattr(ScopedSession, "add", instrument("add")) - setattr(ScopedSession, "add_all", instrument("add_all")) - - .. change:: - :tags: orm - :tickets: - - Fixed non-2.3 compatible usage of set() and generator - expression within relation(). - -.. changelog:: - :version: 0.4.7 - :released: Sat Jul 26 2008 - - .. change:: - :tags: orm - :tickets: 1058 - - The contains() operator when used with many-to-many - will alias() the secondary (association) table so - that multiple contains() calls will not conflict - with each other - - .. change:: - :tags: orm - :tickets: - - fixed bug preventing merge() from functioning in - conjunction with a comparable_property() - - .. change:: - :tags: orm - :tickets: - - the enable_typechecks=False setting on relation() - now only allows subtypes with inheriting mappers. - Totally unrelated types, or subtypes not set up with - mapper inheritance against the target mapper are - still not allowed. - - .. change:: - :tags: orm - :tickets: 976 - - Added is_active flag to Sessions to detect when - a transaction is in progress. This - flag is always True with a "transactional" - (in 0.5 a non-"autocommit") Session. - - .. change:: - :tags: sql - :tickets: - - Fixed bug when calling select([literal('foo')]) - or select([bindparam('foo')]). - - .. change:: - :tags: schema - :tickets: 571 - - create_all(), drop_all(), create(), drop() all raise - an error if the table name or schema name contains - more characters than that dialect's configured - character limit. Some DB's can handle too-long - table names during usage, and SQLA can handle this - as well. But various reflection/ - checkfirst-during-create scenarios fail since we are - looking for the name within the DB's catalog tables. - - .. change:: - :tags: schema - :tickets: 571, 820 - - The index name generated when you say "index=True" - on a Column is truncated to the length appropriate - for the dialect. Additionally, an Index with a too- - long name cannot be explicitly dropped with - Index.drop(), similar to. - - .. change:: - :tags: postgres - :tickets: - - Repaired server_side_cursors to properly detect - text() clauses. - - .. change:: - :tags: postgres - :tickets: 1092 - - Added PGCidr type. - - .. change:: - :tags: mysql - :tickets: - - Added 'CALL' to the list of SQL keywords which return - result rows. - - .. change:: - :tags: oracle - :tickets: - - Oracle get_default_schema_name() "normalizes" the name - before returning, meaning it returns a lower-case name - when the identifier is detected as case insensitive. - - .. change:: - :tags: oracle - :tickets: 709 - - creating/dropping tables takes schema name into account - when searching for the existing table, so that tables - in other owner namespaces with the same name do not - conflict - - .. change:: - :tags: oracle - :tickets: 1062 - - Cursors now have "arraysize" set to 50 by default on - them, the value of which is configurable using the - "arraysize" argument to create_engine() with the - Oracle dialect. This to account for cx_oracle's default - setting of "1", which has the effect of many round trips - being sent to Oracle. This actually works well in - conjunction with BLOB/CLOB-bound cursors, of which - there are any number available but only for the life of - that row request (so BufferedColumnRow is still needed, - but less so). - - .. change:: - :tags: oracle - :tickets: - - sqlite - - add SLFloat type, which matches the SQLite REAL - type affinity. Previously, only SLNumeric was provided - which fulfills NUMERIC affinity, but that's not the - same as REAL. - -.. changelog:: - :version: 0.4.6 - :released: Sat May 10 2008 - - .. change:: - :tags: orm - :tickets: - - Fix to the recent relation() refactoring which fixes - exotic viewonly relations which join between local and - remote table multiple times, with a common column shared - between the joins. - - .. change:: - :tags: orm - :tickets: - - Also re-established viewonly relation() configurations - that join across multiple tables. - - .. change:: - :tags: orm - :tickets: 610 - - Added experimental relation() flag to help with - primaryjoins across functions, etc., - _local_remote_pairs=[tuples]. This complements a complex - primaryjoin condition allowing you to provide the - individual column pairs which comprise the relation's - local and remote sides. Also improved lazy load SQL - generation to handle placing bind params inside of - functions and other expressions. (partial progress - towards) - - .. change:: - :tags: orm - :tickets: 1036 - - repaired single table inheritance such that you - can single-table inherit from a joined-table inheriting - mapper without issue. - - .. change:: - :tags: orm - :tickets: 1027 - - Fixed "concatenate tuple" bug which could occur with - Query.order_by() if clause adaption had taken place. - - .. change:: - :tags: orm - :tickets: - - Removed ancient assertion that mapped selectables require - "alias names" - the mapper creates its own alias now if - none is present. Though in this case you need to use the - class, not the mapped selectable, as the source of column - attributes - so a warning is still issued. - - .. change:: - :tags: orm - :tickets: - - fixes to the "exists" function involving inheritance (any(), - has(), ~contains()); the full target join will be rendered - into the EXISTS clause for relations that link to subclasses. - - .. change:: - :tags: orm - :tickets: - - restored usage of append_result() extension method for primary - query rows, when the extension is present and only a single- - entity result is being returned. - - .. change:: - :tags: orm - :tickets: - - Also re-established viewonly relation() configurations that - join across multiple tables. - - .. change:: - :tags: orm - :tickets: - - removed ancient assertion that mapped selectables require - "alias names" - the mapper creates its own alias now if - none is present. Though in this case you need to use - the class, not the mapped selectable, as the source of - column attributes - so a warning is still issued. - - .. change:: - :tags: orm - :tickets: 1015 - - refined mapper._save_obj() which was unnecessarily calling - __ne__() on scalar values during flush - - .. change:: - :tags: orm - :tickets: 1019 - - added a feature to eager loading whereby subqueries set - as column_property() with explicit label names (which is not - necessary, btw) will have the label anonymized when - the instance is part of the eager join, to prevent - conflicts with a subquery or column of the same name - on the parent object. - - .. change:: - :tags: orm - :tickets: - - set-based collections \|=, -=, ^= and &= are stricter about - their operands and only operate on sets, frozensets or - subclasses of the collection type. Previously, they would - accept any duck-typed set. - - .. change:: - :tags: orm - :tickets: - - added an example dynamic_dict/dynamic_dict.py, illustrating - a simple way to place dictionary behavior on top of - a dynamic_loader. - - .. change:: - :tags: declarative, extension - :tickets: - - Joined table inheritance mappers use a slightly relaxed - function to create the "inherit condition" to the parent - table, so that other foreign keys to not-yet-declared - Table objects don't trigger an error. - - .. change:: - :tags: declarative, extension - :tickets: - - fixed reentrant mapper compile hang when - a declared attribute is used within ForeignKey, - ie. ForeignKey(MyOtherClass.someattribute) - - .. change:: - :tags: sql - :tickets: - - Added COLLATE support via the .collate() - expression operator and collate(, ) sql - function. - - .. change:: - :tags: sql - :tickets: - - Fixed bug with union() when applied to non-Table connected - select statements - - .. change:: - :tags: sql - :tickets: 1014 - - improved behavior of text() expressions when used as - FROM clauses, such as select().select_from(text("sometext")) - - .. change:: - :tags: sql - :tickets: 1021 - - Column.copy() respects the value of "autoincrement", - fixes usage with Migrate - - .. change:: - :tags: engines - :tickets: - - Pool listeners can now be provided as a dictionary of - callables or a (possibly partial) duck-type of - PoolListener, your choice. - - .. change:: - :tags: engines - :tickets: - - added "rollback_returned" option to Pool which will - disable the rollback() issued when connections are - returned. This flag is only safe to use with a database - which does not support transactions (i.e. MySQL/MyISAM). - - .. change:: - :tags: ext - :tickets: - - set-based association proxies \|=, -=, ^= and &= are - stricter about their operands and only operate on sets, - frozensets or other association proxies. Previously, they - would accept any duck-typed set. - - .. change:: - :tags: mssql - :tickets: 1005 - - Added "odbc_autotranslate" parameter to engine / dburi - parameters. Any given string will be passed through to the - ODBC connection string as: - - "AutoTranslate=%s" % odbc_autotranslate - - .. change:: - :tags: mssql - :tickets: - - Added "odbc_options" parameter to engine / dburi - parameters. The given string is simply appended to the - SQLAlchemy-generated odbc connection string. - - This should obviate the need of adding a myriad of ODBC - options in the future. - - .. change:: - :tags: firebird - :tickets: - - Handle the "SUBSTRING(:string FROM :start FOR :length)" - builtin. - -.. changelog:: - :version: 0.4.5 - :released: Fri Apr 04 2008 - - .. change:: - :tags: orm - :tickets: - - A small change in behavior to session.merge() - existing - objects are checked for based on primary key attributes, not - necessarily _instance_key. So the widely requested - capability, that: - - x = MyObject(id=1) - x = sess.merge(x) - - will in fact load MyObject with id #1 from the database if - present, is now available. merge() still copies the state - of the given object to the persistent one, so an example - like the above would typically have copied "None" from all - attributes of "x" onto the persistent copy. These can be - reverted using session.expire(x). - - .. change:: - :tags: orm - :tickets: - - Also fixed behavior in merge() whereby collection elements - present on the destination but not the merged collection - were not being removed from the destination. - - .. change:: - :tags: orm - :tickets: 995 - - Added a more aggressive check for "uncompiled mappers", - helps particularly with declarative layer - - .. change:: - :tags: orm - :tickets: - - The methodology behind "primaryjoin"/"secondaryjoin" has - been refactored. Behavior should be slightly more - intelligent, primarily in terms of error messages which - have been pared down to be more readable. In a slight - number of scenarios it can better resolve the correct - foreign key than before. - - .. change:: - :tags: orm - :tickets: - - Added comparable_property(), adds query Comparator - behavior to regular, unmanaged Python properties - - .. change:: - :tags: orm, Company.employees.of_type(Engineer), 'machines' - :tickets: - - the functionality of query.with_polymorphic() has - been added to mapper() as a configuration option. - - It's set via several forms: - with_polymorphic='*' - with_polymorphic=[mappers] - with_polymorphic=('*', selectable) - with_polymorphic=([mappers], selectable) - - This controls the default polymorphic loading strategy - for inherited mappers. When a selectable is not given, - outer joins are created for all joined-table inheriting - mappers requested. Note that the auto-create of joins - is not compatible with concrete table inheritance. - - The existing select_table flag on mapper() is now - deprecated and is synonymous with - with_polymorphic('*', select_table). Note that the - underlying "guts" of select_table have been - completely removed and replaced with the newer, - more flexible approach. - - The new approach also automatically allows eager loads - to work for subclasses, if they are present, for - example:: - - sess.query(Company).options(eagerload_all()) - - to load Company objects, their employees, and the - 'machines' collection of employees who happen to be - Engineers. A "with_polymorphic" Query option should be - introduced soon as well which would allow per-Query - control of with_polymorphic() on relations. - - .. change:: - :tags: orm - :tickets: - - added two "experimental" features to Query, - "experimental" in that their specific name/behavior - is not carved in stone just yet: _values() and - _from_self(). We'd like feedback on these. - - - _values(\*columns) is given a list of column - expressions, and returns a new Query that only - returns those columns. When evaluated, the return - value is a list of tuples just like when using - add_column() or add_entity(), the only difference is - that "entity zero", i.e. the mapped class, is not - included in the results. This means it finally makes - sense to use group_by() and having() on Query, which - have been sitting around uselessly until now. - - A future change to this method may include that its - ability to join, filter and allow other options not - related to a "resultset" are removed, so the feedback - we're looking for is how people want to use - _values()...i.e. at the very end, or do people prefer - to continue generating after it's called. - - - _from_self() compiles the SELECT statement for the - Query (minus any eager loaders), and returns a new - Query that selects from that SELECT. So basically you - can query from a Query without needing to extract the - SELECT statement manually. This gives meaning to - operations like query[3:5]._from_self().filter(some - criterion). There's not much controversial here - except that you can quickly create highly nested - queries that are less efficient, and we want feedback - on the naming choice. - - .. change:: - :tags: orm - :tickets: - - query.order_by() and query.group_by() will accept - multiple arguments using \*args (like select() - already does). - - .. change:: - :tags: orm - :tickets: - - Added some convenience descriptors to Query: - query.statement returns the full SELECT construct, - query.whereclause returns just the WHERE part of the - SELECT construct. - - .. change:: - :tags: orm - :tickets: - - Fixed/covered case when using a False/0 value as a - polymorphic discriminator. - - .. change:: - :tags: orm - :tickets: - - Fixed bug which was preventing synonym() attributes from - being used with inheritance - - .. change:: - :tags: orm - :tickets: 996 - - Fixed SQL function truncation of trailing underscores - - .. change:: - :tags: orm - :tickets: - - When attributes are expired on a pending instance, an - error will not be raised when the "refresh" action is - triggered and no result is found. - - .. change:: - :tags: orm - :tickets: - - Session.execute can now find binds from metadata - - .. change:: - :tags: orm - :tickets: - - Adjusted the definition of "self-referential" to be any - two mappers with a common parent (this affects whether or - not aliased=True is required when joining with Query). - - .. change:: - :tags: orm - :tickets: - - Made some fixes to the "from_joinpoint" argument to - query.join() so that if the previous join was aliased and - this one isn't, the join still happens successfully. - - .. change:: - :tags: orm - :tickets: 895 - - Assorted "cascade deletes" fixes: - - Fixed "cascade delete" operation of dynamic relations, - which had only been implemented for foreign-key - nulling behavior in 0.4.2 and not actual cascading - deletes - - - Delete cascade without delete-orphan cascade on a - many-to-one will not delete orphans which were - disconnected from the parent before session.delete() - is called on the parent (one-to-many already had - this). - - - Delete cascade with delete-orphan will delete orphans - whether or not it remains attached to its also-deleted - parent. - - - delete-orphan cascade is properly detected on relations - that are present on superclasses when using inheritance. - - .. change:: - :tags: orm - :tickets: - - Fixed order_by calculation in Query to properly alias - mapper-config'ed order_by when using select_from() - - .. change:: - :tags: orm - :tickets: - - Refactored the diffing logic that kicks in when replacing - one collection with another into collections.bulk_replace, - useful to anyone building multi-level collections. - - .. change:: - :tags: orm - :tickets: - - Cascade traversal algorithm converted from recursive to - iterative to support deep object graphs. - - .. change:: - :tags: sql - :tickets: 999 - - schema-qualified tables now will place the schemaname - ahead of the tablename in all column expressions as well - as when generating column labels. This prevents cross- - schema name collisions in all cases - - .. change:: - :tags: sql - :tickets: - - can now allow selects which correlate all FROM clauses - and have no FROM themselves. These are typically - used in a scalar context, i.e. SELECT x, (SELECT x WHERE y) - FROM table. Requires explicit correlate() call. - - .. change:: - :tags: sql - :tickets: - - 'name' is no longer a required constructor argument for - Column(). It (and .key) may now be deferred until the - column is added to a Table. - - .. change:: - :tags: sql - :tickets: 791, 993 - - like(), ilike(), contains(), startswith(), endswith() take - an optional keyword argument "escape=", which - is set as the escape character using the syntax "x LIKE y - ESCAPE ''". - - .. change:: - :tags: sql - :tickets: - - random() is now a generic sql function and will compile to - the database's random implementation, if any. - - .. change:: - :tags: sql - :tickets: - - update().values() and insert().values() take keyword - arguments. - - .. change:: - :tags: sql - :tickets: - - Fixed an issue in select() regarding its generation of - FROM clauses, in rare circumstances two clauses could be - produced when one was intended to cancel out the other. - Some ORM queries with lots of eager loads might have seen - this symptom. - - .. change:: - :tags: sql - :tickets: - - The case() function now also takes a dictionary as its - whens parameter. It also interprets the "THEN" - expressions as values by default, meaning case([(x==y, - "foo")]) will interpret "foo" as a bound value, not a SQL - expression. use text(expr) for literal SQL expressions in - this case. For the criterion itself, these may be literal - strings only if the "value" keyword is present, otherwise - SA will force explicit usage of either text() or - literal(). - - .. change:: - :tags: oracle - :tickets: - - The "owner" keyword on Table is now deprecated, and is - exactly synonymous with the "schema" keyword. Tables can - now be reflected with alternate "owner" attributes, - explicitly stated on the Table object or not using - "schema". - - .. change:: - :tags: oracle - :tickets: - - All of the "magic" searching for synonyms, DBLINKs etc. - during table reflection are disabled by default unless you - specify "oracle_resolve_synonyms=True" on the Table - object. Resolving synonyms necessarily leads to some - messy guessing which we'd rather leave off by default. - When the flag is set, tables and related tables will be - resolved against synonyms in all cases, meaning if a - synonym exists for a particular table, reflection will use - it when reflecting related tables. This is stickier - behavior than before which is why it's off by default. - - .. change:: - :tags: declarative, extension - :tickets: - - The "synonym" function is now directly usable with - "declarative". Pass in the decorated property using the - "descriptor" keyword argument, e.g.: somekey = - synonym('_somekey', descriptor=property(g, s)) - - .. change:: - :tags: declarative, extension - :tickets: - - The "deferred" function is usable with "declarative". - Simplest usage is to declare deferred and Column together, - e.g.: data = deferred(Column(Text)) - - .. change:: - :tags: declarative, extension - :tickets: - - Declarative also gained @synonym_for(...) and - @comparable_using(...), front-ends for synonym and - comparable_property. - - .. change:: - :tags: declarative, extension - :tickets: 995 - - Improvements to mapper compilation when using declarative; - already-compiled mappers will still trigger compiles of - other uncompiled mappers when used - - .. change:: - :tags: declarative, extension - :tickets: - - Declarative will complete setup for Columns lacking names, - allows a more DRY syntax. - - class Foo(Base): - __tablename__ = 'foos' - id = Column(Integer, primary_key=True) - - .. change:: - :tags: declarative, extension - :tickets: - - inheritance in declarative can be disabled when sending - "inherits=None" to __mapper_args__. - - .. change:: - :tags: declarative, extension - :tickets: - - declarative_base() takes optional kwarg "mapper", which - is any callable/class/method that produces a mapper, - such as declarative_base(mapper=scopedsession.mapper). - This property can also be set on individual declarative - classes using the "__mapper_cls__" property. - - .. change:: - :tags: postgres - :tickets: 1001 - - Got PG server side cursors back into shape, added fixed - unit tests as part of the default test suite. Added - better uniqueness to the cursor ID - - .. change:: - :tags: oracle - :tickets: - - The "owner" keyword on Table is now deprecated, and is - exactly synonymous with the "schema" keyword. Tables can - now be reflected with alternate "owner" attributes, - explicitly stated on the Table object or not using - "schema". - - .. change:: - :tags: oracle - :tickets: - - All of the "magic" searching for synonyms, DBLINKs etc. - during table reflection are disabled by default unless you - specify "oracle_resolve_synonyms=True" on the Table - object. Resolving synonyms necessarily leads to some - messy guessing which we'd rather leave off by default. - When the flag is set, tables and related tables will be - resolved against synonyms in all cases, meaning if a - synonym exists for a particular table, reflection will use - it when reflecting related tables. This is stickier - behavior than before which is why it's off by default. - - .. change:: - :tags: mssql - :tickets: 979 - - Reflected tables will now automatically load other tables - which are referenced by Foreign keys in the auto-loaded - table,. - - .. change:: - :tags: mssql - :tickets: 916 - - Added executemany check to skip identity fetch,. - - .. change:: - :tags: mssql - :tickets: 884 - - Added stubs for small date type. - - .. change:: - :tags: mssql - :tickets: - - Added a new 'driver' keyword parameter for the pyodbc dialect. - Will substitute into the ODBC connection string if given, - defaults to 'SQL Server'. - - .. change:: - :tags: mssql - :tickets: - - Added a new 'max_identifier_length' keyword parameter for - the pyodbc dialect. - - .. change:: - :tags: mssql - :tickets: - - Improvements to pyodbc + Unix. If you couldn't get that - combination to work before, please try again. - - .. change:: - :tags: mysql - :tickets: - - The connection.info keys the dialect uses to cache server - settings have changed and are now namespaced. - -.. changelog:: - :version: 0.4.4 - :released: Wed Mar 12 2008 - - .. change:: - :tags: sql - :tickets: 975 - - Can again create aliases of selects against textual FROM - clauses. - - .. change:: - :tags: sql - :tickets: - - The value of a bindparam() can be a callable, in which - case it's evaluated at statement execution time to get the - value. - - .. change:: - :tags: sql - :tickets: 978 - - Added exception wrapping/reconnect support to result set - fetching. Reconnect works for those databases that raise - a catchable data error during results (i.e. doesn't work - on MySQL) - - .. change:: - :tags: sql - :tickets: 936 - - Implemented two-phase API for "threadlocal" engine, via - engine.begin_twophase(), engine.prepare() - - .. change:: - :tags: sql - :tickets: 986 - - Fixed bug which was preventing UNIONS from being - cloneable. - - .. change:: - :tags: sql - :tickets: - - Added "bind" keyword argument to insert(), update(), - delete() and DDL(). The .bind property is now assignable - on those statements as well as on select(). - - .. change:: - :tags: sql - :tickets: - - Insert statements can now be compiled with extra "prefix" - words between INSERT and INTO, for vendor extensions like - MySQL's INSERT IGNORE INTO table. - - .. change:: - :tags: orm - :tickets: - - any(), has(), contains(), ~contains(), attribute level == - and != now work properly with self-referential relations - - the clause inside the EXISTS is aliased on the "remote" - side to distinguish it from the parent table. This - applies to single table self-referential as well as - inheritance-based self-referential. - - .. change:: - :tags: orm - :tickets: 985 - - Repaired behavior of == and != operators at the relation() - level when compared against NULL for one-to-one relations - - .. change:: - :tags: orm - :tickets: - - Fixed bug whereby session.expire() attributes were not - loading on an polymorphically-mapped instance mapped by a - select_table mapper. - - .. change:: - :tags: orm - :tickets: - - Added query.with_polymorphic() - specifies a list of - classes which descend from the base class, which will be - added to the FROM clause of the query. Allows subclasses - to be used within filter() criterion as well as eagerly - loads the attributes of those subclasses. - - .. change:: - :tags: orm - :tickets: - - Your cries have been heard: removing a pending item from - an attribute or collection with delete-orphan expunges the - item from the session; no FlushError is raised. Note that - if you session.save()'ed the pending item explicitly, the - attribute/collection removal still knocks it out. - - .. change:: - :tags: orm - :tickets: - - session.refresh() and session.expire() raise an error when - called on instances which are not persistent within the - session - - .. change:: - :tags: orm - :tickets: - - Fixed potential generative bug when the same Query was - used to generate multiple Query objects using join(). - - .. change:: - :tags: orm - :tickets: - - Fixed bug which was introduced in 0.4.3, whereby loading - an already-persistent instance mapped with joined table - inheritance would trigger a useless "secondary" load from - its joined table, when using the default "select" - polymorphic_fetch. This was due to attributes being - marked as expired during its first load and not getting - unmarked from the previous "secondary" load. Attributes - are now unexpired based on presence in __dict__ after any - load or commit operation succeeds. - - .. change:: - :tags: orm - :tickets: - - Deprecated Query methods apply_sum(), apply_max(), - apply_min(), apply_avg(). Better methodologies are - coming.... - - .. change:: - :tags: orm - :tickets: - - relation() can accept a callable for its first argument, - which returns the class to be related. This is in place - to assist declarative packages to define relations without - classes yet being in place. - - .. change:: - :tags: orm - :tickets: - - Added a new "higher level" operator called "of_type()": - used in join() as well as with any() and has(), qualifies - the subclass which will be used in filter criterion, e.g.: - - query.filter(Company.employees.of_type(Engineer). - any(Engineer.name=='foo')) - - or - - query.join(Company.employees.of_type(Engineer)). - filter(Engineer.name=='foo') - - .. change:: - :tags: orm - :tickets: - - Preventive code against a potential lost-reference bug in - flush(). - - .. change:: - :tags: orm - :tickets: - - Expressions used in filter(), filter_by() and others, when - they make usage of a clause generated from a relation - using the identity of a child object (e.g., - filter(Parent.child==)), evaluate the actual - primary key value of at execution time so that - the autoflush step of the Query can complete, thereby - populating the PK value of in the case that - was pending. - - .. change:: - :tags: orm - :tickets: - - setting the relation()-level order by to a column in the - many-to-many "secondary" table will now work with eager - loading, previously the "order by" wasn't aliased against - the secondary table's alias. - - .. change:: - :tags: orm - :tickets: - - Synonyms riding on top of existing descriptors are now - full proxies to those descriptors. - - .. change:: - :tags: dialects - :tickets: - - Invalid SQLite connection URLs now raise an error. - - .. change:: - :tags: dialects - :tickets: 981 - - postgres TIMESTAMP renders correctly - - .. change:: - :tags: dialects - :tickets: - - postgres PGArray is a "mutable" type by default; when used - with the ORM, mutable-style equality/ copy-on-write - techniques are used to test for changes. - - .. change:: - :tags: extensions - :tickets: - - a new super-small "declarative" extension has been added, - which allows Table and mapper() configuration to take - place inline underneath a class declaration. This - extension differs from ActiveMapper and Elixir in that it - does not redefine any SQLAlchemy semantics at all; literal - Column, Table and relation() constructs are used to define - the class behavior and table definition. - -.. changelog:: - :version: 0.4.3 - :released: Thu Feb 14 2008 - - .. change:: - :tags: sql - :tickets: - - Added "schema.DDL", an executable free-form DDL statement. - DDLs can be executed in isolation or attached to Table or - MetaData instances and executed automatically when those - objects are created and/or dropped. - - .. change:: - :tags: sql - :tickets: - - Table columns and constraints can be overridden on a an - existing table (such as a table that was already reflected) - using the 'useexisting=True' flag, which now takes into - account the arguments passed along with it. - - .. change:: - :tags: sql - :tickets: - - Added a callable-based DDL events interface, adds hooks - before and after Tables and MetaData create and drop. - - .. change:: - :tags: sql - :tickets: - - Added generative where() method to delete() and - update() constructs which return a new object with criterion - joined to existing criterion via AND, just like - select().where(). - - .. change:: - :tags: sql - :tickets: 727 - - Added "ilike()" operator to column operations. Compiles to - ILIKE on postgres, lower(x) LIKE lower(y) on all - others. - - .. change:: - :tags: sql - :tickets: 943 - - Added "now()" as a generic function; on SQLite, Oracle - and MSSQL compiles as "CURRENT_TIMESTAMP"; "now()" on - all others. - - .. change:: - :tags: sql - :tickets: 962 - - The startswith(), endswith(), and contains() operators now - concatenate the wildcard operator with the given operand in - SQL, i.e. "'%' || " in all cases, accept - text('something') operands properly - - .. change:: - :tags: sql - :tickets: 962 - - cast() accepts text('something') and other non-literal - operands properly - - .. change:: - :tags: sql - :tickets: - - fixed bug in result proxy where anonymously generated - column labels would not be accessible using their straight - string name - - .. change:: - :tags: sql - :tickets: - - Deferrable constraints can now be defined. - - .. change:: - :tags: sql - :tickets: 915 - - Added "autocommit=True" keyword argument to select() and - text(), as well as generative autocommit() method on - select(); for statements which modify the database through - some user-defined means other than the usual INSERT/UPDATE/ - DELETE etc. This flag will enable "autocommit" behavior - during execution if no transaction is in progress. - - .. change:: - :tags: sql - :tickets: - - The '.c.' attribute on a selectable now gets an entry for - every column expression in its columns clause. Previously, - "unnamed" columns like functions and CASE statements weren't - getting put there. Now they will, using their full string - representation if no 'name' is available. - - .. change:: - :tags: sql - :tickets: - - a CompositeSelect, i.e. any union(), union_all(), - intersect(), etc. now asserts that each selectable contains - the same number of columns. This conforms to the - corresponding SQL requirement. - - .. change:: - :tags: sql - :tickets: - - The anonymous 'label' generated for otherwise unlabeled - functions and expressions now propagates outwards at compile - time for expressions like select([select([func.foo()])]). - - .. change:: - :tags: sql - :tickets: - - Building on the above ideas, CompositeSelects now build up - their ".c." collection based on the names present in the - first selectable only; corresponding_column() now works - fully for all embedded selectables. - - .. change:: - :tags: sql - :tickets: - - Oracle and others properly encode SQL used for defaults like - sequences, etc., even if no unicode idents are used since - identifier preparer may return a cached unicode identifier. - - .. change:: - :tags: sql - :tickets: - - Column and clause comparisons to datetime objects on the - left hand side of the expression now work (d < table.c.col). - (datetimes on the RHS have always worked, the LHS exception - is a quirk of the datetime implementation.) - - .. change:: - :tags: orm - :tickets: - - Every Session.begin() must now be accompanied by a - corresponding commit() or rollback() unless the session is - closed with Session.close(). This also includes the begin() - which is implicit to a session created with - transactional=True. The biggest change introduced here is - that when a Session created with transactional=True raises - an exception during flush(), you must call - Session.rollback() or Session.close() in order for that - Session to continue after an exception. - - .. change:: - :tags: orm - :tickets: 961 - - Fixed merge() collection-doubling bug when merging transient - entities with backref'ed collections. - - .. change:: - :tags: orm - :tickets: - - merge(dont_load=True) does not accept transient entities, - this is in continuation with the fact that - merge(dont_load=True) does not accept any "dirty" objects - either. - - .. change:: - :tags: orm - :tickets: - - Added standalone "query" class attribute generated by a - scoped_session. This provides MyClass.query without using - Session.mapper. Use via: - - MyClass.query = Session.query_property() - - .. change:: - :tags: orm - :tickets: - - The proper error message is raised when trying to access - expired instance attributes with no session present - - .. change:: - :tags: orm - :tickets: - - dynamic_loader() / lazy="dynamic" now accepts and uses - the order_by parameter in the same way in which it works - with relation(). - - .. change:: - :tags: orm - :tickets: - - Added expire_all() method to Session. Calls expire() for - all persistent instances. This is handy in conjunction - with... - - .. change:: - :tags: orm - :tickets: - - Instances which have been partially or fully expired will - have their expired attributes populated during a regular - Query operation which affects those objects, preventing a - needless second SQL statement for each instance. - - .. change:: - :tags: orm - :tickets: 938 - - Dynamic relations, when referenced, create a strong - reference to the parent object so that the query still has a - parent to call against even if the parent is only created - (and otherwise dereferenced) within the scope of a single - expression. - - .. change:: - :tags: orm - :tickets: - - Added a mapper() flag "eager_defaults". When set to True, - defaults that are generated during an INSERT or UPDATE - operation are post-fetched immediately, instead of being - deferred until later. This mimics the old 0.3 behavior. - - .. change:: - :tags: orm - :tickets: - - query.join() can now accept class-mapped attributes as - arguments. These can be used in place or in any combination - with strings. In particular this allows construction of - joins to subclasses on a polymorphic relation, i.e.: - - query(Company).join(['employees', Engineer.name]) - - .. change:: - :tags: orm, ('employees', people.join(engineer)), Engineer.name - :tickets: - - query.join() can also accept tuples of attribute name/some - selectable as arguments. This allows construction of joins - *from* subclasses of a polymorphic relation, i.e.: - - query(Company).\ - join( - - ) - - .. change:: - :tags: orm - :tickets: - - General improvements to the behavior of join() in - conjunction with polymorphic mappers, i.e. joining from/to - polymorphic mappers and properly applying aliases. - - .. change:: - :tags: orm - :tickets: 933 - - Fixed/improved behavior when a mapper determines the natural - "primary key" of a mapped join, it will more effectively - reduce columns which are equivalent via foreign key - relation. This affects how many arguments need to be sent - to query.get(), among other things. - - .. change:: - :tags: orm - :tickets: 946 - - The lazy loader can now handle a join condition where the - "bound" column (i.e. the one that gets the parent id sent as - a bind parameter) appears more than once in the join - condition. Specifically this allows the common task of a - relation() which contains a parent-correlated subquery, such - as "select only the most recent child item". - - .. change:: - :tags: orm - :tickets: - - Fixed bug in polymorphic inheritance where an incorrect - exception is raised when base polymorphic_on column does not - correspond to any columns within the local selectable of an - inheriting mapper more than one level deep - - .. change:: - :tags: orm - :tickets: - - Fixed bug in polymorphic inheritance which made it difficult - to set a working "order_by" on a polymorphic mapper. - - .. change:: - :tags: orm - :tickets: - - Fixed a rather expensive call in Query that was slowing down - polymorphic queries. - - .. change:: - :tags: orm - :tickets: 954 - - "Passive defaults" and other "inline" defaults can now be - loaded during a flush() call if needed; in particular, this - allows constructing relations() where a foreign key column - references a server-side-generated, non-primary-key - column. - - .. change:: - :tags: orm - :tickets: - - Additional Session transaction fixes/changes: - - Fixed bug with session transaction management: parent - transactions weren't started on the connection when - adding a connection to a nested transaction. - - - session.transaction now always refers to the innermost - active transaction, even when commit/rollback are called - directly on the session transaction object. - - - Two-phase transactions can now be prepared. - - - When preparing a two-phase transaction fails on one - connection, all the connections are rolled back. - - - session.close() didn't close all transactions when - nested transactions were used. - - - rollback() previously erroneously set the current - transaction directly to the parent of the transaction - that could be rolled back to. Now it rolls back the next - transaction up that can handle it, but sets the current - transaction to its parent and inactivates the - transactions in between. Inactive transactions can only - be rolled back or closed, any other call results in an - error. - - - autoflush for commit() wasn't flushing for simple - subtransactions. - - - unitofwork flush didn't close the failed transaction - when the session was not in a transaction and committing - the transaction failed. - - .. change:: - :tags: orm - :tickets: 964, 940 - - Miscellaneous tickets: - - .. change:: - :tags: general - :tickets: - - Fixed a variety of hidden and some not-so-hidden - compatibility issues for Python 2.3, thanks to new support - for running the full test suite on 2.3. - - .. change:: - :tags: general - :tickets: - - Warnings are now issued as type exceptions.SAWarning. - - .. change:: - :tags: dialects - :tickets: - - Better support for schemas in SQLite (linked in by ATTACH - DATABASE ... AS name). In some cases in the past, schema - names were omitted from generated SQL for SQLite. This is - no longer the case. - - .. change:: - :tags: dialects - :tickets: - - table_names on SQLite now picks up temporary tables as well. - - .. change:: - :tags: dialects - :tickets: - - Auto-detect an unspecified MySQL ANSI_QUOTES mode during - reflection operations, support for changing the mode - midstream. Manual mode setting is still required if no - reflection is used. - - .. change:: - :tags: dialects - :tickets: - - Fixed reflection of TIME columns on SQLite. - - .. change:: - :tags: dialects - :tickets: 580 - - Finally added PGMacAddr type to postgres - - .. change:: - :tags: dialects - :tickets: - - Reflect the sequence associated to a PK field (typically - with a BEFORE INSERT trigger) under Firebird - - .. change:: - :tags: dialects - :tickets: 941 - - Oracle assembles the correct columns in the result set - column mapping when generating a LIMIT/OFFSET subquery, - allows columns to map properly to result sets even if - long-name truncation kicks in - - .. change:: - :tags: dialects - :tickets: - - MSSQL now includes EXEC in the _is_select regexp, which - should allow row-returning stored procedures to be used. - - .. change:: - :tags: dialects - :tickets: - - MSSQL now includes an experimental implementation of - LIMIT/OFFSET using the ANSI SQL row_number() function, so it - requires MSSQL-2005 or higher. To enable the feature, add - "has_window_funcs" to the keyword arguments for connect, or - add "?has_window_funcs=1" to your dburi query arguments. - - .. change:: - :tags: ext - :tickets: - - Changed ext.activemapper to use a non-transactional session - for the objectstore. - - .. change:: - :tags: ext - :tickets: - - Fixed output order of "['a'] + obj.proxied" binary operation - on association-proxied lists. - -.. changelog:: - :version: 0.4.2p3 - :released: Wed Jan 09 2008 - - .. change:: - :tags: general - :tickets: - - sub version numbering scheme changed to suite - setuptools version number rules; easy_install -u - should now get this version over 0.4.2. - - .. change:: - :tags: sql - :tickets: 912 - - Text type is properly exported now and does not - raise a warning on DDL create; String types with no - length only raise warnings during CREATE TABLE - - .. change:: - :tags: sql - :tickets: - - new UnicodeText type is added, to specify an - encoded, unlengthed Text type - - .. change:: - :tags: sql - :tickets: - - fixed bug in union() so that select() statements - which don't derive from FromClause objects can be - unioned - - .. change:: - :tags: orm - :tickets: - - fixed bug with session.dirty when using "mutable - scalars" (such as PickleTypes) - - .. change:: - :tags: orm - :tickets: - - added a more descriptive error message when flushing - on a relation() that has non-locally-mapped columns - in its primary or secondary join condition - - .. change:: - :tags: dialects - :tickets: - - Fixed reflection of mysql empty string column - defaults. - - .. change:: - :tags: sql - :tickets: 912 - - changed name of TEXT to Text since its a "generic" - type; TEXT name is deprecated until 0.5. The - "upgrading" behavior of String to Text when no - length is present is also deprecated until 0.5; will - issue a warning when used for CREATE TABLE - statements (String with no length for SQL expression - purposes is still fine) - - .. change:: - :tags: sql - :tickets: 924 - - generative select.order_by(None) / group_by(None) - was not managing to reset order by/group by - criterion, fixed - - .. change:: - :tags: orm - :tickets: - - suppressing *all* errors in - InstanceState.__cleanup() now. - - .. change:: - :tags: orm - :tickets: 922 - - fixed an attribute history bug whereby assigning a - new collection to a collection-based attribute which - already had pending changes would generate incorrect - history - - .. change:: - :tags: orm - :tickets: 925 - - fixed delete-orphan cascade bug whereby setting the - same object twice to a scalar attribute could log it - as an orphan - - .. change:: - :tags: orm - :tickets: - - Fixed cascades on a += assignment to a list-based - relation. - - .. change:: - :tags: orm - :tickets: 919 - - synonyms can now be created against props that don't - exist yet, which are later added via add_property(). - This commonly includes backrefs. (i.e. you can make - synonyms for backrefs without worrying about the - order of operations) - - .. change:: - :tags: orm - :tickets: - - fixed bug which could occur with polymorphic "union" - mapper which falls back to "deferred" loading of - inheriting tables - - .. change:: - :tags: orm - :tickets: - - the "columns" collection on a mapper/mapped class - (i.e. 'c') is against the mapped table, not the - select_table in the case of polymorphic "union" - loading (this shouldn't be noticeable). - - .. change:: - :tags: ext - :tickets: - - '+', '*', '+=' and '\*=' support for association - proxied lists. - - .. change:: - :tags: dialects - :tickets: 923 - - mssql - narrowed down the test for "date"/"datetime" - in MSDate/ MSDateTime subclasses so that incoming - "datetime" objects don't get mis-interpreted as - "date" objects and vice versa. - - .. change:: - :tags: orm - :tickets: - - fixed fairly critical bug whereby the same instance could be listed - more than once in the unitofwork.new collection; most typically - reproduced when using a combination of inheriting mappers and - ScopedSession.mapper, as the multiple __init__ calls per instance - could save() the object with distinct _state objects - - .. change:: - :tags: orm - :tickets: - - added very rudimentary yielding iterator behavior to Query. Call - query.yield_per() and evaluate the Query in an - iterative context; every collection of N rows will be packaged up - and yielded. Use this method with extreme caution since it does - not attempt to reconcile eagerly loaded collections across - result batch boundaries, nor will it behave nicely if the same - instance occurs in more than one batch. This means that an eagerly - loaded collection will get cleared out if it's referenced in more than - one batch, and in all cases attributes will be overwritten on instances - that occur in more than one batch. - - .. change:: - :tags: orm - :tickets: 920 - - Fixed in-place set mutation operators for set collections and association - proxied sets. - - .. change:: - :tags: dialects - :tickets: 913 - - Fixed the missing call to subtype result processor for the PGArray - type. - -.. changelog:: - :version: 0.4.2 - :released: Wed Jan 02 2008 - - .. change:: - :tags: sql - :tickets: 615 - - generic functions ! we introduce a database of known SQL functions, such - as current_timestamp, coalesce, and create explicit function objects - representing them. These objects have constrained argument lists, are - type aware, and can compile in a dialect-specific fashion. So saying - func.char_length("foo", "bar") raises an error (too many args), - func.coalesce(datetime.date(2007, 10, 5), datetime.date(2005, 10, 15)) - knows that its return type is a Date. We only have a few functions - represented so far but will continue to add to the system - - .. change:: - :tags: sql - :tickets: - - auto-reconnect support improved; a Connection can now automatically - reconnect after its underlying connection is invalidated, without - needing to connect() again from the engine. This allows an ORM session - bound to a single Connection to not need a reconnect. - Open transactions on the Connection must be rolled back after an invalidation - of the underlying connection else an error is raised. Also fixed - bug where disconnect detect was not being called for cursor(), rollback(), - or commit(). - - .. change:: - :tags: sql - :tickets: - - added new flag to String and create_engine(), - assert_unicode=(True|False|'warn'\|None). Defaults to `False` or `None` on - create_engine() and String, `'warn'` on the Unicode type. When `True`, - results in all unicode conversion operations raising an exception when a - non-unicode bytestring is passed as a bind parameter. 'warn' results - in a warning. It is strongly advised that all unicode-aware applications - make proper use of Python unicode objects (i.e. u'hello' and not 'hello') - so that data round trips accurately. - - .. change:: - :tags: sql - :tickets: - - generation of "unique" bind parameters has been simplified to use the same - "unique identifier" mechanisms as everything else. This doesn't affect - user code, except any code that might have been hardcoded against the generated - names. Generated bind params now have the form "_", - whereas before only the second bind of the same name would have this form. - - .. change:: - :tags: sql - :tickets: - - select().as_scalar() will raise an exception if the select does not have - exactly one expression in its columns clause. - - .. change:: - :tags: sql - :tickets: - - bindparam() objects themselves can be used as keys for execute(), i.e. - statement.execute({bind1:'foo', bind2:'bar'}) - - .. change:: - :tags: sql - :tickets: - - added new methods to TypeDecorator, process_bind_param() and - process_result_value(), which automatically take advantage of the processing - of the underlying type. Ideal for using with Unicode or Pickletype. - TypeDecorator should now be the primary way to augment the behavior of any - existing type including other TypeDecorator subclasses such as PickleType. - - .. change:: - :tags: sql - :tickets: - - selectables (and others) will issue a warning when two columns in - their exported columns collection conflict based on name. - - .. change:: - :tags: sql - :tickets: 890 - - tables with schemas can still be used in sqlite, firebird, - schema name just gets dropped - - .. change:: - :tags: sql - :tickets: - - changed the various "literal" generation functions to use an anonymous - bind parameter. not much changes here except their labels now look - like ":param_1", ":param_2" instead of ":literal" - - .. change:: - :tags: sql - :tickets: - - column labels in the form "tablename.columname", i.e. with a dot, are now - supported. - - .. change:: - :tags: sql - :tickets: - - from_obj keyword argument to select() can be a scalar or a list. - - .. change:: - :tags: orm - :tickets: 871 - - a major behavioral change to collection-based backrefs: they no - longer trigger lazy loads ! "reverse" adds and removes - are queued up and are merged with the collection when it is - actually read from and loaded; but do not trigger a load beforehand. - For users who have noticed this behavior, this should be much more - convenient than using dynamic relations in some cases; for those who - have not, you might notice your apps using a lot fewer queries than - before in some situations. - - .. change:: - :tags: orm - :tickets: - - mutable primary key support is added. primary key columns can be - changed freely, and the identity of the instance will change upon - flush. In addition, update cascades of foreign key referents (primary - key or not) along relations are supported, either in tandem with the - database's ON UPDATE CASCADE (required for DB's like Postgres) or - issued directly by the ORM in the form of UPDATE statements, by setting - the flag "passive_cascades=False". - - .. change:: - :tags: orm - :tickets: 490 - - inheriting mappers now inherit the MapperExtensions of their parent - mapper directly, so that all methods for a particular MapperExtension - are called for subclasses as well. As always, any MapperExtension - can return either EXT_CONTINUE to continue extension processing - or EXT_STOP to stop processing. The order of mapper resolution is: - . - - Note that if you instantiate the same extension class separately - and then apply it individually for two mappers in the same inheritance - chain, the extension will be applied twice to the inheriting class, - and each method will be called twice. - - To apply a mapper extension explicitly to each inheriting class but - have each method called only once per operation, use the same - instance of the extension for both mappers. - - .. change:: - :tags: orm - :tickets: 907 - - MapperExtension.before_update() and after_update() are now called - symmetrically; previously, an instance that had no modified column - attributes (but had a relation() modification) could be called with - before_update() but not after_update() - - .. change:: - :tags: orm - :tickets: - - columns which are missing from a Query's select statement - now get automatically deferred during load. - - .. change:: - :tags: orm - :tickets: 908 - - mapped classes which extend "object" and do not provide an - __init__() method will now raise TypeError if non-empty \*args - or \**kwargs are present at instance construction time (and are - not consumed by any extensions such as the scoped_session mapper), - consistent with the behavior of normal Python classes - - .. change:: - :tags: orm - :tickets: 899 - - fixed Query bug when filter_by() compares a relation against None - - .. change:: - :tags: orm - :tickets: - - improved support for pickling of mapped entities. Per-instance - lazy/deferred/expired callables are now serializable so that - they serialize and deserialize with _state. - - .. change:: - :tags: orm - :tickets: 801 - - new synonym() behavior: an attribute will be placed on the mapped - class, if one does not exist already, in all cases. if a property - already exists on the class, the synonym will decorate the property - with the appropriate comparison operators so that it can be used in - column expressions just like any other mapped attribute (i.e. usable in - filter(), etc.) the "proxy=True" flag is deprecated and no longer means - anything. Additionally, the flag "map_column=True" will automatically - generate a ColumnProperty corresponding to the name of the synonym, - i.e.: 'somename':synonym('_somename', map_column=True) will map the - column named 'somename' to the attribute '_somename'. See the example - in the mapper docs. - - .. change:: - :tags: orm - :tickets: - - Query.select_from() now replaces all existing FROM criterion with - the given argument; the previous behavior of constructing a list - of FROM clauses was generally not useful as is required - filter() calls to create join criterion, and new tables introduced - within filter() already add themselves to the FROM clause. The - new behavior allows not just joins from the main table, but select - statements as well. Filter criterion, order bys, eager load - clauses will be "aliased" against the given statement. - - .. change:: - :tags: orm - :tickets: - - this month's refactoring of attribute instrumentation changes - the "copy-on-load" behavior we've had since midway through 0.3 - with "copy-on-modify" in most cases. This takes a sizable chunk - of latency out of load operations and overall does less work - as only attributes which are actually modified get their - "committed state" copied. Only "mutable scalar" attributes - (i.e. a pickled object or other mutable item), the reason for - the copy-on-load change in the first place, retain the old - behavior. - - .. change:: - :tags: attrname, orm - :tickets: - - a slight behavioral change to attributes is, del'ing an attribute - does *not* cause the lazyloader of that attribute to fire off again; - the "del" makes the effective value of the attribute "None". To - re-trigger the "loader" for an attribute, use - session.expire(instance,). - - .. change:: - :tags: orm - :tickets: - - query.filter(SomeClass.somechild == None), when comparing - a many-to-one property to None, properly generates "id IS NULL" - including that the NULL is on the right side. - - .. change:: - :tags: orm - :tickets: - - query.order_by() takes into account aliased joins, i.e. - query.join('orders', aliased=True).order_by(Order.id) - - .. change:: - :tags: orm - :tickets: - - eagerload(), lazyload(), eagerload_all() take an optional - second class-or-mapper argument, which will select the mapper - to apply the option towards. This can select among other - mappers which were added using add_entity(). - - .. change:: - :tags: orm - :tickets: - - eagerloading will work with mappers added via add_entity(). - - .. change:: - :tags: orm - :tickets: - - added "cascade delete" behavior to "dynamic" relations just like - that of regular relations. if passive_deletes flag (also just added) - is not set, a delete of the parent item will trigger a full load of - the child items so that they can be deleted or updated accordingly. - - .. change:: - :tags: orm - :tickets: - - also with dynamic, implemented correct count() behavior as well - as other helper methods. - - .. change:: - :tags: orm - :tickets: - - fix to cascades on polymorphic relations, such that cascades - from an object to a polymorphic collection continue cascading - along the set of attributes specific to each element in the collection. - - .. change:: - :tags: orm - :tickets: 893 - - query.get() and query.load() do not take existing filter or other - criterion into account; these methods *always* look up the given id - in the database or return the current instance from the identity map, - disregarding any existing filter, join, group_by or other criterion - which has been configured. - - .. change:: - :tags: orm - :tickets: 883 - - added support for version_id_col in conjunction with inheriting mappers. - version_id_col is typically set on the base mapper in an inheritance - relationship where it takes effect for all inheriting mappers. - - .. change:: - :tags: orm - :tickets: - - relaxed rules on column_property() expressions having labels; any - ColumnElement is accepted now, as the compiler auto-labels non-labeled - ColumnElements now. a selectable, like a select() statement, still - requires conversion to ColumnElement via as_scalar() or label(). - - .. change:: - :tags: orm - :tickets: - - fixed backref bug where you could not del instance.attr if attr - was None - - .. change:: - :tags: orm - :tickets: - - several ORM attributes have been removed or made private: - mapper.get_attr_by_column(), mapper.set_attr_by_column(), - mapper.pks_by_table, mapper.cascade_callable(), - MapperProperty.cascade_callable(), mapper.canload(), - mapper.save_obj(), mapper.delete_obj(), mapper._mapper_registry, - attributes.AttributeManager - - .. change:: - :tags: orm - :tickets: - - Assigning an incompatible collection type to a relation attribute now - raises TypeError instead of sqlalchemy's ArgumentError. - - .. change:: - :tags: orm - :tickets: 886 - - Bulk assignment of a MappedCollection now raises an error if a key in the - incoming dictionary does not match the key that the collection's keyfunc - would use for that value. - - .. change:: - :tags: orm, newval1, newval2 - :tickets: - - Custom collections can now specify a @converter method to translate - objects used in "bulk" assignment into a stream of values, as in: - - .. sourcecode:: text - - obj.col = - # or - obj.dictcol = {'foo': newval1, 'bar': newval2} - - The MappedCollection uses this hook to ensure that incoming key/value - pairs are sane from the collection's perspective. - - .. change:: - :tags: orm - :tickets: 872 - - fixed endless loop issue when using lazy="dynamic" on both - sides of a bi-directional relationship - - .. change:: - :tags: orm - :tickets: 904 - - more fixes to the LIMIT/OFFSET aliasing applied with Query + eagerloads, - in this case when mapped against a select statement - - .. change:: - :tags: orm - :tickets: - - fix to self-referential eager loading such that if the same mapped - instance appears in two or more distinct sets of columns in the same - result set, its eagerly loaded collection will be populated regardless - of whether or not all of the rows contain a set of "eager" columns for - that collection. this would also show up as a KeyError when fetching - results with join_depth turned on. - - .. change:: - :tags: orm - :tickets: - - fixed bug where Query would not apply a subquery to the SQL when LIMIT - was used in conjunction with an inheriting mapper where the eager - loader was only in the parent mapper. - - .. change:: - :tags: orm - :tickets: - - clarified the error message which occurs when you try to update() - an instance with the same identity key as an instance already present - in the session. - - .. change:: - :tags: orm - :tickets: - - some clarifications and fixes to merge(instance, dont_load=True). - fixed bug where lazy loaders were getting disabled on returned instances. - Also, we currently do not support merging an instance which has uncommitted - changes on it, in the case that dont_load=True is used....this will - now raise an error. This is due to complexities in merging the - "committed state" of the given instance to correctly correspond to the - newly copied instance, as well as other modified state. - Since the use case for dont_load=True is caching, the given instances - shouldn't have any uncommitted changes on them anyway. - We also copy the instances over without using any events now, so that - the 'dirty' list on the new session remains unaffected. - - .. change:: - :tags: orm - :tickets: - - fixed bug which could arise when using session.begin_nested() in conjunction - with more than one level deep of enclosing session.begin() statements - - .. change:: - :tags: orm - :tickets: 914 - - fixed session.refresh() with instance that has custom entity_name - - .. change:: - :tags: dialects - :tickets: - - sqlite SLDate type will not erroneously render "microseconds" portion - of a datetime or time object. - - .. change:: - :tags: dialects - :tickets: 902 - - oracle - - added disconnect detection support for Oracle - - some cleanup to binary/raw types so that cx_oracle.LOB is detected - on an ad-hoc basis - - .. change:: - :tags: dialects - :tickets: 824, 839, 842, 901 - - MSSQL - - PyODBC no longer has a global "set nocount on". - - Fix non-identity integer PKs on autoload - - Better support for convert_unicode - - Less strict date conversion for pyodbc/adodbapi - - Schema-qualified tables / autoload - - .. change:: - :tags: firebird, backend - :tickets: 410 - - does properly reflect domains (partially fixing) and - PassiveDefaults - - .. change:: - :tags: 3562, firebird, backend - :tickets: - - reverted to use default poolclass (was set to SingletonThreadPool in - 0.4.0 for test purposes) - - .. change:: - :tags: firebird, backend - :tickets: - - map func.length() to 'char_length' (easily overridable with the UDF - 'strlen' on old versions of Firebird) - -.. changelog:: - :version: 0.4.1 - :released: Sun Nov 18 2007 - - .. change:: - :tags: sql - :tickets: - - the "shortname" keyword parameter on bindparam() has been - deprecated. - - .. change:: - :tags: sql - :tickets: - - Added contains operator (generates a "LIKE %%" clause). - - .. change:: - :tags: sql - :tickets: - - anonymous column expressions are automatically labeled. - e.g. select([x* 5]) produces "SELECT x * 5 AS anon_1". - This allows the labelname to be present in the cursor.description - which can then be appropriately matched to result-column processing - rules. (we can't reliably use positional tracking for result-column - matches since text() expressions may represent multiple columns). - - .. change:: - :tags: sql - :tickets: - - operator overloading is now controlled by TypeEngine objects - the - one built-in operator overload so far is String types overloading - '+' to be the string concatenation operator. - User-defined types can also define their own operator overloading - by overriding the adapt_operator(self, op) method. - - .. change:: - :tags: sql - :tickets: 819 - - untyped bind parameters on the right side of a binary expression - will be assigned the type of the left side of the operation, to better - enable the appropriate bind parameter processing to take effect - - .. change:: - :tags: sql - :tickets: 833 - - Removed regular expression step from most statement compilations. - Also fixes - - .. change:: - :tags: sql - :tickets: - - Fixed empty (zero column) sqlite inserts, allowing inserts on - autoincrementing single column tables. - - .. change:: - :tags: sql - :tickets: - - Fixed expression translation of text() clauses; this repairs various - ORM scenarios where literal text is used for SQL expressions - - .. change:: - :tags: sql - :tickets: - - Removed ClauseParameters object; compiled.params returns a regular - dictionary now, as well as result.last_inserted_params() / - last_updated_params(). - - .. change:: - :tags: sql - :tickets: - - Fixed INSERT statements w.r.t. primary key columns that have - SQL-expression based default generators on them; SQL expression - executes inline as normal but will not trigger a "postfetch" condition - for the column, for those DB's who provide it via cursor.lastrowid - - .. change:: - :tags: sql - :tickets: 844 - - func. objects can be pickled/unpickled - - .. change:: - :tags: sql - :tickets: - - rewrote and simplified the system used to "target" columns across - selectable expressions. On the SQL side this is represented by the - "corresponding_column()" method. This method is used heavily by the ORM - to "adapt" elements of an expression to similar, aliased expressions, - as well as to target result set columns originally bound to a - table or selectable to an aliased, "corresponding" expression. The new - rewrite features completely consistent and accurate behavior. - - .. change:: - :tags: sql - :tickets: 573 - - Added a field ("info") for storing arbitrary data on schema items - - .. change:: - :tags: sql - :tickets: - - The "properties" collection on Connections has been renamed "info" to - match schema's writable collections. Access is still available via - the "properties" name until 0.5. - - .. change:: - :tags: sql - :tickets: - - fixed the close() method on Transaction when using strategy='threadlocal' - - .. change:: - :tags: sql - :tickets: 853 - - fix to compiled bind parameters to not mistakenly populate None - - .. change:: - :tags: sql - :tickets: - - ._execute_clauseelement becomes a public method - Connectable.execute_clauseelement - - .. change:: - :tags: orm - :tickets: 843 - - eager loading with LIMIT/OFFSET applied no longer adds the primary - table joined to a limited subquery of itself; the eager loads now - join directly to the subquery which also provides the primary table's - columns to the result set. This eliminates a JOIN from all eager loads - with LIMIT/OFFSET. - - .. change:: - :tags: orm - :tickets: 802 - - session.refresh() and session.expire() now support an additional argument - "attribute_names", a list of individual attribute keynames to be refreshed - or expired, allowing partial reloads of attributes on an already-loaded - instance. - - .. change:: - :tags: orm - :tickets: 767 - - added op() operator to instrumented attributes; i.e. - User.name.op('ilike')('%somename%') - - .. change:: - :tags: orm - :tickets: 676 - - Mapped classes may now define __eq__, __hash__, and __nonzero__ methods - with arbitrary semantics. The orm now handles all mapped instances on - an identity-only basis. (e.g. 'is' vs '==') - - .. change:: - :tags: orm - :tickets: - - the "properties" accessor on Mapper is removed; it now throws an informative - exception explaining the usage of mapper.get_property() and - mapper.iterate_properties - - .. change:: - :tags: orm - :tickets: - - added having() method to Query, applies HAVING to the generated statement - in the same way as filter() appends to the WHERE clause. - - .. change:: - :tags: orm - :tickets: 777 - - The behavior of query.options() is now fully based on paths, i.e. an - option such as eagerload_all('x.y.z.y.x') will apply eagerloading to - only those paths, i.e. and not 'x.y.x'; eagerload('children.children') - applies only to exactly two-levels deep, etc. - - .. change:: - :tags: orm - :tickets: - - PickleType will compare using `==` when set up with mutable=False, - and not the `is` operator. To use `is` or any other comparator, send - in a custom comparison function using PickleType(comparator=my_custom_comparator). - - .. change:: - :tags: orm - :tickets: 848 - - query doesn't throw an error if you use distinct() and an order_by() - containing UnaryExpressions (or other) together - - .. change:: - :tags: orm - :tickets: 786 - - order_by() expressions from joined tables are properly added to columns - clause when using distinct() - - .. change:: - :tags: orm - :tickets: 858 - - fixed error where Query.add_column() would not accept a class-bound - attribute as an argument; Query also raises an error if an invalid - argument was sent to add_column() (at instances() time) - - .. change:: - :tags: orm - :tickets: - - added a little more checking for garbage-collection dereferences in - InstanceState.__cleanup() to reduce "gc ignored" errors on app - shutdown - - .. change:: - :tags: orm - :tickets: - - The session API has been solidified: - - .. change:: - :tags: orm - :tickets: 840 - - It's an error to session.save() an object which is already - persistent - - .. change:: - :tags: orm - :tickets: - - It's an error to session.delete() an object which is *not* - persistent. - - .. change:: - :tags: orm - :tickets: - - session.update() and session.delete() raise an error when updating - or deleting an instance that is already in the session with a - different identity. - - .. change:: - :tags: orm - :tickets: - - The session checks more carefully when determining "object X already - in another session"; e.g. if you pickle a series of objects and - unpickle (i.e. as in a Pylons HTTP session or similar), they can go - into a new session without any conflict - - .. change:: - :tags: orm - :tickets: - - merge() includes a keyword argument "dont_load=True". setting this - flag will cause the merge operation to not load any data from the - database in response to incoming detached objects, and will accept - the incoming detached object as though it were already present in - that session. Use this to merge detached objects from external - caching systems into the session. - - .. change:: - :tags: orm - :tickets: - - Deferred column attributes no longer trigger a load operation when the - attribute is assigned to. In those cases, the newly assigned value - will be present in the flushes' UPDATE statement unconditionally. - - .. change:: - :tags: orm - :tickets: 834 - - Fixed a truncation error when re-assigning a subset of a collection - (obj.relation = obj.relation[1:]) - - .. change:: - :tags: orm - :tickets: 832 - - De-cruftified backref configuration code, backrefs which step on - existing properties now raise an error - - .. change:: - :tags: orm - :tickets: 831 - - Improved behavior of add_property() etc., fixed involving - synonym/deferred. - - .. change:: - :tags: orm - :tickets: - - Fixed clear_mappers() behavior to better clean up after itself. - - .. change:: - :tags: orm - :tickets: 841 - - Fix to "row switch" behavior, i.e. when an INSERT/DELETE is combined - into a single UPDATE; many-to-many relations on the parent object - update properly. - - .. change:: - :tags: orm - :tickets: - - Fixed __hash__ for association proxy- these collections are unhashable, - just like their mutable Python counterparts. - - .. change:: - :tags: orm - :tickets: - - Added proxying of save_or_update, __contains__ and __iter__ methods for - scoped sessions. - - .. change:: - :tags: orm - :tickets: 852 - - fixed very hard-to-reproduce issue where by the FROM clause of Query - could get polluted by certain generative calls - - .. change:: - :tags: dialects - :tickets: - - Added experimental support for MaxDB (versions >= 7.6.03.007 only). - - .. change:: - :tags: dialects - :tickets: - - oracle will now reflect "DATE" as an OracleDateTime column, not - OracleDate - - .. change:: - :tags: dialects - :tickets: 847 - - added awareness of schema name in oracle table_names() function, - fixes metadata.reflect(schema='someschema') - - .. change:: - :tags: dialects - :tickets: - - MSSQL anonymous labels for selection of functions made deterministic - - .. change:: - :tags: dialects - :tickets: - - sqlite will reflect "DECIMAL" as a numeric column. - - .. change:: - :tags: dialects - :tickets: 828 - - Made access dao detection more reliable - - .. change:: - :tags: dialects - :tickets: - - Renamed the Dialect attribute 'preexecute_sequences' to - 'preexecute_pk_sequences'. An attribute proxy is in place for - out-of-tree dialects using the old name. - - .. change:: - :tags: dialects - :tickets: - - Added test coverage for unknown type reflection. Fixed sqlite/mysql - handling of type reflection for unknown types. - - .. change:: - :tags: dialects - :tickets: - - Added REAL for mysql dialect (for folks exploiting the - REAL_AS_FLOAT sql mode). - - .. change:: - :tags: dialects - :tickets: - - mysql Float, MSFloat and MSDouble constructed without arguments - now produce no-argument DDL, e.g.'FLOAT'. - - .. change:: - :tags: misc - :tickets: - - Removed unused util.hash(). - -.. changelog:: - :version: 0.4.0 - :released: Wed Oct 17 2007 - - .. change:: - :tags: - :tickets: - - (see 0.4.0beta1 for the start of major changes against 0.3, - as well as https://www.sqlalchemy.org/trac/wiki/WhatsNewIn04 ) - - .. change:: - :tags: - :tickets: 785 - - Added initial Sybase support (mxODBC so far) - - .. change:: - :tags: - :tickets: - - Added partial index support for PostgreSQL. Use the postgres_where keyword - on the Index. - - .. change:: - :tags: - :tickets: 817 - - string-based query param parsing/config file parser understands - wider range of string values for booleans - - .. change:: - :tags: - :tickets: 813 - - backref remove object operation doesn't fail if the other-side - collection doesn't contain the item, supports noload collections - - .. change:: - :tags: - :tickets: 818 - - removed __len__ from "dynamic" collection as it would require issuing - a SQL "count()" operation, thus forcing all list evaluations to issue - redundant SQL - - .. change:: - :tags: - :tickets: 816 - - inline optimizations added to locate_dirty() which can greatly speed up - repeated calls to flush(), as occurs with autoflush=True - - .. change:: - :tags: - :tickets: - - The IdentifierPreprarer's _requires_quotes test is now regex based. Any - out-of-tree dialects that provide custom sets of legal_characters or - illegal_initial_characters will need to move to regexes or override - _requires_quotes. - - .. change:: - :tags: - :tickets: - - Firebird has supports_sane_rowcount and supports_sane_multi_rowcount set - to False due to ticket #370 (right way). - - .. change:: - :tags: - :tickets: - - Improvements and fixes on Firebird reflection: - * FBDialect now mimics OracleDialect, regarding case-sensitivity of TABLE and - COLUMN names (see 'case_sensitive remotion' topic on this current file). - * FBDialect.table_names() doesn't bring system tables (ticket:796). - * FB now reflects Column's nullable property correctly. - - .. change:: - :tags: - :tickets: - - Fixed SQL compiler's awareness of top-level column labels as used - in result-set processing; nested selects which contain the same column - names don't affect the result or conflict with result-column metadata. - - .. change:: - :tags: - :tickets: - - query.get() and related functions (like many-to-one lazyloading) - use compile-time-aliased bind parameter names, to prevent - name conflicts with bind parameters that already exist in the - mapped selectable. - - .. change:: - :tags: - :tickets: 795 - - Fixed three- and multi-level select and deferred inheritance loading - (i.e. abc inheritance with no select_table). - - .. change:: - :tags: - :tickets: - - Ident passed to id_chooser in shard.py always a list. - - .. change:: - :tags: - :tickets: - - The no-arg ResultProxy._row_processor() is now the class attribute - `_process_row`. - - .. change:: - :tags: - :tickets: 797 - - Added support for returning values from inserts and updates for - PostgreSQL 8.2+. - - .. change:: - :tags: - :tickets: - - PG reflection, upon seeing the default schema name being used explicitly - as the "schema" argument in a Table, will assume that this is the - user's desired convention, and will explicitly set the "schema" argument - in foreign-key-related reflected tables, thus making them match only - with Table constructors that also use the explicit "schema" argument - (even though its the default schema). - In other words, SA assumes the user is being consistent in this usage. - - .. change:: - :tags: - :tickets: 808 - - fixed sqlite reflection of BOOL/BOOLEAN - - .. change:: - :tags: - :tickets: - - Added support for UPDATE with LIMIT on mysql. - - .. change:: - :tags: - :tickets: 803 - - null foreign key on a m2o doesn't trigger a lazyload - - .. change:: - :tags: - :tickets: 800 - - oracle does not implicitly convert to unicode for non-typed result - sets (i.e. when no TypeEngine/String/Unicode type is even being used; - previously it was detecting DBAPI types and converting regardless). - should fix - - .. change:: - :tags: - :tickets: 806 - - fix to anonymous label generation of long table/column names - - .. change:: - :tags: - :tickets: - - Firebird dialect now uses SingletonThreadPool as poolclass. - - .. change:: - :tags: - :tickets: - - Firebird now uses dialect.preparer to format sequences names - - .. change:: - :tags: - :tickets: 810 - - Fixed breakage with postgres and multiple two-phase transactions. Two-phase - commits and rollbacks didn't automatically end up with a new transaction - as the usual dbapi commits/rollbacks do. - - .. change:: - :tags: - :tickets: - - Added an option to the _ScopedExt mapper extension to not automatically - save new objects to session on object initialization. - - .. change:: - :tags: - :tickets: - - fixed Oracle non-ansi join syntax - - .. change:: - :tags: - :tickets: - - PickleType and Interval types (on db not supporting it natively) are now - slightly faster. - - .. change:: - :tags: - :tickets: - - Added Float and Time types to Firebird (FBFloat and FBTime). Fixed - BLOB SUB_TYPE for TEXT and Binary types. - - .. change:: - :tags: - :tickets: - - Changed the API for the in\_ operator. in_() now accepts a single argument - that is a sequence of values or a selectable. The old API of passing in - values as varargs still works but is deprecated. - -.. changelog:: - :version: 0.4.0beta6 - :released: Thu Sep 27 2007 - - .. change:: - :tags: - :tickets: - - The Session identity map is now *weak referencing* by default, use - weak_identity_map=False to use a regular dict. The weak dict we are using - is customized to detect instances which are "dirty" and maintain a - temporary strong reference to those instances until changes are flushed. - - .. change:: - :tags: - :tickets: 758 - - Mapper compilation has been reorganized such that most compilation occurs - upon mapper construction. This allows us to have fewer calls to - mapper.compile() and also to allow class-based properties to force a - compilation (i.e. User.addresses == 7 will compile all mappers; this is). The only caveat here is that an inheriting mapper now - looks for its inherited mapper upon construction; so mappers within - inheritance relationships need to be constructed in inheritance order - (which should be the normal case anyway). - - .. change:: - :tags: - :tickets: - - added "FETCH" to the keywords detected by Postgres to indicate a - result-row holding statement (i.e. in addition to "SELECT"). - - .. change:: - :tags: - :tickets: - - Added full list of SQLite reserved keywords so that they get escaped - properly. - - .. change:: - :tags: - :tickets: - - Tightened up the relationship between the Query's generation of "eager - load" aliases, and Query.instances() which actually grabs the eagerly - loaded rows. If the aliases were not specifically generated for that - statement by EagerLoader, the EagerLoader will not take effect when the - rows are fetched. This prevents columns from being grabbed accidentally - as being part of an eager load when they were not meant for such, which - can happen with textual SQL as well as some inheritance situations. It's - particularly important since the "anonymous aliasing" of columns uses - simple integer counts now to generate labels. - - .. change:: - :tags: - :tickets: - - Removed "parameters" argument from clauseelement.compile(), replaced with - "column_keys". The parameters sent to execute() only interact with the - insert/update statement compilation process in terms of the column names - present but not the values for those columns. Produces more consistent - execute/executemany behavior, simplifies things a bit internally. - - .. change:: - :tags: - :tickets: 560 - - Added 'comparator' keyword argument to PickleType. By default, "mutable" - PickleType does a "deep compare" of objects using their dumps() - representation. But this doesn't work for dictionaries. Pickled objects - which provide an adequate __eq__() implementation can be set up with - "PickleType(comparator=operator.eq)" - - .. change:: - :tags: - :tickets: - - Added session.is_modified(obj) method; performs the same "history" - comparison operation as occurs within a flush operation; setting - include_collections=False gives the same result as is used when the flush - determines whether or not to issue an UPDATE for the instance's row. - - .. change:: - :tags: - :tickets: 584, 761 - - Added "schema" argument to Sequence; use this with Postgres /Oracle when - the sequence is located in an alternate schema. Implements part of, should fix. - - .. change:: - :tags: - :tickets: - - Fixed reflection of the empty string for mysql enums. - - .. change:: - :tags: - :tickets: 794 - - Changed MySQL dialect to use the older LIMIT , syntax - instead of LIMIT OFFSET for folks using 3.23. - - .. change:: - :tags: - :tickets: - - Added 'passive_deletes="all"' flag to relation(), disables all nulling-out - of foreign key attributes during a flush where the parent object is - deleted. - - .. change:: - :tags: - :tickets: - - Column defaults and onupdates, executing inline, will add parenthesis for - subqueries and other parenthesis-requiring expressions - - .. change:: - :tags: - :tickets: 793 - - The behavior of String/Unicode types regarding that they auto-convert to - TEXT/CLOB when no length is present now occurs *only* for an exact type of - String or Unicode with no arguments. If you use VARCHAR or NCHAR - (subclasses of String/Unicode) with no length, they will be interpreted by - the dialect as VARCHAR/NCHAR; no "magic" conversion happens there. This - is less surprising behavior and in particular this helps Oracle keep - string-based bind parameters as VARCHARs and not CLOBs. - - .. change:: - :tags: - :tickets: 771 - - Fixes to ShardedSession to work with deferred columns. - - .. change:: - :tags: - :tickets: - - User-defined shard_chooser() function must accept "clause=None" argument; - this is the ClauseElement passed to session.execute(statement) and can be - used to determine correct shard id (since execute() doesn't take an - instance.) - - .. change:: - :tags: - :tickets: 764 - - Adjusted operator precedence of NOT to match '==' and others, so that - ~(x y) produces NOT (x y), which is better compatible - with older MySQL versions.. This doesn't apply to "~(x==y)" - as it does in 0.3 since ~(x==y) compiles to "x != y", but still applies - to operators like BETWEEN. - - .. change:: - :tags: - :tickets: 757, 768, 779, 728 - - Other tickets:,,. - -.. changelog:: - :version: 0.4.0beta5 - :released: - - .. change:: - :tags: - :tickets: 754 - - Connection pool fixes; the better performance of beta4 remains but fixes - "connection overflow" and other bugs which were present (like). - - .. change:: - :tags: - :tickets: 769 - - Fixed bugs in determining proper sync clauses from custom inherit - conditions. - - .. change:: - :tags: - :tickets: 763 - - Extended 'engine_from_config' coercion for QueuePool size / overflow. - - .. change:: - :tags: - :tickets: 748 - - mysql views can be reflected again. - - .. change:: - :tags: - :tickets: - - AssociationProxy can now take custom getters and setters. - - .. change:: - :tags: - :tickets: - - Fixed malfunctioning BETWEEN in orm queries. - - .. change:: - :tags: - :tickets: 762 - - Fixed OrderedProperties pickling - - .. change:: - :tags: - :tickets: - - SQL-expression defaults and sequences now execute "inline" for all - non-primary key columns during an INSERT or UPDATE, and for all columns - during an executemany()-style call. inline=True flag on any insert/update - statement also forces the same behavior with a single execute(). - result.postfetch_cols() is a collection of columns for which the previous - single insert or update statement contained a SQL-side default expression. - - .. change:: - :tags: - :tickets: 759 - - Fixed PG executemany() behavior. - - .. change:: - :tags: - :tickets: - - postgres reflects tables with autoincrement=False for primary key columns - which have no defaults. - - .. change:: - :tags: - :tickets: - - postgres no longer wraps executemany() with individual execute() calls, - instead favoring performance. "rowcount"/"concurrency" checks with - deleted items (which use executemany) are disabled with PG since psycopg2 - does not report proper rowcount for executemany(). - - .. change:: - :tags: tickets, fixed - :tickets: 742 - - - - .. change:: - :tags: tickets, fixed - :tickets: 748 - - - - .. change:: - :tags: tickets, fixed - :tickets: 760 - - - - .. change:: - :tags: tickets, fixed - :tickets: 762 - - - - .. change:: - :tags: tickets, fixed - :tickets: 763 - - - -.. changelog:: - :version: 0.4.0beta4 - :released: Wed Aug 22 2007 - - .. change:: - :tags: - :tickets: - - Tidied up what ends up in your namespace when you 'from sqlalchemy import \*': - - .. change:: - :tags: - :tickets: - - 'table' and 'column' are no longer imported. They remain available by - direct reference (as in 'sql.table' and 'sql.column') or a glob import - from the sql package. It was too easy to accidentally use a - sql.expressions.table instead of schema.Table when just starting out - with SQLAlchemy, likewise column. - - .. change:: - :tags: - :tickets: - - Internal-ish classes like ClauseElement, FromClause, NullTypeEngine, - etc., are also no longer imported into your namespace - - .. change:: - :tags: - :tickets: - - The 'Smallinteger' compatibility name (small i!) is no longer imported, - but remains in schema.py for now. SmallInteger (big I!) is still - imported. - - .. change:: - :tags: - :tickets: - - The connection pool uses a "threadlocal" strategy internally to return - the same connection already bound to a thread, for "contextual" connections; - these are the connections used when you do a "connectionless" execution - like insert().execute(). This is like a "partial" version of the - "threadlocal" engine strategy but without the thread-local transaction part - of it. We're hoping it reduces connection pool overhead as well as - database usage. However, if it proves to impact stability in a negative way, - we'll roll it right back. - - .. change:: - :tags: - :tickets: - - Fix to bind param processing such that "False" values (like blank strings) - still get processed/encoded. - - .. change:: - :tags: - :tickets: 752 - - Fix to select() "generative" behavior, such that calling column(), - select_from(), correlate(), and with_prefix() does not modify the - original select object - - .. change:: - :tags: - :tickets: - - Added a "legacy" adapter to types, such that user-defined TypeEngine - and TypeDecorator classes which define convert_bind_param() and/or - convert_result_value() will continue to function. Also supports - calling the super() version of those methods. - - .. change:: - :tags: - :tickets: - - Added session.prune(), trims away instances cached in a session that - are no longer referenced elsewhere. (A utility for strong-ref - identity maps). - - .. change:: - :tags: - :tickets: - - Added close() method to Transaction. Closes out a transaction using - rollback if it's the outermost transaction, otherwise just ends - without affecting the outer transaction. - - .. change:: - :tags: - :tickets: - - Transactional and non-transactional Session integrates better with - bound connection; a close() will ensure that connection - transactional state is the same as that which existed on it before - being bound to the Session. - - .. change:: - :tags: - :tickets: 735 - - Modified SQL operator functions to be module-level operators, - allowing SQL expressions to be pickleable. - - .. change:: - :tags: - :tickets: - - Small adjustment to mapper class.__init__ to allow for Py2.6 - object.__init__() behavior. - - .. change:: - :tags: - :tickets: - - Fixed 'prefix' argument for select() - - .. change:: - :tags: - :tickets: - - Connection.begin() no longer accepts nested=True, this logic is now - all in begin_nested(). - - .. change:: - :tags: - :tickets: - - Fixes to new "dynamic" relation loader involving cascades - - .. change:: - :tags: tickets, fixed - :tickets: 735 - - - - .. change:: - :tags: tickets, fixed - :tickets: 752 - - - -.. changelog:: - :version: 0.4.0beta3 - :released: Thu Aug 16 2007 - - .. change:: - :tags: - :tickets: - - SQL types optimization: - - .. change:: - :tags: - :tickets: - - New performance tests show a combined mass-insert/mass-select test as - having 68% fewer function calls than the same test run against 0.3. - - .. change:: - :tags: - :tickets: - - General performance improvement of result set iteration is around 10-20%. - - .. change:: - :tags: - :tickets: - - In types.AbstractType, convert_bind_param() and convert_result_value() - have migrated to callable-returning bind_processor() and - result_processor() methods. If no callable is returned, no pre/post - processing function is called. - - .. change:: - :tags: - :tickets: - - Hooks added throughout base/sql/defaults to optimize the calling of bind - param/result processors so that method call overhead is minimized. - - .. change:: - :tags: - :tickets: - - Support added for executemany() scenarios such that unneeded "last row id" - logic doesn't kick in, parameters aren't excessively traversed. - - .. change:: - :tags: - :tickets: - - Added 'inherit_foreign_keys' arg to mapper(). - - .. change:: - :tags: - :tickets: - - Added support for string date passthrough in sqlite. - - .. change:: - :tags: tickets, fixed - :tickets: 738 - - - - .. change:: - :tags: tickets, fixed - :tickets: 739 - - - - .. change:: - :tags: tickets, fixed - :tickets: 743 - - - - .. change:: - :tags: tickets, fixed - :tickets: 744 - - - -.. changelog:: - :version: 0.4.0beta2 - :released: Tue Aug 14 2007 - - .. change:: - :tags: oracle, improvements. - :tickets: - - Auto-commit after LOAD DATA INFILE for mysql. - - .. change:: - :tags: oracle, improvements. - :tickets: - - A rudimental SessionExtension class has been added, allowing user-defined - functionality to take place at flush(), commit(), and rollback() boundaries. - - .. change:: - :tags: oracle, improvements. - :tickets: - - Added engine_from_config() function for helping to create_engine() from an - .ini style config. - - .. change:: - :tags: oracle, improvements. - :tickets: - - base_mapper() becomes a plain attribute. - - .. change:: - :tags: oracle, improvements. - :tickets: - - session.execute() and scalar() can search for a Table with which to bind from - using the given ClauseElement. - - .. change:: - :tags: oracle, improvements. - :tickets: - - Session automatically extrapolates tables from mappers with binds, also uses - base_mapper so that inheritance hierarchies bind automatically. - - .. change:: - :tags: oracle, improvements. - :tickets: - - Moved ClauseVisitor traversal back to inlined non-recursive. - - .. change:: - :tags: tickets, fixed - :tickets: 730 - - - - .. change:: - :tags: tickets, fixed - :tickets: 732 - - - - .. change:: - :tags: tickets, fixed - :tickets: 733 - - - - .. change:: - :tags: tickets, fixed - :tickets: 734 - - - -.. changelog:: - :version: 0.4.0beta1 - :released: Sun Aug 12 2007 - - .. change:: - :tags: orm - :tickets: - - Speed! Along with recent speedups to ResultProxy, total number of function - calls significantly reduced for large loads. - - .. change:: - :tags: orm - :tickets: - - test/perf/masseagerload.py reports 0.4 as having the fewest number of - function calls across all SA versions (0.1, 0.2, and 0.3). - - .. change:: - :tags: orm - :tickets: 213 - - New collection_class api and implementation. Collections are - now instrumented via decorations rather than proxying. You can now have - collections that manage their own membership, and your class instance will - be directly exposed on the relation property. The changes are transparent - for most users. - - .. change:: - :tags: orm - :tickets: - - InstrumentedList (as it was) is removed, and relation properties no - longer have 'clear()', '.data', or any other added methods beyond those - provided by the collection type. You are free, of course, to add them to - a custom class. - - .. change:: - :tags: orm - :tickets: - - __setitem__-like assignments now fire remove events for the existing - value, if any. - - .. change:: - :tags: orm - :tickets: - - dict-likes used as collection classes no longer need to change __iter__ - semantics- itervalues() is used by default instead. This is a backwards - incompatible change. - - .. change:: - :tags: orm - :tickets: - - Subclassing dict for a mapped collection is no longer needed in most - cases. orm.collections provides canned implementations that key objects - by a specified column or a custom function of your choice. - - .. change:: - :tags: orm - :tickets: - - Collection assignment now requires a compatible type- assigning None to - clear a collection or assigning a list to a dict collection will now - raise an argument error. - - .. change:: - :tags: orm - :tickets: - - AttributeExtension moved to interfaces, and .delete is now .remove The - event method signature has also been swapped around. - - .. change:: - :tags: orm - :tickets: - - Major overhaul for Query: - - .. change:: - :tags: orm - :tickets: - - All selectXXX methods are deprecated. Generative methods are now the - standard way to do things, i.e. filter(), filter_by(), all(), one(), - etc. Deprecated methods are docstring'ed with their new replacements. - - .. change:: - :tags: orm - :tickets: 643 - - Class-level properties are now usable as query elements... no more - '.c.'! "Class.c.propname" is now superseded by "Class.propname". All - clause operators are supported, as well as higher level operators such - as Class.prop== for scalar attributes, - Class.prop.contains() and Class.prop.any() for collection-based attributes (all are also - negatable). Table-based column expressions as well as columns mounted - on mapped classes via 'c' are of course still fully available and can be - freely mixed with the new attributes. - - .. change:: - :tags: orm - :tickets: - - Removed ancient query.select_by_attributename() capability. - - .. change:: - :tags: orm - :tickets: - - The aliasing logic used by eager loading has been generalized, so that - it also adds full automatic aliasing support to Query. It's no longer - necessary to create an explicit Alias to join to the same tables - multiple times; *even for self-referential relationships*. - - - join() and outerjoin() take arguments "aliased=True". Yhis causes - their joins to be built on aliased tables; subsequent calls to - filter() and filter_by() will translate all table expressions (yes, - real expressions using the original mapped Table) to be that of the - Alias for the duration of that join() (i.e. until reset_joinpoint() or - another join() is called). - - - join() and outerjoin() take arguments "id=". When used - with "aliased=True", the id can be referenced by add_entity(cls, - id=) so that you can select the joined instances even if - they're from an alias. - - - join() and outerjoin() now work with self-referential relationships! - Using "aliased=True", you can join as many levels deep as desired, - i.e. query.join(['children', 'children'], aliased=True); filter - criterion will be against the rightmost joined table - - .. change:: - :tags: orm - :tickets: 660 - - Added query.populate_existing(), marks the query to reload all - attributes and collections of all instances touched in the query, - including eagerly-loaded entities. - - .. change:: - :tags: orm - :tickets: - - Added eagerload_all(), allows eagerload_all('x.y.z') to specify eager - loading of all properties in the given path. - - .. change:: - :tags: orm - :tickets: - - Major overhaul for Session: - - .. change:: - :tags: orm - :tickets: - - New function which "configures" a session called "sessionmaker()". Send - various keyword arguments to this function once, returns a new class - which creates a Session against that stereotype. - - .. change:: - :tags: orm - :tickets: - - SessionTransaction removed from "public" API. You now can call begin()/ - commit()/rollback() on the Session itself. - - .. change:: - :tags: orm - :tickets: - - Session also supports SAVEPOINT transactions; call begin_nested(). - - .. change:: - :tags: orm - :tickets: - - Session supports two-phase commit behavior when vertically or - horizontally partitioning (i.e., using more than one engine). Use - twophase=True. - - .. change:: - :tags: orm - :tickets: - - Session flag "transactional=True" produces a session which always places - itself into a transaction when first used. Upon commit(), rollback() or - close(), the transaction ends; but begins again on the next usage. - - .. change:: - :tags: orm - :tickets: - - Session supports "autoflush=True". This issues a flush() before each - query. Use in conjunction with transactional, and you can just - save()/update() and then query, the new objects will be there. Use - commit() at the end (or flush() if non-transactional) to flush remaining - changes. - - .. change:: - :tags: orm - :tickets: - - New scoped_session() function replaces SessionContext and assignmapper. - Builds onto "sessionmaker()" concept to produce a class whose Session() - construction returns the thread-local session. Or, call all Session - methods as class methods, i.e. Session.save(foo); Session.commit(). - just like the old "objectstore" days. - - .. change:: - :tags: orm - :tickets: - - Added new "binds" argument to Session to support configuration of - multiple binds with sessionmaker() function. - - .. change:: - :tags: orm - :tickets: - - A rudimental SessionExtension class has been added, allowing - user-defined functionality to take place at flush(), commit(), and - rollback() boundaries. - - .. change:: - :tags: orm - :tickets: - - Query-based relation()s available with dynamic_loader(). This is a - *writable* collection (supporting append() and remove()) which is also a - live Query object when accessed for reads. Ideal for dealing with very - large collections where only partial loading is desired. - - .. change:: - :tags: orm - :tickets: - - flush()-embedded inline INSERT/UPDATE expressions. Assign any SQL - expression, like "sometable.c.column + 1", to an instance's attribute. - Upon flush(), the mapper detects the expression and embeds it directly in - the INSERT or UPDATE statement; the attribute gets deferred on the - instance so it loads the new value the next time you access it. - - .. change:: - :tags: orm - :tickets: 618 - - A rudimental sharding (horizontal scaling) system is introduced. This - system uses a modified Session which can distribute read and write - operations among multiple databases, based on user-defined functions - defining the "sharding strategy". Instances and their dependents can be - distributed and queried among multiple databases based on attribute - values, round-robin approaches or any other user-defined - system. - - .. change:: - :tags: orm - :tickets: 659 - - Eager loading has been enhanced to allow even more joins in more places. - It now functions at any arbitrary depth along self-referential and - cyclical structures. When loading cyclical structures, specify - "join_depth" on relation() indicating how many times you'd like the table - to join to itself; each level gets a distinct table alias. The alias - names themselves are generated at compile time using a simple counting - scheme now and are a lot easier on the eyes, as well as of course - completely deterministic. - - .. change:: - :tags: orm - :tickets: 211 - - Added composite column properties. This allows you to create a type which - is represented by more than one column, when using the ORM. Objects of - the new type are fully functional in query expressions, comparisons, - query.get() clauses, etc. and act as though they are regular single-column - scalars... except they're not! Use the function composite(cls, \*columns) - inside of the mapper's "properties" dict, and instances of cls will be - created/mapped to a single attribute, comprised of the values corresponding - to \*columns. - - .. change:: - :tags: orm - :tickets: - - Improved support for custom column_property() attributes which feature - correlated subqueries, works better with eager loading now. - - .. change:: - :tags: orm - :tickets: 611 - - Primary key "collapse" behavior; the mapper will analyze all columns in - its given selectable for primary key "equivalence", that is, columns which - are equivalent via foreign key relationship or via an explicit - inherit_condition. primarily for joined-table inheritance scenarios where - different named PK columns in inheriting tables should "collapse" into a - single-valued (or fewer-valued) primary key. Fixes things like. - - .. change:: - :tags: orm - :tickets: - - Joined-table inheritance will now generate the primary key columns of all - inherited classes against the root table of the join only. This implies - that each row in the root table is distinct to a single instance. If for - some rare reason this is not desirable, explicit primary_key settings on - individual mappers will override it. - - .. change:: - :tags: orm - :tickets: - - When "polymorphic" flags are used with joined-table or single-table - inheritance, all identity keys are generated against the root class of the - inheritance hierarchy; this allows query.get() to work polymorphically - using the same caching semantics as a non-polymorphic get. Note that this - currently does not work with concrete inheritance. - - .. change:: - :tags: orm - :tickets: - - Secondary inheritance loading: polymorphic mappers can be constructed - *without* a select_table argument. inheriting mappers whose tables were - not represented in the initial load will issue a second SQL query - immediately, once per instance (i.e. not very efficient for large lists), - in order to load the remaining columns. - - .. change:: - :tags: orm - :tickets: - - Secondary inheritance loading can also move its second query into a - column-level "deferred" load, via the "polymorphic_fetch" argument, which - can be set to 'select' or 'deferred' - - .. change:: - :tags: orm - :tickets: 696 - - It's now possible to map only a subset of available selectable columns - onto mapper properties, using include_columns/exclude_columns.. - - .. change:: - :tags: orm - :tickets: - - Added undefer_group() MapperOption, sets a set of "deferred" columns - joined by a "group" to load as "undeferred". - - .. change:: - :tags: orm - :tickets: - - Rewrite of the "deterministic alias name" logic to be part of the SQL - layer, produces much simpler alias and label names more in the style of - Hibernate - - .. change:: - :tags: sql - :tickets: - - Speed! Clause compilation as well as the mechanics of SQL constructs have - been streamlined and simplified to a significant degree, for a 20-30% - improvement of the statement construction/compilation overhead of 0.3. - - .. change:: - :tags: sql - :tickets: - - All "type" keyword arguments, such as those to bindparam(), column(), - Column(), and func.(), renamed to "type\_". Those objects still - name their "type" attribute as "type". - - .. change:: - :tags: sql - :tickets: - - case_sensitive=(True|False) setting removed from schema items, since - checking this state added a lot of method call overhead and there was no - decent reason to ever set it to False. Table and column names which are - all lower case will be treated as case-insensitive (yes we adjust for - Oracle's UPPERCASE style too). - - .. change:: - :tags: transactions - :tickets: - - Added context manager (with statement) support for transactions. - - .. change:: - :tags: transactions - :tickets: - - Added support for two phase commit, works with mysql and postgres so far. - - .. change:: - :tags: transactions - :tickets: - - Added a subtransaction implementation that uses savepoints. - - .. change:: - :tags: transactions - :tickets: - - Added support for savepoints. - - .. change:: - :tags: metadata - :tickets: - - Tables can be reflected from the database en-masse without declaring - them in advance. MetaData(engine, reflect=True) will load all tables - present in the database, or use metadata.reflect() for finer control. - - .. change:: - :tags: metadata - :tickets: - - DynamicMetaData has been renamed to ThreadLocalMetaData - - .. change:: - :tags: metadata - :tickets: - - The ThreadLocalMetaData constructor now takes no arguments. - - .. change:: - :tags: metadata - :tickets: - - BoundMetaData has been removed- regular MetaData is equivalent - - .. change:: - :tags: metadata - :tickets: 646 - - Numeric and Float types now have an "asdecimal" flag; defaults to True for - Numeric, False for Float. When True, values are returned as - decimal.Decimal objects; when False, values are returned as float(). The - defaults of True/False are already the behavior for PG and MySQL's DBAPI - modules. - - .. change:: - :tags: metadata - :tickets: 475 - - New SQL operator implementation which removes all hardcoded operators from - expression structures and moves them into compilation; allows greater - flexibility of operator compilation; for example, "+" compiles to "||" - when used in a string context, or "concat(a,b)" on MySQL; whereas in a - numeric context it compiles to "+". Fixes. - - .. change:: - :tags: metadata - :tickets: - - "Anonymous" alias and label names are now generated at SQL compilation - time in a completely deterministic fashion... no more random hex IDs - - .. change:: - :tags: metadata - :tickets: - - Significant architectural overhaul to SQL elements (ClauseElement). All - elements share a common "mutability" framework which allows a consistent - approach to in-place modifications of elements as well as generative - behavior. Improves stability of the ORM which makes heavy usage of - mutations to SQL expressions. - - .. change:: - :tags: metadata - :tickets: - - select() and union()'s now have "generative" behavior. Methods like - order_by() and group_by() return a *new* instance - the original instance - is left unchanged. Non-generative methods remain as well. - - .. change:: - :tags: metadata - :tickets: 569, 52 - - The internals of select/union vastly simplified- all decision making - regarding "is subquery" and "correlation" pushed to SQL generation phase. - select() elements are now *never* mutated by their enclosing containers or - by any dialect's compilation process - - .. change:: - :tags: metadata - :tickets: - - select(scalar=True) argument is deprecated; use select(..).as_scalar(). - The resulting object obeys the full "column" interface and plays better - within expressions. - - .. change:: - :tags: metadata - :tickets: 504 - - Added select().with_prefix('foo') allowing any set of keywords to be - placed before the columns clause of the SELECT - - .. change:: - :tags: metadata - :tickets: 686 - - Added array slice support to row[] - - .. change:: - :tags: metadata - :tickets: - - Result sets make a better attempt at matching the DBAPI types present in - cursor.description to the TypeEngine objects defined by the dialect, which - are then used for result-processing. Note this only takes effect for - textual SQL; constructed SQL statements always have an explicit type map. - - .. change:: - :tags: metadata - :tickets: - - Result sets from CRUD operations close their underlying cursor immediately - and will also autoclose the connection if defined for the operation; this - allows more efficient usage of connections for successive CRUD operations - with less chance of "dangling connections". - - .. change:: - :tags: metadata - :tickets: 559 - - Column defaults and onupdate Python functions (i.e. passed to - ColumnDefault) may take zero or one arguments; the one argument is the - ExecutionContext, from which you can call "context.parameters[someparam]" - to access the other bind parameter values affixed to the statement. The connection used for the execution is available as well - so that you can pre-execute statements. - - .. change:: - :tags: metadata - :tickets: - - Added "explicit" create/drop/execute support for sequences (i.e. you can - pass a "connectable" to each of those methods on Sequence). - - .. change:: - :tags: metadata - :tickets: - - Better quoting of identifiers when manipulating schemas. - - .. change:: - :tags: metadata - :tickets: - - Standardized the behavior for table reflection where types can't be - located; NullType is substituted instead, warning is raised. - - .. change:: - :tags: metadata - :tickets: 606 - - ColumnCollection (i.e. the 'c' attribute on tables) follows dictionary - semantics for "__contains__" - - .. change:: - :tags: engines - :tickets: - - Speed! The mechanics of result processing and bind parameter processing - have been overhauled, streamlined and optimized to issue as little method - calls as possible. Bench tests for mass INSERT and mass rowset iteration - both show 0.4 to be over twice as fast as 0.3, using 68% fewer function - calls. - - .. change:: - :tags: engines - :tickets: - - You can now hook into the pool lifecycle and run SQL statements or other - logic at new each DBAPI connection, pool check-out and check-in. - - .. change:: - :tags: engines - :tickets: - - Connections gain a .properties collection, with contents scoped to the - lifetime of the underlying DBAPI connection - - .. change:: - :tags: engines - :tickets: - - Removed auto_close_cursors and disallow_open_cursors arguments from Pool; - reduces overhead as cursors are normally closed by ResultProxy and - Connection. - - .. change:: - :tags: extensions - :tickets: - - proxyengine is temporarily removed, pending an actually working - replacement. - - .. change:: - :tags: extensions - :tickets: - - SelectResults has been replaced by Query. SelectResults / - SelectResultsExt still exist but just return a slightly modified Query - object for backwards-compatibility. join_to() method from SelectResults - isn't present anymore, need to use join(). - - .. change:: - :tags: mysql - :tickets: - - Table and column names loaded via reflection are now Unicode. - - .. change:: - :tags: mysql - :tickets: - - All standard column types are now supported, including SET. - - .. change:: - :tags: mysql - :tickets: - - Table reflection can now be performed in as little as one round-trip. - - .. change:: - :tags: mysql - :tickets: - - ANSI and ANSI_QUOTES sql modes are now supported. - - .. change:: - :tags: mysql - :tickets: - - Indexes are now reflected. - - .. change:: - :tags: postgres - :tickets: - - Added PGArray datatype for using postgres array datatypes. - - .. change:: - :tags: oracle - :tickets: 507 - - Very rudimental support for OUT parameters added; use sql.outparam(name, - type) to set up an OUT parameter, just like bindparam(); after execution, - values are available via result.out_parameters dictionary. diff --git a/doc/build/changelog/changelog_05.rst b/doc/build/changelog/changelog_05.rst deleted file mode 100644 index e998cb4443..0000000000 --- a/doc/build/changelog/changelog_05.rst +++ /dev/null @@ -1,3775 +0,0 @@ - -============= -0.5 Changelog -============= - - -.. changelog:: - :version: 0.5.9 - :released: - - .. change:: - :tags: sql - :tickets: 1661 - - Fixed erroneous self_group() call in expression package. - -.. changelog:: - :version: 0.5.8 - :released: Sat Jan 16 2010 - - .. change:: - :tags: sql - :tickets: - - The copy() method on Column now supports uninitialized, - unnamed Column objects. This allows easy creation of - declarative helpers which place common columns on multiple - subclasses. - - .. change:: - :tags: sql - :tickets: - - Default generators like Sequence() translate correctly - across a copy() operation. - - .. change:: - :tags: sql - :tickets: - - Sequence() and other DefaultGenerator objects are accepted - as the value for the "default" and "onupdate" keyword - arguments of Column, in addition to being accepted - positionally. - - .. change:: - :tags: sql - :tickets: 1568, 1617 - - Fixed a column arithmetic bug that affected column - correspondence for cloned selectables which contain - free-standing column expressions. This bug is - generally only noticeable when exercising newer - ORM behavior only available in 0.6 via, - but is more correct at the SQL expression level - as well. - - .. change:: - :tags: postgresql - :tickets: 1647 - - The extract() function, which was slightly improved in - 0.5.7, needed a lot more work to generate the correct - typecast (the typecasts appear to be necessary in PG's - EXTRACT quite a lot of the time). The typecast is - now generated using a rule dictionary based - on PG's documentation for date/time/interval arithmetic. - It also accepts text() constructs again, which was broken - in 0.5.7. - - .. change:: - :tags: firebird - :tickets: 1646 - - Recognize more errors as disconnections. - -.. changelog:: - :version: 0.5.7 - :released: Sat Dec 26 2009 - - .. change:: - :tags: orm - :tickets: 1543 - - contains_eager() now works with the automatically - generated subquery that results when you say - "query(Parent).join(Parent.somejoinedsubclass)", i.e. - when Parent joins to a joined-table-inheritance subclass. - Previously contains_eager() would erroneously add the - subclass table to the query separately producing a - cartesian product. An example is in the ticket - description. - - .. change:: - :tags: orm - :tickets: 1553 - - query.options() now only propagate to loaded objects - for potential further sub-loads only for options where - such behavior is relevant, keeping - various unserializable options like those generated - by contains_eager() out of individual instance states. - - .. change:: - :tags: orm - :tickets: 1054 - - Session.execute() now locates table- and - mapper-specific binds based on a passed - in expression which is an insert()/update()/delete() - construct. - - .. change:: - :tags: orm - :tickets: - - Session.merge() now properly overwrites a many-to-one or - uselist=False attribute to None if the attribute - is also None in the given object to be merged. - - .. change:: - :tags: orm - :tickets: 1618 - - Fixed a needless select which would occur when merging - transient objects that contained a null primary key - identifier. - - .. change:: - :tags: orm - :tickets: 1585 - - Mutable collection passed to the "extension" attribute - of relation(), column_property() etc. will not be mutated - or shared among multiple instrumentation calls, preventing - duplicate extensions, such as backref populators, - from being inserted into the list. - - .. change:: - :tags: orm - :tickets: 1504 - - Fixed the call to get_committed_value() on CompositeProperty. - - .. change:: - :tags: orm - :tickets: 1602 - - Fixed bug where Query would crash if a join() with no clear - "left" side were called when a non-mapped column entity - appeared in the columns list. - - .. change:: - :tags: orm - :tickets: 1616, 1480 - - Fixed bug whereby composite columns wouldn't load properly - when configured on a joined-table subclass, introduced in - version 0.5.6 as a result of the fix for. thx to Scott Torborg. - - .. change:: - :tags: orm - :tickets: 1556 - - The "use get" behavior of many-to-one relations, i.e. that a - lazy load will fallback to the possibly cached query.get() - value, now works across join conditions where the two compared - types are not exactly the same class, but share the same - "affinity" - i.e. Integer and SmallInteger. Also allows - combinations of reflected and non-reflected types to work - with 0.5 style type reflection, such as PGText/Text (note 0.6 - reflects types as their generic versions). - - .. change:: - :tags: orm - :tickets: 1436 - - Fixed bug in query.update() when passing Cls.attribute - as keys in the value dict and using synchronize_session='expire' - ('fetch' in 0.6). - - .. change:: - :tags: sql - :tickets: 1603 - - Fixed bug in two-phase transaction whereby commit() method - didn't set the full state which allows subsequent close() - call to succeed. - - .. change:: - :tags: sql - :tickets: - - Fixed the "numeric" paramstyle, which apparently is the - default paramstyle used by Informixdb. - - .. change:: - :tags: sql - :tickets: 1574 - - Repeat expressions in the columns clause of a select - are deduped based on the identity of each clause element, - not the actual string. This allows positional - elements to render correctly even if they all render - identically, such as "qmark" style bind parameters. - - .. change:: - :tags: sql - :tickets: 1632 - - The cursor associated with connection pool connections - (i.e. _CursorFairy) now proxies `__iter__()` to the - underlying cursor correctly. - - .. change:: - :tags: sql - :tickets: 1556 - - types now support an "affinity comparison" operation, i.e. - that an Integer/SmallInteger are "compatible", or - a Text/String, PickleType/Binary, etc. Part of. - - .. change:: - :tags: sql - :tickets: 1641 - - Fixed bug preventing alias() of an alias() from being - cloned or adapted (occurs frequently in ORM operations). - - .. change:: - :tags: sqlite - :tickets: 1439 - - sqlite dialect properly generates CREATE INDEX for a table - that is in an alternate schema. - - .. change:: - :tags: postgresql - :tickets: 1085 - - Added support for reflecting the DOUBLE PRECISION type, - via a new postgres.PGDoublePrecision object. - This is postgresql.DOUBLE_PRECISION in 0.6. - - .. change:: - :tags: postgresql - :tickets: 460 - - Added support for reflecting the INTERVAL YEAR TO MONTH - and INTERVAL DAY TO SECOND syntaxes of the INTERVAL - type. - - .. change:: - :tags: postgresql - :tickets: 1576 - - Corrected the "has_sequence" query to take current schema, - or explicit sequence-stated schema, into account. - - .. change:: - :tags: postgresql - :tickets: 1611 - - Fixed the behavior of extract() to apply operator - precedence rules to the "::" operator when applying - the "timestamp" cast - ensures proper parenthesization. - - .. change:: - :tags: mssql - :tickets: 1561 - - Changed the name of TrustedConnection to - Trusted_Connection when constructing pyodbc connect - arguments - - .. change:: - :tags: oracle - :tickets: 1637 - - The "table_names" dialect function, used by MetaData - .reflect(), omits "index overflow tables", a system - table generated by Oracle when "index only tables" - with overflow are used. These tables aren't accessible - via SQL and can't be reflected. - - .. change:: - :tags: ext - :tickets: 1570, 1523 - - A column can be added to a joined-table declarative - superclass after the class has been constructed - (i.e. via class-level attribute assignment), and - the column will be propagated down to - subclasses. This is the reverse - situation as that of, fixed in 0.5.6. - - .. change:: - :tags: ext - :tickets: 1491 - - Fixed a slight inaccuracy in the sharding example. - Comparing equivalence of columns in the ORM is best - accomplished using col1.shares_lineage(col2). - - .. change:: - :tags: ext - :tickets: 1606 - - Removed unused `load()` method from ShardedQuery. - -.. changelog:: - :version: 0.5.6 - :released: Sat Sep 12 2009 - - .. change:: - :tags: orm - :tickets: 1300 - - Fixed bug whereby inheritance discriminator part of a - composite primary key would fail on updates. - Continuation of. - - .. change:: - :tags: orm - :tickets: 1507 - - Fixed bug which disallowed one side of a many-to-many - bidirectional reference to declare itself as "viewonly" - - .. change:: - :tags: orm - :tickets: 1526 - - Added an assertion that prevents a @validates function - or other AttributeExtension from loading an unloaded - collection such that internal state may be corrupted. - - .. change:: - :tags: orm - :tickets: 1519 - - Fixed bug which prevented two entities from mutually - replacing each other's primary key values within a single - flush() for some orderings of operations. - - .. change:: - :tags: orm - :tickets: 1485 - - Fixed an obscure issue whereby a joined-table subclass - with a self-referential eager load on the base class - would populate the related object's "subclass" table with - data from the "subclass" table of the parent. - - .. change:: - :tags: orm - :tickets: 1477 - - relations() now have greater ability to be "overridden", - meaning a subclass that explicitly specifies a relation() - overriding that of the parent class will be honored - during a flush. This is currently to support - many-to-many relations from concrete inheritance setups. - Outside of that use case, YMMV. - - .. change:: - :tags: orm - :tickets: 1483 - - Squeezed a few more unnecessary "lazy loads" out of - relation(). When a collection is mutated, many-to-one - backrefs on the other side will not fire off to load - the "old" value, unless "single_parent=True" is set. - A direct assignment of a many-to-one still loads - the "old" value in order to update backref collections - on that value, which may be present in the session - already, thus maintaining the 0.5 behavioral contract. - - .. change:: - :tags: orm - :tickets: 1480 - - Fixed bug whereby a load/refresh of joined table - inheritance attributes which were based on - column_property() or similar would fail to evaluate. - - .. change:: - :tags: orm - :tickets: 1488 - - Improved support for MapperProperty objects overriding - that of an inherited mapper for non-concrete - inheritance setups - attribute extensions won't randomly - collide with each other. - - .. change:: - :tags: orm - :tickets: 1487 - - UPDATE and DELETE do not support ORDER BY, LIMIT, OFFSET, - etc. in standard SQL. Query.update() and Query.delete() - now raise an exception if any of limit(), offset(), - order_by(), group_by(), or distinct() have been - called. - - .. change:: - :tags: orm - :tickets: - - Added AttributeExtension to sqlalchemy.orm.__all__ - - .. change:: - :tags: orm - :tickets: 1476 - - Improved error message when query() is called with - a non-SQL /entity expression. - - .. change:: - :tags: orm - :tickets: 1440 - - Using False or 0 as a polymorphic discriminator now - works on the base class as well as a subclass. - - .. change:: - :tags: orm - :tickets: 1424 - - Added enable_assertions(False) to Query which disables - the usual assertions for expected state - used - by Query subclasses to engineer custom state.. See - https://www.sqlalchemy.org/trac/wiki/UsageRecipes/PreFilteredQuery - for an example. - - .. change:: - :tags: orm - :tickets: 1501 - - Fixed recursion issue which occurred if a mapped object's - `__len__()` or `__nonzero__()` method resulted in state - changes. - - .. change:: - :tags: orm - :tickets: 1506 - - Fixed incorrect exception raise in - Weak/StrongIdentityMap.add() - - .. change:: - :tags: orm - :tickets: 1522 - - Fixed the error message for "could not find a FROM clause" - in query.join() which would fail to issue correctly - if the query was against a pure SQL construct. - - .. change:: - :tags: orm - :tickets: 1486 - - Fixed a somewhat hypothetical issue which would result - in the wrong primary key being calculated for a mapper - using the old polymorphic_union function - but this - is old stuff. - - .. change:: - :tags: sql - :tickets: 1373 - - Fixed column.copy() to copy defaults and onupdates. - - .. change:: - :tags: sql - :tickets: - - Fixed a bug in extract() introduced in 0.5.4 whereby - the string "field" argument was getting treated as a - ClauseElement, causing various errors within more - complex SQL transformations. - - .. change:: - :tags: sql - :tickets: 1420 - - Unary expressions such as DISTINCT propagate their - type handling to result sets, allowing conversions like - unicode and such to take place. - - .. change:: - :tags: sql - :tickets: 1482 - - Fixed bug in Table and Column whereby passing empty - dict for "info" argument would raise an exception. - - .. change:: - :tags: oracle - :tickets: 1309 - - Backported 0.6 fix for Oracle alias names not getting - truncated. - - .. change:: - :tags: ext - :tickets: 1446 - - The collection proxies produced by associationproxy are now - pickleable. A user-defined proxy_factory however - is still not pickleable unless it defines __getstate__ - and __setstate__. - - .. change:: - :tags: ext - :tickets: 1468 - - Declarative will raise an informative exception if - __table_args__ is passed as a tuple with no dict argument. - Improved documentation. - - .. change:: - :tags: ext - :tickets: 1527 - - Table objects declared in the MetaData can now be used - in string expressions sent to primaryjoin/secondaryjoin/ - secondary - the name is pulled from the MetaData of the - declarative base. - - .. change:: - :tags: ext - :tickets: 1523 - - A column can be added to a joined-table subclass after - the class has been constructed (i.e. via class-level - attribute assignment). The column is added to the underlying - Table as always, but now the mapper will rebuild its - "join" to include the new column, instead of raising - an error about "no such column, use column_property() - instead". - - .. change:: - :tags: test - :tickets: - - Added examples into the test suite so they get exercised - regularly and cleaned up a couple deprecation warnings. - -.. changelog:: - :version: 0.5.5 - :released: Mon Jul 13 2009 - - .. change:: - :tags: general - :tickets: 970 - - unit tests have been migrated from unittest to nose. See - README.unittests for information on how to run the tests. - - .. change:: - :tags: orm - :tickets: - - The "foreign_keys" argument of relation() will now propagate - automatically to the backref in the same way that primaryjoin - and secondaryjoin do. For the extremely rare use case where - the backref of a relation() has intentionally different - "foreign_keys" configured, both sides now need to be - configured explicitly (if they do in fact require this setting, - see the next note...). - - .. change:: - :tags: orm - :tickets: - - ...the only known (and really, really rare) use case where a - different foreign_keys setting was used on the - forwards/backwards side, a composite foreign key that - partially points to its own columns, has been enhanced such - that the fk->itself aspect of the relation won't be used to - determine relation direction. - - .. change:: - :tags: orm - :tickets: - - Session.mapper is now *deprecated*. - - Call session.add() if you'd like a free-standing object to be - part of your session. Otherwise, a DIY version of - Session.mapper is now documented at - https://www.sqlalchemy.org/trac/wiki/UsageRecipes/SessionAwareMapper - The method will remain deprecated throughout 0.6. - - .. change:: - :tags: orm - :tickets: 1431 - - Fixed Query being able to join() from individual columns of a - joined-table subclass entity, i.e. query(SubClass.foo, - SubClass.bar).join(). In most cases, an error - "Could not find a FROM clause to join from" would be - raised. In a few others, the result would be returned in terms - of the base class rather than the subclass - so applications - which relied on this erroneous result need to be - adjusted. - - .. change:: - :tags: orm - :tickets: 1461 - - Fixed a bug involving contains_eager(), which would apply - itself to a secondary (i.e. lazy) load in a particular rare - case, producing cartesian products. improved the targeting of - query.options() on secondary loads overall. - - .. change:: - :tags: orm - :tickets: - - Fixed bug introduced in 0.5.4 whereby Composite types fail - when default-holding columns are flushed. - - .. change:: - :tags: orm - :tickets: 1426 - - Fixed another 0.5.4 bug whereby mutable attributes - (i.e. PickleType) wouldn't be deserialized correctly when the - whole object was serialized. - - .. change:: - :tags: orm - :tickets: - - Fixed bug whereby session.is_modified() would raise an - exception if any synonyms were in use. - - .. change:: - :tags: orm - :tickets: - - Fixed potential memory leak whereby previously pickled objects - placed back in a session would not be fully garbage collected - unless the Session were explicitly closed out. - - .. change:: - :tags: orm - :tickets: - - Fixed bug whereby list-based attributes, like pickletype and - PGArray, failed to be merged() properly. - - .. change:: - :tags: orm - :tickets: - - Repaired non-working attributes.set_committed_value function. - - .. change:: - :tags: orm - :tickets: - - Trimmed the pickle format for InstanceState which should - further reduce the memory footprint of pickled instances. The - format should be backwards compatible with that of 0.5.4 and - previous. - - .. change:: - :tags: orm - :tickets: 1463 - - sqlalchemy.orm.join and sqlalchemy.orm.outerjoin are now - added to __all__ in sqlalchemy.orm.*. - - .. change:: - :tags: orm - :tickets: 1458 - - Fixed bug where Query exception raise would fail when - a too-short composite primary key value were passed to - get(). - - .. change:: - :tags: sql - :tickets: - - Removed an obscure feature of execute() (including connection, - engine, Session) whereby a bindparam() construct can be sent - as a key to the params dictionary. This usage is undocumented - and is at the core of an issue whereby the bindparam() object - created implicitly by a text() construct may have the same - hash value as a string placed in the params dictionary and may - result in an inappropriate match when computing the final bind - parameters. Internal checks for this condition would add - significant latency to the critical task of parameter - rendering, so the behavior is removed. This is a backwards - incompatible change for any application that may have been - using this feature, however the feature has never been - documented. - - .. change:: - :tags: engine/pool - :tickets: - - Implemented recreate() for StaticPool. - -.. changelog:: - :version: 0.5.4p2 - :released: Tue May 26 2009 - - .. change:: - :tags: sql - :tickets: - - Repaired the printing of SQL exceptions which are not - based on parameters or are not executemany() style. - - .. change:: - :tags: postgresql - :tickets: - - Deprecated the hardcoded TIMESTAMP function, which when - used as func.TIMESTAMP(value) would render "TIMESTAMP value". - This breaks on some platforms as PostgreSQL doesn't allow - bind parameters to be used in this context. The hard-coded - uppercase is also inappropriate and there's lots of other - PG casts that we'd need to support. So instead, use - text constructs i.e. select(["timestamp '12/05/09'"]). - -.. changelog:: - :version: 0.5.4p1 - :released: Mon May 18 2009 - - .. change:: - :tags: orm - :tickets: - - Fixed an attribute error introduced in 0.5.4 which would - occur when merge() was used with an incomplete object. - -.. changelog:: - :version: 0.5.4 - :released: Sun May 17 2009 - - .. change:: - :tags: orm - :tickets: 1398 - - Significant performance enhancements regarding Sessions/flush() - in conjunction with large mapper graphs, large numbers of - objects: - - - Removed all* O(N) scanning behavior from the flush() process, - i.e. operations that were scanning the full session, - including an extremely expensive one that was erroneously - assuming primary key values were changing when this - was not the case. - - * one edge case remains which may invoke a full scan, - if an existing primary key attribute is modified - to a new value. - - - The Session's "weak referencing" behavior is now *full* - - no strong references whatsoever are made to a mapped object - or related items/collections in its __dict__. Backrefs and - other cycles in objects no longer affect the Session's ability - to lose all references to unmodified objects. Objects with - pending changes still are maintained strongly until flush. - - - The implementation also improves performance by moving - the "resurrection" process of garbage collected items - to only be relevant for mappings that map "mutable" - attributes (i.e. PickleType, composite attrs). This removes - overhead from the gc process and simplifies internal - behavior. - - If a "mutable" attribute change is the sole change on an object - which is then dereferenced, the mapper will not have access to - other attribute state when the UPDATE is issued. This may present - itself differently to some MapperExtensions. - - The change also affects the internal attribute API, but not - the AttributeExtension interface nor any of the publicly - documented attribute functions. - - - The unit of work no longer generates a graph of "dependency" - processors for the full graph of mappers during flush(), instead - creating such processors only for those mappers which represent - objects with pending changes. This saves a tremendous number - of method calls in the context of a large interconnected - graph of mappers. - - - Cached a wasteful "table sort" operation that previously - occurred multiple times per flush, also removing significant - method call count from flush(). - - - Other redundant behaviors have been simplified in - mapper._save_obj(). - - .. change:: - :tags: orm - :tickets: - - Modified query_cls on DynamicAttributeImpl to accept a full - mixin version of the AppenderQuery, which allows subclassing - the AppenderMixin. - - .. change:: - :tags: orm - :tickets: 1300 - - The "polymorphic discriminator" column may be part of a - primary key, and it will be populated with the correct - discriminator value. - - .. change:: - :tags: orm - :tickets: - - Fixed the evaluator not being able to evaluate IS NULL clauses. - - .. change:: - :tags: orm - :tickets: 1352 - - Fixed the "set collection" function on "dynamic" relations to - initiate events correctly. Previously a collection could only - be assigned to a pending parent instance, otherwise modified - events would not be fired correctly. Set collection is now - compatible with merge(), fixes. - - .. change:: - :tags: orm - :tickets: - - Allowed pickling of PropertyOption objects constructed with - instrumented descriptors; previously, pickle errors would occur - when pickling an object which was loaded with a descriptor-based - option, such as query.options(eagerload(MyClass.foo)). - - .. change:: - :tags: orm - :tickets: 1357 - - Lazy loader will not use get() if the "lazy load" SQL clause - matches the clause used by get(), but contains some parameters - hardcoded. Previously the lazy strategy would fail with the - get(). Ideally get() would be used with the hardcoded - parameters but this would require further development. - - .. change:: - :tags: orm - :tickets: 1391 - - MapperOptions and other state associated with query.options() - is no longer bundled within callables associated with each - lazy/deferred-loading attribute during a load. - The options are now associated with the instance's - state object just once when it's populated. This removes - the need in most cases for per-instance/attribute loader - objects, improving load speed and memory overhead for - individual instances. - - .. change:: - :tags: orm - :tickets: 1360 - - Fixed another location where autoflush was interfering - with session.merge(). autoflush is disabled completely - for the duration of merge() now. - - .. change:: - :tags: orm - :tickets: 1406 - - Fixed bug which prevented "mutable primary key" dependency - logic from functioning properly on a one-to-one - relation(). - - .. change:: - :tags: orm - :tickets: - - Fixed bug in relation(), introduced in 0.5.3, - whereby a self referential relation - from a base class to a joined-table subclass would - not configure correctly. - - .. change:: - :tags: orm - :tickets: - - Fixed obscure mapper compilation issue when inheriting - mappers are used which would result in un-initialized - attributes. - - .. change:: - :tags: orm - :tickets: - - Fixed documentation for session weak_identity_map - - the default value is True, indicating a weak - referencing map in use. - - .. change:: - :tags: orm - :tickets: 1376 - - Fixed a unit of work issue whereby the foreign - key attribute on an item contained within a collection - owned by an object being deleted would not be set to - None if the relation() was self-referential. - - .. change:: - :tags: orm - :tickets: 1378 - - Fixed Query.update() and Query.delete() failures with eagerloaded - relations. - - .. change:: - :tags: orm - :tickets: - - It is now an error to specify both columns of a binary primaryjoin - condition in the foreign_keys or remote_side collection. Whereas - previously it was just nonsensical, but would succeed in a - non-deterministic way. - - .. change:: - :tags: ticket: 594, 1341, schema - :tickets: - - Added a quote_schema() method to the IdentifierPreparer class - so that dialects can override how schemas get handled. This - enables the MSSQL dialect to treat schemas as multipart - identifiers, such as 'database.owner'. - - .. change:: - :tags: sql - :tickets: - - Back-ported the "compiler" extension from SQLA 0.6. This - is a standardized interface which allows the creation of custom - ClauseElement subclasses and compilers. In particular it's - handy as an alternative to text() when you'd like to - build a construct that has database-specific compilations. - See the extension docs for details. - - .. change:: - :tags: sql - :tickets: 1413 - - Exception messages are truncated when the list of bound - parameters is larger than 10, preventing enormous - multi-page exceptions from filling up screens and logfiles - for large executemany() statements. - - .. change:: - :tags: sql - :tickets: - - ``sqlalchemy.extract()`` is now dialect sensitive and can - extract components of timestamps idiomatically across the - supported databases, including SQLite. - - .. change:: - :tags: sql - :tickets: 1353 - - Fixed __repr__() and other _get_colspec() methods on - ForeignKey constructed from __clause_element__() style - construct (i.e. declarative columns). - - .. change:: - :tags: mysql - :tickets: 1405 - - Reflecting a FOREIGN KEY construct will take into account - a dotted schema.tablename combination, if the foreign key - references a table in a remote schema. - - .. change:: - :tags: mssql - :tickets: - - Modified how savepoint logic works to prevent it from - stepping on non-savepoint oriented routines. Savepoint - support is still very experimental. - - .. change:: - :tags: mssql - :tickets: 1310 - - Added in reserved words for MSSQL that covers version 2008 - and all prior versions. - - .. change:: - :tags: mssql - :tickets: 1343 - - Corrected problem with information schema not working with a - binary collation based database. Cleaned up information schema - since it is only used by mssql now. - - .. change:: - :tags: sqlite - :tickets: 1402 - - Corrected the SLBoolean type so that it properly treats only 1 - as True. - - .. change:: - :tags: sqlite - :tickets: 1273 - - Corrected the float type so that it correctly maps to a - SLFloat type when being reflected. - - .. change:: - :tags: extensions - :tickets: 1379 - - Fixed adding of deferred or other column properties to a - declarative class. - -.. changelog:: - :version: 0.5.3 - :released: Tue Mar 24 2009 - - .. change:: - :tags: orm - :tickets: 1315 - - The "objects" argument to session.flush() is deprecated. - State which represents the linkage between a parent and - child object does not support "flushed" status on - one side of the link and not the other, so supporting - this operation leads to misleading results. - - .. change:: - :tags: orm - :tickets: - - Query now implements __clause_element__() which produces - its selectable, which means a Query instance can be accepted - in many SQL expressions, including col.in_(query), - union(query1, query2), select([foo]).select_from(query), - etc. - - .. change:: - :tags: orm - :tickets: 1337 - - Query.join() can now construct multiple FROM clauses, if - needed. Such as, query(A, B).join(A.x).join(B.y) - might say SELECT A.*, B.* FROM A JOIN X, B JOIN Y. - Eager loading can also tack its joins onto those - multiple FROM clauses. - - .. change:: - :tags: orm - :tickets: 1347 - - Fixed bug in dynamic_loader() where append/remove events - after construction time were not being propagated to the - UOW to pick up on flush(). - - .. change:: - :tags: orm - :tickets: - - Fixed bug where column_prefix wasn't being checked before - not mapping an attribute that already had class-level - name present. - - .. change:: - :tags: orm - :tickets: 1315 - - a session.expire() on a particular collection attribute - will clear any pending backref additions as well, so that - the next access correctly returns only what was present - in the database. Presents some degree of a workaround for, although we are considering removing the - flush([objects]) feature altogether. - - .. change:: - :tags: orm - :tickets: - - Session.scalar() now converts raw SQL strings to text() - the same way Session.execute() does and accepts same - alternative \**kw args. - - .. change:: - :tags: orm - :tickets: - - improvements to the "determine direction" logic of - relation() such that the direction of tricky situations - like mapper(A.join(B)) -> relation-> mapper(B) can be - determined. - - .. change:: - :tags: orm - :tickets: 1306 - - When flushing partial sets of objects using session.flush([somelist]), - pending objects which remain pending after the operation won't - inadvertently be added as persistent. - - .. change:: - :tags: orm - :tickets: 1314 - - Added "post_configure_attribute" method to InstrumentationManager, - so that the "listen_for_events.py" example works again. - - .. change:: - :tags: orm - :tickets: - - a forward and complementing backwards reference which are both - of the same direction, i.e. ONETOMANY or MANYTOONE, - is now detected, and an error message is raised. - Saves crazy CircularDependencyErrors later on. - - .. change:: - :tags: orm - :tickets: - - Fixed bugs in Query regarding simultaneous selection of - multiple joined-table inheritance entities with common base - classes: - - - previously the adaption applied to "B" on - "A JOIN B" would be erroneously partially applied - to "A". - - - comparisons on relations (i.e. A.related==someb) - were not getting adapted when they should. - - - Other filterings, like - query(A).join(A.bs).filter(B.foo=='bar'), were erroneously - adapting "B.foo" as though it were an "A". - - .. change:: - :tags: orm - :tickets: 1325 - - Fixed adaptation of EXISTS clauses via any(), has(), etc. - in conjunction with an aliased object on the left and - of_type() on the right. - - .. change:: - :tags: orm - :tickets: - - Added an attribute helper method ``set_committed_value`` in - sqlalchemy.orm.attributes. Given an object, attribute name, - and value, will set the value on the object as part of its - "committed" state, i.e. state that is understood to have - been loaded from the database. Helps with the creation of - homegrown collection loaders and such. - - .. change:: - :tags: orm - :tickets: - - Query won't fail with weakref error when a non-mapper/class - instrumented descriptor is passed, raises - "Invalid column expression". - - .. change:: - :tags: orm - :tickets: - - Query.group_by() properly takes into account aliasing applied - to the FROM clause, such as with select_from(), using - with_polymorphic(), or using from_self(). - - .. change:: - :tags: sql - :tickets: - - An alias() of a select() will convert to a "scalar subquery" - when used in an unambiguously scalar context, i.e. it's used - in a comparison operation. This applies to - the ORM when using query.subquery() as well. - - .. change:: - :tags: sql - :tickets: 1302 - - Fixed missing _label attribute on Function object, others - when used in a select() with use_labels (such as when used - in an ORM column_property()). - - .. change:: - :tags: sql - :tickets: 1309 - - anonymous alias names now truncate down to the max length - allowed by the dialect. More significant on DBs like - Oracle with very small character limits. - - .. change:: - :tags: sql - :tickets: - - the __selectable__() interface has been replaced entirely - by __clause_element__(). - - .. change:: - :tags: sql - :tickets: 1299 - - The per-dialect cache used by TypeEngine to cache - dialect-specific types is now a WeakKeyDictionary. - This to prevent dialect objects from - being referenced forever for an application that - creates an arbitrarily large number of engines - or dialects. There is a small performance penalty - which will be resolved in 0.6. - - .. change:: - :tags: sqlite - :tickets: - - Fixed SQLite reflection methods so that non-present - cursor.description, which triggers an auto-cursor - close, will be detected so that no results doesn't - fail on recent versions of pysqlite which raise - an error when fetchone() called with no rows present. - - .. change:: - :tags: postgresql - :tickets: - - Index reflection won't fail when an index with - multiple expressions is encountered. - - .. change:: - :tags: postgresql - :tickets: 1327 - - Added PGUuid and PGBit types to - sqlalchemy.databases.postgres. - - .. change:: - :tags: postgresql - :tickets: 1327 - - Refection of unknown PG types won't crash when those - types are specified within a domain. - - .. change:: - :tags: mssql - :tickets: - - Preliminary support for pymssql 1.0.1 - - .. change:: - :tags: mssql - :tickets: - - Corrected issue on mssql where max_identifier_length was - not being respected. - - .. change:: - :tags: extensions - :tickets: - - Fixed a recursive pickling issue in serializer, triggered - by an EXISTS or other embedded FROM construct. - - .. change:: - :tags: extensions - :tickets: - - Declarative locates the "inherits" class using a search - through __bases__, to skip over mixins that are local - to subclasses. - - .. change:: - :tags: extensions - :tickets: - - Declarative figures out joined-table inheritance primary join - condition even if "inherits" mapper argument is given - explicitly. - - .. change:: - :tags: extensions - :tickets: - - Declarative will properly interpret the "foreign_keys" argument - on a backref() if it's a string. - - .. change:: - :tags: extensions - :tickets: - - Declarative will accept a table-bound column as a property - when used in conjunction with __table__, if the column is already - present in __table__. The column will be remapped to the given - key the same way as when added to the mapper() properties dict. - -.. changelog:: - :version: 0.5.2 - :released: Sat Jan 24 2009 - - .. change:: - :tags: orm - :tickets: - - Further refined 0.5.1's warning about delete-orphan cascade - placed on a many-to-many relation. First, the bad news: - the warning will apply to both many-to-many as well as - many-to-one relations. This is necessary since in both - cases, SQLA does not scan the full set of potential parents - when determining "orphan" status - for a persistent object - it only detects an in-python de-association event to establish - the object as an "orphan". Next, the good news: to support - one-to-one via a foreign key or association table, or to - support one-to-many via an association table, a new flag - single_parent=True may be set which indicates objects - linked to the relation are only meant to have a single parent. - The relation will raise an error if multiple parent-association - events occur within Python. - - .. change:: - :tags: orm - :tickets: 1292 - - Adjusted the attribute instrumentation change from 0.5.1 to - fully establish instrumentation for subclasses where the mapper - was created after the superclass had already been fully - instrumented. - - .. change:: - :tags: orm - :tickets: - - Fixed bug in delete-orphan cascade whereby two one-to-one - relations from two different parent classes to the same target - class would prematurely expunge the instance. - - .. change:: - :tags: orm - :tickets: - - Fixed an eager loading bug whereby self-referential eager - loading would prevent other eager loads, self referential or not, - from joining to the parent JOIN properly. Thanks to Alex K - for creating a great test case. - - .. change:: - :tags: orm - :tickets: - - session.expire() and related methods will not expire() unloaded - deferred attributes. This prevents them from being needlessly - loaded when the instance is refreshed. - - .. change:: - :tags: orm - :tickets: 1293 - - query.join()/outerjoin() will now properly join an aliased() - construct to the existing left side, even if query.from_self() - or query.select_from(someselectable) has been called. - - .. change:: - :tags: sql - :tickets: 1284 - - Further fixes to the "percent signs and spaces in column/table - names" functionality. - - .. change:: - :tags: mssql - :tickets: 1291 - - Restored convert_unicode handling. Results were being passed - on through without conversion. - - .. change:: - :tags: mssql - :tickets: 1282 - - Really fixing the decimal handling this time.. - - .. change:: - :tags: Ticket:1289, mssql - :tickets: - - Modified table reflection code to use only kwargs when - constructing tables. - -.. changelog:: - :version: 0.5.1 - :released: Sat Jan 17 2009 - - .. change:: - :tags: orm - :tickets: - - Removed an internal join cache which could potentially leak - memory when issuing query.join() repeatedly to ad-hoc - selectables. - - .. change:: - :tags: orm - :tickets: - - The "clear()", "save()", "update()", "save_or_update()" - Session methods have been deprecated, replaced by - "expunge_all()" and "add()". "expunge_all()" has also - been added to ScopedSession. - - .. change:: - :tags: orm - :tickets: - - Modernized the "no mapped table" exception and added a more - explicit __table__/__tablename__ exception to declarative. - - .. change:: - :tags: orm - :tickets: 1237 - - Concrete inheriting mappers now instrument attributes which - are inherited from the superclass, but are not defined for - the concrete mapper itself, with an InstrumentedAttribute that - issues a descriptive error when accessed. - - .. change:: - :tags: orm - :tickets: 1237, 781 - - Added a new `relation()` keyword `back_populates`. This - allows configuration of backreferences using explicit - relations. This is required when creating - bidirectional relations between a hierarchy of concrete - mappers and another class. - - .. change:: - :tags: orm - :tickets: 1237 - - Test coverage added for `relation()` objects specified on - concrete mappers. - - .. change:: - :tags: orm - :tickets: 1276 - - Query.from_self() as well as query.subquery() both disable - the rendering of eager joins inside the subquery produced. - The "disable all eager joins" feature is available publicly - via a new query.enable_eagerloads() generative. - - .. change:: - :tags: orm - :tickets: - - Added a rudimental series of set operations to Query that - receive Query objects as arguments, including union(), - union_all(), intersect(), except_(), intersect_all(), - except_all(). See the API documentation for - Query.union() for examples. - - .. change:: - :tags: orm - :tickets: - - Fixed bug that prevented Query.join() and eagerloads from - attaching to a query that selected from a union or aliased union. - - .. change:: - :tags: orm - :tickets: 1237 - - A short documentation example added for bidirectional - relations specified on concrete mappers. - - .. change:: - :tags: orm - :tickets: 1269 - - Mappers now instrument class attributes upon construction - with the final InstrumentedAttribute object which remains - persistent. The `_CompileOnAttr`/`__getattribute__()` - methodology has been removed. The net effect is that - Column-based mapped class attributes can now be used fully - at the class level without invoking a mapper compilation - operation, greatly simplifying typical usage patterns - within declarative. - - .. change:: - :tags: orm - :tickets: - - ColumnProperty (and front-end helpers such as ``deferred``) no - longer ignores unknown \**keyword arguments. - - .. change:: - :tags: orm - :tickets: - - Fixed a bug with the unitofwork's "row switch" mechanism, - i.e. the conversion of INSERT/DELETE into an UPDATE, when - combined with joined-table inheritance and an object - which contained no defined values for the child table where - an UPDATE with no SET clause would be rendered. - - .. change:: - :tags: orm - :tickets: 1281 - - Using delete-orphan on a many-to-many relation is deprecated. - This produces misleading or erroneous results since SQLA does - not retrieve the full list of "parents" for m2m. To get delete-orphan - behavior with an m2m table, use an explicit association class - so that the individual association row is treated as a parent. - - .. change:: - :tags: orm - :tickets: 1281 - - delete-orphan cascade always requires delete cascade. Specifying - delete-orphan without delete now raises a deprecation warning. - - .. change:: - :tags: sql - :tickets: 1256 - - Improved the methodology to handling percent signs in column - names from. Added more tests. MySQL and - PostgreSQL dialects still do not issue correct CREATE TABLE - statements for identifiers with percent signs in them. - - .. change:: - :tags: schema - :tickets: 1214 - - Index now accepts column-oriented InstrumentedAttributes - (i.e. column-based mapped class attributes) as column - arguments. - - .. change:: - :tags: schema - :tickets: - - Column with no name (as in declarative) won't raise a - NoneType error when its string output is requested - (such as in a stack trace). - - .. change:: - :tags: schema - :tickets: 1278 - - Fixed bug when overriding a Column with a ForeignKey - on a reflected table, where derived columns (i.e. the - "virtual" columns of a select, etc.) would inadvertently - call upon schema-level cleanup logic intended only - for the original column. - - .. change:: - :tags: declarative - :tickets: - - Can now specify Column objects on subclasses which have no - table of their own (i.e. use single table inheritance). - The columns will be appended to the base table, but only - mapped by the subclass. - - .. change:: - :tags: declarative - :tickets: - - For both joined and single inheriting subclasses, the subclass - will only map those columns which are already mapped on the - superclass and those explicit on the subclass. Other - columns that are present on the `Table` will be excluded - from the mapping by default, which can be disabled - by passing a blank `exclude_properties` collection to the - `__mapper_args__`. This is so that single-inheriting - classes which define their own columns are the only classes - to map those columns. The effect is actually a more organized - mapping than you'd normally get with explicit `mapper()` - calls unless you set up the `exclude_properties` arguments - explicitly. - - .. change:: - :tags: declarative - :tickets: - - It's an error to add new Column objects to a declarative class - that specified an existing table using __table__. - - .. change:: - :tags: mysql - :tickets: - - Added the missing keywords from MySQL 4.1 so they get escaped - properly. - - .. change:: - :tags: mssql - :tickets: 1280 - - Corrected handling of large decimal values with more robust - tests. Removed string manipulation on floats. - - .. change:: - :tags: mssql - :tickets: - - Modified the do_begin handling in mssql to use the Cursor not - the Connection so it is DBAPI compatible. - - .. change:: - :tags: mssql - :tickets: - - Corrected SAVEPOINT support on adodbapi by changing the - handling of savepoint_release, which is unsupported on mssql. - -.. changelog:: - :version: 0.5.0 - :released: Tue Jan 06 2009 - - .. change:: - :tags: general - :tickets: - - Documentation has been converted to Sphinx. In particular, - the generated API documentation has been constructed into a - full blown "API Reference" section which organizes editorial - documentation combined with generated docstrings. Cross - linking between sections and API docs are vastly improved, a - javascript-powered search feature is provided, and a full - index of all classes, functions and members is provided. - - .. change:: - :tags: general - :tickets: - - setup.py now imports setuptools only optionally. If not - present, distutils is used. The new "pip" installer is - recommended over easy_install as it installs in a more - simplified way. - - .. change:: - :tags: general - :tickets: - - added an extremely basic illustration of a PostGIS integration - to the examples folder. - - .. change:: - :tags: orm - :tickets: - - Query.with_polymorphic() now accepts a third argument - "discriminator" which will replace the value of - mapper.polymorphic_on for that query. Mappers themselves no - longer require polymorphic_on to be set, even if the mapper - has a polymorphic_identity. When not set, the mapper will - load non-polymorphically by default. Together, these two - features allow a non-polymorphic concrete inheritance setup to - use polymorphic loading on a per-query basis, since concrete - setups are prone to many issues when used polymorphically in - all cases. - - .. change:: - :tags: orm - :tickets: - - dynamic_loader accepts a query_class= to customize the Query - classes used for both the dynamic collection and the queries - built from it. - - .. change:: - :tags: orm - :tickets: 1079 - - query.order_by() accepts None which will remove any pending - order_by state from the query, as well as cancel out any - mapper/relation configured ordering. This is primarily useful - for overriding the ordering specified on a dynamic_loader(). - - .. change:: - :tags: sql - :tickets: 935 - - RowProxy objects can be used in place of dictionary arguments - sent to connection.execute() and friends. - - .. change:: - :tags: dialect - :tickets: - - Added a new description_encoding attribute on the dialect that - is used for encoding the column name when processing the - metadata. This usually defaults to utf-8. - - .. change:: - :tags: mssql - :tickets: - - Added in a new MSGenericBinary type. This maps to the Binary - type so it can implement the specialized behavior of treating - length specified types as fixed-width Binary types and - non-length types as an unbound variable length Binary type. - - .. change:: - :tags: mssql - :tickets: 1249 - - Added in new types: MSVarBinary and MSImage. - - .. change:: - :tags: mssql - :tickets: - - Added in the MSReal, MSNText, MSSmallDateTime, MSTime, - MSDateTimeOffset, and MSDateTime2 types - - .. change:: - :tags: sqlite - :tickets: 1266 - - Table reflection now stores the actual DefaultClause value for - the column. - - .. change:: - :tags: sqlite - :tickets: - - bugfixes, behavioral changes - - .. change:: - :tags: orm - :tickets: - - Exceptions raised during compile_mappers() are now preserved - to provide "sticky behavior" - if a hasattr() call on a - pre-compiled mapped attribute triggers a failing compile and - suppresses the exception, subsequent compilation is blocked - and the exception will be reiterated on the next compile() - call. This issue occurs frequently when using declarative. - - .. change:: - :tags: orm - :tickets: - - property.of_type() is now recognized on a single-table - inheriting target, when used in the context of - prop.of_type(..).any()/has(), as well as - query.join(prop.of_type(...)). - - .. change:: - :tags: orm - :tickets: - - query.join() raises an error when the target of the join - doesn't match the property-based attribute - while it's - unlikely anyone is doing this, the SQLAlchemy author was - guilty of this particular loosey-goosey behavior. - - .. change:: - :tags: orm - :tickets: 1272 - - Fixed bug when using weak_instance_map=False where modified - events would not be intercepted for a flush(). - - .. change:: - :tags: orm - :tickets: 1268 - - Fixed some deep "column correspondence" issues which could - impact a Query made against a selectable containing multiple - versions of the same table, as well as unions and similar - which contained the same table columns in different column - positions at different levels. - - .. change:: - :tags: orm - :tickets: - - Custom comparator classes used in conjunction with - column_property(), relation() etc. can define new comparison - methods on the Comparator, which will become available via - __getattr__() on the InstrumentedAttribute. In the case of - synonym() or comparable_property(), attributes are resolved - first on the user-defined descriptor, then on the user-defined - comparator. - - .. change:: - :tags: orm - :tickets: 976 - - Added ScopedSession.is_active accessor. - - .. change:: - :tags: orm - :tickets: 1262 - - Can pass mapped attributes and column objects as keys to - query.update({}). - - .. change:: - :tags: orm - :tickets: - - Mapped attributes passed to the values() of an expression - level insert() or update() will use the keys of the mapped - columns, not that of the mapped attribute. - - .. change:: - :tags: orm - :tickets: 1242 - - Corrected problem with Query.delete() and Query.update() not - working properly with bind parameters. - - .. change:: - :tags: orm - :tickets: - - Query.select_from(), from_statement() ensure that the given - argument is a FromClause, or Text/Select/Union, respectively. - - .. change:: - :tags: orm - :tickets: 1253 - - Query() can be passed a "composite" attribute as a column - expression and it will be expanded. Somewhat related to. - - .. change:: - :tags: orm - :tickets: - - Query() is a little more robust when passed various column - expressions such as strings, clauselists, text() constructs - (which may mean it just raises an error more nicely). - - .. change:: - :tags: orm - :tickets: - - first() works as expected with Query.from_statement(). - - .. change:: - :tags: orm - :tickets: - - Fixed bug introduced in 0.5rc4 involving eager loading not - functioning for properties which were added to a mapper - post-compile using add_property() or equivalent. - - .. change:: - :tags: orm - :tickets: - - Fixed bug where many-to-many relation() with viewonly=True - would not correctly reference the link between - secondary->remote. - - .. change:: - :tags: orm - :tickets: 1232 - - Duplicate items in a list-based collection will be maintained - when issuing INSERTs to a "secondary" table in a many-to-many - relation. Assuming the m2m table has a unique or primary key - constraint on it, this will raise the expected constraint - violation instead of silently dropping the duplicate - entries. Note that the old behavior remains for a one-to-many - relation since collection entries in that case don't result in - INSERT statements and SQLA doesn't manually police - collections. - - .. change:: - :tags: orm - :tickets: - - Query.add_column() can accept FromClause objects in the same - manner as session.query() can. - - .. change:: - :tags: orm - :tickets: - - Comparison of many-to-one relation to NULL is properly - converted to IS NOT NULL based on not_(). - - .. change:: - :tags: orm - :tickets: 1087 - - Extra checks added to ensure explicit - primaryjoin/secondaryjoin are ClauseElement instances, to - prevent more confusing errors later on. - - .. change:: - :tags: orm - :tickets: 1236 - - Improved mapper() check for non-class classes. - - .. change:: - :tags: orm - :tickets: 5051 - - comparator_factory argument is now documented and supported by - all MapperProperty types, including column_property(), - relation(), backref(), and synonym(). - - .. change:: - :tags: orm - :tickets: - - Changed the name of PropertyLoader to RelationProperty, to be - consistent with all the other names. PropertyLoader is still - present as a synonym. - - .. change:: - :tags: orm - :tickets: 1099, 1228 - - fixed "double iter()" call causing bus errors in shard API, - removed errant result.close() left over from the 0.4 - version. - - .. change:: - :tags: orm - :tickets: - - made Session.merge cascades not trigger autoflush. Fixes - merged instances getting prematurely inserted with missing - values. - - .. change:: - :tags: orm - :tickets: - - Two fixes to help prevent out-of-band columns from being - rendered in polymorphic_union inheritance scenarios (which - then causes extra tables to be rendered in the FROM clause - causing cartesian products): - - - improvements to "column adaption" for a->b->c inheritance - situations to better locate columns that are related to - one another via multiple levels of indirection, rather - than rendering the non-adapted column. - - - the "polymorphic discriminator" column is only rendered - for the actual mapper being queried against. The column - won't be "pulled in" from a subclass or superclass mapper - since it's not needed. - - .. change:: - :tags: orm - :tickets: 1072 - - Fixed shard_id argument on ShardedSession.execute(). - - .. change:: - :tags: sql - :tickets: 1256 - - Columns can again contain percent signs within their - names. - - .. change:: - :tags: sql - :tickets: - - sqlalchemy.sql.expression.Function is now a public class. It - can be subclassed to provide user-defined SQL functions in an - imperative style, including with pre-established behaviors. - The postgis.py example illustrates one usage of this. - - .. change:: - :tags: sql - :tickets: - - PickleType now favors == comparison by default, if the - incoming object (such as a dict) implements __eq__(). If the - object does not implement __eq__() and mutable=True, a - deprecation warning is raised. - - .. change:: - :tags: sql - :tickets: 1215 - - Fixed the import weirdness in sqlalchemy.sql to not export - __names__. - - .. change:: - :tags: sql - :tickets: 1238 - - Using the same ForeignKey object repeatedly raises an error - instead of silently failing later. - - .. change:: - :tags: sql - :tickets: - - Added NotImplementedError for params() method on - Insert/Update/Delete constructs. These items currently don't - support this functionality, which also would be a little - misleading compared to values(). - - .. change:: - :tags: sql - :tickets: 650 - - Reflected foreign keys will properly locate their referenced - column, even if the column was given a "key" attribute - different from the reflected name. This is achieved via a new - flag on ForeignKey/ForeignKeyConstraint called "link_to_name", - if True means the given name is the referred-to column's name, - not its assigned key. - - .. change:: - :tags: sql - :tickets: 1253 - - select() can accept a ClauseList as a column in the same way - as a Table or other selectable and the interior expressions - will be used as column elements. - - .. change:: - :tags: sql - :tickets: - - the "passive" flag on session.is_modified() is correctly - propagated to the attribute manager. - - .. change:: - :tags: sql - :tickets: - - union() and union_all() will not whack any order_by() that has - been applied to the select()s inside. If you union() a - select() with order_by() (presumably to support LIMIT/OFFSET), - you should also call self_group() on it to apply parenthesis. - - .. change:: - :tags: engine/pool - :tickets: 1246 - - Connection.invalidate() checks for closed status to avoid - attribute errors. - - .. change:: - :tags: engine/pool - :tickets: 1094 - - NullPool supports reconnect on failure behavior. - - .. change:: - :tags: engine/pool - :tickets: 799 - - Added a mutex for the initial pool creation when using - pool.manage(dbapi). This prevents a minor case of "dogpile" - behavior which would otherwise occur upon a heavy load - startup. - - .. change:: - :tags: engine/pool - :tickets: - - _execute_clauseelement() goes back to being a private method. - Subclassing Connection is not needed now that ConnectionProxy - is available. - - .. change:: - :tags: documentation - :tickets: 1149, 1200 - - Tickets. - - .. change:: - :tags: documentation - :tickets: - - Added note about create_session() defaults. - - .. change:: - :tags: documentation - :tickets: - - Added section about metadata.reflect(). - - .. change:: - :tags: documentation - :tickets: - - Updated `TypeDecorator` section. - - .. change:: - :tags: documentation - :tickets: - - Rewrote the "threadlocal" strategy section of the docs due to - recent confusion over this feature. - - .. change:: - :tags: documentation - :tickets: - - Removed badly out of date 'polymorphic_fetch' and - 'select_table' docs from inheritance, reworked the second half - of "joined table inheritance". - - .. change:: - :tags: documentation - :tickets: - - Documented `comparator_factory` kwarg, added new doc section - "Custom Comparators". - - .. change:: - :tags: mssql - :tickets: 1254 - - Refactored the Date/Time types. The ``smalldatetime`` data - type no longer truncates to a date only, and will now be - mapped to the MSSmallDateTime type. - - .. change:: - :tags: mssql - :tickets: - - Corrected an issue with Numerics to accept an int. - - .. change:: - :tags: mssql - :tickets: - - Mapped ``char_length`` to the ``LEN()`` function. - - .. change:: - :tags: mssql - :tickets: - - If an ``INSERT`` includes a subselect the ``INSERT`` is - converted from an ``INSERT INTO VALUES`` construct to a - ``INSERT INTO SELECT`` construct. - - .. change:: - :tags: mssql - :tickets: - - If the column is part of a ``primary_key`` it will be ``NOT - NULL`` since MSSQL doesn't allow ``NULL`` in primary_key - columns. - - .. change:: - :tags: mssql - :tickets: 1249 - - ``MSBinary`` now returns a ``BINARY`` instead of an - ``IMAGE``. This is a backwards incompatible change in that - ``BINARY`` is a fixed length data type whereas ``IMAGE`` is a - variable length data type. - - .. change:: - :tags: mssql - :tickets: 1258 - - ``get_default_schema_name`` is now reflected from the database - based on the user's default schema. This only works with MSSQL - 2005 and later. - - .. change:: - :tags: mssql - :tickets: 1248 - - Added collation support through the use of a new collation - argument. This is supported on the following types: char, - nchar, varchar, nvarchar, text, ntext. - - .. change:: - :tags: mssql - :tickets: - - Changes to the connection string parameters favor DSN as the - default specification for pyodbc. See the mssql.py docstring - for detailed usage instructions. - - .. change:: - :tags: mssql - :tickets: - - Added experimental support of savepoints. It currently does - not work fully with sessions. - - .. change:: - :tags: mssql - :tickets: 1243 - - Support for three levels of column nullability: NULL, NOT - NULL, and the database's configured default. The default - Column configuration (nullable=True) will now generate NULL in - the DDL. Previously no specification was emitted and the - database default would take effect (usually NULL, but not - always). To explicitly request the database default, - configure columns with nullable=None and no specification will - be emitted in DDL. This is backwards incompatible - behavior. - - .. change:: - :tags: postgres - :tickets: 1267 - - "%" signs in text() constructs are automatically escaped to - "%%". Because of the backwards incompatible nature of this - change, a warning is emitted if '%%' is detected in the - string. - - .. change:: - :tags: postgres - :tickets: - - Calling alias.execute() in conjunction with - server_side_cursors won't raise AttributeError. - - .. change:: - :tags: postgres - :tickets: 714 - - Added Index reflection support to PostgreSQL, using a great - patch we long neglected, submitted by Ken - Kuhlman. - - .. change:: - :tags: oracle - :tickets: - - Adjusted the format of create_xid() to repair two-phase - commit. We now have field reports of Oracle two-phase commit - working properly with this change. - - .. change:: - :tags: oracle - :tickets: 1233 - - Added OracleNVarchar type, produces NVARCHAR2, and also - subclasses Unicode so that convert_unicode=True by default. - NVARCHAR2 reflects into this type automatically so these - columns pass unicode on a reflected table with no explicit - convert_unicode=True flags. - - .. change:: - :tags: oracle - :tickets: 1265 - - Fixed bug which was preventing out params of certain types - from being received; thanks a ton to huddlej at wwu.edu ! - - .. change:: - :tags: mysql - :tickets: - - "%" signs in text() constructs are automatically escaped to - "%%". Because of the backwards incompatible nature of this - change, a warning is emitted if '%%' is detected in the - string. - - .. change:: - :tags: mysql - :tickets: 1241 - - Fixed bug in exception raise when FK columns not present - during reflection. - - .. change:: - :tags: mysql - :tickets: - - Fixed bug involving reflection of a remote-schema table with a - foreign key ref to another table in that schema. - - .. change:: - :tags: associationproxy - :tickets: - - The association proxy properties are make themselves available - at the class level, e.g. MyClass.aproxy. Previously this - evaluated to None. - - .. change:: - :tags: declarative - :tickets: - - The full list of arguments accepted as string by backref() - includes 'primaryjoin', 'secondaryjoin', 'secondary', - 'foreign_keys', 'remote_side', 'order_by'. - -.. changelog:: - :version: 0.5.0rc4 - :released: Fri Nov 14 2008 - - .. change:: - :tags: orm - :tickets: - - Query.count() has been enhanced to do the "right thing" in a - wider variety of cases. It can now count multiple-entity - queries, as well as column-based queries. Note that this means - if you say query(A, B).count() without any joining criterion, - it's going to count the cartesian product of A*B. Any query - which is against column-based entities will automatically - issue "SELECT count(1) FROM (SELECT...)" so that the real - rowcount is returned, meaning a query such as - query(func.count(A.name)).count() will return a value of one, - since that query would return one row. - - .. change:: - :tags: orm - :tickets: - - Lots of performance tuning. A rough guesstimate over various - ORM operations places it 10% faster over 0.5.0rc3, 25-30% over - 0.4.8. - - .. change:: - :tags: orm - :tickets: - - bugfixes and behavioral changes - - .. change:: - :tags: general - :tickets: - - global "propigate"->"propagate" change. - - .. change:: - :tags: orm - :tickets: - - Adjustments to the enhanced garbage collection on - InstanceState to better guard against errors due to lost - state. - - .. change:: - :tags: orm - :tickets: 1220 - - Query.get() returns a more informative error message when - executed against multiple entities. - - .. change:: - :tags: orm - :tickets: 1140, 1221 - - Restored NotImplementedError on Cls.relation.in_() - - .. change:: - :tags: orm - :tickets: 1226 - - Fixed PendingDeprecationWarning involving order_by parameter - on relation(). - - .. change:: - :tags: sql - :tickets: - - Removed the 'properties' attribute of the Connection object, - Connection.info should be used. - - .. change:: - :tags: sql - :tickets: - - Restored "active rowcount" fetch before ResultProxy autocloses - the cursor. This was removed in 0.5rc3. - - .. change:: - :tags: sql - :tickets: - - Rearranged the `load_dialect_impl()` method in `TypeDecorator` - such that it will take effect even if the user-defined - `TypeDecorator` uses another `TypeDecorator` as its impl. - - .. change:: - :tags: access - :tickets: - - Added support for Currency type. - - .. change:: - :tags: access - :tickets: 1017 - - Functions were not return their result. - - .. change:: - :tags: access - :tickets: 1017 - - Corrected problem with joins. Access only support LEFT OUTER - or INNER not just JOIN by itself. - - .. change:: - :tags: mssql - :tickets: - - Lots of cleanup and fixes to correct problems with limit and - offset. - - .. change:: - :tags: mssql - :tickets: - - Correct situation where subqueries as part of a binary - expression need to be translated to use the IN and NOT IN - syntax. - - .. change:: - :tags: mssql - :tickets: 1216 - - Fixed E Notation issue that prevented the ability to insert - decimal values less than 1E-6. - - .. change:: - :tags: mssql - :tickets: 1217 - - Corrected problems with reflection when dealing with schemas, - particularly when those schemas are the default - schema. - - .. change:: - :tags: mssql - :tickets: - - Corrected problem with casting a zero length item to a - varchar. It now correctly adjusts the CAST. - - .. change:: - :tags: ext - :tickets: - - Can now use a custom "inherit_condition" in __mapper_args__ - when using declarative. - - .. change:: - :tags: ext - :tickets: - - fixed string-based "remote_side", "order_by" and others not - propagating correctly when used in backref(). - -.. changelog:: - :version: 0.5.0rc3 - :released: Fri Nov 07 2008 - - .. change:: - :tags: orm - :tickets: - - Added two new hooks to SessionExtension: after_bulk_delete() - and after_bulk_update(). after_bulk_delete() is called after - a bulk delete() operation on a query. after_bulk_update() is - called after a bulk update() operation on a query. - - .. change:: - :tags: sql - :tickets: - - SQL compiler optimizations and complexity reduction. The call - count for compiling a typical select() construct is 20% less - versus 0.5.0rc2. - - .. change:: - :tags: sql - :tickets: 1211 - - Dialects can now generate label names of adjustable - length. Pass in the argument "label_length=" to - create_engine() to adjust how many characters max will be - present in dynamically generated column labels, i.e. - "somecolumn AS somelabel". Any value less than 6 will result - in a label of minimal size, consisting of an underscore and a - numeric counter. The compiler uses the value of - dialect.max_identifier_length as a default. - - .. change:: - :tags: ext - :tickets: - - Added a new extension sqlalchemy.ext.serializer. Provides - Serializer/Deserializer "classes" which mirror - Pickle/Unpickle, as well as dumps() and loads(). This - serializer implements an "external object" pickler which keeps - key context-sensitive objects, including engines, sessions, - metadata, Tables/Columns, and mappers, outside of the pickle - stream, and can later restore the pickle using any - engine/metadata/session provider. This is used not for - pickling regular object instances, which are pickleable - without any special logic, but for pickling expression objects - and full Query objects, such that all mapper/engine/session - dependencies can be restored at unpickle time. - - .. change:: - :tags: oracle - :tickets: - - Wrote a docstring for Oracle dialect. Apparently that Ohloh - "few source code comments" label is starting to string :). - - .. change:: - :tags: oracle - :tickets: 536 - - Removed FIRST_ROWS() optimize flag when using LIMIT/OFFSET, - can be re-enabled with optimize_limits=True create_engine() - flag. - - .. change:: - :tags: oracle - :tickets: - - bugfixes and behavioral changes - - .. change:: - :tags: orm - :tickets: - - "not equals" comparisons of simple many-to-one relation to an - instance will not drop into an EXISTS clause and will compare - foreign key columns instead. - - .. change:: - :tags: orm - :tickets: - - Removed not-really-working use cases of comparing a collection - to an iterable. Use contains() to test for collection - membership. - - .. change:: - :tags: orm - :tickets: 1171 - - Improved the behavior of aliased() objects such that they more - accurately adapt the expressions generated, which helps - particularly with self-referential comparisons. - - .. change:: - :tags: orm - :tickets: - - Fixed bug involving primaryjoin/secondaryjoin conditions - constructed from class-bound attributes (as often occurs when - using declarative), which later would be inappropriately - aliased by Query, particularly with the various EXISTS based - comparators. - - .. change:: - :tags: orm - :tickets: - - Fixed bug when using multiple query.join() with an - aliased-bound descriptor which would lose the left alias. - - .. change:: - :tags: orm - :tickets: - - Improved weakref identity map memory management to no longer - require mutexing, resurrects garbage collected instance on a - lazy basis for an InstanceState with pending changes. - - .. change:: - :tags: orm - :tickets: - - InstanceState object now removes circular references to itself - upon disposal to keep it outside of cyclic garbage collection. - - .. change:: - :tags: orm - :tickets: - - relation() won't hide unrelated ForeignKey errors inside of - the "please specify primaryjoin" message when determining join - condition. - - .. change:: - :tags: orm - :tickets: 1218 - - Fixed bug in Query involving order_by() in conjunction with - multiple aliases of the same class (will add tests in) - - .. change:: - :tags: orm - :tickets: - - When using Query.join() with an explicit clause for the ON - clause, the clause will be aliased in terms of the left side - of the join, allowing scenarios like query(Source). - from_self().join((Dest, Source.id==Dest.source_id)) to work - properly. - - .. change:: - :tags: orm - :tickets: - - polymorphic_union() function respects the "key" of each Column - if they differ from the column's name. - - .. change:: - :tags: orm - :tickets: 1183 - - Repaired support for "passive-deletes" on a many-to-one - relation() with "delete" cascade. - - .. change:: - :tags: orm - :tickets: 1213 - - Fixed bug in composite types which prevented a primary-key - composite type from being mutated. - - .. change:: - :tags: orm - :tickets: 1202 - - Added more granularity to internal attribute access, such that - cascade and flush operations will not initialize unloaded - attributes and collections, leaving them intact for a - lazy-load later on. Backref events still initialize attributes - and collections for pending instances. - - .. change:: - :tags: sql - :tickets: 1212 - - Simplified the check for ResultProxy "autoclose without - results" to be based solely on presence of - cursor.description. All the regexp-based guessing about - statements returning rows has been removed. - - .. change:: - :tags: sql - :tickets: 1194 - - Direct execution of a union() construct will properly set up - result-row processing. - - .. change:: - :tags: sql - :tickets: - - The internal notion of an "OID" or "ROWID" column has been - removed. It's basically not used by any dialect, and the - possibility of its usage with psycopg2's cursor.lastrowid is - basically gone now that INSERT..RETURNING is available. - - .. change:: - :tags: sql - :tickets: - - Removed "default_order_by()" method on all FromClause objects. - - .. change:: - :tags: sql - :tickets: - - Repaired the table.tometadata() method so that a passed-in - schema argument is propagated to ForeignKey constructs. - - .. change:: - :tags: sql - :tickets: - - Slightly changed behavior of IN operator for comparing to - empty collections. Now results in inequality comparison - against self. More portable, but breaks with stored procedures - that aren't pure functions. - - .. change:: - :tags: oracle - :tickets: - - Setting the auto_convert_lobs to False on create_engine() will - also instruct the OracleBinary type to return the cx_oracle - LOB object unchanged. - - .. change:: - :tags: mysql - :tickets: - - Fixed foreign key reflection in the edge case where a Table's - explicit schema= is the same as the schema (database) the - connection is attached to. - - .. change:: - :tags: mysql - :tickets: - - No longer expects include_columns in table reflection to be - lower case. - - .. change:: - :tags: ext - :tickets: 1174 - - Fixed bug preventing declarative-bound "column" objects from - being used in column_mapped_collection(). - - .. change:: - :tags: misc - :tickets: 1077 - - util.flatten_iterator() func doesn't interpret strings with - __iter__() methods as iterators, such as in pypy. - -.. changelog:: - :version: 0.5.0rc2 - :released: Sun Oct 12 2008 - - .. change:: - :tags: orm - :tickets: - - Fixed bug involving read/write relation()s that contain - literal or other non-column expressions within their - primaryjoin condition equated to a foreign key column. - - .. change:: - :tags: orm - :tickets: - - "non-batch" mode in mapper(), a feature which allows mapper - extension methods to be called as each instance is - updated/inserted, now honors the insert order of the objects - given. - - .. change:: - :tags: orm - :tickets: - - Fixed RLock-related bug in mapper which could deadlock upon - reentrant mapper compile() calls, something that occurs when - using declarative constructs inside of ForeignKey objects. - - .. change:: - :tags: orm - :tickets: - - ScopedSession.query_property now accepts a query_cls factory, - overriding the session's configured query_cls. - - .. change:: - :tags: orm - :tickets: - - Fixed shared state bug interfering with ScopedSession.mapper's - ability to apply default __init__ implementations on object - subclasses. - - .. change:: - :tags: orm - :tickets: 1177 - - Fixed up slices on Query (i.e. query[x:y]) to work properly - for zero length slices, slices with None on either end. - - .. change:: - :tags: orm - :tickets: - - Added an example illustrating Celko's "nested sets" as a - SQLA mapping. - - .. change:: - :tags: orm - :tickets: - - contains_eager() with an alias argument works even when - the alias is embedded in a SELECT, as when sent to the - Query via query.select_from(). - - .. change:: - :tags: orm - :tickets: 1180 - - contains_eager() usage is now compatible with a Query that - also contains a regular eager load and limit/offset, in that - the columns are added to the Query-generated subquery. - - .. change:: - :tags: orm - :tickets: - - session.execute() will execute a Sequence object passed to - it (regression from 0.4). - - .. change:: - :tags: orm - :tickets: - - Removed the "raiseerror" keyword argument from object_mapper() - and class_mapper(). These functions raise in all cases - if the given class/instance is not mapped. - - .. change:: - :tags: orm - :tickets: - - Fixed session.transaction.commit() on a autocommit=False - session not starting a new transaction. - - .. change:: - :tags: orm - :tickets: - - Some adjustments to Session.identity_map's weak referencing - behavior to reduce asynchronous GC side effects. - - .. change:: - :tags: orm - :tickets: 1182 - - Adjustment to Session's post-flush accounting of newly - "clean" objects to better protect against operating on - objects as they're asynchronously gc'ed. - - .. change:: - :tags: sql - :tickets: 1074 - - column.in_(someselect) can now be used as a columns-clause - expression without the subquery bleeding into the FROM clause - - .. change:: - :tags: sqlite - :tickets: 968 - - Overhauled SQLite date/time bind/result processing to use - regular expressions and format strings, rather than - strptime/strftime, to generically support pre-1900 dates, - dates with microseconds. - - .. change:: - :tags: sqlite - :tickets: - - String's (and Unicode's, UnicodeText's, etc.) convert_unicode - logic disabled in the sqlite dialect, to adjust for pysqlite - 2.5.0's new requirement that only Python unicode objects are - accepted; - https://itsystementwicklung.de/pipermail/list-pysqlite/2008-March/000018.html - - .. change:: - :tags: mysql - :tickets: - - Temporary tables are now reflectable. - - .. change:: - :tags: oracle - :tickets: 1187 - - Oracle will detect string-based statements which contain - comments at the front before a SELECT as SELECT statements. - -.. changelog:: - :version: 0.5.0rc1 - :released: Thu Sep 11 2008 - - .. change:: - :tags: orm - :tickets: - - Query now has delete() and update(values) methods. This allows - to perform bulk deletes/updates with the Query object. - - .. change:: - :tags: orm - :tickets: - - The RowTuple object returned by Query(\*cols) now features - keynames which prefer mapped attribute names over column keys, - column keys over column names, i.e. Query(Class.foo, - Class.bar) will have names "foo" and "bar" even if those are - not the names of the underlying Column objects. Direct Column - objects such as Query(table.c.col) will return the "key" - attribute of the Column. - - .. change:: - :tags: orm - :tickets: - - Added scalar() and value() methods to Query, each return a - single scalar value. scalar() takes no arguments and is - roughly equivalent to first()[0], value() - takes a single column expression and is roughly equivalent to - values(expr).next()[0]. - - .. change:: - :tags: orm - :tickets: - - Improved the determination of the FROM clause when placing SQL - expressions in the query() list of entities. In particular - scalar subqueries should not "leak" their inner FROM objects - out into the enclosing query. - - .. change:: - :tags: orm - :tickets: - - Joins along a relation() from a mapped class to a mapped - subclass, where the mapped subclass is configured with single - table inheritance, will include an IN clause which limits the - subtypes of the joined class to those requested, within the ON - clause of the join. This takes effect for eager load joins as - well as query.join(). Note that in some scenarios the IN - clause will appear in the WHERE clause of the query as well - since this discrimination has multiple trigger points. - - .. change:: - :tags: orm - :tickets: - - AttributeExtension has been refined such that the event - is fired before the mutation actually occurs. Additionally, - the append() and set() methods must now return the given value, - which is used as the value to be used in the mutation operation. - This allows creation of validating AttributeListeners which - raise before the action actually occurs, and which can change - the given value into something else before its used. - - .. change:: - :tags: orm - :tickets: - - column_property(), composite_property(), and relation() now - accept a single or list of AttributeExtensions using the - "extension" keyword argument. - - .. change:: - :tags: orm - :tickets: - - query.order_by().get() silently drops the "ORDER BY" from - the query issued by GET but does not raise an exception. - - .. change:: - :tags: orm - :tickets: - - Added a Validator AttributeExtension, as well as a - @validates decorator which is used in a similar fashion - as @reconstructor, and marks a method as validating - one or more mapped attributes. - - .. change:: - :tags: orm - :tickets: 1140 - - class.someprop.in_() raises NotImplementedError pending the - implementation of "in\_" for relation - - .. change:: - :tags: orm - :tickets: 1127 - - Fixed primary key update for many-to-many collections where - the collection had not been loaded yet - - .. change:: - :tags: orm - :tickets: - - Fixed bug whereby deferred() columns with a group in conjunction - with an otherwise unrelated synonym() would produce - an AttributeError during deferred load. - - .. change:: - :tags: orm - :tickets: 1128 - - The before_flush() hook on SessionExtension takes place before - the list of new/dirty/deleted is calculated for the final - time, allowing routines within before_flush() to further - change the state of the Session before the flush proceeds. - - .. change:: - :tags: orm - :tickets: - - The "extension" argument to Session and others can now - optionally be a list, supporting events sent to multiple - SessionExtension instances. Session places SessionExtensions - in Session.extensions. - - .. change:: - :tags: orm - :tickets: - - Reentrant calls to flush() raise an error. This also serves - as a rudimentary, but not foolproof, check against concurrent - calls to Session.flush(). - - .. change:: - :tags: orm - :tickets: - - Improved the behavior of query.join() when joining to - joined-table inheritance subclasses, using explicit join - criteria (i.e. not on a relation). - - .. change:: - :tags: orm - :tickets: - - @orm.attributes.reconstitute and - MapperExtension.reconstitute have been renamed to - @orm.reconstructor and MapperExtension.reconstruct_instance - - .. change:: - :tags: orm - :tickets: 1129 - - Fixed @reconstructor hook for subclasses which inherit from a - base class. - - .. change:: - :tags: orm - :tickets: 1132 - - The composite() property type now supports a - __set_composite_values__() method on the composite class which - is required if the class represents state using attribute - names other than the column's keynames; default-generated - values now get populated properly upon flush. Also, - composites with attributes set to None compare correctly. - - .. change:: - :tags: orm - :tickets: - - The 3-tuple of iterables returned by attributes.get_history() - may now be a mix of lists and tuples. (Previously members - were always lists.) - - .. change:: - :tags: orm - :tickets: 1151 - - Fixed bug whereby changing a primary key attribute on an - entity where the attribute's previous value had been expired - would produce an error upon flush(). - - .. change:: - :tags: orm - :tickets: - - Fixed custom instrumentation bug whereby get_instance_dict() - was not called for newly constructed instances not loaded - by the ORM. - - .. change:: - :tags: orm - :tickets: 1150 - - Session.delete() adds the given object to the session if - not already present. This was a regression bug from 0.4. - - .. change:: - :tags: orm - :tickets: - - The `echo_uow` flag on `Session` is deprecated, and unit-of-work - logging is now application-level only, not per-session level. - - .. change:: - :tags: orm - :tickets: 1153 - - Removed conflicting `contains()` operator from - `InstrumentedAttribute` which didn't accept `escape` kwaarg. - - .. change:: - :tags: declarative - :tickets: 1161 - - Fixed bug whereby mapper couldn't initialize if a composite - primary key referenced another table that was not defined - yet. - - .. change:: - :tags: declarative - :tickets: - - Fixed exception throw which would occur when string-based - primaryjoin condition was used in conjunction with backref. - - .. change:: - :tags: schema - :tickets: 1033 - - Added "sorted_tables" accessor to MetaData, which returns - Table objects sorted in order of dependency as a list. - This deprecates the MetaData.table_iterator() method. - The "reverse=False" keyword argument has also been - removed from util.sort_tables(); use the Python - 'reversed' function to reverse the results. - - .. change:: - :tags: schema - :tickets: - - The 'length' argument to all Numeric types has been renamed - to 'scale'. 'length' is deprecated and is still accepted - with a warning. - - .. change:: - :tags: schema - :tickets: - - Dropped 0.3-compatibility for user defined types - (convert_result_value, convert_bind_param). - - .. change:: - :tags: sql - :tickets: 1068 - - Temporarily rolled back the "ORDER BY" enhancement from. This feature is on hold pending further - development. - - .. change:: - :tags: sql - :tickets: - - The exists() construct won't "export" its contained list - of elements as FROM clauses, allowing them to be used more - effectively in the columns clause of a SELECT. - - .. change:: - :tags: sql - :tickets: 798 - - and_() and or_() now generate a ColumnElement, allowing - boolean expressions as result columns, i.e. - select([and_(1, 0)]). - - .. change:: - :tags: sql - :tickets: - - Bind params now subclass ColumnElement which allows them to be - selectable by orm.query (they already had most ColumnElement - semantics). - - .. change:: - :tags: sql - :tickets: - - Added select_from() method to exists() construct, which becomes - more and more compatible with a regular select(). - - .. change:: - :tags: sql - :tickets: 1160 - - Added func.min(), func.max(), func.sum() as "generic functions", - which basically allows for their return type to be determined - automatically. Helps with dates on SQLite, decimal types, - others. - - .. change:: - :tags: sql - :tickets: - - added decimal.Decimal as an "auto-detect" type; bind parameters - and generic functions will set their type to Numeric when a - Decimal is used. - - .. change:: - :tags: mysql - :tickets: - - The 'length' argument to MSInteger, MSBigInteger, MSTinyInteger, - MSSmallInteger and MSYear has been renamed to 'display_width'. - - .. change:: - :tags: mysql - :tickets: 1146 - - Added MSMediumInteger type. - - .. change:: - :tags: mysql - :tickets: - - the function func.utc_timestamp() compiles to UTC_TIMESTAMP, without - the parenthesis, which seem to get in the way when using in - conjunction with executemany(). - - .. change:: - :tags: oracle - :tickets: 536 - - limit/offset no longer uses ROW NUMBER OVER to limit rows, - and instead uses subqueries in conjunction with a special - Oracle optimization comment. Allows LIMIT/OFFSET to work - in conjunction with DISTINCT. - - .. change:: - :tags: oracle - :tickets: 1155 - - has_sequence() now takes the current "schema" argument into - account - - .. change:: - :tags: oracle - :tickets: 1121 - - added BFILE to reflected type names - -.. changelog:: - :version: 0.5.0beta3 - :released: Mon Aug 04 2008 - - .. change:: - :tags: orm - :tickets: - - The "entity_name" feature of SQLAlchemy mappers has been - removed. For rationale, see https://tinyurl.com/6nm2ne - - .. change:: - :tags: orm - :tickets: - - the "autoexpire" flag on Session, sessionmaker(), and - scoped_session() has been renamed to "expire_on_commit". It - does not affect the expiration behavior of rollback(). - - .. change:: - :tags: orm - :tickets: - - fixed endless loop bug which could occur within a mapper's - deferred load of inherited attributes. - - .. change:: - :tags: orm - :tickets: - - a legacy-support flag "_enable_transaction_accounting" flag - added to Session which when False, disables all - transaction-level object accounting, including expire on - rollback, expire on commit, new/deleted list maintenance, and - autoflush on begin. - - .. change:: - :tags: orm - :tickets: - - The 'cascade' parameter to relation() accepts None as a value, - which is equivalent to no cascades. - - .. change:: - :tags: orm - :tickets: - - A critical fix to dynamic relations allows the "modified" - history to be properly cleared after a flush(). - - .. change:: - :tags: orm - :tickets: - - user-defined @properties on a class are detected and left in - place during mapper initialization. This means that a - table-bound column of the same name will not be mapped at all - if a @property is in the way (and the column is not remapped - to a different name), nor will an instrumented attribute from - an inherited class be applied. The same rules apply for names - excluded using the include_properties/exclude_properties - collections. - - .. change:: - :tags: orm - :tickets: - - Added a new SessionExtension hook called after_attach(). This - is called at the point of attachment for objects via add(), - add_all(), delete(), and merge(). - - .. change:: - :tags: orm - :tickets: 1111 - - A mapper which inherits from another, when inheriting the - columns of its inherited mapper, will use any reassigned - property names specified in that inheriting mapper. - Previously, if "Base" had reassigned "base_id" to the name - "id", "SubBase(Base)" would still get an attribute called - "base_id". This could be worked around by explicitly stating - the column in each submapper as well but this is fairly - unworkable and also impossible when using declarative. - - .. change:: - :tags: orm - :tickets: - - Fixed a series of potential race conditions in Session whereby - asynchronous GC could remove unmodified, no longer referenced - items from the session as they were present in a list of items - to be processed, typically during session.expunge_all() and - dependent methods. - - .. change:: - :tags: orm - :tickets: - - Some improvements to the _CompileOnAttr mechanism which should - reduce the probability of "Attribute x was not replaced during - compile" warnings. (this generally applies to SQLA hackers, - like Elixir devs). - - .. change:: - :tags: orm - :tickets: - - Fixed bug whereby the "unsaved, pending instance" FlushError - raised for a pending orphan would not take superclass mappers - into account when generating the list of relations responsible - for the error. - - .. change:: - :tags: sql - :tickets: - - func.count() with no arguments renders as COUNT(*), equivalent - to func.count(text('*')). - - .. change:: - :tags: sql - :tickets: 1068 - - simple label names in ORDER BY expressions render as - themselves, and not as a re-statement of their corresponding - expression. This feature is currently enabled only for - SQLite, MySQL, and PostgreSQL. It can be enabled on other - dialects as each is shown to support this - behavior. - - .. change:: - :tags: ext - :tickets: - - Class-bound attributes sent as arguments to relation()'s - remote_side and foreign_keys parameters are now accepted, - allowing them to be used with declarative. Additionally fixed - bugs involving order_by being specified as a class-bound - attribute in conjunction with eager loading. - - .. change:: - :tags: ext - :tickets: - - declarative initialization of Columns adjusted so that - non-renamed columns initialize in the same way as a non - declarative mapper. This allows an inheriting mapper to set - up its same-named "id" columns in particular such that the - parent "id" column is favored over the child column, reducing - database round trips when this value is requested. - - .. change:: - :tags: mysql - :tickets: 1110 - - Quoting of MSEnum values for use in CREATE TABLE is now - optional & will be quoted on demand as required. (Quoting was - always optional for use with existing tables.) - -.. changelog:: - :version: 0.5.0beta2 - :released: Mon Jul 14 2008 - - .. change:: - :tags: orm - :tickets: 870 - - In addition to expired attributes, deferred attributes also - load if their data is present in the result set. - - .. change:: - :tags: orm - :tickets: - - session.refresh() raises an informative error message if the - list of attributes does not include any column-based - attributes. - - .. change:: - :tags: orm - :tickets: - - query() raises an informative error message if no columns or - mappers are specified. - - .. change:: - :tags: orm - :tickets: - - lazy loaders now trigger autoflush before proceeding. This - allows expire() of a collection or scalar relation to function - properly in the context of autoflush. - - .. change:: - :tags: orm - :tickets: 887 - - column_property() attributes which represent SQL expressions - or columns that are not present in the mapped tables (such as - those from views) are automatically expired after an INSERT or - UPDATE, assuming they have not been locally modified, so that - they are refreshed with the most recent data upon access. - - .. change:: - :tags: orm - :tickets: 1082 - - Fixed explicit, self-referential joins between two - joined-table inheritance mappers when using query.join(cls, - aliased=True). - - .. change:: - :tags: orm - :tickets: - - Fixed query.join() when used in conjunction with a - columns-only clause and a SQL-expression ON clause in the - join. - - .. change:: - :tags: orm - :tickets: - - The "allow_column_override" flag from mapper() has been - removed. This flag is virtually always misunderstood. Its - specific functionality is available via the - include_properties/exclude_properties mapper arguments. - - .. change:: - :tags: orm - :tickets: 1066 - - Repaired `__str__()` method on Query. - - .. change:: - :tags: orm - :tickets: - - Session.bind gets used as a default even when table/mapper - specific binds are defined. - - .. change:: - :tags: schema - :tickets: 1075 - - Added prefixes option to `Table` that accepts a list of - strings to insert after CREATE in the CREATE TABLE statement. - - .. change:: - :tags: schema - :tickets: - - Unicode, UnicodeText types now set "assert_unicode" and - "convert_unicode" by default, but accept overriding - \**kwargs for these values. - - .. change:: - :tags: sql - :tickets: - - Added new match() operator that performs a full-text search. - Supported on PostgreSQL, SQLite, MySQL, MS-SQL, and Oracle - backends. - - .. change:: - :tags: sqlite - :tickets: 1090 - - Modified SQLite's representation of "microseconds" to match - the output of str(somedatetime), i.e. in that the microseconds - are represented as fractional seconds in string format. This - makes SQLA's SQLite date type compatible with datetimes that - were saved directly using Pysqlite (which just calls str()). - Note that this is incompatible with the existing microseconds - values in a SQLA 0.4 generated SQLite database file. - - To get the old behavior globally: - - from sqlalchemy.databases.sqlite import DateTimeMixin - DateTimeMixin.__legacy_microseconds__ = True - - To get the behavior on individual DateTime types: - - t = sqlite.SLDateTime() - t.__legacy_microseconds__ = True - - Then use "t" as the type on the Column. - - .. change:: - :tags: sqlite - :tickets: - - SQLite Date, DateTime, and Time types only accept Python - datetime objects now, not strings. If you'd like to format - dates as strings yourself with SQLite, use a String type. If - you'd like them to return datetime objects anyway despite - their accepting strings as input, make a TypeDecorator around - String - SQLA doesn't encourage this pattern. - - .. change:: - :tags: extensions - :tickets: 1096 - - Declarative supports a __table_args__ class variable, which is - either a dictionary, or tuple of the form (arg1, arg2, ..., - {kwarg1:value, ...}) which contains positional + kw arguments - to be passed to the Table constructor. - -.. changelog:: - :version: 0.5.0beta1 - :released: Thu Jun 12 2008 - - .. change:: - :tags: - :tickets: - - The "__init__" trigger/decorator added by mapper now attempts - to exactly mirror the argument signature of the original - __init__. The pass-through for '_sa_session' is no longer - implicit- you must allow for this keyword argument in your - constructor. - - .. change:: - :tags: - :tickets: - - ClassState is renamed to ClassManager. - - .. change:: - :tags: - :tickets: - - Classes may supply their own InstrumentationManager by - providing a __sa_instrumentation_manager__ property. - - .. change:: - :tags: - :tickets: - - Custom instrumentation may use any mechanism to associate a - ClassManager with a class and an InstanceState with an - instance. Attributes on those objects are still the default - association mechanism used by SQLAlchemy's native - instrumentation. - - .. change:: - :tags: - :tickets: - - Moved entity_name, _sa_session_id, and _instance_key from the - instance object to the instance state. These values are still - available in the old way, which is now deprecated, using - descriptors attached to the class. A deprecation warning will - be issued when accessed. - - .. change:: - :tags: - :tickets: - - The _prepare_instrumentation alias for prepare_instrumentation - has been removed. - - .. change:: - :tags: - :tickets: - - sqlalchemy.exceptions has been renamed to sqlalchemy.exc. The - module may be imported under either name. - - .. change:: - :tags: - :tickets: - - ORM-related exceptions are now defined in sqlalchemy.orm.exc. - ConcurrentModificationError, FlushError, and - UnmappedColumnError compatibility aliases are installed in - sqlalchemy.exc during the import of sqlalchemy.orm. - - .. change:: - :tags: - :tickets: - - sqlalchemy.logging has been renamed to sqlalchemy.log. - - .. change:: - :tags: - :tickets: - - The transitional sqlalchemy.log.SADeprecationWarning alias for - the warning's definition in sqlalchemy.exc has been removed. - - .. change:: - :tags: - :tickets: - - exc.AssertionError has been removed and usage replaced with - Python's built-in AssertionError. - - .. change:: - :tags: - :tickets: - - The behavior of MapperExtensions attached to multiple, - entity_name= primary mappers for a single class has been - altered. The first mapper() defined for a class is the only - mapper eligible for the MapperExtension 'instrument_class', - 'init_instance' and 'init_failed' events. This is backwards - incompatible; previously the extensions of last mapper defined - would receive these events. - - .. change:: - :tags: firebird - :tickets: - - Added support for returning values from inserts (2.0+ only), - updates and deletes (2.1+ only). - - .. change:: - :tags: general - :tickets: - - global "propigate"->"propagate" change. - - .. change:: - :tags: orm - :tickets: - - polymorphic_union() function respects the "key" of each - Column if they differ from the column's name. - - .. change:: - :tags: orm - :tickets: 1199 - - Fixed 0.4-only bug preventing composite columns - from working properly with inheriting mappers - - .. change:: - :tags: orm - :tickets: - - Fixed RLock-related bug in mapper which could deadlock upon - reentrant mapper compile() calls, something that occurs when - using declarative constructs inside of ForeignKey objects. - Ported from 0.5. - - .. change:: - :tags: orm - :tickets: 1213 - - Fixed bug in composite types which prevented a primary-key - composite type from being mutated. - - .. change:: - :tags: orm - :tickets: 976 - - Added ScopedSession.is_active accessor. - - .. change:: - :tags: orm - :tickets: 939 - - Class-bound accessor can be used as the argument to - relation() order_by. - - .. change:: - :tags: orm - :tickets: 1072 - - Fixed shard_id argument on ShardedSession.execute(). - - .. change:: - :tags: sql - :tickets: 1246 - - Connection.invalidate() checks for closed status - to avoid attribute errors. - - .. change:: - :tags: sql - :tickets: 1094 - - NullPool supports reconnect on failure behavior. - - .. change:: - :tags: sql - :tickets: 1299 - - The per-dialect cache used by TypeEngine to cache - dialect-specific types is now a WeakKeyDictionary. - This to prevent dialect objects from - being referenced forever for an application that - creates an arbitrarily large number of engines - or dialects. There is a small performance penalty - which will be resolved in 0.6. - - .. change:: - :tags: sql - :tickets: - - Fixed SQLite reflection methods so that non-present - cursor.description, which triggers an auto-cursor - close, will be detected so that no results doesn't - fail on recent versions of pysqlite which raise - an error when fetchone() called with no rows present. - - .. change:: - :tags: postgres - :tickets: 714 - - Added Index reflection support to Postgres, using a - great patch we long neglected, submitted by - Ken Kuhlman. - - .. change:: - :tags: mysql - :tickets: 1241 - - Fixed bug in exception raise when FK columns not present - during reflection. - - .. change:: - :tags: oracle - :tickets: 1265 - - Fixed bug which was preventing out params of certain types - from being received; thanks a ton to huddlej at wwu.edu ! diff --git a/doc/build/changelog/changelog_06.rst b/doc/build/changelog/changelog_06.rst deleted file mode 100644 index 739df36b23..0000000000 --- a/doc/build/changelog/changelog_06.rst +++ /dev/null @@ -1,5411 +0,0 @@ -============= -0.6 Changelog -============= - - -.. changelog:: - :version: 0.6.9 - :released: Sat May 05 2012 - - .. change:: - :tags: general - :tickets: 2279 - - Adjusted the "importlater" mechanism, which is - used internally to resolve import cycles, - such that the usage of __import__ is completed - when the import of sqlalchemy or sqlalchemy.orm - is done, thereby avoiding any usage of __import__ - after the application starts new threads, - fixes. - - .. change:: - :tags: orm - :tickets: 2197 - - Fixed bug whereby the source clause - used by query.join() would be inconsistent - if against a column expression that combined - multiple entities together. - - .. change:: - :tags: orm, bug - :tickets: 2310 - - fixed inappropriate evaluation of user-mapped - object in a boolean context within query.get(). - - .. change:: - :tags: orm - :tickets: 2228 - - Fixed bug apparent only in Python 3 whereby - sorting of persistent + pending objects during - flush would produce an illegal comparison, - if the persistent object primary key - is not a single integer. - - .. change:: - :tags: orm - :tickets: 2234 - - Fixed bug where query.join() + aliased=True - from a joined-inh structure to itself on - relationship() with join condition on the child - table would convert the lead entity into the - joined one inappropriately. - - .. change:: - :tags: orm - :tickets: 2287 - - Fixed bug whereby mapper.order_by attribute would - be ignored in the "inner" query within a - subquery eager load. . - - .. change:: - :tags: orm - :tickets: 2215 - - Fixed bug whereby if a mapped class - redefined __hash__() or __eq__() to something - non-standard, which is a supported use case - as SQLA should never consult these, - the methods would be consulted if the class - was part of a "composite" (i.e. non-single-entity) - result set. - - .. change:: - :tags: orm - :tickets: 2188 - - Fixed subtle bug that caused SQL to blow - up if: column_property() against subquery + - joinedload + LIMIT + order by the column - property() occurred. . - - .. change:: - :tags: orm - :tickets: 2207 - - The join condition produced by with_parent - as well as when using a "dynamic" relationship - against a parent will generate unique - bindparams, rather than incorrectly repeating - the same bindparam. . - - .. change:: - :tags: orm - :tickets: 2199 - - Repaired the "no statement condition" - assertion in Query which would attempt - to raise if a generative method were called - after from_statement() were called.. - - .. change:: - :tags: orm - :tickets: 1776 - - Cls.column.collate("some collation") now - works. - - .. change:: - :tags: orm, bug - :tickets: 2297 - - Fixed the error formatting raised when - a tuple is inadvertently passed to session.query(). - - .. change:: - :tags: engine - :tickets: 2317 - - Backported the fix for introduced - in 0.7.4, which ensures that the connection - is in a valid state before attempting to call - rollback()/prepare()/release() on savepoint - and two-phase transactions. - - .. change:: - :tags: sql - :tickets: 2188 - - Fixed two subtle bugs involving column - correspondence in a selectable, - one with the same labeled subquery repeated, the other - when the label has been "grouped" and - loses itself. Affects. - - .. change:: - :tags: sql - :tickets: - - Fixed bug whereby "warn on unicode" flag - would get set for the String type - when used with certain dialects. This - bug is not in 0.7. - - .. change:: - :tags: sql - :tickets: 2270 - - Fixed bug whereby with_only_columns() method of - Select would fail if a selectable were passed.. However, the FROM behavior is - still incorrect here, so you need 0.7 in - any case for this use case to be usable. - - .. change:: - :tags: schema - :tickets: - - Added an informative error message when - ForeignKeyConstraint refers to a column name in - the parent that is not found. - - .. change:: - :tags: postgresql - :tickets: 2291, 2141 - - Fixed bug related to whereby the - same modified index behavior in PG 9 affected - primary key reflection on a renamed column.. - - .. change:: - :tags: mysql - :tickets: 2186 - - Fixed OurSQL dialect to use ansi-neutral - quote symbol "'" for XA commands instead - of '"'. . - - .. change:: - :tags: mysql - :tickets: 2225 - - a CREATE TABLE will put the COLLATE option - after CHARSET, which appears to be part of - MySQL's arbitrary rules regarding if it will actually - work or not. - - .. change:: - :tags: mssql, bug - :tickets: 2269 - - Decode incoming values when retrieving - list of index names and the names of columns - within those indexes. - - .. change:: - :tags: oracle - :tickets: 2200 - - Added ORA-00028 to disconnect codes, use - cx_oracle _Error.code to get at the code,. - - .. change:: - :tags: oracle - :tickets: 2220 - - repaired the oracle.RAW type which did not - generate the correct DDL. - - .. change:: - :tags: oracle - :tickets: 2212 - - added CURRENT to reserved word list. - - .. change:: - :tags: examples - :tickets: 2266 - - Adjusted dictlike-polymorphic.py example - to apply the CAST such that it works on - PG, other databases. - -.. changelog:: - :version: 0.6.8 - :released: Sun Jun 05 2011 - - .. change:: - :tags: orm - :tickets: 2144 - - Calling query.get() against a column-based entity is - invalid, this condition now raises a deprecation warning. - - .. change:: - :tags: orm - :tickets: 2151 - - a non_primary mapper will inherit the _identity_class - of the primary mapper. This so that a non_primary - established against a class that's normally in an - inheritance mapping will produce results that are - identity-map compatible with that of the primary - mapper - - .. change:: - :tags: orm - :tickets: 2148 - - Backported 0.7's identity map implementation, which - does not use a mutex around removal. This as some users - were still getting deadlocks despite the adjustments - in 0.6.7; the 0.7 approach that doesn't use a mutex - does not appear to produce "dictionary changed size" - issues, the original rationale for the mutex. - - .. change:: - :tags: orm - :tickets: 2163 - - Fixed the error message emitted for "can't - execute syncrule for destination column 'q'; - mapper 'X' does not map this column" to - reference the correct mapper. . - - .. change:: - :tags: orm - :tickets: 2149 - - Fixed bug where determination of "self referential" - relationship would fail with no workaround - for joined-inh subclass related to itself, - or joined-inh subclass related to a subclass - of that with no cols in the sub-sub class - in the join condition. - - .. change:: - :tags: orm - :tickets: 2153 - - mapper() will ignore non-configured foreign keys - to unrelated tables when determining inherit - condition between parent and child class. - This is equivalent to behavior already - applied to declarative. Note that 0.7 has a - more comprehensive solution to this, altering - how join() itself determines an FK error. - - .. change:: - :tags: orm - :tickets: 2171 - - Fixed bug whereby mapper mapped to an anonymous - alias would fail if logging were used, due to - unescaped % sign in the alias name. - - .. change:: - :tags: orm - :tickets: 2170 - - Modify the text of the message which occurs - when the "identity" key isn't detected on - flush, to include the common cause that - the Column isn't set up to detect - auto-increment correctly;. - - .. change:: - :tags: orm - :tickets: 2182 - - Fixed bug where transaction-level "deleted" - collection wouldn't be cleared of expunged - states, raising an error if they later - became transient. - - .. change:: - :tags: sql - :tickets: 2147 - - Fixed bug whereby if FetchedValue was passed - to column server_onupdate, it would not - have its parent "column" assigned, added - test coverage for all column default assignment - patterns. - - .. change:: - :tags: sql - :tickets: 2167 - - Fixed bug whereby nesting a label of a select() - with another label in it would produce incorrect - exported columns. Among other things this would - break an ORM column_property() mapping against - another column_property(). . - - .. change:: - :tags: engine - :tickets: 2178 - - Adjusted the __contains__() method of - a RowProxy result row such that no exception - throw is generated internally; - NoSuchColumnError() also will generate its - message regardless of whether or not the column - construct can be coerced to a string.. - - .. change:: - :tags: postgresql - :tickets: 2141 - - Fixed bug affecting PG 9 whereby index reflection - would fail if against a column whose name - had changed. . - - .. change:: - :tags: postgresql - :tickets: 2175 - - Some unit test fixes regarding numeric arrays, - MATCH operator. A potential floating-point - inaccuracy issue was fixed, and certain tests - of the MATCH operator only execute within an - EN-oriented locale for now. . - - .. change:: - :tags: mssql - :tickets: 2169 - - Fixed bug in MSSQL dialect whereby the aliasing - applied to a schema-qualified table would leak - into enclosing select statements. - - .. change:: - :tags: mssql - :tickets: 2159 - - Fixed bug whereby DATETIME2 type would fail on - the "adapt" step when used in result sets or - bound parameters. This issue is not in 0.7. - -.. changelog:: - :version: 0.6.7 - :released: Wed Apr 13 2011 - - .. change:: - :tags: orm - :tickets: 2087 - - Tightened the iterate vs. remove mutex around the - identity map iteration, attempting to reduce the - chance of an (extremely rare) reentrant gc operation - causing a deadlock. Might remove the mutex in - 0.7. - - .. change:: - :tags: orm - :tickets: 2030 - - Added a `name` argument to `Query.subquery()`, to allow - a fixed name to be assigned to the alias object. - - .. change:: - :tags: orm - :tickets: 2019 - - A warning is emitted when a joined-table inheriting mapper - has no primary keys on the locally mapped table - (but has pks on the superclass table). - - .. change:: - :tags: orm - :tickets: 2038 - - Fixed bug where "middle" class in a polymorphic hierarchy - would have no 'polymorphic_on' column if it didn't also - specify a 'polymorphic_identity', leading to strange - errors upon refresh, wrong class loaded when querying - from that target. Also emits the correct WHERE criterion - when using single table inheritance. - - .. change:: - :tags: orm - :tickets: 1995 - - Fixed bug where a column with a SQL or server side default - that was excluded from a mapping with include_properties - or exclude_properties would result in UnmappedColumnError. - - .. change:: - :tags: orm - :tickets: 2046 - - A warning is emitted in the unusual case that an - append or similar event on a collection occurs after - the parent object has been dereferenced, which - prevents the parent from being marked as "dirty" - in the session. This will be an exception in 0.7. - - .. change:: - :tags: orm - :tickets: 2098 - - Fixed bug in query.options() whereby a path - applied to a lazyload using string keys could - overlap a same named attribute on the wrong - entity. Note 0.7 has an updated version of this - fix. - - .. change:: - :tags: orm - :tickets: 2063 - - Reworded the exception raised when a flush - is attempted of a subclass that is not polymorphic - against the supertype. - - .. change:: - :tags: orm - :tickets: 2123 - - Some fixes to the state handling regarding - backrefs, typically when autoflush=False, where - the back-referenced collection wouldn't - properly handle add/removes with no net - change. Thanks to Richard Murri for the - test case + patch. - - .. change:: - :tags: orm - :tickets: 2130 - - a "having" clause would be copied from the - inside to the outside query if from_self() - were used.. - - .. change:: - :tags: sql - :tickets: 2028 - - Column.copy(), as used in table.tometadata(), copies the - 'doc' attribute. - - .. change:: - :tags: sql - :tickets: 2023 - - Added some defs to the resultproxy.c extension so that - the extension compiles and runs on Python 2.4. - - .. change:: - :tags: sql - :tickets: 2042 - - The compiler extension now supports overriding the default - compilation of expression._BindParamClause including that - the auto-generated binds within the VALUES/SET clause - of an insert()/update() statement will also use the new - compilation rules. - - .. change:: - :tags: sql - :tickets: 2089 - - Added accessors to ResultProxy "returns_rows", "is_insert" - - .. change:: - :tags: sql - :tickets: 2116 - - The limit/offset keywords to select() as well - as the value passed to select.limit()/offset() - will be coerced to integer. - - .. change:: - :tags: engine - :tickets: 2102 - - Fixed bug in QueuePool, SingletonThreadPool whereby - connections that were discarded via overflow or periodic - cleanup() were not explicitly closed, leaving garbage - collection to the task instead. This generally only - affects non-reference-counting backends like Jython - and PyPy. Thanks to Jaimy Azle for spotting - this. - - .. change:: - :tags: sqlite - :tickets: 2115 - - Fixed bug where reflection of foreign key - created as "REFERENCES " without - col name would fail. - - .. change:: - :tags: postgresql - :tickets: 1083 - - When explicit sequence execution derives the name - of the auto-generated sequence of a SERIAL column, - which currently only occurs if implicit_returning=False, - now accommodates if the table + column name is greater - than 63 characters using the same logic PostgreSQL uses. - - .. change:: - :tags: postgresql - :tickets: 2044 - - Added an additional libpq message to the list of "disconnect" - exceptions, "could not receive data from server" - - .. change:: - :tags: postgresql - :tickets: 2092 - - Added RESERVED_WORDS for postgresql dialect. - - .. change:: - :tags: postgresql - :tickets: 2073 - - Fixed the BIT type to allow a "length" parameter, "varying" - parameter. Reflection also fixed. - - .. change:: - :tags: informix - :tickets: 2092 - - Added RESERVED_WORDS informix dialect. - - .. change:: - :tags: mssql - :tickets: 2071 - - Rewrote the query used to get the definition of a view, - typically when using the Inspector interface, to - use sys.sql_modules instead of the information schema, - thereby allowing views definitions longer than 4000 - characters to be fully returned. - - .. change:: - :tags: mysql - :tickets: 2047 - - oursql dialect accepts the same "ssl" arguments in - create_engine() as that of MySQLdb. - - .. change:: - :tags: firebird - :tickets: 2083 - - The "implicit_returning" flag on create_engine() is - honored if set to False. - - .. change:: - :tags: oracle - :tickets: 2100 - - Using column names that would require quotes - for the column itself or for a name-generated - bind parameter, such as names with special - characters, underscores, non-ascii characters, - now properly translate bind parameter keys when - talking to cx_oracle. - - .. change:: - :tags: oracle - :tickets: 2116 - - Oracle dialect adds use_binds_for_limits=False - create_engine() flag, will render the LIMIT/OFFSET - values inline instead of as binds, reported to - modify the execution plan used by Oracle. - - .. change:: - :tags: ext - :tickets: 2090 - - The horizontal_shard ShardedSession class accepts the common - Session argument "query_cls" as a constructor argument, - to enable further subclassing of ShardedQuery. - - .. change:: - :tags: declarative - :tickets: 2050 - - Added an explicit check for the case that the name - 'metadata' is used for a column attribute on a - declarative class. - - .. change:: - :tags: declarative - :tickets: 2061 - - Fix error message referencing old @classproperty - name to reference @declared_attr - - .. change:: - :tags: declarative - :tickets: 2091 - - Arguments in __mapper_args__ that aren't "hashable" - aren't mistaken for always-hashable, possibly-column - arguments. - - .. change:: - :tags: documentation - :tickets: 2029 - - Documented SQLite DATE/TIME/DATETIME types. - - .. change:: - :tags: examples - :tickets: 2090 - - The Beaker caching example allows a "query_cls" argument - to the query_callable() function. - -.. changelog:: - :version: 0.6.6 - :released: Sat Jan 08 2011 - - .. change:: - :tags: orm - :tickets: - - Fixed bug whereby a non-"mutable" attribute modified event - which occurred on an object that was clean except for - preceding mutable attribute changes would fail to strongly - reference itself in the identity map. This would cause the - object to be garbage collected, losing track of any changes - that weren't previously saved in the "mutable changes" - dictionary. - - .. change:: - :tags: orm - :tickets: 2013 - - Fixed bug whereby "passive_deletes='all'" wasn't passing - the correct symbols to lazy loaders during flush, thereby - causing an unwarranted load. - - .. change:: - :tags: orm - :tickets: 1997 - - Fixed bug which prevented composite mapped - attributes from being used on a mapped select statement.. Note the workings of composite are slated to - change significantly in 0.7. - - .. change:: - :tags: orm - :tickets: 1976 - - active_history flag also added to composite(). - The flag has no effect in 0.6, but is instead - a placeholder flag for forwards compatibility, - as it applies in 0.7 for composites. - - .. change:: - :tags: orm - :tickets: 2002 - - Fixed uow bug whereby expired objects passed to - Session.delete() would not have unloaded references - or collections taken into account when deleting - objects, despite passive_deletes remaining at - its default of False. - - .. change:: - :tags: orm - :tickets: 1987 - - A warning is emitted when version_id_col is specified - on an inheriting mapper when the inherited mapper - already has one, if those column expressions are not - the same. - - .. change:: - :tags: orm - :tickets: 1954 - - "innerjoin" flag doesn't take effect along the chain - of joinedload() joins if a previous join in that chain - is an outer join, thus allowing primary rows without - a referenced child row to be correctly returned - in results. - - .. change:: - :tags: orm - :tickets: 1964 - - Fixed bug regarding "subqueryload" strategy whereby - strategy would fail if the entity was an aliased() - construct. - - .. change:: - :tags: orm - :tickets: 2014 - - Fixed bug regarding "subqueryload" strategy whereby - the join would fail if using a multi-level load - of the form from A->joined-subclass->C - - .. change:: - :tags: orm - :tickets: 1968 - - Fixed indexing of Query objects by -1. It was erroneously - transformed to the empty slice -1:0 that resulted in - IndexError. - - .. change:: - :tags: orm - :tickets: 1971 - - The mapper argument "primary_key" can be passed as a - single column as well as a list or tuple. - The documentation examples that illustrated it as a - scalar value have been changed to lists. - - .. change:: - :tags: orm - :tickets: 1961 - - Added active_history flag to relationship() - and column_property(), forces attribute events to - always load the "old" value, so that it's available to - attributes.get_history(). - - .. change:: - :tags: orm - :tickets: 1977 - - Query.get() will raise if the number of params - in a composite key is too large, as well as too - small. - - .. change:: - :tags: orm - :tickets: 1992 - - Backport of "optimized get" fix from 0.7, - improves the generation of joined-inheritance - "load expired row" behavior. - - .. change:: - :tags: orm - :tickets: - - A little more verbiage to the "primaryjoin" error, - in an unusual condition that the join condition - "works" for viewonly but doesn't work for non-viewonly, - and foreign_keys wasn't used - adds "foreign_keys" to - the suggestion. Also add "foreign_keys" to the - suggestion for the generic "direction" error. - - .. change:: - :tags: sql - :tickets: 1984 - - Fixed operator precedence rules for multiple - chains of a single non-associative operator. - I.e. "x - (y - z)" will compile as "x - (y - z)" - and not "x - y - z". Also works with labels, - i.e. "x - (y - z).label('foo')" - - .. change:: - :tags: sql - :tickets: 1967 - - The 'info' attribute of Column is copied during - Column.copy(), i.e. as occurs when using columns - in declarative mixins. - - .. change:: - :tags: sql - :tickets: - - Added a bind processor for booleans which coerces - to int, for DBAPIs such as pymssql that naively call - str() on values. - - .. change:: - :tags: sql - :tickets: 2000 - - CheckConstraint will copy its 'initially', 'deferrable', - and '_create_rule' attributes within a copy()/tometadata() - - .. change:: - :tags: engine - :tickets: - - The "unicode warning" against non-unicode bind data - is now raised only when the - Unicode type is used explicitly; not when - convert_unicode=True is used on the engine - or String type. - - .. change:: - :tags: engine - :tickets: 1978 - - Fixed memory leak in C version of Decimal result - processor. - - .. change:: - :tags: engine - :tickets: 1871 - - Implemented sequence check capability for the C - version of RowProxy, as well as 2.7 style - "collections.Sequence" registration for RowProxy. - - .. change:: - :tags: engine - :tickets: 1998 - - Threadlocal engine methods rollback(), commit(), - prepare() won't raise if no transaction is in progress; - this was a regression introduced in 0.6. - - .. change:: - :tags: engine - :tickets: 2004 - - Threadlocal engine returns itself upon begin(), - begin_nested(); engine then implements contextmanager - methods to allow the "with" statement. - - .. change:: - :tags: postgresql - :tickets: 1984 - - Single element tuple expressions inside an IN clause - parenthesize correctly, also from - - .. change:: - :tags: postgresql - :tickets: 1955 - - Ensured every numeric, float, int code, scalar + array, - are recognized by psycopg2 and pg8000's "numeric" - base type. - - .. change:: - :tags: postgresql - :tickets: 1956 - - Added as_uuid=True flag to the UUID type, will receive - and return values as Python UUID() objects rather than - strings. Currently, the UUID type is only known to - work with psycopg2. - - .. change:: - :tags: postgresql - :tickets: 1989 - - Fixed bug whereby KeyError would occur with non-ENUM - supported PG versions after a pool dispose+recreate - would occur. - - .. change:: - :tags: mysql - :tickets: 1960 - - Fixed error handling for Jython + zxjdbc, such that - has_table() property works again. Regression from - 0.6.3 (we don't have a Jython buildbot, sorry) - - .. change:: - :tags: sqlite - :tickets: 1851 - - The REFERENCES clause in a CREATE TABLE that includes - a remote schema to another table with the same schema - name now renders the remote name without - the schema clause, as required by SQLite. - - .. change:: - :tags: sqlite - :tickets: - - On the same theme, the REFERENCES clause in a CREATE TABLE - that includes a remote schema to a *different* schema - than that of the parent table doesn't render at all, - as cross-schema references do not appear to be supported. - - .. change:: - :tags: mssql - :tickets: 1770 - - The rewrite of index reflection in was - unfortunately not tested correctly, and returned incorrect - results. This regression is now fixed. - - .. change:: - :tags: oracle - :tickets: 1953 - - The cx_oracle "decimal detection" logic, which takes place - for result set columns with ambiguous numeric characteristics, - now uses the decimal point character determined by the locale/ - NLS_LANG setting, using an on-first-connect detection of - this character. cx_oracle 5.0.3 or greater is also required - when using a non-period-decimal-point NLS_LANG setting.. - - .. change:: - :tags: firebird - :tickets: 2012 - - Firebird numeric type now checks for Decimal explicitly, - lets float() pass right through, thereby allowing - special values such as float('inf'). - - .. change:: - :tags: declarative - :tickets: 1972 - - An error is raised if __table_args__ is not in tuple - or dict format, and is not None. - - .. change:: - :tags: sqlsoup - :tickets: 1975 - - Added "map_to()" method to SqlSoup, which is a "master" - method which accepts explicit arguments for each aspect of - the selectable and mapping, including a base class per - mapping. - - .. change:: - :tags: sqlsoup - :tickets: - - Mapped selectables used with the map(), with_labels(), - join() methods no longer put the given argument into the - internal "cache" dictionary. Particularly since the - join() and select() objects are created in the method - itself this was pretty much a pure memory leaking behavior. - - .. change:: - :tags: examples - :tickets: - - The versioning example now supports detection of changes - in an associated relationship(). - -.. changelog:: - :version: 0.6.5 - :released: Sun Oct 24 2010 - - .. change:: - :tags: orm - :tickets: 1914 - - Added a new "lazyload" option "immediateload". - Issues the usual "lazy" load operation automatically - as the object is populated. The use case - here is when loading objects to be placed in - an offline cache, or otherwise used after - the session isn't available, and straight 'select' - loading, not 'joined' or 'subquery', is desired. - - .. change:: - :tags: orm - :tickets: 1920 - - New Query methods: query.label(name), query.as_scalar(), - return the query's statement as a scalar subquery - with /without label; - query.with_entities(\*ent), replaces the SELECT list of - the query with new entities. - Roughly equivalent to a generative form of query.values() - which accepts mapped entities as well as column - expressions. - - .. change:: - :tags: orm - :tickets: - - Fixed recursion bug which could occur when moving - an object from one reference to another, with - backrefs involved, where the initiating parent - was a subclass (with its own mapper) of the - previous parent. - - .. change:: - :tags: orm - :tickets: 1918 - - Fixed a regression in 0.6.4 which occurred if you - passed an empty list to "include_properties" on - mapper() - - .. change:: - :tags: orm - :tickets: - - Fixed labeling bug in Query whereby the NamedTuple - would mis-apply labels if any of the column - expressions were un-labeled. - - .. change:: - :tags: orm - :tickets: 1925 - - Patched a case where query.join() would adapt the - right side to the right side of the left's join - inappropriately - - .. change:: - :tags: orm - :tickets: - - Query.select_from() has been beefed up to help - ensure that a subsequent call to query.join() - will use the select_from() entity, assuming it's - a mapped entity and not a plain selectable, - as the default "left" side, not the first entity - in the Query object's list of entities. - - .. change:: - :tags: orm - :tickets: - - The exception raised by Session when it is used - subsequent to a subtransaction rollback (which is what - happens when a flush fails in autocommit=False mode) has - now been reworded (this is the "inactive due to a - rollback in a subtransaction" message). In particular, - if the rollback was due to an exception during flush(), - the message states this is the case, and reiterates the - string form of the original exception that occurred - during flush. If the session is closed due to explicit - usage of subtransactions (not very common), the message - just states this is the case. - - .. change:: - :tags: orm - :tickets: - - The exception raised by Mapper when repeated requests to - its initialization are made after initialization already - failed no longer assumes the "hasattr" case, since - there's other scenarios in which this message gets - emitted, and the message also does not compound onto - itself multiple times - you get the same message for - each attempt at usage. The misnomer "compiles" is being - traded out for "initialize". - - .. change:: - :tags: orm - :tickets: 1935 - - Fixed bug in query.update() where 'evaluate' or 'fetch' - expiration would fail if the column expression key was - a class attribute with a different keyname as the - actual column name. - - .. change:: - :tags: orm - :tickets: - - Added an assertion during flush which ensures - that no NULL-holding identity keys were generated - on "newly persistent" objects. - This can occur when user defined code inadvertently - triggers flushes on not-fully-loaded objects. - - .. change:: - :tags: orm - :tickets: 1910 - - lazy loads for relationship attributes now use - the current state, not the "committed" state, - of foreign and primary key attributes - when issuing SQL, if a flush is not in process. - Previously, only the database-committed state would - be used. In particular, this would cause a many-to-one - get()-on-lazyload operation to fail, as autoflush - is not triggered on these loads when the attributes are - determined and the "committed" state may not be - available. - - .. change:: - :tags: orm - :tickets: - - A new flag on relationship(), load_on_pending, allows - the lazy loader to fire off on pending objects without a - flush taking place, as well as a transient object that's - been manually "attached" to the session. Note that this - flag blocks attribute events from taking place when an - object is loaded, so backrefs aren't available until - after a flush. The flag is only intended for very - specific use cases. - - .. change:: - :tags: orm - :tickets: - - Another new flag on relationship(), cascade_backrefs, - disables the "save-update" cascade when the event was - initiated on the "reverse" side of a bidirectional - relationship. This is a cleaner behavior so that - many-to-ones can be set on a transient object without - it getting sucked into the child object's session, - while still allowing the forward collection to - cascade. We *might* default this to False in 0.7. - - .. change:: - :tags: orm - :tickets: - - Slight improvement to the behavior of - "passive_updates=False" when placed only on the - many-to-one side of a relationship; documentation has - been clarified that passive_updates=False should really - be on the one-to-many side. - - .. change:: - :tags: orm - :tickets: - - Placing passive_deletes=True on a many-to-one emits - a warning, since you probably intended to put it on - the one-to-many side. - - .. change:: - :tags: orm - :tickets: - - Fixed bug that would prevent "subqueryload" from - working correctly with single table inheritance - for a relationship from a subclass - the "where - type in (x, y, z)" only gets placed on the inside, - instead of repeatedly. - - .. change:: - :tags: orm - :tickets: - - When using from_self() with single table inheritance, - the "where type in (x, y, z)" is placed on the outside - of the query only, instead of repeatedly. May make - some more adjustments to this. - - .. change:: - :tags: orm - :tickets: 1924 - - scoped_session emits a warning when configure() is - called if a Session is already present (checks only the - current thread) - - .. change:: - :tags: orm - :tickets: 1932 - - reworked the internals of mapper.cascade_iterator() to - cut down method calls by about 9% in some circumstances. - - .. change:: - :tags: sql - :tickets: - - Fixed bug in TypeDecorator whereby the dialect-specific - type was getting pulled in to generate the DDL for a - given type, which didn't always return the correct result. - - .. change:: - :tags: sql - :tickets: - - TypeDecorator can now have a fully constructed type - specified as its "impl", in addition to a type class. - - .. change:: - :tags: sql - :tickets: - - TypeDecorator will now place itself as the resulting - type for a binary expression where the type coercion - rules would normally return its impl type - previously, - a copy of the impl type would be returned which would - have the TypeDecorator embedded into it as the "dialect" - impl, this was probably an unintentional way of achieving - the desired effect. - - .. change:: - :tags: sql - :tickets: - - TypeDecorator.load_dialect_impl() returns "self.impl" by - default, i.e. not the dialect implementation type of - "self.impl". This to support compilation correctly. - Behavior can be user-overridden in exactly the same way - as before to the same effect. - - .. change:: - :tags: sql - :tickets: - - Added type_coerce(expr, type\_) expression element. - Treats the given expression as the given type when evaluating - expressions and processing result rows, but does not - affect the generation of SQL, other than an anonymous - label. - - .. change:: - :tags: sql - :tickets: - - Table.tometadata() now copies Index objects associated - with the Table as well. - - .. change:: - :tags: sql - :tickets: - - Table.tometadata() issues a warning if the given Table - is already present in the target MetaData - the existing - Table object is returned. - - .. change:: - :tags: sql - :tickets: - - An informative error message is raised if a Column - which has not yet been assigned a name, i.e. as in - declarative, is used in a context where it is - exported to the columns collection of an enclosing - select() construct, or if any construct involving - that column is compiled before its name is - assigned. - - .. change:: - :tags: sql - :tickets: 1862 - - as_scalar(), label() can be called on a selectable - which contains a Column that is not yet named. - - .. change:: - :tags: sql - :tickets: 1907 - - Fixed recursion overflow which could occur when operating - with two expressions both of type "NullType", but - not the singleton NULLTYPE instance. - - .. change:: - :tags: declarative - :tickets: 1922 - - @classproperty (soon/now @declared_attr) takes effect for - __mapper_args__, __table_args__, __tablename__ on - a base class that is not a mixin, as well as mixins. - - .. change:: - :tags: declarative - :tickets: 1915 - - @classproperty 's official name/location for usage - with declarative is sqlalchemy.ext.declarative.declared_attr. - Same thing, but moving there since it is more of a - "marker" that's specific to declarative, - not just an attribute technique. - - .. change:: - :tags: declarative - :tickets: 1931, 1930 - - Fixed bug whereby columns on a mixin wouldn't propagate - correctly to a single-table, or joined-table, - inheritance scheme where the attribute name is - different than that of the column.,. - - .. change:: - :tags: declarative - :tickets: - - A mixin can now specify a column that overrides - a column of the same name associated with a superclass. - Thanks to Oystein Haaland. - - .. change:: - :tags: engine - :tickets: - - Fixed a regression in 0.6.4 whereby the change that - allowed cursor errors to be raised consistently broke - the result.lastrowid accessor. Test coverage has - been added for result.lastrowid. Note that lastrowid - is only supported by Pysqlite and some MySQL drivers, - so isn't super-useful in the general case. - - .. change:: - :tags: engine - :tickets: - - the logging message emitted by the engine when - a connection is first used is now "BEGIN (implicit)" - to emphasize that DBAPI has no explicit begin(). - - .. change:: - :tags: engine - :tickets: 1936 - - added "views=True" option to metadata.reflect(), - will add the list of available views to those - being reflected. - - .. change:: - :tags: engine - :tickets: 1899 - - engine_from_config() now accepts 'debug' for - 'echo', 'echo_pool', 'force' for 'convert_unicode', - boolean values for 'use_native_unicode'. - - .. change:: - :tags: postgresql - :tickets: - - Added "as_tuple" flag to ARRAY type, returns results - as tuples instead of lists to allow hashing. - - .. change:: - :tags: postgresql - :tickets: 1933 - - Fixed bug which prevented "domain" built from a - custom type such as "enum" from being reflected. - - .. change:: - :tags: mysql - :tickets: 1940 - - Fixed bug involving reflection of CURRENT_TIMESTAMP - default used with ON UPDATE clause, thanks to - Taavi Burns - - .. change:: - :tags: oracle - :tickets: 1878 - - The implicit_returning argument to create_engine() - is now honored regardless of detected version of - Oracle. Previously, the flag would be forced - to False if server version info was < 10. - - .. change:: - :tags: mssql - :tickets: 1946 - - Fixed reflection bug which did not properly handle - reflection of unknown types. - - .. change:: - :tags: mssql - :tickets: 1943 - - Fixed bug where aliasing of tables with "schema" would - fail to compile properly. - - .. change:: - :tags: mssql - :tickets: 1770 - - Rewrote the reflection of indexes to use sys. - catalogs, so that column names of any configuration - (spaces, embedded commas, etc.) can be reflected. - Note that reflection of indexes requires SQL - Server 2005 or greater. - - .. change:: - :tags: mssql - :tickets: 1952 - - mssql+pymssql dialect now honors the "port" portion - of the URL instead of discarding it. - - .. change:: - :tags: informix - :tickets: 1906 - - *Major* cleanup / modernization of the Informix - dialect for 0.6, courtesy Florian Apolloner. - - .. change:: - :tags: tests - :tickets: - - the NoseSQLAlchemyPlugin has been moved to a - new package "sqlalchemy_nose" which installs - along with "sqlalchemy". This so that the "nosetests" - script works as always but also allows the - --with-coverage option to turn on coverage before - SQLAlchemy modules are imported, allowing coverage - to work correctly. - - .. change:: - :tags: misc - :tickets: 1890 - - CircularDependencyError now has .cycles and .edges - members, which are the set of elements involved in - one or more cycles, and the set of edges as 2-tuples. - -.. changelog:: - :version: 0.6.4 - :released: Tue Sep 07 2010 - - .. change:: - :tags: orm - :tickets: - - The name ConcurrentModificationError has been - changed to StaleDataError, and descriptive - error messages have been revised to reflect - exactly what the issue is. Both names will - remain available for the foreseeable future - for schemes that may be specifying - ConcurrentModificationError in an "except:" - clause. - - .. change:: - :tags: orm - :tickets: 1891 - - Added a mutex to the identity map which mutexes - remove operations against iteration methods, - which now pre-buffer before returning an - iterable. This because asynchronous gc - can remove items via the gc thread at any time. - - .. change:: - :tags: orm - :tickets: - - The Session class is now present in sqlalchemy.orm.*. - We're moving away from the usage of create_session(), - which has non-standard defaults, for those situations - where a one-step Session constructor is desired. Most - users should stick with sessionmaker() for general use, - however. - - .. change:: - :tags: orm - :tickets: - - query.with_parent() now accepts transient objects - and will use the non-persistent values of their pk/fk - attributes in order to formulate the criterion. - Docs are also clarified as to the purpose of with_parent(). - - .. change:: - :tags: orm - :tickets: - - The include_properties and exclude_properties arguments - to mapper() now accept Column objects as members in - addition to strings. This so that same-named Column - objects, such as those within a join(), can be - disambiguated. - - .. change:: - :tags: orm - :tickets: 1896 - - A warning is now emitted if a mapper is created against a - join or other single selectable that includes multiple - columns with the same name in its .c. collection, - and those columns aren't explicitly named as part of - the same or separate attributes (or excluded). - In 0.7 this warning will be an exception. Note that - this warning is not emitted when the combination occurs - as a result of inheritance, so that attributes - still allow being overridden naturally.. In 0.7 this will be improved further. - - .. change:: - :tags: orm - :tickets: 1896 - - The primary_key argument to mapper() can now specify - a series of columns that are only a subset of - the calculated "primary key" columns of the mapped - selectable, without an error being raised. This - helps for situations where a selectable's effective - primary key is simpler than the number of columns - in the selectable that are actually marked as - "primary_key", such as a join against two - tables on their primary key columns. - - .. change:: - :tags: orm - :tickets: - - An object that's been deleted now gets a flag - 'deleted', which prohibits the object from - being re-add()ed to the session, as previously - the object would live in the identity map - silently until its attributes were accessed. - The make_transient() function now resets this - flag along with the "key" flag. - - .. change:: - :tags: orm - :tickets: - - make_transient() can be safely called on an - already transient instance. - - .. change:: - :tags: orm - :tickets: - - a warning is emitted in mapper() if the polymorphic_on - column is not present either in direct or derived - form in the mapped selectable or in the - with_polymorphic selectable, instead of silently - ignoring it. Look for this to become an - exception in 0.7. - - .. change:: - :tags: orm - :tickets: - - Another pass through the series of error messages - emitted when relationship() is configured with - ambiguous arguments. The "foreign_keys" - setting is no longer mentioned, as it is almost - never needed and it is preferable users set up - correct ForeignKey metadata, which is now the - recommendation. If 'foreign_keys' - is used and is incorrect, the message suggests - the attribute is probably unnecessary. Docs - for the attribute are beefed up. This - because all confused relationship() users on the - ML appear to be attempting to use foreign_keys - due to the message, which only confuses them - further since Table metadata is much clearer. - - .. change:: - :tags: orm - :tickets: 1877 - - If the "secondary" table has no ForeignKey metadata - and no foreign_keys is set, even though the - user is passing screwed up information, it is assumed - that primary/secondaryjoin expressions should - consider only and all cols in "secondary" to be - foreign. It's not possible with "secondary" for - the foreign keys to be elsewhere in any case. - A warning is now emitted instead of an error, - and the mapping succeeds. - - .. change:: - :tags: orm - :tickets: 1856 - - Moving an o2m object from one collection to - another, or vice versa changing the referenced - object by an m2o, where the foreign key is also a - member of the primary key, will now be more - carefully checked during flush if the change in - value of the foreign key on the "many" side is the - result of a change in the primary key of the "one" - side, or if the "one" is just a different object. - In one case, a cascade-capable DB would have - cascaded the value already and we need to look at - the "new" PK value to do an UPDATE, in the other we - need to continue looking at the "old". We now look - at the "old", assuming passive_updates=True, - unless we know it was a PK switch that - triggered the change. - - .. change:: - :tags: orm - :tickets: 1857 - - The value of version_id_col can be changed - manually, and this will result in an UPDATE - of the row. Versioned UPDATEs and DELETEs - now use the "committed" value of the - version_id_col in the WHERE clause and - not the pending changed value. The - version generator is also bypassed if - manual changes are present on the attribute. - - .. change:: - :tags: orm - :tickets: - - Repaired the usage of merge() when used with - concrete inheriting mappers. Such mappers frequently - have so-called "concrete" attributes, which are - subclass attributes that "disable" propagation from - the parent - these needed to allow a merge() - operation to pass through without effect. - - .. change:: - :tags: orm - :tickets: 1863 - - Specifying a non-column based argument - for column_mapped_collection, including string, - text() etc., will raise an error message that - specifically asks for a column element, no longer - misleads with incorrect information about - text() or literal(). - - .. change:: - :tags: orm - :tickets: - - Similarly, for relationship(), foreign_keys, - remote_side, order_by - all column-based - expressions are enforced - lists of strings - are explicitly disallowed since this is a - very common error - - .. change:: - :tags: orm - :tickets: 1864 - - Dynamic attributes don't support collection - population - added an assertion for when - set_committed_value() is called, as well as - when joinedload() or subqueryload() options - are applied to a dynamic attribute, instead - of failure / silent failure. - - .. change:: - :tags: orm - :tickets: 1852 - - Fixed bug whereby generating a Query derived - from one which had the same column repeated - with different label names, typically - in some UNION situations, would fail to - propagate the inner columns completely to - the outer query. - - .. change:: - :tags: orm - :tickets: 1881 - - object_session() raises the proper - UnmappedInstanceError when presented with an - unmapped instance. - - .. change:: - :tags: orm - :tickets: - - Applied further memoizations to calculated Mapper - properties, with significant (~90%) runtime mapper.py - call count reduction in heavily polymorphic mapping - configurations. - - .. change:: - :tags: orm - :tickets: - - mapper _get_col_to_prop private method used - by the versioning example is deprecated; - now use mapper.get_property_by_column() which - will remain the public method for this. - - .. change:: - :tags: orm - :tickets: - - the versioning example works correctly now - if versioning on a col that was formerly - NULL. - - .. change:: - :tags: sql - :tickets: - - Calling execute() on an alias() construct is pending - deprecation for 0.7, as it is not itself an - "executable" construct. It currently "proxies" its - inner element and is conditionally "executable" but - this is not the kind of ambiguity we like these days. - - .. change:: - :tags: sql - :tickets: - - The execute() and scalar() methods of ClauseElement - are now moved appropriately to the Executable - subclass. ClauseElement.execute()/ scalar() are still - present and are pending deprecation in 0.7, but note - these would always raise an error anyway if you were - not an Executable (unless you were an alias(), see - previous note). - - .. change:: - :tags: sql - :tickets: - - Added basic math expression coercion for - Numeric->Integer, - so that resulting type is Numeric regardless - of the direction of the expression. - - .. change:: - :tags: sql - :tickets: 1855 - - Changed the scheme used to generate truncated - "auto" index names when using the "index=True" - flag on Column. The truncation only takes - place with the auto-generated name, not one - that is user-defined (an error would be - raised instead), and the truncation scheme - itself is now based on a fragment of an md5 - hash of the identifier name, so that multiple - indexes on columns with similar names still - have unique names. - - .. change:: - :tags: sql - :tickets: 1412 - - The generated index name also is based on - a "max index name length" attribute which is - separate from the "max identifier length" - - this to appease MySQL who has a max length - of 64 for index names, separate from their - overall max length of 255. - - .. change:: - :tags: sql - :tickets: - - the text() construct, if placed in a column - oriented situation, will at least return NULLTYPE - for its type instead of None, allowing it to - be used a little more freely for ad-hoc column - expressions than before. literal_column() - is still the better choice, however. - - .. change:: - :tags: sql - :tickets: - - Added full description of parent table/column, - target table/column in error message raised when - ForeignKey can't resolve target. - - .. change:: - :tags: sql - :tickets: 1865 - - Fixed bug whereby replacing composite foreign key - columns in a reflected table would cause an attempt - to remove the reflected constraint from the table - a second time, raising a KeyError. - - .. change:: - :tags: sql - :tickets: - - the _Label construct, i.e. the one that is produced - whenever you say somecol.label(), now counts itself - in its "proxy_set" unioned with that of its - contained column's proxy set, instead of - directly returning that of the contained column. - This allows column correspondence - operations which depend on the identity of the - _Labels themselves to return the correct result - - .. change:: - :tags: sql - :tickets: 1852 - - fixes ORM bug. - - .. change:: - :tags: engine - :tickets: - - Calling fetchone() or similar on a result that - has already been exhausted, has been closed, - or is not a result-returning result now - raises ResourceClosedError, a subclass of - InvalidRequestError, in all cases, regardless - of backend. Previously, some DBAPIs would - raise ProgrammingError (i.e. pysqlite), others - would return None leading to downstream breakages - (i.e. MySQL-python). - - .. change:: - :tags: engine - :tickets: 1894 - - Fixed bug in Connection whereby if a "disconnect" - event occurred in the "initialize" phase of the - first connection pool connect, an AttributeError - would be raised when the Connection would attempt - to invalidate the DBAPI connection. - - .. change:: - :tags: engine - :tickets: - - Connection, ResultProxy, as well as Session use - ResourceClosedError for all "this - connection/transaction/result is closed" types of - errors. - - .. change:: - :tags: engine - :tickets: - - Connection.invalidate() can be called more than - once and subsequent calls do nothing. - - .. change:: - :tags: declarative - :tickets: - - if @classproperty is used with a regular class-bound - mapper property attribute, it will be called to get the - actual attribute value during initialization. Currently, - there's no advantage to using @classproperty on a column - or relationship attribute of a declarative class that - isn't a mixin - evaluation is at the same time as if - @classproperty weren't used. But here we at least allow - it to function as expected. - - .. change:: - :tags: declarative - :tickets: - - Fixed bug where "Can't add additional column" message - would display the wrong name. - - .. change:: - :tags: postgresql - :tickets: - - Fixed the psycopg2 dialect to use its - set_isolation_level() method instead of relying - upon the base "SET SESSION ISOLATION" command, - as psycopg2 resets the isolation level on each new - transaction otherwise. - - .. change:: - :tags: mssql - :tickets: - - Fixed "default schema" query to work with - pymssql backend. - - .. change:: - :tags: firebird - :tickets: - - Fixed bug whereby a column default would fail to - reflect if the "default" keyword were lower case. - - .. change:: - :tags: oracle - :tickets: 1879 - - Added ROWID type to the Oracle dialect, for those - cases where an explicit CAST might be needed. - - .. change:: - :tags: oracle - :tickets: 1867 - - Oracle reflection of indexes has been tuned so - that indexes which include some or all primary - key columns, but not the same set of columns - as that of the primary key, are reflected. - Indexes which contain the identical columns - as that of the primary key are skipped within - reflection, as the index in that case is assumed - to be the auto-generated primary key index. - Previously, any index with PK columns present - would be skipped. Thanks to Kent Bower - for the patch. - - .. change:: - :tags: oracle - :tickets: 1868 - - Oracle now reflects the names of primary key - constraints - also thanks to Kent Bower. - - .. change:: - :tags: informix - :tickets: 1904 - - Applied patches from to get - basic Informix functionality up again. We - rely upon end-user testing to ensure that - Informix is working to some degree. - - .. change:: - :tags: documentation - :tickets: - - The docs have been reorganized such that the "API - Reference" section is gone - all the docstrings from - there which were public API are moved into the - context of the main doc section that talks about it. - Main docs divided into "SQLAlchemy Core" and - "SQLAlchemy ORM" sections, mapper/relationship docs - have been broken out. Lots of sections rewritten - and/or reorganized. - - .. change:: - :tags: examples - :tickets: - - The beaker_caching example has been reorganized - such that the Session, cache manager, - declarative_base are part of environment, and - custom cache code is portable and now within - "caching_query.py". This allows the example to - be easier to "drop in" to existing projects. - - .. change:: - :tags: examples - :tickets: 1887 - - the history_meta versioning recipe sets "unique=False" - when copying columns, so that the versioning - table handles multiple rows with repeating values. - -.. changelog:: - :version: 0.6.3 - :released: Thu Jul 15 2010 - - .. change:: - :tags: orm - :tickets: 1845 - - Removed errant many-to-many load in unitofwork - which triggered unnecessarily on expired/unloaded - collections. This load now takes place only if - passive_updates is False and the parent primary - key has changed, or if passive_deletes is False - and a delete of the parent has occurred. - - .. change:: - :tags: orm - :tickets: 1853 - - Column-entities (i.e. query(Foo.id)) copy their - state more fully when queries are derived from - themselves + a selectable (i.e. from_self(), - union(), etc.), so that join() and such have the - correct state to work from. - - .. change:: - :tags: orm - :tickets: 1853 - - Fixed bug where Query.join() would fail if - querying a non-ORM column then joining without - an on clause when a FROM clause is already - present, now raises a checked exception the - same way it does when the clause is not - present. - - .. change:: - :tags: orm - :tickets: 1142 - - Improved the check for an "unmapped class", - including the case where the superclass is mapped - but the subclass is not. Any attempts to access - cls._sa_class_manager.mapper now raise - UnmappedClassError(). - - .. change:: - :tags: orm - :tickets: - - Added "column_descriptions" accessor to Query, - returns a list of dictionaries containing - naming/typing information about the entities - the Query will return. Can be helpful for - building GUIs on top of ORM queries. - - .. change:: - :tags: mysql - :tickets: 1848 - - The _extract_error_code() method now works - correctly with each MySQL dialect ( - MySQL-python, OurSQL, MySQL-Connector-Python, - PyODBC). Previously, - the reconnect logic would fail for OperationalError - conditions, however since MySQLdb and OurSQL - have their own reconnect feature, there was no - symptom for these drivers here unless one - watched the logs. - - .. change:: - :tags: oracle - :tickets: 1840 - - More tweaks to cx_oracle Decimal handling. - "Ambiguous" numerics with no decimal place - are coerced to int at the connection handler - level. The advantage here is that ints - come back as ints without SQLA type - objects being involved and without needless - conversion to Decimal first. - - Unfortunately, some exotic subquery cases - can even see different types between - individual result rows, so the Numeric - handler, when instructed to return Decimal, - can't take full advantage of "native decimal" - mode and must run isinstance() on every value - to check if its Decimal already. Reopen of - -.. changelog:: - :version: 0.6.2 - :released: Tue Jul 06 2010 - - .. change:: - :tags: orm - :tickets: - - Query.join() will check for a call of the - form query.join(target, clause_expression), - i.e. missing the tuple, and raise an informative - error message that this is the wrong calling form. - - .. change:: - :tags: orm - :tickets: 1824 - - Fixed bug regarding flushes on self-referential - bi-directional many-to-many relationships, where - two objects made to mutually reference each other - in one flush would fail to insert a row for both - sides. Regression from 0.5. - - .. change:: - :tags: orm - :tickets: - - the post_update feature of relationship() has been - reworked architecturally to integrate more closely - with the new 0.6 unit of work. The motivation - for the change is so that multiple "post update" - calls, each affecting different foreign key - columns of the same row, are executed in a single - UPDATE statement, rather than one UPDATE - statement per column per row. Multiple row - updates are also batched into executemany()s as - possible, while maintaining consistent row ordering. - - .. change:: - :tags: orm - :tickets: - - Query.statement, Query.subquery(), etc. now transfer - the values of bind parameters, i.e. those specified - by query.params(), into the resulting SQL expression. - Previously the values would not be transferred - and bind parameters would come out as None. - - .. change:: - :tags: orm - :tickets: - - Subquery-eager-loading now works with Query objects - which include params(), as well as get() Queries. - - .. change:: - :tags: orm - :tickets: - - Can now call make_transient() on an instance that - is referenced by parent objects via many-to-one, - without the parent's foreign key value getting - temporarily set to None - this was a function - of the "detect primary key switch" flush handler. - It now ignores objects that are no longer - in the "persistent" state, and the parent's - foreign key identifier is left unaffected. - - .. change:: - :tags: orm - :tickets: - - query.order_by() now accepts False, which cancels - any existing order_by() state on the Query, allowing - subsequent generative methods to be called which do - not support ORDER BY. This is not the same as the - already existing feature of passing None, which - suppresses any existing order_by() settings, including - those configured on the mapper. False will make it - as though order_by() was never called, while - None is an active setting. - - .. change:: - :tags: orm - :tickets: - - An instance which is moved to "transient", has - an incomplete or missing set of primary key - attributes, and contains expired attributes, will - raise an InvalidRequestError if an expired attribute - is accessed, instead of getting a recursion overflow. - - .. change:: - :tags: orm - :tickets: - - The make_transient() function is now in the generated - documentation. - - .. change:: - :tags: orm - :tickets: - - make_transient() removes all "loader" callables from - the state being made transient, removing any - "expired" state - all unloaded attributes reset back - to undefined, None/empty on access. - - .. change:: - :tags: sql - :tickets: 1822 - - The warning emitted by the Unicode and String types - with convert_unicode=True no longer embeds the actual - value passed. This so that the Python warning - registry does not continue to grow in size, the warning - is emitted once as per the warning filter settings, - and large string values don't pollute the output. - - .. change:: - :tags: sql - :tickets: - - Fixed bug that would prevent overridden clause - compilation from working for "annotated" expression - elements, which are often generated by the ORM. - - .. change:: - :tags: sql - :tickets: 1400 - - The argument to "ESCAPE" of a LIKE operator or similar - is passed through render_literal_value(), which may - implement escaping of backslashes. - - .. change:: - :tags: sql - :tickets: - - Fixed bug in Enum type which blew away native_enum - flag when used with TypeDecorators or other adaption - scenarios. - - .. change:: - :tags: sql - :tickets: - - Inspector hits bind.connect() when invoked to ensure - initialize has been called. the internal name ".conn" - is changed to ".bind", since that's what it is. - - .. change:: - :tags: sql - :tickets: - - Modified the internals of "column annotation" such that - a custom Column subclass can safely override - _constructor to return Column, for the purposes of - making "configurational" column classes that aren't - involved in proxying, etc. - - .. change:: - :tags: sql - :tickets: 1829 - - Column.copy() takes along the "unique" attribute - among others, fixes regarding declarative - mixins - - .. change:: - :tags: postgresql - :tickets: 1400 - - render_literal_value() is overridden which escapes - backslashes, currently applies to the ESCAPE clause - of LIKE and similar expressions. - Ultimately this will have to detect the value of - "standard_conforming_strings" for full behavior. - - .. change:: - :tags: postgresql - :tickets: 1836 - - Won't generate "CREATE TYPE" / "DROP TYPE" if - using types.Enum on a PG version prior to 8.3 - - the supports_native_enum flag is fully - honored. - - .. change:: - :tags: mysql - :tickets: 1826 - - MySQL dialect doesn't emit CAST() for MySQL version - detected < 4.0.2. This allows the unicode - check on connect to proceed. - - .. change:: - :tags: mysql - :tickets: - - MySQL dialect now detects NO_BACKSLASH_ESCAPES sql - mode, in addition to ANSI_QUOTES. - - .. change:: - :tags: mysql - :tickets: 1400 - - render_literal_value() is overridden which escapes - backslashes, currently applies to the ESCAPE clause - of LIKE and similar expressions. This behavior - is derived from detecting the value of - NO_BACKSLASH_ESCAPES. - - .. change:: - :tags: oracle - :tickets: 1819 - - Fixed ora-8 compatibility flags such that they - don't cache a stale value from before the first - database connection actually occurs. - - .. change:: - :tags: oracle - :tickets: 1840 - - Oracle's "native decimal" metadata begins to return - ambiguous typing information about numerics - when columns are embedded in subqueries as well - as when ROWNUM is consulted with subqueries, as we - do for limit/offset. We've added these ambiguous - conditions to the cx_oracle "convert to Decimal()" - handler, so that we receive numerics as Decimal - in more cases instead of as floats. These are - then converted, if requested, into Integer - or Float, or otherwise kept as the lossless - Decimal. - - .. change:: - :tags: mssql - :tickets: 1825 - - If server_version_info is outside the usual - range of (8, ), (9, ), (10, ), a warning is emitted - which suggests checking that the FreeTDS version - configuration is using 7.0 or 8.0, not 4.2. - - .. change:: - :tags: firebird - :tickets: 1823 - - Fixed incorrect signature in do_execute(), error - introduced in 0.6.1. - - .. change:: - :tags: firebird - :tickets: 1813 - - Firebird dialect adds CHAR, VARCHAR types which - accept a "charset" flag, to support Firebird - "CHARACTER SET" clause. - - .. change:: - :tags: declarative - :tickets: 1805, 1796, 1751 - - Added support for @classproperty to provide - any kind of schema/mapping construct from a - declarative mixin, including columns with foreign - keys, relationships, column_property, deferred. - This solves all such issues on declarative mixins. - An error is raised if any MapperProperty subclass - is specified on a mixin without using @classproperty. - - .. change:: - :tags: declarative - :tickets: 1821 - - a mixin class can now define a column that matches - one which is present on a __table__ defined on a - subclass. It cannot, however, define one that is - not present in the __table__, and the error message - here now works. - - .. change:: - :tags: extension, compiler - :tickets: 1838 - - The 'default' compiler is automatically copied over - when overriding the compilation of a built in - clause construct, so no KeyError is raised if the - user-defined compiler is specific to certain - backends and compilation for a different backend - is invoked. - - .. change:: - :tags: documentation - :tickets: 1820 - - Added documentation for the Inspector. - - .. change:: - :tags: documentation - :tickets: 1830 - - Fixed @memoized_property and @memoized_instancemethod - decorators so that Sphinx documentation picks up - these attributes and methods, such as - ResultProxy.inserted_primary_key. - -.. changelog:: - :version: 0.6.1 - :released: Mon May 31 2010 - - .. change:: - :tags: orm - :tickets: 1782 - - Fixed regression introduced in 0.6.0 involving improper - history accounting on mutable attributes. - - .. change:: - :tags: orm - :tickets: 1807 - - Fixed regression introduced in 0.6.0 unit of work refactor - that broke updates for bi-directional relationship() - with post_update=True. - - .. change:: - :tags: orm - :tickets: 1789 - - session.merge() will not expire attributes on the returned - instance if that instance is "pending". - - .. change:: - :tags: orm - :tickets: 1802 - - fixed __setstate__ method of CollectionAdapter to not - fail during deserialize where parent InstanceState not - yet unserialized. - - .. change:: - :tags: orm - :tickets: 1797 - - Added internal warning in case an instance without a - full PK happened to be expired and then was asked - to refresh. - - .. change:: - :tags: orm - :tickets: - - Added more aggressive caching to the mapper's usage of - UPDATE, INSERT, and DELETE expressions. Assuming the - statement has no per-object SQL expressions attached, - the expression objects are cached by the mapper after - the first create, and their compiled form is stored - persistently in a cache dictionary for the duration of - the related Engine. The cache is an LRUCache for the - rare case that a mapper receives an extremely - high number of different column patterns as UPDATEs. - - .. change:: - :tags: sql - :tickets: 1793 - - expr.in_() now accepts a text() construct as the argument. - Grouping parenthesis are added automatically, i.e. usage - is like `col.in_(text("select id from table"))`. - - .. change:: - :tags: sql - :tickets: - - Columns of _Binary type (i.e. LargeBinary, BLOB, etc.) - will coerce a "basestring" on the right side into a - _Binary as well so that required DBAPI processing - takes place. - - .. change:: - :tags: sql - :tickets: 1801 - - Added table.add_is_dependent_on(othertable), allows manual - placement of dependency rules between two Table objects - for use within create_all(), drop_all(), sorted_tables. - - .. change:: - :tags: sql - :tickets: 1778 - - Fixed bug that prevented implicit RETURNING from functioning - properly with composite primary key that contained zeroes. - - .. change:: - :tags: sql - :tickets: - - Fixed errant space character when generating ADD CONSTRAINT - for a named UNIQUE constraint. - - .. change:: - :tags: sql - :tickets: 1571 - - Fixed "table" argument on constructor of ForeignKeyConstraint - - .. change:: - :tags: sql - :tickets: 1786 - - Fixed bug in connection pool cursor wrapper whereby if a - cursor threw an exception on close(), the logging of the - message would fail. - - .. change:: - :tags: sql - :tickets: - - the _make_proxy() method of ColumnClause and Column now use - self.__class__ to determine the class of object to be returned - instead of hardcoding to ColumnClause/Column, making it slightly - easier to produce specific subclasses of these which work in - alias/subquery situations. - - .. change:: - :tags: sql - :tickets: 1798 - - func.XXX() doesn't inadvertently resolve to non-Function - classes (e.g. fixes func.text()). - - .. change:: - :tags: engines - :tickets: 1781 - - Fixed building the C extensions on Python 2.4. - - .. change:: - :tags: engines - :tickets: - - Pool classes will reuse the same "pool_logging_name" setting - after a dispose() occurs. - - .. change:: - :tags: engines - :tickets: - - Engine gains an "execution_options" argument and - update_execution_options() method, which will apply to - all connections generated by this engine. - - .. change:: - :tags: mysql - :tickets: 1794 - - func.sysdate() emits "SYSDATE()", i.e. with the ending - parenthesis, on MySQL. - - .. change:: - :tags: sqlite - :tickets: 1812 - - Fixed concatenation of constraints when "PRIMARY KEY" - constraint gets moved to column level due to SQLite - AUTOINCREMENT keyword being rendered. - - .. change:: - :tags: oracle - :tickets: 1775 - - Added a check for cx_oracle versions lower than version 5, - in which case the incompatible "output type handler" won't - be used. This will impact decimal accuracy and some - unicode handling issues. - - .. change:: - :tags: oracle - :tickets: 1790 - - Fixed use_ansi=False mode, which was producing broken - WHERE clauses in pretty much all cases. - - .. change:: - :tags: oracle - :tickets: 1808 - - Re-established support for Oracle 8 with cx_oracle, - including that use_ansi is set to False automatically, - NVARCHAR2 and NCLOB are not rendered for Unicode, - "native unicode" check doesn't fail, cx_oracle - "native unicode" mode is disabled, VARCHAR() is emitted - with bytes count instead of char count. - - .. change:: - :tags: oracle - :tickets: 1670 - - oracle_xe 5 doesn't accept a Python unicode object in - its connect string in normal Python 2.x mode - so we coerce - to str() directly. non-ascii characters aren't supported - in connect strings here since we don't know what encoding - we could use. - - .. change:: - :tags: oracle - :tickets: 1815 - - FOR UPDATE is emitted in the syntactically correct position - when limit/offset is used, i.e. the ROWNUM subquery. - However, Oracle can't really handle FOR UPDATE with ORDER BY - or with subqueries, so its still not very usable, but at - least SQLA gets the SQL past the Oracle parser. - - .. change:: - :tags: firebird - :tickets: 1521 - - Added a label to the query used within has_table() and - has_sequence() to work with older versions of Firebird - that don't provide labels for result columns. - - .. change:: - :tags: firebird - :tickets: 1779 - - Added integer coercion to the "type_conv" attribute when - passed via query string, so that it is properly interpreted - by Kinterbasdb. - - .. change:: - :tags: firebird - :tickets: 1646 - - Added 'connection shutdown' to the list of exception strings - which indicate a dropped connection. - - .. change:: - :tags: sqlsoup - :tickets: 1783 - - the SqlSoup constructor accepts a `base` argument which specifies - the base class to use for mapped classes, the default being - `object`. - -.. changelog:: - :version: 0.6.0 - :released: Sun Apr 18 2010 - - .. change:: - :tags: orm - :tickets: 1742, 1081 - - Unit of work internals have been rewritten. Units of work - with large numbers of objects interdependent objects - can now be flushed without recursion overflows - as there is no longer reliance upon recursive calls. The number of internal structures now stays - constant for a particular session state, regardless of - how many relationships are present on mappings. The flow - of events now corresponds to a linear list of steps, - generated by the mappers and relationships based on actual - work to be done, filtered through a single topological sort - for correct ordering. Flush actions are assembled using - far fewer steps and less memory. - - .. change:: - :tags: orm - :tickets: - - Along with the UOW rewrite, this also removes an issue - introduced in 0.6beta3 regarding topological cycle detection - for units of work with long dependency cycles. We now use - an algorithm written by Guido (thanks Guido!). - - .. change:: - :tags: orm - :tickets: 1764 - - one-to-many relationships now maintain a list of positive - parent-child associations within the flush, preventing - previous parents marked as deleted from cascading a - delete or NULL foreign key set on those child objects, - despite the end-user not removing the child from the old - association. - - .. change:: - :tags: orm - :tickets: 1495 - - A collection lazy load will switch off default - eagerloading on the reverse many-to-one side, since - that loading is by definition unnecessary. - - .. change:: - :tags: orm - :tickets: - - Session.refresh() now does an equivalent expire() - on the given instance first, so that the "refresh-expire" - cascade is propagated. Previously, refresh() was - not affected in any way by the presence of "refresh-expire" - cascade. This is a change in behavior versus that - of 0.6beta2, where the "lockmode" flag passed to refresh() - would cause a version check to occur. Since the instance - is first expired, refresh() always upgrades the object - to the most recent version. - - .. change:: - :tags: orm - :tickets: 1754 - - The 'refresh-expire' cascade, when reaching a pending object, - will expunge the object if the cascade also includes - "delete-orphan", or will simply detach it otherwise. - - .. change:: - :tags: orm - :tickets: 1756 - - id(obj) is no longer used internally within topological.py, - as the sorting functions now require hashable objects - only. - - .. change:: - :tags: orm - :tickets: - - The ORM will set the docstring of all generated descriptors - to None by default. This can be overridden using 'doc' - (or if using Sphinx, attribute docstrings work too). - - .. change:: - :tags: orm - :tickets: - - Added kw argument 'doc' to all mapper property callables - as well as Column(). Will assemble the string 'doc' as - the '__doc__' attribute on the descriptor. - - .. change:: - :tags: orm - :tickets: 1761 - - Usage of version_id_col on a backend that supports - cursor.rowcount for execute() but not executemany() now works - when a delete is issued (already worked for saves, since those - don't use executemany()). For a backend that doesn't support - cursor.rowcount at all, a warning is emitted the same - as with saves. - - .. change:: - :tags: orm - :tickets: - - The ORM now short-term caches the "compiled" form of - insert() and update() constructs when flushing lists of - objects of all the same class, thereby avoiding redundant - compilation per individual INSERT/UPDATE within an - individual flush() call. - - .. change:: - :tags: orm - :tickets: - - internal getattr(), setattr(), getcommitted() methods - on ColumnProperty, CompositeProperty, RelationshipProperty - have been underscored (i.e. are private), signature has - changed. - - .. change:: - :tags: engines - :tickets: 1757 - - The C extension now also works with DBAPIs which use custom - sequences as row (and not only tuples). - - .. change:: - :tags: sql - :tickets: 1755 - - Restored some bind-labeling logic from 0.5 which ensures - that tables with column names that overlap another column - of the form "_" won't produce - errors if column._label is used as a bind name during - an UPDATE. Test coverage which wasn't present in 0.5 - has been added. - - .. change:: - :tags: sql - :tickets: 1729 - - somejoin.select(fold_equivalents=True) is no longer - deprecated, and will eventually be rolled into a more - comprehensive version of the feature for. - - .. change:: - :tags: sql - :tickets: 1759 - - the Numeric type raises an *enormous* warning when expected - to convert floats to Decimal from a DBAPI that returns floats. - This includes SQLite, Sybase, MS-SQL. - - .. change:: - :tags: sql - :tickets: - - Fixed an error in expression typing which caused an endless - loop for expressions with two NULL types. - - .. change:: - :tags: sql - :tickets: - - Fixed bug in execution_options() feature whereby the existing - Transaction and other state information from the parent - connection would not be propagated to the sub-connection. - - .. change:: - :tags: sql - :tickets: - - Added new 'compiled_cache' execution option. A dictionary - where Compiled objects will be cached when the Connection - compiles a clause expression into a dialect- and parameter- - specific Compiled object. It is the user's responsibility to - manage the size of this dictionary, which will have keys - corresponding to the dialect, clause element, the column - names within the VALUES or SET clause of an INSERT or UPDATE, - as well as the "batch" mode for an INSERT or UPDATE statement. - - .. change:: - :tags: sql - :tickets: 1769 - - Added get_pk_constraint() to reflection.Inspector, similar - to get_primary_keys() except returns a dict that includes the - name of the constraint, for supported backends (PG so far). - - .. change:: - :tags: sql - :tickets: 1771 - - Table.create() and Table.drop() no longer apply metadata- - level create/drop events. - - .. change:: - :tags: ext - :tickets: - - the compiler extension now allows @compiles decorators - on base classes that extend to child classes, @compiles - decorators on child classes that aren't broken by a - @compiles decorator on the base class. - - .. change:: - :tags: ext - :tickets: - - Declarative will raise an informative error message - if a non-mapped class attribute is referenced in the - string-based relationship() arguments. - - .. change:: - :tags: ext - :tickets: - - Further reworked the "mixin" logic in declarative to - additionally allow __mapper_args__ as a @classproperty - on a mixin, such as to dynamically assign polymorphic_identity. - - .. change:: - :tags: postgresql - :tickets: 1071 - - PostgreSQL now reflects sequence names associated with - SERIAL columns correctly, after the name of the sequence - has been changed. Thanks to Kumar McMillan for the patch. - - .. change:: - :tags: postgresql - :tickets: - - Repaired missing import in psycopg2._PGNumeric type when - unknown numeric is received. - - .. change:: - :tags: postgresql - :tickets: - - psycopg2/pg8000 dialects now aware of REAL[], FLOAT[], - DOUBLE_PRECISION[], NUMERIC[] return types without - raising an exception. - - .. change:: - :tags: postgresql - :tickets: 1769 - - PostgreSQL reflects the name of primary key constraints, - if one exists. - - .. change:: - :tags: oracle - :tickets: - - Now using cx_oracle output converters so that the - DBAPI returns natively the kinds of values we prefer: - - .. change:: - :tags: oracle - :tickets: 1759 - - NUMBER values with positive precision + scale convert - to cx_oracle.STRING and then to Decimal. This - allows perfect precision for the Numeric type when - using cx_oracle. - - .. change:: - :tags: oracle - :tickets: - - STRING/FIXED_CHAR now convert to unicode natively. - SQLAlchemy's String types then don't need to - apply any kind of conversions. - - .. change:: - :tags: firebird - :tickets: - - The functionality of result.rowcount can be disabled on a - per-engine basis by setting 'enable_rowcount=False' - on create_engine(). Normally, cursor.rowcount is called - after any UPDATE or DELETE statement unconditionally, - because the cursor is then closed and Firebird requires - an open cursor in order to get a rowcount. This - call is slightly expensive however so it can be disabled. - To re-enable on a per-execution basis, the - 'enable_rowcount=True' execution option may be used. - - .. change:: - :tags: examples - :tickets: - - Updated attribute_shard.py example to use a more robust - method of searching a Query for binary expressions which - compare columns against literal values. - -.. changelog:: - :version: 0.6beta3 - :released: Sun Mar 28 2010 - - .. change:: - :tags: orm - :tickets: 1675 - - Major feature: Added new "subquery" loading capability to - relationship(). This is an eager loading option which - generates a second SELECT for each collection represented - in a query, across all parents at once. The query - re-issues the original end-user query wrapped in a subquery, - applies joins out to the target collection, and loads - all those collections fully in one result, similar to - "joined" eager loading but using all inner joins and not - re-fetching full parent rows repeatedly (as most DBAPIs seem - to do, even if columns are skipped). Subquery loading is - available at mapper config level using "lazy='subquery'" and - at the query options level using "subqueryload(props..)", - "subqueryload_all(props...)". - - .. change:: - :tags: orm - :tickets: - - To accommodate the fact that there are now two kinds of eager - loading available, the new names for eagerload() and - eagerload_all() are joinedload() and joinedload_all(). The - old names will remain as synonyms for the foreseeable future. - - .. change:: - :tags: orm - :tickets: - - The "lazy" flag on the relationship() function now accepts - a string argument for all kinds of loading: "select", "joined", - "subquery", "noload" and "dynamic", where the default is now - "select". The old values of True/ - False/None still retain their usual meanings and will remain - as synonyms for the foreseeable future. - - .. change:: - :tags: orm - :tickets: 921 - - Added with_hint() method to Query() construct. This calls - directly down to select().with_hint() and also accepts - entities as well as tables and aliases. See with_hint() in the - SQL section below. - - .. change:: - :tags: orm - :tickets: - - Fixed bug in Query whereby calling q.join(prop).from_self(...). - join(prop) would fail to render the second join outside the - subquery, when joining on the same criterion as was on the - inside. - - .. change:: - :tags: orm - :tickets: - - Fixed bug in Query whereby the usage of aliased() constructs - would fail if the underlying table (but not the actual alias) - were referenced inside the subquery generated by - q.from_self() or q.select_from(). - - .. change:: - :tags: orm - :tickets: - - Fixed bug which affected all eagerload() and similar options - such that "remote" eager loads, i.e. eagerloads off of a lazy - load such as query(A).options(eagerload(A.b, B.c)) - wouldn't eagerload anything, but using eagerload("b.c") would - work fine. - - .. change:: - :tags: orm - :tickets: - - Query gains an add_columns(\*columns) method which is a multi- - version of add_column(col). add_column(col) is future - deprecated. - - .. change:: - :tags: orm - :tickets: - - Query.join() will detect if the end result will be - "FROM A JOIN A", and will raise an error if so. - - .. change:: - :tags: orm - :tickets: - - Query.join(Cls.propname, from_joinpoint=True) will check more - carefully that "Cls" is compatible with the current joinpoint, - and act the same way as Query.join("propname", from_joinpoint=True) - in that regard. - - .. change:: - :tags: sql - :tickets: 921 - - Added with_hint() method to select() construct. Specify - a table/alias, hint text, and optional dialect name, and - "hints" will be rendered in the appropriate place in the - statement. Works for Oracle, Sybase, MySQL. - - .. change:: - :tags: sql - :tickets: 1747 - - Fixed bug introduced in 0.6beta2 where column labels would - render inside of column expressions already assigned a label. - - .. change:: - :tags: postgresql - :tickets: 877 - - The psycopg2 dialect will log NOTICE messages via the - "sqlalchemy.dialects.postgresql" logger name. - - .. change:: - :tags: postgresql - :tickets: 997 - - the TIME and TIMESTAMP types are now available from the - postgresql dialect directly, which add the PG-specific - argument 'precision' to both. 'precision' and - 'timezone' are correctly reflected for both TIME and - TIMEZONE types. - - .. change:: - :tags: mysql - :tickets: 1752 - - No longer guessing that TINYINT(1) should be BOOLEAN - when reflecting - TINYINT(1) is returned. Use Boolean/ - BOOLEAN in table definition to get boolean conversion - behavior. - - .. change:: - :tags: oracle - :tickets: 1744 - - The Oracle dialect will issue VARCHAR type definitions - using character counts, i.e. VARCHAR2(50 CHAR), so that - the column is sized in terms of characters and not bytes. - Column reflection of character types will also use - ALL_TAB_COLUMNS.CHAR_LENGTH instead of - ALL_TAB_COLUMNS.DATA_LENGTH. Both of these behaviors take - effect when the server version is 9 or higher - for - version 8, the old behaviors are used. - - .. change:: - :tags: declarative - :tickets: 1746 - - Using a mixin won't break if the mixin implements an - unpredictable __getattribute__(), i.e. Zope interfaces. - - .. change:: - :tags: declarative - :tickets: 1749 - - Using @classdecorator and similar on mixins to define - __tablename__, __table_args__, etc. now works if - the method references attributes on the ultimate - subclass. - - .. change:: - :tags: declarative - :tickets: 1751 - - relationships and columns with foreign keys aren't - allowed on declarative mixins, sorry. - - .. change:: - :tags: ext - :tickets: - - The sqlalchemy.orm.shard module now becomes an extension, - sqlalchemy.ext.horizontal_shard. The old import - works with a deprecation warning. - -.. changelog:: - :version: 0.6beta2 - :released: Sat Mar 20 2010 - - .. change:: - :tags: py3k - :tickets: - - Improved the installation/test setup regarding Python 3, - now that Distribute runs on Py3k. distribute_setup.py - is now included. See README.py3k for Python 3 installation/ - testing instructions. - - .. change:: - :tags: orm - :tickets: 1740 - - The official name for the relation() function is now - relationship(), to eliminate confusion over the relational - algebra term. relation() however will remain available - in equal capacity for the foreseeable future. - - .. change:: - :tags: orm - :tickets: 1692 - - Added "version_id_generator" argument to Mapper, this is a - callable that, given the current value of the "version_id_col", - returns the next version number. Can be used for alternate - versioning schemes such as uuid, timestamps. - - .. change:: - :tags: orm - :tickets: - - added "lockmode" kw argument to Session.refresh(), will - pass through the string value to Query the same as - in with_lockmode(), will also do version check for a - version_id_col-enabled mapping. - - .. change:: - :tags: orm - :tickets: 1188 - - Fixed bug whereby calling query(A).join(A.bs).add_entity(B) - in a joined inheritance scenario would double-add B as a - target and produce an invalid query. - - .. change:: - :tags: orm - :tickets: 1674 - - Fixed bug in session.rollback() which involved not removing - formerly "pending" objects from the session before - re-integrating "deleted" objects, typically occurred with - natural primary keys. If there was a primary key conflict - between them, the attach of the deleted would fail - internally. The formerly "pending" objects are now expunged - first. - - .. change:: - :tags: orm - :tickets: 1719 - - Removed a lot of logging that nobody really cares about, - logging that remains will respond to live changes in the - log level. No significant overhead is added. - - .. change:: - :tags: orm - :tickets: - - Fixed bug in session.merge() which prevented dict-like - collections from merging. - - .. change:: - :tags: orm - :tickets: - - session.merge() works with relations that specifically - don't include "merge" in their cascade options - the target - is ignored completely. - - .. change:: - :tags: orm - :tickets: 1681 - - session.merge() will not expire existing scalar attributes - on an existing target if the target has a value for that - attribute, even if the incoming merged doesn't have - a value for the attribute. This prevents unnecessary loads - on existing items. Will still mark the attr as expired - if the destination doesn't have the attr, though, which - fulfills some contracts of deferred cols. - - .. change:: - :tags: orm - :tickets: 1680 - - The "allow_null_pks" flag is now called "allow_partial_pks", - defaults to True, acts like it did in 0.5 again. Except, - it also is implemented within merge() such that a SELECT - won't be issued for an incoming instance with partially - NULL primary key if the flag is False. - - .. change:: - :tags: orm - :tickets: 1737 - - Fixed bug in 0.6-reworked "many-to-one" optimizations - such that a many-to-one that is against a non-primary key - column on the remote table (i.e. foreign key against a - UNIQUE column) will pull the "old" value in from the - database during a change, since if it's in the session - we will need it for proper history/backref accounting, - and we can't pull from the local identity map on a - non-primary key column. - - .. change:: - :tags: orm - :tickets: 1731 - - fixed internal error which would occur if calling has() - or similar complex expression on a single-table inheritance - relation(). - - .. change:: - :tags: orm - :tickets: 1688 - - query.one() no longer applies LIMIT to the query, this to - ensure that it fully counts all object identities present - in the result, even in the case where joins may conceal - multiple identities for two or more rows. As a bonus, - one() can now also be called with a query that issued - from_statement() to start with since it no longer modifies - the query. - - .. change:: - :tags: orm - :tickets: 1727 - - query.get() now returns None if queried for an identifier - that is present in the identity map with a different class - than the one requested, i.e. when using polymorphic loading. - - .. change:: - :tags: orm - :tickets: 1706 - - A major fix in query.join(), when the "on" clause is an - attribute of an aliased() construct, but there is already - an existing join made out to a compatible target, query properly - joins to the right aliased() construct instead of sticking - onto the right side of the existing join. - - .. change:: - :tags: orm - :tickets: 1362 - - Slight improvement to the fix for to not issue - needless updates of the primary key column during a so-called - "row switch" operation, i.e. add + delete of two objects - with the same PK. - - .. change:: - :tags: orm - :tickets: - - Now uses sqlalchemy.orm.exc.DetachedInstanceError when an - attribute load or refresh action fails due to object - being detached from any Session. UnboundExecutionError - is specific to engines bound to sessions and statements. - - .. change:: - :tags: orm - :tickets: - - Query called in the context of an expression will render - disambiguating labels in all cases. Note that this does - not apply to the existing .statement and .subquery() - accessor/method, which still honors the .with_labels() - setting that defaults to False. - - .. change:: - :tags: orm - :tickets: 1676 - - Query.union() retains disambiguating labels within the - returned statement, thus avoiding various SQL composition - errors which can result from column name conflicts. - - .. change:: - :tags: orm - :tickets: - - Fixed bug in attribute history that inadvertently invoked - __eq__ on mapped instances. - - .. change:: - :tags: orm - :tickets: - - Some internal streamlining of object loading grants a - small speedup for large results, estimates are around - 10-15%. Gave the "state" internals a good solid - cleanup with less complexity, datamembers, - method calls, blank dictionary creates. - - .. change:: - :tags: orm - :tickets: 1689 - - Documentation clarification for query.delete() - - .. change:: - :tags: orm - :tickets: - - Fixed cascade bug in many-to-one relation() when attribute - was set to None, introduced in r6711 (cascade deleted - items into session during add()). - - .. change:: - :tags: orm - :tickets: 1736 - - Calling query.order_by() or query.distinct() before calling - query.select_from(), query.with_polymorphic(), or - query.from_statement() raises an exception now instead of - silently dropping those criterion. - - .. change:: - :tags: orm - :tickets: 1735 - - query.scalar() now raises an exception if more than one - row is returned. All other behavior remains the same. - - .. change:: - :tags: orm - :tickets: 1692 - - Fixed bug which caused "row switch" logic, that is an - INSERT and DELETE replaced by an UPDATE, to fail when - version_id_col was in use. - - .. change:: - :tags: sql - :tickets: 1714 - - join() will now simulate a NATURAL JOIN by default. Meaning, - if the left side is a join, it will attempt to join the right - side to the rightmost side of the left first, and not raise - any exceptions about ambiguous join conditions if successful - even if there are further join targets across the rest of - the left. - - .. change:: - :tags: sql - :tickets: - - The most common result processors conversion function were - moved to the new "processors" module. Dialect authors are - encouraged to use those functions whenever they correspond - to their needs instead of implementing custom ones. - - .. change:: - :tags: sql - :tickets: 1694, 1698 - - SchemaType and subclasses Boolean, Enum are now serializable, - including their ddl listener and other event callables. - - .. change:: - :tags: sql - :tickets: - - Some platforms will now interpret certain literal values - as non-bind parameters, rendered literally into the SQL - statement. This to support strict SQL-92 rules that are - enforced by some platforms including MS-SQL and Sybase. - In this model, bind parameters aren't allowed in the - columns clause of a SELECT, nor are certain ambiguous - expressions like "?=?". When this mode is enabled, the base - compiler will render the binds as inline literals, but only across - strings and numeric values. Other types such as dates - will raise an error, unless the dialect subclass defines - a literal rendering function for those. The bind parameter - must have an embedded literal value already or an error - is raised (i.e. won't work with straight bindparam('x')). - Dialects can also expand upon the areas where binds are not - accepted, such as within argument lists of functions - (which don't work on MS-SQL when native SQL binding is used). - - .. change:: - :tags: sql - :tickets: - - Added "unicode_errors" parameter to String, Unicode, etc. - Behaves like the 'errors' keyword argument to - the standard library's string.decode() functions. This flag - requires that `convert_unicode` is set to `"force"` - otherwise, - SQLAlchemy is not guaranteed to handle the task of unicode - conversion. Note that this flag adds significant performance - overhead to row-fetching operations for backends that already - return unicode objects natively (which most DBAPIs do). This - flag should only be used as an absolute last resort for reading - strings from a column with varied or corrupted encodings, - which only applies to databases that accept invalid encodings - in the first place (i.e. MySQL. *not* PG, Sqlite, etc.) - - .. change:: - :tags: sql - :tickets: - - Added math negation operator support, -x. - - .. change:: - :tags: sql - :tickets: - - FunctionElement subclasses are now directly executable the - same way any func.foo() construct is, with automatic - SELECT being applied when passed to execute(). - - .. change:: - :tags: sql - :tickets: - - The "type" and "bind" keyword arguments of a func.foo() - construct are now local to "func." constructs and are - not part of the FunctionElement base class, allowing - a "type" to be handled in a custom constructor or - class-level variable. - - .. change:: - :tags: sql - :tickets: - - Restored the keys() method to ResultProxy. - - .. change:: - :tags: sql - :tickets: 1647, 1683 - - The type/expression system now does a more complete job - of determining the return type from an expression - as well as the adaptation of the Python operator into - a SQL operator, based on the full left/right/operator - of the given expression. In particular - the date/time/interval system created for PostgreSQL - EXTRACT in has now been generalized into - the type system. The previous behavior which often - occurred of an expression "column + literal" forcing - the type of "literal" to be the same as that of "column" - will now usually not occur - the type of - "literal" is first derived from the Python type of the - literal, assuming standard native Python types + date - types, before falling back to that of the known type - on the other side of the expression. If the - "fallback" type is compatible (i.e. CHAR from String), - the literal side will use that. TypeDecorator - types override this by default to coerce the "literal" - side unconditionally, which can be changed by implementing - the coerce_compared_value() method. Also part of. - - .. change:: - :tags: sql - :tickets: - - Made sqlalchemy.sql.expressions.Executable part of public - API, used for any expression construct that can be sent to - execute(). FunctionElement now inherits Executable so that - it gains execution_options(), which are also propagated - to the select() that's generated within execute(). - Executable in turn subclasses _Generative which marks - any ClauseElement that supports the @_generative - decorator - these may also become "public" for the benefit - of the compiler extension at some point. - - .. change:: - :tags: sql - :tickets: 1579 - - A change to the solution for - an end-user - defined bind parameter name that directly conflicts with - a column-named bind generated directly from the SET or - VALUES clause of an update/insert generates a compile error. - This reduces call counts and eliminates some cases where - undesirable name conflicts could still occur. - - .. change:: - :tags: sql - :tickets: 1705 - - Column() requires a type if it has no foreign keys (this is - not new). An error is now raised if a Column() has no type - and no foreign keys. - - .. change:: - :tags: sql - :tickets: 1717 - - the "scale" argument of the Numeric() type is honored when - coercing a returned floating point value into a string - on its way to Decimal - this allows accuracy to function - on SQLite, MySQL. - - .. change:: - :tags: sql - :tickets: - - the copy() method of Column now copies over uninitialized - "on table attach" events. Helps with the new declarative - "mixin" capability. - - .. change:: - :tags: engines - :tickets: - - Added an optional C extension to speed up the sql layer by - reimplementing RowProxy and the most common result processors. - The actual speedups will depend heavily on your DBAPI and - the mix of datatypes used in your tables, and can vary from - a 30% improvement to more than 200%. It also provides a modest - (~15-20%) indirect improvement to ORM speed for large queries. - Note that it is *not* built/installed by default. - See README for installation instructions. - - .. change:: - :tags: engines - :tickets: - - the execution sequence pulls all rowcount/last inserted ID - info from the cursor before commit() is called on the - DBAPI connection in an "autocommit" scenario. This helps - mxodbc with rowcount and is probably a good idea overall. - - .. change:: - :tags: engines - :tickets: 1719 - - Opened up logging a bit such that isEnabledFor() is called - more often, so that changes to the log level for engine/pool - will be reflected on next connect. This adds a small - amount of method call overhead. It's negligible and will make - life a lot easier for all those situations when logging - just happens to be configured after create_engine() is called. - - .. change:: - :tags: engines - :tickets: - - The assert_unicode flag is deprecated. SQLAlchemy will raise - a warning in all cases where it is asked to encode a non-unicode - Python string, as well as when a Unicode or UnicodeType type - is explicitly passed a bytestring. The String type will do nothing - for DBAPIs that already accept Python unicode objects. - - .. change:: - :tags: engines - :tickets: - - Bind parameters are sent as a tuple instead of a list. Some - backend drivers will not accept bind parameters as a list. - - .. change:: - :tags: engines - :tickets: - - threadlocal engine wasn't properly closing the connection - upon close() - fixed that. - - .. change:: - :tags: engines - :tickets: - - Transaction object doesn't rollback or commit if it isn't - "active", allows more accurate nesting of begin/rollback/commit. - - .. change:: - :tags: engines - :tickets: - - Python unicode objects as binds result in the Unicode type, - not string, thus eliminating a certain class of unicode errors - on drivers that don't support unicode binds. - - .. change:: - :tags: engines - :tickets: 1555 - - Added "logging_name" argument to create_engine(), Pool() constructor - as well as "pool_logging_name" argument to create_engine() which - filters down to that of Pool. Issues the given string name - within the "name" field of logging messages instead of the default - hex identifier string. - - .. change:: - :tags: engines - :tickets: - - The visit_pool() method of Dialect is removed, and replaced with - on_connect(). This method returns a callable which receives - the raw DBAPI connection after each one is created. The callable - is assembled into a first_connect/connect pool listener by the - connection strategy if non-None. Provides a simpler interface - for dialects. - - .. change:: - :tags: engines - :tickets: 1728 - - StaticPool now initializes, disposes and recreates without - opening a new connection - the connection is only opened when - first requested. dispose() also works on AssertionPool now. - - .. change:: - :tags: ticket: 1673, metadata - :tickets: - - Added the ability to strip schema information when using - "tometadata" by passing "schema=None" as an argument. If schema - is not specified then the table's schema is retained. - - .. change:: - :tags: declarative - :tickets: - - DeclarativeMeta exclusively uses cls.__dict__ (not dict\_) - as the source of class information; _as_declarative exclusively - uses the dict\_ passed to it as the source of class information - (which when using DeclarativeMeta is cls.__dict__). This should - in theory make it easier for custom metaclasses to modify - the state passed into _as_declarative. - - .. change:: - :tags: declarative - :tickets: 1707 - - declarative now accepts mixin classes directly, as a means - to provide common functional and column-based elements on - all subclasses, as well as a means to propagate a fixed - set of __table_args__ or __mapper_args__ to subclasses. - For custom combinations of __table_args__/__mapper_args__ from - an inherited mixin to local, descriptors can now be used. - New details are all up in the Declarative documentation. - Thanks to Chris Withers for putting up with my strife - on this. - - .. change:: - :tags: declarative - :tickets: 1393 - - the __mapper_args__ dict is copied when propagating to a subclass, - and is taken straight off the class __dict__ to avoid any - propagation from the parent. mapper inheritance already - propagates the things you want from the parent mapper. - - .. change:: - :tags: declarative - :tickets: 1732 - - An exception is raised when a single-table subclass specifies - a column that is already present on the base class. - - .. change:: - :tags: mysql - :tickets: 1655 - - Fixed reflection bug whereby when COLLATE was present, - nullable flag and server defaults would not be reflected. - - .. change:: - :tags: mysql - :tickets: - - Fixed reflection of TINYINT(1) "boolean" columns defined with - integer flags like UNSIGNED. - - .. change:: - :tags: mysql - :tickets: 1668 - - Further fixes for the mysql-connector dialect. - - .. change:: - :tags: mysql - :tickets: 1496 - - Composite PK table on InnoDB where the "autoincrement" column - isn't first will emit an explicit "KEY" phrase within - CREATE TABLE thereby avoiding errors. - - .. change:: - :tags: mysql - :tickets: 1634 - - Added reflection/create table support for a wide range - of MySQL keywords. - - .. change:: - :tags: mysql - :tickets: 1580 - - Fixed import error which could occur reflecting tables on - a Windows host - - .. change:: - :tags: mssql - :tickets: - - Re-established support for the pymssql dialect. - - .. change:: - :tags: mssql - :tickets: - - Various fixes for implicit returning, reflection, - etc. - the MS-SQL dialects aren't quite complete - in 0.6 yet (but are close) - - .. change:: - :tags: mssql - :tickets: 1710 - - Added basic support for mxODBC. - - .. change:: - :tags: mssql - :tickets: - - Removed the text_as_varchar option. - - .. change:: - :tags: oracle - :tickets: - - "out" parameters require a type that is supported by - cx_oracle. An error will be raised if no cx_oracle - type can be found. - - .. change:: - :tags: oracle - :tickets: - - Oracle 'DATE' now does not perform any result processing, - as the DATE type in Oracle stores full date+time objects, - that's what you'll get. Note that the generic types.Date - type *will* still call value.date() on incoming values, - however. When reflecting a table, the reflected type - will be 'DATE'. - - .. change:: - :tags: oracle - :tickets: 1670 - - Added preliminary support for Oracle's WITH_UNICODE - mode. At the very least this establishes initial - support for cx_Oracle with Python 3. When WITH_UNICODE - mode is used in Python 2.xx, a large and scary warning - is emitted asking that the user seriously consider - the usage of this difficult mode of operation. - - .. change:: - :tags: oracle - :tickets: 1712 - - The except_() method now renders as MINUS on Oracle, - which is more or less equivalent on that platform. - - .. change:: - :tags: oracle - :tickets: 651 - - Added support for rendering and reflecting - TIMESTAMP WITH TIME ZONE, i.e. TIMESTAMP(timezone=True). - - .. change:: - :tags: oracle - :tickets: - - Oracle INTERVAL type can now be reflected. - - .. change:: - :tags: sqlite - :tickets: 1685 - - Added "native_datetime=True" flag to create_engine(). - This will cause the DATE and TIMESTAMP types to skip - all bind parameter and result row processing, under - the assumption that PARSE_DECLTYPES has been enabled - on the connection. Note that this is not entirely - compatible with the "func.current_date()", which - will be returned as a string. - - .. change:: - :tags: sybase - :tickets: - - Implemented a preliminary working dialect for Sybase, - with sub-implementations for Python-Sybase as well - as Pyodbc. Handles table - creates/drops and basic round trip functionality. - Does not yet include reflection or comprehensive - support of unicode/special expressions/etc. - - .. change:: - :tags: examples - :tickets: - - Changed the beaker cache example a bit to have a separate - RelationCache option for lazyload caching. This object - does a lookup among any number of potential attributes - more efficiently by grouping several into a common structure. - Both FromCache and RelationCache are simpler individually. - - .. change:: - :tags: documentation - :tickets: 1700 - - Major cleanup work in the docs to link class, function, and - method names into the API docs. - -.. changelog:: - :version: 0.6beta1 - :released: Wed Feb 03 2010 - - .. change:: - :tags: release, major - :tickets: - - For the full set of feature descriptions, see - https://docs.sqlalchemy.org/en/latest/changelog/migration_06.html . - This document is a work in progress. - - .. change:: - :tags: release, major - :tickets: - - All bug fixes and feature enhancements from the most - recent 0.5 version and below are also included within 0.6. - - .. change:: - :tags: release, major - :tickets: - - Platforms targeted now include Python 2.4/2.5/2.6, Python - 3.1, Jython2.5. - - .. change:: - :tags: orm - :tickets: - - Changes to query.update() and query.delete(): - - the 'expire' option on query.update() has been renamed to - 'fetch', thus matching that of query.delete(). - 'expire' is deprecated and issues a warning. - - - query.update() and query.delete() both default to - 'evaluate' for the synchronize strategy. - - - the 'synchronize' strategy for update() and delete() - raises an error on failure. There is no implicit fallback - onto "fetch". Failure of evaluation is based on the - structure of criteria, so success/failure is deterministic - based on code structure. - - .. change:: - :tags: orm - :tickets: 1186, 1492, 1544 - - Enhancements on many-to-one relations: - - many-to-one relations now fire off a lazyload in fewer - cases, including in most cases will not fetch the "old" - value when a new one is replaced. - - - many-to-one relation to a joined-table subclass now uses - get() for a simple load (known as the "use_get" - condition), i.e. Related->Sub(Base), without the need to - redefine the primaryjoin condition in terms of the base - table. - - - specifying a foreign key with a declarative column, i.e. - ForeignKey(MyRelatedClass.id) doesn't break the "use_get" - condition from taking place - - - relation(), eagerload(), and eagerload_all() now feature - an option called "innerjoin". Specify `True` or `False` to - control whether an eager join is constructed as an INNER - or OUTER join. Default is `False` as always. The mapper - options will override whichever setting is specified on - relation(). Should generally be set for many-to-one, not - nullable foreign key relations to allow improved join - performance. - - - the behavior of eagerloading such that the main query is - wrapped in a subquery when LIMIT/OFFSET are present now - makes an exception for the case when all eager loads are - many-to-one joins. In those cases, the eager joins are - against the parent table directly along with the - limit/offset without the extra overhead of a subquery, - since a many-to-one join does not add rows to the result. - - .. change:: - :tags: orm - :tickets: - - Enhancements / Changes on Session.merge(): - - .. change:: - :tags: orm - :tickets: - - the "dont_load=True" flag on Session.merge() is deprecated - and is now "load=False". - - .. change:: - :tags: orm - :tickets: - - Session.merge() is performance optimized, using half the - call counts for "load=False" mode compared to 0.5 and - significantly fewer SQL queries in the case of collections - for "load=True" mode. - - .. change:: - :tags: orm - :tickets: - - merge() will not issue a needless merge of attributes if the - given instance is the same instance which is already present. - - .. change:: - :tags: orm - :tickets: - - merge() now also merges the "options" associated with a given - state, i.e. those passed through query.options() which follow - along with an instance, such as options to eagerly- or - lazyily- load various attributes. This is essential for - the construction of highly integrated caching schemes. This - is a subtle behavioral change vs. 0.5. - - .. change:: - :tags: orm - :tickets: - - A bug was fixed regarding the serialization of the "loader - path" present on an instance's state, which is also necessary - when combining the usage of merge() with serialized state - and associated options that should be preserved. - - .. change:: - :tags: orm - :tickets: - - The all new merge() is showcased in a new comprehensive - example of how to integrate Beaker with SQLAlchemy. See - the notes in the "examples" note below. - - .. change:: - :tags: orm - :tickets: 1362 - - Primary key values can now be changed on a joined-table inheritance - object, and ON UPDATE CASCADE will be taken into account when - the flush happens. Set the new "passive_updates" flag to False - on mapper() when using SQLite or MySQL/MyISAM. - - .. change:: - :tags: orm - :tickets: 1671 - - flush() now detects when a primary key column was updated by - an ON UPDATE CASCADE operation from another primary key, and - can then locate the row for a subsequent UPDATE on the new PK - value. This occurs when a relation() is there to establish - the relationship as well as passive_updates=True. - - .. change:: - :tags: orm - :tickets: - - the "save-update" cascade will now cascade the pending *removed* - values from a scalar or collection attribute into the new session - during an add() operation. This so that the flush() operation - will also delete or modify rows of those disconnected items. - - .. change:: - :tags: orm - :tickets: 1531 - - Using a "dynamic" loader with a "secondary" table now produces - a query where the "secondary" table is *not* aliased. This - allows the secondary Table object to be used in the "order_by" - attribute of the relation(), and also allows it to be used - in filter criterion against the dynamic relation. - - .. change:: - :tags: orm - :tickets: 1643 - - relation() with uselist=False will emit a warning when - an eager or lazy load locates more than one valid value for - the row. This may be due to primaryjoin/secondaryjoin - conditions which aren't appropriate for an eager LEFT OUTER - JOIN or for other conditions. - - .. change:: - :tags: orm - :tickets: 1633 - - an explicit check occurs when a synonym() is used with - map_column=True, when a ColumnProperty (deferred or otherwise) - exists separately in the properties dictionary sent to mapper - with the same keyname. Instead of silently replacing - the existing property (and possible options on that property), - an error is raised. - - .. change:: - :tags: orm - :tickets: - - a "dynamic" loader sets up its query criterion at construction - time so that the actual query is returned from non-cloning - accessors like "statement". - - .. change:: - :tags: orm - :tickets: - - the "named tuple" objects returned when iterating a - Query() are now pickleable. - - .. change:: - :tags: orm - :tickets: 1542 - - mapping to a select() construct now requires that you - make an alias() out of it distinctly. This to eliminate - confusion over such issues as - - .. change:: - :tags: orm - :tickets: 1537 - - query.join() has been reworked to provide more consistent - behavior and more flexibility (includes) - - .. change:: - :tags: orm - :tickets: - - query.select_from() accepts multiple clauses to produce - multiple comma separated entries within the FROM clause. - Useful when selecting from multiple-homed join() clauses. - - .. change:: - :tags: orm - :tickets: - - query.select_from() also accepts mapped classes, aliased() - constructs, and mappers as arguments. In particular this - helps when querying from multiple joined-table classes to ensure - the full join gets rendered. - - .. change:: - :tags: orm - :tickets: 1135 - - query.get() can be used with a mapping to an outer join - where one or more of the primary key values are None. - - .. change:: - :tags: orm - :tickets: 1568 - - query.from_self(), query.union(), others which do a - "SELECT * from (SELECT...)" type of nesting will do - a better job translating column expressions within the subquery - to the columns clause of the outer query. This is - potentially backwards incompatible with 0.5, in that this - may break queries with literal expressions that do not have labels - applied (i.e. literal('foo'), etc.) - - .. change:: - :tags: orm - :tickets: 1622 - - relation primaryjoin and secondaryjoin now check that they - are column-expressions, not just clause elements. this prohibits - things like FROM expressions being placed there directly. - - .. change:: - :tags: orm - :tickets: 1415 - - `expression.null()` is fully understood the same way - None is when comparing an object/collection-referencing - attribute within query.filter(), filter_by(), etc. - - .. change:: - :tags: orm - :tickets: 1052 - - added "make_transient()" helper function which transforms a - persistent/ detached instance into a transient one (i.e. - deletes the instance_key and removes from any session.) - - .. change:: - :tags: orm - :tickets: 1339 - - the allow_null_pks flag on mapper() is deprecated, and - the feature is turned "on" by default. This means that - a row which has a non-null value for any of its primary key - columns will be considered an identity. The need for this - scenario typically only occurs when mapping to an outer join. - - .. change:: - :tags: orm - :tickets: - - the mechanics of "backref" have been fully merged into the - finer grained "back_populates" system, and take place entirely - within the _generate_backref() method of RelationProperty. This - makes the initialization procedure of RelationProperty - simpler and allows easier propagation of settings (such as from - subclasses of RelationProperty) into the reverse reference. - The internal BackRef() is gone and backref() returns a plain - tuple that is understood by RelationProperty. - - .. change:: - :tags: orm - :tickets: 1569 - - The version_id_col feature on mapper() will raise a warning when - used with dialects that don't support "rowcount" adequately. - - .. change:: - :tags: orm - :tickets: - - added "execution_options()" to Query, to so options can be - passed to the resulting statement. Currently only - Select-statements have these options, and the only option - used is "stream_results", and the only dialect which knows - "stream_results" is psycopg2. - - .. change:: - :tags: orm - :tickets: - - Query.yield_per() will set the "stream_results" statement - option automatically. - - .. change:: - :tags: orm - :tickets: - - Deprecated or removed: - * 'allow_null_pks' flag on mapper() is deprecated. It does - nothing now and the setting is "on" in all cases. - * 'transactional' flag on sessionmaker() and others is - removed. Use 'autocommit=True' to indicate 'transactional=False'. - * 'polymorphic_fetch' argument on mapper() is removed. - Loading can be controlled using the 'with_polymorphic' - option. - * 'select_table' argument on mapper() is removed. Use - 'with_polymorphic=("*", )' for this - functionality. - * 'proxy' argument on synonym() is removed. This flag - did nothing throughout 0.5, as the "proxy generation" - behavior is now automatic. - * Passing a single list of elements to eagerload(), - eagerload_all(), contains_eager(), lazyload(), - defer(), and undefer() instead of multiple positional - \*args is deprecated. - * Passing a single list of elements to query.order_by(), - query.group_by(), query.join(), or query.outerjoin() - instead of multiple positional \*args is deprecated. - * query.iterate_instances() is removed. Use query.instances(). - * Query.query_from_parent() is removed. Use the - sqlalchemy.orm.with_parent() function to produce a - "parent" clause, or alternatively query.with_parent(). - * query._from_self() is removed, use query.from_self() - instead. - * the "comparator" argument to composite() is removed. - Use "comparator_factory". - * RelationProperty._get_join() is removed. - * the 'echo_uow' flag on Session is removed. Use - logging on the "sqlalchemy.orm.unitofwork" name. - * session.clear() is removed. use session.expunge_all(). - * session.save(), session.update(), session.save_or_update() - are removed. Use session.add() and session.add_all(). - * the "objects" flag on session.flush() remains deprecated. - * the "dont_load=True" flag on session.merge() is deprecated - in favor of "load=False". - * ScopedSession.mapper remains deprecated. See the - usage recipe at - https://www.sqlalchemy.org/trac/wiki/UsageRecipes/SessionAwareMapper - * passing an InstanceState (internal SQLAlchemy state object) to - attributes.init_collection() or attributes.get_history() is - deprecated. These functions are public API and normally - expect a regular mapped object instance. - * the 'engine' parameter to declarative_base() is removed. - Use the 'bind' keyword argument. - - .. change:: - :tags: sql - :tickets: - - the "autocommit" flag on select() and text() as well - as select().autocommit() are deprecated - now call - .execution_options(autocommit=True) on either of those - constructs, also available directly on Connection and orm.Query. - - .. change:: - :tags: sql - :tickets: - - the autoincrement flag on column now indicates the column - which should be linked to cursor.lastrowid, if that method - is used. See the API docs for details. - - .. change:: - :tags: sql - :tickets: 1566 - - an executemany() now requires that all bound parameter - sets require that all keys are present which are - present in the first bound parameter set. The structure - and behavior of an insert/update statement is very much - determined by the first parameter set, including which - defaults are going to fire off, and a minimum of - guesswork is performed with all the rest so that performance - is not impacted. For this reason defaults would otherwise - silently "fail" for missing parameters, so this is now guarded - against. - - .. change:: - :tags: sql - :tickets: - - returning() support is native to insert(), update(), - delete(). Implementations of varying levels of - functionality exist for PostgreSQL, Firebird, MSSQL and - Oracle. returning() can be called explicitly with column - expressions which are then returned in the resultset, - usually via fetchone() or first(). - - insert() constructs will also use RETURNING implicitly to - get newly generated primary key values, if the database - version in use supports it (a version number check is - performed). This occurs if no end-user returning() was - specified. - - .. change:: - :tags: sql - :tickets: 1665 - - union(), intersect(), except() and other "compound" types - of statements have more consistent behavior w.r.t. - parenthesizing. Each compound element embedded within - another will now be grouped with parenthesis - previously, - the first compound element in the list would not be grouped, - as SQLite doesn't like a statement to start with - parenthesis. However, PostgreSQL in particular has - precedence rules regarding INTERSECT, and it is - more consistent for parenthesis to be applied equally - to all sub-elements. So now, the workaround for SQLite - is also what the workaround for PG was previously - - when nesting compound elements, the first one usually needs - ".alias().select()" called on it to wrap it inside - of a subquery. - - .. change:: - :tags: sql - :tickets: 1579 - - insert() and update() constructs can now embed bindparam() - objects using names that match the keys of columns. These - bind parameters will circumvent the usual route to those - keys showing up in the VALUES or SET clause of the generated - SQL. - - .. change:: - :tags: sql - :tickets: 1524 - - the Binary type now returns data as a Python string - (or a "bytes" type in Python 3), instead of the built- - in "buffer" type. This allows symmetric round trips - of binary data. - - .. change:: - :tags: sql - :tickets: - - Added a tuple_() construct, allows sets of expressions - to be compared to another set, typically with IN against - composite primary keys or similar. Also accepts an - IN with multiple columns. The "scalar select can - have only one column" error message is removed - will - rely upon the database to report problems with - col mismatch. - - .. change:: - :tags: sql - :tickets: - - User-defined "default" and "onupdate" callables which - accept a context should now call upon - "context.current_parameters" to get at the dictionary - of bind parameters currently being processed. This - dict is available in the same way regardless of - single-execute or executemany-style statement execution. - - .. change:: - :tags: sql - :tickets: 1428 - - multi-part schema names, i.e. with dots such as - "dbo.master", are now rendered in select() labels - with underscores for dots, i.e. "dbo_master_table_column". - This is a "friendly" label that behaves better - in result sets. - - .. change:: - :tags: sql - :tickets: - - removed needless "counter" behavior with select() - labelnames that match a column name in the table, - i.e. generates "tablename_id" for "id", instead of - "tablename_id_1" in an attempt to avoid naming - conflicts, when the table has a column actually - named "tablename_id" - this is because - the labeling logic is always applied to all columns - so a naming conflict will never occur. - - .. change:: - :tags: sql - :tickets: 1628 - - calling expr.in_([]), i.e. with an empty list, emits a warning - before issuing the usual "expr != expr" clause. The - "expr != expr" can be very expensive, and it's preferred - that the user not issue in_() if the list is empty, - instead simply not querying, or modifying the criterion - as appropriate for more complex situations. - - .. change:: - :tags: sql - :tickets: - - Added "execution_options()" to select()/text(), which set the - default options for the Connection. See the note in "engines". - - .. change:: - :tags: sql - :tickets: 1131 - - Deprecated or removed: - * "scalar" flag on select() is removed, use - select.as_scalar(). - * "shortname" attribute on bindparam() is removed. - * postgres_returning, firebird_returning flags on - insert(), update(), delete() are deprecated, use - the new returning() method. - * fold_equivalents flag on join is deprecated (will remain - until is implemented) - - .. change:: - :tags: engines - :tickets: 443 - - transaction isolation level may be specified with - create_engine(... isolation_level="..."); available on - postgresql and sqlite. - - .. change:: - :tags: engines - :tickets: - - Connection has execution_options(), generative method - which accepts keywords that affect how the statement - is executed w.r.t. the DBAPI. Currently supports - "stream_results", causes psycopg2 to use a server - side cursor for that statement, as well as - "autocommit", which is the new location for the "autocommit" - option from select() and text(). select() and - text() also have .execution_options() as well as - ORM Query(). - - .. change:: - :tags: engines - :tickets: 1630 - - fixed the import for entrypoint-driven dialects to - not rely upon silly tb_info trick to determine import - error status. - - .. change:: - :tags: engines - :tickets: - - added first() method to ResultProxy, returns first row and - closes result set immediately. - - .. change:: - :tags: engines - :tickets: - - RowProxy objects are now pickleable, i.e. the object returned - by result.fetchone(), result.fetchall() etc. - - .. change:: - :tags: engines - :tickets: - - RowProxy no longer has a close() method, as the row no longer - maintains a reference to the parent. Call close() on - the parent ResultProxy instead, or use autoclose. - - .. change:: - :tags: engines - :tickets: 1586 - - ResultProxy internals have been overhauled to greatly reduce - method call counts when fetching columns. Can provide a large - speed improvement (up to more than 100%) when fetching large - result sets. The improvement is larger when fetching columns - that have no type-level processing applied and when using - results as tuples (instead of as dictionaries). Many - thanks to Elixir's Gaëtan de Menten for this dramatic - improvement ! - - .. change:: - :tags: engines - :tickets: - - Databases which rely upon postfetch of "last inserted id" - to get at a generated sequence value (i.e. MySQL, MS-SQL) - now work correctly when there is a composite primary key - where the "autoincrement" column is not the first primary - key column in the table. - - .. change:: - :tags: engines - :tickets: - - the last_inserted_ids() method has been renamed to the - descriptor "inserted_primary_key". - - .. change:: - :tags: engines - :tickets: 1554 - - setting echo=False on create_engine() now sets the loglevel - to WARN instead of NOTSET. This so that logging can be - disabled for a particular engine even if logging - for "sqlalchemy.engine" is enabled overall. Note that the - default setting of "echo" is `None`. - - .. change:: - :tags: engines - :tickets: - - ConnectionProxy now has wrapper methods for all transaction - lifecycle events, including begin(), rollback(), commit() - begin_nested(), begin_prepared(), prepare(), release_savepoint(), - etc. - - .. change:: - :tags: engines - :tickets: - - Connection pool logging now uses both INFO and DEBUG - log levels for logging. INFO is for major events such - as invalidated connections, DEBUG for all the acquire/return - logging. `echo_pool` can be False, None, True or "debug" - the same way as `echo` works. - - .. change:: - :tags: engines - :tickets: 1621 - - All pyodbc-dialects now support extra pyodbc-specific - kw arguments 'ansi', 'unicode_results', 'autocommit'. - - .. change:: - :tags: engines - :tickets: - - the "threadlocal" engine has been rewritten and simplified - and now supports SAVEPOINT operations. - - .. change:: - :tags: engines - :tickets: - - deprecated or removed - * result.last_inserted_ids() is deprecated. Use - result.inserted_primary_key - * dialect.get_default_schema_name(connection) is now - public via dialect.default_schema_name. - * the "connection" argument from engine.transaction() and - engine.run_callable() is removed - Connection itself - now has those methods. All four methods accept - \*args and \**kwargs which are passed to the given callable, - as well as the operating connection. - - .. change:: - :tags: schema - :tickets: 1541 - - the `__contains__()` method of `MetaData` now accepts - strings or `Table` objects as arguments. If given - a `Table`, the argument is converted to `table.key` first, - i.e. "[schemaname.]" - - .. change:: - :tags: schema - :tickets: - - deprecated MetaData.connect() and - ThreadLocalMetaData.connect() have been removed - send - the "bind" attribute to bind a metadata. - - .. change:: - :tags: schema - :tickets: - - deprecated metadata.table_iterator() method removed (use - sorted_tables) - - .. change:: - :tags: schema - :tickets: - - deprecated PassiveDefault - use DefaultClause. - - .. change:: - :tags: schema - :tickets: - - the "metadata" argument is removed from DefaultGenerator - and subclasses, but remains locally present on Sequence, - which is a standalone construct in DDL. - - .. change:: - :tags: schema - :tickets: - - Removed public mutability from Index and Constraint - objects: - - * ForeignKeyConstraint.append_element() - * Index.append_column() - * UniqueConstraint.append_column() - * PrimaryKeyConstraint.add() - * PrimaryKeyConstraint.remove() - - These should be constructed declaratively (i.e. in one - construction). - - .. change:: - :tags: schema - :tickets: 1545 - - The "start" and "increment" attributes on Sequence now - generate "START WITH" and "INCREMENT BY" by default, - on Oracle and PostgreSQL. Firebird doesn't support - these keywords right now. - - .. change:: - :tags: schema - :tickets: - - UniqueConstraint, Index, PrimaryKeyConstraint all accept - lists of column names or column objects as arguments. - - .. change:: - :tags: schema - :tickets: - - Other removed things: - - Table.key (no idea what this was for) - - Table.primary_key is not assignable - use - table.append_constraint(PrimaryKeyConstraint(...)) - - Column.bind (get via column.table.bind) - - Column.metadata (get via column.table.metadata) - - Column.sequence (use column.default) - - ForeignKey(constraint=some_parent) (is now private _constraint) - - .. change:: - :tags: schema - :tickets: - - The use_alter flag on ForeignKey is now a shortcut option - for operations that can be hand-constructed using the - DDL() event system. A side effect of this refactor is - that ForeignKeyConstraint objects with use_alter=True - will *not* be emitted on SQLite, which does not support - ALTER for foreign keys. - - .. change:: - :tags: schema - :tickets: 1605 - - ForeignKey and ForeignKeyConstraint objects now correctly - copy() all their public keyword arguments. - - .. change:: - :tags: reflection/inspection - :tickets: - - Table reflection has been expanded and generalized into - a new API called "sqlalchemy.engine.reflection.Inspector". - The Inspector object provides fine-grained information about - a wide variety of schema information, with room for expansion, - including table names, column names, view definitions, sequences, - indexes, etc. - - .. change:: - :tags: reflection/inspection - :tickets: - - Views are now reflectable as ordinary Table objects. The same - Table constructor is used, with the caveat that "effective" - primary and foreign key constraints aren't part of the reflection - results; these have to be specified explicitly if desired. - - .. change:: - :tags: reflection/inspection - :tickets: - - The existing autoload=True system now uses Inspector underneath - so that each dialect need only return "raw" data about tables - and other objects - Inspector is the single place that information - is compiled into Table objects so that consistency is at a maximum. - - .. change:: - :tags: ddl - :tickets: - - the DDL system has been greatly expanded. the DDL() class - now extends the more generic DDLElement(), which forms the basis - of many new constructs: - - - CreateTable() - - DropTable() - - AddConstraint() - - DropConstraint() - - CreateIndex() - - DropIndex() - - CreateSequence() - - DropSequence() - - These support "on" and "execute-at()" just like plain DDL() - does. User-defined DDLElement subclasses can be created and - linked to a compiler using the sqlalchemy.ext.compiler extension. - - .. change:: - :tags: ddl - :tickets: - - The signature of the "on" callable passed to DDL() and - DDLElement() is revised as follows: - - ddl - the DDLElement object itself - event - the string event name. - target - previously "schema_item", the Table or MetaData object triggering the event. - connection - the Connection object in use for the operation. - \**kw - keyword arguments. In the case of MetaData before/after - create/drop, the list of Table objects for which - CREATE/DROP DDL is to be issued is passed as the kw - argument "tables". This is necessary for metadata-level - DDL that is dependent on the presence of specific tables. - - The "schema_item" attribute of DDL has been renamed to - "target". - - .. change:: - :tags: dialect, refactor - :tickets: - - Dialect modules are now broken into database dialects - plus DBAPI implementations. Connect URLs are now - preferred to be specified using dialect+driver://..., - i.e. "mysql+mysqldb://scott:tiger@localhost/test". See - the 0.6 documentation for examples. - - .. change:: - :tags: dialect, refactor - :tickets: - - the setuptools entrypoint for external dialects is now - called "sqlalchemy.dialects". - - .. change:: - :tags: dialect, refactor - :tickets: - - the "owner" keyword argument is removed from Table. Use - "schema" to represent any namespaces to be prepended to - the table name. - - .. change:: - :tags: dialect, refactor - :tickets: - - server_version_info becomes a static attribute. - - .. change:: - :tags: dialect, refactor - :tickets: - - dialects receive an initialize() event on initial - connection to determine connection properties. - - .. change:: - :tags: dialect, refactor - :tickets: - - dialects receive a visit_pool event have an opportunity - to establish pool listeners. - - .. change:: - :tags: dialect, refactor - :tickets: - - cached TypeEngine classes are cached per-dialect class - instead of per-dialect. - - .. change:: - :tags: dialect, refactor - :tickets: - - new UserDefinedType should be used as a base class for - new types, which preserves the 0.5 behavior of - get_col_spec(). - - .. change:: - :tags: dialect, refactor - :tickets: - - The result_processor() method of all type classes now - accepts a second argument "coltype", which is the DBAPI - type argument from cursor.description. This argument - can help some types decide on the most efficient processing - of result values. - - .. change:: - :tags: dialect, refactor - :tickets: - - Deprecated Dialect.get_params() removed. - - .. change:: - :tags: dialect, refactor - :tickets: - - Dialect.get_rowcount() has been renamed to a descriptor - "rowcount", and calls cursor.rowcount directly. Dialects - which need to hardwire a rowcount in for certain calls - should override the method to provide different behavior. - - .. change:: - :tags: dialect, refactor - :tickets: 1566 - - DefaultRunner and subclasses have been removed. The job - of this object has been simplified and moved into - ExecutionContext. Dialects which support sequences should - add a `fire_sequence()` method to their execution context - implementation. - - .. change:: - :tags: dialect, refactor - :tickets: - - Functions and operators generated by the compiler now use - (almost) regular dispatch functions of the form - "visit_" and "visit__fn" to provide - customed processing. This replaces the need to copy the - "functions" and "operators" dictionaries in compiler - subclasses with straightforward visitor methods, and also - allows compiler subclasses complete control over - rendering, as the full _Function or _BinaryExpression - object is passed in. - - .. change:: - :tags: postgresql - :tickets: - - New dialects: pg8000, zxjdbc, and pypostgresql - on py3k. - - .. change:: - :tags: postgresql - :tickets: - - The "postgres" dialect is now named "postgresql" ! - Connection strings look like: - - postgresql://scott:tiger@localhost/test - postgresql+pg8000://scott:tiger@localhost/test - - The "postgres" name remains for backwards compatibility - in the following ways: - - - There is a "postgres.py" dummy dialect which - allows old URLs to work, i.e. - postgres://scott:tiger@localhost/test - - - The "postgres" name can be imported from the old - "databases" module, i.e. "from - sqlalchemy.databases import postgres" as well as - "dialects", "from sqlalchemy.dialects.postgres - import base as pg", will send a deprecation - warning. - - - Special expression arguments are now named - "postgresql_returning" and "postgresql_where", but - the older "postgres_returning" and - "postgres_where" names still work with a - deprecation warning. - - .. change:: - :tags: postgresql - :tickets: - - "postgresql_where" now accepts SQL expressions which - can also include literals, which will be quoted as needed. - - .. change:: - :tags: postgresql - :tickets: - - The psycopg2 dialect now uses psycopg2's "unicode extension" - on all new connections, which allows all String/Text/etc. - types to skip the need to post-process bytestrings into - unicode (an expensive step due to its volume). Other - dialects which return unicode natively (pg8000, zxjdbc) - also skip unicode post-processing. - - .. change:: - :tags: postgresql - :tickets: 1511 - - Added new ENUM type, which exists as a schema-level - construct and extends the generic Enum type. Automatically - associates itself with tables and their parent metadata - to issue the appropriate CREATE TYPE/DROP TYPE - commands as needed, supports unicode labels, supports - reflection. - - .. change:: - :tags: postgresql - :tickets: - - INTERVAL supports an optional "precision" argument - corresponding to the argument that PG accepts. - - .. change:: - :tags: postgresql - :tickets: - - using new dialect.initialize() feature to set up - version-dependent behavior. - - .. change:: - :tags: postgresql - :tickets: 1279 - - somewhat better support for % signs in table/column names; - psycopg2 can't handle a bind parameter name of - %(foobar)s however and SQLA doesn't want to add overhead - just to treat that one non-existent use case. - - .. change:: - :tags: postgresql - :tickets: 1516 - - Inserting NULL into a primary key + foreign key column - will allow the "not null constraint" error to raise, - not an attempt to execute a nonexistent "col_id_seq" - sequence. - - .. change:: - :tags: postgresql - :tickets: - - autoincrement SELECT statements, i.e. those which - select from a procedure that modifies rows, now work - with server-side cursor mode (the named cursor isn't - used for such statements.) - - .. change:: - :tags: postgresql - :tickets: 1636 - - postgresql dialect can properly detect pg "devel" version - strings, i.e. "8.5devel" - - .. change:: - :tags: postgresql - :tickets: 1619 - - The psycopg2 now respects the statement option - "stream_results". This option overrides the connection setting - "server_side_cursors". If true, server side cursors will be - used for the statement. If false, they will not be used, even - if "server_side_cursors" is true on the - connection. - - .. change:: - :tags: mysql - :tickets: - - New dialects: oursql, a new native dialect, - MySQL Connector/Python, a native Python port of MySQLdb, - and of course zxjdbc on Jython. - - .. change:: - :tags: mysql - :tickets: - - VARCHAR/NVARCHAR will not render without a length, raises - an error before passing to MySQL. Doesn't impact - CAST since VARCHAR is not allowed in MySQL CAST anyway, - the dialect renders CHAR/NCHAR in those cases. - - .. change:: - :tags: mysql - :tickets: - - all the _detect_XXX() functions now run once underneath - dialect.initialize() - - .. change:: - :tags: mysql - :tickets: 1279 - - somewhat better support for % signs in table/column names; - MySQLdb can't handle % signs in SQL when executemany() is used, - and SQLA doesn't want to add overhead just to treat that one - non-existent use case. - - .. change:: - :tags: mysql - :tickets: - - the BINARY and MSBinary types now generate "BINARY" in all - cases. Omitting the "length" parameter will generate - "BINARY" with no length. Use BLOB to generate an unlengthed - binary column. - - .. change:: - :tags: mysql - :tickets: - - the "quoting='quoted'" argument to MSEnum/ENUM is deprecated. - It's best to rely upon the automatic quoting. - - .. change:: - :tags: mysql - :tickets: - - ENUM now subclasses the new generic Enum type, and also handles - unicode values implicitly, if the given labelnames are unicode - objects. - - .. change:: - :tags: mysql - :tickets: 1539 - - a column of type TIMESTAMP now defaults to NULL if - "nullable=False" is not passed to Column(), and no default - is present. This is now consistent with all other types, - and in the case of TIMESTAMP explicitly renders "NULL" - due to MySQL's "switching" of default nullability - for TIMESTAMP columns. - - .. change:: - :tags: oracle - :tickets: - - unit tests pass 100% with cx_oracle ! - - .. change:: - :tags: oracle - :tickets: - - support for cx_Oracle's "native unicode" mode which does - not require NLS_LANG to be set. Use the latest 5.0.2 or - later of cx_oracle. - - .. change:: - :tags: oracle - :tickets: - - an NCLOB type is added to the base types. - - .. change:: - :tags: oracle - :tickets: - - use_ansi=False won't leak into the FROM/WHERE clause of - a statement that's selecting from a subquery that also - uses JOIN/OUTERJOIN. - - .. change:: - :tags: oracle - :tickets: 1467 - - added native INTERVAL type to the dialect. This supports - only the DAY TO SECOND interval type so far due to lack - of support in cx_oracle for YEAR TO MONTH. - - .. change:: - :tags: oracle - :tickets: - - usage of the CHAR type results in cx_oracle's - FIXED_CHAR dbapi type being bound to statements. - - .. change:: - :tags: oracle - :tickets: 885 - - the Oracle dialect now features NUMBER which intends - to act justlike Oracle's NUMBER type. It is the primary - numeric type returned by table reflection and attempts - to return Decimal()/float/int based on the precision/scale - parameters. - - .. change:: - :tags: oracle - :tickets: - - func.char_length is a generic function for LENGTH - - .. change:: - :tags: oracle - :tickets: - - ForeignKey() which includes onupdate= will emit a - warning, not emit ON UPDATE CASCADE which is unsupported - by oracle - - .. change:: - :tags: oracle - :tickets: - - the keys() method of RowProxy() now returns the result - column names *normalized* to be SQLAlchemy case - insensitive names. This means they will be lower case for - case insensitive names, whereas the DBAPI would normally - return them as UPPERCASE names. This allows row keys() to - be compatible with further SQLAlchemy operations. - - .. change:: - :tags: oracle - :tickets: - - using new dialect.initialize() feature to set up - version-dependent behavior. - - .. change:: - :tags: oracle - :tickets: 1125 - - using types.BigInteger with Oracle will generate - NUMBER(19) - - .. change:: - :tags: oracle - :tickets: - - "case sensitivity" feature will detect an all-lowercase - case-sensitive column name during reflect and add - "quote=True" to the generated Column, so that proper - quoting is maintained. - - .. change:: - :tags: firebird - :tickets: - - the keys() method of RowProxy() now returns the result - column names *normalized* to be SQLAlchemy case - insensitive names. This means they will be lower case for - case insensitive names, whereas the DBAPI would normally - return them as UPPERCASE names. This allows row keys() to - be compatible with further SQLAlchemy operations. - - .. change:: - :tags: firebird - :tickets: - - using new dialect.initialize() feature to set up - version-dependent behavior. - - .. change:: - :tags: firebird - :tickets: - - "case sensitivity" feature will detect an all-lowercase - case-sensitive column name during reflect and add - "quote=True" to the generated Column, so that proper - quoting is maintained. - - .. change:: - :tags: mssql - :tickets: - - MSSQL + Pyodbc + FreeTDS now works for the most part, - with possible exceptions regarding binary data as well as - unicode schema identifiers. - - .. change:: - :tags: mssql - :tickets: - - the "has_window_funcs" flag is removed. LIMIT/OFFSET - usage will use ROW NUMBER as always, and if on an older - version of SQL Server, the operation fails. The behavior - is exactly the same except the error is raised by SQL - server instead of the dialect, and no flag setting is - required to enable it. - - .. change:: - :tags: mssql - :tickets: - - the "auto_identity_insert" flag is removed. This feature - always takes effect when an INSERT statement overrides a - column that is known to have a sequence on it. As with - "has_window_funcs", if the underlying driver doesn't - support this, then you can't do this operation in any - case, so there's no point in having a flag. - - .. change:: - :tags: mssql - :tickets: - - using new dialect.initialize() feature to set up - version-dependent behavior. - - .. change:: - :tags: mssql - :tickets: - - removed references to sequence which is no longer used. - implicit identities in mssql work the same as implicit - sequences on any other dialects. Explicit sequences are - enabled through the use of "default=Sequence()". See - the MSSQL dialect documentation for more information. - - .. change:: - :tags: sqlite - :tickets: - - DATE, TIME and DATETIME types can now take optional storage_format - and regexp argument. storage_format can be used to store those types - using a custom string format. regexp allows to use a custom regular - expression to match string values from the database. - - .. change:: - :tags: sqlite - :tickets: - - Time and DateTime types now use by a default a stricter regular - expression to match strings from the database. Use the regexp - argument if you are using data stored in a legacy format. - - .. change:: - :tags: sqlite - :tickets: - - __legacy_microseconds__ on SQLite Time and DateTime types is not - supported anymore. You should use the storage_format argument - instead. - - .. change:: - :tags: sqlite - :tickets: - - Date, Time and DateTime types are now stricter in what they accept as - bind parameters: Date type only accepts date objects (and datetime - ones, because they inherit from date), Time only accepts time - objects, and DateTime only accepts date and datetime objects. - - .. change:: - :tags: sqlite - :tickets: 1016 - - Table() supports a keyword argument "sqlite_autoincrement", which - applies the SQLite keyword "AUTOINCREMENT" to the single integer - primary key column when generating DDL. Will prevent generation of - a separate PRIMARY KEY constraint. - - .. change:: - :tags: types - :tickets: - - The construction of types within dialects has been totally - overhauled. Dialects now define publicly available types - as UPPERCASE names exclusively, and internal implementation - types using underscore identifiers (i.e. are private). - The system by which types are expressed in SQL and DDL - has been moved to the compiler system. This has the - effect that there are much fewer type objects within - most dialects. A detailed document on this architecture - for dialect authors is in - lib/sqlalchemy/dialects/type_migration_guidelines.txt . - - .. change:: - :tags: types - :tickets: - - Types no longer make any guesses as to default - parameters. In particular, Numeric, Float, NUMERIC, - FLOAT, DECIMAL don't generate any length or scale unless - specified. - - .. change:: - :tags: types - :tickets: 1664 - - types.Binary is renamed to types.LargeBinary, it only - produces BLOB, BYTEA, or a similar "long binary" type. - New base BINARY and VARBINARY - types have been added to access these MySQL/MS-SQL specific - types in an agnostic way. - - .. change:: - :tags: types - :tickets: - - String/Text/Unicode types now skip the unicode() check - on each result column value if the dialect has - detected the DBAPI as returning Python unicode objects - natively. This check is issued on first connect - using "SELECT CAST 'some text' AS VARCHAR(10)" or - equivalent, then checking if the returned object - is a Python unicode. This allows vast performance - increases for native-unicode DBAPIs, including - pysqlite/sqlite3, psycopg2, and pg8000. - - .. change:: - :tags: types - :tickets: - - Most types result processors have been checked for possible speed - improvements. Specifically, the following generic types have been - optimized, resulting in varying speed improvements: - Unicode, PickleType, Interval, TypeDecorator, Binary. - Also the following dbapi-specific implementations have been improved: - Time, Date and DateTime on Sqlite, ARRAY on PostgreSQL, - Time on MySQL, Numeric(as_decimal=False) on MySQL, oursql and - pypostgresql, DateTime on cx_oracle and LOB-based types on cx_oracle. - - .. change:: - :tags: types - :tickets: - - Reflection of types now returns the exact UPPERCASE - type within types.py, or the UPPERCASE type within - the dialect itself if the type is not a standard SQL - type. This means reflection now returns more accurate - information about reflected types. - - .. change:: - :tags: types - :tickets: 1511, 1109 - - Added a new Enum generic type. Enum is a schema-aware object - to support databases which require specific DDL in order to - use enum or equivalent; in the case of PG it handles the - details of `CREATE TYPE`, and on other databases without - native enum support will by generate VARCHAR + an inline CHECK - constraint to enforce the enum. - - .. change:: - :tags: types - :tickets: 1467 - - The Interval type includes a "native" flag which controls - if native INTERVAL types (postgresql + oracle) are selected - if available, or not. "day_precision" and "second_precision" - arguments are also added which propagate as appropriately - to these native types. Related to. - - .. change:: - :tags: types - :tickets: 1589 - - The Boolean type, when used on a backend that doesn't - have native boolean support, will generate a CHECK - constraint "col IN (0, 1)" along with the int/smallint- - based column type. This can be switched off if - desired with create_constraint=False. - Note that MySQL has no native boolean *or* CHECK constraint - support so this feature isn't available on that platform. - - .. change:: - :tags: types - :tickets: - - PickleType now uses == for comparison of values when - mutable=True, unless the "comparator" argument with a - comparison function is specified to the type. Objects - being pickled will be compared based on identity (which - defeats the purpose of mutable=True) if __eq__() is not - overridden or a comparison function is not provided. - - .. change:: - :tags: types - :tickets: - - The default "precision" and "scale" arguments of Numeric - and Float have been removed and now default to None. - NUMERIC and FLOAT will be rendered with no numeric - arguments by default unless these values are provided. - - .. change:: - :tags: types - :tickets: - - AbstractType.get_search_list() is removed - the games - that was used for are no longer necessary. - - .. change:: - :tags: types - :tickets: 1125 - - Added a generic BigInteger type, compiles to - BIGINT or NUMBER(19). - - .. change:: - :tags: types - :tickets: - - sqlsoup has been overhauled to explicitly support an 0.5 style - session, using autocommit=False, autoflush=True. Default - behavior of SQLSoup now requires the usual usage of commit() - and rollback(), which have been added to its interface. An - explicit Session or scoped_session can be passed to the - constructor, allowing these arguments to be overridden. - - .. change:: - :tags: types - :tickets: - - sqlsoup db..update() and delete() now call - query(cls).update() and delete(), respectively. - - .. change:: - :tags: types - :tickets: - - sqlsoup now has execute() and connection(), which call upon - the Session methods of those names, ensuring that the bind is - in terms of the SqlSoup object's bind. - - .. change:: - :tags: types - :tickets: - - sqlsoup objects no longer have the 'query' attribute - it's - not needed for sqlsoup's usage paradigm and it gets in the - way of a column that is actually named 'query'. - - .. change:: - :tags: types - :tickets: 1259 - - The signature of the proxy_factory callable passed to - association_proxy is now (lazy_collection, creator, - value_attr, association_proxy), adding a fourth argument - that is the parent AssociationProxy argument. Allows - serializability and subclassing of the built in collections. - - .. change:: - :tags: types - :tickets: 1372 - - association_proxy now has basic comparator methods .any(), - .has(), .contains(), ==, !=, thanks to Scott Torborg. diff --git a/doc/build/changelog/changelog_07.rst b/doc/build/changelog/changelog_07.rst deleted file mode 100644 index 300985f021..0000000000 --- a/doc/build/changelog/changelog_07.rst +++ /dev/null @@ -1,4699 +0,0 @@ -============= -0.7 Changelog -============= - - -.. changelog:: - :version: 0.7.11 - :released: - - .. change:: - :tags: bug, engine - :tickets: 2851 - :versions: 0.8.3, 0.9.0b1 - - The regexp used by the :func:`~sqlalchemy.engine.url.make_url` function now parses - ipv6 addresses, e.g. surrounded by brackets. - - .. change:: - :tags: bug, orm - :tickets: 2807 - :versions: 0.8.3, 0.9.0b1 - - Fixed bug where list instrumentation would fail to represent a - setslice of ``[0:0]`` correctly, which in particular could occur - when using ``insert(0, item)`` with the association proxy. Due - to some quirk in Python collections, the issue was much more likely - with Python 3 rather than 2. - - .. change:: - :tags: bug, sql - :tickets: 2801 - :versions: 0.8.3, 0.9.0b1 - - Fixed regression dating back to 0.7.9 whereby the name of a CTE might - not be properly quoted if it was referred to in multiple FROM clauses. - - .. change:: - :tags: mysql, bug - :tickets: 2791 - :versions: 0.8.3, 0.9.0b1 - - Updates to MySQL reserved words for versions 5.5, 5.6, courtesy - Hanno Schlichting. - - .. change:: - :tags: sql, bug, cte - :tickets: 2783 - :versions: 0.8.3, 0.9.0b1 - - Fixed bug in common table expression system where if the CTE were - used only as an ``alias()`` construct, it would not render using the - WITH keyword. - - .. change:: - :tags: bug, sql - :tickets: 2784 - :versions: 0.8.3, 0.9.0b1 - - Fixed bug in :class:`.CheckConstraint` DDL where the "quote" flag from a - :class:`_schema.Column` object would not be propagated. - - .. change:: - :tags: bug, orm - :tickets: 2699 - :versions: 0.8.1 - - Fixed bug when a query of the form: - ``query(SubClass).options(subqueryload(Baseclass.attrname))``, - where ``SubClass`` is a joined inh of ``BaseClass``, - would fail to apply the ``JOIN`` inside the subquery - on the attribute load, producing a cartesian product. - The populated results still tended to be correct as additional - rows are just ignored, so this issue may be present as a - performance degradation in applications that are - otherwise working correctly. - - .. change:: - :tags: bug, orm - :tickets: 2689 - :versions: 0.8.1 - - Fixed bug in unit of work whereby a joined-inheritance - subclass could insert the row for the "sub" table - before the parent table, if the two tables had no - ForeignKey constraints set up between them. - - .. change:: - :tags: feature, postgresql - :tickets: 2676 - :versions: 0.8.0 - - Added support for PostgreSQL's traditional SUBSTRING - function syntax, renders as "SUBSTRING(x FROM y FOR z)" - when regular ``func.substring()`` is used. - Courtesy Gunnlaugur Þór Briem. - - .. change:: - :tags: bug, tests - :tickets: 2669 - :pullreq: 41 - - Fixed an import of "logging" in test_execute which was not - working on some linux platforms. - - .. change:: - :tags: bug, orm - :tickets: 2674 - - Improved the error message emitted when a "backref loop" is detected, - that is when an attribute event triggers a bidirectional - assignment between two other attributes with no end. - This condition can occur not just when an object of the wrong - type is assigned, but also when an attribute is mis-configured - to backref into an existing backref pair. - - .. change:: - :tags: bug, orm - :tickets: 2674 - - A warning is emitted when a MapperProperty is assigned to a mapper - that replaces an existing property, if the properties in question - aren't plain column-based properties. Replacement of relationship - properties is rarely (ever?) what is intended and usually refers to a - mapper mis-configuration. This will also warn if a backref configures - itself on top of an existing one in an inheritance relationship - (which is an error in 0.8). - -.. changelog:: - :version: 0.7.10 - :released: Thu Feb 7 2013 - - .. change:: - :tags: engine, bug - :tickets: 2604 - :versions: 0.8.0b2 - - Fixed :meth:`_schema.MetaData.reflect` to correctly use - the given :class:`_engine.Connection`, if given, without - opening a second connection from that connection's - :class:`_engine.Engine`. - - .. change:: - :tags: mssql, bug - :tickets:2607 - :versions: 0.8.0b2 - - Fixed bug whereby using "key" with Column - in conjunction with "schema" for the owning - Table would fail to locate result rows due - to the MSSQL dialect's "schema rendering" - logic's failure to take .key into account. - - .. change:: - :tags: sql, mysql, gae - :tickets: 2649 - - Added a conditional import to the ``gaerdbms`` dialect which attempts - to import rdbms_apiproxy vs. rdbms_googleapi to work - on both dev and production platforms. Also now honors the - ``instance`` attribute. Courtesy Sean Lynch. Also backported - enhancements to allow username/password as well as - fixing error code interpretation from 0.8. - - .. change:: - :tags: sql, bug - :tickets: 2594, 2584 - - Backported adjustment to ``__repr__`` for - :class:`.TypeDecorator` to 0.7, allows :class:`.PickleType` - to produce a clean ``repr()`` to help with Alembic. - - .. change:: - :tags: sql, bug - :tickets: 2643 - - Fixed bug where :meth:`_schema.Table.tometadata` would fail if a - :class:`_schema.Column` had both a foreign key as well as an - alternate ".key" name for the column. - - .. change:: - :tags: mssql, bug - :tickets: 2638 - - Added a Py3K conditional around unnecessary .decode() - call in mssql information schema, fixes reflection - in Py3k. - - .. change:: - :tags: orm, bug - :tickets: 2650 - - Fixed potential memory leak which could occur if an - arbitrary number of :class:`.sessionmaker` objects - were created. The anonymous subclass created by - the sessionmaker, when dereferenced, would not be garbage - collected due to remaining class-level references from the - event package. This issue also applies to any custom system - that made use of ad-hoc subclasses in conjunction with - an event dispatcher. - - .. change:: - :tags: orm, bug - :tickets: 2640 - - :meth:`_query.Query.merge_result` can now load rows from an outer join - where an entity may be ``None`` without throwing an error. - - .. change:: - :tags: sqlite, bug - :tickets: 2568 - :versions: 0.8.0b2 - - More adjustment to this SQLite related issue which was released in - 0.7.9, to intercept legacy SQLite quoting characters when reflecting - foreign keys. In addition to intercepting double quotes, other - quoting characters such as brackets, backticks, and single quotes - are now also intercepted. - - .. change:: - :tags: sql, bug - :tickets: 2631 - :versions: 0.8.0b2 - - Fixed bug where using server_onupdate= - without passing the "for_update=True" flag would apply the default - object to the server_default, blowing away whatever was there. - The explicit for_update=True argument shouldn't be needed with this usage - (especially since the documentation shows an example without it being - used) so it is now arranged internally using a copy of the given default - object, if the flag isn't set to what corresponds to that argument. - - .. change:: - :tags: oracle, bug - :tickets: 2620 - - The Oracle LONG type, while an unbounded text type, does not appear - to use the cx_Oracle.LOB type when result rows are returned, - so the dialect has been repaired to exclude LONG from - having cx_Oracle.LOB filtering applied. - - .. change:: - :tags: oracle, bug - :tickets: 2611 - - Repaired the usage of ``.prepare()`` in conjunction with - cx_Oracle so that a return value of ``False`` will result - in no call to ``connection.commit()``, hence avoiding - "no transaction" errors. Two-phase transactions have - now been shown to work in a rudimental fashion with - SQLAlchemy and cx_oracle, however are subject to caveats - observed with the driver; check the documentation - for details. - - .. change:: - :tags: orm, bug - :tickets: 2624 - - The :class:`.MutableComposite` type did not allow for the - :meth:`.MutableBase.coerce` method to be used, even though - the code seemed to indicate this intent, so this now works - and a brief example is added. As a side-effect, - the mechanics of this event handler have been changed so that - new :class:`.MutableComposite` types no longer add per-type - global event handlers. Also in 0.8.0b2. - - .. change:: - :tags: orm, bug - :tickets: 2583 - - Fixed Session accounting bug whereby replacing - a deleted object in the identity map with another - object of the same primary key would raise a - "conflicting state" error on rollback(), - if the replaced primary key were established either - via non-unitofwork-established INSERT statement - or by primary key switch of another instance. - - .. change:: - :tags: oracle, bug - :tickets: 2561 - - changed the list of cx_oracle types that are - excluded from the setinputsizes() step to only include - STRING and UNICODE; CLOB and NCLOB are removed. This - is to work around cx_oracle behavior which is broken - for the executemany() call. In 0.8, this same change - is applied however it is also configurable via the - exclude_setinputsizes argument. - - .. change:: - :tags: feature, mysql - :tickets: 2523 - - Added "raise_on_warnings" flag to OurSQL - dialect. - - .. change:: - :tags: feature, mysql - :tickets: 2554 - - Added "read_timeout" flag to MySQLdb - dialect. - -.. changelog:: - :version: 0.7.9 - :released: Mon Oct 01 2012 - - .. change:: - :tags: orm, bug - :tickets: - - Fixed bug mostly local to new - AbstractConcreteBase helper where the "type" - attribute from the superclass would not - be overridden on the subclass to produce the - "reserved for base" error message, instead placing - a do-nothing attribute there. This was inconsistent - vs. using ConcreteBase as well as all the behavior - of classical concrete mappings, where the "type" - column from the polymorphic base would be explicitly - disabled on subclasses, unless overridden - explicitly. - - .. change:: - :tags: orm, bug - :tickets: - - A warning is emitted when lazy='dynamic' - is combined with uselist=False. This is an - exception raise in 0.8. - - .. change:: - :tags: orm, bug - :tickets: - - Fixed bug whereby user error in related-object - assignment could cause recursion overflow if the - assignment triggered a backref of the same name - as a bi-directional attribute on the incorrect - class to the same target. An informative - error is raised now. - - .. change:: - :tags: orm, bug - :tickets: 2539 - - Fixed bug where incorrect type information - would be passed when the ORM would bind the - "version" column, when using the "version" feature. - Tests courtesy Daniel Miller. - - .. change:: - :tags: orm, bug - :tickets: 2566 - - Extra logic has been added to the "flush" - that occurs within Session.commit(), such that the - extra state added by an after_flush() or - after_flush_postexec() hook is also flushed in a - subsequent flush, before the "commit" completes. - Subsequent calls to flush() will continue until - the after_flush hooks stop adding new state. - An "overflow" counter of 100 is also in place, - in the event of a broken after_flush() hook - adding new content each time. - - .. change:: - :tags: bug, sql - :tickets: 2571 - - Fixed the DropIndex construct to support - an Index associated with a Table in a remote - schema. - - .. change:: - :tags: bug, sql - :tickets: 2574 - - Fixed bug in over() construct whereby - passing an empty list for either partition_by - or order_by, as opposed to None, would fail - to generate correctly. - Courtesy Gunnlaugur Þór Briem. - - .. change:: - :tags: bug, sql - :tickets: 2521 - - Fixed CTE bug whereby positional - bound parameters present in the CTEs themselves - would corrupt the overall ordering of - bound parameters. This primarily - affected SQL Server as the platform with - positional binds + CTE support. - - .. change:: - :tags: bug, sql - :tickets: - - Fixed more un-intuitivenesses in CTEs - which prevented referring to a CTE in a union - of itself without it being aliased. - CTEs now render uniquely - on name, rendering the outermost CTE of a given - name only - all other references are rendered - just as the name. This even includes other - CTE/SELECTs that refer to different versions - of the same CTE object, such as a SELECT - or a UNION ALL of that SELECT. We are - somewhat loosening the usual link between object - identity and lexical identity in this case. - A true name conflict between two unrelated - CTEs now raises an error. - - .. change:: - :tags: bug, sql - :tickets: 2512 - - quoting is applied to the column names - inside the WITH RECURSIVE clause of a - common table expression according to the - quoting rules for the originating Column. - - .. change:: - :tags: bug, sql - :tickets: 2518 - - Fixed regression introduced in 0.7.6 - whereby the FROM list of a SELECT statement - could be incorrect in certain "clone+replace" - scenarios. - - .. change:: - :tags: bug, sql - :tickets: 2552 - - Fixed bug whereby usage of a UNION - or similar inside of an embedded subquery - would interfere with result-column targeting, - in the case that a result-column had the same - ultimate name as a name inside the embedded - UNION. - - .. change:: - :tags: bug, sql - :tickets: 2558 - - Fixed a regression since 0.6 regarding - result-row targeting. It should be possible - to use a select() statement with string - based columns in it, that is - select(['id', 'name']).select_from('mytable'), - and have this statement be targetable by - Column objects with those names; this is the - mechanism by which - query(MyClass).from_statement(some_statement) - works. At some point the specific case of - using select(['id']), which is equivalent to - select([literal_column('id')]), stopped working - here, so this has been re-instated and of - course tested. - - .. change:: - :tags: bug, sql - :tickets: 2544 - - Added missing operators is_(), isnot() - to the ColumnOperators base, so that these long-available - operators are present as methods like all - the other operators. - - .. change:: - :tags: engine, bug - :tickets: 2522 - - Fixed bug whereby - a disconnect detect + dispose that occurs - when the QueuePool has threads waiting - for connections would leave those - threads waiting for the duration of - the timeout on the old pool (or indefinitely - if timeout was disabled). The fix - now notifies those waiters with a special - exception case and has them move onto - the new pool. - - .. change:: - :tags: engine, feature - :tickets: 2516 - - Dramatic improvement in memory - usage of the event system; instance-level - collections are no longer created for a - particular type of event until - instance-level listeners are established - for that event. - - .. change:: - :tags: engine, bug - :tickets: 2529 - - Added gaerdbms import to mysql/__init__.py, - the absence of which was preventing the new - GAE dialect from being loaded. - - .. change:: - :tags: engine, bug - :tickets: 2553 - - Fixed cextension bug whereby the - "ambiguous column error" would fail to - function properly if the given index were - a Column object and not a string. - Note there are still some column-targeting - issues here which are fixed in 0.8. - - .. change:: - :tags: engine, bug - :tickets: - - Fixed the repr() of Enum to include - the "name" and "native_enum" flags. Helps - Alembic autogenerate. - - .. change:: - :tags: sqlite, bug - :tickets: 2568 - - Adjusted a very old bugfix which attempted - to work around a SQLite issue that itself was - "fixed" as of sqlite 3.6.14, regarding quotes - surrounding a table name when using - the "foreign_key_list" pragma. The fix has been - adjusted to not interfere with quotes that - are *actually in the name* of a column or table, - to as much a degree as possible; sqlite still - doesn't return the correct result for foreign_key_list() - if the target table actually has quotes surrounding - its name, as *part* of its name (i.e. """mytable"""). - - .. change:: - :tags: sqlite, bug - :tickets: 2265 - - Adjusted column default reflection code to - convert non-string values to string, to accommodate - old SQLite versions that don't deliver - default info as a string. - - .. change:: - :tags: sqlite, feature - :tickets: - - Added support for the localtimestamp() - SQL function implemented in SQLite, courtesy - Richard Mitchell. - - .. change:: - :tags: postgresql, bug - :tickets: 2531 - - Columns in reflected primary key constraint - are now returned in the order in which the constraint - itself defines them, rather than how the table - orders them. Courtesy Gunnlaugur Þór Briem.. - - .. change:: - :tags: postgresql, bug - :tickets: 2570 - - Added 'terminating connection' to the list - of messages we use to detect a disconnect with PG, which - appears to be present in some versions when the server - is restarted. - - .. change:: - :tags: bug, mysql - :tickets: - - Updated mysqlconnector interface to use - updated "client flag" and "charset" APIs, - courtesy David McNelis. - - .. change:: - :tags: mssql, bug - :tickets: 2538 - - Fixed compiler bug whereby using a correlated - subquery within an ORDER BY would fail to render correctly - if the statement also used LIMIT/OFFSET, due to mis-rendering - within the ROW_NUMBER() OVER clause. Fix courtesy - sayap - - .. change:: - :tags: mssql, bug - :tickets: 2545 - - Fixed compiler bug whereby a given - select() would be modified if it had an "offset" - attribute, causing the construct to not compile - correctly a second time. - - .. change:: - :tags: mssql, bug - :tickets: - - Fixed bug where reflection of primary key constraint - would double up columns if the same constraint/table - existed in multiple schemas. - -.. changelog:: - :version: 0.7.8 - :released: Sat Jun 16 2012 - - .. change:: - :tags: orm, bug - :tickets: 2480 - - Fixed bug whereby subqueryload() from - a polymorphic mapping to a target would incur - a new invocation of the query for each - distinct class encountered in the polymorphic - result. - - .. change:: - :tags: orm, bug - :tickets: 2491, 1892 - - Fixed bug in declarative - whereby the precedence of columns - in a joined-table, composite - column (typically for id) would fail to - be correct if the columns contained - names distinct from their attribute - names. This would cause things like - primaryjoin conditions made against the - entity attributes to be incorrect. Related - to as this was supposed - to be part of that, this is. - - .. change:: - :tags: orm, feature - :tickets: - - The 'objects' argument to - flush() is no longer deprecated, as some - valid use cases have been identified. - - .. change:: - :tags: orm, bug - :tickets: 2508 - - Fixed identity_key() function which - was not accepting a scalar argument - for the identity. . - - .. change:: - :tags: orm, bug - :tickets: 2497 - - Fixed bug whereby populate_existing - option would not propagate to subquery - eager loaders. . - - .. change:: - :tags: bug, sql - :tickets: 2499 - - added BIGINT to types.__all__, - BIGINT, BINARY, VARBINARY to sqlalchemy - module namespace, plus test to ensure - this breakage doesn't occur again. - - .. change:: - :tags: bug, sql - :tickets: 2490 - - Repaired common table expression - rendering to function correctly when the - SELECT statement contains UNION or other - compound expressions, courtesy btbuilder. - - .. change:: - :tags: bug, sql - :tickets: 2482 - - Fixed bug whereby append_column() - wouldn't function correctly on a cloned - select() construct, courtesy - Gunnlaugur Þór Briem. - - .. change:: - :tags: engine, bug - :tickets: 2489 - - Fixed memory leak in C version of - result proxy whereby DBAPIs which don't deliver - pure Python tuples for result rows would - fail to decrement refcounts correctly. - The most prominently affected DBAPI - is pyodbc. - - .. change:: - :tags: engine, bug - :tickets: 2503 - - Fixed bug affecting Py3K whereby - string positional parameters passed to - engine/connection execute() would fail to be - interpreted correctly, due to __iter__ - being present on Py3K string.. - - .. change:: - :tags: postgresql, bug - :tickets: 2510 - - removed unnecessary table clause when - reflecting enums,. Courtesy - Gunnlaugur Þór Briem. - - .. change:: - :tags: oracle, bug - :tickets: 2483 - - Added ROWID to oracle.*. - - .. change:: - :tags: feature, mysql - :tickets: 2484 - - Added a new dialect for Google App - Engine. Courtesy Richie Foreman. - -.. changelog:: - :version: 0.7.7 - :released: Sat May 05 2012 - - .. change:: - :tags: orm, bug - :tickets: 2477 - - Fixed issue in unit of work - whereby setting a non-None self-referential - many-to-one relationship to None - would fail to persist the change if the - former value was not already loaded.. - - .. change:: - :tags: orm, feature - :tickets: 2443 - - Added prefix_with() method - to Query, calls upon select().prefix_with() - to allow placement of MySQL SELECT - directives in statements. Courtesy - Diana Clarke - - .. change:: - :tags: orm, bug - :tickets: 2409 - - Fixed bug in 0.7.6 introduced by whereby column_mapped_collection - used against columns that were mapped as - joins or other indirect selectables - would fail to function. - - .. change:: - :tags: orm, feature - :tickets: - - Added new flag to @validates - include_removes. When True, collection - remove and attribute del events - will also be sent to the validation function, - which accepts an additional argument - "is_remove" when this flag is used. - - .. change:: - :tags: orm, bug - :tickets: 2449 - - Fixed bug whereby polymorphic_on - column that's not otherwise mapped on the - class would be incorrectly included - in a merge() operation, raising an error. - - .. change:: - :tags: orm, bug - :tickets: 2453 - - Fixed bug in expression annotation - mechanics which could lead to incorrect - rendering of SELECT statements with aliases - and joins, particularly when using - column_property(). - - .. change:: - :tags: orm, bug - :tickets: 2454 - - Fixed bug which would prevent - OrderingList from being pickleable. Courtesy Jeff Dairiki - - .. change:: - :tags: orm, bug - :tickets: - - Fixed bug in relationship comparisons - whereby calling unimplemented methods like - SomeClass.somerelationship.like() would - produce a recursion overflow, instead - of NotImplementedError. - - .. change:: - :tags: bug, sql - :tickets: - - Removed warning when Index is created - with no columns; while this might not be what - the user intended, it is a valid use case - as an Index could be a placeholder for just an - index of a certain name. - - .. change:: - :tags: feature, sql - :tickets: - - Added new connection event - dbapi_error(). Is called for all DBAPI-level - errors passing the original DBAPI exception - before SQLAlchemy modifies the state - of the cursor. - - .. change:: - :tags: bug, sql - :tickets: - - If conn.begin() fails when calling - "with engine.begin()", the newly acquired - Connection is closed explicitly before - propagating the exception onward normally. - - .. change:: - :tags: bug, sql - :tickets: 2474 - - Add BINARY, VARBINARY to types.__all__. - - .. change:: - :tags: mssql, feature - :tickets: - - Added interim create_engine flag - supports_unicode_binds to PyODBC dialect, - to force whether or not the dialect - passes Python unicode literals to PyODBC - or not. - - .. change:: - :tags: mssql, bug - :tickets: - - Repaired the use_scope_identity - create_engine() flag when using the pyodbc - dialect. Previously this flag would be - ignored if set to False. When set to False, - you'll get "SELECT @@identity" after each - INSERT to get at the last inserted ID, - for those tables which have "implicit_returning" - set to False. - - .. change:: - :tags: mssql, bug - :tickets: 2468 - - UPDATE..FROM syntax with SQL Server - requires that the updated table be present - in the FROM clause when an alias of that - table is also present in the FROM clause. - The updated table is now always present - in the FROM, when FROM is present - in the first place. Courtesy sayap. - - .. change:: - :tags: postgresql, feature - :tickets: 2445 - - Added new for_update/with_lockmode() - options for PostgreSQL: for_update="read"/ - with_lockmode("read"), - for_update="read_nowait"/ - with_lockmode("read_nowait"). - These emit "FOR SHARE" and "FOR SHARE NOWAIT", - respectively. Courtesy Diana Clarke - - .. change:: - :tags: postgresql, bug - :tickets: 2473 - - removed unnecessary table clause - when reflecting domains. - - .. change:: - :tags: bug, mysql - :tickets: 2460 - - Fixed bug whereby column name inside - of "KEY" clause for autoincrement composite - column with InnoDB would double quote a - name that's a reserved word. Courtesy Jeff - Dairiki. - - .. change:: - :tags: bug, mysql - :tickets: - - Fixed bug whereby get_view_names() for - "information_schema" schema would fail - to retrieve views marked as "SYSTEM VIEW". - courtesy Matthew Turland. - - .. change:: - :tags: bug, mysql - :tickets: 2467 - - Fixed bug whereby if cast() is used - on a SQL expression whose type is not supported - by cast() and therefore CAST isn't rendered by - the dialect, the order of evaluation could change - if the casted expression required that it be - grouped; grouping is now applied to those - expressions. - - .. change:: - :tags: sqlite, feature - :tickets: 2475 - - Added SQLite execution option - "sqlite_raw_colnames=True", will bypass - attempts to remove "." from column names - returned by SQLite cursor.description. - - .. change:: - :tags: sqlite, bug - :tickets: 2525 - - When the primary key column of a Table - is replaced, such as via extend_existing, - the "auto increment" column used by insert() - constructs is reset. Previously it would - remain referring to the previous primary - key column. - -.. changelog:: - :version: 0.7.6 - :released: Wed Mar 14 2012 - - .. change:: - :tags: orm, bug - :tickets: 2424 - - Fixed event registration bug - which would primarily show up as - events not being registered with - sessionmaker() instances created - after the event was associated - with the Session class. - - .. change:: - :tags: orm, bug - :tickets: 2425 - - Fixed bug whereby a primaryjoin - condition with a "literal" in it would - raise an error on compile with certain - kinds of deeply nested expressions - which also needed to render the same - bound parameter name more than once. - - .. change:: - :tags: orm, feature - :tickets: - - Added "no_autoflush" context - manager to Session, used with with: - will temporarily disable autoflush. - - .. change:: - :tags: orm, feature - :tickets: 1859 - - Added cte() method to Query, - invokes common table expression support - from the Core (see below). - - .. change:: - :tags: orm, bug - :tickets: 2403 - - Removed the check for number of - rows affected when doing a multi-delete - against mapped objects. If an ON DELETE - CASCADE exists between two rows, we can't - get an accurate rowcount from the DBAPI; - this particular count is not supported - on most DBAPIs in any case, MySQLdb - is the notable case where it is. - - .. change:: - :tags: orm, bug - :tickets: 2409 - - Fixed bug whereby objects using - attribute_mapped_collection or - column_mapped_collection could not be - pickled. - - .. change:: - :tags: orm, bug - :tickets: 2406 - - Fixed bug whereby MappedCollection - would not get the appropriate collection - instrumentation if it were only used - in a custom subclass that used - @collection.internally_instrumented. - - .. change:: - :tags: orm, bug - :tickets: 2419 - - Fixed bug whereby SQL adaption mechanics - would fail in a very nested scenario involving - joined-inheritance, joinedload(), limit(), and a - derived function in the columns clause. - - .. change:: - :tags: orm, bug - :tickets: 2417 - - Fixed the repr() for CascadeOptions to - include refresh-expire. Also reworked - CascadeOptions to be a . - - .. change:: - :tags: orm, feature - :tickets: 2400 - - Added the ability to query for - Table-bound column names when using - query(sometable).filter_by(colname=value). - - .. change:: - :tags: orm, bug - :tickets: - - Improved the "declarative reflection" - example to support single-table inheritance, - multiple calls to prepare(), tables that - are present in alternate schemas, - establishing only a subset of classes - as reflected. - - .. change:: - :tags: orm, bug - :tickets: 2390 - - Scaled back the test applied within - flush() to check for UPDATE against partially - NULL PK within one table to only actually - happen if there's really an UPDATE to occur. - - .. change:: - :tags: orm, bug - :tickets: 2352 - - Fixed bug whereby if a method name - conflicted with a column name, a - TypeError would be raised when the mapper - tried to inspect the __get__() method - on the method object. - - .. change:: - :tags: bug, sql - :tickets: 2427 - - Fixed memory leak in core which would - occur when C extensions were used with - particular types of result fetches, - in particular when orm query.count() - were called. - - .. change:: - :tags: bug, sql - :tickets: 2398 - - Fixed issue whereby attribute-based - column access on a row would raise - AttributeError with non-C version, - NoSuchColumnError with C version. Now - raises AttributeError in both cases. - - .. change:: - :tags: feature, sql - :tickets: 1859 - - Added support for SQL standard - common table expressions (CTE), allowing - SELECT objects as the CTE source (DML - not yet supported). This is invoked via - the cte() method on any select() construct. - - .. change:: - :tags: bug, sql - :tickets: 2392 - - Added support for using the .key - of a Column as a string identifier in a - result set row. The .key is currently - listed as an "alternate" name for a column, - and is superseded by the name of a column - which has that key value as its regular name. - For the next major release - of SQLAlchemy we may reverse this precedence - so that .key takes precedence, but this - is not decided on yet. - - .. change:: - :tags: bug, sql - :tickets: 2413 - - A warning is emitted when a not-present - column is stated in the values() clause - of an insert() or update() construct. - Will move to an exception in 0.8. - - .. change:: - :tags: bug, sql - :tickets: 2396 - - A significant change to how labeling - is applied to columns in SELECT statements - allows "truncated" labels, that is label names - that are generated in Python which exceed - the maximum identifier length (note this is - configurable via label_length on create_engine()), - to be properly referenced when rendered inside - of a subquery, as well as to be present - in a result set row using their original - in-Python names. - - .. change:: - :tags: bug, sql - :tickets: 2402 - - Fixed bug in new "autoload_replace" flag - which would fail to preserve the primary - key constraint of the reflected table. - - .. change:: - :tags: bug, sql - :tickets: 2380 - - Index will raise when arguments passed - cannot be interpreted as columns or expressions. - Will warn when Index is created - with no columns at all. - - .. change:: - :tags: engine, feature - :tickets: 2407 - - Added "no_parameters=True" execution - option for connections. If no parameters - are present, will pass the statement - as cursor.execute(statement), thereby invoking - the DBAPIs behavior when no parameter collection - is present; for psycopg2 and mysql-python, this - means not interpreting % signs in the string. - This only occurs with this option, and not - just if the param list is blank, as otherwise - this would produce inconsistent behavior - of SQL expressions that normally escape percent - signs (and while compiling, can't know ahead of - time if parameters will be present in - some cases). - - .. change:: - :tags: engine, bug - :tickets: - - Added execution_options() call to - MockConnection (i.e., that used with - strategy="mock") which acts as a pass through - for arguments. - - .. change:: - :tags: engine, feature - :tickets: 2378 - - Added pool_reset_on_return argument - to create_engine, allows control over - "connection return" behavior. Also added - new arguments 'rollback', 'commit', None - to pool.reset_on_return to allow more control - over connection return activity. - - .. change:: - :tags: engine, feature - :tickets: - - Added some decent context managers - to Engine, Connection:: - - with engine.begin() as conn: - # - ... - - and:: - - with engine.connect() as conn: - # - ... - - Both close out the connection when done, - commit or rollback transaction with errors - on engine.begin(). - - .. change:: - :tags: sqlite, bug - :tickets: 2432 - - Fixed bug in C extensions whereby - string format would not be applied to a - Numeric value returned as integer; this - affected primarily SQLite which does - not maintain numeric scale settings. - - .. change:: - :tags: mssql, feature - :tickets: 2430 - - Added support for MSSQL INSERT, - UPDATE, and DELETE table hints, using - new with_hint() method on UpdateBase. - - .. change:: - :tags: feature, mysql - :tickets: 2386 - - Added support for MySQL index and - primary key constraint types - (i.e. USING) via new mysql_using parameter - to Index and PrimaryKeyConstraint, - courtesy Diana Clarke. - - .. change:: - :tags: feature, mysql - :tickets: 2394 - - Added support for the "isolation_level" - parameter to all MySQL dialects. Thanks - to mu_mind for the patch here. - - .. change:: - :tags: oracle, feature - :tickets: 2399 - - Added a new create_engine() flag - coerce_to_decimal=False, disables the precision - numeric handling which can add lots of overhead - by converting all numeric values to - Decimal. - - .. change:: - :tags: oracle, bug - :tickets: 2401 - - Added missing compilation support for - LONG - - .. change:: - :tags: oracle, bug - :tickets: 2435 - - Added 'LEVEL' to the list of reserved - words for Oracle. - - .. change:: - :tags: examples, bug - :tickets: - - Altered _params_from_query() function - in Beaker example to pull bindparams from the - fully compiled statement, as a quick means - to get everything including subqueries in the - columns clause, etc. - -.. changelog:: - :version: 0.7.5 - :released: Sat Jan 28 2012 - - .. change:: - :tags: orm, bug - :tickets: 2389 - - Fixed issue where modified session state - established after a failed flush would be committed - as part of the subsequent transaction that - begins automatically after manual call - to rollback(). The state of the session is - checked within rollback(), and if new state - is present, a warning is emitted and - restore_snapshot() is called a second time, - discarding those changes. - - .. change:: - :tags: orm, bug - :tickets: 2345 - - Fixed regression from 0.7.4 whereby - using an already instrumented column from a - superclass as "polymorphic_on" failed to resolve - the underlying Column. - - .. change:: - :tags: orm, bug - :tickets: 2370 - - Raise an exception if xyzload_all() is - used inappropriately with two non-connected - relationships. - - .. change:: - :tags: orm, feature - :tickets: - - Added "class_registry" argument to - declarative_base(). Allows two or more declarative - bases to share the same registry of class names. - - .. change:: - :tags: orm, feature - :tickets: - - query.filter() accepts multiple - criteria which will join via AND, i.e. - query.filter(x==y, z>q, ...) - - .. change:: - :tags: orm, feature - :tickets: 2351 - - Added new capability to relationship - loader options to allow "default" loader strategies. - Pass '*' to any of joinedload(), lazyload(), - subqueryload(), or noload() and that becomes the - loader strategy used for all relationships, - except for those explicitly stated in the - Query. Thanks to up-and-coming contributor - Kent Bower for an exhaustive and well - written test suite ! - - .. change:: - :tags: orm, bug - :tickets: 2367 - - Fixed bug whereby event.listen(SomeClass) - forced an entirely unnecessary compile of the - mapper, making events very hard to set up - at module import time (nobody noticed this ??) - - .. change:: - :tags: orm, bug - :tickets: - - Fixed bug whereby hybrid_property didn't - work as a kw arg in any(), has(). - - .. change:: - :tags: orm - :tickets: - - Fixed regression from 0.6 whereby if - "load_on_pending" relationship() flag were used - where a non-"get()" lazy clause needed to be - emitted on a pending object, it would fail - to load. - - .. change:: - :tags: orm, bug - :tickets: 2371 - - ensure pickleability of all ORM exceptions - for multiprocessing compatibility. - - .. change:: - :tags: orm, bug - :tickets: 2353 - - implemented standard "can't set attribute" / - "can't delete attribute" AttributeError when - setattr/delattr used on a hybrid that doesn't - define fset or fdel. - - .. change:: - :tags: orm, bug - :tickets: 2362 - - Fixed bug where unpickled object didn't - have enough of its state set up to work - correctly within the unpickle() event established - by the mutable object extension, if the object - needed ORM attribute access within - __eq__() or similar. - - .. change:: - :tags: orm, bug - :tickets: 2374 - - Fixed bug where "merge" cascade could - mis-interpret an unloaded attribute, if the - load_on_pending flag were used with - relationship(). Thanks to Kent Bower - for tests. - - .. change:: - :tags: orm, feature - :tickets: 2356 - - New declarative reflection example - added, illustrates how best to mix table reflection - with declarative as well as uses some new features - from. - - .. change:: - :tags: feature, sql - :tickets: 2356 - - New reflection feature "autoload_replace"; - when set to False on Table, the Table can be autoloaded - without existing columns being replaced. Allows - more flexible chains of Table construction/reflection - to be constructed, including that it helps with - combining Declarative with table reflection. - See the new example on the wiki. - - .. change:: - :tags: bug, sql - :tickets: 2356 - - Improved the API for add_column() such that - if the same column is added to its own table, - an error is not raised and the constraints - don't get doubled up. Also helps with some - reflection/declarative patterns. - - .. change:: - :tags: feature, sql - :tickets: - - Added "false()" and "true()" expression - constructs to sqlalchemy.sql namespace, though - not part of __all__ as of yet. - - .. change:: - :tags: feature, sql - :tickets: 2361 - - Dialect-specific compilers now raise - CompileError for all type/statement compilation - issues, instead of InvalidRequestError or ArgumentError. - The DDL for CREATE TABLE will re-raise - CompileError to include table/column information - for the problematic column. - - .. change:: - :tags: bug, sql - :tickets: 2381 - - Fixed issue where the "required" exception - would not be raised for bindparam() with required=True, - if the statement were given no parameters at all. - - .. change:: - :tags: engine, bug - :tickets: 2371 - - Added __reduce__ to StatementError, - DBAPIError, column errors so that exceptions - are pickleable, as when using multiprocessing. - However, not - all DBAPIs support this yet, such as - psycopg2. - - .. change:: - :tags: engine, bug - :tickets: 2382 - - Improved error messages when a non-string - or invalid string is passed to any of the - date/time processors used by SQLite, including - C and Python versions. - - .. change:: - :tags: engine, bug - :tickets: 2377 - - Fixed bug whereby a table-bound Column - object named "_" which matched a column - labeled as "_" could match - inappropriately when targeting in a result - set row. - - .. change:: - :tags: engine, bug - :tickets: 2384 - - Fixed bug in "mock" strategy whereby - correct DDL visit method wasn't called, resulting - in "CREATE/DROP SEQUENCE" statements being - duplicated - - .. change:: - :tags: sqlite, bug - :tickets: 2364 - - the "name" of an FK constraint in SQLite - is reflected as "None", not "0" or other - integer value. - SQLite does not appear to support constraint - naming in any case. - - .. change:: - :tags: sqlite, bug - :tickets: 2368 - - sql.false() and sql.true() compile to - 0 and 1, respectively in sqlite - - .. change:: - :tags: sqlite, bug - :tickets: - - removed an erroneous "raise" in the - SQLite dialect when getting table names - and view names, where logic is in place - to fall back to an older version of - SQLite that doesn't have the - "sqlite_temp_master" table. - - .. change:: - :tags: bug, mysql - :tickets: 2376 - - fixed regexp that filters out warnings - for non-reflected "PARTITION" directives, - thanks to George Reilly - - .. change:: - :tags: mssql, bug - :tickets: 2340 - - Adjusted the regexp used in the - mssql.TIME type to ensure only six digits - are received for the "microseconds" portion - of the value, which is expected by - Python's datetime.time(). Note that - support for sending microseconds doesn't - seem to be possible yet with pyodbc - at least. - - .. change:: - :tags: mssql, bug - :tickets: 2347 - - Dropped the "30 char" limit on pymssql, - based on reports that it's doing things - better these days. pymssql hasn't been - well tested and as the DBAPI is in flux - it's still not clear what the status - is on this driver and how SQLAlchemy's - implementation should adapt. - - .. change:: - :tags: oracle, bug - :tickets: 2388 - - Added ORA-03135 to the never ending - list of oracle "connection lost" errors - - .. change:: - :tags: core, bug - :tickets: 2379 - - Changed LRUCache, used by the mapper - to cache INSERT/UPDATE/DELETE statements, - to use an incrementing counter instead - of a timestamp to track entries, for greater - reliability versus using time.time(), which - can cause test failures on some platforms. - - .. change:: - :tags: core, bug - :tickets: 2383 - - Added a boolean check for the "finalize" - function within the pool connection proxy's - weakref callback before calling it, so that a - warning isn't emitted that this function is None - when the application is exiting and gc has - removed the function from the module before the - weakref callback was invoked. - - .. change:: - :tags: bug, py3k - :tickets: 2348 - - Fixed inappropriate usage of util.py3k - flag and renamed it to util.py3k_warning, since - this flag is intended to detect the -3 flag - series of import restrictions only. - - .. change:: - :tags: examples, feature - :tickets: 2313 - - Simplified the versioning example - a bit to use a declarative mixin as well - as an event listener, instead of a metaclass + - SessionExtension. - - .. change:: - :tags: examples, bug - :tickets: 2346 - - Fixed large_collection.py to close the - session before dropping tables. - -.. changelog:: - :version: 0.7.4 - :released: Fri Dec 09 2011 - - .. change:: - :tags: orm, bug - :tickets: 2315 - - Fixed backref behavior when "popping" the - value off of a many-to-one in response to - a removal from a stale one-to-many - the operation - is skipped, since the many-to-one has since - been updated. - - .. change:: - :tags: orm, bug - :tickets: 2264 - - After some years of not doing this, added - more granularity to the "is X a parent of Y" - functionality, which is used when determining - if the FK on "Y" needs to be "nulled out" as well - as if "Y" should be deleted with delete-orphan - cascade. The test now takes into account the - Python identity of the parent as well its identity - key, to see if the last known parent of Y is - definitely X. If a decision - can't be made, a StaleDataError is raised. The - conditions where this error is raised are fairly - rare, requiring that the previous parent was - garbage collected, and previously - could very well inappropriately update/delete - a record that's since moved onto a new parent, - though there may be some cases where - "silent success" occurred previously that will now - raise in the face of ambiguity. - Expiring "Y" resets the "parent" tracker, meaning - X.remove(Y) could then end up deleting Y even - if X is stale, but this is the same behavior - as before; it's advised to expire X also in that - case. - - .. change:: - :tags: orm, bug - :tickets: 2310 - - fixed inappropriate evaluation of user-mapped - object in a boolean context within query.get(). Also in 0.6.9. - - .. change:: - :tags: orm, bug - :tickets: 2304 - - Added missing comma to PASSIVE_RETURN_NEVER_SET - symbol - - .. change:: - :tags: orm, bug - :tickets: 1776 - - Cls.column.collate("some collation") now - works. Also in 0.6.9 - - .. change:: - :tags: orm, bug - :tickets: 2309 - - the value of a composite attribute is now - expired after an insert or update operation, instead - of regenerated in place. This ensures that a - column value which is expired within a flush - will be loaded first, before the composite - is regenerated using that value. - - .. change:: - :tags: orm, bug - :tickets: 2309, 2308 - - The fix in also emits the - "refresh" event when the composite value is - loaded on access, even if all column - values were already present, as is appropriate. - This fixes the "mutable" extension which relies - upon the "load" event to ensure the _parents - dictionary is up to date, fixes. - Thanks to Scott Torborg for the test case here. - - .. change:: - :tags: orm, bug - :tickets: 2312 - - Fixed bug whereby a subclass of a subclass - using concrete inheritance in conjunction with - the new ConcreteBase or AbstractConcreteBase - would fail to apply the subclasses deeper than - one level to the "polymorphic loader" of each - base - - .. change:: - :tags: orm, bug - :tickets: 2312 - - Fixed bug whereby a subclass of a subclass - using the new AbstractConcreteBase would fail - to acquire the correct "base_mapper" attribute - when the "base" mapper was generated, thereby - causing failures later on. - - .. change:: - :tags: orm, bug - :tickets: 2316 - - Fixed bug whereby column_property() created - against ORM-level column could be treated as - a distinct entity when producing certain - kinds of joined-inh joins. - - .. change:: - :tags: orm, bug - :tickets: 2297 - - Fixed the error formatting raised when - a tuple is inadvertently passed to session.query(). Also in 0.6.9. - - .. change:: - :tags: orm, bug - :tickets: 2328 - - Calls to query.join() to a single-table - inheritance subclass are now tracked, and - are used to eliminate the additional WHERE.. - IN criterion normally tacked on with single - table inheritance, since the join should - accommodate it. This allows OUTER JOIN - to a single table subclass to produce - the correct results, and overall will produce - fewer WHERE criterion when dealing with - single table inheritance joins. - - .. change:: - :tags: orm, bug - :tickets: 2339 - - __table_args__ can now be passed as - an empty tuple as well as an empty dict.. Thanks to Fayaz Yusuf Khan - for the patch. - - .. change:: - :tags: orm, bug - :tickets: 2325 - - Updated warning message when setting - delete-orphan without delete to no longer - refer to 0.6, as we never got around to - upgrading this to an exception. Ideally - this might be better as an exception but - it's not critical either way. - - .. change:: - :tags: orm, feature - :tickets: 2345, 2238 - - polymorphic_on now accepts many - new kinds of values: - - * standalone expressions that aren't - otherwise mapped - * column_property() objects - * string names of any column_property() - or attribute name of a mapped Column - - The docs include an example using - the case() construct, which is likely to be - a common constructed used here. and part of - - Standalone expressions in polymorphic_on - propagate to single-table inheritance - subclasses so that they are used in the - WHERE /JOIN clause to limit rows to that - subclass as is the usual behavior. - - .. change:: - :tags: orm, feature - :tickets: 2301 - - IdentitySet supports the - operator - as the same as difference(), handy when dealing - with Session.dirty etc. - - .. change:: - :tags: orm, feature - :tickets: - - Added new value for Column autoincrement - called "ignore_fk", can be used to force autoincrement - on a column that's still part of a ForeignKeyConstraint. - New example in the relationship docs illustrates - its use. - - .. change:: - :tags: orm, bug - :tickets: - - Fixed bug in get_history() when referring - to a composite attribute that has no value; - added coverage for get_history() regarding - composites which is otherwise just a userland - function. - - .. change:: - :tags: bug, sql - :tickets: 2316, 2261 - - related to, made some - adjustments to the change from - regarding the "from" list on a select(). The - _froms collection is no longer memoized, as this - simplifies various use cases and removes the - need for a "warning" if a column is attached - to a table after it was already used in an - expression - the select() construct will now - always produce the correct expression. - There's probably no real-world - performance hit here; select() objects are - almost always made ad-hoc, and systems that - wish to optimize the re-use of a select() - would be using the "compiled_cache" feature. - A hit which would occur when calling select.bind - has been reduced, but the vast majority - of users shouldn't be using "bound metadata" - anyway :). - - .. change:: - :tags: feature, sql - :tickets: 2166, 1944 - - The update() construct can now accommodate - multiple tables in the WHERE clause, which will - render an "UPDATE..FROM" construct, recognized by - PostgreSQL and MSSQL. When compiled on MySQL, - will instead generate "UPDATE t1, t2, ..". MySQL - additionally can render against multiple tables in the - SET clause, if Column objects are used as keys - in the "values" parameter or generative method. - - .. change:: - :tags: feature, sql - :tickets: 77 - - Added accessor to types called "python_type", - returns the rudimentary Python type object - for a particular TypeEngine instance, if known, - else raises NotImplementedError. - - .. change:: - :tags: bug, sql - :tickets: 2261, 2319 - - further tweak to the fix from, - so that generative methods work a bit better - off of cloned (this is almost a non-use case though). - In particular this allows with_only_columns() - to behave more consistently. Added additional - documentation to with_only_columns() to clarify - expected behavior, which changed as a result - of. - - .. change:: - :tags: engine, bug - :tickets: 2317 - - Fixed bug whereby transaction.rollback() - would throw an error on an invalidated - connection if the transaction were a - two-phase or savepoint transaction. - For plain transactions, rollback() is a no-op - if the connection is invalidated, so while - it wasn't 100% clear if it should be a no-op, - at least now the interface is consistent. - - .. change:: - :tags: feature, schema - :tickets: - - Added new support for remote "schemas": - - .. change:: - :tags: schema - :tickets: - - MetaData() accepts "schema" and "quote_schema" - arguments, which will be applied to the same-named - arguments of a Table - or Sequence which leaves these at their default - of ``None``. - - .. change:: - :tags: schema - :tickets: - - Sequence accepts "quote_schema" argument - - .. change:: - :tags: schema - :tickets: - - tometadata() for Table will use the "schema" - of the incoming MetaData for the new Table - if the schema argument is explicitly "None" - - .. change:: - :tags: schema - :tickets: - - Added CreateSchema and DropSchema DDL - constructs - these accept just the string - name of a schema and a "quote" flag. - - .. change:: - :tags: schema - :tickets: - - When using default "schema" with MetaData, - ForeignKey will also assume the "default" schema - when locating remote table. This allows the "schema" - argument on MetaData to be applied to any - set of Table objects that otherwise don't have - a "schema". - - .. change:: - :tags: schema - :tickets: 1679 - - a "has_schema" method has been implemented - on dialect, but only works on PostgreSQL so far. - Courtesy Manlio Perillo. - - .. change:: - :tags: feature, schema - :tickets: 1410 - - The "extend_existing" flag on Table - now allows for the reflection process to take - effect for a Table object that's already been - defined; when autoload=True and extend_existing=True - are both set, the full set of columns will be - reflected from the Table which will then - *overwrite* those columns already present, - rather than no activity occurring. Columns that - are present directly in the autoload run - will be used as always, however. - - .. change:: - :tags: bug, schema - :tickets: - - Fixed bug whereby TypeDecorator would - return a stale value for _type_affinity, when - using a TypeDecorator that "switches" types, - like the CHAR/UUID type. - - .. change:: - :tags: bug, schema - :tickets: - - Fixed bug whereby "order_by='foreign_key'" - option to Inspector.get_table_names - wasn't implementing the sort properly, replaced - with the existing sort algorithm - - .. change:: - :tags: bug, schema - :tickets: 2305 - - the "name" of a column-level CHECK constraint, - if present, is now rendered in the CREATE TABLE - statement using "CONSTRAINT CHECK ". - - .. change:: - :tags: pyodbc, bug - :tickets: 2318 - - pyodbc-based dialects now parse the - pyodbc accurately as far as observed - pyodbc strings, including such gems - as "py3-3.0.1-beta4" - - .. change:: - :tags: postgresql, bug - :tickets: 2311 - - PostgreSQL dialect memoizes that an ENUM of a - particular name was processed - during a create/drop sequence. This allows - a create/drop sequence to work without any - calls to "checkfirst", and also means with - "checkfirst" turned on it only needs to - check for the ENUM once. - - .. change:: - :tags: postgresql, feature - :tickets: - - Added create_type constructor argument - to pg.ENUM. When False, no CREATE/DROP or - checking for the type will be performed as part - of a table create/drop event; only the - create()/drop)() methods called directly - will do this. Helps with Alembic "offline" - scripts. - - .. change:: - :tags: mssql, feature - :tickets: 822 - - lifted the restriction on SAVEPOINT - for SQL Server. All tests pass using it, - it's not known if there are deeper issues - however. - - .. change:: - :tags: mssql, bug - :tickets: 2336 - - repaired the with_hint() feature which - wasn't implemented correctly on MSSQL - - usually used for the "WITH (NOLOCK)" hint - (which you shouldn't be using anyway ! - use snapshot isolation instead :) ) - - .. change:: - :tags: mssql, bug - :tickets: 2318 - - use new pyodbc version detection for - _need_decimal_fix option. - - .. change:: - :tags: mssql, bug - :tickets: 2343 - - don't cast "table name" as NVARCHAR - on SQL Server 2000. Still mostly in the dark - what incantations are needed to make PyODBC - work fully with FreeTDS 0.91 here, however. - - .. change:: - :tags: mssql, bug - :tickets: 2269 - - Decode incoming values when retrieving - list of index names and the names of columns - within those indexes. - - .. change:: - :tags: bug, mysql - :tickets: - - Unicode adjustments allow latest pymysql - (post 0.4) to pass 100% on Python 2. - - .. change:: - :tags: ext, feature - :tickets: - - Added an example to the hybrid docs - of a "transformer" - a hybrid that returns a - query-transforming callable in combination - with a custom comparator. Uses a new method - on Query called with_transformation(). The use - case here is fairly experimental, but only - adds one line of code to Query. - - .. change:: - :tags: ext, bug - :tickets: - - the @compiles decorator raises an - informative error message when no "default" - compilation handler is present, rather - than KeyError. - - .. change:: - :tags: examples, bug - :tickets: - - Fixed bug in history_meta.py example where - the "unique" flag was not removed from a - single-table-inheritance subclass which - generates columns to put up onto the base. - -.. changelog:: - :version: 0.7.3 - :released: Sun Oct 16 2011 - - .. change:: - :tags: general - :tickets: 2279 - - Adjusted the "importlater" mechanism, which is - used internally to resolve import cycles, - such that the usage of __import__ is completed - when the import of sqlalchemy or sqlalchemy.orm - is done, thereby avoiding any usage of __import__ - after the application starts new threads, - fixes. Also in 0.6.9. - - .. change:: - :tags: orm - :tickets: 2298 - - Improved query.join() such that the "left" side - can more flexibly be a non-ORM selectable, - such as a subquery. A selectable placed - in select_from() will now be used as the left - side, favored over implicit usage - of a mapped entity. - If the join still fails based on lack of - foreign keys, the error message includes - this detail. Thanks to brianrhude - on IRC for the test case. - - .. change:: - :tags: orm - :tickets: 2241 - - Added after_soft_rollback() Session event. This - event fires unconditionally whenever rollback() - is called, regardless of if an actual DBAPI - level rollback occurred. This event - is specifically designed to allow operations - with the Session to proceed after a rollback - when the Session.is_active is True. - - .. change:: - :tags: orm - :tickets: - - added "adapt_on_names" boolean flag to orm.aliased() - construct. Allows an aliased() construct - to link the ORM entity to a selectable that contains - aggregates or other derived forms of a particular - attribute, provided the name is the same as that - of the entity mapped column. - - .. change:: - :tags: orm - :tickets: - - Added new flag expire_on_flush=False to column_property(), - marks those properties that would otherwise be considered - to be "readonly", i.e. derived from SQL expressions, - to retain their value after a flush has occurred, including - if the parent object itself was involved in an update. - - .. change:: - :tags: orm - :tickets: 2237 - - Enhanced the instrumentation in the ORM to support - Py3K's new argument style of "required kw arguments", - i.e. fn(a, b, \*, c, d), fn(a, b, \*args, c, d). - Argument signatures of mapped object's __init__ - method will be preserved, including required kw rules. - - .. change:: - :tags: orm - :tickets: 2282 - - Fixed bug in unit of work whereby detection of - "cycles" among classes in highly interlinked patterns - would not produce a deterministic - result; thereby sometimes missing some nodes that - should be considered cycles and causing further - issues down the road. Note this bug is in 0.6 - also; not backported at the moment. - - .. change:: - :tags: orm - :tickets: - - Fixed a variety of synonym()-related regressions - from 0.6: - - * making a synonym against a synonym now works. - * synonyms made against a relationship() can - be passed to query.join(), options sent - to query.options(), passed by name - to query.with_parent(). - - .. change:: - :tags: orm - :tickets: 2287 - - Fixed bug whereby mapper.order_by attribute would - be ignored in the "inner" query within a - subquery eager load. . - Also in 0.6.9. - - .. change:: - :tags: orm - :tickets: 2267 - - Identity map .discard() uses dict.pop(,None) - internally instead of "del" to avoid KeyError/warning - during a non-determinate gc teardown - - .. change:: - :tags: orm - :tickets: 2253 - - Fixed regression in new composite rewrite where - deferred=True option failed due to missing - import - - .. change:: - :tags: orm - :tickets: 2248 - - Reinstated "comparator_factory" argument to - composite(), removed when 0.7 was released. - - .. change:: - :tags: orm - :tickets: 2247 - - Fixed bug in query.join() which would occur - in a complex multiple-overlapping path scenario, - where the same table could be joined to - twice. Thanks *much* to Dave Vitek - for the excellent fix here. - - .. change:: - :tags: orm - :tickets: - - Query will convert an OFFSET of zero when - slicing into None, so that needless OFFSET - clauses are not invoked. - - .. change:: - :tags: orm - :tickets: - - Repaired edge case where mapper would fail - to fully update internal state when a relationship - on a new mapper would establish a backref on the - first mapper. - - .. change:: - :tags: orm - :tickets: 2260 - - Fixed bug whereby if __eq__() was - redefined, a relationship many-to-one lazyload - would hit the __eq__() and fail. - Does not apply to 0.6.9. - - .. change:: - :tags: orm - :tickets: 2196 - - Calling class_mapper() and passing in an object - that is not a "type" (i.e. a class that could - potentially be mapped) now raises an informative - ArgumentError, rather than UnmappedClassError. - - .. change:: - :tags: orm - :tickets: - - New event hook, MapperEvents.after_configured(). - Called after a configure() step has completed and - mappers were in fact affected. Theoretically this - event is called once per application, unless new mappings - are constructed after existing ones have been used - already. - - .. change:: - :tags: orm - :tickets: 2281 - - When an open Session is garbage collected, the objects - within it which remain are considered detached again - when they are add()-ed to a new Session. - This is accomplished by an extra check that the previous - "session_key" doesn't actually exist among the pool - of Sessions. - - .. change:: - :tags: orm - :tickets: 2239 - - New declarative features: - - * __declare_last__() method, establishes an event - listener for the class method that will be called - when mappers are completed with the final "configure" - step. - * __abstract__ flag. The class will not be mapped - at all when this flag is present on the class. - * New helper classes ConcreteBase, AbstractConcreteBase. - Allow concrete mappings using declarative which automatically - set up the "polymorphic_union" when the "configure" - mapper step is invoked. - * The mapper itself has semi-private methods that allow - the "with_polymorphic" selectable to be assigned - to the mapper after it has already been configured. - - .. change:: - :tags: orm - :tickets: 2283 - - Declarative will warn when a subclass' base uses - @declared_attr for a regular column - this attribute - does not propagate to subclasses. - - .. change:: - :tags: orm - :tickets: 2280 - - The integer "id" used to link a mapped instance with - its owning Session is now generated by a sequence - generation function rather than id(Session), to - eliminate the possibility of recycled id() values - causing an incorrect result, no need to check that - object actually in the session. - - .. change:: - :tags: orm - :tickets: 2257 - - Behavioral improvement: empty - conjunctions such as and_() and or_() will be - flattened in the context of an enclosing conjunction, - i.e. and_(x, or_()) will produce 'X' and not 'X AND - ()'.. - - .. change:: - :tags: orm - :tickets: 2261 - - Fixed bug regarding calculation of "from" list - for a select() element. The "from" calc is now - delayed, so that if the construct uses a Column - object that is not yet attached to a Table, - but is later associated with a Table, it generates - SQL using the table as a FROM. This change - impacted fairly deeply the mechanics of how - the FROM list as well as the "correlates" collection - is calculated, as some "clause adaption" schemes - (these are used very heavily in the ORM) - were relying upon the fact that the "froms" - collection would typically be cached before the - adaption completed. The rework allows it - such that the "froms" collection can be cleared - and re-generated at any time. - - .. change:: - :tags: orm - :tickets: 2270 - - Fixed bug whereby with_only_columns() method of - Select would fail if a selectable were passed.. Also in 0.6.9. - - .. change:: - :tags: schema - :tickets: 2284 - - Modified Column.copy() to use _constructor(), - which defaults to self.__class__, in order to - create the new object. This allows easier support - of subclassing Column. - - .. change:: - :tags: schema - :tickets: 2223 - - Added a slightly nicer __repr__() to SchemaItem - classes. Note the repr here can't fully support - the "repr is the constructor" idea since schema - items can be very deeply nested/cyclical, have - late initialization of some things, etc. - - .. change:: - :tags: engine - :tickets: 2254 - - The recreate() method in all pool classes uses - self.__class__ to get at the type of pool - to produce, in the case of subclassing. Note - there's no usual need to subclass pools. - - .. change:: - :tags: engine - :tickets: 2243 - - Improvement to multi-param statement logging, - long lists of bound parameter sets will be - compressed with an informative indicator - of the compression taking place. Exception - messages use the same improved formatting. - - .. change:: - :tags: engine - :tickets: - - Added optional "sa_pool_key" argument to - pool.manage(dbapi).connect() so that serialization - of args is not necessary. - - .. change:: - :tags: engine - :tickets: 2286 - - The entry point resolution supported by - create_engine() now supports resolution of - individual DBAPI drivers on top of a built-in - or entry point-resolved dialect, using the - standard '+' notation - it's converted to - a '.' before being resolved as an entry - point. - - .. change:: - :tags: engine - :tickets: 2299 - - Added an exception catch + warning for the - "return unicode detection" step within connect, - allows databases that crash on NVARCHAR to - continue initializing, assuming no NVARCHAR - type implemented. - - .. change:: - :tags: types - :tickets: 2258 - - Extra keyword arguments to the base Float - type beyond "precision" and "asdecimal" are ignored; - added a deprecation warning here and additional - docs, related to - - .. change:: - :tags: sqlite - :tickets: - - Ensured that the same ValueError is raised for - illegal date/time/datetime string parsed from - the database regardless of whether C - extensions are in use or not. - - .. change:: - :tags: postgresql - :tickets: 2290 - - Added "postgresql_using" argument to Index(), produces - USING clause to specify index implementation for - PG. . Thanks to Ryan P. Kelly for - the patch. - - .. change:: - :tags: postgresql - :tickets: 1839 - - Added client_encoding parameter to create_engine() - when the postgresql+psycopg2 dialect is used; - calls the psycopg2 set_client_encoding() method - with the value upon connect. - - .. change:: - :tags: postgresql - :tickets: 2291, 2141 - - Fixed bug related to whereby the - same modified index behavior in PG 9 affected - primary key reflection on a renamed column.. Also in 0.6.9. - - .. change:: - :tags: postgresql - :tickets: 2256 - - Reflection functions for Table, Sequence no longer - case insensitive. Names can be differ only in case - and will be correctly distinguished. - - .. change:: - :tags: postgresql - :tickets: - - Use an atomic counter as the "random number" - source for server side cursor names; - conflicts have been reported in rare cases. - - .. change:: - :tags: postgresql - :tickets: 2249 - - Narrowed the assumption made when reflecting - a foreign-key referenced table with schema in - the current search path; an explicit schema will - be applied to the referenced table only if - it actually matches that of the referencing table, - which also has an explicit schema. Previously - it was assumed that "current" schema was synonymous - with the full search_path. - - .. change:: - :tags: mysql - :tickets: 2225 - - a CREATE TABLE will put the COLLATE option - after CHARSET, which appears to be part of - MySQL's arbitrary rules regarding if it will actually - work or not. Also in 0.6.9. - - .. change:: - :tags: mysql - :tickets: 2293 - - Added mysql_length parameter to Index construct, - specifies "length" for indexes. - - .. change:: - :tags: mssql - :tickets: 2273 - - Changes to attempt support of FreeTDS 0.91 with - Pyodbc. This includes that string binds are sent as - Python unicode objects when FreeTDS 0.91 is detected, - and a CAST(? AS NVARCHAR) is used when we detect - for a table. However, I'd continue - to characterize Pyodbc + FreeTDS 0.91 behavior as - pretty crappy, there are still many queries such - as used in reflection which cause a core dump on - Linux, and it is not really usable at all - on OSX, MemoryErrors abound and just plain broken - unicode support. - - .. change:: - :tags: mssql - :tickets: 2277 - - The behavior of =/!= when comparing a scalar select - to a value will no longer produce IN/NOT IN as of 0.8; - this behavior is a little too heavy handed (use ``in_()`` if - you want to emit IN) and now emits a deprecation warning. - To get the 0.8 behavior immediately and remove the warning, - a compiler recipe is given at - https://www.sqlalchemy.org/docs/07/dialects/mssql.html#scalar-select-comparisons - to override the behavior of visit_binary(). - - .. change:: - :tags: mssql - :tickets: 2222 - - "0" is accepted as an argument for limit() which - will produce "TOP 0". - - .. change:: - :tags: oracle - :tickets: 2272 - - Fixed ReturningResultProxy for zxjdbc dialect.. Regression from 0.6. - - .. change:: - :tags: oracle - :tickets: 2252 - - The String type now generates VARCHAR2 on Oracle - which is recommended as the default VARCHAR. - Added an explicit VARCHAR2 and NVARCHAR2 to the Oracle - dialect as well. Using NVARCHAR still generates - "NVARCHAR2" - there is no "NVARCHAR" on Oracle - - this remains a slight breakage of the "uppercase types - always give exactly that" policy. VARCHAR still - generates "VARCHAR", keeping with the policy. If - Oracle were to ever define "VARCHAR" as something - different as they claim (IMHO this will never happen), - the type would be available. - - .. change:: - :tags: ext - :tickets: 2262 - - SQLSoup will not be included in version 0.8 - of SQLAlchemy; while useful, we would like to - keep SQLAlchemy itself focused on one ORM - usage paradigm. SQLSoup will hopefully - soon be superseded by a third party - project. - - .. change:: - :tags: ext - :tickets: 2236 - - Added local_attr, remote_attr, attr accessors - to AssociationProxy, providing quick access - to the proxied attributes at the class - level. - - .. change:: - :tags: ext - :tickets: 2275 - - Changed the update() method on association proxy - dictionary to use a duck typing approach, i.e. - checks for "keys", to discern between update({}) - and update((a, b)). Previously, passing a - dictionary that had tuples as keys would be misinterpreted - as a sequence. - - .. change:: - :tags: examples - :tickets: 2266 - - Adjusted dictlike-polymorphic.py example - to apply the CAST such that it works on - PG, other databases. - Also in 0.6.9. - -.. changelog:: - :version: 0.7.2 - :released: Sun Jul 31 2011 - - .. change:: - :tags: orm - :tickets: 2213 - - Feature enhancement: joined and subquery - loading will now traverse already-present related - objects and collections in search of unpopulated - attributes throughout the scope of the eager load - being defined, so that the eager loading that is - specified via mappings or query options - unconditionally takes place for the full depth, - populating whatever is not already populated. - Previously, this traversal would stop if a related - object or collection were already present leading - to inconsistent behavior (though would save on - loads/cycles for an already-loaded graph). For a - subqueryload, this means that the additional - SELECT statements emitted by subqueryload will - invoke unconditionally, no matter how much of the - existing graph is already present (hence the - controversy). The previous behavior of "stopping" - is still in effect when a query is the result of - an attribute-initiated lazyload, as otherwise an - "N+1" style of collection iteration can become - needlessly expensive when the same related object - is encountered repeatedly. There's also an - as-yet-not-public generative Query method - _with_invoke_all_eagers() - which selects old/new behavior - - .. change:: - :tags: orm - :tickets: 2195 - - A rework of "replacement traversal" within - the ORM as it alters selectables to be against - aliases of things (i.e. clause adaption) includes - a fix for multiply-nested any()/has() constructs - against a joined table structure. - - .. change:: - :tags: orm - :tickets: 2234 - - Fixed bug where query.join() + aliased=True - from a joined-inh structure to itself on - relationship() with join condition on the child - table would convert the lead entity into the - joined one inappropriately. - Also in 0.6.9. - - .. change:: - :tags: orm - :tickets: 2205 - - Fixed regression from 0.6 where Session.add() - against an object which contained None in a - collection would raise an internal exception. - Reverted this to 0.6's behavior which is to - accept the None but obviously nothing is - persisted. Ideally, collections with None - present or on append() should at least emit a - warning, which is being considered for 0.8. - - .. change:: - :tags: orm - :tickets: 2191 - - Load of a deferred() attribute on an object - where row can't be located raises - ObjectDeletedError instead of failing later - on; improved the message in ObjectDeletedError - to include other conditions besides a simple - "delete". - - .. change:: - :tags: orm - :tickets: 2224 - - Fixed regression from 0.6 where a get history - operation on some relationship() based attributes - would fail when a lazyload would emit; this could - trigger within a flush() under certain conditions. Thanks to the user who submitted - the great test for this. - - .. change:: - :tags: orm - :tickets: 2228 - - Fixed bug apparent only in Python 3 whereby - sorting of persistent + pending objects during - flush would produce an illegal comparison, - if the persistent object primary key - is not a single integer. - Also in 0.6.9 - - .. change:: - :tags: orm - :tickets: 2197 - - Fixed bug whereby the source clause - used by query.join() would be inconsistent - if against a column expression that combined - multiple entities together. - Also in 0.6.9 - - .. change:: - :tags: orm - :tickets: 2215 - - Fixed bug whereby if a mapped class - redefined __hash__() or __eq__() to something - non-standard, which is a supported use case - as SQLA should never consult these, - the methods would be consulted if the class - was part of a "composite" (i.e. non-single-entity) - result set. - Also in 0.6.9. - - .. change:: - :tags: orm - :tickets: 2240 - - Added public attribute ".validators" to - Mapper, an immutable dictionary view of - all attributes that have been decorated - with the @validates decorator. courtesy Stefano Fontanelli - - .. change:: - :tags: orm - :tickets: 2188 - - Fixed subtle bug that caused SQL to blow - up if: column_property() against subquery + - joinedload + LIMIT + order by the column - property() occurred. . - Also in 0.6.9 - - .. change:: - :tags: orm - :tickets: 2207 - - The join condition produced by with_parent - as well as when using a "dynamic" relationship - against a parent will generate unique - bindparams, rather than incorrectly repeating - the same bindparam. . - Also in 0.6.9. - - .. change:: - :tags: orm - :tickets: - - Added the same "columns-only" check to - mapper.polymorphic_on as used when - receiving user arguments to - relationship.order_by, foreign_keys, - remote_side, etc. - - .. change:: - :tags: orm - :tickets: 2190 - - Fixed bug whereby comparison of column - expression to a Query() would not call - as_scalar() on the underlying SELECT - statement to produce a scalar subquery, - in the way that occurs if you called - it on Query().subquery(). - - .. change:: - :tags: orm - :tickets: 2194 - - Fixed declarative bug where a class inheriting - from a superclass of the same name would fail - due to an unnecessary lookup of the name - in the _decl_class_registry. - - .. change:: - :tags: orm - :tickets: 2199 - - Repaired the "no statement condition" - assertion in Query which would attempt - to raise if a generative method were called - after from_statement() were called.. Also in 0.6.9. - - .. change:: - :tags: sql - :tickets: 2188 - - Fixed two subtle bugs involving column - correspondence in a selectable, - one with the same labeled subquery repeated, the other - when the label has been "grouped" and - loses itself. Affects. - - .. change:: - :tags: schema - :tickets: 2187 - - New feature: with_variant() method on - all types. Produces an instance of Variant(), - a special TypeDecorator which will select - the usage of a different type based on the - dialect in use. - - .. change:: - :tags: schema - :tickets: - - Added an informative error message when - ForeignKeyConstraint refers to a column name in - the parent that is not found. Also in 0.6.9. - - .. change:: - :tags: schema - :tickets: 2206 - - Fixed bug whereby adaptation of old append_ddl_listener() - function was passing unexpected \**kw through - to the Table event. Table gets no kws, the MetaData - event in 0.6 would get "tables=somecollection", - this behavior is preserved. - - .. change:: - :tags: schema - :tickets: - - Fixed bug where "autoincrement" detection on - Table would fail if the type had no "affinity" - value, in particular this would occur when using - the UUID example on the site that uses TypeEngine - as the "impl". - - .. change:: - :tags: schema - :tickets: 2209 - - Added an improved repr() to TypeEngine objects - that will only display constructor args which - are positional or kwargs that deviate - from the default. - - .. change:: - :tags: engine - :tickets: - - Context manager provided by Connection.begin() - will issue rollback() if the commit() fails, - not just if an exception occurs. - - .. change:: - :tags: engine - :tickets: 1682 - - Use urllib.parse_qsl() in Python 2.6 and above, - no deprecation warning about cgi.parse_qsl() - - .. change:: - :tags: engine - :tickets: - - Added mixin class sqlalchemy.ext.DontWrapMixin. - User-defined exceptions of this type are never - wrapped in StatementException when they - occur in the context of a statement - execution. - - .. change:: - :tags: engine - :tickets: - - StatementException wrapping will display the - original exception class in the message. - - .. change:: - :tags: engine - :tickets: 2201 - - Failures on connect which raise dbapi.Error - will forward the error to dialect.is_disconnect() - and set the "connection_invalidated" flag if - the dialect knows this to be a potentially - "retryable" condition. Only Oracle ORA-01033 - implemented for now. - - .. change:: - :tags: sqlite - :tickets: 2189 - - SQLite dialect no longer strips quotes - off of reflected default value, allowing - a round trip CREATE TABLE to work. - This is consistent with other dialects - that also maintain the exact form of - the default. - - .. change:: - :tags: postgresql - :tickets: 2198 - - Added new "postgresql_ops" argument to - Index, allows specification of PostgreSQL - operator classes for indexed columns. Courtesy Filip Zyzniewski. - - .. change:: - :tags: mysql - :tickets: 2186 - - Fixed OurSQL dialect to use ansi-neutral - quote symbol "'" for XA commands instead - of '"'. . Also in 0.6.9. - - .. change:: - :tags: mssql - :tickets: - - Adjusted the pyodbc dialect such that bound - values are passed as bytes and not unicode - if the "Easysoft" unix drivers are detected. - This is the same behavior as occurs with - FreeTDS. Easysoft appears to segfault - if Python unicodes are passed under - certain circumstances. - - .. change:: - :tags: oracle - :tickets: 2200 - - Added ORA-00028 to disconnect codes, use - cx_oracle _Error.code to get at the code,. Also in 0.6.9. - - .. change:: - :tags: oracle - :tickets: 2201 - - Added ORA-01033 to disconnect codes, which - can be caught during a connection - event. - - .. change:: - :tags: oracle - :tickets: 2220 - - repaired the oracle.RAW type which did not - generate the correct DDL. - Also in 0.6.9. - - .. change:: - :tags: oracle - :tickets: 2212 - - added CURRENT to reserved word list. Also in 0.6.9. - - .. change:: - :tags: oracle - :tickets: - - Fixed bug in the mutable extension whereby - if the same type were used twice in one - mapping, the attributes beyond the first - would not get instrumented. - - .. change:: - :tags: oracle - :tickets: - - Fixed bug in the mutable extension whereby - if None or a non-corresponding type were set, - an error would be raised. None is now accepted - which assigns None to all attributes, - illegal values raise ValueError. - - .. change:: - :tags: examples - :tickets: - - Repaired the examples/versioning test runner - to not rely upon SQLAlchemy test libs, - nosetests must be run from within - examples/versioning to get around setup.cfg - breaking it. - - .. change:: - :tags: examples - :tickets: - - Tweak to examples/versioning to pick the - correct foreign key in a multi-level - inheritance situation. - - .. change:: - :tags: examples - :tickets: - - Fixed the attribute shard example to check - for bind param callable correctly in 0.7 - style. - -.. changelog:: - :version: 0.7.1 - :released: Sun Jun 05 2011 - - .. change:: - :tags: general - :tickets: 2184 - - Added a workaround for Python bug 7511 where - failure of C extension build does not - raise an appropriate exception on Windows 64 - bit + VC express - - .. change:: - :tags: orm - :tickets: 1912 - - "delete-orphan" cascade is now allowed on - self-referential relationships - this since - SQLA 0.7 no longer enforces "parent with no - child" at the ORM level; this check is left - up to foreign key nullability. - Related to - - .. change:: - :tags: orm - :tickets: 2180 - - Repaired new "mutable" extension to propagate - events to subclasses correctly; don't - create multiple event listeners for - subclasses either. - - .. change:: - :tags: orm - :tickets: 2170 - - Modify the text of the message which occurs - when the "identity" key isn't detected on - flush, to include the common cause that - the Column isn't set up to detect - auto-increment correctly;. - Also in 0.6.8. - - .. change:: - :tags: orm - :tickets: 2182 - - Fixed bug where transaction-level "deleted" - collection wouldn't be cleared of expunged - states, raising an error if they later - became transient. - Also in 0.6.8. - - .. change:: - :tags: sql - :tickets: - - Fixed bug whereby metadata.reflect(bind) - would close a Connection passed as a - bind argument. Regression from 0.6. - - .. change:: - :tags: sql - :tickets: - - Streamlined the process by which a Select - determines what's in its '.c' collection. - Behaves identically, except that a - raw ClauseList() passed to select([]) - (which is not a documented case anyway) will - now be expanded into its individual column - elements instead of being ignored. - - .. change:: - :tags: engine - :tickets: - - Deprecate schema/SQL-oriented methods on - Connection/Engine that were never well known - and are redundant: reflecttable(), create(), - drop(), text(), engine.func - - .. change:: - :tags: engine - :tickets: 2178 - - Adjusted the __contains__() method of - a RowProxy result row such that no exception - throw is generated internally; - NoSuchColumnError() also will generate its - message regardless of whether or not the column - construct can be coerced to a string.. Also in 0.6.8. - - .. change:: - :tags: sqlite - :tickets: 2173 - - Accept None from cursor.fetchone() when - "PRAGMA read_uncommitted" is called to determine - current isolation mode at connect time and - default to SERIALIZABLE; this to support SQLite - versions pre-3.3.0 that did not have this - feature. - - .. change:: - :tags: postgresql - :tickets: 2175 - - Some unit test fixes regarding numeric arrays, - MATCH operator. A potential floating-point - inaccuracy issue was fixed, and certain tests - of the MATCH operator only execute within an - EN-oriented locale for now. . - Also in 0.6.8. - - .. change:: - :tags: mysql - :tickets: - - Unit tests pass 100% on MySQL installed - on windows. - - .. change:: - :tags: mysql - :tickets: 2181 - - Removed the "adjust casing" step that would - fail when reflecting a table on MySQL - on windows with a mixed case name. After some - experimenting with a windows MySQL server, it's - been determined that this step wasn't really - helping the situation much; MySQL does not return - FK names with proper casing on non-windows - platforms either, and removing the step at - least allows the reflection to act more like - it does on other OSes. A warning here - has been considered but its difficult to - determine under what conditions such a warning - can be raised, so punted on that for now - - added some docs instead. - - .. change:: - :tags: mysql - :tickets: - - supports_sane_rowcount will be set to False - if using MySQLdb and the DBAPI doesn't provide - the constants.CLIENT module. - -.. changelog:: - :version: 0.7.0 - :released: Fri May 20 2011 - - .. change:: - :tags: - :tickets: - - This section documents those changes from 0.7b4 - to 0.7.0. For an overview of what's new in - SQLAlchemy 0.7, see - https://docs.sqlalchemy.org/en/latest/changelog/migration_07.html - - .. change:: - :tags: orm - :tickets: 2069 - - Fixed regression introduced in 0.7b4 (!) whereby - query.options(someoption("nonexistent name")) would - fail to raise an error. Also added additional - error catching for cases where the option would - try to build off a column-based element, further - fixed up some of the error messages tailored - in - - .. change:: - :tags: orm - :tickets: 2162 - - query.count() emits "count(*)" instead of - "count(1)". - - .. change:: - :tags: orm - :tickets: 2155 - - Fine tuning of Query clause adaptation when - from_self(), union(), or other "select from - myself" operation, such that plain SQL expression - elements added to filter(), order_by() etc. - which are present in the nested "from myself" - query *will* be adapted in the same way an ORM - expression element will, since these - elements are otherwise not easily accessible. - - .. change:: - :tags: orm - :tickets: 2149 - - Fixed bug where determination of "self referential" - relationship would fail with no workaround - for joined-inh subclass related to itself, - or joined-inh subclass related to a subclass - of that with no cols in the sub-sub class - in the join condition. - Also in 0.6.8. - - .. change:: - :tags: orm - :tickets: 2153 - - mapper() will ignore non-configured foreign keys - to unrelated tables when determining inherit - condition between parent and child class, - but will raise as usual for unresolved - columns and table names regarding the inherited - table. This is an enhanced generalization of - behavior that was already applied to declarative - previously. 0.6.8 has a more - conservative version of this which doesn't - fundamentally alter how join conditions - are determined. - - .. change:: - :tags: orm - :tickets: 2144 - - It is an error to call query.get() when the - given entity is not a single, full class - entity or mapper (i.e. a column). This is - a deprecation warning in 0.6.8. - - .. change:: - :tags: orm - :tickets: 2148 - - Fixed a potential KeyError which under some - circumstances could occur with the identity - map, part of - - .. change:: - :tags: orm - :tickets: - - added Query.with_session() method, switches - Query to use a different session. - - .. change:: - :tags: orm - :tickets: 2131 - - horizontal shard query should use execution - options per connection as per - - .. change:: - :tags: orm - :tickets: 2151 - - a non_primary mapper will inherit the _identity_class - of the primary mapper. This so that a non_primary - established against a class that's normally in an - inheritance mapping will produce results that are - identity-map compatible with that of the primary - mapper (also in 0.6.8) - - .. change:: - :tags: orm - :tickets: 2163 - - Fixed the error message emitted for "can't - execute syncrule for destination column 'q'; - mapper 'X' does not map this column" to - reference the correct mapper. . - Also in 0.6.8. - - .. change:: - :tags: orm - :tickets: 1502 - - polymorphic_union() gets a "cast_nulls" option, - disables the usage of CAST when it renders - the labeled NULL columns. - - .. change:: - :tags: orm - :tickets: - - polymorphic_union() renders the columns in their - original table order, as according to the first - table/selectable in the list of polymorphic - unions in which they appear. (which is itself - an unordered mapping unless you pass an OrderedDict). - - .. change:: - :tags: orm - :tickets: 2171 - - Fixed bug whereby mapper mapped to an anonymous - alias would fail if logging were used, due to - unescaped % sign in the alias name. - Also in 0.6.8. - - .. change:: - :tags: sql - :tickets: 2167 - - Fixed bug whereby nesting a label of a select() - with another label in it would produce incorrect - exported columns. Among other things this would - break an ORM column_property() mapping against - another column_property(). . - Also in 0.6.8 - - .. change:: - :tags: sql - :tickets: - - Changed the handling in determination of join - conditions such that foreign key errors are - only considered between the two given tables. - That is, t1.join(t2) will report FK errors - that involve 't1' or 't2', but anything - involving 't3' will be skipped. This affects - join(), as well as ORM relationship and - inherit condition logic. - - .. change:: - :tags: sql - :tickets: - - Some improvements to error handling inside - of the execute procedure to ensure auto-close - connections are really closed when very - unusual DBAPI errors occur. - - .. change:: - :tags: sql - :tickets: - - metadata.reflect() and reflection.Inspector() - had some reliance on GC to close connections - which were internally procured, fixed this. - - .. change:: - :tags: sql - :tickets: 2140 - - Added explicit check for when Column .name - is assigned as blank string - - .. change:: - :tags: sql - :tickets: 2147 - - Fixed bug whereby if FetchedValue was passed - to column server_onupdate, it would not - have its parent "column" assigned, added - test coverage for all column default assignment - patterns. also in 0.6.8 - - .. change:: - :tags: postgresql - :tickets: - - Fixed the psycopg2_version parsing in the - psycopg2 dialect. - - .. change:: - :tags: postgresql - :tickets: 2141 - - Fixed bug affecting PG 9 whereby index reflection - would fail if against a column whose name - had changed. . Also in 0.6.8. - - .. change:: - :tags: mssql - :tickets: 2169 - - Fixed bug in MSSQL dialect whereby the aliasing - applied to a schema-qualified table would leak - into enclosing select statements. - Also in 0.6.8. - - .. change:: - :tags: documentation - :tickets: 2152 - - Removed the usage of the "collections.MutableMapping" - abc from the ext.mutable docs as it was being used - incorrectly and makes the example more difficult - to understand in any case. - - .. change:: - :tags: examples - :tickets: - - removed the ancient "polymorphic association" - examples and replaced with an updated set of - examples that use declarative mixins, - "generic_associations". Each presents an alternative - table layout. - - .. change:: - :tags: ext - :tickets: 2143 - - Fixed bugs in sqlalchemy.ext.mutable extension where - `None` was not appropriately handled, replacement - events were not appropriately handled. - -.. changelog:: - :version: 0.7.0b4 - :released: Sun Apr 17 2011 - - .. change:: - :tags: general - :tickets: - - Changes to the format of CHANGES, this file. - The format changes have been applied to - the 0.7 releases. - - .. change:: - :tags: general - :tickets: - - The "-declarative" changes will now be listed - directly under the "-orm" section, as these - are closely related. - - .. change:: - :tags: general - :tickets: - - The 0.5 series changes have been moved to - the file CHANGES_PRE_06 which replaces - CHANGES_PRE_05. - - .. change:: - :tags: general - :tickets: - - The changelog for 0.6.7 and subsequent within - the 0.6 series is now listed only in the - CHANGES file within the 0.6 branch. - In the 0.7 CHANGES file (i.e. this file), all the - 0.6 changes are listed inline within the 0.7 - section in which they were also applied - (since all 0.6 changes are in 0.7 as well). - Changes that apply to an 0.6 version here - are noted as are if any differences in - implementation/behavior are present. - - .. change:: - :tags: orm - :tickets: 2122 - - Some fixes to "evaluate" and "fetch" evaluation - when query.update(), query.delete() are called. - The retrieval of records is done after autoflush - in all cases, and before update/delete is - emitted, guarding against unflushed data present - as well as expired objects failing during - the evaluation. - - .. change:: - :tags: orm - :tickets: 2063 - - Reworded the exception raised when a flush - is attempted of a subclass that is not polymorphic - against the supertype. - - .. change:: - :tags: orm - :tickets: - - Still more wording adjustments when a query option - can't find the target entity. Explain that the - path must be from one of the root entities. - - .. change:: - :tags: orm - :tickets: 2123 - - Some fixes to the state handling regarding - backrefs, typically when autoflush=False, where - the back-referenced collection wouldn't - properly handle add/removes with no net - change. Thanks to Richard Murri for the - test case + patch. - (also in 0.6.7). - - .. change:: - :tags: orm - :tickets: 2127 - - Added checks inside the UOW to detect the unusual - condition of being asked to UPDATE or DELETE - on a primary key value that contains NULL - in it. - - .. change:: - :tags: orm - :tickets: 2127 - - Some refinements to attribute history. More - changes are pending possibly in 0.8, but - for now history has been modified such that - scalar history doesn't have a "side effect" - of populating None for a non-present value. - This allows a slightly better ability to - distinguish between a None set and no actual - change, affects as well. - - .. change:: - :tags: orm - :tickets: 2130 - - a "having" clause would be copied from the - inside to the outside query if from_self() - were used; in particular this would break - an 0.7 style count() query. - (also in 0.6.7) - - .. change:: - :tags: orm - :tickets: 2131 - - the Query.execution_options() method now passes - those options to the Connection rather than - the SELECT statement, so that all available - options including isolation level and - compiled cache may be used. - - .. change:: - :tags: sql - :tickets: 2131 - - The "compiled_cache" execution option now raises - an error when passed to a SELECT statement - rather than a Connection. Previously it was - being ignored entirely. We may look into - having this option work on a per-statement - level at some point. - - .. change:: - :tags: sql - :tickets: - - Restored the "catchall" constructor on the base - TypeEngine class, with a deprecation warning. - This so that code which does something like - Integer(11) still succeeds. - - .. change:: - :tags: sql - :tickets: 2104 - - Fixed regression whereby MetaData() coming - back from unpickling did not keep track of - new things it keeps track of now, i.e. - collection of Sequence objects, list - of schema names. - - .. change:: - :tags: sql - :tickets: 2116 - - The limit/offset keywords to select() as well - as the value passed to select.limit()/offset() - will be coerced to integer. - (also in 0.6.7) - - .. change:: - :tags: sql - :tickets: - - fixed bug where "from" clause gathering from an - over() clause would be an itertools.chain() and - not a list, causing "can only concatenate list" - TypeError when combined with other clauses. - - .. change:: - :tags: sql - :tickets: 2134 - - Fixed incorrect usage of "," in over() clause - being placed between the "partition" and "order by" - clauses. - - .. change:: - :tags: sql - :tickets: 2105 - - Before/after attach events for PrimaryKeyConstraint - now function, tests added for before/after events - on all constraint types. - - .. change:: - :tags: sql - :tickets: 2117 - - Added explicit true()/false() constructs to expression - lib - coercion rules will intercept "False"/"True" - into these constructs. In 0.6, the constructs were - typically converted straight to string, which was - no longer accepted in 0.7. - - .. change:: - :tags: engine - :tickets: 2129 - - The C extension is now enabled by default on CPython - 2.x with a fallback to pure python if it fails to - compile. - - .. change:: - :tags: schema - :tickets: 2109 - - The 'useexisting' flag on Table has been superseded - by a new pair of flags 'keep_existing' and - 'extend_existing'. 'extend_existing' is equivalent - to 'useexisting' - the existing Table is returned, - and additional constructor elements are added. - With 'keep_existing', the existing Table is returned, - but additional constructor elements are not added - - these elements are only applied when the Table - is newly created. - - .. change:: - :tags: types - :tickets: 2081 - - REAL has been added to the core types. Supported - by PostgreSQL, SQL Server, MySQL, SQLite. Note - that the SQL Server and MySQL versions, which - add extra arguments, are also still available - from those dialects. - - .. change:: - :tags: types - :tickets: 2106 - - Added @event.listens_for() decorator, given - target + event name, applies the decorated - function as a listener. - - .. change:: - :tags: pool - :tickets: 2103 - - AssertionPool now stores the traceback indicating - where the currently checked out connection was - acquired; this traceback is reported within - the assertion raised upon a second concurrent - checkout; courtesy Gunnlaugur Briem - - .. change:: - :tags: pool - :tickets: - - The "pool.manage" feature doesn't use pickle - anymore to hash the arguments for each pool. - - .. change:: - :tags: sqlite - :tickets: 2115 - - Fixed bug where reflection of foreign key - created as "REFERENCES " without - col name would fail. - (also in 0.6.7) - - .. change:: - :tags: postgresql - :tickets: - - Psycopg2 for Python 3 is now supported. - - .. change:: - :tags: postgresql - :tickets: 2132 - - Fixed support for precision numerics when using - pg8000. - - .. change:: - :tags: oracle - :tickets: 2100 - - Using column names that would require quotes - for the column itself or for a name-generated - bind parameter, such as names with special - characters, underscores, non-ascii characters, - now properly translate bind parameter keys when - talking to cx_oracle. (Also - in 0.6.7) - - .. change:: - :tags: oracle - :tickets: 2116 - - Oracle dialect adds use_binds_for_limits=False - create_engine() flag, will render the LIMIT/OFFSET - values inline instead of as binds, reported to - modify the execution plan used by Oracle. (Also in 0.6.7) - - .. change:: - :tags: documentation - :tickets: 2029 - - Documented SQLite DATE/TIME/DATETIME types. (also in 0.6.7) - - .. change:: - :tags: documentation - :tickets: 2118 - - Fixed mutable extension docs to show the - correct type-association methods. - -.. changelog:: - :version: 0.7.0b3 - :released: Sun Mar 20 2011 - - .. change:: - :tags: general - :tickets: - - Lots of fixes to unit tests when run under PyPy - (courtesy Alex Gaynor). - - .. change:: - :tags: orm - :tickets: 2093 - - Changed the underlying approach to query.count(). - query.count() is now in all cases exactly: - - query. - from_self(func.count(literal_column('1'))). - scalar() - - That is, "select count(1) from ()". - This produces a subquery in all cases, but - vastly simplifies all the guessing count() - tried to do previously, which would still - fail in many scenarios particularly when - joined table inheritance and other joins - were involved. If the subquery produced - for an otherwise very simple count is really - an issue, use query(func.count()) as an - optimization. - - .. change:: - :tags: orm - :tickets: 2087 - - some changes to the identity map regarding - rare weakref callbacks during iterations. - The mutex has been removed as it apparently - can cause a reentrant (i.e. in one thread) deadlock, - perhaps when gc collects objects at the point of - iteration in order to gain more memory. It is hoped - that "dictionary changed during iteration" will - be exceedingly rare as iteration methods internally - acquire the full list of objects in a single values() - call. Note 0.6.7 has a more conservative fix here - which still keeps the mutex in place. - - .. change:: - :tags: orm - :tickets: 2082 - - A tweak to the unit of work causes it to order - the flush along relationship() dependencies even if - the given objects don't have any inter-attribute - references in memory, which was the behavior in - 0.5 and earlier, so a flush of Parent/Child with - only foreign key/primary key set will succeed. - This while still maintaining 0.6 and above's not - generating a ton of useless internal dependency - structures within the flush that don't correspond - to state actually within the current flush. - - .. change:: - :tags: orm - :tickets: 2069 - - Improvements to the error messages emitted when - querying against column-only entities in conjunction - with (typically incorrectly) using loader options, - where the parent entity is not fully present. - - .. change:: - :tags: orm - :tickets: 2098 - - Fixed bug in query.options() whereby a path - applied to a lazyload using string keys could - overlap a same named attribute on the wrong - entity. Note 0.6.7 has a more conservative fix - to this. - - .. change:: - :tags: declarative - :tickets: 2091 - - Arguments in __mapper_args__ that aren't "hashable" - aren't mistaken for always-hashable, possibly-column - arguments. (also in 0.6.7) - - .. change:: - :tags: sql - :tickets: - - Added a fully descriptive error message for the - case where Column is subclassed and _make_proxy() - fails to make a copy due to TypeError on the - constructor. The method _constructor should - be implemented in this case. - - .. change:: - :tags: sql - :tickets: 2095 - - Added new event "column_reflect" for Table objects. - Receives the info dictionary about a Column before - the object is generated within reflection, and allows - modification to the dictionary for control over - most aspects of the resulting Column including - key, name, type, info dictionary. - - .. change:: - :tags: sql - :tickets: - - To help with the "column_reflect" event being used - with specific Table objects instead of all instances - of Table, listeners can be added to a Table object - inline with its construction using a new argument - "listeners", a list of tuples of the form - (, ), which are applied to the Table - before the reflection process begins. - - .. change:: - :tags: sql - :tickets: 2085 - - Added new generic function "next_value()", accepts - a Sequence object as its argument and renders the - appropriate "next value" generation string on the - target platform, if supported. Also provides - ".next_value()" method on Sequence itself. - - .. change:: - :tags: sql - :tickets: 2084 - - func.next_value() or other SQL expression can - be embedded directly into an insert() construct, - and if implicit or explicit "returning" is used - in conjunction with a primary key column, - the newly generated value will be present in - result.inserted_primary_key. - - .. change:: - :tags: sql - :tickets: 2089 - - Added accessors to ResultProxy "returns_rows", - "is_insert" (also in 0.6.7) - - .. change:: - :tags: engine - :tickets: 2097 - - Fixed AssertionPool regression bug. - - .. change:: - :tags: engine - :tickets: 2060 - - Changed exception raised to ArgumentError when an - invalid dialect is specified. - - .. change:: - :tags: postgresql - :tickets: 2092 - - Added RESERVED_WORDS for postgresql dialect. - (also in 0.6.7) - - .. change:: - :tags: postgresql - :tickets: 2073 - - Fixed the BIT type to allow a "length" parameter, "varying" - parameter. Reflection also fixed. - (also in 0.6.7) - - .. change:: - :tags: mssql - :tickets: 2071 - - Rewrote the query used to get the definition of a view, - typically when using the Inspector interface, to - use sys.sql_modules instead of the information schema, - thereby allowing views definitions longer than 4000 - characters to be fully returned. - (also in 0.6.7) - - .. change:: - :tags: firebird - :tickets: 2083 - - The "implicit_returning" flag on create_engine() is - honored if set to False. (also in 0.6.7) - - .. change:: - :tags: informix - :tickets: 2092 - - Added RESERVED_WORDS informix dialect. - (also in 0.6.7) - - .. change:: - :tags: ext - :tickets: 2090 - - The horizontal_shard ShardedSession class accepts the common - Session argument "query_cls" as a constructor argument, - to enable further subclassing of ShardedQuery. (also in 0.6.7) - - .. change:: - :tags: examples - :tickets: - - Updated the association, association proxy examples - to use declarative, added a new example - dict_of_sets_with_default.py, a "pushing the envelope" - example of association proxy. - - .. change:: - :tags: examples - :tickets: 2090 - - The Beaker caching example allows a "query_cls" argument - to the query_callable() function. - (also in 0.6.7) - -.. changelog:: - :version: 0.7.0b2 - :released: Sat Feb 19 2011 - - .. change:: - :tags: orm - :tickets: 2053 - - Fixed bug whereby Session.merge() would call the - load() event with one too few arguments. - - .. change:: - :tags: orm - :tickets: 2052 - - Added logic which prevents the generation of - events from a MapperExtension or SessionExtension - from generating do-nothing events for all the methods - not overridden. - - .. change:: - :tags: declarative - :tickets: 2058 - - Fixed regression whereby composite() with - Column objects placed inline would fail - to initialize. The Column objects can now - be inline with the composite() or external - and pulled in via name or object ref. - - .. change:: - :tags: declarative - :tickets: 2061 - - Fix error message referencing old @classproperty - name to reference @declared_attr - (also in 0.6.7) - - .. change:: - :tags: declarative - :tickets: 1468 - - the dictionary at the end of the __table_args__ - tuple is now optional. - - .. change:: - :tags: sql - :tickets: 2059 - - Renamed the EngineEvents event class to - ConnectionEvents. As these classes are never - accessed directly by end-user code, this strictly - is a documentation change for end users. Also - simplified how events get linked to engines - and connections internally. - - .. change:: - :tags: sql - :tickets: 2055 - - The Sequence() construct, when passed a MetaData() - object via its 'metadata' argument, will be - included in CREATE/DROP statements within - metadata.create_all() and metadata.drop_all(), - including "checkfirst" logic. - - .. change:: - :tags: sql - :tickets: 2064 - - The Column.references() method now returns True - if it has a foreign key referencing the - given column exactly, not just its parent - table. - - .. change:: - :tags: postgresql - :tickets: 2065 - - Fixed regression from 0.6 where SMALLINT and - BIGINT types would both generate SERIAL - on an integer PK column, instead of - SMALLINT and BIGSERIAL - - .. change:: - :tags: ext - :tickets: 2054 - - Association proxy now has correct behavior for - any(), has(), and contains() when proxying - a many-to-one scalar attribute to a one-to-many - collection (i.e. the reverse of the 'typical' - association proxy use case) - - .. change:: - :tags: examples - :tickets: - - Beaker example now takes into account 'limit' - and 'offset', bind params within embedded - FROM clauses (like when you use union() or - from_self()) when generating a cache key. - -.. changelog:: - :version: 0.7.0b1 - :released: Sat Feb 12 2011 - - .. change:: - :tags: - :tickets: - - Detailed descriptions of each change below are - described at: - https://docs.sqlalchemy.org/en/latest/changelog/migration_07.html - - .. change:: - :tags: general - :tickets: 1902 - - New event system, supersedes all extensions, listeners, - etc. - - .. change:: - :tags: general - :tickets: 1926 - - Logging enhancements - - .. change:: - :tags: general - :tickets: 1949 - - Setup no longer installs a Nose plugin - - .. change:: - :tags: general - :tickets: - - The "sqlalchemy.exceptions" alias in sys.modules - has been removed. Base SQLA exceptions are - available via "from sqlalchemy import exc". - The "exceptions" alias for "exc" remains in - "sqlalchemy" for now, it's just not patched into - sys.modules. - - .. change:: - :tags: orm - :tickets: 1923 - - More succinct form of query.join(target, onclause) - - .. change:: - :tags: orm - :tickets: 1903 - - Hybrid Attributes, implements/supersedes synonym() - - .. change:: - :tags: orm - :tickets: 2008 - - Rewrite of composites - - .. change:: - :tags: orm - :tickets: - - Mutation Event Extension, supersedes "mutable=True" - - .. seealso:: - - :ref:`07_migration_mutation_extension` - - .. change:: - :tags: orm - :tickets: 1980 - - PickleType and ARRAY mutability turned off by default - - .. change:: - :tags: orm - :tickets: 1895 - - Simplified polymorphic_on assignment - - .. change:: - :tags: orm - :tickets: 1912 - - Flushing of Orphans that have no parent is allowed - - .. change:: - :tags: orm - :tickets: 2041 - - Adjusted flush accounting step to occur before - the commit in the case of autocommit=True. This allows - autocommit=True to work appropriately with - expire_on_commit=True, and also allows post-flush session - hooks to operate in the same transactional context - as when autocommit=False. - - .. change:: - :tags: orm - :tickets: 1973 - - Warnings generated when collection members, scalar referents - not part of the flush - - .. change:: - :tags: orm - :tickets: 1876 - - Non-`Table`-derived constructs can be mapped - - .. change:: - :tags: orm - :tickets: 1942 - - Tuple label names in Query Improved - - .. change:: - :tags: orm - :tickets: 1892 - - Mapped column attributes reference the most specific - column first - - .. change:: - :tags: orm - :tickets: 1896 - - Mapping to joins with two or more same-named columns - requires explicit declaration - - .. change:: - :tags: orm - :tickets: 1875 - - Mapper requires that polymorphic_on column be present - in the mapped selectable - - .. change:: - :tags: orm - :tickets: 1966 - - compile_mappers() renamed configure_mappers(), simplified - configuration internals - - .. change:: - :tags: orm - :tickets: 2018 - - the aliased() function, if passed a SQL FromClause element - (i.e. not a mapped class), will return element.alias() - instead of raising an error on AliasedClass. - - .. change:: - :tags: orm - :tickets: 2027 - - Session.merge() will check the version id of the incoming - state against that of the database, assuming the mapping - uses version ids and incoming state has a version_id - assigned, and raise StaleDataError if they don't - match. - - .. change:: - :tags: orm - :tickets: 1996 - - Session.connection(), Session.execute() accept 'bind', - to allow execute/connection operations to participate - in the open transaction of an engine explicitly. - - .. change:: - :tags: orm - :tickets: - - Query.join(), Query.outerjoin(), eagerload(), - eagerload_all(), others no longer allow lists - of attributes as arguments (i.e. option([x, y, z]) - form, deprecated since 0.5) - - .. change:: - :tags: orm - :tickets: - - ScopedSession.mapper is removed (deprecated since 0.5). - - .. change:: - :tags: orm - :tickets: 2031 - - Horizontal shard query places 'shard_id' in - context.attributes where it's accessible by the - "load()" event. - - .. change:: - :tags: orm - :tickets: 2032 - - A single contains_eager() call across - multiple entities will indicate all collections - along that path should load, instead of requiring - distinct contains_eager() calls for each endpoint - (which was never correctly documented). - - .. change:: - :tags: orm - :tickets: - - The "name" field used in orm.aliased() now renders - in the resulting SQL statement. - - .. change:: - :tags: orm - :tickets: 1473 - - Session weak_instance_dict=False is deprecated. - - .. change:: - :tags: orm - :tickets: 2046 - - An exception is raised in the unusual case that an - append or similar event on a collection occurs after - the parent object has been dereferenced, which - prevents the parent from being marked as "dirty" - in the session. Was a warning in 0.6.6. - - .. change:: - :tags: orm - :tickets: 1069 - - Query.distinct() now accepts column expressions - as \*args, interpreted by the PostgreSQL dialect - as DISTINCT ON (). - - .. change:: - :tags: orm - :tickets: 2049 - - Additional tuning to "many-to-one" relationship - loads during a flush(). A change in version 0.6.6 - ([ticket:2002]) required that more "unnecessary" m2o - loads during a flush could occur. Extra loading modes have - been added so that the SQL emitted in this - specific use case is trimmed back, while still - retrieving the information the flush needs in order - to not miss anything. - - .. change:: - :tags: orm - :tickets: - - the value of "passive" as passed to - attributes.get_history() should be one of the - constants defined in the attributes package. Sending - True or False is deprecated. - - .. change:: - :tags: orm - :tickets: 2030 - - Added a `name` argument to `Query.subquery()`, to allow - a fixed name to be assigned to the alias object. (also in 0.6.7) - - .. change:: - :tags: orm - :tickets: 2019 - - A warning is emitted when a joined-table inheriting mapper - has no primary keys on the locally mapped table - (but has pks on the superclass table). - (also in 0.6.7) - - .. change:: - :tags: orm - :tickets: 2038 - - Fixed bug where "middle" class in a polymorphic hierarchy - would have no 'polymorphic_on' column if it didn't also - specify a 'polymorphic_identity', leading to strange - errors upon refresh, wrong class loaded when querying - from that target. Also emits the correct WHERE criterion - when using single table inheritance. - (also in 0.6.7) - - .. change:: - :tags: orm - :tickets: 1995 - - Fixed bug where a column with a SQL or server side default - that was excluded from a mapping with include_properties - or exclude_properties would result in UnmappedColumnError. (also in 0.6.7) - - .. change:: - :tags: orm - :tickets: 2046 - - A warning is emitted in the unusual case that an - append or similar event on a collection occurs after - the parent object has been dereferenced, which - prevents the parent from being marked as "dirty" - in the session. This will be an exception in 0.7. (also in 0.6.7) - - .. change:: - :tags: declarative - :tickets: 2050 - - Added an explicit check for the case that the name - 'metadata' is used for a column attribute on a - declarative class. (also in 0.6.7) - - .. change:: - :tags: sql - :tickets: 1844 - - Added over() function, method to FunctionElement - classes, produces the _Over() construct which - in turn generates "window functions", i.e. - " OVER (PARTITION BY , - ORDER BY )". - - .. change:: - :tags: sql - :tickets: 805 - - LIMIT/OFFSET clauses now use bind parameters - - .. change:: - :tags: sql - :tickets: 1069 - - select.distinct() now accepts column expressions - as \*args, interpreted by the PostgreSQL dialect - as DISTINCT ON (). Note this was already - available via passing a list to the `distinct` - keyword argument to select(). - - .. change:: - :tags: sql - :tickets: - - select.prefix_with() accepts multiple expressions - (i.e. \*expr), 'prefix' keyword argument to select() - accepts a list or tuple. - - .. change:: - :tags: sql - :tickets: - - Passing a string to the `distinct` keyword argument - of `select()` for the purpose of emitting special - MySQL keywords (DISTINCTROW etc.) is deprecated - - use `prefix_with()` for this. - - .. change:: - :tags: sql - :tickets: 2006, 2005 - - TypeDecorator works with primary key columns - - .. change:: - :tags: sql - :tickets: 1897 - - DDL() constructs now escape percent signs - - .. change:: - :tags: sql - :tickets: 1917, 1893 - - Table.c / MetaData.tables refined a bit, don't allow direct - mutation - - .. change:: - :tags: sql - :tickets: 1950 - - Callables passed to `bindparam()` don't get evaluated - - .. change:: - :tags: sql - :tickets: 1870 - - types.type_map is now private, types._type_map - - .. change:: - :tags: sql - :tickets: 1982 - - Non-public Pool methods underscored - - .. change:: - :tags: sql - :tickets: 723 - - Added NULLS FIRST and NULLS LAST support. It's implemented - as an extension to the asc() and desc() operators, called - nullsfirst() and nullslast(). - - .. change:: - :tags: sql - :tickets: - - The Index() construct can be created inline with a Table - definition, using strings as column names, as an alternative - to the creation of the index outside of the Table. - - .. change:: - :tags: sql - :tickets: 2001 - - execution_options() on Connection accepts - "isolation_level" argument, sets transaction isolation - level for that connection only until returned to the - connection pool, for those backends which support it - (SQLite, PostgreSQL) - - .. change:: - :tags: sql - :tickets: 2005 - - A TypeDecorator of Integer can be used with a primary key - column, and the "autoincrement" feature of various dialects - as well as the "sqlite_autoincrement" flag will honor - the underlying database type as being Integer-based. - - .. change:: - :tags: sql - :tickets: 2020, 2021 - - Established consistency when server_default is present - on an Integer PK column. SQLA doesn't pre-fetch these, - nor do they come back in cursor.lastrowid (DBAPI). - Ensured all backends consistently return None - in result.inserted_primary_key for these. Regarding - reflection for this case, reflection of an int PK col - with a server_default sets the "autoincrement" flag to False, - except in the case of a PG SERIAL col where we detected a - sequence default. - - .. change:: - :tags: sql - :tickets: 2006 - - Result-row processors are applied to pre-executed SQL - defaults, as well as cursor.lastrowid, when determining - the contents of result.inserted_primary_key. - - .. change:: - :tags: sql - :tickets: - - Bind parameters present in the "columns clause" of a select - are now auto-labeled like other "anonymous" clauses, - which among other things allows their "type" to be meaningful - when the row is fetched, as in result row processors. - - .. change:: - :tags: sql - :tickets: - - TypeDecorator is present in the "sqlalchemy" import space. - - .. change:: - :tags: sql - :tickets: 2015 - - Non-DBAPI errors which occur in the scope of an `execute()` - call are now wrapped in sqlalchemy.exc.StatementError, - and the text of the SQL statement and repr() of params - is included. This makes it easier to identify statement - executions which fail before the DBAPI becomes - involved. - - .. change:: - :tags: sql - :tickets: 2048 - - The concept of associating a ".bind" directly with a - ClauseElement has been explicitly moved to Executable, - i.e. the mixin that describes ClauseElements which represent - engine-executable constructs. This change is an improvement - to internal organization and is unlikely to affect any - real-world usage. - - .. change:: - :tags: sql - :tickets: 2028 - - Column.copy(), as used in table.tometadata(), copies the - 'doc' attribute. (also in 0.6.7) - - .. change:: - :tags: sql - :tickets: 2023 - - Added some defs to the resultproxy.c extension so that - the extension compiles and runs on Python 2.4. (also in 0.6.7) - - .. change:: - :tags: sql - :tickets: 2042 - - The compiler extension now supports overriding the default - compilation of expression._BindParamClause including that - the auto-generated binds within the VALUES/SET clause - of an insert()/update() statement will also use the new - compilation rules. (also in 0.6.7) - - .. change:: - :tags: sql - :tickets: 1921 - - SQLite dialect now uses `NullPool` for file-based databases - - .. change:: - :tags: sql - :tickets: 2036 - - The path given as the location of a sqlite database is now - normalized via os.path.abspath(), so that directory changes - within the process don't affect the ultimate location - of a relative file path. - - .. change:: - :tags: postgresql - :tickets: 1083 - - When explicit sequence execution derives the name - of the auto-generated sequence of a SERIAL column, - which currently only occurs if implicit_returning=False, - now accommodates if the table + column name is greater - than 63 characters using the same logic PostgreSQL uses. (also in 0.6.7) - - .. change:: - :tags: postgresql - :tickets: 2044 - - Added an additional libpq message to the list of "disconnect" - exceptions, "could not receive data from server" (also in 0.6.7) - - .. change:: - :tags: mssql - :tickets: 1833 - - the String/Unicode types, and their counterparts VARCHAR/ - NVARCHAR, emit "max" as the length when no length is - specified, so that the default length, normally '1' - as per SQL server documentation, is instead - 'unbounded'. This also occurs for the VARBINARY type.. - - This behavior makes these types more closely compatible - with PostgreSQL's VARCHAR type which is similarly unbounded - when no length is specified. - - .. change:: - :tags: mysql - :tickets: 1991 - - New DBAPI support for pymysql, a pure Python port - of MySQL-python. - - .. change:: - :tags: mysql - :tickets: 2047 - - oursql dialect accepts the same "ssl" arguments in - create_engine() as that of MySQLdb. - (also in 0.6.7) - - .. change:: - :tags: firebird - :tickets: 1885 - - Some adjustments so that Interbase is supported as well. - FB/Interbase version idents are parsed into a structure - such as (8, 1, 1, 'interbase') or (2, 1, 588, 'firebird') - so they can be distinguished. diff --git a/doc/build/changelog/changelog_08.rst b/doc/build/changelog/changelog_08.rst deleted file mode 100644 index 363f5aeb1b..0000000000 --- a/doc/build/changelog/changelog_08.rst +++ /dev/null @@ -1,3740 +0,0 @@ -============= -0.8 Changelog -============= - -.. changelog_imports:: - - .. include:: changelog_07.rst - :start-line: 5 - - -.. changelog:: - :version: 0.8.7 - :released: July 22, 2014 - - .. change:: - :tags: bug, mssql - :versions: 1.0.0b1, 0.9.7 - - Added statement encoding to the "SET IDENTITY_INSERT" - statements which operate when an explicit INSERT is being - interjected into an IDENTITY column, to support non-ascii table - identifiers on drivers such as pyodbc + unix + py2k that don't - support unicode statements. - - .. change:: - :tags: bug, mssql - :versions: 1.0.0b1, 0.9.7 - :tickets: 3091 - - In the SQL Server pyodbc dialect, repaired the implementation - for the ``description_encoding`` dialect parameter, which when - not explicitly set was preventing cursor.description from - being parsed correctly in the case of result sets that - contained names in alternate encodings. This parameter - shouldn't be needed going forward. - - .. change:: - :tags: bug, sql - :versions: 1.0.0b1, 0.9.7 - :tickets: 3124 - - Fixed bug in :class:`.Enum` and other :class:`.SchemaType` - subclasses where direct association of the type with a - :class:`_schema.MetaData` would lead to a hang when events - (like create events) were emitted on the :class:`_schema.MetaData`. - - .. change:: - :tags: bug, sql - :versions: 1.0.0b1, 0.9.7 - :tickets: 3102 - - Fixed a bug within the custom operator plus :meth:`.TypeEngine.with_variant` - system, whereby using a :class:`.TypeDecorator` in conjunction with - variant would fail with an MRO error when a comparison operator was used. - - .. change:: - :tags: bug, mysql - :versions: 1.0.0b1, 0.9.7 - :tickets: 3101 - - MySQL error 2014 "commands out of sync" appears to be raised as a - ProgrammingError, not OperationalError, in modern MySQL-Python versions; - all MySQL error codes that are tested for "is disconnect" are now - checked within OperationalError and ProgrammingError regardless. - - .. change:: - :tags: bug, mysql - :versions: 1.0.0b1, 0.9.5 - :tickets: 3085 - - Fixed bug where column names added to ``mysql_length`` parameter - on an index needed to have the same quoting for quoted names in - order to be recognized. The fix makes the quotes optional but - also provides the old behavior for backwards compatibility with those - using the workaround. - - .. change:: - :tags: bug, declarative - :versions: 1.0.0b1, 0.9.5 - :tickets: 3062 - - The ``__mapper_args__`` dictionary is copied from a declarative - mixin or abstract class when accessed, so that modifications made - to this dictionary by declarative itself won't conflict with that - of other mappings. The dictionary is modified regarding the - ``version_id_col`` and ``polymorphic_on`` arguments, replacing the - column within with the one that is officially mapped to the local - class/table. - - .. change:: - :tags: bug, sql - :versions: 0.9.5, 1.0.0b1 - :tickets: 3044 - - Fixed bug in INSERT..FROM SELECT construct where selecting from a - UNION would wrap the union in an anonymous (e.g. unlabeled) subquery. - - .. change:: - :tags: bug, postgresql - :versions: 0.9.5, 1.0.0b1 - :tickets: 3053 - - Added the ``hashable=False`` flag to the PG :class:`.HSTORE` type, which - is needed to allow the ORM to skip over trying to "hash" an ORM-mapped - HSTORE column when requesting it in a mixed column/entity list. - Patch courtesy Gunnlaugur Þór Briem. - - .. change:: - :tags: bug, orm - :versions: 0.9.5, 1.0.0b1 - :tickets: 3055 - - Fixed bug in subquery eager loading where a long chain of - eager loads across a polymorphic-subclass boundary in conjunction - with polymorphic loading would fail to locate the subclass-link in the - chain, erroring out with a missing property name on an - :class:`.AliasedClass`. - - .. change:: - :tags: bug, ext - :versions: 0.9.5, 1.0.0b1 - :tickets: 3051, 3093 - - Fixed bug in mutable extension where :class:`.MutableDict` did not - report change events for the ``setdefault()`` dictionary operation. - - .. change:: - :tags: bug, ext - :versions: 0.9.5, 1.0.0b1 - :tickets: 3093, 3051 - - Fixed bug where :meth:`.MutableDict.setdefault` didn't return the - existing or new value (this bug was not released in any 0.8 version). - Pull request courtesy Thomas Hervé. - - .. change:: - :tags: bug, mysql - :versions: 0.9.5, 1.0.0b1 - - Added support for reflecting tables where an index includes - KEY_BLOCK_SIZE using an equal sign. Pull request courtesy - Sean McGivern. - - .. change:: - :tags: bug, orm - :tickets: 3047 - :versions: 0.9.5, 1.0.0b1 - - Fixed ORM bug where the :func:`.class_mapper` function would mask - AttributeErrors or KeyErrors that should raise during mapper - configuration due to user errors. The catch for attribute/keyerror - has been made more specific to not include the configuration step. - - .. change:: - :tags: bug, sql - :tickets: 3045 - :versions: 0.9.5, 1.0.0b1 - - Fixed bug where :meth:`_schema.Table.update` and :meth:`_schema.Table.delete` - would produce an empty WHERE clause when an empty :func:`.and_()` - or :func:`.or_()` or other blank expression were applied. This is - now consistent with that of :func:`_expression.select`. - - .. change:: - :tags: bug, postgresql - :versions: 0.9.5, 1.0.0b1 - - Added a new "disconnect" message "connection has been closed unexpectedly". - This appears to be related to newer versions of SSL. - Pull request courtesy Antti Haapala. - -.. changelog:: - :version: 0.8.6 - :released: March 28, 2014 - - .. change:: - :tags: bug, orm - :tickets: 3006 - :versions: 0.9.4 - - Fixed ORM bug where changing the primary key of an object, then marking - it for DELETE would fail to target the correct row for DELETE. - - .. change:: - :tags: feature, postgresql - :versions: 0.9.4 - - Enabled "sane multi-row count" checking for the psycopg2 DBAPI, as - this seems to be supported as of psycopg2 2.0.9. - - .. change:: - :tags: bug, postgresql - :tickets: 3000 - :versions: 0.9.4 - - Fixed regression caused by release 0.8.5 / 0.9.3's compatibility - enhancements where index reflection on PostgreSQL versions specific - to only the 8.1, 8.2 series again - broke, surrounding the ever problematic int2vector type. While - int2vector supports array operations as of 8.1, apparently it only - supports CAST to a varchar as of 8.3. - - .. change:: - :tags: bug, orm - :tickets: 2995, - :versions: 0.9.4 - - Fixed regression from 0.8.3 as a result of :ticket:`2818` - where :meth:`_query.Query.exists` wouldn't work on a query that only - had a :meth:`_query.Query.select_from` entry but no other entities. - - .. change:: - :tags: bug, general - :tickets: 2986 - :versions: 0.9.4 - - Adjusted ``setup.py`` file to support the possible future - removal of the ``setuptools.Feature`` extension from setuptools. - If this keyword isn't present, the setup will still succeed - with setuptools rather than falling back to distutils. C extension - building can be disabled now also by setting the - DISABLE_SQLALCHEMY_CEXT environment variable. This variable works - whether or not setuptools is even available. - - .. change:: - :tags: bug, ext - :versions: 0.9.4 - :tickets: 2997 - - Fixed bug in mutable extension as well as - :func:`.attributes.flag_modified` where the change event would not be - propagated if the attribute had been reassigned to itself. - - .. change:: - :tags: bug, orm - :versions: 0.9.4 - - Improved an error message which would occur if a query() were made - against a non-selectable, such as a :func:`_expression.literal_column`, and then - an attempt was made to use :meth:`_query.Query.join` such that the "left" - side would be determined as ``None`` and then fail. This condition - is now detected explicitly. - - .. change:: - :tags: bug, sql - :versions: 0.9.4 - :tickets: 2977 - - Fixed bug in :func:`.tuple_` construct where the "type" of essentially - the first SQL expression would be applied as the "comparison type" - to a compared tuple value; this has the effect in some cases of an - inappropriate "type coercion" occurring, such as when a tuple that - has a mix of String and Binary values improperly coerces target - values to Binary even though that's not what they are on the left - side. :func:`.tuple_` now expects heterogeneous types within its - list of values. - - .. change:: - :tags: orm, bug - :versions: 0.9.4 - :tickets: 2975 - - Removed stale names from ``sqlalchemy.orm.interfaces.__all__`` and - refreshed with current names, so that an ``import *`` from this - module again works. - -.. changelog:: - :version: 0.8.5 - :released: February 19, 2014 - - .. change:: - :tags: postgresql, bug - :versions: 0.9.3 - :tickets: 2936 - - Added an additional message to psycopg2 disconnect detection, - "could not send data to server", which complements the existing - "could not receive data from server" and has been observed by users. - - .. change:: - :tags: postgresql, bug - :versions: 0.9.3 - - Support has been improved for PostgreSQL reflection behavior on very old - (pre 8.1) versions of PostgreSQL, and potentially other PG engines - such as Redshift (assuming Redshift reports the version as < 8.1). - The query for "indexes" as well as "primary keys" relies upon inspecting - a so-called "int2vector" datatype, which refuses to coerce to an array - prior to 8.1 causing failures regarding the "ANY()" operator used - in the query. Extensive googling has located the very hacky, but - recommended-by-PG-core-developer query to use when PG version < 8.1 - is in use, so index and primary key constraint reflection now work - on these versions. - - - .. change:: - :tags: feature, mysql - :versions: 0.9.3 - :tickets: 2941 - - Added new MySQL-specific :class:`.mysql.DATETIME` which includes - fractional seconds support; also added fractional seconds support - to :class:`.mysql.TIMESTAMP`. DBAPI support is limited, though - fractional seconds are known to be supported by MySQL Connector/Python. - Patch courtesy Geert JM Vanderkelen. - - .. change:: - :tags: bug, mysql - :versions: 0.9.3 - :tickets: 2966 - - Added support for the ``PARTITION BY`` and ``PARTITIONS`` - MySQL table keywords, specified as ``mysql_partition_by='value'`` and - ``mysql_partitions='value'`` to :class:`_schema.Table`. Pull request - courtesy Marcus McCurdy. - - .. change:: - :tags: bug, sql - :versions: 0.9.3 - :tickets: 2944 - - Fixed bug where calling :meth:`_expression.Insert.values` with an empty list - or tuple would raise an IndexError. It now produces an empty - insert construct as would be the case with an empty dictionary. - - .. change:: - :tags: bug, engine, pool - :versions: 0.9.3 - :tickets: 2880, 2964 - - Fixed a critical regression caused by :ticket:`2880` where the newly - concurrent ability to return connections from the pool means that the - "first_connect" event is now no longer synchronized either, thus leading - to dialect mis-configurations under even minimal concurrency situations. - - .. change:: - :tags: bug, sqlite - - Restored a change that was missed in the backport of unique - constraint reflection to 0.8, where :class:`.UniqueConstraint` - with SQLite would fail if reserved keywords were included in the - names of columns. Pull request courtesy Roman Podolyaka. - - .. change:: - :tags: bug, postgresql - :tickets: 2291 - :versions: 0.9.3 - - Revised this very old issue where the PostgreSQL "get primary key" - reflection query were updated to take into account primary key constraints - that were renamed; the newer query fails on very old versions of - PostgreSQL such as version 7, so the old query is restored in those cases - when server_version_info < (8, 0) is detected. - - .. change:: - :tags: bug, sql - :tickets: 2957 - :versions: 0.9.3 - - Fixed bug where :meth:`.ColumnOperators.in_` would go into an endless - loop if erroneously passed a column expression whose comparator - included the ``__getitem__()`` method, such as a column that uses the - :class:`_postgresql.ARRAY` type. - - .. change:: - :tags: bug, orm - :tickets: 2951 - :versions: 0.9.3 - - Fixed bug where :meth:`_query.Query.get` would fail to consistently - raise the :class:`.InvalidRequestError` that invokes when called - on a query with existing criterion, when the given identity is - already present in the identity map. - - .. change:: - :tags: bug, mysql - :tickets: 2933 - :versions: 0.9.3 - - Fixed bug which prevented MySQLdb-based dialects (e.g. - pymysql) from working in Py3K, where a check for "connection - charset" would fail due to Py3K's more strict value comparison - rules. The call in question wasn't taking the database - version into account in any case as the server version was - still None at that point, so the method overall has been - simplified to rely upon connection.character_set_name(). - - .. change:: - :tags: bug, mysql - :versions: 0.9.2 - - Some missing methods added to the cymysql dialect, including - _get_server_version_info() and _detect_charset(). Pullreq - courtesy Hajime Nakagami. - - .. change:: - :tags: bug, py3k - - Fixed Py3K bug where a missing import would cause "literal binary" - mode to fail to import "util.binary_type" when rendering a bound - parameter. 0.9 handles this differently. Pull request courtesy - Andreas Zeidler. - - .. change:: - :tags: bug, orm - :versions: 0.9.2 - - Fixed error message when an iterator object is passed to - :func:`.class_mapper` or similar, where the error would fail to - render on string formatting. Pullreq courtesy Kyle Stark. - - .. change:: - :tags: bug, firebird - :versions: 0.9.0 - :tickets: 2897 - - The firebird dialect will quote identifiers which begin with an - underscore. Courtesy Treeve Jelbert. - - .. change:: - :tags: bug, firebird - :versions: 0.9.0 - - Fixed bug in Firebird index reflection where the columns within the - index were not sorted correctly; they are now sorted - in order of RDB$FIELD_POSITION. - - .. change:: - :tags: bug, mssql, firebird - :versions: 0.9.0 - - The "asdecimal" flag used with the :class:`.Float` type will now - work with Firebird as well as the mssql+pyodbc dialects; previously the - decimal conversion was not occurring. - - .. change:: - :tags: bug, mssql, pymssql - :versions: 0.9.0 - - Added "Net-Lib error during Connection reset by peer" message - to the list of messages checked for "disconnect" within the - pymssql dialect. Courtesy John Anderson. - - .. change:: - :tags: bug, sql - :versions: 0.9.0 - :tickets: 2896 - - Fixed issue where a primary key column that has a Sequence on it, - yet the column is not the "auto increment" column, either because - it has a foreign key constraint or ``autoincrement=False`` set, - would attempt to fire the Sequence on INSERT for backends that don't - support sequences, when presented with an INSERT missing the primary - key value. This would take place on non-sequence backends like - SQLite, MySQL. - - .. change:: - :tags: bug, sql - :versions: 0.9.0 - :tickets: 2895 - - Fixed bug with :meth:`_expression.Insert.from_select` method where the order - of the given names would not be taken into account when generating - the INSERT statement, thus producing a mismatch versus the column - names in the given SELECT statement. Also noted that - :meth:`_expression.Insert.from_select` implies that Python-side insert defaults - cannot be used, since the statement has no VALUES clause. - - .. change:: - :tags: enhancement, sql - :versions: 0.9.0 - - The exception raised when a :class:`.BindParameter` is present - in a compiled statement without a value now includes the key name - of the bound parameter in the error message. - - .. change:: - :tags: bug, orm - :versions: 0.9.0 - :tickets: 2887 - - An adjustment to the :func:`.subqueryload` strategy which ensures that - the query runs after the loading process has begun; this is so that - the subqueryload takes precedence over other loaders that may be - hitting the same attribute due to other eager/noload situations - at the wrong time. - - .. change:: - :tags: bug, orm - :versions: 0.9.0 - :tickets: 2885 - - Fixed bug when using joined table inheritance from a table to a - select/alias on the base, where the PK columns were also not same - named; the persistence system would fail to copy primary key values - from the base table to the inherited table upon INSERT. - - .. change:: - :tags: bug, orm - :versions: 0.9.0 - :tickets: 2889 - - :func:`.composite` will raise an informative error message when the - columns/attribute (names) passed don't resolve to a Column or mapped - attribute (such as an erroneous tuple); previously raised an unbound - local. - - .. change:: - :tags: bug, declarative - :versions: 0.9.0 - :tickets: 2888 - - Error message when a string arg sent to :func:`_orm.relationship` which - doesn't resolve to a class or mapper has been corrected to work - the same way as when a non-string arg is received, which indicates - the name of the relationship which had the configurational error. - -.. changelog:: - :version: 0.8.4 - :released: December 8, 2013 - - .. change:: - :tags: bug, engine - :versions: 0.9.0 - :tickets: 2881 - - A DBAPI that raises an error on ``connect()`` which is not a subclass - of dbapi.Error (such as ``TypeError``, ``NotImplementedError``, etc.) - will propagate the exception unchanged. Previously, - the error handling specific to the ``connect()`` routine would both - inappropriately run the exception through the dialect's - :meth:`.Dialect.is_disconnect` routine as well as wrap it in - a :class:`sqlalchemy.exc.DBAPIError`. It is now propagated unchanged - in the same way as occurs within the execute process. - - .. change:: - :tags: bug, engine, pool - :versions: 0.9.0 - :tickets: 2880 - - The :class:`.QueuePool` has been enhanced to not block new connection - attempts when an existing connection attempt is blocking. Previously, - the production of new connections was serialized within the block - that monitored overflow; the overflow counter is now altered within - its own critical section outside of the connection process itself. - - .. change:: - :tags: bug, engine, pool - :versions: 0.9.0 - :tickets: 2522 - - Made a slight adjustment to the logic which waits for a pooled - connection to be available, such that for a connection pool - with no timeout specified, it will every half a second break out of - the wait to check for the so-called "abort" flag, which allows the - waiter to break out in case the whole connection pool was dumped; - normally the waiter should break out due to a notify_all() but it's - possible this notify_all() is missed in very slim cases. - This is an extension of logic first introduced in 0.8.0, and the - issue has only been observed occasionally in stress tests. - - .. change:: - :tags: bug, mssql - :versions: 0.9.0 - - Fixed bug introduced in 0.8.0 where the ``DROP INDEX`` - statement for an index in MSSQL would render incorrectly if the - index were in an alternate schema; the schemaname/tablename - would be reversed. The format has been also been revised to - match current MSSQL documentation. Courtesy Derek Harland. - - .. change:: - :tags: feature, sql - :tickets: 1443 - :versions: 0.9.0b1 - - Added support for "unique constraint" reflection, via the - :meth:`_reflection.Inspector.get_unique_constraints` method. - Thanks for Roman Podolyaka for the patch. - - .. change:: - :tags: bug, oracle - :tickets: 2864 - :versions: 0.9.0 - - Added ORA-02396 "maximum idle time" error code to list of - "is disconnect" codes with cx_oracle. - - .. change:: - :tags: bug, engine - :tickets: 2871 - :versions: 0.9.0 - - Fixed bug where SQL statement would be improperly ASCII-encoded - when a pre-DBAPI :class:`.StatementError` were raised within - :meth:`_engine.Connection.execute`, causing encoding errors for - non-ASCII statements. The stringification now remains within - Python unicode thus avoiding encoding errors. - - .. change:: - :tags: bug, oracle - :tickets: 2870 - :versions: 0.9.0 - - Fixed bug where Oracle ``VARCHAR`` types given with no length - (e.g. for a ``CAST`` or similar) would incorrectly render ``None CHAR`` - or similar. - - .. change:: - :tags: bug, ext - :tickets: 2869 - :versions: 0.9.0 - - Fixed bug which prevented the ``serializer`` extension from working - correctly with table or column names that contain non-ASCII - characters. - - .. change:: - :tags: bug, orm - :tickets: 2818 - :versions: 0.9.0 - - Fixed a regression introduced by :ticket:`2818` where the EXISTS - query being generated would produce a "columns being replaced" - warning for a statement with two same-named columns, - as the internal SELECT wouldn't have use_labels set. - - .. change:: - :tags: bug, postgresql - :tickets: 2855 - :versions: 0.9.0 - - Fixed bug where index reflection would mis-interpret indkey values - when using the pypostgresql adapter, which returns these values - as lists vs. psycopg2's return type of string. - -.. changelog:: - :version: 0.8.3 - :released: October 26, 2013 - - .. change:: - :tags: bug, oracle - :tickets: 2853 - :versions: 0.9.0b1 - - Fixed bug where Oracle table reflection using synonyms would fail - if the synonym and the table were in different remote schemas. - Patch to fix courtesy Kyle Derr. - - .. change:: - :tags: bug, sql - :tickets: 2849 - :versions: 0.9.0b1 - - Fixed bug where :func:`.type_coerce` would not interpret ORM - elements with a ``__clause_element__()`` method properly. - - .. change:: - :tags: bug, sql - :tickets: 2842 - :versions: 0.9.0b1 - - The :class:`.Enum` and :class:`.Boolean` types now bypass - any custom (e.g. TypeDecorator) type in use when producing the - CHECK constraint for the "non native" type. This so that the custom type - isn't involved in the expression within the CHECK, since this - expression is against the "impl" value and not the "decorated" value. - - .. change:: - :tags: bug, postgresql - :tickets: 2844 - :versions: 0.9.0b1 - - Removed a 128-character truncation from the reflection of the - server default for a column; this code was original from - PG system views which truncated the string for readability. - - .. change:: - :tags: bug, mysql - :tickets: 2721, 2839 - :versions: 0.9.0b1 - - The change in :ticket:`2721`, which is that the ``deferrable`` keyword - of :class:`_schema.ForeignKeyConstraint` is silently ignored on the MySQL - backend, will be reverted as of 0.9; this keyword will now render again, raising - errors on MySQL as it is not understood - the same behavior will also - apply to the ``initially`` keyword. In 0.8, the keywords will remain - ignored but a warning is emitted. Additionally, the ``match`` keyword - now raises a :exc:`.CompileError` on 0.9 and emits a warning on 0.8; - this keyword is not only silently ignored by MySQL but also breaks - the ON UPDATE/ON DELETE options. - - To use a :class:`_schema.ForeignKeyConstraint` - that does not render or renders differently on MySQL, use a custom - compilation option. An example of this usage has been added to the - documentation, see :ref:`mysql_foreign_keys`. - - .. change:: - :tags: bug, sql - :tickets: 2825 - :versions: 0.9.0b1 - - The ``.unique`` flag on :class:`.Index` could be produced as ``None`` - if it was generated from a :class:`_schema.Column` that didn't specify ``unique`` - (where it defaults to ``None``). The flag will now always be ``True`` or - ``False``. - - .. change:: - :tags: feature, orm - :tickets: 2836 - :versions: 0.9.0b1 - - Added new option to :func:`_orm.relationship` ``distinct_target_key``. - This enables the subquery eager loader strategy to apply a DISTINCT - to the innermost SELECT subquery, to assist in the case where - duplicate rows are generated by the innermost query which corresponds - to this relationship (there's not yet a general solution to the issue - of dupe rows within subquery eager loading, however, when joins outside - of the innermost subquery produce dupes). When the flag - is set to ``True``, the DISTINCT is rendered unconditionally, and when - it is set to ``None``, DISTINCT is rendered if the innermost relationship - targets columns that do not comprise a full primary key. - The option defaults to False in 0.8 (e.g. off by default in all cases), - None in 0.9 (e.g. automatic by default). Thanks to Alexander Koval - for help with this. - - .. seealso:: - - :ref:`change_2836` - - .. change:: - :tags: bug, mysql - :tickets: 2515 - :versions: 0.9.0b1 - - MySQL-connector dialect now allows options in the create_engine - query string to override those defaults set up in the connect, - including "buffered" and "raise_on_warnings". - - .. change:: - :tags: bug, postgresql - :tickets: 2742 - :versions: 0.9.0b1 - - Parenthesis will be applied to a compound SQL expression as - rendered in the column list of a CREATE INDEX statement. - - .. change:: - :tags: bug, sql - :tickets: 2742 - :versions: 0.9.0b1 - - Fixed bug in default compiler plus those of postgresql, mysql, and - mssql to ensure that any literal SQL expression values are - rendered directly as literals, instead of as bound parameters, - within a CREATE INDEX statement. This also changes the rendering - scheme for other DDL such as constraints. - - .. change:: - :tags: bug, sql - :tickets: 2815 - :versions: 0.9.0b1 - - A :func:`_expression.select` that is made to refer to itself in its FROM clause, - typically via in-place mutation, will raise an informative error - message rather than causing a recursion overflow. - - .. change:: - :tags: bug, orm - :tickets: 2813 - :versions: 0.9.0b1 - - Fixed bug where using an annotation such as :func:`.remote` or - :func:`.foreign` on a :class:`_schema.Column` before association with a parent - :class:`_schema.Table` could produce issues related to the parent table not - rendering within joins, due to the inherent copy operation performed - by an annotation. - - .. change:: - :tags: bug, sql - :tickets: 2831 - - Non-working "schema" argument on :class:`_schema.ForeignKey` is deprecated; - raises a warning. Removed in 0.9. - - .. change:: - :tags: bug, postgresql - :tickets: 2819 - :versions: 0.9.0b1 - - Fixed bug where PostgreSQL version strings that had a prefix preceding - the words "PostgreSQL" or "EnterpriseDB" would not parse. - Courtesy Scott Schaefer. - - .. change:: - :tags: feature, engine - :tickets: 2821 - :versions: 0.9.0b1 - - ``repr()`` for the :class:`.URL` of an :class:`_engine.Engine` - will now conceal the password using asterisks. - Courtesy Gunnlaugur Þór Briem. - - .. change:: - :tags: bug, orm - :tickets: 2818 - :versions: 0.9.0b1 - - Fixed bug where :meth:`_query.Query.exists` failed to work correctly - without any WHERE criterion. Courtesy Vladimir Magamedov. - - .. change:: - :tags: bug, sql - :tickets: 2811 - :versions: 0.9.0b1 - - Fixed bug where using the ``column_reflect`` event to change the ``.key`` - of the incoming :class:`_schema.Column` would prevent primary key constraints, - indexes, and foreign key constraints from being correctly reflected. - - .. change:: - :tags: feature - :versions: 0.9.0b1 - - Added a new flag ``system=True`` to :class:`_schema.Column`, which marks - the column as a "system" column which is automatically made present - by the database (such as PostgreSQL ``oid`` or ``xmin``). The - column will be omitted from the ``CREATE TABLE`` statement but will - otherwise be available for querying. In addition, the - :class:`.CreateColumn` construct can be applied to a custom - compilation rule which allows skipping of columns, by producing - a rule that returns ``None``. - - .. change:: - :tags: bug, orm - :tickets: 2779 - - Backported a change from 0.9 whereby the iteration of a hierarchy - of mappers used in polymorphic inheritance loads is sorted, - which allows the SELECT statements generated for polymorphic queries - to have deterministic rendering, which in turn helps with caching - schemes that cache on the SQL string itself. - - .. change:: - :tags: bug, orm - :tickets: 2794 - :versions: 0.9.0b1 - - Fixed a potential issue in an ordered sequence implementation used - by the ORM to iterate mapper hierarchies; under the Jython interpreter - this implementation wasn't ordered, even though cPython and PyPy - maintained ordering. - - .. change:: - :tags: bug, examples - :versions: 0.9.0b1 - - Added "autoincrement=False" to the history table created in the - versioning example, as this table shouldn't have autoinc on it - in any case, courtesy Patrick Schmid. - - .. change:: - :tags: bug, sql - :versions: 0.9.0b1 - - The :meth:`.ColumnOperators.notin_` operator added in 0.8 now properly - produces the negation of the expression "IN" returns - when used against an empty collection. - - .. change:: - :tags: feature, examples - :versions: 0.9.0b1 - - Improved the examples in ``examples/generic_associations``, including - that ``discriminator_on_association.py`` makes use of single table - inheritance do the work with the "discriminator". Also - added a true "generic foreign key" example, which works similarly - to other popular frameworks in that it uses an open-ended integer - to point to any other table, foregoing traditional referential - integrity. While we don't recommend this pattern, information wants - to be free. - - .. change:: - :tags: feature, orm, declarative - :versions: 0.9.0b1 - - Added a convenience class decorator :func:`.as_declarative`, is - a wrapper for :func:`.declarative_base` which allows an existing base - class to be applied using a nifty class-decorated approach. - - .. change:: - :tags: bug, orm - :tickets: 2786 - :versions: 0.9.0b1 - - Fixed bug in ORM-level event registration where the "raw" or - "propagate" flags could potentially be mis-configured in some - "unmapped base class" configurations. - - .. change:: - :tags: bug, orm - :tickets: 2778 - :versions: 0.9.0b1 - - A performance fix related to the usage of the :func:`.defer` option - when loading mapped entities. The function overhead of applying - a per-object deferred callable to an instance at load time was - significantly higher than that of just loading the data from the row - (note that ``defer()`` is meant to reduce DB/network overhead, not - necessarily function call count); the function call overhead is now - less than that of loading data from the column in all cases. There - is also a reduction in the number of "lazy callable" objects created - per load from N (total deferred values in the result) to 1 (total - number of deferred cols). - - .. change:: - :tags: bug, sqlite - :tickets: 2781 - :versions: 0.9.0b1 - - The newly added SQLite DATETIME arguments storage_format and - regexp apparently were not fully implemented correctly; while the - arguments were accepted, in practice they would have no effect; - this has been fixed. - - .. change:: - :tags: bug, sql, postgresql - :tickets: 2780 - :versions: 0.9.0b1 - - Fixed bug where the expression system relied upon the ``str()`` - form of a some expressions when referring to the ``.c`` collection - on a ``select()`` construct, but the ``str()`` form isn't available - since the element relies on dialect-specific compilation constructs, - notably the ``__getitem__()`` operator as used with a PostgreSQL - ``ARRAY`` element. The fix also adds a new exception class - :exc:`.UnsupportedCompilationError` which is raised in those cases - where a compiler is asked to compile something it doesn't know - how to. - - .. change:: - :tags: bug, engine, oracle - :tickets: 2776 - :versions: 0.9.0b1 - - Dialect.initialize() is not called a second time if an :class:`_engine.Engine` - is recreated, due to a disconnect error. This fixes a particular - issue in the Oracle 8 dialect, but in general the dialect.initialize() - phase should only be once per dialect. - - .. change:: - :tags: feature, sql - :tickets: 722 - - Added new method to the :func:`_expression.insert` construct - :meth:`_expression.Insert.from_select`. Given a list of columns and - a selectable, renders ``INSERT INTO (table) (columns) SELECT ..``. - - .. change:: - :tags: feature, sql - :versions: 0.9.0b1 - - The :func:`_expression.update`, :func:`_expression.insert`, and :func:`_expression.delete` constructs - will now interpret ORM entities as target tables to be operated upon, - e.g.:: - - from sqlalchemy import insert, update, delete - - ins = insert(SomeMappedClass).values(x=5) - - del_ = delete(SomeMappedClass).where(SomeMappedClass.id == 5) - - upd = update(SomeMappedClass).where(SomeMappedClass.id == 5).values(name="ed") - - .. change:: - :tags: bug, orm - :tickets: 2773 - :versions: 0.9.0b1 - - Fixed bug whereby attribute history functions would fail - when an object we moved from "persistent" to "pending" - using the :func:`.make_transient` function, for operations - involving collection-based backrefs. - - .. change:: - :tags: bug, engine, pool - :tickets: 2772 - :versions: 0.9.0b1 - - Fixed bug where :class:`.QueuePool` would lose the correct - checked out count if an existing pooled connection failed to reconnect - after an invalidate or recycle event. - -.. changelog:: - :version: 0.8.2 - :released: July 3, 2013 - - .. change:: - :tags: bug, mysql - :tickets: 2768 - :versions: 0.9.0b1 - - Fixed bug when using multi-table UPDATE where a supplemental - table is a SELECT with its own bound parameters, where the positioning - of the bound parameters would be reversed versus the statement - itself when using MySQL's special syntax. - - .. change:: - :tags: bug, sqlite - :tickets: 2764 - :versions: 0.9.0b1 - - Added :class:`sqlalchemy.types.BIGINT` to the list of type names that can be - reflected by the SQLite dialect; courtesy Russell Stuart. - - .. change:: - :tags: feature, orm, declarative - :tickets: 2761 - :versions: 0.9.0b1 - - ORM descriptors such as hybrid properties can now be referenced - by name in a string argument used with ``order_by``, - ``primaryjoin``, or similar in :func:`_orm.relationship`, - in addition to column-bound attributes. - - .. change:: - :tags: feature, firebird - :tickets: 2763 - :versions: 0.9.0b1 - - Added new flag ``retaining=True`` to the kinterbasdb and fdb dialects. - This controls the value of the ``retaining`` flag sent to the - ``commit()`` and ``rollback()`` methods of the DBAPI connection. - Due to historical concerns, this flag defaults to ``True`` in 0.8.2, - however in 0.9.0b1 this flag defaults to ``False``. - - .. change:: - :tags: requirements - :versions: 0.9.0b1 - - The Python `mock `_ library - is now required in order to run the unit test suite. While part - of the standard library as of Python 3.3, previous Python installations - will need to install this in order to run unit tests or to - use the ``sqlalchemy.testing`` package for external dialects. - - .. change:: - :tags: bug, orm - :tickets: 2750 - :versions: 0.9.0b1 - - A warning is emitted when trying to flush an object of an inherited - class where the polymorphic discriminator has been assigned - to a value that is invalid for the class. - - .. change:: - :tags: bug, postgresql - :tickets: 2740 - :versions: 0.9.0b1 - - The behavior of :func:`.extract` has been simplified on the - PostgreSQL dialect to no longer inject a hardcoded ``::timestamp`` - or similar cast into the given expression, as this interfered - with types such as timezone-aware datetimes, but also - does not appear to be at all necessary with modern versions - of psycopg2. - - - .. change:: - :tags: bug, firebird - :tickets: 2757 - :versions: 0.9.0b1 - - Type lookup when reflecting the Firebird types LONG and - INT64 has been fixed so that LONG is treated as INTEGER, - INT64 treated as BIGINT, unless the type has a "precision" - in which case it's treated as NUMERIC. Patch courtesy - Russell Stuart. - - .. change:: - :tags: bug, postgresql - :tickets: 2766 - :versions: 0.9.0b1 - - Fixed bug in HSTORE type where keys/values that contained - backslashed quotes would not be escaped correctly when - using the "non native" (i.e. non-psycopg2) means - of translating HSTORE data. Patch courtesy Ryan Kelly. - - .. change:: - :tags: bug, postgresql - :tickets: 2767 - :versions: 0.9.0b1 - - Fixed bug where the order of columns in a multi-column - PostgreSQL index would be reflected in the wrong order. - Courtesy Roman Podolyaka. - - .. change:: - :tags: bug, sql - :tickets: 2746, 2668 - :versions: 0.9.0b1 - - Multiple fixes to the correlation behavior of - :class:`_expression.Select` constructs, first introduced in 0.8.0: - - * To satisfy the use case where FROM entries should be - correlated outwards to a SELECT that encloses another, - which then encloses this one, correlation now works - across multiple levels when explicit correlation is - established via :meth:`_expression.Select.correlate`, provided - that the target select is somewhere along the chain - contained by a WHERE/ORDER BY/columns clause, not - just nested FROM clauses. This makes - :meth:`_expression.Select.correlate` act more compatibly to - that of 0.7 again while still maintaining the new - "smart" correlation. - - * When explicit correlation is not used, the usual - "implicit" correlation limits its behavior to just - the immediate enclosing SELECT, to maximize compatibility - with 0.7 applications, and also prevents correlation - across nested FROMs in this case, maintaining compatibility - with 0.8.0/0.8.1. - - * The :meth:`_expression.Select.correlate_except` method was not - preventing the given FROM clauses from correlation in - all cases, and also would cause FROM clauses to be incorrectly - omitted entirely (more like what 0.7 would do), - this has been fixed. - - * Calling `select.correlate_except(None)` will enter - all FROM clauses into correlation as would be expected. - - .. change:: - :tags: bug, ext - :versions: 0.9.0b1 - - Fixed bug whereby if a composite type were set up - with a function instead of a class, the mutable extension - would trip up when it tried to check that column - for being a :class:`.MutableComposite` (which it isn't). - Courtesy asldevi. - - .. change:: - :tags: feature, sql - :tickets: 2744, 2734 - - Provided a new attribute for :class:`.TypeDecorator` - called :attr:`.TypeDecorator.coerce_to_is_types`, - to make it easier to control how comparisons using - ``==`` or ``!=`` to ``None`` and boolean types goes - about producing an ``IS`` expression, or a plain - equality expression with a bound parameter. - - .. change:: - :tags: feature, postgresql - :versions: 0.9.0b1 - - Support for PostgreSQL 9.2 range types has been added. - Currently, no type translation is provided, so works - directly with strings or psycopg2 2.5 range extension types - at the moment. Patch courtesy Chris Withers. - - .. change:: - :tags: bug, examples - :versions: 0.9.0b1 - - Fixed an issue with the "versioning" recipe whereby a many-to-one - reference could produce a meaningless version for the target, - even though it was not changed, when backrefs were present. - Patch courtesy Matt Chisholm. - - .. change:: - :tags: feature, postgresql - :tickets: 2072 - :versions: 0.9.0b1 - - Added support for "AUTOCOMMIT" isolation when using the psycopg2 - DBAPI. The keyword is available via the ``isolation_level`` - execution option. Patch courtesy Roman Podolyaka. - - .. change:: - :tags: bug, orm - :tickets: 2759 - :versions: 0.9.0b1 - - Fixed bug in polymorphic SQL generation where multiple joined-inheritance - entities against the same base class joined to each other as well - would not track columns on the base table independently of each other if - the string of joins were more than two entities long. - - .. change:: - :tags: bug, engine - :versions: 0.9.0b1 - - Fixed bug where the ``reset_on_return`` argument to various :class:`_pool.Pool` - implementations would not be propagated when the pool was regenerated. - Courtesy Eevee. - - .. change:: - :tags: bug, orm - :tickets: 2754 - :versions: 0.9.0b1 - - Fixed bug where sending a composite attribute into :meth:`_query.Query.order_by` - would produce a parenthesized expression not accepted by some databases. - - .. change:: - :tags: bug, orm - :tickets: 2755 - :versions: 0.9.0b1 - - Fixed the interaction between composite attributes and - the :func:`.aliased` function. Previously, composite attributes - wouldn't work correctly in comparison operations when aliasing - was applied. - - .. change:: - :tags: bug, mysql - :tickets: 2715 - :versions: 0.9.0b1 - - Added another conditional to the ``mysql+gaerdbms`` dialect to - detect so-called "development" mode, where we should use the - ``rdbms_mysqldb`` DBAPI. Patch courtesy Brett Slatkin. - - .. change:: - :tags: feature, mysql - :tickets: 2704 - :versions: 0.9.0b1 - - The ``mysql_length`` parameter used with :class:`.Index` can now - be passed as a dictionary of column names/lengths, for use - with composite indexes. Big thanks to Roman Podolyaka for the - patch. - - .. change:: - :tags: bug, mssql - :tickets: 2747 - :versions: 0.9.0b1 - - When querying the information schema on SQL Server 2000, removed - a CAST call that was added in 0.8.1 to help with driver issues, - which apparently is not compatible on 2000. - The CAST remains in place for SQL Server 2005 and greater. - - .. change:: - :tags: bug, mysql - :tickets: 2721 - :versions: 0.9.0b1 - - The ``deferrable`` keyword argument on :class:`_schema.ForeignKey` and - :class:`_schema.ForeignKeyConstraint` will not render the ``DEFERRABLE`` keyword - on the MySQL dialect. For a long time we left this in place because - a non-deferrable foreign key would act very differently than a deferrable - one, but some environments just disable FKs on MySQL, so we'll be less - opinionated here. - - .. change:: - :tags: bug, ext, orm - :tickets: 2730 - :versions: 0.9.0b1 - - Fixed bug where :class:`.MutableDict` didn't report a change event - when ``clear()`` was called. - - .. change:: - :tags: bug, sql - :tickets: 2738 - :versions: 0.9.0b1 - - Fixed bug whereby joining a select() of a table "A" with multiple - foreign key paths to a table "B", to that table "B", would fail - to produce the "ambiguous join condition" error that would be - reported if you join table "A" directly to "B"; it would instead - produce a join condition with multiple criteria. - - .. change:: - :tags: bug, sql, reflection - :tickets: 2728 - :versions: 0.9.0b1 - - Fixed bug whereby using :meth:`_schema.MetaData.reflect` across a remote - schema as well as a local schema could produce wrong results - in the case where both schemas had a table of the same name. - - .. change:: - :tags: bug, sql - :tickets: 2726 - :versions: 0.9.0b1 - - Removed the "not implemented" ``__iter__()`` call from the base - :class:`.ColumnOperators` class, while this was introduced - in 0.8.0 to prevent an endless, memory-growing loop when one also - implements a ``__getitem__()`` method on a custom - operator and then calls erroneously ``list()`` on that object, - it had the effect of causing column elements to report that they - were in fact iterable types which then throw an error when you try - to iterate. There's no real way to have both sides here so we - stick with Python best practices. Careful with implementing - ``__getitem__()`` on your custom operators! - - .. change:: - :tags: feature, orm - :tickets: 2736 - - Added a new method :meth:`_query.Query.select_entity_from` which - will in 0.9 replace part of the functionality of - :meth:`_query.Query.select_from`. In 0.8, the two methods perform - the same function, so that code can be migrated to use the - :meth:`_query.Query.select_entity_from` method as appropriate. - See the 0.9 migration guide for details. - - .. change:: - :tags: bug, orm - :tickets: 2737 - - Fixed a regression caused by :ticket:`2682` whereby the - evaluation invoked by :meth:`_query.Query.update` and :meth:`_query.Query.delete` - would hit upon unsupported ``True`` and ``False`` symbols - which now appear due to the usage of ``IS``. - - .. change:: - :tags: bug, postgresql - :tickets: 2735 - - Fixed the HSTORE type to correctly encode/decode for unicode. - This is always on, as the hstore is a textual type, and - matches the behavior of psycopg2 when using Python 3. - Courtesy Dmitry Mugtasimov. - - .. change:: - :tags: bug, examples - - Fixed a small bug in the dogpile example where the generation - of SQL cache keys wasn't applying deduping labels to the - statement the same way :class:`_query.Query` normally does. - - .. change:: - :tags: bug, engine, sybase - :tickets: 2732 - - Fixed a bug where the routine to detect the correct kwargs - being sent to :func:`_sa.create_engine` would fail in some cases, - such as with the Sybase dialect. - - .. change:: - :tags: bug, orm - :tickets: 2481 - - Fixed a regression from 0.7 caused by this ticket, which - made the check for recursion overflow in self-referential - eager joining too loose, missing a particular circumstance - where a subclass had lazy="joined" or "subquery" configured - and the load was a "with_polymorphic" against the base. - - .. change:: - :tags: bug, orm - :tickets: 2718 - - Fixed a regression from 0.7 where the contextmanager feature - of :meth:`.Session.begin_nested` would fail to correctly - roll back the transaction when a flush error occurred, instead - raising its own exception while leaving the session still - pending a rollback. - - .. change:: - :tags: bug, mysql - - Updated mysqlconnector dialect to check for disconnect based - on the apparent string message sent in the exception; tested - against mysqlconnector 1.0.9. - - .. change:: - :tags: bug, sql, mssql - :tickets: 2682 - - Regression from this ticket caused the unsupported keyword - "true" to render, added logic to convert this to 1/0 - for SQL server. - -.. changelog:: - :version: 0.8.1 - :released: April 27, 2013 - - .. change:: - :tags: bug, orm - :tickets: 2698 - - Fixes to the ``sqlalchemy.ext.serializer`` extension, including - that the "id" passed from the pickler is turned into a string - to prevent against bytes being parsed on Py3K, as well as that - ``relationship()`` and ``orm.join()`` constructs are now properly - serialized. - - .. change:: - :tags: bug, orm - :tickets: 2714 - - A significant improvement to the inner workings of query.join(), - such that the decisionmaking involved on how to join has been - dramatically simplified. New test cases now pass such as - multiple joins extending from the middle of an already complex - series of joins involving inheritance and such. Joining from - deeply nested subquery structures is still complicated and - not without caveats, but with these improvements the edge - cases are hopefully pushed even farther out to the edges. - - .. change:: - :tags: feature, orm - :tickets: 2673 - - Added a convenience method to Query that turns a query into an - EXISTS subquery of the form - ``EXISTS (SELECT 1 FROM ... WHERE ...)``. - - .. change:: - :tags: bug, orm - - Added a conditional to the unpickling process for ORM - mapped objects, such that if the reference to the object - were lost when the object was pickled, we don't - erroneously try to set up _sa_instance_state - fixes - a NoneType error. - - .. change:: - :tags: bug, postgresql - :tickets: 2712 - - Opened up the checking for "disconnect" with psycopg2/libpq - to check for all the various "disconnect" messages within - the full exception hierarchy. Specifically the - "closed the connection unexpectedly" message has now been - seen in at least three different exception types. - Courtesy Eli Collins. - - .. change:: - :tags: bug, sql, mysql - :tickets: 2682 - - Fully implemented the IS and IS NOT operators with - regards to the True/False constants. An expression like - ``col.is_(True)`` will now render ``col IS true`` - on the target platform, rather than converting the True/ - False constant to an integer bound parameter. - This allows the ``is_()`` operator to work on MySQL when - given True/False constants. - - .. change:: - :tags: bug, postgresql - :tickets: 2681 - - The operators for the PostgreSQL ARRAY type supports - input types of sets, generators, etc. even when - a dimension is not specified, by turning the given - iterable into a collection unconditionally. - - .. change:: - :tags: bug, mysql - - Fixes to support the latest cymysql DBAPI, courtesy - Hajime Nakagami. - - .. change:: - :tags: bug, mysql - :tickets: 2663 - - Improvements to the operation of the pymysql dialect on - Python 3, including some important decode/bytes steps. - Issues remain with BLOB types due to driver issues. - Courtesy Ben Trofatter. - - .. change:: - :tags: bug, orm - :tickets: 2710 - - Fixed bug where many-to-many relationship with uselist=False - would fail to delete the association row and raise an error - if the scalar attribute were set to None. This was a - regression introduced by the changes for :ticket:`2229`. - - .. change:: - :tags: bug, orm - :tickets: 2708 - - Improved the behavior of instance management regarding - the creation of strong references within the Session; - an object will no longer have an internal reference cycle - created if it's in the transient state or moves into the - detached state - the strong ref is created only when the - object is attached to a Session and is removed when the - object is detached. This makes it somewhat safer for an - object to have a `__del__()` method, even though this is - not recommended, as relationships with backrefs produce - cycles too. A warning has been added when a class with - a `__del__()` method is mapped. - - .. change:: - :tags: bug, sql - :tickets: 2702 - - A major fix to the way in which a select() object produces - labeled columns when apply_labels() is used; this mode - produces a SELECT where each column is labeled as in - _, to remove column name collisions - for a multiple table select. The fix is that if two labels - collide when combined with the table name, i.e. - "foo.bar_id" and "foo_bar.id", anonymous aliasing will be - applied to one of the dupes. This allows the ORM to handle - both columns independently; previously, 0.7 - would in some cases silently emit a second SELECT for the - column that was "duped", and in 0.8 an ambiguous column error - would be emitted. The "keys" applied to the .c. collection - of the select() will also be deduped, so that the "column - being replaced" warning will no longer emit for any select() - that specifies use_labels, though the dupe key will be given - an anonymous label which isn't generally user-friendly. - - .. change:: - :tags: bug, mysql - - Updated a regexp to correctly extract error code on - google app engine v1.7.5 and newer. Courtesy - Dan Ring. - - .. change:: - :tags: bug, examples - - Fixed a long-standing bug in the caching example, where - the limit/offset parameter values wouldn't be taken into - account when computing the cache key. The - _key_from_query() function has been simplified to work - directly from the final compiled statement in order to get - at both the full statement as well as the fully processed - parameter list. - - .. change:: - :tags: bug, mssql - :tickets: 2355 - - Part of a longer series of fixes needed for pyodbc+ - mssql, a CAST to NVARCHAR(max) has been added to the bound - parameter for the table name and schema name in all information schema - queries to avoid the issue of comparing NVARCHAR to NTEXT, - which seems to be rejected by the ODBC driver in some cases, - such as FreeTDS (0.91 only?) plus unicode bound parameters being passed. - The issue seems to be specific to the SQL Server information - schema tables and the workaround is harmless for those cases - where the problem doesn't exist in the first place. - - .. change:: - :tags: bug, sql - :tickets: 2691 - - Fixed bug where disconnect detect on error would - raise an attribute error if the error were being - raised after the Connection object had already - been closed. - - .. change:: - :tags: bug, sql - :tickets: 2703 - - Reworked internal exception raises that emit - a rollback() before re-raising, so that the stack - trace is preserved from sys.exc_info() before entering - the rollback. This so that the traceback is preserved - when using coroutine frameworks which may have switched - contexts before the rollback function returns. - - .. change:: - :tags: bug, orm - :tickets: 2697 - - Fixed bug whereby ORM would run the wrong kind of - query when refreshing an inheritance-mapped class - where the superclass was mapped to a non-Table - object, like a custom join() or a select(), - running a query that assumed a hierarchy that's - mapped to individual Table-per-class. - - .. change:: - :tags: bug, orm - - Fixed `__repr__()` on mapper property constructs - to work before the object is initialized, so - that Sphinx builds with recent Sphinx versions - can read them. - - .. change:: - :tags: bug, sql, postgresql - - The _Binary base type now converts values through - the bytes() callable when run on Python 3; in particular - psycopg2 2.5 with Python 3.3 seems to now be returning - the "memoryview" type, so this is converted to bytes - before return. - - .. change:: - :tags: bug, sql - :tickets: 2695 - - Improvements to Connection auto-invalidation - handling. If a non-disconnect error occurs, - but leads to a delayed disconnect error within error - handling (happens with MySQL), the disconnect condition - is detected. The Connection can now also be closed - when in an invalid state, meaning it will raise "closed" - on next usage, and additionally the "close with result" - feature will work even if the autorollback in an error - handling routine fails and regardless of whether the - condition is a disconnect or not. - - - .. change:: - :tags: bug, orm, declarative - :tickets: 2656 - - Fixed indirect regression regarding :func:`.has_inherited_table`, - where since it considers the current class' ``__table__``, was - sensitive to when it was called. This is 0.7's behavior also, - but in 0.7 things tended to "work out" within events like - ``__mapper_args__()``. :func:`.has_inherited_table` now only - considers superclasses, so should return the same answer - regarding the current class no matter when it's called - (obviously assuming the state of the superclass). - - .. change:: - :tags: bug, mssql - - Added support for additional "disconnect" messages - to the pymssql dialect. Courtesy John Anderson. - - .. change:: - :tags: feature, sql - - Loosened the check on dialect-specific argument names - passed to Table(); since we want to support external dialects - and also want to support args without a certain dialect - being installed, it only checks the format of the arg now, - rather than looking for that dialect in sqlalchemy.dialects. - - .. change:: - :tags: bug, sql - - Fixed bug whereby a DBAPI that can return "0" - for cursor.lastrowid would not function correctly - in conjunction with :attr:`_engine.ResultProxy.inserted_primary_key`. - - .. change:: - :tags: bug, mssql - :tickets: 2683 - - Fixed Py3K bug regarding "binary" types and - pymssql. Courtesy Marc Abramowitz. - - .. change:: - :tags: bug, postgresql - :tickets: 2680 - - Added missing HSTORE type to postgresql type names - so that the type can be reflected. - -.. changelog:: - :version: 0.8.0 - :released: March 9, 2013 - - .. note:: - - There are some new behavioral changes as of 0.8.0 - not present in 0.8.0b2. They are present in the - migration document as follows: - - * :ref:`legacy_is_orphan_addition` - - * :ref:`metadata_create_drop_tables` - - * :ref:`correlation_context_specific` - - .. change:: - :tags: feature, orm - :tickets: 2675 - - A meaningful :attr:`.QueryableAttribute.info` attribute is - added, which proxies down to the ``.info`` attribute on either - the :class:`_schema.Column` object if directly present, or - the :class:`.MapperProperty` otherwise. The full behavior - is documented and ensured by tests to remain stable. - - .. change:: - :tags: bug, sql - :tickets: 2668 - - The behavior of SELECT correlation has been improved such that - the :meth:`_expression.Select.correlate` and :meth:`_expression.Select.correlate_except` - methods, as well as their ORM analogues, will still retain - "auto-correlation" behavior in that the FROM clause is modified - only if the output would be legal SQL; that is, the FROM clause - is left intact if the correlated SELECT is not used in the context - of an enclosing SELECT inside of the WHERE, columns, or HAVING clause. - The two methods now only specify conditions to the default - "auto correlation", rather than absolute FROM lists. - - .. change:: - :tags: feature, mysql - - New dialect for CyMySQL added, courtesy Hajime Nakagami. - - .. change:: - :tags: bug, orm - :tickets: 2674 - - Improved checking for an existing backref name conflict during - mapper configuration; will now test for name conflicts on - superclasses and subclasses, in addition to the current mapper, - as these conflicts break things just as much. This is new for - 0.8, but see below for a warning that will also be triggered - in 0.7.11. - - .. change:: - :tags: bug, orm - :tickets: 2674 - - Improved the error message emitted when a "backref loop" is detected, - that is when an attribute event triggers a bidirectional - assignment between two other attributes with no end. - This condition can occur not just when an object of the wrong - type is assigned, but also when an attribute is mis-configured - to backref into an existing backref pair. Also in 0.7.11. - - .. change:: - :tags: bug, orm - :tickets: 2674 - - A warning is emitted when a MapperProperty is assigned to a mapper - that replaces an existing property, if the properties in question - aren't plain column-based properties. Replacement of relationship - properties is rarely (ever?) what is intended and usually refers to a - mapper mis-configuration. Also in 0.7.11. - - .. change:: - :tags: feature, orm - - Can set/change the "cascade" attribute on a :func:`_orm.relationship` - construct after it's been constructed already. This is not - a pattern for normal use but we like to change the setting - for demonstration purposes in tutorials. - - .. change:: - :tags: bug, schema - :tickets: 2664 - - :meth:`_schema.MetaData.create_all` and :meth:`_schema.MetaData.drop_all` will - now accommodate an empty list as an instruction to not create/drop - any items, rather than ignoring the collection. - - - .. change:: - :tags: bug, tests - :tickets: 2669 - - Fixed an import of "logging" in test_execute which was not - working on some linux platforms. Also in 0.7.11. - - .. change:: - :tags: bug, orm - :tickets: 2662 - - A clear error message is emitted if an event handler - attempts to emit SQL on a Session within the after_commit() - handler, where there is not a viable transaction in progress. - - .. change:: - :tags: bug, orm - :tickets: 2665 - - Detection of a primary key change within the process - of cascading a natural primary key update will succeed - even if the key is composite and only some of the - attributes have changed. - - .. change:: - :tags: feature, orm - :tickets: 2658 - - Added new helper function :func:`.was_deleted`, returns True - if the given object was the subject of a :meth:`.Session.delete` - operation. - - .. change:: - :tags: bug, orm - :tickets: 2658 - - An object that's deleted from a session will be de-associated with - that session fully after the transaction is committed, that is - the :func:`.object_session` function will return None. - - .. change:: - :tags: bug, oracle - - The cx_oracle dialect will no longer run the bind parameter names - through ``encode()``, as this is not valid on Python 3, and prevented - statements from functioning correctly on Python 3. We now - encode only if ``supports_unicode_binds`` is False, which is not - the case for cx_oracle when at least version 5 of cx_oracle is used. - - .. change:: - :tags: bug, orm - :tickets: 2661 - - Fixed bug whereby :meth:`_query.Query.yield_per` would set the execution - options incorrectly, thereby breaking subsequent usage of the - :meth:`_query.Query.execution_options` method. Courtesy Ryan Kelly. - - .. change:: - :tags: bug, orm - :tickets: 1768 - - Fixed the consideration of the ``between()`` operator - so that it works correctly with the new relationship local/remote - system. - - .. change:: - :tags: bug, sql - :tickets: 2660, 1768 - - Fixed a bug regarding column annotations which in particular - could impact some usages of the new :func:`_orm.remote` and - :func:`_orm.local` annotation functions, where annotations - could be lost when the column were used in a subsequent - expression. - - .. change:: - :tags: bug, mysql, gae - :tickets: 2649 - - Added a conditional import to the ``gaerdbms`` dialect which attempts - to import rdbms_apiproxy vs. rdbms_googleapi to work - on both dev and production platforms. Also now honors the - ``instance`` attribute. Courtesy Sean Lynch. - Also in 0.7.10. - - .. change:: - :tags: bug, sql - :tickets: 2496 - - The :meth:`.ColumnOperators.in_` operator will now coerce - values of ``None`` to :func:`.null`. - - .. change:: - :tags: feature, sql - :tickets: 2657 - - Added a new argument to :class:`.Enum` and its base - :class:`.SchemaType` ``inherit_schema``. When set to ``True``, - the type will set its ``schema`` attribute of that of the - :class:`_schema.Table` to which it is associated. This also occurs - during a :meth:`_schema.Table.tometadata` operation; the :class:`.SchemaType` - is now copied in all cases when :meth:`_schema.Table.tometadata` happens, - and if ``inherit_schema=True``, the type will take on the new - schema name passed to the method. The ``schema`` is important - when used with the PostgreSQL backend, as the type results in - a ``CREATE TYPE`` statement. - - .. change:: - :tags: feature, postgresql - - Added :meth:`.postgresql.ARRAY.Comparator.any` and - :meth:`.postgresql.ARRAY.Comparator.all` - methods, as well as standalone expression constructs. Big thanks - to Audrius Kažukauskas for the terrific work here. - - .. change:: - :tags: sql, bug - :tickets: 2643 - - Fixed bug where :meth:`_schema.Table.tometadata` would fail if a - :class:`_schema.Column` had both a foreign key as well as an - alternate ".key" name for the column. Also in 0.7.10. - - .. change:: - :tags: sql, bug - :tickets: 2629 - - insert().returning() raises an informative CompileError if attempted - to compile on a dialect that doesn't support RETURNING. - - .. change:: - :tags: orm, bug - :tickets: 2655 - - the consideration of a pending object as - an "orphan" has been modified to more closely match the - behavior as that of persistent objects, which is that the object - is expunged from the :class:`.Session` as soon as it is - de-associated from any of its orphan-enabled parents. Previously, - the pending object would be expunged only if de-associated - from all of its orphan-enabled parents. The new flag ``legacy_is_orphan`` - is added to :class:`_orm.Mapper` which re-establishes the - legacy behavior. - - See the change note and example case at :ref:`legacy_is_orphan_addition` - for a detailed discussion of this change. - - .. change:: - :tags: orm, bug - :tickets: 2653 - - Fixed the (most likely never used) "@collection.link" collection - method, which fires off each time the collection is associated - or de-associated with a mapped object - the decorator - was not tested or functional. The decorator method - is now named :meth:`.collection.linker` though the name "link" - remains for backwards compatibility. Courtesy Luca Wehrstedt. - - .. change:: - :tags: orm, bug - :tickets: 2654 - - Made some fixes to the system of producing custom instrumented - collections, mainly that the usage of the @collection decorators - will now honor the __mro__ of the given class, applying the - logic of the sub-most classes' version of a particular collection - method. Previously, it wasn't predictable when subclassing - an existing instrumented class such as :class:`.MappedCollection` - whether or not custom methods would resolve correctly. - - .. change:: - :tags: orm, removed - - The undocumented (and hopefully unused) system of producing - custom collections using an ``__instrumentation__`` datastructure - associated with the collection has been removed, as this was a complex - and untested feature which was also essentially redundant versus the - decorator approach. Other internal simplifications to the - orm.collections module have been made as well. - - .. change:: - :tags: mssql, feature - - Added ``mssql_include`` and ``mssql_clustered`` options to - :class:`.Index`, renders the ``INCLUDE`` and ``CLUSTERED`` keywords, - respectively. Courtesy Derek Harland. - - .. change:: - :tags: sql, feature - :tickets: 695 - - :class:`.Index` now supports arbitrary SQL expressions and/or - functions, in addition to straight columns. Common modifiers - include using ``somecolumn.desc()`` for a descending index and - ``func.lower(somecolumn)`` for a case-insensitive index, depending on the - capabilities of the target backend. - - .. change:: - :tags: mssql, bug - :tickets: 2638 - - Added a py3K conditional around unnecessary .decode() - call in mssql information schema, fixes reflection - in Py3K. Also in 0.7.10. - - .. change:: - :tags: orm, bug - :tickets: 2650 - - Fixed potential memory leak which could occur if an - arbitrary number of :class:`.sessionmaker` objects - were created. The anonymous subclass created by - the sessionmaker, when dereferenced, would not be garbage - collected due to remaining class-level references from the - event package. This issue also applies to any custom system - that made use of ad-hoc subclasses in conjunction with - an event dispatcher. Also in 0.7.10. - - .. change:: - :tags: mssql, bug - - Fixed a regression whereby the "collation" parameter - of the character types CHAR, NCHAR, etc. stopped working, - as "collation" is now supported by the base string types. - The TEXT, NCHAR, CHAR, VARCHAR types within the - MSSQL dialect are now synonyms for the base types. - - .. change:: - :tags: mssql, feature - :tickets: 2644 - - DDL for IDENTITY columns is now supported on - non-primary key columns, by establishing a - :class:`.Sequence` construct on any - integer column. Courtesy Derek Harland. - - .. change:: - :tags: examples, bug - - Fixed a regression in the examples/dogpile_caching example - which was due to the change in :ticket:`2614`. - - .. change:: - :tags: orm, bug - :tickets: 2640 - - :meth:`_query.Query.merge_result` can now load rows from an outer join - where an entity may be ``None`` without throwing an error. - Also in 0.7.10. - - .. change:: - :tags: sql, bug - :tickets: 2648 - - Tweaked the "REQUIRED" symbol used by the compiler to identify - INSERT/UPDATE bound parameters that need to be passed, so that - it's more easily identifiable when writing custom bind-handling - code. - - .. change:: - :tags: postgresql, bug - - Fixed bug in :class:`~sqlalchemy.dialects.postgresql.array()` construct whereby using it - inside of an :func:`_expression.insert` construct would produce an - error regarding a parameter issue in the ``self_group()`` method. - - .. change:: - :tags: orm, feature - - Extended the :doc:`/core/inspection` system so that all Python descriptors - associated with the ORM or its extensions can be retrieved. - This fulfills the common request of being able to inspect - all :class:`.QueryableAttribute` descriptors in addition to - extension types such as :class:`.hybrid_property` and - :class:`.AssociationProxy`. See :attr:`_orm.Mapper.all_orm_descriptors`. - - .. change:: - :tags: mysql, feature - - GAE dialect now accepts username/password arguments in the URL, - courtesy Owen Nelson. - - .. change:: - :tags: mysql, bug - - GAE dialect won't fail on None match if the error code can't be extracted - from the exception throw; courtesy Owen Nelson. - - .. change:: - :tags: orm, bug - :tickets: 2637 - - Fixes to the "dynamic" loader on :func:`_orm.relationship`, includes - that backrefs will work properly even when autoflush is disabled, - history events are more accurate in scenarios where multiple add/remove - of the same object occurs. - -.. changelog:: - :version: 0.8.0b2 - :released: December 14, 2012 - - .. change:: - :tags: orm, bug - :tickets: 2635 - - The :meth:`_query.Query.select_from` method can now be used with a - :func:`.aliased` construct without it interfering with the entities - being selected. Basically, a statement like this:: - - ua = aliased(User) - session.query(User.name).select_from(ua).join(User, User.name > ua.name) - - Will maintain the columns clause of the SELECT as coming from the - unaliased "user", as specified; the select_from only takes place in the - FROM clause: - - .. sourcecode:: sql - - SELECT users.name AS users_name FROM users AS users_1 - JOIN users ON users.name < users_1.name - - Note that this behavior is in contrast - to the original, older use case for :meth:`_query.Query.select_from`, which is that - of restating the mapped entity in terms of a different selectable:: - - session.query(User.name).select_from(user_table.select().where(user_table.c.id > 5)) - - Which produces: - - .. sourcecode:: sql - - SELECT anon_1.name AS anon_1_name FROM (SELECT users.id AS id, - users.name AS name FROM users WHERE users.id > :id_1) AS anon_1 - - It was the "aliasing" behavior of the latter use case that was - getting in the way of the former use case. The method now - specifically considers a SQL expression like - :func:`_expression.select` or :func:`_expression.alias` - separately from a mapped entity like a :func:`.aliased` - construct. - - .. change:: - :tags: sql, bug - :tickets: 2633 - - Fixed a regression caused by :ticket:`2410` whereby a - :class:`.CheckConstraint` would apply itself back to the - original table during a :meth:`_schema.Table.tometadata` operation, as - it would parse the SQL expression for a parent table. The - operation now copies the given expression to correspond to the - new table. - - .. change:: - :tags: oracle, bug - :tickets: 2619 - - Fixed table reflection for Oracle when accessing a synonym that refers - to a DBLINK remote database; while the syntax has been present in the - Oracle dialect for some time, up until now it has never been tested. - The syntax has been tested against a sample database linking to itself, - however there's still some uncertainty as to what should be used for the - "owner" when querying the remote database for table information. - Currently, the value of "username" from user_db_links is used to - match the "owner". - - .. change:: - :tags: orm, feature - :tickets: 2601 - - Added :meth:`.KeyedTuple._asdict` and :attr:`.KeyedTuple._fields` - to the :class:`.KeyedTuple` class to provide some degree of compatibility - with the Python standard library ``collections.namedtuple()``. - - .. change:: - :tags: sql, bug - :tickets: 2610 - - Fixed bug whereby using a label_length on dialect that was smaller - than the size of actual column identifiers would fail to render - the columns correctly in a SELECT statement. - - .. change:: - :tags: sql, feature - :tickets: 2623 - - The :class:`_expression.Insert` construct now supports multi-valued inserts, - that is, an INSERT that renders like - "INSERT INTO table VALUES (...), (...), ...". - Supported by PostgreSQL, SQLite, and MySQL. - Big thanks to Idan Kamara for doing the legwork on this one. - - .. seealso:: - - :ref:`feature_2623` - - .. change:: - :tags: oracle, bug - :tickets: 2620 - - The Oracle LONG type, while an unbounded text type, does not appear - to use the cx_Oracle.LOB type when result rows are returned, - so the dialect has been repaired to exclude LONG from - having cx_Oracle.LOB filtering applied. Also in 0.7.10. - - .. change:: - :tags: oracle, bug - :tickets: 2611 - - Repaired the usage of ``.prepare()`` in conjunction with - cx_Oracle so that a return value of ``False`` will result - in no call to ``connection.commit()``, hence avoiding - "no transaction" errors. Two-phase transactions have - now been shown to work in a rudimental fashion with - SQLAlchemy and cx_oracle, however are subject to caveats - observed with the driver; check the documentation - for details. Also in 0.7.10. - - .. change:: - :tags: sql, bug - :tickets: 2618 - - The :class:`~sqlalchemy.types.DECIMAL` type now honors the "precision" and - "scale" arguments when rendering DDL. - - .. change:: - :tags: orm, bug - :tickets: 2624 - - The :class:`.MutableComposite` type did not allow for the - :meth:`.MutableBase.coerce` method to be used, even though - the code seemed to indicate this intent, so this now works - and a brief example is added. As a side-effect, - the mechanics of this event handler have been changed so that - new :class:`.MutableComposite` types no longer add per-type - global event handlers. Also in 0.7.10. - - .. change:: - :tags: sql, bug - :tickets: 2621 - - Made an adjustment to the "boolean", (i.e. ``__nonzero__``) - evaluation of binary expressions, i.e. ``x1 == x2``, such - that the "auto-grouping" applied by :class:`.BinaryExpression` - in some cases won't get in the way of this comparison. - Previously, an expression like:: - - expr1 = mycolumn > 2 - bool(expr1 == expr1) - - Would evaluate as ``False``, even though this is an identity - comparison, because ``mycolumn > 2`` would be "grouped" before - being placed into the :class:`.BinaryExpression`, thus changing - its identity. :class:`.BinaryExpression` now keeps track - of the "original" objects passed in. - Additionally the ``__nonzero__`` method now only returns if - the operator is ``==`` or ``!=`` - all others raise ``TypeError``. - - .. change:: - :tags: firebird, bug - :tickets: 2622 - - Added missing import for "fdb" to the experimental - "firebird+fdb" dialect. - - .. change:: - :tags: orm, feature - - Allow synonyms to be used when defining primary and secondary - joins for relationships. - - .. change:: - :tags: orm, bug - :tickets: 2614 - - A second overhaul of aliasing/internal pathing mechanics - now allows two subclasses to have different relationships - of the same name, supported with subquery or joined eager - loading on both simultaneously when a full polymorphic - load is used. - - .. change:: - :tags: orm, bug - :tickets: 2617 - - Fixed bug whereby a multi-hop subqueryload within - a particular with_polymorphic load would produce a KeyError. - Takes advantage of the same internal pathing overhaul - as :ticket:`2614`. - - .. change:: - :tags: sql, bug - - Fixed a gotcha where inadvertently calling list() on a - :class:`_expression.ColumnElement` would go into an endless loop, if - :meth:`.ColumnOperators.__getitem__` were implemented. - A new NotImplementedError is emitted via ``__iter__()``. - - .. change:: - :tags: orm, extensions, feature - - The :mod:`sqlalchemy.ext.mutable` extension now includes the - example :class:`.MutableDict` class as part of the extension. - - .. change:: - :tags: postgresql, feature - :tickets: 2606 - - :class:`.HSTORE` is now available in the PostgreSQL dialect. - Will also use psycopg2's extensions if available. Courtesy - Audrius Kažukauskas. - - .. change:: - :tags: sybase, feature - :tickets: 1753 - - Reflection support has been added to the Sybase dialect. - Big thanks to Ben Trofatter for all the work developing and - testing this. - - .. change:: - :tags: engine, feature - - The :meth:`_engine.Connection.connect` and :meth:`_engine.Connection.contextual_connect` - methods now return a "branched" version so that the :meth:`_engine.Connection.close` - method can be called on the returned connection without affecting the - original. Allows symmetry when using :class:`_engine.Engine` and - :class:`_engine.Connection` objects as context managers:: - - with conn.connect() as c: # leaves the Connection open - c.execute("...") - - with engine.connect() as c: # closes the Connection - c.execute("...") - - .. change:: - :tags: engine - - The "reflect=True" argument to :class:`~sqlalchemy.schema.MetaData` is deprecated. - Please use the :meth:`_schema.MetaData.reflect` method. - - .. change:: - :tags: sql, bug - :tickets: 2603 - - Fixed bug in type_coerce() whereby typing information - could be lost if the statement were used as a subquery - inside of another statement, as well as other similar - situations. Among other things, would cause - typing information to be lost when the Oracle/mssql dialects - would apply limit/offset wrappings. - - .. change:: - :tags: orm, bug - :tickets: 2602 - - Fixed regression where query.update() would produce - an error if an object matched by the "fetch" - synchronization strategy wasn't locally present. - Courtesy Scott Torborg. - - .. change:: - :tags: sql, bug - :tickets: 2597 - - Fixed bug whereby the ".key" of a Column wasn't being - used when producing a "proxy" of the column against - a selectable. This probably didn't occur in 0.7 - since 0.7 doesn't respect the ".key" in a wider - range of scenarios. - - .. change:: - :tags: mssql, feature - :tickets: 2600 - - Support for reflection of the "name" of primary key - constraints added, courtesy Dave Moore. - - .. change:: - :tags: informix - - Some cruft regarding informix transaction handling has been - removed, including a feature that would skip calling - commit()/rollback() as well as some hardcoded isolation level - assumptions on begin().. The status of this dialect is not - well understood as we don't have any users working with it, - nor any access to an Informix database. If someone with - access to Informix wants to help test this dialect, please - let us know. - - .. change:: - :tags: pool, feature - - The :class:`_pool.Pool` will now log all connection.close() - operations equally, including closes which occur for - invalidated connections, detached connections, and connections - beyond the pool capacity. - - .. change:: - :tags: pool, feature - :tickets: 2611 - - The :class:`_pool.Pool` now consults the :class:`.Dialect` for - functionality regarding how the connection should be - "auto rolled back", as well as closed. This grants more - control of transaction scope to the dialect, so that we - will be better able to implement transactional workarounds - like those potentially needed for pysqlite and cx_oracle. - - .. change:: - :tags: pool, feature - - Added new :meth:`_events.PoolEvents.reset` hook to capture - the event before a connection is auto-rolled back, upon - return to the pool. Together with - :meth:`_events.ConnectionEvents.rollback` this allows all rollback - events to be intercepted. - -.. changelog:: - :version: 0.8.0b1 - :released: October 30, 2012 - - .. change:: - :tags: sql, bug - :tickets: 2593 - - Fixed bug where keyword arguments passed to - :meth:`.Compiler.process` wouldn't get propagated - to the column expressions present in the columns - clause of a SELECT statement. In particular this would - come up when used by custom compilation schemes that - relied upon special flags. - - .. change:: - :tags: sql, feature - - Added a new method :meth:`_engine.Engine.execution_options` - to :class:`_engine.Engine`. This method works similarly to - :meth:`_engine.Connection.execution_options` in that it creates - a copy of the parent object which will refer to the new - set of options. The method can be used to build - sharding schemes where each engine shares the same - underlying pool of connections. The method - has been tested against the horizontal shard - recipe in the ORM as well. - - .. seealso:: - - :meth:`_engine.Engine.execution_options` - - .. change:: - :tags: sql, orm, bug - :tickets: 2595 - - The auto-correlation feature of :func:`_expression.select`, and - by proxy that of :class:`_query.Query`, will not - take effect for a SELECT statement that is being - rendered directly in the FROM list of the enclosing - SELECT. Correlation in SQL only applies to column - expressions such as those in the WHERE, ORDER BY, - columns clause. - - .. change:: - :tags: sqlite - :changeset: c3addcc9ffad - - Added :class:`_types.NCHAR`, :class:`_types.NVARCHAR` - to the SQLite dialect's list of recognized type names - for reflection. SQLite returns the name given - to a type as the name returned. - - .. change:: - :tags: examples - :tickets: 2589 - - The Beaker caching example has been converted - to use `dogpile.cache `_. - This is a new caching library written by the same - creator of Beaker's caching internals, and represents a - vastly improved, simplified, and modernized system of caching. - - .. seealso:: - - :ref:`examples_caching` - - .. change:: - :tags: general - :tickets: - - SQLAlchemy 0.8 now targets Python 2.5 and - above. Python 2.4 is no longer supported. - - .. change:: - :tags: removed, general - :tickets: 2433 - - The "sqlalchemy.exceptions" - synonym for "sqlalchemy.exc" is removed - fully. - - .. change:: - :tags: removed, orm - :tickets: 2442 - - The legacy "mutable" system of the - ORM, including the MutableType class as well - as the mutable=True flag on PickleType - and postgresql.ARRAY has been removed. - In-place mutations are detected by the ORM - using the sqlalchemy.ext.mutable extension, - introduced in 0.7. The removal of MutableType - and associated constructs removes a great - deal of complexity from SQLAlchemy's internals. - The approach performed poorly as it would incur - a scan of the full contents of the Session - when in use. - - .. change:: - :tags: orm, moved - :tickets: - - The InstrumentationManager interface - and the entire related system of alternate - class implementation is now moved out - to sqlalchemy.ext.instrumentation. This is - a seldom used system that adds significant - complexity and overhead to the mechanics of - class instrumentation. The new architecture - allows it to remain unused until - InstrumentationManager is actually imported, - at which point it is bootstrapped into - the core. - - .. change:: - :tags: orm, feature - :tickets: 1401 - - Major rewrite of relationship() - internals now allow join conditions which - include columns pointing to themselves - within composite foreign keys. A new - API for very specialized primaryjoin conditions - is added, allowing conditions based on - SQL functions, CAST, etc. to be handled - by placing the annotation functions - remote() and foreign() inline within the - expression when necessary. Previous recipes - using the semi-private _local_remote_pairs - approach can be upgraded to this new - approach. - - .. seealso:: - - :ref:`feature_relationship_08` - - .. change:: - :tags: orm, bug - :tickets: 2527 - - ORM will perform extra effort to determine - that an FK dependency between two tables is - not significant during flush if the tables - are related via joined inheritance and the FK - dependency is not part of the inherit_condition, - saves the user a use_alter directive. - - .. change:: - :tags: orm, feature - :tickets: 2333 - - New standalone function with_polymorphic() - provides the functionality of query.with_polymorphic() - in a standalone form. It can be applied to any - entity within a query, including as the target - of a join in place of the "of_type()" modifier. - - .. change:: - :tags: orm, feature - :tickets: 1106, 2438 - - The of_type() construct on attributes - now accepts aliased() class constructs as well - as with_polymorphic constructs, and works with - query.join(), any(), has(), and also - eager loaders subqueryload(), joinedload(), - contains_eager() - - .. change:: - :tags: orm, feature - :tickets: 2585 - - Improvements to event listening for - mapped classes allows that unmapped classes - can be specified for instance- and mapper-events. - The established events will be automatically - set up on subclasses of that class when the - propagate=True flag is passed, and the - events will be set up for that class itself - if and when it is ultimately mapped. - - .. change:: - :tags: orm, bug - :tickets: 2590 - - The instrumentation events class_instrument(), - class_uninstrument(), and attribute_instrument() - will now fire off only for descendant classes - of the class assigned to listen(). Previously, - an event listener would be assigned to listen - for all classes in all cases regardless of the - "target" argument passed. - - .. change:: - :tags: orm, bug - :tickets: 1900 - - with_polymorphic() produces JOINs - in the correct order and with correct inheriting - tables in the case of sending multi-level - subclasses in an arbitrary order or with - intermediary classes missing. - - .. change:: - :tags: orm, feature - :tickets: 2485 - - The "deferred declarative - reflection" system has been moved into the - declarative extension itself, using the - new DeferredReflection class. This - class is now tested with both single - and joined table inheritance use cases. - - .. change:: - :tags: orm, feature - :tickets: 2208 - - Added new core function "inspect()", - which serves as a generic gateway to - introspection into mappers, objects, - others. The Mapper and InstanceState - objects have been enhanced with a public - API that allows inspection of mapped - attributes, including filters for column-bound - or relationship-bound properties, inspection - of current object state, history of - attributes, etc. - - .. change:: - :tags: orm, feature - :tickets: 2452 - - Calling rollback() within a - session.begin_nested() will now only expire - those objects that had net changes within the - scope of that transaction, that is objects which - were dirty or were modified on a flush. This - allows the typical use case for begin_nested(), - that of altering a small subset of objects, to - leave in place the data from the larger enclosing - set of objects that weren't modified in - that sub-transaction. - - .. change:: - :tags: orm, feature - :tickets: 2372 - - Added utility feature - Session.enable_relationship_loading(), - supersedes relationship.load_on_pending. - Both features should be avoided, however. - - .. change:: - :tags: orm, feature - :tickets: - - Added support for .info dictionary argument to - column_property(), relationship(), composite(). - All MapperProperty classes have an auto-creating .info - dict available overall. - - .. change:: - :tags: orm, feature - :tickets: 2229 - - Adding/removing None from a mapped collection - now generates attribute events. Previously, a None - append would be ignored in some cases. Related - to. - - .. change:: - :tags: orm, feature - :tickets: 2229 - - The presence of None in a mapped collection - now raises an error during flush. Previously, - None values in collections would be silently ignored. - - .. change:: - :tags: orm, feature - :tickets: - - The Query.update() method is now - more lenient as to the table - being updated. Plain Table objects are better - supported now, and additional a joined-inheritance - subclass may be used with update(); the subclass - table will be the target of the update, - and if the parent table is referenced in the - WHERE clause, the compiler will call upon - UPDATE..FROM syntax as allowed by the dialect - to satisfy the WHERE clause. MySQL's multi-table - update feature is also supported if columns - are specified by object in the "values" dictionary. - PG's DELETE..USING is also not available - in Core yet. - - .. change:: - :tags: orm, feature - :tickets: - - New session events after_transaction_create - and after_transaction_end - allows tracking of new SessionTransaction objects. - If the object is inspected, can be used to determine - when a session first becomes active and when - it deactivates. - - .. change:: - :tags: orm, feature - :tickets: 2592 - - The Query can now load entity/scalar-mixed - "tuple" rows that contain - types which aren't hashable, by setting the flag - "hashable=False" on the corresponding TypeEngine object - in use. Custom types that return unhashable types - (typically lists) can set this flag to False. - - .. change:: - :tags: orm, bug - :tickets: 2481 - - Improvements to joined/subquery eager - loading dealing with chains of subclass entities - sharing a common base, with no specific "join depth" - provided. Will chain out to - each subclass mapper individually before detecting - a "cycle", rather than considering the base class - to be the source of the "cycle". - - .. change:: - :tags: orm, bug - :tickets: 2320 - - The "passive" flag on Session.is_modified() - no longer has any effect. is_modified() in - all cases looks only at local in-memory - modified flags and will not emit any - SQL or invoke loader callables/initializers. - - .. change:: - :tags: orm, bug - :tickets: 2405 - - The warning emitted when using - delete-orphan cascade with one-to-many - or many-to-many without single-parent=True - is now an error. The ORM - would fail to function subsequent to this - warning in any case. - - .. change:: - :tags: orm, bug - :tickets: 2350 - - Lazy loads emitted within flush events - such as before_flush(), before_update(), - etc. will now function as they would - within non-event code, regarding consideration - of the PK/FK values used in the lazy-emitted - query. Previously, - special flags would be established that - would cause lazy loads to load related items - based on the "previous" value of the - parent PK/FK values specifically when called - upon within a flush; the signal to load - in this way is now localized to where the - unit of work actually needs to load that - way. Note that the UOW does - sometimes load these collections before - the before_update() event is called, - so the usage of "passive_updates" or not - can affect whether or not a collection will - represent the "old" or "new" data, when - accessed within a flush event, based - on when the lazy load was emitted. - The change is backwards incompatible in - the exceedingly small chance that - user event code depended on the old - behavior. - - .. change:: - :tags: orm, feature - :tickets: 2179 - - Query now "auto correlates" by - default in the same way as select() does. - Previously, a Query used as a subquery - in another would require the correlate() - method be called explicitly in order to - correlate a table on the inside to the - outside. As always, correlate(None) - disables correlation. - - .. change:: - :tags: orm, feature - :tickets: 2464 - - The after_attach event is now - emitted after the object is established - in Session.new or Session.identity_map - upon Session.add(), Session.merge(), - etc., so that the object is represented - in these collections when the event - is called. Added before_attach - event to accommodate use cases that - need autoflush w pre-attached object. - - .. change:: - :tags: orm, feature - :tickets: - - The Session will produce warnings - when unsupported methods are used inside the - "execute" portion of the flush. These are - the familiar methods add(), delete(), etc. - as well as collection and related-object - manipulations, as called within mapper-level - flush events - like after_insert(), after_update(), etc. - It's been prominently documented for a long - time that SQLAlchemy cannot guarantee - results when the Session is manipulated within - the execution of the flush plan, - however users are still doing it, so now - there's a warning. Maybe someday the Session - will be enhanced to support these operations - inside of the flush, but for now, results - can't be guaranteed. - - .. change:: - :tags: orm, bug - :tickets: 2582, 2566 - - Continuing regarding extra - state post-flush due to event listeners; - any states that are marked as "dirty" from an - attribute perspective, usually via column-attribute - set events within after_insert(), after_update(), - etc., will get the "history" flag reset - in all cases, instead of only those instances - that were part of the flush. This has the effect - that this "dirty" state doesn't carry over - after the flush and won't result in UPDATE - statements. A warning is emitted to this - effect; the set_committed_state() - method can be used to assign attributes on objects - without producing history events. - - .. change:: - :tags: orm, feature - :tickets: 2245 - - ORM entities can be passed - to the core select() construct as well - as to the select_from(), - correlate(), and correlate_except() - methods of select(), where they will be unwrapped - into selectables. - - .. change:: - :tags: orm, feature - :tickets: 2245 - - Some support for auto-rendering of a - relationship join condition based on the mapped - attribute, with usage of core SQL constructs. - E.g. select([SomeClass]).where(SomeClass.somerelationship) - would render SELECT from "someclass" and use the - primaryjoin of "somerelationship" as the WHERE - clause. This changes the previous meaning - of "SomeClass.somerelationship" when used in a - core SQL context; previously, it would "resolve" - to the parent selectable, which wasn't generally - useful. Also works with query.filter(). - Related to. - - .. change:: - :tags: orm, feature - :tickets: 2526 - - The registry of classes - in declarative_base() is now a - WeakValueDictionary. So subclasses of - "Base" that are dereferenced will be - garbage collected, *if they are not - referred to by any other mappers/superclass - mappers*. See the next note for this ticket. - - .. change:: - :tags: orm, feature - :tickets: 2472 - - Conflicts between columns on - single-inheritance declarative subclasses, - with or without using a mixin, can be resolved - using a new @declared_attr usage described - in the documentation. - - .. change:: - :tags: orm, feature - :tickets: 2472 - - declared_attr can now be used - on non-mixin classes, even though this is generally - only useful for single-inheritance subclass - column conflict resolution. - - .. change:: - :tags: orm, feature - :tickets: 2517 - - declared_attr can now be used with - attributes that are not Column or MapperProperty; - including any user-defined value as well - as association proxy objects. - - .. change:: - :tags: orm, bug - :tickets: 2565 - - Fixed a disconnect that slowly evolved - between a @declared_attr Column and a - directly-defined Column on a mixin. In both - cases, the Column will be applied to the - declared class' table, but not to that of a - joined inheritance subclass. Previously, - the directly-defined Column would be placed - on both the base and the sub table, which isn't - typically what's desired. - - .. change:: - :tags: orm, feature - :tickets: 2526 - - *Very limited* support for - inheriting mappers to be GC'ed when the - class itself is deferenced. The mapper - must not have its own table (i.e. - single table inh only) without polymorphic - attributes in place. - This allows for the use case of - creating a temporary subclass of a declarative - mapped class, with no table or mapping - directives of its own, to be garbage collected - when dereferenced by a unit test. - - .. change:: - :tags: orm, feature - :tickets: 2338 - - Declarative now maintains a registry - of classes by string name as well as by full - module-qualified name. Multiple classes with the - same name can now be looked up based on a module-qualified - string within relationship(). Simple class name - lookups where more than one class shares the same - name now raises an informative error message. - - .. change:: - :tags: orm, feature - :tickets: 2535 - - Can now provide class-bound attributes - that override columns which are of any - non-ORM type, not just descriptors. - - .. change:: - :tags: orm, feature - :tickets: 1729 - - Added with_labels and - reduce_columns keyword arguments to - Query.subquery(), to provide two alternate - strategies for producing queries with uniquely- - named columns. . - - .. change:: - :tags: orm, feature - :tickets: 2476 - - A warning is emitted when a reference - to an instrumented collection is no longer - associated with the parent class due to - expiration/attribute refresh/collection - replacement, but an append - or remove operation is received on the - now-detached collection. - - .. change:: - :tags: orm, bug - :tickets: 2549 - - Declarative can now propagate a column - declared on a single-table inheritance subclass - up to the parent class' table, when the parent - class is itself mapped to a join() or select() - statement, directly or via joined inheritance, - and not just a Table. - - .. change:: - :tags: orm, bug - :tickets: - - An error is emitted when uselist=False - is combined with a "dynamic" loader. - This is a warning in 0.7.9. - - .. change:: - :tags: removed, orm - :tickets: - - Deprecated identifiers removed: - - * allow_null_pks mapper() argument - (use allow_partial_pks) - - * _get_col_to_prop() mapper method - (use get_property_by_column()) - - * dont_load argument to Session.merge() - (use load=True) - - * sqlalchemy.orm.shard module - (use sqlalchemy.ext.horizontal_shard) - - .. change:: - :tags: engine, feature - :tickets: 2511 - - Connection event listeners can - now be associated with individual - Connection objects, not just Engine - objects. - - .. change:: - :tags: engine, feature - :tickets: 2459 - - The before_cursor_execute event - fires off for so-called "_cursor_execute" - events, which are usually special-case - executions of primary-key bound sequences - and default-generation SQL - phrases that invoke separately when RETURNING - is not used with INSERT. - - .. change:: - :tags: engine, feature - :tickets: - - The libraries used by the test suite - have been moved around a bit so that they are - part of the SQLAlchemy install again. In addition, - a new suite of tests is present in the - new sqlalchemy.testing.suite package. This is - an under-development system that hopes to provide - a universal testing suite for external dialects. - Dialects which are maintained outside of SQLAlchemy - can use the new test fixture as the framework - for their own tests, and will get for free a - "compliance" suite of dialect-focused tests, - including an improved "requirements" system - where specific capabilities and features can - be enabled or disabled for testing. - - .. change:: - :tags: engine, bug - :tickets: - - The Inspector.get_table_names() - order_by="foreign_key" feature now sorts - tables by dependee first, to be consistent - with util.sort_tables and metadata.sorted_tables. - - .. change:: - :tags: engine, bug - :tickets: 2522 - - Fixed bug whereby if a database restart - affected multiple connections, each - connection would individually invoke a new - disposal of the pool, even though only - one disposal is needed. - - .. change:: - :tags: engine, feature - :tickets: 2462 - - Added a new system - for registration of new dialects in-process - without using an entrypoint. See the - docs for "Registering New Dialects". - - .. change:: - :tags: engine, feature - :tickets: 2556 - - The "required" flag is set to - True by default, if not passed explicitly, - on bindparam() if the "value" or "callable" - parameters are not passed. - This will cause statement execution to check - for the parameter being present in the final - collection of bound parameters, rather than - implicitly assigning None. - - .. change:: - :tags: engine, feature - :tickets: - - Various API tweaks to the "dialect" - API to better support highly specialized - systems such as the Akiban database, including - more hooks to allow an execution context to - access type processors. - - .. change:: - :tags: engine, bug - :tickets: 2397 - - The names of the columns on the - .c. attribute of a select().apply_labels() - is now based on _ instead - of _, for those columns - that have a distinctly named .key. - - .. change:: - :tags: engine, feature - :tickets: 2422 - - Inspector.get_primary_keys() is - deprecated; use Inspector.get_pk_constraint(). - Courtesy Diana Clarke. - - .. change:: - :tags: engine, bug - :tickets: - - The autoload_replace flag on Table, - when False, will cause any reflected foreign key - constraints which refer to already-declared - columns to be skipped, assuming that the - in-Python declared column will take over - the task of specifying in-Python ForeignKey - or ForeignKeyConstraint declarations. - - .. change:: - :tags: engine, bug - :tickets: 2498 - - The ResultProxy methods inserted_primary_key, - last_updated_params(), last_inserted_params(), - postfetch_cols(), prefetch_cols() all - assert that the given statement is a compiled - construct, and is an insert() or update() - statement as is appropriate, else - raise InvalidRequestError. - - .. change:: - :tags: engine, feature - :tickets: - - New C extension module "utils" has - been added for additional function speedups - as we have time to implement. - - .. change:: - :tags: engine - :tickets: - - ResultProxy.last_inserted_ids is removed, - replaced by inserted_primary_key. - - .. change:: - :tags: feature, sql - :tickets: 2547 - - Major rework of operator system - in Core, to allow redefinition of existing - operators as well as addition of new operators - at the type level. New types can be created - from existing ones which add or redefine - operations that are exported out to column - expressions, in a similar manner to how the - ORM has allowed comparator_factory. The new - architecture moves this capability into the - Core so that it is consistently usable in - all cases, propagating cleanly using existing - type propagation behavior. - - .. change:: - :tags: feature, sql - :tickets: 1534, 2547 - - To complement, types - can now provide "bind expressions" and - "column expressions" which allow compile-time - injection of SQL expressions into statements - on a per-column or per-bind level. This is - to suit the use case of a type which needs - to augment bind- and result- behavior at the - SQL level, as opposed to in the Python level. - Allows for schemes like transparent encryption/ - decryption, usage of PostGIS functions, etc. - - .. change:: - :tags: feature, sql - :tickets: - - The Core operator system now includes - the `getitem` operator, i.e. the bracket - operator in Python. This is used at first - to provide index and slice behavior to the - PostgreSQL ARRAY type, and also provides a hook - for end-user definition of custom __getitem__ - schemes which can be applied at the type - level as well as within ORM-level custom - operator schemes. `lshift` (<<) - and `rshift` (>>) are also supported as - optional operators. - - Note that this change has the effect that - descriptor-based __getitem__ schemes used by - the ORM in conjunction with synonym() or other - "descriptor-wrapped" schemes will need - to start using a custom comparator in order - to maintain this behavior. - - .. change:: - :tags: feature, sql - :tickets: 2537 - - Revised the rules used to determine - the operator precedence for the user-defined - operator, i.e. that granted using the ``op()`` - method. Previously, the smallest precedence - was applied in all cases, now the default - precedence is zero, lower than all operators - except "comma" (such as, used in the argument - list of a ``func`` call) and "AS", and is - also customizable via the "precedence" argument - on the ``op()`` method. - - .. change:: - :tags: feature, sql - :tickets: 2276 - - Added "collation" parameter to all - String types. When present, renders as - COLLATE . This to support the - COLLATE keyword now supported by several - databases including MySQL, SQLite, and PostgreSQL. - - .. change:: - :tags: change, sql - :tickets: - - The Text() type renders the length - given to it, if a length was specified. - - .. change:: - :tags: feature, sql - :tickets: - - Custom unary operators can now be - used by combining operators.custom_op() with - UnaryExpression(). - - .. change:: - :tags: bug, sql - :tickets: 2564 - - A tweak to column precedence which moves the - "concat" and "match" operators to be the same as - that of "is", "like", and others; this helps with - parenthesization rendering when used in conjunction - with "IS". - - .. change:: - :tags: feature, sql - :tickets: - - Enhanced GenericFunction and func.* - to allow for user-defined GenericFunction - subclasses to be available via the func.* - namespace automatically by classname, - optionally using a package name, as well - as with the ability to have the rendered - name different from the identified name - in func.*. - - .. change:: - :tags: feature, sql - :tickets: 2562 - - The cast() and extract() constructs - will now be produced via the func.* accessor - as well, as users naturally try to access these - names from func.* they might as well do - what's expected, even though the returned - object is not a FunctionElement. - - .. change:: - :tags: changed, sql - :tickets: - - Most classes in expression.sql - are no longer preceded with an underscore, - i.e. Label, SelectBase, Generative, CompareMixin. - _BindParamClause is also renamed to - BindParameter. The old underscore names for - these classes will remain available as synonyms - for the foreseeable future. - - .. change:: - :tags: feature, sql - :tickets: 2208 - - The Inspector object can now be - acquired using the new inspect() service, - part of - - .. change:: - :tags: feature, sql - :tickets: 2418 - - The column_reflect event now - accepts the Inspector object as the first - argument, preceding "table". Code which - uses the 0.7 version of this very new - event will need modification to add the - "inspector" object as the first argument. - - .. change:: - :tags: feature, sql - :tickets: 2423 - - The behavior of column targeting - in result sets is now case sensitive by - default. SQLAlchemy for many years would - run a case-insensitive conversion on these values, - probably to alleviate early case sensitivity - issues with dialects like Oracle and - Firebird. These issues have been more cleanly - solved in more modern versions so the performance - hit of calling lower() on identifiers is removed. - The case insensitive comparisons can be re-enabled - by setting "case_insensitive=False" on - create_engine(). - - .. change:: - :tags: bug, sql - :tickets: 2591 - - Applying a column expression to a select - statement using a label with or without other - modifying constructs will no longer "target" that - expression to the underlying Column; this affects - ORM operations that rely upon Column targeting - in order to retrieve results. That is, a query - like query(User.id, User.id.label('foo')) will now - track the value of each "User.id" expression separately - instead of munging them together. It is not expected - that any users will be impacted by this; however, - a usage that uses select() in conjunction with - query.from_statement() and attempts to load fully - composed ORM entities may not function as expected - if the select() named Column objects with arbitrary - .label() names, as these will no longer target to - the Column objects mapped by that entity. - - .. change:: - :tags: feature, sql - :tickets: 2415 - - The "unconsumed column names" warning emitted - when keys are present in insert.values() or update.values() - that aren't in the target table is now an exception. - - .. change:: - :tags: feature, sql - :tickets: 2502 - - Added "MATCH" clause to ForeignKey, - ForeignKeyConstraint, courtesy Ryan Kelly. - - .. change:: - :tags: feature, sql - :tickets: 2507 - - Added support for DELETE and UPDATE from - an alias of a table, which would assumedly - be related to itself elsewhere in the query, - courtesy Ryan Kelly. - - .. change:: - :tags: feature, sql - :tickets: - - select() features a correlate_except() - method, auto correlates all selectables except those - passed. - - .. change:: - :tags: feature, sql - :tickets: 2431 - - The prefix_with() method is now available - on each of select(), insert(), update(), delete(), - all with the same API, accepting multiple - prefix calls, as well as a "dialect name" so that - the prefix can be limited to one kind of dialect. - - .. change:: - :tags: feature, sql - :tickets: 1729 - - Added reduce_columns() method - to select() construct, replaces columns inline - using the util.reduce_columns utility function - to remove equivalent columns. reduce_columns() - also adds "with_only_synonyms" to limit the - reduction just to those columns which have the same - name. The deprecated fold_equivalents() feature is - removed. - - .. change:: - :tags: feature, sql - :tickets: 2470 - - Reworked the startswith(), endswith(), - contains() operators to do a better job with - negation (NOT LIKE), and also to assemble them - at compilation time so that their rendered SQL - can be altered, such as in the case for Firebird - STARTING WITH - - .. change:: - :tags: feature, sql - :tickets: 2463 - - Added a hook to the system of rendering - CREATE TABLE that provides access to the render for each - Column individually, by constructing a @compiles - function against the new schema.CreateColumn - construct. - - .. change:: - :tags: feature, sql - :tickets: - - "scalar" selects now have a WHERE method - to help with generative building. Also slight adjustment - regarding how SS "correlates" columns; the new methodology - no longer applies meaning to the underlying - Table column being selected. This improves - some fairly esoteric situations, and the logic - that was there didn't seem to have any purpose. - - .. change:: - :tags: bug, sql - :tickets: 2520 - - Fixes to the interpretation of the - Column "default" parameter as a callable - to not pass ExecutionContext into a keyword - argument parameter. - - .. change:: - :tags: bug, sql - :tickets: 2410 - - All of UniqueConstraint, ForeignKeyConstraint, - CheckConstraint, and PrimaryKeyConstraint will - attach themselves to their parent table automatically - when they refer to a Table-bound Column object directly - (i.e. not just string column name), and refer to - one and only one Table. Prior to 0.8 this behavior - occurred for UniqueConstraint and PrimaryKeyConstraint, - but not ForeignKeyConstraint or CheckConstraint. - - .. change:: - :tags: bug, sql - :tickets: 2594 - - TypeDecorator now includes a generic repr() - that works in terms of the "impl" type by default. - This is a behavioral change for those TypeDecorator - classes that specify a custom __init__ method; those - types will need to re-define __repr__() if they need - __repr__() to provide a faithful constructor representation. - - .. change:: - :tags: bug, sql - :tickets: 2168 - - column.label(None) now produces an - anonymous label, instead of returning the - column object itself, consistent with the behavior - of label(column, None). - - .. change:: - :tags: feature, sql - :tickets: 2455 - - An explicit error is raised when - a ForeignKeyConstraint() that was - constructed to refer to multiple remote tables - is first used. - - .. change:: - :tags: access, feature - :tickets: - - the MS Access dialect has been - moved to its own project on Bitbucket, - taking advantage of the new SQLAlchemy - dialect compliance suite. The dialect is - still in very rough shape and probably not - ready for general use yet, however - it does have *extremely* rudimental - functionality now. - https://bitbucket.org/zzzeek/sqlalchemy-access - - .. change:: - :tags: maxdb, moved - :tickets: - - The MaxDB dialect, which hasn't been - functional for several years, is - moved out to a pending bitbucket project, - https://bitbucket.org/zzzeek/sqlalchemy-maxdb. - - .. change:: - :tags: sqlite, feature - :tickets: 2363 - - the SQLite date and time types - have been overhauled to support a more open - ended format for input and output, using - name based format strings and regexps. A - new argument "microseconds" also provides - the option to omit the "microseconds" - portion of timestamps. Thanks to - Nathan Wright for the work and tests on - this. - - .. change:: - :tags: mssql, feature - :tickets: - - SQL Server dialect can be given - database-qualified schema names, - i.e. "schema='mydatabase.dbo'"; reflection - operations will detect this, split the schema - among the "." to get the owner separately, - and emit a "USE mydatabase" statement before - reflecting targets within the "dbo" owner; - the existing database returned from - DB_NAME() is then restored. - - .. change:: - :tags: mssql, bug - :tickets: 2277 - - removed legacy behavior whereby - a column comparison to a scalar SELECT via - == would coerce to an IN with the SQL server - dialect. This is implicit - behavior which fails in other scenarios - so is removed. Code which relies on this - needs to be modified to use column.in_(select) - explicitly. - - .. change:: - :tags: mssql, feature - :tickets: - - updated support for the mxodbc - driver; mxodbc 3.2.1 is recommended for full - compatibility. - - .. change:: - :tags: postgresql, feature - :tickets: 2441 - - postgresql.ARRAY features an optional - "dimension" argument, will assign a specific - number of dimensions to the array which will - render in DDL as ARRAY[][]..., also improves - performance of bind/result processing. - - .. change:: - :tags: postgresql, feature - :tickets: - - postgresql.ARRAY now supports - indexing and slicing. The Python [] operator - is available on all SQL expressions that are - of type ARRAY; integer or simple slices can be - passed. The slices can also be used on the - assignment side in the SET clause of an UPDATE - statement by passing them into Update.values(); - see the docs for examples. - - .. change:: - :tags: postgresql, feature - :tickets: - - Added new "array literal" construct - postgresql.array(). Basically a "tuple" that - renders as ARRAY[1,2,3]. - - .. change:: - :tags: postgresql, feature - :tickets: 2506 - - Added support for the PostgreSQL ONLY - keyword, which can appear corresponding to a - table in a SELECT, UPDATE, or DELETE statement. - The phrase is established using with_hint(). - Courtesy Ryan Kelly - - .. change:: - :tags: postgresql, feature - :tickets: - - The "ischema_names" dictionary of the - PostgreSQL dialect is "unofficially" customizable. - Meaning, new types such as PostGIS types can - be added into this dictionary, and the PG type - reflection code should be able to handle simple - types with variable numbers of arguments. - The functionality here is "unofficial" for - three reasons: - - 1. this is not an "official" API. Ideally - an "official" API would allow custom type-handling - callables at the dialect or global level - in a generic way. - 2. This is only implemented for the PG dialect, - in particular because PG has broad support - for custom types vs. other database backends. - A real API would be implemented at the - default dialect level. - 3. The reflection code here is only tested against - simple types and probably has issues with more - compositional types. - - patch courtesy Éric Lemoine. - - .. change:: - :tags: firebird, feature - :tickets: 2470 - - The "startswith()" operator renders - as "STARTING WITH", "~startswith()" renders - as "NOT STARTING WITH", using FB's more efficient - operator. - - .. change:: - :tags: firebird, bug - :tickets: 2505 - - CompileError is raised when VARCHAR with - no length is attempted to be emitted, same - way as MySQL. - - .. change:: - :tags: firebird, bug - :tickets: - - Firebird now uses strict "ansi bind rules" - so that bound parameters don't render in the - columns clause of a statement - they render - literally instead. - - .. change:: - :tags: firebird, bug - :tickets: - - Support for passing datetime as date when - using the DateTime type with Firebird; other - dialects support this. - - .. change:: - :tags: firebird, feature - :tickets: 2504 - - An experimental dialect for the fdb - driver is added, but is untested as I cannot - get the fdb package to build. - - .. change:: - :tags: bug, mysql - :tickets: 2404 - - Dialect no longer emits expensive server - collations query, as well as server casing, - on first connect. These functions are still - available as semi-private. - - .. change:: - :tags: feature, mysql - :tickets: 2534 - - Added TIME type to mysql dialect, - accepts "fst" argument which is the new - "fractional seconds" specifier for recent - MySQL versions. The datatype will interpret - a microseconds portion received from the driver, - however note that at this time most/all MySQL - DBAPIs do not support returning this value. - - .. change:: - :tags: oracle, bug - :tickets: 2437 - - Quoting information is now passed along - from a Column with quote=True when generating - a same-named bound parameter to the bindparam() - object, as is the case in generated INSERT and UPDATE - statements, so that unknown reserved names can - be fully supported. - - .. change:: - :tags: oracle, feature - :tickets: 2561 - - The types of columns excluded from the - setinputsizes() set can be customized by sending - a list of string DBAPI type names to exclude, - using the exclude_setinputsizes dialect parameter. - This list was previously fixed. The list also - now defaults to STRING, UNICODE, removing - CLOB, NCLOB from the list. - - .. change:: - :tags: oracle, bug - :tickets: - - The CreateIndex construct in Oracle - will now schema-qualify the name of the index - to be that of the parent table. Previously this - name was omitted which apparently creates the - index in the default schema, rather than that - of the table. - - .. change:: - :tags: sql, feature - :tickets: 2580 - - Added :meth:`.ColumnOperators.notin_`, - :meth:`.ColumnOperators.notlike`, - :meth:`.ColumnOperators.notilike` to :class:`.ColumnOperators`. - - .. change:: - :tags: sql, removed - - The long-deprecated and non-functional ``assert_unicode`` flag on - :func:`_sa.create_engine` as well as :class:`.String` is removed. diff --git a/doc/build/changelog/changelog_09.rst b/doc/build/changelog/changelog_09.rst deleted file mode 100644 index d00e043326..0000000000 --- a/doc/build/changelog/changelog_09.rst +++ /dev/null @@ -1,3340 +0,0 @@ -============= -0.9 Changelog -============= - -.. changelog_imports:: - - .. include:: changelog_08.rst - :start-line: 5 - - - .. include:: changelog_07.rst - :start-line: 5 - - -.. _unreleased_changelog:: - :version: 0.9.11 - - .. change:: - :tags: bug, oracle, py3k - :tickets: 3491 - :versions: 1.0.9 - - Fixed support for cx_Oracle version 5.2, which was tripping - up SQLAlchemy's version detection under Python 3 and inadvertently - not using the correct unicode mode for Python 3. This would cause - issues such as bound variables mis-interpreted as NULL and rows - silently not being returned. - - .. change:: - :tags: bug, engine - :tickets: 3497 - :versions: 1.0.8 - - Fixed critical issue whereby the pool "checkout" event handler - may be called against a stale connection without the "connect" - event handler having been called, in the case where the pool - attempted to reconnect after being invalidated and failed; the stale - connection would remain present and would be used on a subsequent - attempt. This issue has a greater impact in the 1.0 series subsequent - to 1.0.2, as it also delivers a blanked-out ``.info`` dictionary to - the event handler; prior to 1.0.2 the ``.info`` dictionary is still - the previous one. - -.. changelog:: - :version: 0.9.10 - :released: July 22, 2015 - - .. change:: - :tags: bug, sqlite - :tickets: 3495 - :versions: 1.0.8 - - Fixed bug in SQLite dialect where reflection of UNIQUE constraints - that included non-alphabetic characters in the names, like dots or - spaces, would not be reflected with their name. - - .. change:: - :tags: feature, sql - :tickets: 3418 - :versions: 1.0.5 - - Added official support for a CTE used by the SELECT present - inside of :meth:`_expression.Insert.from_select`. This behavior worked - accidentally up until 0.9.9, when it no longer worked due to - unrelated changes as part of :ticket:`3248`. Note that this - is the rendering of the WITH clause after the INSERT, before the - SELECT; the full functionality of CTEs rendered at the top - level of INSERT, UPDATE, DELETE is a new feature targeted for a - later release. - - .. change:: - :tags: bug, ext - :tickets: 3408 - :versions: 1.0.4 - - Fixed bug where when using extended attribute instrumentation system, - the correct exception would not be raised when :func:`.class_mapper` - were called with an invalid input that also happened to not - be weak referencable, such as an integer. - - .. change:: - :tags: bug, tests, pypy - :tickets: 3406 - :versions: 1.0.4 - - Fixed an import that prevented "pypy setup.py test" from working - correctly. - - .. change:: - :tags: bug, engine - :tickets: 3375 - :versions: 1.0.1 - - Added the string value ``"none"`` to those accepted by the - :paramref:`_pool.Pool.reset_on_return` parameter as a synonym for ``None``, - so that string values can be used for all settings, allowing - utilities like :func:`.engine_from_config` to be usable without - issue. - - .. change:: - :tags: bug, sql - :tickets: 3362 - :versions: 1.0.0 - - Fixed issue where a :class:`_schema.MetaData` object that used a naming - convention would not properly work with pickle. The attribute was - skipped leading to inconsistencies and failures if the unpickled - :class:`_schema.MetaData` object were used to base additional tables - from. - - .. change:: - :tags: bug, postgresql - :tickets: 3354 - :versions: 1.0.0 - - Fixed a long-standing bug where the :class:`.Enum` type as used - with the psycopg2 dialect in conjunction with non-ascii values - and ``native_enum=False`` would fail to decode return results properly. - This stemmed from when the PG :class:`_postgresql.ENUM` type used - to be a standalone type without a "non native" option. - - .. change:: - :tags: bug, orm - :tickets: 3349 - - :class:`_query.Query` doesn't support joins, subselects, or special - FROM clauses when using the :meth:`_query.Query.update` or - :meth:`_query.Query.delete` methods; instead of silently ignoring these - fields if methods like :meth:`_query.Query.join` or - :meth:`_query.Query.select_from` has been called, a warning is emitted. - As of 1.0.0b5 this will raise an error. - - .. change:: - :tags: bug, orm - :tickets: 3352 - :versions: 1.0.0b5 - - Fixed bug where the state tracking within multiple, nested - :meth:`.Session.begin_nested` operations would fail to propagate - the "dirty" flag for an object that had been updated within - the inner savepoint, such that if the enclosing savepoint were - rolled back, the object would not be part of the state that was - expired and therefore reverted to its database state. - - .. change:: - :tags: bug, mysql, pymysql - :tickets: 3337 - :versions: 1.0.0b4 - - Fixed unicode support for PyMySQL when using an "executemany" - operation with unicode parameters. SQLAlchemy now passes both - the statement as well as the bound parameters as unicode - objects, as PyMySQL generally uses string interpolation - internally to produce the final statement, and in the case of - executemany does the "encode" step only on the final statement. - - .. change:: - :tags: bug, py3k, mysql - :tickets: 3333 - :versions: 1.0.0b2 - - Fixed the :class:`.mysql.BIT` type on Py3K which was not using the - ``ord()`` function correctly. Pull request courtesy David Marin. - - .. change:: - :tags: bug, ext - :tickets: 3324 - - Fixed regression from 0.9.9 where the :func:`.as_declarative` - symbol was removed from the ``sqlalchemy.ext.declarative`` - namespace. - - .. change:: - :tags: feature, orm - :tickets: 3320 - :versions: 1.0.0b1 - - Added a new entry ``"entity"`` to the dictionaries returned by - :attr:`_query.Query.column_descriptions`. This refers to the primary ORM - mapped class or aliased class that is referred to by the expression. - Compared to the existing entry for ``"type"``, it will always be - a mapped entity, even if extracted from a column expression, or - None if the given expression is a pure core expression. - See also :ticket:`3403` which repaired a regression in this feature - which was unreleased in 0.9.10 but was released in the 1.0 version. - - -.. changelog:: - :version: 0.9.9 - :released: March 10, 2015 - - .. change:: - :tags: feature, postgresql - :versions: 1.0.0b1 - - Added support for the ``CONCURRENTLY`` keyword with PostgreSQL - indexes, established using ``postgresql_concurrently``. Pull - request courtesy Iuri de Silvio. - - .. seealso:: - - :ref:`postgresql_index_concurrently` - - .. change:: - :tags: bug, ext, py3k - :versions: 1.0.0b1 - - Fixed bug where the association proxy list class would not interpret - slices correctly under Py3K. Pull request courtesy - Gilles Dartiguelongue. - - .. change:: - :tags: feature, sqlite - :versions: 1.0.0b1 - - Added support for partial indexes (e.g. with a WHERE clause) on - SQLite. Pull request courtesy Kai Groner. - - .. seealso:: - - :ref:`sqlite_partial_index` - - .. change:: - :tags: bug, orm - :tickets: 3310 - :versions: 1.0.0b1 - - Fixed bugs in ORM object comparisons where comparison of - many-to-one ``!= None`` would fail if the source were an aliased - class, or if the query needed to apply special aliasing to the - expression due to aliased joins or polymorphic querying; also fixed - bug in the case where comparing a many-to-one to an object state - would fail if the query needed to apply special aliasing - due to aliased joins or polymorphic querying. - - .. change:: - :tags: bug, orm - :tickets: 3309 - :versions: 1.0.0b1 - - Fixed bug where internal assertion would fail in the case where - an ``after_rollback()`` handler for a :class:`.Session` incorrectly - adds state to that :class:`.Session` within the handler, and the task - to warn and remove this state (established by :ticket:`2389`) attempts - to proceed. - - .. change:: - :tags: bug, orm - :versions: 1.0.0b1 - - Fixed bug where TypeError raised when :meth:`_query.Query.join` called - with unknown kw arguments would raise its own TypeError due - to broken formatting. Pull request courtesy Malthe Borch. - - .. change:: - :tags: bug, engine - :tickets: 3302 - :versions: 1.0.0b1 - - Fixed bug in :class:`_engine.Connection` and pool where the - :meth:`_engine.Connection.invalidate` method, or an invalidation due - to a database disconnect, would fail if the - ``isolation_level`` parameter had been used with - :meth:`_engine.Connection.execution_options`; the "finalizer" that resets - the isolation level would be called on the no longer opened connection. - - .. change:: - :tags: feature, orm - :tickets: 3296 - :versions: 1.0.0b1 - - Added new parameter :paramref:`.Session.connection.execution_options` - which may be used to set up execution options on a :class:`_engine.Connection` - when it is first checked out, before the transaction has begun. - This is used to set up options such as isolation level on the - connection before the transaction starts. - - .. seealso:: - - :ref:`session_transaction_isolation` - new documentation section - detailing best practices for setting transaction isolation with - sessions. - - .. change:: - :tags: bug, engine - :tickets: 3296 - :versions: 1.0.0b1 - - A warning is emitted if the ``isolation_level`` parameter is used - with :meth:`_engine.Connection.execution_options` when a :class:`.Transaction` - is in play; DBAPIs and/or SQLAlchemy dialects such as psycopg2, - MySQLdb may implicitly rollback or commit the transaction, or - not change the setting til next transaction, so this is never safe. - - .. change:: - :tags: bug, orm - :tickets: 3300 - :versions: 1.0.0b1 - - Fixed bug in lazy loading SQL construction whereby a complex - primaryjoin that referred to the same "local" column multiple - times in the "column that points to itself" style of self-referential - join would not be substituted in all cases. The logic to determine - substitutions here has been reworked to be more open-ended. - - .. change:: - :tags: bug, postgresql - :tickets: 2940 - :versions: 1.0.0b1 - - Repaired support for PostgreSQL UUID types in conjunction with - the ARRAY type when using psycopg2. The psycopg2 dialect now - employs use of the psycopg2.extras.register_uuid() hook - so that UUID values are always passed to/from the DBAPI as - UUID() objects. The :paramref:`.UUID.as_uuid` flag is still - honored, except with psycopg2 we need to convert returned - UUID objects back into strings when this is disabled. - - .. change:: - :tags: bug, postgresql - :versions: 1.0.0b1 - - Added support for the :class:`postgresql.JSONB` datatype when - using psycopg2 2.5.4 or greater, which features native conversion - of JSONB data so that SQLAlchemy's converters must be disabled; - additionally, the newly added psycopg2 extension - ``extras.register_default_jsonb`` is used to establish a JSON - deserializer passed to the dialect via the ``json_deserializer`` - argument. Also repaired the PostgreSQL integration tests which - weren't actually round-tripping the JSONB type as opposed to the - JSON type. Pull request courtesy Mateusz Susik. - - .. change:: - :tags: bug, postgresql - :versions: 1.0.0b1 - - Repaired the use of the "array_oid" flag when registering the - HSTORE type with older psycopg2 versions < 2.4.3, which does not - support this flag, as well as use of the native json serializer - hook "register_default_json" with user-defined ``json_deserializer`` - on psycopg2 versions < 2.5, which does not include native json. - - .. change:: - :tags: bug, schema - :tickets: 3298, 1765 - - Fixed bug in 0.9's foreign key setup system, such that - the logic used to link a :class:`_schema.ForeignKey` to its parent could fail - when the foreign key used "link_to_name=True" in conjunction with - a target :class:`_schema.Table` that would not receive its parent column until - later, such as within a reflection + "useexisting" scenario, - if the target column in fact had a key value different from its name, - as would occur in reflection if column reflect events were used to - alter the .key of reflected :class:`_schema.Column` objects so that the - link_to_name becomes significant. Also repaired support for column - type via FK transmission in a similar way when target columns had a - different key and were referenced using link_to_name. - - .. change:: - :tags: feature, engine - :versions: 1.0.0b1 - - Added new user-space accessors for viewing transaction isolation - levels; :meth:`_engine.Connection.get_isolation_level`, - :attr:`_engine.Connection.default_isolation_level`. - - .. change:: - :tags: bug, postgresql - :versions: 1.0.0b1 - :tickets: 3174 - - Fixed bug where PostgreSQL dialect would fail to render an - expression in an :class:`.Index` that did not correspond directly - to a table-bound column; typically when a :func:`_expression.text` construct - was one of the expressions within the index; or could misinterpret the - list of expressions if one or more of them were such an expression. - - .. change:: - :tags: bug, orm - :versions: 1.0.0b1 - :tickets: 3287 - - The "wildcard" loader options, in particular the one set up by - the :func:`_orm.load_only` option to cover all attributes not - explicitly mentioned, now takes into account the superclasses - of a given entity, if that entity is mapped with inheritance mapping, - so that attribute names within the superclasses are also omitted - from the load. Additionally, the polymorphic discriminator column - is unconditionally included in the list, just in the same way that - primary key columns are, so that even with load_only() set up, - polymorphic loading of subtypes continues to function correctly. - - .. change:: - :tags: bug, sql - :versions: 1.0.0b1 - - Added the ``native_enum`` flag to the ``__repr__()`` output - of :class:`.Enum`, which is mostly important when using it with - Alembic autogenerate. Pull request courtesy Dimitris Theodorou. - - .. change:: - :tags: bug, orm, pypy - :versions: 1.0.0b1 - :tickets: 3285 - - Fixed bug where if an exception were thrown at the start of a - :class:`_query.Query` before it fetched results, particularly when - row processors can't be formed, the cursor would stay open with - results pending and not actually be closed. This is typically only - an issue on an interpreter like PyPy where the cursor isn't - immediately GC'ed, and can in some circumstances lead to transactions/ - locks being open longer than is desirable. - - .. change:: - :tags: change, mysql - :versions: 1.0.0b1 - :tickets: 3275 - - The ``gaerdbms`` dialect is no longer necessary, and emits a - deprecation warning. Google now recommends using the MySQLdb - dialect directly. - - .. change:: - :tags: bug, sql - :versions: 1.0.0b1 - :tickets: 3278 - - Fixed bug where using a :class:`.TypeDecorator` that implemented - a type that was also a :class:`.TypeDecorator` would fail with - Python's "Cannot create a consistent method resolution order (MRO)" - error, when any kind of SQL comparison expression were used against - an object using this type. - - .. change:: - :tags: bug, mysql - :versions: 1.0.0b1 - :tickets: 3274 - - Added a version check to the MySQLdb dialect surrounding the - check for 'utf8_bin' collation, as this fails on MySQL server < 5.0. - - .. change:: - :tags: feature, orm - :versions: 1.0.0b1 - - Added new method :meth:`.Session.invalidate`, functions similarly - to :meth:`.Session.close`, except also calls - :meth:`_engine.Connection.invalidate` - on all connections, guaranteeing that they will not be returned to - the connection pool. This is useful in situations e.g. dealing - with gevent timeouts when it is not safe to use the connection further, - even for rollbacks. - - .. change:: - :tags: bug, examples - :versions: 1.0.0b1 - - Updated the :ref:`examples_versioned_history` example such that - mapped columns are re-mapped to - match column names as well as grouping of columns; in particular, - this allows columns that are explicitly grouped in a same-column-named - joined inheritance scenario to be mapped in the same way in the - history mappings, avoiding warnings added in the 0.9 series - regarding this pattern and allowing the same view of attribute - keys. - - .. change:: - :tags: bug, examples - :versions: 1.0.0b1 - - Fixed a bug in the examples/generic_associations/discriminator_on_association.py - example, where the subclasses of AddressAssociation were not being - mapped as "single table inheritance", leading to problems when trying - to use the mappings further. - - .. change:: - :tags: bug, orm - :versions: 1.0.0b1 - :tickets: 3251 - - Fixed a leak which would occur in the unsupported and highly - non-recommended use case of replacing a relationship on a fixed - mapped class many times, referring to an arbitrarily growing number of - target mappers. A warning is emitted when the old relationship is - replaced, however if the mapping were already used for querying, the - old relationship would still be referenced within some registries. - - .. change:: - :tags: bug, sql - :versions: 1.0.0b1 - :tickets: 3248 - - Fixed issue where the columns from a SELECT embedded in an - INSERT, either through the values clause or as a "from select", - would pollute the column types used in the result set produced by - the RETURNING clause when columns from both statements shared the - same name, leading to potential errors or mis-adaptation when - retrieving the returning rows. - - .. change:: - :tags: bug, orm, sqlite - :versions: 1.0.0b1 - :tickets: 3241 - - Fixed bug regarding expression mutations which could express - itself as a "Could not locate column" error when using - :class:`_query.Query` to select from multiple, anonymous column - entities when querying against SQLite, as a side effect of the - "join rewriting" feature used by the SQLite dialect. - - .. change:: - :tags: feature, sqlite - :versions: 1.0.0b1 - - Added a new SQLite backend for the SQLCipher backend. This backend - provides for encrypted SQLite databases using the pysqlcipher Python - driver, which is very similar to the pysqlite driver. - - .. seealso:: - - :mod:`~sqlalchemy.dialects.sqlite.pysqlcipher` - - .. change:: - :tags: bug, orm - :tickets: 3232 - :versions: 1.0.0b1 - - Fixed bug where the ON clause for :meth:`_query.Query.join`, - and :meth:`_query.Query.outerjoin` to a single-inheritance subclass - using ``of_type()`` would not render the "single table criteria" in - the ON clause if the ``from_joinpoint=True`` flag were set. - -.. changelog:: - :version: 0.9.8 - :released: October 13, 2014 - - .. change:: - :tags: bug, mysql, mysqlconnector - :versions: 1.0.0b1 - - Mysqlconnector as of version 2.0, probably as a side effect of - the python 3 merge, now does not expect percent signs (e.g. - as used as the modulus operator and others) to be doubled, - even when using the "pyformat" bound parameter format (this - change is not documented by Mysqlconnector). The dialect now - checks for py2k and for mysqlconnector less than version 2.0 - when detecting if the modulus operator should be rendered as - ``%%`` or ``%``. - - .. change:: - :tags: bug, mysql, mysqlconnector - :versions: 1.0.0b1 - - Unicode SQL is now passed for MySQLconnector version 2.0 and above; - for Py2k and MySQL < 2.0, strings are encoded. - - - .. change:: - :tags: bug, oracle - :versions: 1.0.0b1 - :tickets: 2138 - - Fixed long-standing bug in Oracle dialect where bound parameter - names that started with numbers would not be quoted, as Oracle - doesn't like numerics in bound parameter names. - - .. change:: - :tags: bug, sql - :versions: 1.0.0b1 - :tickets: 3195 - - Fixed bug where a fair number of SQL elements within - the sql package would fail to ``__repr__()`` successfully, - due to a missing ``description`` attribute that would then invoke - a recursion overflow when an internal AttributeError would then - re-invoke ``__repr__()``. - - .. change:: - :tags: bug, declarative, orm - :versions: 1.0.0b1 - :tickets: 3185 - - Fixed "'NoneType' object has no attribute 'concrete'" error - when using :class:`.AbstractConcreteBase` in conjunction with - a subclass that declares ``__abstract__``. - - .. change:: - :tags: bug, engine - :versions: 1.0.0b1 - :tickets: 3200 - - The execution options passed to an :class:`_engine.Engine` either via - :paramref:`_sa.create_engine.execution_options` or - :meth:`_engine.Engine.update_execution_options` are not passed to the - special :class:`_engine.Connection` used to initialize the dialect - within the "first connect" event; dialects will usually - perform their own queries in this phase, and none of the - current available options should be applied here. In - particular, the "autocommit" option was causing an attempt to - autocommit within this initial connect which would fail with - an AttributeError due to the non-standard state of the - :class:`_engine.Connection`. - - .. change:: - :tags: bug, sqlite - :versions: 1.0.0b1 - :tickets: 3211 - - When selecting from a UNION using an attached database file, - the pysqlite driver reports column names in cursor.description - as 'dbname.tablename.colname', instead of 'tablename.colname' as - it normally does for a UNION (note that it's supposed to just be - 'colname' for both, but we work around it). The column translation - logic here has been adjusted to retrieve the rightmost token, rather - than the second token, so it works in both cases. Workaround - courtesy Tony Roberts. - - .. change:: - :tags: bug, postgresql - :versions: 1.0.0b1 - :tickets: 3021 - - A revisit to this issue first patched in 0.9.5, apparently - psycopg2's ``.closed`` accessor is not as reliable as we assumed, - so we have added an explicit check for the exception messages - "SSL SYSCALL error: Bad file descriptor" and - "SSL SYSCALL error: EOF detected" when detecting an - is-disconnect scenario. We will continue to consult psycopg2's - connection.closed as a first check. - - .. change:: - :tags: bug, orm, engine - :versions: 1.0.0b1 - :tickets: 3197 - - Fixed bug that affected generally the same classes of event - as that of :ticket:`3199`, when the ``named=True`` parameter - would be used. Some events would fail to register, and others - would not invoke the event arguments correctly, generally in the - case of when an event was "wrapped" for adaption in some other way. - The "named" mechanics have been rearranged to not interfere with - the argument signature expected by internal wrapper functions. - - .. change:: - :tags: bug, declarative - :versions: 1.0.0b1 - :tickets: 3208 - - Fixed an unlikely race condition observed in some exotic end-user - setups, where the attempt to check for "duplicate class name" in - declarative would hit upon a not-totally-cleaned-up weak reference - related to some other class being removed; the check here now ensures - the weakref still references an object before calling upon it further. - - .. change:: - :tags: bug, orm - :versions: 1.0.0b1 - :tickets: 3199 - - Fixed bug that affected many classes of event, particularly - ORM events but also engine events, where the usual logic of - "de duplicating" a redundant call to :func:`.event.listen` - with the same arguments would fail, for those events where the - listener function is wrapped. An assertion would be hit within - registry.py. This assertion has now been integrated into the - deduplication check, with the added bonus of a simpler means - of checking deduplication across the board. - - .. change:: - :tags: bug, mssql - :versions: 1.0.0b1 - :tickets: 3151 - - Fixed the version string detection in the pymssql dialect to - work with Microsoft SQL Azure, which changes the word "SQL Server" - to "SQL Azure". - - .. change:: - :tags: bug, orm - :versions: 1.0.0b1 - :tickets: 3194 - - Fixed warning that would emit when a complex self-referential - primaryjoin contained functions, while at the same time remote_side - was specified; the warning would suggest setting "remote side". - It now only emits if remote_side isn't present. - - .. change:: - :tags: bug, ext - :versions: 1.0.0b1 - :tickets: 3191 - - Fixed bug in ordering list where the order of items would be - thrown off during a collection replace event, if the - reorder_on_append flag were set to True. The fix ensures that the - ordering list only impacts the list that is explicitly associated - with the object. - - .. change:: - :tags: bug, sql - :versions: 1.0.0b1 - :tickets: 3180 - - An adjustment to table/index reflection such that if an index - reports a column that isn't found to be present in the table, - a warning is emitted and the column is skipped. This can occur - for some special system column situations as has been observed - with Oracle. - - .. change:: - :tags: bug, ext - :versions: 1.0.0b1 - - Fixed bug where :class:`.ext.mutable.MutableDict` - failed to implement the ``update()`` dictionary method, thus - not catching changes. Pull request courtesy Matt Chisholm. - - .. change:: - :tags: bug, ext - :versions: 1.0.0b1 - - Fixed bug where a custom subclass of :class:`.ext.mutable.MutableDict` - would not show up in a "coerce" operation, and would instead - return a plain :class:`.ext.mutable.MutableDict`. Pull request - courtesy Matt Chisholm. - - .. change:: - :tags: bug, pool - :versions: 1.0.0b1 - :tickets: 3168 - - Fixed bug in connection pool logging where the "connection checked out" - debug logging message would not emit if the logging were set up using - ``logging.setLevel()``, rather than using the ``echo_pool`` flag. - Tests to assert this logging have been added. This is a - regression that was introduced in 0.9.0. - - .. change:: - :tags: feature, postgresql, pg8000 - :versions: 1.0.0b1 - - Support is added for "sane multi row count" with the pg8000 driver, - which applies mostly to when using versioning with the ORM. - The feature is version-detected based on pg8000 1.9.14 or greater - in use. Pull request courtesy Tony Locke. - - .. change:: - :tags: bug, engine - :versions: 1.0.0b1 - :tickets: 3165 - - The string keys that are used to determine the columns impacted - for an INSERT or UPDATE are now sorted when they contribute towards - the "compiled cache" cache key. These keys were previously not - deterministically ordered, meaning the same statement could be - cached multiple times on equivalent keys, costing both in terms of - memory as well as performance. - - .. change:: - :tags: bug, postgresql - :versions: 1.0.0b1 - :tickets: 3159 - - Fixed bug where PostgreSQL JSON type was not able to persist or - otherwise render a SQL NULL column value, rather than a JSON-encoded - ``'null'``. To support this case, changes are as follows: - - * The value :func:`.null` can now be specified, which will always - result in a NULL value resulting in the statement. - - * A new parameter :paramref:`_types.JSON.none_as_null` is added, which - when True indicates that the Python ``None`` value should be - persisted as SQL NULL, rather than JSON-encoded ``'null'``. - - Retrieval of NULL as None is also repaired for DBAPIs other than - psycopg2, namely pg8000. - - .. change:: - :tags: bug, sql - :versions: 1.0.0b1 - :tickets: 3154 - - Fixed bug in CTE where ``literal_binds`` compiler argument would not - be always be correctly propagated when one CTE referred to another - aliased CTE in a statement. - - .. change:: - :tags: bug, postgresql - :versions: 1.0.0b1 - :tickets: 3075 - - The exception wrapping system for DBAPI errors can now accommodate - non-standard DBAPI exceptions, such as the psycopg2 - TransactionRollbackError. These exceptions will now be raised - using the closest available subclass in ``sqlalchemy.exc``, in the - case of TransactionRollbackError, ``sqlalchemy.exc.OperationalError``. - - .. change:: - :tags: bug, sql - :versions: 1.0.0b1 - :tickets: 3144, 3067 - - Fixed 0.9.7 regression caused by :ticket:`3067` in conjunction with - a mis-named unit test such that so-called "schema" types like - :class:`.Boolean` and :class:`.Enum` could no longer be pickled. - - .. change:: - :tags: bug, postgresql - :versions: 1.0.0b1 - :tickets: 3141 - - Fixed bug in :class:`_postgresql.array` object where comparison - to a plain Python list would fail to use the correct array constructor. - Pull request courtesy Andrew. - - .. change:: - :tags: bug, postgresql - :versions: 1.0.0b1 - :tickets: 3137 - - Added a supported :meth:`.FunctionElement.alias` method to functions, - e.g. the ``func`` construct. Previously, behavior for this method - was undefined. The current behavior mimics that of pre-0.9.4, - which is that the function is turned into a single-column FROM - clause with the given alias name, where the column itself is - anonymously named. - -.. changelog:: - :version: 0.9.7 - :released: July 22, 2014 - - .. change:: - :tags: bug, postgresql, pg8000 - :tickets: 3134 - :versions: 1.0.0b1 - - Fixed bug introduced in 0.9.5 by new pg8000 isolation level feature - where engine-level isolation level parameter would raise an error - on connect. - - .. change:: - :tags: bug, oracle, tests - :tickets: 3128 - :versions: 1.0.0b1 - - Fixed bug in oracle dialect test suite where in one test, - 'username' was assumed to be in the database URL, even though - this might not be the case. - - .. change:: - :tags: bug, orm, eagerloading - :tickets: 3131 - :versions: 1.0.0b1 - - Fixed a regression caused by :ticket:`2976` released in 0.9.4 where - the "outer join" propagation along a chain of joined eager loads - would incorrectly convert an "inner join" along a sibling join path - into an outer join as well, when only descendant paths should be - receiving the "outer join" propagation; additionally, fixed related - issue where "nested" join propagation would take place inappropriately - between two sibling join paths. - - .. change:: - :tags: bug, sqlite - :tickets: 3130 - :versions: 1.0.0b1 - - Fixed a SQLite join rewriting issue where a subquery that is embedded - as a scalar subquery such as within an IN would receive inappropriate - substitutions from the enclosing query, if the same table were present - inside the subquery as were in the enclosing query such as in a - joined inheritance scenario. - - .. change:: - :tags: bug, sql - :tickets: 3067 - :versions: 1.0.0b1 - - Fix bug in naming convention feature where using a check - constraint convention that includes ``constraint_name`` would - then force all :class:`.Boolean` and :class:`.Enum` types to - require names as well, as these implicitly create a - constraint, even if the ultimate target backend were one that does - not require generation of the constraint such as PostgreSQL. - The mechanics of naming conventions for these particular - constraints has been reorganized such that the naming - determination is done at DDL compile time, rather than at - constraint/table construction time. - - .. change:: - :tags: bug, mssql - :tickets: 3025 - - Fixed a regression from 0.9.5 caused by :ticket:`3025` where the - query used to determine "default schema" is invalid in SQL Server 2000. - For SQL Server 2000 we go back to defaulting to the "schema name" - parameter of the dialect, which is configurable but defaults - to 'dbo'. - - .. change:: - :tags: bug, orm - :tickets: 3083, 2736 - :versions: 1.0.0b1 - - Fixed a regression from 0.9.0 due to :ticket:`2736` where the - :meth:`_query.Query.select_from` method no longer set up the "from - entity" of the :class:`_query.Query` object correctly, so that - subsequent :meth:`_query.Query.filter_by` or :meth:`_query.Query.join` - calls would fail to check the appropriate "from" entity when - searching for attributes by string name. - - .. change:: - :tags: bug, sql - :tickets: 3090 - :versions: 1.0.0b1 - - Fixed bug in common table expressions whereby positional bound - parameters could be expressed in the wrong final order - when CTEs were nested in certain ways. - - .. change:: - :tags: bug, sql - :tickets: 3069 - :versions: 1.0.0b1 - - Fixed bug where multi-valued :class:`_expression.Insert` construct would fail - to check subsequent values entries beyond the first one given - for literal SQL expressions. - - .. change:: - :tags: bug, sql - :tickets: 3123 - :versions: 1.0.0b1 - - Added a "str()" step to the dialect_kwargs iteration for - Python version < 2.6.5, working around the - "no unicode keyword arg" bug as these args are passed along as - keyword args within some reflection processes. - - .. change:: - :tags: bug, sql - :tickets: 3122 - :versions: 1.0.0b1 - - The :meth:`.TypeEngine.with_variant` method will now accept a - type class as an argument which is internally converted to an - instance, using the same convention long established by other - constructs such as :class:`_schema.Column`. - - .. change:: - :tags: bug, orm - :tickets: 3117 - - The "evaluator" for query.update()/delete() won't work with multi-table - updates, and needs to be set to `synchronize_session=False` or - `synchronize_session='fetch'`; a warning is now emitted. In - 1.0 this will be promoted to a full exception. - - .. change:: - :tags: bug, tests - :versions: 1.0.0b1 - - Fixed bug where "python setup.py test" wasn't calling into - distutils appropriately, and errors would be emitted at the end - of the test suite. - - .. change:: - :tags: feature, postgresql - :versions: 1.0.0b1 - :tickets: 3078 - - Added kw argument ``postgresql_regconfig`` to the - :meth:`.ColumnOperators.match` operator, allows the "reg config" argument - to be specified to the ``to_tsquery()`` function emitted. - Pull request courtesy Jonathan Vanasco. - - .. change:: - :tags: feature, postgresql - :versions: 1.0.0b1 - - Added support for PostgreSQL JSONB via :class:`_postgresql.JSONB`. Pull request - courtesy Damian Dimmich. - - .. change:: - :tags: feature, mssql - :versions: 1.0.0b1 - - Enabled "multivalues insert" for SQL Server 2008. Pull request - courtesy Albert Cervin. Also expanded the checks for "IDENTITY INSERT" - mode to include when the identity key is present in the - VALUEs clause of the statement. - - .. change:: - :tags: feature, engine - :tickets: 3076 - :versions: 1.0.0b1 - - Added new event :meth:`_events.ConnectionEvents.handle_error`, a more - fully featured and comprehensive replacement for - :meth:`_events.ConnectionEvents.dbapi_error`. - - .. change:: - :tags: bug, orm - :tickets: 3108 - :versions: 1.0.0b1 - - Fixed bug where items that were persisted, deleted, or had a - primary key change within a savepoint block would not - participate in being restored to their former state (not in - session, in session, previous PK) after the outer transaction - were rolled back. - - .. change:: - :tags: bug, orm - :tickets: 3106 - :versions: 1.0.0b1 - - Fixed bug in subquery eager loading in conjunction with - :func:`.with_polymorphic`, the targeting of entities and columns - in the subquery load has been made more accurate with respect - to this type of entity and others. - - .. change:: - :tags: bug, orm - :tickets: 3099 - - Fixed bug involving dynamic attributes, that was again a regression - of :ticket:`3060` from version 0.9.5. A self-referential relationship - with lazy='dynamic' would raise a TypeError within a flush operation. - - .. change:: - :tags: bug, declarative - :tickets: 3097 - :versions: 1.0.0b1 - - Fixed bug when the declarative ``__abstract__`` flag was not being - distinguished for when it was actually the value ``False``. - The ``__abstract__`` flag needs to actually evaluate to a True - value at the level being tested. - -.. changelog:: - :version: 0.9.6 - :released: June 23, 2014 - - .. change:: - :tags: bug, orm - :tickets: 3060 - - Reverted the change for :ticket:`3060` - this is a unit of work - fix that is updated more comprehensively in 1.0 via :ticket:`3061`. - The fix in :ticket:`3060` unfortunately produces a new issue whereby - an eager load of a many-to-one attribute can produce an event - that is interpreted into an attribute change. - -.. changelog:: - :version: 0.9.5 - :released: June 23, 2014 - - .. change:: - :tags: bug, orm - :tickets: 3042 - :versions: 1.0.0b1 - - Additional checks have been added for the case where an inheriting - mapper is implicitly combining one of its column-based attributes - with that of the parent, where those columns normally don't necessarily - share the same value. This is an extension of an existing check that - was added via :ticket:`1892`; however this new check emits only a - warning, instead of an exception, to allow for applications that may - be relying upon the existing behavior. - - .. seealso:: - - :ref:`faq_combining_columns` - - .. change:: - :tags: bug, sql - :tickets: 3023 - :versions: 1.0.0b1 - - The :paramref:`_schema.Column.nullable` flag is implicitly set to ``False`` - when that :class:`_schema.Column` is referred to in an explicit - :class:`.PrimaryKeyConstraint` for that table. This behavior now - matches that of when the :class:`_schema.Column` itself has the - :paramref:`_schema.Column.primary_key` flag set to ``True``, which is - intended to be an exactly equivalent case. - - .. change:: - :tags: enhancement, postgresql - :tickets: 3002 - :versions: 1.0.0b1 - - Added a new type :class:`_postgresql.OID` to the PostgreSQL dialect. - While "oid" is generally a private type within PG that is not exposed - in modern versions, there are some PG use cases such as large object - support where these types might be exposed, as well as within some - user-reported schema reflection use cases. - - .. change:: - :tags: bug, orm - :tickets: 3080 - :versions: 1.0.0b1 - - Modified the behavior of :func:`_orm.load_only` such that primary key - columns are always added to the list of columns to be "undeferred"; - otherwise, the ORM can't load the row's identity. Apparently, - one can defer the mapped primary keys and the ORM will fail, that - hasn't been changed. But as load_only is essentially saying - "defer all but X", it's more critical that PK cols not be part of this - deferral. - - .. change:: - :tags: feature, examples - :versions: 1.0.0b1 - - Added a new example illustrating materialized paths, using the - latest relationship features. Example courtesy Jack Zhou. - - .. change:: - :tags: bug, testsuite - :versions: 1.0.0b1 - - In public test suite, changed to use of ``String(40)`` from - less-supported ``Text`` in ``StringTest.test_literal_backslashes``. - Pullreq courtesy Jan. - - .. change:: - :tags: bug, engine - :versions: 1.0.0b1 - :tickets: 3063 - - Fixed bug which would occur if a DBAPI exception - occurs when the engine first connects and does its initial checks, - and the exception is not a disconnect exception, yet the cursor - raises an error when we try to close it. In this case the real - exception would be quashed as we tried to log the cursor close - exception via the connection pool and failed, as we were trying - to access the pool's logger in a way that is inappropriate - in this very specific scenario. - - .. change:: - :tags: feature, postgresql - :versions: 1.0.0b1 - - Added support for AUTOCOMMIT isolation level when using the pg8000 - DBAPI. Pull request courtesy Tony Locke. - - .. change:: - :tags: bug, postgresql - :tickets: 3021 - :versions: 1.0.0b1 - - The psycopg2 ``.closed`` accessor is now consulted when determining - if an exception is a "disconnect" error; ideally, this should remove - the need for any other inspection of the exception message to detect - disconnect, however we will leave those existing messages in place - as a fallback. This should be able to handle newer cases like - "SSL EOF" conditions. Pull request courtesy Dirk Mueller. - - .. change:: - :tags: bug, orm - :tickets: 3060 - :versions: 1.0.0b1 - - Fixed a few edge cases which arise in the so-called "row switch" - scenario, where an INSERT/DELETE can be turned into an UPDATE. - In this situation, a many-to-one relationship set to None, or - in some cases a scalar attribute set to None, may not be detected - as a net change in value, and therefore the UPDATE would not reset - what was on the previous row. This is due to some as-yet - unresolved side effects of the way attribute history works in terms - of implicitly assuming None isn't really a "change" for a previously - un-set attribute. See also :ticket:`3061`. - - .. note:: - - This change has been **REVERTED** in 0.9.6. The full fix - will be in version 1.0 of SQLAlchemy. - - - .. change:: - :tags: bug, orm - :versions: 1.0.0b1 - - Related to :ticket:`3060`, an adjustment has been made to the unit - of work such that loading for related many-to-one objects is slightly - more aggressive, in the case of a graph of self-referential objects - that are to be deleted; the load of related objects is to help - determine the correct order for deletion if passive_deletes is - not set. - - .. change:: - :tags: bug, orm - :tickets: 3057 - :versions: 1.0.0b1 - - Fixed bug in SQLite join rewriting where anonymized column names - due to repeats would not correctly be rewritten in subqueries. - This would affect SELECT queries with any kind of subquery + join. - - .. change:: - :tags: bug, sql - :tickets: 3012 - :versions: 1.0.0b1 - - Fixed bug where the :meth:`.Operators.__and__`, - :meth:`.Operators.__or__` and :meth:`.Operators.__invert__` - operator overload methods could not be overridden within a custom - :class:`.TypeEngine.Comparator` implementation. - - .. change:: - :tags: feature, postgresql - :tickets: 2785 - :versions: 1.0.0b1 - - Added a new flag :paramref:`_types.ARRAY.zero_indexes` to the PostgreSQL - :class:`_types.ARRAY` type. When set to ``True``, a value of one will be - added to all array index values before passing to the database, allowing - better interoperability between Python style zero-based indexes and - PostgreSQL one-based indexes. Pull request courtesy Alexey Terentev. - - .. change:: - :tags: bug, engine - :tickets: 3043 - :versions: 1.0.0b1 - - Fixed some "double invalidate" situations were detected where - a connection invalidation could occur within an already critical section - like a connection.close(); ultimately, these conditions are caused - by the change in :ticket:`2907`, in that the "reset on return" feature - calls out to the Connection/Transaction in order to handle it, where - "disconnect detection" might be caught. However, it's possible that - the more recent change in :ticket:`2985` made it more likely for this - to be seen as the "connection invalidate" operation is much quicker, - as the issue is more reproducible on 0.9.4 than 0.9.3. - - Checks are now added within any section that - an invalidate might occur to halt further disallowed operations - on the invalidated connection. This includes two fixes both at the - engine level and at the pool level. While the issue was observed - with highly concurrent gevent cases, it could in theory occur in - any kind of scenario where a disconnect occurs within the connection - close operation. - - .. change:: - :tags: feature, orm - :tickets: 3029 - :versions: 1.0.0b1 - - The "primaryjoin" model has been stretched a bit further to allow - a join condition that is strictly from a single column to itself, - translated through some kind of SQL function or expression. This - is kind of experimental, but the first proof of concept is a - "materialized path" join condition where a path string is compared - to itself using "like". The :meth:`.ColumnOperators.like` operator has - also been added to the list of valid operators to use in a primaryjoin - condition. - - .. change:: - :tags: feature, sql - :tickets: 3028 - :versions: 1.0.0b1 - - Liberalized the contract for :class:`.Index` a bit in that you can - specify a :func:`_expression.text` expression as the target; the index no longer - needs to have a table-bound column present if the index is to be - manually added to the table, either via inline declaration or via - :meth:`_schema.Table.append_constraint`. - - .. change:: - :tags: bug, firebird - :tickets: 3038 - - Fixed bug where the combination of "limit" rendering as - "SELECT FIRST n ROWS" using a bound parameter (only firebird has both), - combined with column-level subqueries - which also feature "limit" as well as "positional" bound parameters - (e.g. qmark style) would erroneously assign the subquery-level positions - before that of the enclosing SELECT, thus returning parameters which - are out of order. - - .. change:: - :tags: bug, mssql - :tickets: 3025 - :versions: 1.0.0b1 - - Revised the query used to determine the current default schema name - to use the ``database_principal_id()`` function in conjunction with - the ``sys.database_principals`` view so that we can determine - the default schema independently of the type of login in progress - (e.g., SQL Server, Windows, etc). - - .. change:: - :tags: bug, sql - :tickets: 3024 - :versions: 1.0.0b1 - - Fixed bug in new :meth:`.DialectKWArgs.argument_for` method where - adding an argument for a construct not previously included for any - special arguments would fail. - - .. change:: - :tags: bug, py3k, tests - :tickets: 2830 - :versions: 1.0.0b1 - - Corrected for some deprecation warnings involving the ``imp`` - module and Python 3.3 or greater, when running tests. Pull - request courtesy Matt Chisholm. - - .. change:: - :tags: bug, sql - :tickets: 3020, 1068 - :versions: 1.0.0b1 - - Fixed regression introduced in 0.9 where new "ORDER BY " - feature from :ticket:`1068` would not apply quoting rules to the - label name as rendered in the ORDER BY. - - .. change:: - :tags: feature, orm - :tickets: 3017 - :versions: 1.0.0b1 - - Added new utility function :func:`.make_transient_to_detached` which can - be used to manufacture objects that behave as though they were loaded - from a session, then detached. Attributes that aren't present - are marked as expired, and the object can be added to a Session - where it will act like a persistent one. - - .. change:: - :tags: bug, sql - :versions: 1.0.0b1 - - Restored the import for :class:`.Function` to the ``sqlalchemy.sql.expression`` - import namespace, which was removed at the beginning of 0.9. - - .. change:: - :tags: bug, orm, sql - :tickets: 3013 - :versions: 1.0.0b1 - - Fixes to the newly enhanced boolean coercion in :ticket:`2804` where - the new rules for "where" and "having" wouldn't take effect for the - "whereclause" and "having" kw arguments of the :func:`_expression.select` construct, - which is also what :class:`_query.Query` uses so wasn't working in the - ORM either. - - .. change:: - :tags: feature, sql - :tickets: 2990 - :versions: 1.0.0b1 - - Added new flag :paramref:`.expression.between.symmetric`, when set to True - renders "BETWEEN SYMMETRIC". Also added a new negation operator - "notbetween_op", which now allows an expression like ``~col.between(x, y)`` - to render as "col NOT BETWEEN x AND y", rather than a parenthesized NOT - string. - -.. changelog:: - :version: 0.9.4 - :released: March 28, 2014 - - .. change:: - :tags: feature, orm - :tickets: 3007 - - Added new parameter :paramref:`.orm.mapper.confirm_deleted_rows`. Defaults - to True, indicates that a series of DELETE statements should confirm - that the cursor rowcount matches the number of primary keys that should - have matched; this behavior had been taken off in most cases - (except when version_id is used) to support the unusual edge case of - self-referential ON DELETE CASCADE; to accommodate this, the message - is now just a warning, not an exception, and the flag can be used - to indicate a mapping that expects self-referential cascaded - deletes of this nature. See also :ticket:`2403` for background on the - original change. - - .. change:: - :tags: bug, ext, automap - :tickets: 3004 - - Added support to automap for the case where a relationship should - not be created between two classes that are in a joined inheritance - relationship, for those foreign keys that link the subclass back to - the superclass. - - .. change:: - :tags: bug, orm - :tickets: 2948 - - Fixed a very old behavior where the lazy load emitted for a one-to-many - could inappropriately pull in the parent table, and also return results - inconsistent based on what's in the parent table, when the primaryjoin - includes some kind of discriminator against the parent table, such - as ``and_(parent.id == child.parent_id, parent.deleted == False)``. - While this primaryjoin doesn't make that much sense for a one-to-many, - it is slightly more common when applied to the many-to-one side, and - the one-to-many comes as a result of a backref. - Loading rows from ``child`` in this case would keep ``parent.deleted == False`` - as is within the query, thereby yanking it into the FROM clause - and doing a cartesian product. The new behavior will now substitute - the value of the local "parent.deleted" for that parameter as is - appropriate. Though typically, a real-world app probably wants to use a - different primaryjoin for the o2m side in any case. - - .. change:: - :tags: bug, orm - :tickets: 2965 - - Improved the check for "how to join from A to B" such that when - a table has multiple, composite foreign keys targeting a parent table, - the :paramref:`_orm.relationship.foreign_keys` argument will be properly - interpreted in order to resolve the ambiguity; previously this condition - would raise that there were multiple FK paths when in fact the - foreign_keys argument should be establishing which one is expected. - - .. change:: - :tags: bug, mysql - - Tweaked the settings for mysql-connector-python; in Py2K, the - "supports unicode statements" flag is now False, so that SQLAlchemy - will encode the *SQL string* (note: *not* the parameters) - to bytes before sending to the database. This seems to allow - all unicode-related tests to pass for mysql-connector, including those - that use non-ascii table/column names, as well as some tests for the - TEXT type using unicode under cursor.executemany(). - - .. change:: - :tags: feature, engine - - Added some new event mechanics for dialect-level events; the initial - implementation allows an event handler to redefine the specific mechanics - by which an arbitrary dialect invokes execute() or executemany() on a - DBAPI cursor. The new events, at this point semi-public and experimental, - are in support of some upcoming transaction-related extensions. - - .. change:: - :tags: feature, engine - :tickets: 2978 - - An event listener can now be associated with a :class:`_engine.Engine`, - after one or more :class:`_engine.Connection` objects have been created - (such as by an orm :class:`.Session` or via explicit connect) - and the listener will pick up events from those connections. - Previously, performance concerns pushed the event transfer from - :class:`_engine.Engine` to :class:`_engine.Connection` at init-time only, but - we've inlined a bunch of conditional checks to make this possible - without any additional function calls. - - .. change:: - :tags: bug, tests - :tickets: 2980 - - Fixed a few errant ``u''`` strings that would prevent tests from passing - in Py3.2. Patch courtesy Arfrever Frehtes Taifersar Arahesis. - - .. change:: - :tags: bug, engine - :tickets: 2985 - - A major improvement made to the mechanics by which the :class:`_engine.Engine` - recycles the connection pool when a "disconnect" condition is detected; - instead of discarding the pool and explicitly closing out connections, - the pool is retained and a "generational" timestamp is updated to - reflect the current time, thereby causing all existing connections - to be recycled when they are next checked out. This greatly simplifies - the recycle process, removes the need for "waking up" connect attempts - waiting on the old pool and eliminates the race condition that many - immediately-discarded "pool" objects could be created during the - recycle operation. - - .. change:: - :tags: bug, oracle - :tickets: 2987 - - Added new datatype :class:`_oracle.DATE`, which is a subclass of - :class:`.DateTime`. As Oracle has no "datetime" type per se, - it instead has only ``DATE``, it is appropriate here that the - ``DATE`` type as present in the Oracle dialect be an instance of - :class:`.DateTime`. This issue doesn't change anything as far as - the behavior of the type, as data conversion is handled by the - DBAPI in any case, however the improved subclass layout will help - the use cases of inspecting types for cross-database compatibility. - Also removed uppercase ``DATETIME`` from the Oracle dialect as this - type isn't functional in that context. - - .. change:: - :tags: bug, sql - :tickets: 2988 - - Fixed an 0.9 regression where a :class:`_schema.Table` that failed to - reflect correctly wouldn't be removed from the parent - :class:`_schema.MetaData`, even though in an invalid state. Pullreq - courtesy Roman Podoliaka. - - .. change:: - :tags: bug, engine - - The :meth:`_events.ConnectionEvents.after_cursor_execute` event is now - emitted for the "_cursor_execute()" method of :class:`_engine.Connection`; - this is the "quick" executor that is used for things like - when a sequence is executed ahead of an INSERT statement, as well as - for dialect startup checks like unicode returns, charset, etc. - the :meth:`_events.ConnectionEvents.before_cursor_execute` event was already - invoked here. The "executemany" flag is now always set to False - here, as this event always corresponds to a single execution. - Previously the flag could be True if we were acting on behalf of - an executemany INSERT statement. - - .. change:: - :tags: bug, orm - - Added support for the not-quite-yet-documented ``insert=True`` - flag for :func:`.event.listen` to work with mapper / instance events. - - .. change:: - :tags: feature, sql - - Added support for literal rendering of boolean values, e.g. - "true" / "false" or "1" / "0". - - .. change:: - :tags: feature, sql - - Added a new feature :func:`_schema.conv`, the purpose of which is to - mark a constraint name as already having had a naming convention applied. - This token will be used by Alembic migrations as of Alembic 0.6.4 - in order to render constraints in migration scripts with names marked - as already having been subject to a naming convention. - - .. change:: - :tags: bug, sql - - :paramref:`_schema.MetaData.naming_convention` feature will now also - apply to :class:`.CheckConstraint` objects that are associated - directly with a :class:`_schema.Column` instead of just on the - :class:`_schema.Table`. - - .. change:: - :tags: bug, sql - :tickets: 2991 - - Fixed bug in new :paramref:`_schema.MetaData.naming_convention` feature - where the name of a check constraint making use of the - `"%(constraint_name)s"` token would get doubled up for the - constraint generated by a boolean or enum type, and overall - duplicate events would cause the `"%(constraint_name)s"` token - to keep compounding itself. - - .. change:: - :tags: feature, orm - - A warning is emitted if the :meth:`.MapperEvents.before_configured` - or :meth:`.MapperEvents.after_configured` events are applied to a - specific mapper or mapped class, as the events are only invoked - for the :class:`_orm.Mapper` target at the general level. - - .. change:: - :tags: feature, orm - - Added a new keyword argument ``once=True`` to :func:`.event.listen` - and :func:`.event.listens_for`. This is a convenience feature which - will wrap the given listener such that it is only invoked once. - - .. change:: - :tags: feature, oracle - :tickets: 2911 - - Added a new engine option ``coerce_to_unicode=True`` to the - cx_Oracle dialect, which restores the cx_Oracle outputtypehandler - approach to Python unicode conversion under Python 2, which was - removed in 0.9.2 as a result of :ticket:`2911`. Some use cases would - prefer that unicode coercion is unconditional for all string values, - despite performance concerns. Pull request courtesy - Christoph Zwerschke. - - .. change:: - :tags: bug, pool - - Fixed small issue in :class:`.SingletonThreadPool` where the current - connection to be returned might get inadvertently cleaned out during - the "cleanup" process. Patch courtesy jd23. - - .. change:: - :tags: bug, ext, py3k - - Fixed bug in association proxy where assigning an empty slice - (e.g. ``x[:] = [...]``) would fail on Py3k. - - .. change:: - :tags: bug, general - :tickets: 2979 - - Fixed some test/feature failures occurring in Python 3.4, - in particular the logic used to wrap "column default" callables - wouldn't work properly for Python built-ins. - - .. change:: - :tags: feature, general - - Support has been added for pytest to run tests. This runner - is currently being supported in addition to nose, and will likely - be preferred to nose going forward. The nose plugin system used - by SQLAlchemy has been split out so that it works under pytest as - well. There are no plans to drop support for nose at the moment - and we hope that the test suite itself can continue to remain as - agnostic of testing platform as possible. See the file - README.unittests.rst for updated information on running tests - with pytest. - - The test plugin system has also been enhanced to support running - tests against multiple database URLs at once, by specifying the ``--db`` - and/or ``--dburi`` flags multiple times. This does not run the entire test - suite for each database, but instead allows test cases that are specific - to certain backends make use of that backend as the test is run. - When using pytest as the test runner, the system will also run - specific test suites multiple times, once for each database, particularly - those tests within the "dialect suite". The plan is that the enhanced - system will also be used by Alembic, and allow Alembic to run - migration operation tests against multiple backends in one run, including - third-party backends not included within Alembic itself. - Third party dialects and extensions are also encouraged to standardize - on SQLAlchemy's test suite as a basis; see the file README.dialects.rst - for background on building out from SQLAlchemy's test platform. - - .. change:: - :tags: feature, orm - :tickets: 2976 - - Added a new option to :paramref:`_orm.relationship.innerjoin` which is - to specify the string ``"nested"``. When set to ``"nested"`` as opposed - to ``True``, the "chaining" of joins will parenthesize the inner join on the - right side of an existing outer join, instead of chaining as a string - of outer joins. This possibly should have been the default behavior - when 0.9 was released, as we introduced the feature of right-nested - joins in the ORM, however we are keeping it as a non-default for now - to avoid further surprises. - - .. seealso:: - - :ref:`feature_2976` - - .. change:: - :tags: bug, ext - :tickets: 2810 - - Fixed a regression in association proxy caused by :ticket:`2810` which - caused a user-provided "getter" to no longer receive values of ``None`` - when fetching scalar values from a target that is non-present. The - check for None introduced by this change is now moved into the default - getter, so a user-provided getter will also again receive values of - None. - - .. change:: - :tags: bug, sql - :tickets: 2974 - - Adjusted the logic which applies names to the .c collection when - a no-name :class:`.BindParameter` is received, e.g. via :func:`_expression.literal` - or similar; the "key" of the bind param is used as the key within - .c. rather than the rendered name. Since these binds have "anonymous" - names in any case, this allows individual bound parameters to - have their own name within a selectable if they are otherwise unlabeled. - - .. change:: - :tags: bug, sql - :tickets: 2974 - - Some changes to how the :attr:`_expression.FromClause.c` collection behaves - when presented with duplicate columns. The behavior of emitting a - warning and replacing the old column with the same name still - remains to some degree; the replacement in particular is to maintain - backwards compatibility. However, the replaced column still remains - associated with the ``c`` collection now in a collection ``._all_columns``, - which is used by constructs such as aliases and unions, to deal with - the set of columns in ``c`` more towards what is actually in the - list of columns rather than the unique set of key names. This helps - with situations where SELECT statements with same-named columns - are used in unions and such, so that the union can match the columns - up positionally and also there's some chance of :meth:`_expression.FromClause.corresponding_column` - still being usable here (it can now return a column that is only - in selectable.c._all_columns and not otherwise named). - The new collection is underscored as we still need to decide where this - list might end up. Theoretically it - would become the result of iter(selectable.c), however this would mean - that the length of the iteration would no longer match the length of - keys(), and that behavior needs to be checked out. - - .. change:: - :tags: bug, sql - - Fixed issue in new :meth:`_expression.TextClause.columns` method where the ordering - of columns given positionally would not be preserved. This could - have potential impact in positional situations such as applying the - resulting :class:`.TextAsFrom` object to a union. - - .. change:: - :tags: feature, sql - :tickets: 2962, 2866 - - The new dialect-level keyword argument system for schema-level - constructs has been enhanced in order to assist with existing - schemes that rely upon addition of ad-hoc keyword arguments to - constructs. - - E.g., a construct such as :class:`.Index` will again accept - ad-hoc keyword arguments within the :attr:`.Index.kwargs` collection, - after construction:: - - idx = Index("a", "b") - idx.kwargs["mysql_someargument"] = True - - To suit the use case of allowing custom arguments at construction time, - the :meth:`.DialectKWArgs.argument_for` method now allows this registration:: - - Index.argument_for("mysql", "someargument", False) - - idx = Index("a", "b", mysql_someargument=True) - - .. seealso:: - - :meth:`.DialectKWArgs.argument_for` - - .. change:: - :tags: bug, orm, engine - :tickets: 2973 - - Fixed bug where events set to listen at the class - level (e.g. on the :class:`_orm.Mapper` or :class:`.ClassManager` - level, as opposed to on an individual mapped class, and also on - :class:`_engine.Connection`) that also made use of internal argument conversion - (which is most within those categories) would fail to be removable. - - .. change:: - :tags: bug, orm - - Fixed regression from 0.8 where using an option like - :func:`_orm.lazyload` with the "wildcard" expression, e.g. ``"*"``, - would raise an assertion error in the case where the query didn't - contain any actual entities. This assertion is meant for other cases - and was catching this one inadvertently. - - .. change:: - :tags: bug, examples - - Fixed bug in the versioned_history example where column-level INSERT - defaults would prevent history values of NULL from being written. - - .. change:: - :tags: orm, bug, sqlite - :tickets: 2969 - - More fixes to SQLite "join rewriting"; the fix from :ticket:`2967` - implemented right before the release of 0.9.3 affected the case where - a UNION contained nested joins in it. "Join rewriting" is a feature - with a wide range of possibilities and is the first intricate - "SQL rewriting" feature we've introduced in years, so we're sort of - going through a lot of iterations with it (not unlike eager loading - back in the 0.2/0.3 series, polymorphic loading in 0.4/0.5). We should - be there soon so thanks for bearing with us :). - - -.. changelog:: - :version: 0.9.3 - :released: February 19, 2014 - - .. change:: - :tags: orm, bug, sqlite - :tickets: 2967 - - Fixed bug in SQLite "join rewriting" where usage of an exists() construct - would fail to be rewritten properly, such as when the exists is - mapped to a column_property in an intricate nested-join scenario. - Also fixed a somewhat related issue where join rewriting would fail - on the columns clause of the SELECT statement if the targets were - aliased tables, as opposed to individual aliased columns. - - .. change:: - :tags: sqlite, bug - - The SQLite dialect will now skip unsupported arguments when reflecting - types; such as if it encounters a string like ``INTEGER(5)``, the - :class:`_types.INTEGER` type will be instantiated without the "5" being included, - based on detecting a ``TypeError`` on the first attempt. - - .. change:: - :tags: sqlite, bug - - Support has been added to SQLite type reflection to fully support - the "type affinity" contract specified at https://www.sqlite.org/datatype3.html. - In this scheme, keywords like ``INT``, ``CHAR``, ``BLOB`` or - ``REAL`` located in the type name generically associate the type with - one of five affinities. Pull request courtesy Erich Blume. - - .. seealso:: - - :ref:`sqlite_type_reflection` - - .. change:: - :tags: postgresql, feature - - Added the :attr:`.TypeEngine.python_type` convenience accessor onto the - :class:`_postgresql.ARRAY` type. Pull request courtesy Alexey Terentev. - - .. change:: - :tags: examples, feature - - Added optional "changed" column to the versioned rows example, as well - as support for when the versioned :class:`_schema.Table` has an explicit - :paramref:`_schema.Table.schema` argument. Pull request - courtesy jplaverdure. - - .. change:: - :tags: bug, postgresql - :tickets: 2946 - - Added server version detection to the newly added dialect startup - query for "show standard_conforming_strings"; as this variable was - added as of PG 8.2, we skip the query for PG versions who report a - version string earlier than that. - - .. change:: - :tags: bug, orm, declarative - :tickets: 2950 - - Fixed bug where :class:`.AbstractConcreteBase` would fail to be - fully usable within declarative relationship configuration, as its - string classname would not be available in the registry of classnames - at mapper configuration time. The class now explicitly adds itself - to the class registry, and additionally both :class:`.AbstractConcreteBase` - as well as :class:`.ConcreteBase` set themselves up *before* mappers - are configured within the :func:`.configure_mappers` setup, using - the new :meth:`.MapperEvents.before_configured` event. - - .. change:: - :tags: feature, orm - - Added new :meth:`.MapperEvents.before_configured` event which allows - an event at the start of :func:`.configure_mappers`, as well - as ``__declare_first__()`` hook within declarative to complement - ``__declare_last__()``. - - .. change:: - :tags: bug, mysql, cymysql - :tickets: 2934 - - Fixed bug in cymysql dialect where a version string such as - ``'33a-MariaDB'`` would fail to parse properly. Pull request - courtesy Matt Schmidt. - - .. change:: - :tags: bug, orm - :tickets: 2949 - - Fixed an 0.9 regression where ORM instance or mapper events applied - to a base class such as a declarative base with the propagate=True - flag would fail to apply to existing mapped classes which also - used inheritance due to an assertion. Additionally, repaired an - attribute error which could occur during removal of such an event, - depending on how it was first assigned. - - .. change:: - :tags: bug, ext - - Fixed bug where the :class:`.AutomapBase` class of the - new automap extension would fail if classes - were pre-arranged in single or potentially joined inheritance patterns. - The repaired joined inheritance issue could also potentially apply when - using :class:`.DeferredReflection` as well. - - - .. change:: - :tags: bug, sql - - Fixed regression in new "naming convention" feature where conventions - would fail if the referred table in a foreign key contained a schema - name. Pull request courtesy Thomas Farvour. - - .. change:: - :tags: bug, sql - - Fixed bug where so-called "literal render" of :func:`.bindparam` - constructs would fail if the bind were constructed with a callable, - rather than a direct value. This prevented ORM expressions - from being rendered with the "literal_binds" compiler flag. - - .. change:: - :tags: bug, orm - :tickets: 2935 - - Improved the initialization logic of composite attributes such that - calling ``MyClass.attribute`` will not require that the configure - mappers step has occurred, e.g. it will just work without throwing - any error. - - .. change:: - :tags: bug, orm - :tickets: 2932 - - More issues with [ticket:2932] first resolved in 0.9.2 where - using a column key of the form ``_`` - matching that of an aliased column in the text would still not - match at the ORM level, which is ultimately due to a core - column-matching issue. Additional rules have been added so that the - column ``_label`` is taken into account when working with a - :class:`.TextAsFrom` construct or with literal columns. - -.. changelog:: - :version: 0.9.2 - :released: February 2, 2014 - - .. change:: - :tags: bug, examples - - Added a tweak to the "history_meta" example where the check for - "history" on a relationship-bound attribute will now no longer emit - any SQL if the relationship is unloaded. - - .. change:: - :tags: feature, sql - - Added :paramref:`.MetaData.reflect.dialect_kwargs` - to support dialect-level reflection options for all :class:`_schema.Table` - objects reflected. - - .. change:: - :tags: feature, postgresql - :tickets: 2922 - - Added a new dialect-level argument ``postgresql_ignore_search_path``; - this argument is accepted by both the :class:`_schema.Table` constructor - as well as by the :meth:`_schema.MetaData.reflect` method. When in use - against PostgreSQL, a foreign-key referenced table which specifies - a remote schema name will retain that schema name even if the name - is present in the ``search_path``; the default behavior since 0.7.3 - has been that schemas present in ``search_path`` would not be copied - to reflected :class:`_schema.ForeignKey` objects. The documentation has been - updated to describe in detail the behavior of the ``pg_get_constraintdef()`` - function and how the ``postgresql_ignore_search_path`` feature essentially - determines if we will honor the schema qualification reported by - this function or not. - - .. seealso:: - - :ref:`postgresql_schema_reflection` - - .. change:: - :tags: bug, sql - :tickets: 2913 - - The behavior of :meth:`_schema.Table.tometadata` has been adjusted such that - the schema target of a :class:`_schema.ForeignKey` will not be changed unless - that schema matches that of the parent table. That is, if - a table "schema_a.user" has a foreign key to "schema_b.order.id", - the "schema_b" target will be maintained whether or not the - "schema" argument is passed to :meth:`_schema.Table.tometadata`. However - if a table "schema_a.user" refers to "schema_a.order.id", the presence - of "schema_a" will be updated on both the parent and referred tables. - This is a behavioral change hence isn't likely to be backported to - 0.8; it is assumed that the previous behavior is pretty buggy - however and that it's unlikely anyone was relying upon it. - - Additionally, a new parameter has been added - :paramref:`.Table.tometadata.referred_schema_fn`. This refers to a - callable function which will be used to determine the new referred - schema for any :class:`_schema.ForeignKeyConstraint` encountered in the - tometadata operation. This callable can be used to revert to the - previous behavior or to customize how referred schemas are treated - on a per-constraint basis. - - .. change:: - :tags: bug, orm - :tickets: 2932 - - Fixed bug in new :class:`.TextAsFrom` construct where :class:`_schema.Column`- - oriented row lookups were not matching up to the ad-hoc :class:`.ColumnClause` - objects that :class:`.TextAsFrom` generates, thereby making it not - usable as a target in :meth:`_query.Query.from_statement`. Also fixed - :meth:`_query.Query.from_statement` mechanics to not mistake a :class:`.TextAsFrom` - for a :class:`_expression.Select` construct. This bug is also an 0.9 regression - as the :meth:`_expression.TextClause.columns` method is called to accommodate the - :paramref:`_expression.text.typemap` argument. - - .. change:: - :tags: feature, sql - :tickets: 2923 - - Added a new feature which allows automated naming conventions to be - applied to :class:`.Constraint` and :class:`.Index` objects. Based - on a recipe in the wiki, the new feature uses schema-events to set up - names as various schema objects are associated with each other. The - events then expose a configuration system through a new argument - :paramref:`_schema.MetaData.naming_convention`. This system allows production - of both simple and custom naming schemes for constraints and indexes - on a per-:class:`_schema.MetaData` basis. - - .. seealso:: - - :ref:`constraint_naming_conventions` - - .. change:: - :tags: bug, orm - :tickets: 2921 - - Added a new directive used within the scope of an attribute "set" operation - to disable autoflush, in the case that the attribute needs to lazy-load - the "old" value, as in when replacing one-to-one values or some - kinds of many-to-one. A flush at this point otherwise occurs - at the point that the attribute is None and can cause NULL violations. - - .. change:: - :tags: feature, orm - - Added a new parameter :paramref:`.Operators.op.is_comparison`. This - flag allows a custom op from :meth:`.Operators.op` to be considered - as a "comparison" operator, thus usable for custom - :paramref:`_orm.relationship.primaryjoin` conditions. - - .. seealso:: - - :ref:`relationship_custom_operator` - - .. change:: - :tags: bug, sqlite - - Fixed bug whereby SQLite compiler failed to propagate compiler arguments - such as "literal binds" into a CAST expression. - - .. change:: - :tags: bug, sql - - Fixed bug whereby binary type would fail in some cases - if used with a "test" dialect, such as a DefaultDialect or other - dialect with no DBAPI. - - .. change:: - :tags: bug, sql, py3k - - Fixed bug where "literal binds" wouldn't work with a bound parameter - that's a binary type. A similar, but different, issue is fixed - in 0.8. - - .. change:: - :tags: bug, sql - :tickets: 2927 - - Fixed regression whereby the "annotation" system used by the ORM was leaking - into the names used by standard functions in :mod:`sqlalchemy.sql.functions`, - such as ``func.coalesce()`` and ``func.max()``. Using these functions - in ORM attributes and thus producing annotated versions of them could - corrupt the actual function name rendered in the SQL. - - .. change:: - :tags: bug, sql - :tickets: 2924, 2848 - - Fixed 0.9 regression where the new sortable support for :class:`.RowProxy` - would lead to ``TypeError`` when compared to non-tuple types as it attempted - to apply tuple() to the "other" object unconditionally. The - full range of Python comparison operators have now been implemented on - :class:`.RowProxy`, using an approach that guarantees a comparison - system that is equivalent to that of a tuple, and the "other" object - is only coerced if it's an instance of RowProxy. - - .. change:: - :tags: bug, orm - :tickets: 2918 - - Fixed an 0.9 regression where the automatic aliasing applied by - :class:`_query.Query` and in other situations where selects or joins - were aliased (such as joined table inheritance) could fail if a - user-defined :class:`_schema.Column` subclass were used in the expression. - In this case, the subclass would fail to propagate ORM-specific - "annotations" along needed by the adaptation. The "expression - annotations" system has been corrected to account for this case. - - .. change:: - :tags: feature, orm - - Support is improved for supplying a :func:`_expression.join` construct as the - target of :paramref:`_orm.relationship.secondary` for the purposes - of creating very complex :func:`_orm.relationship` join conditions. - The change includes adjustments to query joining, joined eager loading - to not render a SELECT subquery, changes to lazy loading such that - the "secondary" target is properly included in the SELECT, and - changes to declarative to better support specification of a - join() object with classes as targets. - - The new use case is somewhat experimental, but a new documentation section - has been added. - - .. seealso:: - - :ref:`composite_secondary_join` - - .. change:: - :tags: bug, mysql, sql - :tickets: 2917 - - Added new test coverage for so-called "down adaptions" of SQL types, - where a more specific type is adapted to a more generic one - this - use case is needed by some third party tools such as ``sqlacodegen``. - The specific cases that needed repair within this test suite were that - of :class:`.mysql.ENUM` being downcast into a :class:`_types.Enum`, - and that of SQLite date types being cast into generic date types. - The ``adapt()`` method needed to become more specific here to counteract - the removal of a "catch all" ``**kwargs`` collection on the base - :class:`.TypeEngine` class that was removed in 0.9. - - .. change:: - :tags: feature, sql - :tickets: 2910 - - Options can now be specified on a :class:`.PrimaryKeyConstraint` object - independently of the specification of columns in the table with - the ``primary_key=True`` flag; use a :class:`.PrimaryKeyConstraint` - object with no columns in it to achieve this result. - - Previously, an explicit :class:`.PrimaryKeyConstraint` would have the - effect of those columns marked as ``primary_key=True`` being ignored; - since this is no longer the case, the :class:`.PrimaryKeyConstraint` - will now assert that either one style or the other is used to specify - the columns, or if both are present, that the column lists match - exactly. If an inconsistent set of columns in the - :class:`.PrimaryKeyConstraint` - and within the :class:`_schema.Table` marked as ``primary_key=True`` are - present, a warning is emitted, and the list of columns is taken - only from the :class:`.PrimaryKeyConstraint` alone as was the case - in previous releases. - - - - .. seealso:: - - :class:`.PrimaryKeyConstraint` - - .. change:: - :tags: feature, sql - :tickets: 2866 - - The system by which schema constructs and certain SQL constructs - accept dialect-specific keyword arguments has been enhanced. This - system includes commonly the :class:`_schema.Table` and :class:`.Index` constructs, - which accept a wide variety of dialect-specific arguments such as - ``mysql_engine`` and ``postgresql_where``, as well as the constructs - :class:`.PrimaryKeyConstraint`, :class:`.UniqueConstraint`, - :class:`_expression.Update`, :class:`_expression.Insert` and :class:`_expression.Delete`, and also - newly added kwarg capability to :class:`_schema.ForeignKeyConstraint` - and :class:`_schema.ForeignKey`. The change is that participating dialects - can now specify acceptable argument lists for these constructs, allowing - an argument error to be raised if an invalid keyword is specified for - a particular dialect. If the dialect portion of the keyword is unrecognized, - a warning is emitted only; while the system will actually make use - of setuptools entrypoints in order to locate non-local dialects, - the use case where certain dialect-specific arguments are used - in an environment where that third-party dialect is uninstalled remains - supported. Dialects also have to explicitly opt-in to this system, - so that external dialects which aren't making use of this system - will remain unaffected. - - .. change:: - :tags: bug, sql - - A :class:`.UniqueConstraint` created inline with a :class:`_schema.Table` - that has no columns within it will be skipped. Pullreq courtesy - Derek Harland. - - .. change:: - :tags: feature, mssql - - Added an option ``mssql_clustered`` to the :class:`.UniqueConstraint` - and :class:`.PrimaryKeyConstraint` constructs; on SQL Server, this adds - the ``CLUSTERED`` keyword to the constraint construct within DDL. - Pullreq courtesy Derek Harland. - - .. change:: - :tags: bug, sql, orm - :tickets: 2912 - - Fixed the multiple-table "UPDATE..FROM" construct, only usable on - MySQL, to correctly render the SET clause among multiple columns - with the same name across tables. This also changes the name used for - the bound parameter in the SET clause to "_" for - the non-primary table only; as this parameter is typically specified - using the :class:`_schema.Column` object directly this should not have an - impact on applications. The fix takes effect for both - :meth:`_schema.Table.update` as well as :meth:`_query.Query.update` in the ORM. - - .. change:: - :tags: bug, oracle - :tickets: 2911 - - It's been observed that the usage of a cx_Oracle "outputtypehandler" - in Python 2.xx in order to coerce string values to Unicode is inordinately - expensive; even though cx_Oracle is written in C, when you pass the - Python ``unicode`` primitive to cursor.var() and associate with an output - handler, the library counts every conversion as a Python function call - with all the requisite overhead being recorded; this *despite* the fact - when running in Python 3, all strings are also unconditionally coerced - to unicode but it does *not* incur this overhead, - meaning that cx_Oracle is failing to use performant techniques in Py2K. - As SQLAlchemy cannot easily select for this style of type handler on a - per-column basis, the handler was assembled unconditionally thereby - adding the overhead to all string access. - - So this logic has been replaced with SQLAlchemy's own unicode - conversion system, which now - only takes effect in Py2K for columns that are requested as unicode. - When C extensions are used, SQLAlchemy's system appears to be 2-3x faster than - cx_Oracle's. Additionally, SQLAlchemy's unicode conversion has been - enhanced such that when the "conditional" converter is required - (now needed for the Oracle backend), the check for "already unicode" is now - performed in C and no longer introduces significant overhead. - - This change has two impacts on the cx_Oracle backend. One is that - string values in Py2K which aren't specifically requested with the - Unicode type or convert_unicode=True will now come back as ``str``, - not ``unicode`` - this behavior is similar to a backend such as - MySQL. Additionally, when unicode values are requested with the cx_Oracle - backend, if the C extensions are *not* used, there is now an additional - overhead of an isinstance() check per column. This tradeoff has been - made as it can be worked around and no longer places a performance burden - on the likely majority of Oracle result columns that are non-unicode - strings. - - .. change:: - :tags: bug, orm - :tickets: 2908 - - Fixed a bug involving the new flattened JOIN structures which - are used with :func:`_orm.joinedload()` (thereby causing a regression - in joined eager loading) as well as :func:`.aliased` - in conjunction with the ``flat=True`` flag and joined-table inheritance; - basically multiple joins across a "parent JOIN sub" entity using different - paths to get to a target class wouldn't form the correct ON conditions. - An adjustment / simplification made in the mechanics of figuring - out the "left side" of the join in the case of an aliased, joined-inh - class repairs the issue. - - .. change:: - :tags: bug, mysql - - The MySQL CAST compilation now takes into account aspects of a string - type such as "charset" and "collation". While MySQL wants all character- - based CAST calls to use the CHAR type, we now create a real CHAR - object at CAST time and copy over all the parameters it has, so that - an expression like ``cast(x, mysql.TEXT(charset='utf8'))`` will - render ``CAST(t.col AS CHAR CHARACTER SET utf8)``. - - .. change:: - :tags: bug, mysql - :tickets: 2906 - - Added new "unicode returns" detection to the MySQL dialect and - to the default dialect system overall, such that any dialect - can add extra "tests" to the on-first-connect "does this DBAPI - return unicode directly?" detection. In this case, we are - adding a check specifically against the "utf8" encoding with - an explicit "utf8_bin" collation type (after checking that - this collation is available) to test for some buggy unicode - behavior observed with MySQLdb version 1.2.3. While MySQLdb - has resolved this issue as of 1.2.4, the check here should - guard against regressions. The change also allows the "unicode" - checks to log in the engine logs, which was not previously - the case. - - .. change:: - :tags: bug, mysql, pool, engine - :tickets: 2907 - - :class:`_engine.Connection` now associates a new - :class:`.RootTransaction` or :class:`.TwoPhaseTransaction` - with its immediate :class:`._ConnectionFairy` as a "reset handler" - for the span of that transaction, which takes over the task - of calling commit() or rollback() for the "reset on return" behavior - of :class:`_pool.Pool` if the transaction was not otherwise completed. - This resolves the issue that a picky transaction - like that of MySQL two-phase will be - properly closed out when the connection is closed without an - explicit rollback or commit (e.g. no longer raises "XAER_RMFAIL" - in this case - note this only shows up in logging as the exception - is not propagated within pool reset). - This issue would arise e.g. when using an orm - :class:`.Session` with ``twophase`` set, and then - :meth:`.Session.close` is called without an explicit rollback or - commit. The change also has the effect that you will now see - an explicit "ROLLBACK" in the logs when using a :class:`.Session` - object in non-autocommit mode regardless of how that session was - discarded. Thanks to Jeff Dairiki and Laurence Rowe for isolating - the issue here. - - .. change:: - :tags: feature, pool, engine - - Added a new pool event :meth:`_events.PoolEvents.invalidate`. Called when - a DBAPI connection is to be marked as "invalidated" and discarded - from the pool. - - .. change:: - :tags: bug, pool - - The argument names for the :meth:`_events.PoolEvents.reset` event have been - renamed to ``dbapi_connection`` and ``connection_record`` in order - to maintain consistency with all the other pool events. It is expected - that any existing listeners for this relatively new and - seldom-used event are using positional style to receive arguments in - any case. - - .. change:: - :tags: bug, py3k, cextensions - - Fixed an issue where the C extensions in Py3K are using the wrong API - to specify the top-level module function, which breaks - in Python 3.4b2. Py3.4b2 changes PyMODINIT_FUNC to return - "void" instead of ``PyObject *``, so we now make sure to use - "PyMODINIT_FUNC" instead of ``PyObject *`` directly. Pull request - courtesy cgohlke. - - .. change:: - :tags: bug, schema - - Restored :class:`sqlalchemy.schema.SchemaVisitor` to the ``.schema`` - module. Pullreq courtesy Sean Dague. - -.. changelog:: - :version: 0.9.1 - :released: January 5, 2014 - - .. change:: - :tags: bug, orm, events - :tickets: 2905 - - Fixed regression where using a ``functools.partial()`` with the event - system would cause a recursion overflow due to usage of inspect.getargspec() - on it in order to detect a legacy calling signature for certain events, - and apparently there's no way to do this with a partial object. Instead - we skip the legacy check and assume the modern style; the check itself - now only occurs for the SessionEvents.after_bulk_update and - SessionEvents.after_bulk_delete events. Those two events will require - the new signature style if assigned to a "partial" event listener. - - .. change:: - :tags: feature, orm, extensions - - A new, **experimental** extension :mod:`sqlalchemy.ext.automap` is added. - This extension expands upon the functionality of Declarative as well as - the :class:`.DeferredReflection` class to produce a base class which - automatically generates mapped classes *and relationships* based on - table metadata. - - .. seealso:: - - :ref:`feature_automap` - - :ref:`automap_toplevel` - - .. change:: - :tags: feature, sql - - Conjunctions like :func:`.and_` and :func:`.or_` can now accept - Python generators as a single argument, e.g.:: - - and_(x == y for x, y in tuples) - - The logic here looks for a single argument ``*args`` where the first - element is an instance of ``types.GeneratorType``. - - .. change:: - :tags: feature, schema - - The :paramref:`_schema.Table.extend_existing` and :paramref:`_schema.Table.autoload_replace` - parameters are now available on the :meth:`_schema.MetaData.reflect` - method. - - .. change:: - :tags: bug, orm, declarative - - Fixed an extremely unlikely memory issue where when using - :class:`.DeferredReflection` - to define classes pending for reflection, if some subset of those - classes were discarded before the :meth:`.DeferredReflection.prepare` - method were called to reflect and map the class, a strong reference - to the class would remain held within the declarative internals. - This internal collection of "classes to map" now uses weak - references against the classes themselves. - - .. change:: - :tags: bug, orm - - Fixed bug where using new :attr:`.Session.info` attribute would fail - if the ``.info`` argument were only passed to the :class:`.sessionmaker` - creation call but not to the object itself. Courtesy Robin Schoonover. - - .. change:: - :tags: bug, orm - :tickets: 2901 - - Fixed regression where we don't check the given name against the - correct string class when setting up a backref based on a name, - therefore causing the error "too many values to unpack". This was - related to the Py3k conversion. - - .. change:: - :tags: bug, orm, declarative - :tickets: 2900 - - A quasi-regression where apparently in 0.8 you can set a class-level - attribute on declarative to simply refer directly to an :class:`.InstrumentedAttribute` - on a superclass or on the class itself, and it - acts more or less like a synonym; in 0.9, this fails to set up enough - bookkeeping to keep up with the more liberalized backref logic - from :ticket:`2789`. Even though this use case was never directly - considered, it is now detected by declarative at the "setattr()" level - as well as when setting up a subclass, and the mirrored/renamed attribute - is now set up as a :func:`.synonym` instead. - - .. change:: - :tags: bug, orm - :tickets: 2903 - - Fixed regression where we apparently still create an implicit - alias when saying query(B).join(B.cs), where "C" is a joined inh - class; however, this implicit alias was created only considering - the immediate left side, and not a longer chain of joins along different - joined-inh subclasses of the same base. As long as we're still - implicitly aliasing in this case, the behavior is dialed back a bit - so that it will alias the right side in a wider variety of cases. - -.. changelog:: - :version: 0.9.0 - :released: December 30, 2013 - - .. change:: - :tags: bug, orm, declarative - :tickets: 2828 - - Declarative does an extra check to detect if the same - :class:`_schema.Column` is mapped multiple times under different properties - (which typically should be a :func:`.synonym` instead) or if two - or more :class:`_schema.Column` objects are given the same name, raising - a warning if this condition is detected. - - .. change:: - :tags: bug, firebird - :tickets: 2898 - - Changed the queries used by Firebird to list table and view names - to query from the ``rdb$relations`` view instead of the - ``rdb$relation_fields`` and ``rdb$view_relations`` views. - Variants of both the old and new queries are mentioned on many - FAQ and blogs, however the new queries are taken straight from - the "Firebird FAQ" which appears to be the most official source - of info. - - .. change:: - :tags: bug, mysql - :tickets: 2893 - - Improvements to the system by which SQL types generate within - ``__repr__()``, particularly with regards to the MySQL integer/numeric/ - character types which feature a wide variety of keyword arguments. - The ``__repr__()`` is important for use with Alembic autogenerate - for when Python code is rendered in a migration script. - - .. change:: - :tags: feature, postgresql - :tickets: 2581 - - Support for PostgreSQL JSON has been added, using the new - :class:`_types.JSON` type. Huge thanks to Nathan Rice for - implementing and testing this. - - .. change:: - :tags: bug, sql - - The :func:`.cast` function, when given a plain literal value, - will now apply the given type to the given literal value on the - bind parameter side according to the type given to the cast, - in the same manner as that of the :func:`.type_coerce` function. - However unlike :func:`.type_coerce`, this only takes effect if a - non-clauseelement value is passed to :func:`.cast`; an existing typed - construct will retain its type. - - .. change:: - :tags: bug, postgresql - - Now using psycopg2 UNICODEARRAY extension for handling unicode arrays - with psycopg2 + normal "native unicode" mode, in the same way the - UNICODE extension is used. - - .. change:: - :tags: bug, sql - :tickets: 2883 - - The :class:`_schema.ForeignKey` class more aggressively checks the given - column argument. If not a string, it checks that the object is - at least a :class:`.ColumnClause`, or an object that resolves to one, - and that the ``.table`` attribute, if present, refers to a - :class:`_expression.TableClause` or subclass, and not something like an - :class:`_expression.Alias`. Otherwise, a :class:`.ArgumentError` is raised. - - - .. change:: - :tags: feature, orm - - The :class:`.exc.StatementError` or DBAPI-related subclass - now can accommodate additional information about the "reason" for - the exception; the :class:`.Session` now adds some detail to it - when the exception occurs within an autoflush. This approach - is taken as opposed to combining :class:`.FlushError` with - a Python 3 style "chained exception" approach so as to maintain - compatibility both with Py2K code as well as code that already - catches ``IntegrityError`` or similar. - - .. change:: - :tags: feature, postgresql - - Added support for PostgreSQL TSVECTOR via the - :class:`_postgresql.TSVECTOR` type. Pull request courtesy - Noufal Ibrahim. - - .. change:: - :tags: feature, engine - :tickets: 2875 - - The :func:`.engine_from_config` function has been improved so that - we will be able to parse dialect-specific arguments from string - configuration dictionaries. Dialect classes can now provide their - own list of parameter types and string-conversion routines. - The feature is not yet used by the built-in dialects, however. - - .. change:: - :tags: bug, sql - :tickets: 2879 - - The precedence rules for the :meth:`.ColumnOperators.collate` operator - have been modified, such that the COLLATE operator is now of lower - precedence than the comparison operators. This has the effect that - a COLLATE applied to a comparison will not render parenthesis - around the comparison, which is not parsed by backends such as - MSSQL. The change is backwards incompatible for those setups that - were working around the issue by applying :meth:`.Operators.collate` - to an individual element of the comparison expression, - rather than the comparison expression as a whole. - - .. seealso:: - - :ref:`migration_2879` - - .. change:: - :tags: bug, orm, declarative - :tickets: 2865 - - The :class:`.DeferredReflection` class has been enhanced to provide - automatic reflection support for the "secondary" table referred - to by a :func:`_orm.relationship`. "secondary", when specified - either as a string table name, or as a :class:`_schema.Table` object with - only a name and :class:`_schema.MetaData` object will also be included - in the reflection process when :meth:`.DeferredReflection.prepare` - is called. - - .. change:: - :tags: feature, orm, backrefs - :tickets: 1535 - - Added new argument ``include_backrefs=True`` to the - :func:`.validates` function; when set to False, a validation event - will not be triggered if the event was initiated as a backref to - an attribute operation from the other side. - - .. seealso:: - - :ref:`feature_1535` - - .. change:: - :tags: bug, orm, collections, py3k - - Added support for the Python 3 method ``list.clear()`` within - the ORM collection instrumentation system; pull request - courtesy Eduardo Schettino. - - .. change:: - :tags: bug, postgresql - :tickets: 2878 - - Fixed bug where values within an ENUM weren't escaped for single - quote signs. Note that this is backwards-incompatible for existing - workarounds that manually escape the single quotes. - - .. seealso:: - - :ref:`migration_2878` - - .. change:: - :tags: bug, orm, declarative - - Fixed bug where in Py2K a unicode literal would not be accepted - as the string name of a class or other argument within - declarative using :func:`_orm.relationship`. - - .. change:: - :tags: feature, sql - :tickets: 2877, 2882 - - New improvements to the :func:`_expression.text` construct, including - more flexible ways to set up bound parameters and return types; - in particular, a :func:`_expression.text` can now be turned into a full - FROM-object, embeddable in other statements as an alias or CTE - using the new method :meth:`_expression.TextClause.columns`. The :func:`_expression.text` - construct can also render "inline" bound parameters when the construct - is compiled in a "literal bound" context. - - .. seealso:: - - :ref:`feature_2877` - - .. change:: - :tags: feature, sql - - A new API for specifying the ``FOR UPDATE`` clause of a ``SELECT`` - is added with the new :meth:`_expression.GenerativeSelect.with_for_update` method. - This method supports a more straightforward system of setting - dialect-specific options compared to the ``for_update`` keyword - argument of :func:`_expression.select`, and also includes support for the - SQL standard ``FOR UPDATE OF`` clause. The ORM also includes - a new corresponding method :meth:`_query.Query.with_for_update`. - Pull request courtesy Mario Lassnig. - - .. seealso:: - - :ref:`feature_github_42` - - .. change:: - :tags: feature, orm - - A new API for specifying the ``FOR UPDATE`` clause of a ``SELECT`` - is added with the new :meth:`_query.Query.with_for_update` method, - to complement the new :meth:`_expression.GenerativeSelect.with_for_update` method. - Pull request courtesy Mario Lassnig. - - .. seealso:: - - :ref:`feature_github_42` - - .. change:: - :tags: bug, engine - :tickets: 2873 - - The :func:`_sa.create_engine` routine and the related :func:`.make_url` - function no longer considers the ``+`` sign to be a space within the - password field. The parsing in this area has been adjusted to match - more closely to how RFC 1738 handles these tokens, in that both - ``username`` and ``password`` expect only ``:``, ``@``, and ``/`` to be - encoded. - - .. seealso:: - - :ref:`migration_2873` - - - .. change:: - :tags: bug, orm - :tickets: 2872 - - Some refinements to the :class:`.AliasedClass` construct with regards - to descriptors, like hybrids, synonyms, composites, user-defined - descriptors, etc. The attribute - adaptation which goes on has been made more robust, such that if a descriptor - returns another instrumented attribute, rather than a compound SQL - expression element, the operation will still proceed. - Additionally, the "adapted" operator will retain its class; previously, - a change in class from ``InstrumentedAttribute`` to ``QueryableAttribute`` - (a superclass) would interact with Python's operator system such that - an expression like ``aliased(MyClass.x) > MyClass.x`` would reverse itself - to read ``myclass.x < myclass_1.x``. The adapted attribute will also - refer to the new :class:`.AliasedClass` as its parent which was not - always the case before. - - .. change:: - :tags: feature, sql - :tickets: 2867 - - The precision used when coercing a returned floating point value to - Python ``Decimal`` via string is now configurable. The - flag ``decimal_return_scale`` is now supported by all :class:`.Numeric` - and :class:`.Float` types, which will ensure this many digits are taken - from the native floating point value when it is converted to string. - If not present, the type will make use of the value of ``.scale``, if - the type supports this setting and it is non-None. Otherwise the original - default length of 10 is used. - - .. seealso:: - - :ref:`feature_2867` - - .. change:: - :tags: bug, schema - :tickets: 2868 - - Fixed a regression caused by :ticket:`2812` where the repr() for - table and column names would fail if the name contained non-ascii - characters. - - .. change:: - :tags: bug, engine - :tickets: 2848 - - The :class:`.RowProxy` object is now sortable in Python as a regular - tuple is; this is accomplished via ensuring tuple() conversion on - both sides within the ``__eq__()`` method as well as - the addition of a ``__lt__()`` method. - - .. seealso:: - - :ref:`migration_2848` - - .. change:: - :tags: bug, orm - :tickets: 2833 - - The ``viewonly`` flag on :func:`_orm.relationship` will now prevent - attribute history from being written on behalf of the target attribute. - This has the effect of the object not being written to the - Session.dirty list if it is mutated. Previously, the object would - be present in Session.dirty, but no change would take place on behalf - of the modified attribute during flush. The attribute still emits - events such as backref events and user-defined events and will still - receive mutations from backrefs. - - .. seealso:: - - :ref:`migration_2833` - - .. change:: - :tags: bug, orm - - Added support for new :attr:`.Session.info` attribute to - :class:`.scoped_session`. - - .. change:: - :tags: removed - - The "informix" and "informixdb" dialects have been removed; the code - is now available as a separate repository on Bitbucket. The IBM-DB - project has provided production-level Informix support since the - informixdb dialect was first added. - - .. change:: - :tags: bug, orm - - Fixed bug where usage of new :class:`.Bundle` object would cause - the :attr:`_query.Query.column_descriptions` attribute to fail. - - .. change:: - :tags: bug, examples - - Fixed bug which prevented history_meta recipe from working with - joined inheritance schemes more than one level deep. - - .. change:: - :tags: bug, orm, sql, sqlite - :tickets: 2858 - - Fixed a regression introduced by the join rewriting feature of - :ticket:`2369` and :ticket:`2587` where a nested join with one side - already an aliased select would fail to translate the ON clause on the - outside correctly; in the ORM this could be seen when using a - SELECT statement as a "secondary" table. - -.. changelog:: - :version: 0.9.0b1 - :released: October 26, 2013 - - .. change:: - :tags: feature, orm - :tickets: 2810 - - The association proxy now returns ``None`` when fetching a scalar - attribute off of a scalar relationship, where the scalar relationship - itself points to ``None``, instead of raising an ``AttributeError``. - - .. seealso:: - - :ref:`migration_2810` - - .. change:: - :tags: feature, sql, postgresql, mysql - :tickets: 2183 - - The PostgreSQL and MySQL dialects now support reflection/inspection - of foreign key options, including ON UPDATE, ON DELETE. PostgreSQL - also reflects MATCH, DEFERRABLE, and INITIALLY. Courtesy ijl. - - .. change:: - :tags: bug, mysql - :tickets: 2839 - - Fix and test parsing of MySQL foreign key options within reflection; - this complements the work in :ticket:`2183` where we begin to support - reflection of foreign key options such as ON UPDATE/ON DELETE - cascade. - - .. change:: - :tags: bug, orm - :tickets: 2787 - - :func:`.attributes.get_history()` when used with a scalar column-mapped - attribute will now honor the "passive" flag - passed to it; as this defaults to ``PASSIVE_OFF``, the function will - by default query the database if the value is not present. - This is a behavioral change vs. 0.8. - - .. seealso:: - - :ref:`change_2787` - - .. change:: - :tags: feature, orm - :tickets: 2787 - - Added new method :meth:`.AttributeState.load_history`, works like - :attr:`.AttributeState.history` but also fires loader callables. - - .. seealso:: - - :ref:`change_2787` - - - .. change:: - :tags: feature, sql - :tickets: 2850 - - A :func:`.bindparam` construct with a "null" type (e.g. no type - specified) is now copied when used in a typed expression, and the - new copy is assigned the actual type of the compared column. Previously, - this logic would occur on the given :func:`.bindparam` in place. - Additionally, a similar process now occurs for :func:`.bindparam` constructs - passed to :meth:`.ValuesBase.values` for an :class:`_expression.Insert` or - :class:`_expression.Update` construct, within the compilation phase of the - construct. - - These are both subtle behavioral changes which may impact some - usages. - - .. seealso:: - - :ref:`migration_2850` - - .. change:: - :tags: feature, sql - :tickets: 2804, 2823, 2734 - - An overhaul of expression handling for special symbols particularly - with conjunctions, e.g. - ``None`` :func:`_expression.null` :func:`_expression.true` - :func:`_expression.false`, including consistency in rendering NULL - in conjunctions, "short-circuiting" of :func:`.and_` and :func:`.or_` - expressions which contain boolean constants, and rendering of - boolean constants and expressions as compared to "1" or "0" for backends - that don't feature ``true``/``false`` constants. - - .. seealso:: - - :ref:`migration_2804` - - .. change:: - :tags: feature, sql - :tickets: 2838 - - The typing system now handles the task of rendering "literal bind" values, - e.g. values that are normally bound parameters but due to context must - be rendered as strings, typically within DDL constructs such as - CHECK constraints and indexes (note that "literal bind" values - become used by DDL as of :ticket:`2742`). A new method - :meth:`.TypeEngine.literal_processor` serves as the base, and - :meth:`.TypeDecorator.process_literal_param` is added to allow wrapping - of a native literal rendering method. - - .. seealso:: - - :ref:`change_2838` - - .. change:: - :tags: feature, sql - :tickets: 2716 - - The :meth:`_schema.Table.tometadata` method now produces copies of - all :attr:`.SchemaItem.info` dictionaries from all :class:`.SchemaItem` - objects within the structure including columns, constraints, - foreign keys, etc. As these dictionaries - are copies, they are independent of the original dictionary. - Previously, only the ``.info`` dictionary of :class:`_schema.Column` was transferred - within this operation, and it was only linked in place, not copied. - - .. change:: - :tags: feature, postgresql - :tickets: 2840 - - Added support for rendering ``SMALLSERIAL`` when a :class:`.SmallInteger` - type is used on a primary key autoincrement column, based on server - version detection of PostgreSQL version 9.2 or greater. - - .. change:: - :tags: feature, mysql - :tickets: 2817 - - The MySQL :class:`.mysql.SET` type now features the same auto-quoting - behavior as that of :class:`.mysql.ENUM`. Quotes are not required when - setting up the value, but quotes that are present will be auto-detected - along with a warning. This also helps with Alembic where - the SET type doesn't render with quotes. - - .. change:: - :tags: feature, sql - - The ``default`` argument of :class:`_schema.Column` now accepts a class - or object method as an argument, in addition to a standalone function; - will properly detect if the "context" argument is accepted or not. - - .. change:: - :tags: bug, sql - :tickets: 2835 - - The "name" attribute is set on :class:`.Index` before the "attach" - events are called, so that attachment events can be used to dynamically - generate a name for the index based on the parent table and/or - columns. - - .. change:: - :tags: bug, engine - :tickets: 2748 - - The method signature of :meth:`.Dialect.reflecttable`, which in - all known cases is provided by :class:`.DefaultDialect`, has been - tightened to expect ``include_columns`` and ``exclude_columns`` - arguments without any kw option, reducing ambiguity - previously - ``exclude_columns`` was missing. - - .. change:: - :tags: bug, sql - :tickets: 2831 - - The erroneous kw arg "schema" has been removed from the :class:`_schema.ForeignKey` - object. this was an accidental commit that did nothing; a warning is raised - in 0.8.3 when this kw arg is used. - - .. change:: - :tags: feature, orm - :tickets: 1418 - - Added a new load option :func:`_orm.load_only`. This allows a series - of column names to be specified as loading "only" those attributes, - deferring the rest. - - .. change:: - :tags: feature, orm - :tickets: 1418 - - The system of loader options has been entirely rearchitected to build - upon a much more comprehensive base, the :class:`_orm.Load` object. This - base allows any common loader option like :func:`_orm.joinedload`, - :func:`.defer`, etc. to be used in a "chained" style for the purpose - of specifying options down a path, such as ``joinedload("foo").subqueryload("bar")``. - The new system supersedes the usage of dot-separated path names, - multiple attributes within options, and the usage of ``_all()`` options. - - .. seealso:: - - :ref:`feature_1418` - - .. change:: - :tags: feature, orm - :tickets: 2824 - - The :func:`.composite` construct now maintains the return object - when used in a column-oriented :class:`_query.Query`, rather than expanding - out into individual columns. This makes use of the new :class:`.Bundle` - feature internally. This behavior is backwards incompatible; to - select from a composite column which will expand out, use - ``MyClass.some_composite.clauses``. - - .. seealso:: - - :ref:`migration_2824` - - .. change:: - :tags: feature, orm - :tickets: 2824 - - A new construct :class:`.Bundle` is added, which allows for specification - of groups of column expressions to a :class:`_query.Query` construct. - The group of columns are returned as a single tuple by default. The - behavior of :class:`.Bundle` can be overridden however to provide - any sort of result processing to the returned row. The behavior - of :class:`.Bundle` is also embedded into composite attributes now - when they are used in a column-oriented :class:`_query.Query`. - - .. seealso:: - - :ref:`change_2824` - - :ref:`migration_2824` - - .. change:: - :tags: bug, sql - :tickets: 2812 - - A rework to the way that "quoted" identifiers are handled, in that - instead of relying upon various ``quote=True`` flags being passed around, - these flags are converted into rich string objects with quoting information - included at the point at which they are passed to common schema constructs - like :class:`_schema.Table`, :class:`_schema.Column`, etc. This solves the issue - of various methods that don't correctly honor the "quote" flag such - as :meth:`_engine.Engine.has_table` and related methods. The :class:`.quoted_name` - object is a string subclass that can also be used explicitly if needed; - the object will hold onto the quoting preferences passed and will - also bypass the "name normalization" performed by dialects that - standardize on uppercase symbols, such as Oracle, Firebird and DB2. - The upshot is that the "uppercase" backends can now work with force-quoted - names, such as lowercase-quoted names and new reserved words. - - .. seealso:: - - :ref:`change_2812` - - .. change:: - :tags: feature, orm - :tickets: 2793 - - The ``version_id_generator`` parameter of ``Mapper`` can now be specified - to rely upon server generated version identifiers, using triggers - or other database-provided versioning features, or via an optional programmatic - value, by setting ``version_id_generator=False``. - When using a server-generated version identifier, the ORM will use RETURNING when - available to immediately - load the new version value, else it will emit a second SELECT. - - .. change:: - :tags: feature, orm - :tickets: 2793 - - The ``eager_defaults`` flag of :class:`_orm.Mapper` will now allow the - newly generated default values to be fetched using an inline - RETURNING clause, rather than a second SELECT statement, for backends - that support RETURNING. - - .. change:: - :tags: feature, core - :tickets: 2793 - - Added a new variant to :meth:`.UpdateBase.returning` called - :meth:`.ValuesBase.return_defaults`; this allows arbitrary columns - to be added to the RETURNING clause of the statement without interfering - with the compilers usual "implicit returning" feature, which is used to - efficiently fetch newly generated primary key values. For supporting - backends, a dictionary of all fetched values is present at - :attr:`_engine.ResultProxy.returned_defaults`. - - .. change:: - :tags: bug, mysql - - Improved support for the cymysql driver, supporting version 0.6.5, - courtesy Hajime Nakagami. - - .. change:: - :tags: general - - A large refactoring of packages has reorganized - the import structure of many Core modules as well as some aspects - of the ORM modules. In particular ``sqlalchemy.sql`` has been broken - out into several more modules than before so that the very large size - of ``sqlalchemy.sql.expression`` is now pared down. The effort - has focused on a large reduction in import cycles. Additionally, - the system of API functions in ``sqlalchemy.sql.expression`` and - ``sqlalchemy.orm`` has been reorganized to eliminate redundancy - in documentation between the functions vs. the objects they produce. - - .. change:: - :tags: orm, feature, orm - - Added a new attribute :attr:`.Session.info` to :class:`.Session`; - this is a dictionary where applications can store arbitrary - data local to a :class:`.Session`. - The contents of :attr:`.Session.info` can be also be initialized - using the ``info`` argument of :class:`.Session` or - :class:`.sessionmaker`. - - - .. change:: - :tags: feature, general, py3k - :tickets: 2161 - - The C extensions are ported to Python 3 and will build under - any supported CPython 2 or 3 environment. - - .. change:: - :tags: feature, orm - :tickets: 2268 - - Removal of event listeners is now implemented. The feature is - provided via the :func:`.event.remove` function. - - .. seealso:: - - :ref:`feature_2268` - - .. change:: - :tags: feature, orm - :tickets: 2789 - - The mechanism by which attribute events pass along an - :class:`.AttributeImpl` as an "initiator" token has been changed; - the object is now an event-specific object called :class:`.attributes.Event`. - Additionally, the attribute system no longer halts events based - on a matching "initiator" token; this logic has been moved to be - specific to ORM backref event handlers, which are the typical source - of the re-propagation of an attribute event onto subsequent append/set/remove - operations. End user code which emulates the behavior of backrefs - must now ensure that recursive event propagation schemes are halted, - if the scheme does not use the backref handlers. Using this new system, - backref handlers can now perform a - "two-hop" operation when an object is appended to a collection, - associated with a new many-to-one, de-associated with the previous - many-to-one, and then removed from a previous collection. Before this - change, the last step of removal from the previous collection would - not occur. - - .. seealso:: - - :ref:`migration_2789` - - .. change:: - :tags: feature, sql - :tickets: 722 - - Added new method to the :func:`_expression.insert` construct - :meth:`_expression.Insert.from_select`. Given a list of columns and - a selectable, renders ``INSERT INTO (table) (columns) SELECT ..``. - While this feature is highlighted as part of 0.9 it is also - backported to 0.8.3. - - .. seealso:: - - :ref:`feature_722` - - .. change:: - :tags: feature, engine - :tickets: 2770 - - New events added to :class:`_events.ConnectionEvents`: - - * :meth:`_events.ConnectionEvents.engine_connect` - * :meth:`_events.ConnectionEvents.set_connection_execution_options` - * :meth:`_events.ConnectionEvents.set_engine_execution_options` - - .. change:: - :tags: bug, sql - :tickets: 1765 - - The resolution of :class:`_schema.ForeignKey` objects to their - target :class:`_schema.Column` has been reworked to be as - immediate as possible, based on the moment that the - target :class:`_schema.Column` is associated with the same - :class:`_schema.MetaData` as this :class:`_schema.ForeignKey`, rather - than waiting for the first time a join is constructed, - or similar. This along with other improvements allows - earlier detection of some foreign key configuration - issues. Also included here is a rework of the - type-propagation system, so that - it should be reliable now to set the type as ``None`` - on any :class:`_schema.Column` that refers to another via - :class:`_schema.ForeignKey` - the type will be copied from the - target column as soon as that other column is associated, - and now works for composite foreign keys as well. - - .. seealso:: - - :ref:`migration_1765` - - .. change:: - :tags: feature, sql - :tickets: 2744, 2734 - - Provided a new attribute for :class:`.TypeDecorator` - called :attr:`.TypeDecorator.coerce_to_is_types`, - to make it easier to control how comparisons using - ``==`` or ``!=`` to ``None`` and boolean types goes - about producing an ``IS`` expression, or a plain - equality expression with a bound parameter. - - .. change:: - :tags: feature, pool - :tickets: 2752 - - Added pool logging for "rollback-on-return" and the less used - "commit-on-return". This is enabled with the rest of pool - "debug" logging. - - .. change:: - :tags: bug, orm, associationproxy - :tickets: 2751 - - Added additional criterion to the ==, != comparators, used with - scalar values, for comparisons to None to also take into account - the association record itself being non-present, in addition to the - existing test for the scalar endpoint on the association record - being NULL. Previously, comparing ``Cls.scalar == None`` would return - records for which ``Cls.associated`` were present and - ``Cls.associated.scalar`` is None, but not rows for which - ``Cls.associated`` is non-present. More significantly, the - inverse operation ``Cls.scalar != None`` *would* return ``Cls`` - rows for which ``Cls.associated`` was non-present. - - The case for ``Cls.scalar != 'somevalue'`` is also modified - to act more like a direct SQL comparison; only rows for - which ``Cls.associated`` is present and ``Associated.scalar`` - is non-NULL and not equal to ``'somevalue'`` are returned. - Previously, this would be a simple ``NOT EXISTS``. - - Also added a special use case where you - can call ``Cls.scalar.has()`` with no arguments, - when ``Cls.scalar`` is a column-based value - this returns whether or - not ``Cls.associated`` has any rows present, regardless of whether - or not ``Cls.associated.scalar`` is NULL or not. - - .. seealso:: - - :ref:`migration_2751` - - - .. change:: - :tags: feature, orm - :tickets: 2587 - - A major change regarding how the ORM constructs joins where - the right side is itself a join or left outer join. The ORM - is now configured to allow simple nesting of joins of - the form ``a JOIN (b JOIN c ON b.id=c.id) ON a.id=b.id``, - rather than forcing the right side into a ``SELECT`` subquery. - This should allow significant performance improvements on most - backends, most particularly MySQL. The one database backend - that has for many years held back this change, SQLite, is now addressed by - moving the production of the ``SELECT`` subquery from the - ORM to the SQL compiler; so that a right-nested join on SQLite will still - ultimately render with a ``SELECT``, while all other backends - are no longer impacted by this workaround. - - As part of this change, a new argument ``flat=True`` has been added - to the :func:`_orm.aliased`, :meth:`_expression.Join.alias`, and - :func:`_orm.with_polymorphic` functions, which allows an "alias" of a - JOIN to be produced which applies an anonymous alias to each component - table within the join, rather than producing a subquery. - - .. seealso:: - - :ref:`feature_joins_09` - - - .. change:: - :tags: bug, orm - :tickets: 2369 - - Fixed an obscure bug where the wrong results would be - fetched when joining/joinedloading across a many-to-many - relationship to a single-table-inheriting - subclass with a specific discriminator value, due to "secondary" - rows that would come back. The "secondary" and right-side - tables are now inner joined inside of parenthesis for all - ORM joins on many-to-many relationships so that the left->right - join can accurately filtered. This change was made possible - by finally addressing the issue with right-nested joins - outlined in :ticket:`2587`. - - .. seealso:: - - :ref:`feature_joins_09` - - .. change:: - :tags: bug, mssql, pyodbc - :tickets: 2355 - - Fixes to MSSQL with Python 3 + pyodbc, including that statements - are passed correctly. - - .. change:: - :tags: feature, sql - :tickets: 1068 - - A :func:`~sqlalchemy.sql.expression.label` construct will now render as its name alone - in an ``ORDER BY`` clause, if that label is also referred to - in the columns clause of the select, instead of rewriting the - full expression. This gives the database a better chance to - optimize the evaluation of the same expression in two different - contexts. - - .. seealso:: - - :ref:`migration_1068` - - .. change:: - :tags: feature, firebird - :tickets: 2504 - - The ``fdb`` dialect is now the default dialect when - specified without a dialect qualifier, i.e. ``firebird://``, - per the Firebird project publishing ``fdb`` as their - official Python driver. - - .. change:: - :tags: feature, general, py3k - :tickets: 2671 - - The codebase is now "in-place" for Python - 2 and 3, the need to run 2to3 has been removed. - Compatibility is now against Python 2.6 on forward. - - .. change:: - :tags: feature, oracle, py3k - - The Oracle unit tests with cx_oracle now pass - fully under Python 3. - - .. change:: - :tags: bug, orm - :tickets: 2736 - - The "auto-aliasing" behavior of the :meth:`_query.Query.select_from` - method has been turned off. The specific behavior is now - available via a new method :meth:`_query.Query.select_entity_from`. - The auto-aliasing behavior here was never well documented and - is generally not what's desired, as :meth:`_query.Query.select_from` - has become more oriented towards controlling how a JOIN is - rendered. :meth:`_query.Query.select_entity_from` will also be made - available in 0.8 so that applications which rely on the auto-aliasing - can shift their applications to use this method. - - .. seealso:: - - :ref:`migration_2736` diff --git a/doc/build/changelog/changelog_10.rst b/doc/build/changelog/changelog_10.rst deleted file mode 100644 index 1db674078f..0000000000 --- a/doc/build/changelog/changelog_10.rst +++ /dev/null @@ -1,3404 +0,0 @@ -============= -1.0 Changelog -============= - -.. changelog_imports:: - - .. include:: changelog_09.rst - :start-line: 5 - - - .. include:: changelog_08.rst - :start-line: 5 - - - .. include:: changelog_07.rst - :start-line: 5 - - - -.. changelog:: - :version: 1.0.19 - :released: August 3, 2017 - - .. change:: - :tags: bug, oracle, performance, py2k - :tickets: 4035 - :versions: 1.0.19, 1.1.13, 1.2.0b3 - - Fixed performance regression caused by the fix for :ticket:`3937` where - cx_Oracle as of version 5.3 dropped the ``.UNICODE`` symbol from its - namespace, which was interpreted as cx_Oracle's "WITH_UNICODE" mode being - turned on unconditionally, which invokes functions on the SQLAlchemy - side which convert all strings to unicode unconditionally and causing - a performance impact. In fact, per cx_Oracle's author the - "WITH_UNICODE" mode has been removed entirely as of 5.1, so the expensive unicode - conversion functions are no longer necessary and are disabled if - cx_Oracle 5.1 or greater is detected under Python 2. The warning against - "WITH_UNICODE" mode that was removed under :ticket:`3937` is also restored. - -.. changelog:: - :version: 1.0.18 - :released: July 24, 2017 - - .. change:: - :tags: bug, tests, py3k - :tickets: 4034 - :versions: 1.0.18, 1.1.12, 1.2.0b2 - - Fixed issue in testing fixtures which was incompatible with a change - made as of Python 3.6.2 involving context managers. - - .. change:: 3937 - :tags: bug, oracle - :tickets: 3937 - :versions: 1.1.7 - - A fix to cx_Oracle's WITH_UNICODE mode which was uncovered by the - fact that cx_Oracle 5.3 now seems to hardcode this flag on in - the build; an internal method that uses this mode wasn't using - the correct signature. - - -.. changelog:: - :version: 1.0.17 - :released: January 17, 2017 - - .. change:: - :tags: bug, py3k - :tickets: 3886 - :versions: 1.1.5 - - Fixed Python 3.6 DeprecationWarnings related to escaped strings without - the 'r' modifier, and added test coverage for Python 3.6. - - .. change:: - :tags: bug, orm - :tickets: 3884 - :versions: 1.1.5 - - Fixed bug involving joined eager loading against multiple entities - when polymorphic inheritance is also in use which would throw - "'NoneType' object has no attribute 'isa'". The issue was introduced - by the fix for :ticket:`3611`. - -.. changelog:: - :version: 1.0.16 - :released: November 15, 2016 - - .. change:: - :tags: bug, orm - :tickets: 3849 - :versions: 1.1.4 - - Fixed bug in :meth:`.Session.bulk_update_mappings` where an alternate-named - primary key attribute would not track properly into the UPDATE statement. - - .. change:: - :tags: bug, mssql - :tickets: 3810 - :versions: 1.1.0 - - Changed the query used to get "default schema name", from one that - queries the database principals table to using the - "schema_name()" function, as issues have been reported that the - former system was unavailable on the Azure Data Warehouse edition. - It is hoped that this will finally work across all SQL Server - versions and authentication styles. - - .. change:: - :tags: bug, mssql - :tickets: 3814 - :versions: 1.1.0 - - Updated the server version info scheme for pyodbc to use SQL Server - SERVERPROPERTY(), rather than relying upon pyodbc.SQL_DBMS_VER, which - continues to be unreliable particularly with FreeTDS. - - .. change:: - :tags: bug, orm - :tickets: 3800 - :versions: 1.1.0 - - Fixed bug where joined eager loading would fail for a polymorphically- - loaded mapper, where the polymorphic_on was set to an un-mapped - expression such as a CASE expression. - - .. change:: - :tags: bug, orm - :tickets: 3798 - :versions: 1.1.0 - - Fixed bug where the ArgumentError raised for an invalid bind - sent to a Session via :meth:`.Session.bind_mapper`, - :meth:`.Session.bind_table`, - or the constructor would fail to be correctly raised. - - .. change:: - :tags: bug, mssql - :tickets: 3791 - :versions: 1.1.0 - - Added error code 20017 "unexpected EOF from the server" to the list of - disconnect exceptions that result in a connection pool reset. Pull - request courtesy Ken Robbins. - - .. change:: - :tags: bug, orm.declarative - :tickets: 3797 - :versions: 1.1.0 - - Fixed bug where setting up a single-table inh subclass of a joined-table - subclass which included an extra column would corrupt the foreign keys - collection of the mapped table, thereby interfering with the - initialization of relationships. - - .. change:: - :tags: bug, orm - :tickets: 3781 - :versions: 1.1.4 - - Fixed bug in :meth:`.Session.bulk_save` where an UPDATE would - not function correctly in conjunction with a mapping that - implements a version id counter. - - .. 3778 - - .. change:: - :tags: bug, orm - :tickets: 3778 - :versions: 1.1.4 - - Fixed bug where the :attr:`_orm.Mapper.attrs`, - :attr:`_orm.Mapper.all_orm_descriptors` and other derived attributes would - fail to refresh when mapper properties or other ORM constructs were - added to the mapper/class after these accessors were first called. - - .. change:: 3762 - :tags: bug, mssql - :tickets: 3762 - :versions: 1.1.4 - - Fixed bug in pyodbc dialect (as well as in the mostly non-working - adodbapi dialect) whereby a semicolon present in the password - or username fields could be interpreted as a separator for another - token; the values are now quoted when semicolons are present. - -.. changelog:: - :version: 1.0.15 - :released: September 1, 2016 - - .. change:: - :tags: bug, mysql - :tickets: 3787 - :versions: 1.1.0 - - Added support for parsing MySQL/Connector boolean and integer - arguments within the URL query string: connection_timeout, - connect_timeout, pool_size, get_warnings, - raise_on_warnings, raw, consume_results, ssl_verify_cert, force_ipv6, - pool_reset_session, compress, allow_local_infile, use_pure. - - .. change:: - :tags: bug, orm - :tickets: 3773, 3774 - :versions: 1.1.0 - - Fixed bug in subquery eager loading where a subqueryload - of an "of_type()" object linked to a second subqueryload of a plain - mapped class, or a longer chain of several "of_type()" attributes, - would fail to link the joins correctly. - - .. change:: - :tags: bug, sql - :tickets: 3755 - :versions: 1.1.0 - - Fixed bug in :class:`_schema.Table` where the internal method - ``_reset_exported()`` would corrupt the state of the object. This - method is intended for selectable objects and is called by the ORM - in some cases; an erroneous mapper configuration would could lead the - ORM to call this on a :class:`_schema.Table` object. - - .. change:: - :tags: bug, ext - :tickets: 3743 - :versions: 1.1.0b3 - - Fixed bug in ``sqlalchemy.ext.baked`` where the unbaking of a - subquery eager loader query would fail due to a variable scoping - issue, when multiple subquery loaders were involved. Pull request - courtesy Mark Hahnenberg. - -.. changelog:: - :version: 1.0.14 - :released: July 6, 2016 - - .. change:: - :tags: bug, postgresql - :tickets: 3739 - :versions: 1.1.0b3 - - Fixed bug whereby :class:`.TypeDecorator` and :class:`.Variant` - types were not deeply inspected enough by the PostgreSQL dialect - to determine if SMALLSERIAL or BIGSERIAL needed to be rendered - rather than SERIAL. - - .. change:: - :tags: bug, oracle - :tickets: 3741 - :versions: 1.1.0b3 - - Fixed bug in :paramref:`.Select.with_for_update.of`, where the Oracle - "rownum" approach to LIMIT/OFFSET would fail to accommodate for the - expressions inside the "OF" clause, which must be stated at the topmost - level referring to expression within the subquery. The expressions are - now added to the subquery if needed. - - .. change:: - :tags: bug, sql - :tickets: 3735 - :versions: 1.1.0b2 - - Fixed issue in SQL math negation operator where the type of the - expression would no longer be the numeric type of the original. - This would cause issues where the type determined result set - behaviors. - - .. change:: - :tags: bug, sql - :tickets: 3728 - :versions: 1.1.0b2 - - Fixed bug whereby the ``__getstate__`` / ``__setstate__`` - methods for sqlalchemy.util.Properties were - non-working due to the transition in the 1.0 series to ``__slots__``. - The issue potentially impacted some third-party applications. - Pull request courtesy Pieter Mulder. - - .. change:: - :tags: bug, sql - :tickets: 3724 - - :meth:`_expression.FromClause.count` is pending deprecation for 1.1. This function - makes use of an arbitrary column in the table and is not reliable; - for Core use, ``func.count()`` should be preferred. - - .. change:: - :tags: bug, sql - :tickets: 3722 - - Fixed bug in :class:`_expression.CTE` structure which would cause it to not - clone properly when a union was used, as is common in a recursive - CTE. The improper cloning would cause errors when the CTE is used - in various ORM contexts such as that of a :func:`.column_property`. - - .. change:: - :tags: bug, sql - :tickets: 3721 - - Fixed bug whereby :meth:`_schema.Table.tometadata` would make a duplicate - :class:`.UniqueConstraint` for each :class:`_schema.Column` object that - featured the ``unique=True`` parameter. - - .. change:: - :tags: bug, engine, postgresql - :tickets: 3716 - - Fixed bug in cross-schema foreign key reflection in conjunction - with the :paramref:`_schema.MetaData.schema` argument, where a referenced - table that is present in the "default" schema would fail since there - would be no way to indicate a :class:`_schema.Table` that has "blank" for - a schema. The special symbol :attr:`_schema.BLANK_SCHEMA` has been - added as an available value for :paramref:`_schema.Table.schema` and - :paramref:`.Sequence.schema`, indicating that the schema name - should be forced to be ``None`` even if :paramref:`_schema.MetaData.schema` - is specified. - - .. change:: - :tags: bug, examples - :tickets: 3704 - - Fixed a regression that occurred in the - examples/vertical/dictlike-polymorphic.py example which prevented it - from running. - -.. changelog:: - :version: 1.0.13 - :released: May 16, 2016 - - .. change:: - :tags: bug, orm - :tickets: 3700 - - Fixed bug in "evaluate" strategy of :meth:`_query.Query.update` and - :meth:`_query.Query.delete` which would fail to accommodate a bound - parameter with a "callable" value, as which occurs when filtering - by a many-to-one equality expression along a relationship. - - .. change:: - :tags: bug, postgresql - :tickets: 3715 - - Added disconnect detection support for the error string - "SSL error: decryption failed or bad record mac". Pull - request courtesy Iuri de Silvio. - - .. change:: - :tags: bug, mssql - :tickets: 3711 - - Fixed bug where by ROW_NUMBER OVER clause applied for OFFSET - selects in SQL Server would inappropriately substitute a plain column - from the local statement that overlaps with a label name used by - the ORDER BY criteria of the statement. - - .. change:: - :tags: bug, orm - :tickets: 3710 - - Fixed bug whereby the event listeners used for backrefs could - be inadvertently applied multiple times, when using a deep class - inheritance hierarchy in conjunction with multiple mapper configuration - steps. - - .. change:: - :tags: bug, orm - :tickets: 3706 - - Fixed bug whereby passing a :func:`_expression.text` construct to the - :meth:`_query.Query.group_by` method would raise an error, instead - of interpreting the object as a SQL fragment. - - .. change:: - :tags: bug, oracle - :tickets: 3705 - - Fixed a bug in the cx_Oracle connect process that caused a TypeError - when the either the user, password or dsn was empty. This prevented - external authentication to Oracle databases, and prevented connecting - to the default dsn. The connect string oracle:// now logs into the - default dsn using the Operating System username, equivalent to - connecting using '/' with sqlplus. - - .. change:: - :tags: bug, oracle - :tickets: 3699 - - Fixed a bug in the result proxy used mainly by Oracle when binary and - other LOB types are in play, such that when query / statement caching - were used, the type-level result processors, notably that required by - the binary type itself but also any other processor, would become lost - after the first run of the statement due to it being removed from the - cached result metadata. - - .. change:: - :tags: bug, examples - :tickets: 3698 - - Changed the "directed graph" example to no longer consider - integer identifiers of nodes as significant; the "higher" / "lower" - references now allow mutual edges in both directions. - - .. change:: - :tags: bug, sql - :tickets: 3690 - - Fixed bug where when using ``case_sensitive=False`` with an - :class:`_engine.Engine`, the result set would fail to correctly accommodate - for duplicate column names in the result set, causing an error - when the statement is executed in 1.0, and preventing the - "ambiguous column" exception from functioning in 1.1. - - .. change:: - :tags: bug, sql - :tickets: 3682 - - Fixed bug where the negation of an EXISTS expression would not - be properly typed as boolean in the result, and also would fail to be - anonymously aliased in a SELECT list as is the case with a - non-negated EXISTS construct. - - .. change:: - :tags: bug, sql - :tickets: 3666 - - Fixed bug where "unconsumed column names" exception would fail to - be raised in the case where :meth:`_expression.Insert.values` were called - with a list of parameter mappings, instead of a single mapping - of parameters. Pull request courtesy Athena Yao. - - .. change:: - :tags: bug, orm - :tickets: 3663 - - Anonymous labeling is applied to a :attr:`.func` construct that is - passed to :func:`.column_property`, so that if the same attribute - is referred to as a column expression twice the names are de-duped, - thus avoiding "ambiguous column" errors. Previously, the - ``.label(None)`` would need to be applied in order for the name - to be de-anonymized. - - .. change:: - :tags: bug, py3k - :tickets: 3660 - - Fixed bug in "to_list" conversion where a single bytes object - would be turned into a list of individual characters. This would - impact among other things using the :meth:`_query.Query.get` method - on a primary key that's a bytes object. - - .. change:: - :tags: bug, orm - :tickets: 3658 - - Fixed regression appearing in the 1.0 series in ORM loading where the - exception raised for an expected column missing would incorrectly - be a ``NoneType`` error, rather than the expected - :class:`.NoSuchColumnError`. - - .. change:: - :tags: bug, mssql, oracle - :tickets: 3657 - - Fixed regression appearing in the 1.0 series which would cause the Oracle - and SQL Server dialects to incorrectly account for result set columns - when these dialects would wrap a SELECT in a subquery in order to - provide LIMIT/OFFSET behavior, and the original SELECT statement - referred to the same column multiple times, such as a column and - a label of that same column. This issue is related - to :ticket:`3658` in that when the error occurred, it would also - cause a ``NoneType`` error, rather than reporting that it couldn't - locate a column. - -.. changelog:: - :version: 1.0.12 - :released: February 15, 2016 - - .. change:: - :tags: bug, orm - :tickets: 3647 - - Fixed bug in :meth:`.Session.merge` where an object with a composite - primary key that has values for some but not all of the PK fields - would emit a SELECT statement leaking the internal NEVER_SET symbol - into the query, rather than detecting that this object does not have - a searchable primary key and no SELECT should be emitted. - - .. change:: - :tags: bug, postgresql - :tickets: 3644 - - Fixed bug in :func:`_expression.text` construct where a double-colon - expression would not escape properly, e.g. ``some\:\:expr``, as is most - commonly required when rendering PostgreSQL-style CAST expressions. - - .. change:: - :tags: bug, sql - :tickets: 3643 - - Fixed issue where the "literal_binds" flag was not propagated - for :func:`_expression.insert`, :func:`_expression.update` or - :func:`_expression.delete` constructs when compiled to string - SQL. Pull request courtesy Tim Tate. - - .. change:: - :tags: bug, oracle, jython - :tickets: 3621 - - Fixed a small issue in the Jython Oracle compiler involving the - rendering of "RETURNING" which allows this currently - unsupported/untested dialect to work rudimentarily with the 1.0 series. - Pull request courtesy Carlos Rivas. - - .. change:: - :tags: bug, sql - :tickets: 3642 - - Fixed issue where inadvertent use of the Python ``__contains__`` - override with a column expression (e.g. by using ``'x' in col``) - would cause an endless loop in the case of an ARRAY type, as Python - defers this to ``__getitem__`` access which never raises for this - type. Overall, all use of ``__contains__`` now raises - NotImplementedError. - - .. change:: - :tags: bug, engine, mysql - :tickets: 2696 - - Revisiting :ticket:`2696`, first released in 1.0.10, which attempts to - work around Python 2's lack of exception context reporting by emitting - a warning for an exception that was interrupted by a second exception - when attempting to roll back the already-failed transaction; this - issue continues to occur for MySQL backends in conjunction with a - savepoint that gets unexpectedly lost, which then causes a - "no such savepoint" error when the rollback is attempted, obscuring - what the original condition was. - - The approach has been generalized to the Core "safe - reraise" function which takes place across the ORM and Core in any - place that a transaction is being rolled back in response to an error - which occurred trying to commit, including the context managers - provided by :class:`.Session` and :class:`_engine.Connection`, and taking - place for operations such as a failure on "RELEASE SAVEPOINT". - Previously, the fix was only in place for a specific path within - the ORM flush/commit process; it now takes place for all transactional - context managers as well. - - .. change:: - :tags: bug, sql - :tickets: 3632 - - Fixed bug in :class:`_schema.Table` metadata construct which appeared - around the 0.9 series where adding columns to a :class:`_schema.Table` - that was unpickled would fail to correctly establish the - :class:`_schema.Column` within the 'c' collection, leading to issues in - areas such as ORM configuration. This could impact use cases such - as ``extend_existing`` and others. - - .. change:: - :tags: bug, py3k - :tickets: 3625 - - Fixed bug where some exception re-raise scenarios would attach - the exception to itself as the "cause"; while the Python 3 interpreter - is OK with this, it could cause endless loops in iPython. - - .. change:: - :tags: bug, mssql - :tickets: 3624 - - Fixed the syntax of the :func:`.extract` function when used on - MSSQL against a datetime value; the quotes around the keyword - are removed. Pull request courtesy Guillaume Doumenc. - - .. change:: - :tags: bug, orm - :tickets: 3623 - - Fixed regression since 0.9 where the 0.9 style loader options - system failed to accommodate for multiple :func:`.undefer_group` - loader options in a single query. Multiple :func:`.undefer_group` - options will now be taken into account even against the same - entity. - - .. change:: - :tags: bug, mssql, firebird - :tickets: 3622 - - Fixed 1.0 regression where the eager fetch of cursor.rowcount was - no longer called for an UPDATE or DELETE statement emitted via plain - text or via the :func:`_expression.text` construct, affecting those drivers - that erase cursor.rowcount once the cursor is closed such as SQL - Server ODBC and Firebird drivers. - - -.. changelog:: - :version: 1.0.11 - :released: December 22, 2015 - - .. change:: - :tags: bug, mysql - :tickets: 3613 - - An adjustment to the regular expression used to parse MySQL views, - such that we no longer assume the "ALGORITHM" keyword is present in - the reflected view source, as some users have reported this not being - present in some Amazon RDS environments. - - .. change:: - :tags: bug, mysql - - Added new reserved words for MySQL 5.7 to the MySQL dialect, - including 'generated', 'optimizer_costs', 'stored', 'virtual'. - Pull request courtesy Hanno Schlichting. - - .. change:: - :tags: bug, ext - :tickets: 3605 - - Further fixes to :ticket:`3605`, pop method on :class:`.MutableDict`, - where the "default" argument was not included. - - .. change:: - :tags: bug, ext - :tickets: 3612 - - Fixed bug in baked loader system where the systemwide monkeypatch - for setting up baked lazy loaders would interfere with other - loader strategies that rely on lazy loading as a fallback, e.g. - joined and subquery eager loaders, leading to ``IndexError`` - exceptions at mapper configuration time. - - .. change:: - :tags: bug, orm - :tickets: 3611 - - Fixed regression caused in 1.0.10 by the fix for :ticket:`3593` where - the check added for a polymorphic joinedload from a - poly_subclass->class->poly_baseclass connection would fail for the - scenario of class->poly_subclass->class. - - .. change:: - :tags: bug, orm - :tickets: 3610 - - Fixed bug where :meth:`.Session.bulk_update_mappings` and related - would not bump a version id counter when in use. The experience - here is still a little rough as the original version id is required - in the given dictionaries and there's not clean error reporting - on that yet. - - .. change:: - :tags: bug, sql - :tickets: 3609 - - Fixed bug in :meth:`_expression.Update.return_defaults` which would cause all - insert-default holding columns not otherwise included in the SET - clause (such as primary key cols) to get rendered into the RETURNING - even though this is an UPDATE. - - .. change:: - :tags: bug, orm - :tickets: 3609 - - Major fixes to the :paramref:`_orm.Mapper.eager_defaults` flag, this - flag would not be honored correctly in the case that multiple - UPDATE statements were to be emitted, either as part of a flush - or a bulk update operation. Additionally, RETURNING - would be emitted unnecessarily within update statements. - - .. change:: - :tags: bug, orm - :tickets: 3606 - - Fixed bug where use of the :meth:`_query.Query.select_from` method would - cause a subsequent call to the :meth:`_query.Query.with_parent` method to - fail. - -.. changelog:: - :version: 1.0.10 - :released: December 11, 2015 - - .. change:: - :tags: bug, ext - :tickets: 3605 - - Added support for the ``dict.pop()`` and ``dict.popitem()`` methods - to the :class:`.mutable.MutableDict` class. - - .. change:: - :tags: change, tests - - The ORM and Core tutorials, which have always been in doctest format, - are now exercised within the normal unit test suite in both Python - 2 and Python 3. - - .. change:: - :tags: bug, sql - :tickets: 3603 - - Fixed issue within the :meth:`_expression.Insert.from_select` construct whereby - the :class:`_expression.Select` construct would have its ``._raw_columns`` - collection mutated in-place when compiling the :class:`_expression.Insert` - construct, when the target :class:`_schema.Table` has Python-side defaults. - The :class:`_expression.Select` construct would compile standalone with the - erroneous column present subsequent to compilation of the - :class:`_expression.Insert`, and the :class:`_expression.Insert` statement itself would - fail on a second compile attempt due to duplicate bound parameters. - - .. change:: - :tags: bug, mysql - :tickets: 3602 - - Fixed bug in MySQL reflection where the "fractional sections portion" - of the :class:`.mysql.DATETIME`, :class:`.mysql.TIMESTAMP` and - :class:`.mysql.TIME` types would be incorrectly placed into the - ``timezone`` attribute, which is unused by MySQL, instead of the - ``fsp`` attribute. - - .. change:: - :tags: bug, orm - :tickets: 3599 - - Fixed issue where post_update on a many-to-one relationship would - fail to emit an UPDATE in the case where the attribute were set to - None and not previously loaded. - - .. change:: - :tags: bug, sql, postgresql - :tickets: 3598 - - Fixed bug where CREATE TABLE with a no-column table, but a constraint - such as a CHECK constraint would render an erroneous comma in the - definition; this scenario can occur such as with a PostgreSQL - INHERITS table that has no columns of its own. - - .. change:: - :tags: bug, mssql - :tickets: 3585 - - - Added the error "20006: Write to the server failed" to the list - of disconnect errors for the pymssql driver, as this has been observed - to render a connection unusable. - - .. change:: - :tags: bug, postgresql - :tickets: 3573 - - - Fixed issue where the "FOR UPDATE OF" PostgreSQL-specific SELECT - modifier would fail if the referred table had a schema qualifier; - PG needs the schema name to be omitted. Pull request courtesy - Diana Clarke. - - .. change:: - :tags: bug, postgresql - - - Fixed bug where some varieties of SQL expression passed to the - "where" clause of :class:`_postgresql.ExcludeConstraint` would fail - to be accepted correctly. Pull request courtesy aisch. - - .. change:: - :tags: bug, orm, declarative - - - Fixed bug where in Py2K a unicode literal would not be accepted as the - string name of a class or other argument within declarative using - :func:`.backref` on :func:`_orm.relationship`. Pull request courtesy - Nils Philippsen. - - .. change:: - :tags: bug, mssql - - A descriptive ValueError is now raised in the event that SQL server - returns an invalid date or time format from a DATE or TIME - column, rather than failing with a NoneType error. Pull request - courtesy Ed Avis. - - .. change:: - :tags: bug, py3k - - Updates to internal getargspec() calls, some py36-related - fixture updates, and alterations to two iterators to "return" instead - of raising StopIteration, to allow tests to pass without - errors or warnings on Py3.5, Py3.6, pull requests courtesy - Jacob MacDonald, Luri de Silvio, and Phil Jones. - - .. change:: - :tags: bug, ext - :tickets: 3597 - - Fixed an issue in baked queries where the .get() method, used either - directly or within lazy loads, didn't consider the mapper's "get clause" - as part of the cache key, causing bound parameter mismatches if the - clause got re-generated. This clause is cached by mappers - on the fly but in highly concurrent scenarios may be generated more - than once when first accessed. - - .. change:: - :tags: feature, sql - - Added support for parameter-ordered SET clauses in an UPDATE - statement. This feature is available by passing the - :paramref:`~.sqlalchemy.sql.expression.update.preserve_parameter_order` - flag either to the core :class:`_expression.Update` construct or alternatively - adding it to the :paramref:`.Query.update.update_args` dictionary at - the ORM-level, also passing the parameters themselves as a list of 2-tuples. - Thanks to Gorka Eguileor for implementation and tests. - - .. seealso:: - - :ref:`tutorial_parameter_ordered_updates` - - .. change:: - :tags: bug, orm - :tickets: 3593 - - Fixed bug which is actually a regression that occurred between - versions 0.8.0 and 0.8.1, due :ticket:`2714`. The case where - joined eager loading needs to join out over a subclass-bound - relationship when "with_polymorphic" were also used would fail - to join from the correct entity. - - .. change:: - :tags: bug, orm - :tickets: 3592 - - Fixed joinedload bug which would occur when a. the query includes - limit/offset criteria that forces a subquery b. the relationship - uses "secondary" c. the primaryjoin of the relationship refers to - a column that is either not part of the primary key, or is a PK - col in a joined-inheritance subclass table that is under a different - attribute name than the parent table's primary key column d. the - query defers the columns that are present in the primaryjoin, typically - via not being included in load_only(); the necessary column(s) would - not be present in the subquery and produce invalid SQL. - - .. change:: - :tags: bug, orm - :tickets: 2696 - - A rare case which occurs when a :meth:`.Session.rollback` fails in the - scope of a :meth:`.Session.flush` operation that's raising an - exception, as has been observed in some MySQL SAVEPOINT cases, prevents - the original database exception from being observed when it was - emitted during flush, but only on Py2K because Py2K does not support - exception chaining; on Py3K the originating exception is chained. As - a workaround, a warning is emitted in this specific case showing at - least the string message of the original database error before we - proceed to raise the rollback-originating exception. - - .. change:: - :tags: bug, postgresql - :tickets: 3571 - - Fixed the ``.python_type`` attribute of :class:`_postgresql.INTERVAL` - to return ``datetime.timedelta`` in the same way as that of - :obj:`.types.Interval.python_type`, rather than raising - ``NotImplementedError``. - - .. change:: - :tags: bug, mssql - - - Fixed issue where DDL generated for the MSSQL types DATETIME2, - TIME and DATETIMEOFFSET with a precision of "zero" would not generate - the precision field. Pull request courtesy Jacobo de Vera. - - -.. changelog:: - :version: 1.0.9 - :released: October 20, 2015 - - .. change:: - :tags: bug, orm, postgresql - :tickets: 3556 - - Fixed regression in 1.0 where new feature of using "executemany" - for UPDATE statements in the ORM (e.g. :ref:`feature_updatemany`) - would break on PostgreSQL and other RETURNING backends - when using server-side version generation - schemes, as the server side value is retrieved via RETURNING which - is not supported with executemany. - - .. change:: - :tags: feature, ext - :tickets: 3551 - - Added the :paramref:`.AssociationProxy.info` parameter to the - :class:`.AssociationProxy` constructor, to suit the - :attr:`.AssociationProxy.info` accessor that was added in - :ticket:`2971`. This is possible because :class:`.AssociationProxy` - is constructed explicitly, unlike a hybrid which is constructed - implicitly via the decorator syntax. - - .. change:: - :tags: bug, oracle - :tickets: 3548 - - Fixed bug in Oracle dialect where reflection of tables and other - symbols with names quoted to force all-lower-case would not be - identified properly in reflection queries. The :class:`.quoted_name` - construct is now applied to incoming symbol names that detect as - forced into all-lower-case within the "name normalize" process. - - .. change:: - :tags: feature, orm - - Added new method :meth:`_query.Query.one_or_none`; same as - :meth:`_query.Query.one` but returns None if no row found. Pull request - courtesy esiegerman. - - .. change:: - :tags: bug, orm - :tickets: 3539 - - Fixed rare TypeError which could occur when stringifying certain - kinds of internal column loader options within internal logging. - - .. change:: - :tags: bug, orm - :tickets: 3525 - - Fixed bug in :meth:`.Session.bulk_save_objects` where a mapped - column that had some kind of "fetch on update" value and was not - locally present in the given object would cause an AttributeError - within the operation. - - .. change:: - :tags: bug, sql - :tickets: 3520 - - Fixed regression in 1.0-released default-processor for multi-VALUES - insert statement, :ticket:`3288`, where the column type for the - default-holding column would not be propagated to the compiled - statement in the case where the default was being used, - leading to bind-level type handlers not being invoked. - - .. change:: - :tags: bug, examples - - - Fixed two issues in the "history_meta" example where history tracking - could encounter empty history, and where a column keyed to an alternate - attribute name would fail to track properly. Fixes courtesy - Alex Fraser. - - .. change:: - :tags: bug, orm - :tickets: 3510 - - - Fixed 1.0 regression where the "noload" loader strategy would fail - to function for a many-to-one relationship. The loader used an - API to place "None" into the dictionary which no longer actually - writes a value; this is a side effect of :ticket:`3061`. - - .. change:: - :tags: bug, sybase - :tickets: 3508, 3509 - - - Fixed two issues regarding Sybase reflection, allowing tables - without primary keys to be reflected as well as ensured that - a SQL statement involved in foreign key detection is pre-fetched up - front to avoid driver issues upon nested queries. Fixes here - courtesy Eugene Zapolsky; note that we cannot currently test - Sybase to locally verify these changes. - - .. change:: - :tags: bug, postgresql - - - An adjustment to the new PostgreSQL feature of reflecting storage - options and USING of :ticket:`3455` released in 1.0.6, - to disable the feature for PostgreSQL versions < 8.2 where the - ``reloptions`` column is not provided; this allows Amazon Redshift - to again work as it is based on an 8.0.x version of PostgreSQL. - Fix courtesy Pete Hollobon. - - -.. changelog:: - :version: 1.0.8 - :released: July 22, 2015 - - .. change:: - :tags: bug, misc - :tickets: 3494 - - Fixed an issue where a particular base class within utils - didn't implement ``__slots__``, and therefore meant all subclasses - of that class didn't either, negating the rationale for ``__slots__`` - to be in use. Didn't cause any issue except on IronPython - which apparently does not implement ``__slots__`` behavior compatibly - with cPython. - - -.. changelog:: - :version: 1.0.7 - :released: July 20, 2015 - - .. change:: - :tags: feature, sql - :tickets: 3459 - - Added a :meth:`_expression.ColumnElement.cast` method which performs the same - purpose as the standalone :func:`_expression.cast` function. Pull - request courtesy Sebastian Bank. - - .. change:: - :tags: bug, engine - :tickets: 3481 - - Fixed regression where new methods on :class:`_engine.ResultProxy` used - by the ORM :class:`_query.Query` object (part of the performance - enhancements of :ticket:`3175`) would not raise the "this result - does not return rows" exception in the case where the driver - (typically MySQL) fails to generate cursor.description correctly; - an AttributeError against NoneType would be raised instead. - - .. change:: - :tags: bug, engine - :tickets: 3483 - - Fixed regression where :meth:`_engine.ResultProxy.keys` would return - un-adjusted internal symbol names for "anonymous" labels, which - are the "foo_1" types of labels we see generated for SQL functions - without labels and similar. This was a side effect of the - performance enhancements implemented as part of #918. - - - .. change:: - :tags: bug, sql - :tickets: 3490 - - Fixed bug where coercion of literal ``True`` or ``False`` constant - in conjunction with :func:`.and_` or :func:`.or_` would fail - with an AttributeError. - - .. change:: - :tags: bug, sql - :tickets: 3485 - - Fixed potential issue where a custom subclass - of :class:`.FunctionElement` or other column element that incorrectly - states 'None' or any other invalid object as the ``.type`` - attribute will report this exception instead of recursion overflow. - - .. change:: - :tags: bug, sql - - Fixed bug where the modulus SQL operator wouldn't work in reverse - due to a missing ``__rmod__`` method. Pull request courtesy - dan-gittik. - - .. change:: - :tags: feature, schema - - Added support for the MINVALUE, MAXVALUE, NO MINVALUE, NO MAXVALUE, - and CYCLE arguments for CREATE SEQUENCE as supported by PostgreSQL - and Oracle. Pull request courtesy jakeogh. - - .. change:: - :tags: bug, orm, declarative - :tickets: 3480 - - Fixed bug in :class:`.AbstractConcreteBase` extension where - a column setup on the ABC base which had a different attribute - name vs. column name would not be correctly mapped on the final - base class. The failure on 0.9 would be silent whereas on - 1.0 it raised an ArgumentError, so may not have been noticed - prior to 1.0. - - .. change:: - :tags: bug, orm - :tickets: 3469 - - Fixed 1.0 regression where value objects that override - ``__eq__()`` to return a non-boolean-capable object, such as - some geoalchemy types as well as numpy types, were being tested - for ``bool()`` during a unit of work update operation, where in - 0.9 the return value of ``__eq__()`` was tested against "is True" - to guard against this. - - .. change:: - :tags: bug, orm - :tickets: 3468 - - Fixed 1.0 regression where a "deferred" attribute would not populate - correctly if it were loaded within the "optimized inheritance load", - which is a special SELECT emitted in the case of joined table - inheritance used to populate expired or unloaded attributes against - a joined table without loading the base table. This is related to - the fact that SQLA 1.0 no longer guesses about loading deferred - columns and must be directed explicitly. - - .. change:: - :tags: bug, orm - :tickets: 3466 - - Fixed 1.0 regression where the "parent entity" of a synonym- - mapped attribute on top of an :func:`.aliased` object would - resolve to the original mapper, not the :func:`.aliased` - version of it, thereby causing problems for a :class:`_query.Query` - that relies on this attribute (e.g. it's the only representative - attribute given in the constructor) to figure out the correct FROM - clause for the query. - -.. changelog:: - :version: 1.0.6 - :released: June 25, 2015 - - .. change:: - :tags: bug, orm - :tickets: 3465 - - Fixed a major regression in the 1.0 series where the version_id_counter - feature would cause an object's version counter to be incremented - when there was no net change to the object's row, but instead an object - related to it via relationship (e.g. typically many-to-one) - were associated or de-associated with it, resulting in an UPDATE - statement that updates the object's version counter and nothing else. - In the use case where the relatively recent "server side" and/or - "programmatic/conditional" version counter feature were used - (e.g. setting version_id_generator to False), the bug could cause an - UPDATE without a valid SET clause to be emitted. - - .. change:: - :tags: bug, mssql - :tickets: 3464 - - Fixed issue when using :class:`_types.VARBINARY` type in conjunction with - an INSERT of NULL + pyodbc; pyodbc requires a special - object be passed in order to persist NULL. As the :class:`_types.VARBINARY` - type is now usually the default for :class:`.LargeBinary` due to - :ticket:`3039`, this issue is partially a regression in 1.0. - The pymssql driver appears to be unaffected. - - .. change:: - :tags: bug, postgresql, pypy - :tickets: 3439 - - Re-fixed this issue first released in 1.0.5 to fix psycopg2cffi - JSONB support once again, as they suddenly - switched on unconditional decoding of JSONB types in version 2.7.1. - Version detection now specifies 2.7.1 as where we should expect - the DBAPI to do json encoding for us. - - .. change:: - :tags: feature, postgresql - :tickets: 3455 - - Added support for storage parameters under CREATE INDEX, using - a new keyword argument ``postgresql_with``. Also added support for - reflection to support both the ``postgresql_with`` flag as well - as the ``postgresql_using`` flag, which will now be set on - :class:`.Index` objects that are reflected, as well present - in a new "dialect_options" dictionary in the result of - :meth:`_reflection.Inspector.get_indexes`. Pull request courtesy Pete Hollobon. - - .. seealso:: - - :ref:`postgresql_index_storage` - - .. change:: - :tags: bug, orm - :tickets: 3462 - - Fixed 1.0 regression where the enhanced behavior of single-inheritance - joins of :ticket:`3222` takes place inappropriately - for a JOIN along explicit join criteria with a single-inheritance - subclass that does not make use of any discriminator, resulting - in an additional "AND NULL" clause. - - .. change:: - :tags: bug, postgresql - :tickets: 3454 - - Repaired the :class:`.ExcludeConstraint` construct to support common - features that other objects like :class:`.Index` now do, that - the column expression may be specified as an arbitrary SQL - expression such as :obj:`_expression.cast` or :obj:`_expression.text`. - - .. change:: - :tags: feature, postgresql - - Added new execution option ``max_row_buffer`` which is interpreted - by the psycopg2 dialect when the ``stream_results`` option is - used, which sets a limit on the size of the row buffer that may be - allocated. This value is also provided based on the integer - value sent to :meth:`_query.Query.yield_per`. Pull request courtesy - mcclurem. - - .. change:: - :tags: bug, orm - :tickets: 3451 - - Fixed bug in new :meth:`.Session.bulk_update_mappings` feature where - the primary key columns used in the WHERE clause to locate the row - would also be included in the SET clause, setting their value to - themselves unnecessarily. Pull request courtesy Patrick Hayes. - - .. change:: - :tags: bug, orm - :tickets: 3448 - - Fixed an unexpected-use regression whereby custom - :class:`.types.TypeEngine.Comparator` objects that made use of the - ``__clause_element__()`` method and returned an object that was an - ORM-mapped :class:`.InstrumentedAttribute` and not explicitly a - :class:`_expression.ColumnElement` would fail to be correctly handled when passed - as an expression to :meth:`.Session.query`. The logic in 0.9 happened - to succeed on this, so this use case is now supported. - - .. change:: - :tags: bug, sql - :tickets: 3445 - - Fixed a bug where clause adaption as applied to a :class:`.Label` - object would fail to accommodate the labeled SQL expression - in all cases, such that any SQL operation that made use of - :meth:`.Label.self_group` would use the original unadapted - expression. One effect of this would be that an ORM :func:`.aliased` - construct would not fully accommodate attributes mapped by - :obj:`.column_property`, such that the un-aliased table could - leak out when the property were used in some kinds of SQL - comparisons. - - .. change:: - :tags: bug, documentation - :tickets: 2077 - - Fixed an internal "memoization" routine for method types such - that a Python descriptor is no longer used; repairs inspectability - of these methods including support for Sphinx documentation. - -.. changelog:: - :version: 1.0.5 - :released: June 7, 2015 - - .. change:: - :tags: feature, engine - - Added new engine event :meth:`_events.ConnectionEvents.engine_disposed`. - Called after the :meth:`_engine.Engine.dispose` method is called. - - .. change:: - :tags: bug, postgresql, pypy - :tickets: 3439 - - Repaired some typing and test issues related to the pypy - psycopg2cffi dialect, in particular that the current 2.7.0 version - does not have native support for the JSONB type. The version detection - for psycopg2 features has been tuned into a specific sub-version - for psycopg2cffi. Additionally, test coverage has been enabled - for the full series of psycopg2 features under psycopg2cffi. - - .. change:: - :tags: feature, ext - - Added support for ``*args`` to be passed to the baked query - initial callable, in the same way that ``*args`` are supported - for the :meth:`.BakedQuery.add_criteria` and - :meth:`.BakedQuery.with_criteria` methods. Initial PR courtesy - Naoki INADA. - - .. change:: - :tags: bug, engine - :tickets: 3435 - - Fixed bug where known boolean values used by - :func:`.engine_from_config` were not being parsed correctly; - these included ``pool_threadlocal`` and the psycopg2 argument - ``use_native_unicode``. - - .. change:: - :tags: bug, mssql - :tickets: 3424, 3430 - - Added a new dialect flag to the MSSQL dialect - ``legacy_schema_aliasing`` which when set to False will disable a - very old and obsolete behavior, that of the compiler's - attempt to turn all schema-qualified table names into alias names, - to work around old and no longer locatable issues where SQL - server could not parse a multi-part identifier name in all - circumstances. The behavior prevented more - sophisticated statements from working correctly, including those which - use hints, as well as CRUD statements that embed correlated SELECT - statements. Rather than continue to repair the feature to work - with more complex statements, it's better to just disable it - as it should no longer be needed for any modern SQL server - version. The flag defaults to True for the 1.0.x series, leaving - current behavior unchanged for this version series. In the 1.1 - series, it will default to False. For the 1.0 series, - when not set to either value explicitly, a warning is emitted - when a schema-qualified table is first used in a statement, which - suggests that the flag be set to False for all modern SQL Server - versions. - - .. seealso:: - - :ref:`legacy_schema_rendering` - - .. change:: - :tags: feature, engine - :tickets: 3379 - - Adjustments to the engine plugin hook, such that the - :meth:`.URL.get_dialect` method will continue to return the - ultimate :class:`.Dialect` object when a dialect plugin is used, - without the need for the caller to be aware of the - :meth:`.Dialect.get_dialect_cls` method. - - - .. change:: - :tags: bug, ext - :tickets: 3427 - - Fixed regression in the :mod:`sqlalchemy.ext.mutable` extension - as a result of the bugfix for :ticket:`3167`, - where attribute and validation events are no longer - called within the flush process. The mutable - extension was relying upon this behavior in the case where a column - level Python-side default were responsible for generating the new value - on INSERT or UPDATE, or when a value were fetched from the RETURNING - clause for "eager defaults" mode. The new value would not be subject - to any event when populated and the mutable extension could not - establish proper coercion or history listening. A new event - :meth:`.InstanceEvents.refresh_flush` is added which the mutable - extension now makes use of for this use case. - - .. change:: - :tags: feature, orm - :tickets: 3427 - - Added new event :meth:`.InstanceEvents.refresh_flush`, invoked - when an INSERT or UPDATE level default value fetched via RETURNING - or Python-side default is invoked within the flush process. This - is to provide a hook that is no longer present as a result of - :ticket:`3167`, where attribute and validation events are no longer - called within the flush process. - - .. change:: - :tags: feature, ext - :tickets: 3427 - - Added a new semi-public method to :class:`.MutableBase` - :meth:`.MutableBase._get_listen_keys`. Overriding this method - is needed in the case where a :class:`.MutableBase` subclass needs - events to propagate for attribute keys other than the key to which - the mutable type is associated with, when intercepting the - :meth:`.InstanceEvents.refresh` or - :meth:`.InstanceEvents.refresh_flush` events. The current example of - this is composites using :class:`.MutableComposite`. - - .. change:: - :tags: bug, engine - :tickets: 3421 - - Added support for the case of the misbehaving DBAPI that has - pep-249 exception names linked to exception classes of an entirely - different name, preventing SQLAlchemy's own exception wrapping from - wrapping the error appropriately. - The SQLAlchemy dialect in use needs to implement a new - accessor :attr:`.DefaultDialect.dbapi_exception_translation_map` - to support this feature; this is implemented now for the py-postgresql - dialect. - - .. change:: - :tags: bug, orm - :tickets: 3420 - - The "lightweight named tuple" used when a :class:`_query.Query` returns - rows failed to implement ``__slots__`` correctly such that it still - had a ``__dict__``. This is resolved, but in the extremely - unlikely case someone was assigning values to the returned tuples, - that will no longer work. - - .. change:: - :tags: bug, engine - :tickets: 3419 - - Fixed bug involving the case when pool checkout event handlers are used - and connection attempts are made in the handler itself which fail, - the owning connection record would not be freed until the stack trace - of the connect error itself were freed. For the case where a test - pool of only a single connection were used, this means the pool would - be fully checked out until that stack trace were freed. This mostly - impacts very specific debugging scenarios and is unlikely to have been - noticeable in any production application. The fix applies an - explicit checkin of the record before re-raising the caught exception. - - -.. changelog:: - :version: 1.0.4 - :released: May 7, 2015 - - .. change:: - :tags: bug, orm - :tickets: 3416 - - Fixed unexpected-use regression where in the odd case that the - primaryjoin of a relationship involved comparison to an unhashable - type such as an HSTORE, lazy loads would fail due to a hash-oriented - check on the statement parameters, modified in 1.0 as a result of - :ticket:`3061` to use hashing and modified in :ticket:`3368` - to occur in cases more common than "load on pending". - The values are now checked for the ``__hash__`` attribute beforehand. - - .. change:: - :tags: bug, orm - :tickets: 3412, 3347 - - Liberalized an assertion that was added as part of :ticket:`3347` - to protect against unknown conditions when splicing inner joins - together within joined eager loads with ``innerjoin=True``; if - some of the joins use a "secondary" table, the assertion needs to - unwrap further joins in order to pass. - - .. change:: - :tags: bug, schema - :tickets: 3411 - - Fixed bug in enhanced constraint-attachment logic introduced in - :ticket:`3341` where in the unusual case of a constraint that refers - to a mixture of :class:`_schema.Column` objects and string column names - at the same time, the auto-attach-on-column-attach logic will be - skipped; for the constraint to be auto-attached in this case, - all columns must be assembled on the target table up front. - Added a new section to the migration document regarding the - original feature as well as this change. - - .. seealso:: - - :ref:`change_3341` - - .. change:: - :tags: bug, orm - :tickets: 3409, 3320 - - Repaired / added to tests yet more expressions that were reported - as failing with the new 'entity' key value added to - :attr:`_query.Query.column_descriptions`, the logic to discover the "from" - clause is again reworked to accommodate columns from aliased classes, - as well as to report the correct value for the "aliased" flag in these - cases. - - -.. changelog:: - :version: 1.0.3 - :released: April 30, 2015 - - .. change:: - :tags: bug, orm, pypy - :tickets: 3405 - - Fixed regression from 0.9.10 prior to release due to :ticket:`3349` - where the check for query state on :meth:`_query.Query.update` or - :meth:`_query.Query.delete` compared the empty tuple to itself using ``is``, - which fails on PyPy to produce ``True`` in this case; this would - erroneously emit a warning in 0.9 and raise an exception in 1.0. - - .. change:: - :tags: feature, engine - :tickets: 3379 - - New features added to support engine/pool plugins with advanced - functionality. Added a new "soft invalidate" feature to the - connection pool at the level of the checked out connection wrapper - as well as the :class:`._ConnectionRecord`. This works similarly - to a modern pool invalidation in that connections aren't actively - closed, but are recycled only on next checkout; this is essentially - a per-connection version of that feature. A new event - :meth:`_events.PoolEvents.soft_invalidate` is added to complement it. - - Also added new flag - :attr:`.ExceptionContext.invalidate_pool_on_disconnect`. - Allows an error handler within :meth:`_events.ConnectionEvents.handle_error` - to maintain a "disconnect" condition, but to handle calling invalidate - on individual connections in a specific manner within the event. - - .. change:: - :tags: feature, engine - :tickets: 3355 - - Added new event :class:`.DialectEvents.do_connect`, which allows - interception / replacement of when the :meth:`.Dialect.connect` - hook is called to create a DBAPI connection. Also added - dialect plugin hooks :meth:`.Dialect.get_dialect_cls` and - :meth:`.Dialect.engine_created` which allow external plugins to - add events to existing dialects using entry points. - - .. change:: - :tags: bug, orm - :tickets: 3403, 3320 - - Fixed regression from 0.9.10 prior to release where the new addition - of ``entity`` to the :attr:`_query.Query.column_descriptions` accessor - would fail if the target entity was produced from a core selectable - such as a :class:`_schema.Table` or :class:`_expression.CTE` object. - - .. change:: - :tags: feature, sql - - Added a placeholder method :meth:`.TypeEngine.compare_against_backend` - which is now consumed by Alembic migrations as of 0.7.6. User-defined - types can implement this method to assist in the comparison of - a type against one reflected from the database. - - .. change:: - :tags: bug, orm - :tickets: 3402 - - Fixed regression within the flush process when an attribute were - set to a SQL expression for an UPDATE, and the SQL expression when - compared to the previous value of the attribute would produce a SQL - comparison other than ``==`` or ``!=``, the exception "Boolean value - of this clause is not defined" would raise. The fix ensures that - the unit of work will not interpret the SQL expression in this way. - - .. change:: - :tags: bug, ext - :tickets: 3397 - - Fixed bug in association proxy where an any()/has() - on an relationship->scalar non-object attribute comparison would fail, - e.g. - ``filter(Parent.some_collection_to_attribute.any(Child.attr == 'foo'))`` - - .. change:: - :tags: bug, sql - :tickets: 3396 - - Fixed bug where the truncation of long labels in SQL could produce - a label that overlapped another label that is not truncated; this - because the length threshold for truncation was greater than - the portion of the label that remains after truncation. These - two values have now been made the same; label_length - 6. - The effect here is that shorter column labels will be "truncated" - where they would not have been truncated before. - - .. change:: - :tags: bug, orm - :tickets: 3392 - - Fixed unexpected use regression due to :ticket:`2992` where - textual elements placed - into the :meth:`_query.Query.order_by` clause in conjunction with joined - eager loading would be added to the columns clause of the inner query - in such a way that they were assumed to be table-bound column names, - in the case where the joined eager load needs to wrap the query - in a subquery to accommodate for a limit/offset. - - Originally, the behavior here was intentional, in that a query such - as ``query(User).order_by('name').limit(1)`` - would order by ``user.name`` even if the query was modified by - joined eager loading to be within a subquery, as ``'name'`` would - be interpreted as a symbol to be located within the FROM clauses, - in this case ``User.name``, which would then be copied into the - columns clause to ensure it were present for ORDER BY. However, the - feature fails to anticipate the case where ``order_by("name")`` refers - to a specific label name present in the local columns clause already - and not a name bound to a selectable in the FROM clause. - - Beyond that, the feature also fails for deprecated cases such as - ``order_by("name desc")``, which, while it emits a - warning that :func:`_expression.text` should be used here (note that the issue - does not impact cases where :func:`_expression.text` is used explicitly), - still produces a different query than previously where the "name desc" - expression is copied into the columns clause inappropriately. The - resolution is such that the "joined eager loading" aspect of the - feature will skip over these so-called "label reference" expressions - when augmenting the inner columns clause, as though they were - :func:`_expression.text` constructs already. - - .. change:: - :tags: bug, sql - :tickets: 3391 - - Fixed regression due to :ticket:`3282` where the ``tables`` collection - passed as a keyword argument to the :meth:`.DDLEvents.before_create`, - :meth:`.DDLEvents.after_create`, :meth:`.DDLEvents.before_drop`, and - :meth:`.DDLEvents.after_drop` events would no longer be a list - of tables, but instead a list of tuples which contained a second - entry with foreign keys to be added or dropped. As the ``tables`` - collection, while documented as not necessarily stable, has come - to be relied upon, this change is considered a regression. - Additionally, in some cases for "drop", this collection would - be an iterator that would cause the operation to fail if - prematurely iterated. The collection is now a list of table - objects in all cases and test coverage for the format of this - collection is now added. - - - .. change:: - :tags: bug, orm - :tickets: 3388 - - Fixed a regression regarding the :meth:`.MapperEvents.instrument_class` - event where its invocation was moved to be after the class manager's - instrumentation of the class, which is the opposite of what the - documentation for the event explicitly states. The rationale for the - switch was due to Declarative taking the step of setting up - the full "instrumentation manager" for a class before it was mapped - for the purpose of the new ``@declared_attr`` features - described in :ref:`feature_3150`, but the change was also made - against the classical use of :class:`_orm.Mapper` for consistency. - However, SQLSoup relies upon the instrumentation event happening - before any instrumentation under classical mapping. - The behavior is reverted in the case of classical and declarative - mapping, the latter implemented by using a simple memoization - without using class manager. - - .. change:: - :tags: bug, orm - :tickets: 3387 - - Fixed issue in new :meth:`.QueryEvents.before_compile` event where - changes made to the :class:`_query.Query` object's collection of entities - to load within the event would render in the SQL, but would not - be reflected during the loading process. - -.. changelog:: - :version: 1.0.2 - :released: April 24, 2015 - - .. change:: - :tags: bug, sql - :tickets: 3338, 3385 - - Fixed a regression that was incorrectly fixed in 1.0.0b4 - (hence becoming two regressions); reports that - SELECT statements would GROUP BY a label name and fail was misconstrued - that certain backends such as SQL Server should not be emitting - ORDER BY or GROUP BY on a simple label name at all; when in fact, - we had forgotten that 0.9 was already emitting ORDER BY on a simple - label name for all backends, as described in :ref:`migration_1068`, - even though 1.0 includes a rewrite of this logic as part of - :ticket:`2992`. As far - as emitting GROUP BY against a simple label, even PostgreSQL has - cases where it will raise an error even though the label to group - on should be apparent, so it is clear that GROUP BY should never - be rendered in this way automatically. - - In 1.0.2, SQL Server, Firebird and others will again emit ORDER BY on - a simple label name when passed a - :class:`.Label` construct that is also present in the columns clause. - Additionally, no backend will emit GROUP BY against the simple label - name only when passed a :class:`.Label` construct. - - .. change:: - :tags: bug, orm, declarative - :tickets: 3383 - - Fixed unexpected use regression regarding the declarative - ``__declare_first__`` and ``__declare_last__`` accessors where these - would no longer be called on the superclass of the declarative base. - -.. changelog:: - :version: 1.0.1 - :released: April 23, 2015 - - .. change:: - :tags: bug, firebird - :tickets: 3380 - - Fixed a regression due to :ticket:`3034` where limit/offset - clauses were not properly interpreted by the Firebird dialect. - Pull request courtesy effem-git. - - .. change:: - :tags: bug, firebird - :tickets: 3381 - - Fixed support for "literal_binds" mode when using limit/offset - with Firebird, so that the values are again rendered inline when - this is selected. Related to :ticket:`3034`. - - .. change:: - :tags: bug, sqlite - :tickets: 3378 - - Fixed a regression due to :ticket:`3282`, where due to the fact that - we attempt to assume the availability of ALTER when creating/dropping - schemas, in the case of SQLite we simply said to not worry about - foreign keys at all, since ALTER is not available, when creating - and dropping tables. This meant that the sorting of tables was - basically skipped in the case of SQLite, and for the vast majority - of SQLite use cases, this is not an issue. - - However, users who were doing DROPs on SQLite - with tables that contained data and with referential integrity - turned on would then experience errors, as the - dependency sorting *does* matter in the case of DROP with - enforced constraints, when those tables have data (SQLite will still - happily let you create foreign keys to nonexistent tables and drop - tables referring to existing ones with constraints enabled, as long as - there's no data being referenced). - - In order to maintain the new feature of :ticket:`3282` while still - allowing a SQLite DROP operation to maintain ordering, we now - do the sort with full FKs taken under consideration, and if we encounter - an unresolvable cycle, only *then* do we forego attempting to sort - the tables; we instead emit a warning and go with the unsorted list. - If an environment needs both ordered DROPs *and* has foreign key - cycles, then the warning notes they will need to restore the - ``use_alter`` flag to their :class:`_schema.ForeignKey` and - :class:`_schema.ForeignKeyConstraint` objects so that just those objects will - be omitted from the dependency sort. - - .. seealso:: - - :ref:`feature_3282` - contains an updated note about SQLite. - - .. change:: - :tags: bug, sql - :tickets: 3372 - - Fixed issue where a straight SELECT EXISTS query would fail to - assign the proper result type of Boolean to the result mapping, and - instead would leak column types from within the query into the - result map. This issue exists in 0.9 and earlier as well, however - has less of an impact in those versions. In 1.0, due to :ticket:`918` - this becomes a regression in that we now rely upon the result mapping - to be very accurate, else we can assign result-type processors to - the wrong column. In all versions, this issue also has the effect - that a simple EXISTS will not apply the Boolean type handler, leading - to simple 1/0 values for backends without native boolean instead of - True/False. The fix includes that an EXISTS columns argument - will be anon-labeled like other column expressions; a similar fix is - implemented for pure-boolean expressions like ``not_(True())``. - - .. change:: - :tags: bug, orm - :tickets: 3374 - - Fixed issue where a query of the form - ``query(B).filter(B.a != A(id=7))`` would render the ``NEVER_SET`` - symbol, when - given a transient object. For a persistent object, it would - always use the persisted database value and not the currently - set value. Assuming autoflush is turned on, this usually would - not be apparent for persistent values, as any pending changes - would be flushed first in any case. However, this is inconsistent - vs. the logic used for the non-negated comparison, - ``query(B).filter(B.a == A(id=7))``, which does use the - current value and additionally allows comparisons to transient - objects. The comparison now uses the current value and not - the database-persisted value. - - Unlike the other ``NEVER_SET`` issues that are repaired as regressions - caused by :ticket:`3061` in this release, this particular issue is - present at least as far back as 0.8 and possibly earlier, however it - was discovered as a result of repairing the related ``NEVER_SET`` - issues. - - .. seealso:: - - :ref:`bug_3374` - - .. change:: - :tags: bug, orm - :tickets: 3371 - - Fixed unexpected use regression cause by :ticket:`3061` where - the NEVER_SET - symbol could leak into relationship-oriented queries, including - ``filter()`` and ``with_parent()`` queries. The ``None`` symbol - is returned in all cases, however many of these queries have never - been correctly supported in any case, and produce comparisons - to NULL without using the IS operator. For this reason, a warning - is also added to that subset of relationship queries that don't - currently provide for ``IS NULL``. - - .. seealso:: - - :ref:`bug_3371` - - - .. change:: - :tags: bug, orm - :tickets: 3368 - - Fixed a regression caused by :ticket:`3061` where the - NEVER_SET symbol could leak into a lazyload query, subsequent - to the flush of a pending object. This would occur typically - for a many-to-one relationship that does not use a simple - "get" strategy. The good news is that the fix improves efficiency - vs. 0.9, because we can now skip the SELECT statement entirely - when we detect NEVER_SET symbols present in the parameters; prior to - :ticket:`3061`, we couldn't discern if the None here were set or not. - - -.. changelog:: - :version: 1.0.0 - :released: April 16, 2015 - - .. change:: - :tags: bug, orm - :tickets: 3367 - - Identified an inconsistency when handling :meth:`_query.Query.join` to the - same target more than once; it implicitly dedupes only in the case of - a relationship join, and due to :ticket:`3233`, in 1.0 a join - to the same table twice behaves differently than 0.9 in that it no - longer erroneously aliases. To help document this change, - the verbiage regarding :ticket:`3233` in the migration notes has - been generalized, and a warning has been added when :meth:`_query.Query.join` - is called against the same target relationship more than once. - - .. change:: - :tags: bug, orm - :tickets: 3364 - - Made a small improvement to the heuristics of relationship when - determining remote side with semi-self-referential (e.g. two joined - inh subclasses referring to each other), non-simple join conditions - such that the parententity is taken into account and can reduce the - need for using the ``remote()`` annotation; this can restore some - cases that might have worked without the annotation prior to 0.9.4 - via :ticket:`2948`. - - .. change:: - :tags: bug, mssql - :tickets: 3360 - - Fixed a regression where the "last inserted id" mechanics would - fail to store the correct value for MSSQL on an INSERT where the - primary key value was present in the insert params before execution, - as well as in the case where an INSERT from SELECT would state the - target columns as column objects, instead of string keys. - - - .. change:: - :tags: bug, mssql - - Using the ``Binary`` constructor now present in pymssql rather than - patching one in. Pull request courtesy Ramiro Morales. - - .. change:: - :tags: bug, tests - :tickets: 3356 - - Fixed the pathing used when tests run; for sqla_nose.py and py.test, - the "./lib" prefix is again inserted at the head of sys.path but - only if sys.flags.no_user_site isn't set; this makes it act just - like the way Python puts "." in the current path by default. - For tox, we are setting the PYTHONNOUSERSITE flag now. - - .. change:: - :tags: feature, sql - :tickets: 3084 - - The topological sorting used to sort :class:`_schema.Table` objects - and available via the :attr:`_schema.MetaData.sorted_tables` collection - will now produce a **deterministic** ordering; that is, the same - ordering each time given a set of tables with particular names - and dependencies. This is to help with comparison of DDL scripts - and other use cases. The tables are sent to the topological sort - sorted by name, and the topological sort itself will process - the incoming data in an ordered fashion. Pull request - courtesy Sebastian Bank. - - .. seealso:: - - :ref:`feature_3084` - - .. change:: - :tags: feature, orm - - Added new argument :paramref:`.Query.update.update_args` which allows - kw arguments such as ``mysql_limit`` to be passed to the underlying - :class:`_expression.Update` construct. Pull request courtesy Amir Sadoughi. - -.. changelog:: - :version: 1.0.0b5 - :released: April 3, 2015 - - .. change:: - :tags: bug, orm - :tickets: 3349 - - :class:`_query.Query` doesn't support joins, subselects, or special - FROM clauses when using the :meth:`_query.Query.update` or - :meth:`_query.Query.delete` methods; instead of silently ignoring these - fields if methods like :meth:`_query.Query.join` or - :meth:`_query.Query.select_from` has been called, an error is raised. - In 0.9.10 this only emits a warning. - - .. change:: - :tags: bug, orm - - Added a list() call around a weak dictionary used within the - commit phase of the session, which without it could cause - a "dictionary changed size during iter" error if garbage collection - interacted within the process. Change was introduced by - #3139. - - .. change:: - :tags: bug, postgresql - :tickets: 3343 - - Fixed bug where updated PG index reflection as a result of - :ticket:`3184` would cause index operations to fail on PostgreSQL - versions 8.4 and earlier. The enhancements are now - disabled when using an older version of PostgreSQL. - - .. change:: - :tags: bug, sql - :tickets: 3346 - - The warning emitted by the unicode type for a non-unicode type - has been liberalized to warn for values that aren't even string - values, such as integers; previously, the updated warning system - of 1.0 made use of string formatting operations which - would raise an internal TypeError. While these cases should ideally - raise totally, some backends like SQLite and MySQL do accept them - and are potentially in use by legacy code, not to mention that they - will always pass through if unicode conversion is turned off - for the target backend. - - .. change:: - :tags: bug, orm - :tickets: 3347 - - Fixed a bug related to "nested" inner join eager loading, which - exists in 0.9 as well but is more of a regression in 1.0 due to - :ticket:`3008` which turns on "nested" by default, such that - a joined eager load that travels across sibling paths from a common - ancestor using innerjoin=True will correctly splice each "innerjoin" - sibling into the appropriate part of the join, when a series of - inner/outer joins are mixed together. - -.. changelog:: - :version: 1.0.0b4 - :released: March 29, 2015 - - .. change:: - :tags: bug, mssql, oracle, firebird, sybase - :tickets: 3338 - - Turned off the "simple order by" flag on the MSSQL, Oracle dialects; - this is the flag that per :ticket:`2992` causes an order by or group by - an expression that's also in the columns clause to be copied by - label, even if referenced as the expression object. The behavior - for MSSQL is now the old behavior that copies the whole expression - in by default, as MSSQL can be picky on these particularly in - GROUP BY expressions. The flag is also turned off defensively - for the Firebird and Sybase dialects. - - .. note:: this resolution was incorrect, please see version 1.0.2 - for a rework of this resolution. - - .. change:: - :tags: feature, schema - :tickets: 3341 - - The "auto-attach" feature of constraints such as :class:`.UniqueConstraint` - and :class:`.CheckConstraint` has been further enhanced such that - when the constraint is associated with non-table-bound :class:`_schema.Column` - objects, the constraint will set up event listeners with the - columns themselves such that the constraint auto attaches at the - same time the columns are associated with the table. This in particular - helps in some edge cases in declarative but is also of general use. - - .. seealso:: - - :ref:`change_3341` - - .. change:: - :tags: bug, sql - :tickets: 3340 - - Fixed bug in new "label resolution" feature of :ticket:`2992` where - a label that was anonymous, then labeled again with a name, would - fail to be locatable via a textual label. This situation occurs - naturally when a mapped :func:`.column_property` is given an - explicit label in a query. - - .. change:: - :tags: bug, sql - :tickets: 3335 - - Fixed bug in new "label resolution" feature of :ticket:`2992` where - the string label placed in the order_by() or group_by() of a statement - would place higher priority on the name as found - inside the FROM clause instead of a more locally available name - inside the columns clause. - -.. changelog:: - :version: 1.0.0b3 - :released: March 20, 2015 - - .. change:: - :tags: bug, mysql - :tickets: 2771 - - Repaired the commit for issue #2771 which was inadvertently commented - out. - - -.. changelog:: - :version: 1.0.0b2 - :released: March 20, 2015 - - .. change:: - :tags: bug, mysql - :tickets: 2771 - - Fixes to fully support using the ``'utf8mb4'`` MySQL-specific charset - with MySQL dialects, in particular MySQL-Python and PyMySQL. In - addition, MySQL databases that report more unusual charsets such as - 'koi8u' or 'eucjpms' will also work correctly. Pull request - courtesy Thomas Grainger. - - .. change:: - :tags: change, orm, declarative - :tickets: 3331 - - Loosened some restrictions that were added to ``@declared_attr`` - objects, such that they were prevented from being called outside - of the declarative process; this is related to the enhancements - of #3150 which allow ``@declared_attr`` to return a value that is - cached based on the current class as it's being configured. - The exception raise has been removed, and the behavior changed - so that outside of the declarative process, the function decorated by - ``@declared_attr`` is called every time just like a regular - ``@property``, without using any caching, as none is available - at this stage. - - .. change:: - :tags: bug, engine - :tickets: 3330, 3329 - - The "auto close" for :class:`_engine.ResultProxy` is now a "soft" close. - That is, after exhausting all rows using the fetch methods, the - DBAPI cursor is released as before and the object may be safely - discarded, but the fetch methods may continue to be called for which - they will return an end-of-result object (None for fetchone, empty list - for fetchmany and fetchall). Only if :meth:`_engine.ResultProxy.close` - is called explicitly will these methods raise the "result is closed" - error. - - .. seealso:: - - :ref:`change_3330` - - .. change:: - :tags: bug, orm - :tickets: 3327 - - Fixed unexpected use regression from pullreq github:137 where - Py2K unicode literals (e.g. ``u""``) would not be accepted by the - :paramref:`_orm.relationship.cascade` option. - Pull request courtesy Julien Castets. - - -.. changelog:: - :version: 1.0.0b1 - :released: March 13, 2015 - - Version 1.0.0b1 is the first release of the 1.0 series. Many changes - described here are also present in the 0.9 and sometimes the 0.8 - series as well. For changes that are specific to 1.0 with an emphasis - on compatibility concerns, see :doc:`/changelog/migration_10`. - - .. change:: - :tags: feature, ext - :tickets: 3054 - - Added a new extension suite :mod:`sqlalchemy.ext.baked`. This - simple but unusual system allows for a dramatic savings in Python - overhead for the construction and processing of orm :class:`_query.Query` - objects, from query construction up through rendering of a string - SQL statement. - - .. seealso:: - - :ref:`baked_toplevel` - - .. change:: - :tags: bug, postgresql - :tickets: 3319 - - The PostgreSQL :class:`_postgresql.ENUM` type will emit a - DROP TYPE instruction when a plain ``table.drop()`` is called, - assuming the object is not associated directly with a - :class:`_schema.MetaData` object. In order to accommodate the use case of - an enumerated type shared between multiple tables, the type should - be associated directly with the :class:`_schema.MetaData` object; in this - case the type will only be created at the metadata level, or if - created directly. The rules for create/drop of - PostgreSQL enumerated types have been highly reworked in general. - - .. seealso:: - - :ref:`change_3319` - - .. change:: - :tags: feature, orm - :tickets: 3317 - - Added a new event suite :class:`.QueryEvents`. The - :meth:`.QueryEvents.before_compile` event allows the creation - of functions which may place additional modifications to - :class:`_query.Query` objects before the construction of the SELECT - statement. It is hoped that this event be made much more - useful via the advent of a new inspection system that will - allow for detailed modifications to be made against - :class:`_query.Query` objects in an automated fashion. - - .. seealso:: - - :class:`.QueryEvents` - - - .. change:: - :tags: feature, orm - :tickets: 3249 - - The subquery wrapping which occurs when joined eager loading - is used with a one-to-many query that also features LIMIT, - OFFSET, or DISTINCT has been disabled in the case of a one-to-one - relationship, that is a one-to-many with - :paramref:`_orm.relationship.uselist` set to False. This will produce - more efficient queries in these cases. - - .. seealso:: - - :ref:`change_3249` - - - .. change:: - :tags: bug, orm - :tickets: 3301 - - Fixed bug where the session attachment error "object is already - attached to session X" would fail to prevent the object from - also being attached to the new session, in the case that execution - continued after the error raise occurred. - - .. change:: - :tags: bug, orm, declarative - :tickets: 3219, 3240 - - Fixed bug where using an ``__abstract__`` mixin in the middle - of a declarative inheritance hierarchy would prevent attributes - and configuration being correctly propagated from the base class - to the inheriting class. - - .. change:: - :tags: feature, sql - :tickets: 918 - - The SQL compiler now generates the mapping of expected columns - such that they are matched to the received result set positionally, - rather than by name. Originally, this was seen as a way to handle - cases where we had columns returned with difficult-to-predict names, - though in modern use that issue has been overcome by anonymous - labeling. In this version, the approach basically reduces function - call count per-result by a few dozen calls, or more for larger - sets of result columns. The approach still degrades into a modern - version of the old approach if any discrepancy in size exists between - the compiled set of columns versus what was received, so there's no - issue for partially or fully textual compilation scenarios where these - lists might not line up. - - .. change:: - :tags: feature, postgresql - - The PG8000 dialect now supports the - :paramref:`_sa.create_engine.encoding` parameter, by setting up - the client encoding on the connection which is then intercepted - by pg8000. Pull request courtesy Tony Locke. - - .. change:: - :tags: feature, postgresql - - Added support for PG8000's native JSONB feature. Pull request - courtesy Tony Locke. - - .. change:: - :tags: change, orm - - Mapped attributes marked as deferred without explicit undeferral - will now remain "deferred" even if their column is otherwise - present in the result set in some way. This is a performance - enhancement in that an ORM load no longer spends time searching - for each deferred column when the result set is obtained. However, - for an application that has been relying upon this, an explicit - :func:`.undefer` or similar option should now be used. - - .. change:: - :tags: feature, orm - :tickets: 3307 - - Mapped state internals have been reworked to allow for a 50% reduction - in callcounts specific to the "expiration" of objects, as in - the "auto expire" feature of :meth:`.Session.commit` and - for :meth:`.Session.expire_all`, as well as in the "cleanup" step - which occurs when object states are garbage collected. - - .. change:: - :tags: bug, mysql - - The MySQL dialect now supports CAST on types that are constructed - as :class:`.TypeDecorator` objects. - - .. change:: - :tags: bug, mysql - :tickets: 3237 - - A warning is emitted when :func:`.cast` is used with the MySQL - dialect on a type where MySQL does not support CAST; MySQL only - supports CAST on a subset of datatypes. SQLAlchemy has for a long - time just omitted the CAST for unsupported types in the case of - MySQL. While we don't want to change this now, we emit a warning - to show that it's taken place. A warning is also emitted when - a CAST is used with an older MySQL version (< 4) that doesn't support - CAST at all, it's skipped in this case as well. - - .. change:: - :tags: feature, sql - :tickets: 3087 - - Literal values within a :class:`.DefaultClause`, which is invoked - when using the :paramref:`_schema.Column.server_default` parameter, will - now be rendered using the "inline" compiler, so that they are rendered - as-is, rather than as bound parameters. - - .. seealso:: - - :ref:`change_3087` - - .. change:: - :tags: feature, oracle - - Added support for cx_oracle connections to a specific service - name, as opposed to a tns name, by passing ``?service_name=`` - to the URL. Pull request courtesy Sławomir Ehlert. - - .. change:: - :tags: feature, mysql - :tickets: 3155 - - The MySQL dialect now renders TIMESTAMP with NULL / NOT NULL in - all cases, so that MySQL 5.6.6 with the - ``explicit_defaults_for_timestamp`` flag enabled will - will allow TIMESTAMP to continue to work as expected when - ``nullable=False``. Existing applications are unaffected as - SQLAlchemy has always emitted NULL for a TIMESTAMP column that - is ``nullable=True``. - - .. seealso:: - - :ref:`change_3155` - - :ref:`mysql_timestamp_null` - - .. change:: - :tags: bug, schema - :tickets: 3299, 3067 - - The :class:`.CheckConstraint` construct now supports naming - conventions that include the token ``%(column_0_name)s``; the - constraint expression is scanned for columns. Additionally, - naming conventions for check constraints that don't include the - ``%(constraint_name)s`` token will now work for :class:`.SchemaType`- - generated constraints, such as those of :class:`.Boolean` and - :class:`.Enum`; this stopped working in 0.9.7 due to :ticket:`3067`. - - .. seealso:: - - :ref:`naming_check_constraints` - - :ref:`naming_schematypes` - - - .. change:: - :tags: feature, postgresql, pypy - :tickets: 3052 - - Added support for the psycopg2cffi DBAPI on pypy. Pull request - courtesy shauns. - - .. seealso:: - - :mod:`sqlalchemy.dialects.postgresql.psycopg2cffi` - - .. change:: - :tags: feature, orm - :tickets: 3262 - - A warning is emitted when the same polymorphic identity is assigned - to two different mappers in the same hierarchy. This is typically a - user error and means that the two different mapping types cannot be - correctly distinguished at load time. Pull request courtesy - Sebastian Bank. - - .. change:: - :tags: feature, sql - - The type of expression is reported when an object passed to a - SQL expression unit can't be interpreted as a SQL fragment; - pull request courtesy Ryan P. Kelly. - - .. change:: - :tags: bug, orm - :tickets: 3227, 3242, 1326 - - The primary :class:`_orm.Mapper` of a :class:`_query.Query` is now passed to the - :meth:`.Session.get_bind` method when calling upon - :meth:`_query.Query.count`, :meth:`_query.Query.update`, :meth:`_query.Query.delete`, - as well as queries against mapped columns, - :obj:`.column_property` objects, and SQL functions and expressions - derived from mapped columns. This allows sessions that rely upon - either customized :meth:`.Session.get_bind` schemes or "bound" metadata - to work in all relevant cases. - - .. seealso:: - - :ref:`bug_3227` - - .. change:: - :tags: enhancement, sql - :tickets: 3074 - - Custom dialects that implement :class:`.GenericTypeCompiler` can - now be constructed such that the visit methods receive an indication - of the owning expression object, if any. Any visit method that - accepts keyword arguments (e.g. ``**kw``) will in most cases - receive a keyword argument ``type_expression``, referring to the - expression object that the type is contained within. For columns - in DDL, the dialect's compiler class may need to alter its - ``get_column_specification()`` method to support this as well. - The ``UserDefinedType.get_col_spec()`` method will also receive - ``type_expression`` if it provides ``**kw`` in its argument - signature. - - .. change:: - :tags: bug, sql - :tickets: 3288 - - The multi-values version of :meth:`_expression.Insert.values` has been - repaired to work more usefully with tables that have Python- - side default values and/or functions, as well as server-side - defaults. The feature will now work with a dialect that uses - "positional" parameters; a Python callable will also be - invoked individually for each row just as is the case with an - "executemany" style invocation; a server- side default column - will no longer implicitly receive the value explicitly - specified for the first row, instead refusing to invoke - without an explicit value. - - .. seealso:: - - :ref:`bug_3288` - - .. change:: - :tags: feature, general - - Structural memory use has been improved via much more significant use - of ``__slots__`` for many internal objects. This optimization is - particularly geared towards the base memory size of large applications - that have lots of tables and columns, and greatly reduces memory - size for a variety of high-volume objects including event listening - internals, comparator objects and parts of the ORM attribute and - loader strategy system. - - .. seealso:: - - :ref:`feature_slots` - - .. change:: - :tags: bug, mysql - :tickets: 3283 - - The :class:`.mysql.SET` type has been overhauled to no longer - assume that the empty string, or a set with a single empty string - value, is in fact a set with a single empty string; instead, this - is by default treated as the empty set. In order to handle persistence - of a :class:`.mysql.SET` that actually wants to include the blank - value ``''`` as a legitimate value, a new bitwise operational mode - is added which is enabled by the - :paramref:`.mysql.SET.retrieve_as_bitwise` flag, which will persist - and retrieve values unambiguously using their bitflag positioning. - Storage and retrieval of unicode values for driver configurations - that aren't converting unicode natively is also repaired. - - .. seealso:: - - :ref:`change_3283` - - - .. change:: - :tags: feature, schema - :tickets: 3282 - - The DDL generation system of :meth:`_schema.MetaData.create_all` - and :meth:`_schema.MetaData.drop_all` has been enhanced to in most - cases automatically handle the case of mutually dependent - foreign key constraints; the need for the - :paramref:`_schema.ForeignKeyConstraint.use_alter` flag is greatly - reduced. The system also works for constraints which aren't given - a name up front; only in the case of DROP is a name required for - at least one of the constraints involved in the cycle. - - .. seealso:: - - :ref:`feature_3282` - - .. change:: - :tags: feature, schema - - Added a new accessor :attr:`_schema.Table.foreign_key_constraints` - to complement the :attr:`_schema.Table.foreign_keys` collection, - as well as :attr:`_schema.ForeignKeyConstraint.referred_table`. - - .. change:: - :tags: bug, sqlite - :tickets: 3244, 3261 - - UNIQUE and FOREIGN KEY constraints are now fully reflected on - SQLite both with and without names. Previously, foreign key - names were ignored and unnamed unique constraints were skipped. - Thanks to Jon Nelson for assistance with this. - - .. change:: - :tags: feature, examples - - A new suite of examples dedicated to providing a detailed study - into performance of SQLAlchemy ORM and Core, as well as the DBAPI, - from multiple perspectives. The suite runs within a container - that provides built in profiling displays both through console - output as well as graphically via the RunSnake tool. - - .. seealso:: - - :ref:`examples_performance` - - .. change:: - :tags: feature, orm - :tickets: 3100 - - A new series of :class:`.Session` methods which provide hooks - directly into the unit of work's facility for emitting INSERT - and UPDATE statements has been created. When used correctly, - this expert-oriented system can allow ORM-mappings to be used - to generate bulk insert and update statements batched into - executemany groups, allowing the statements to proceed at - speeds that rival direct use of the Core. - - .. seealso:: - - :ref:`bulk_operations` - - .. change:: - :tags: feature, mssql - :tickets: 3039 - - SQL Server 2012 now recommends VARCHAR(max), NVARCHAR(max), - VARBINARY(max) for large text/binary types. The MSSQL dialect will - now respect this based on version detection, as well as the new - ``deprecate_large_types`` flag. - - .. seealso:: - - :ref:`mssql_large_type_deprecation` - - .. change:: - :tags: bug, sqlite - :tickets: 3257 - - The SQLite dialect, when using the :class:`_sqlite.DATE`, - :class:`_sqlite.TIME`, - or :class:`_sqlite.DATETIME` types, and given a ``storage_format`` that - only renders numbers, will render the types in DDL as - ``DATE_CHAR``, ``TIME_CHAR``, and ``DATETIME_CHAR``, so that despite the - lack of alpha characters in the values, the column will still - deliver the "text affinity". Normally this is not needed, as the - textual values within the default storage formats already - imply text. - - .. seealso:: - - :ref:`sqlite_datetime` - - .. change:: - :tags: bug, engine - :tickets: 3266 - - The engine-level error handling and wrapping routines will now - take effect in all engine connection use cases, including - when user-custom connect routines are used via the - :paramref:`_sa.create_engine.creator` parameter, as well as when - the :class:`_engine.Connection` encounters a connection error on - revalidation. - - .. seealso:: - - :ref:`change_3266` - - .. change:: - :tags: feature, oracle - - New Oracle DDL features for tables, indexes: COMPRESS, BITMAP. - Patch courtesy Gabor Gombas. - - .. change:: - :tags: bug, oracle - - An alias name will be properly quoted when referred to using the - ``%(name)s`` token inside the :meth:`_expression.Select.with_hint` method. - Previously, the Oracle backend hadn't implemented this quoting. - - .. change:: - :tags: feature, oracle - :tickets: 3220 - - Added support for CTEs under Oracle. This includes some tweaks - to the aliasing syntax, as well as a new CTE feature - :meth:`_expression.CTE.suffix_with`, which is useful for adding in special - Oracle-specific directives to the CTE. - - .. seealso:: - - :ref:`change_3220` - - .. change:: - :tags: feature, mysql - :tickets: 3121 - - Updated the "supports_unicode_statements" flag to True for MySQLdb - and Pymysql under Python 2. This refers to the SQL statements - themselves, not the parameters, and affects issues such as table - and column names using non-ASCII characters. These drivers both - appear to support Python 2 Unicode objects without issue in modern - versions. - - .. change:: - :tags: bug, mysql - :tickets: 3263 - - The :meth:`.ColumnOperators.match` operator is now handled such that the - return type is not strictly assumed to be boolean; it now - returns a :class:`.Boolean` subclass called :class:`.MatchType`. - The type will still produce boolean behavior when used in Python - expressions, however the dialect can override its behavior at - result time. In the case of MySQL, while the MATCH operator - is typically used in a boolean context within an expression, - if one actually queries for the value of a match expression, a - floating point value is returned; this value is not compatible - with SQLAlchemy's C-based boolean processor, so MySQL's result-set - behavior now follows that of the :class:`.Float` type. - A new operator object ``notmatch_op`` is also added to better allow - dialects to define the negation of a match operation. - - .. seealso:: - - :ref:`change_3263` - - .. change:: - :tags: bug, postgresql - :tickets: 3264 - - The :meth:`.PGDialect.has_table` method will now query against - ``pg_catalog.pg_table_is_visible(c.oid)``, rather than testing - for an exact schema match, when the schema name is None; this - so that the method will also illustrate that temporary tables - are present. Note that this is a behavioral change, as PostgreSQL - allows a non-temporary table to silently overwrite an existing - temporary table of the same name, so this changes the behavior - of ``checkfirst`` in that unusual scenario. - - .. seealso:: - - :ref:`change_3264` - - .. change:: - :tags: bug, sql - :tickets: 3260 - - Fixed bug in :meth:`_schema.Table.tometadata` method where the - :class:`.CheckConstraint` associated with a :class:`.Boolean` - or :class:`.Enum` type object would be doubled in the target table. - The copy process now tracks the production of this constraint object - as local to a type object. - - .. change:: - :tags: feature, orm - :tickets: 3217 - - Added a parameter :paramref:`.Query.join.isouter` which is synonymous - with calling :meth:`_query.Query.outerjoin`; this flag is to provide a more - consistent interface compared to Core :meth:`_expression.FromClause.join`. - Pull request courtesy Jonathan Vanasco. - - .. change:: - :tags: bug, sql - :tickets: 3243 - - The behavioral contract of the :attr:`_schema.ForeignKeyConstraint.columns` - collection has been made consistent; this attribute is now a - :class:`_expression.ColumnCollection` like that of all other constraints and - is initialized at the point when the constraint is associated with - a :class:`_schema.Table`. - - .. seealso:: - - :ref:`change_3243` - - .. change:: - :tags: bug, orm - :tickets: 3256 - - The :meth:`.PropComparator.of_type` modifier has been - improved in conjunction with loader directives such as - :func:`_orm.joinedload` and :func:`.contains_eager` such that if - two :meth:`.PropComparator.of_type` modifiers of the same - base type/path are encountered, they will be joined together - into a single "polymorphic" entity, rather than replacing - the entity of type A with the one of type B. E.g. - a joinedload of ``A.b.of_type(BSub1)->BSub1.c`` combined with - joinedload of ``A.b.of_type(BSub2)->BSub2.c`` will create a - single joinedload of ``A.b.of_type((BSub1, BSub2)) -> BSub1.c, BSub2.c``, - without the need for the ``with_polymorphic`` to be explicit - in the query. - - .. seealso:: - - :ref:`eagerloading_polymorphic_subtypes` - contains an updated - example illustrating the new format. - - .. change:: - :tags: bug, sql - :tickets: 3245 - - The :attr:`_schema.Column.key` attribute is now used as the source of - anonymous bound parameter names within expressions, to match the - existing use of this value as the key when rendered in an INSERT - or UPDATE statement. This allows :attr:`_schema.Column.key` to be used - as a "substitute" string to work around a difficult column name - that doesn't translate well into a bound parameter name. Note that - the paramstyle is configurable on :func:`_sa.create_engine` in any case, - and most DBAPIs today support a named and positional style. - - .. change:: - :tags: bug, sql - - Fixed the name of the :paramref:`.PoolEvents.reset.dbapi_connection` - parameter as passed to this event; in particular this affects - usage of the "named" argument style for this event. Pull request - courtesy Jason Goldberger. - - .. change:: - :tags: feature, sql - - Added a new parameter :paramref:`.Table.tometadata.name` to - the :meth:`_schema.Table.tometadata` method. Similar to - :paramref:`.Table.tometadata.schema`, this argument causes the newly - copied :class:`_schema.Table` to take on the new name instead of - the existing one. An interesting capability this adds is that of - copying a :class:`_schema.Table` object to the *same* :class:`_schema.MetaData` - target with a new name. Pull request courtesy n.d. parker. - - .. change:: - :tags: bug, orm - - Repaired support of the ``copy.deepcopy()`` call when used by the - :class:`.orm.util.CascadeOptions` argument, which occurs - if ``copy.deepcopy()`` is being used with :func:`_orm.relationship` - (not an officially supported use case). Pull request courtesy - duesenfranz. - - .. change:: - :tags: bug, sql - :tickets: 3170 - - Reversing a change that was made in 0.9, the "singleton" nature - of the "constants" :func:`.null`, :func:`.true`, and :func:`.false` - has been reverted. These functions returning a "singleton" object - had the effect that different instances would be treated as the - same regardless of lexical use, which in particular would impact - the rendering of the columns clause of a SELECT statement. - - .. seealso:: - - :ref:`bug_3170` - - .. change:: - :tags: bug, orm - :tickets: 3139 - - Fixed bug where :meth:`.Session.expunge` would not fully detach - the given object if the object had been subject to a delete - operation that was flushed, but not committed. This would also - affect related operations like :func:`.make_transient`. - - .. seealso:: - - :ref:`bug_3139` - - .. change:: - :tags: bug, orm - :tickets: 3230 - - A warning is emitted in the case of multiple relationships that - ultimately will populate a foreign key column in conflict with - another, where the relationships are attempting to copy values - from different source columns. This occurs in the case where - composite foreign keys with overlapping columns are mapped to - relationships that each refer to a different referenced column. - A new documentation section illustrates the example as well as how - to overcome the issue by specifying "foreign" columns specifically - on a per-relationship basis. - - .. seealso:: - - :ref:`relationship_overlapping_foreignkeys` - - .. change:: - :tags: feature, sql - :tickets: 3172 - - Exception messages have been spiffed up a bit. The SQL statement - and parameters are not displayed if None, reducing confusion for - error messages that weren't related to a statement. The full - module and classname for the DBAPI-level exception is displayed, - making it clear that this is a wrapped DBAPI exception. The - statement and parameters themselves are bounded within a bracketed - sections to better isolate them from the error message and from - each other. - - .. change:: - :tags: bug, orm - :tickets: 3228 - - The :meth:`_query.Query.update` method will now convert string key - names in the given dictionary of values into mapped attribute names - against the mapped class being updated. Previously, string names - were taken in directly and passed to the core update statement without - any means to resolve against the mapped entity. Support for synonyms - and hybrid attributes as the subject attributes of - :meth:`_query.Query.update` are also supported. - - .. seealso:: - - :ref:`bug_3228` - - .. change:: - :tags: bug, orm - :tickets: 3035 - - Improvements to the mechanism used by :class:`.Session` to locate - "binds" (e.g. engines to use), such engines can be associated with - mixin classes, concrete subclasses, as well as a wider variety - of table metadata such as joined inheritance tables. - - .. seealso:: - - :ref:`bug_3035` - - .. change:: - :tags: bug, general - :tickets: 3218 - - The ``__module__`` attribute is now set for all those SQL and - ORM functions that are derived as "public factory" symbols, which - should assist with documentation tools being able to report on the - target module. - - .. change:: - :tags: feature, sql - - :meth:`_expression.Insert.from_select` now includes Python and SQL-expression - defaults if otherwise unspecified; the limitation where non- - server column defaults aren't included in an INSERT FROM - SELECT is now lifted and these expressions are rendered as - constants into the SELECT statement. - - .. seealso:: - - :ref:`feature_insert_from_select_defaults` - - .. change:: - :tags: bug, orm - :tickets: 3233 - - Fixed bug in single table inheritance where a chain of joins - that included the same single inh entity more than once - (normally this should raise an error) could, in some cases - depending on what was being joined "from", implicitly alias the - second case of the single inh entity, producing - a query that "worked". But as this implicit aliasing is not - intended in the case of single table inheritance, it didn't - really "work" fully and was very misleading, since it wouldn't - always appear. - - .. seealso:: - - :ref:`bug_3233` - - - .. change:: - :tags: bug, orm - :tickets: 3222 - - The ON clause rendered when using :meth:`_query.Query.join`, - :meth:`_query.Query.outerjoin`, or the standalone :func:`_orm.join` / - :func:`_orm.outerjoin` functions to a single-inheritance subclass will - now include the "single table criteria" in the ON clause even - if the ON clause is otherwise hand-rolled; it is now added to the - criteria using AND, the same way as if joining to a single-table - target using relationship or similar. - - This is sort of in-between feature and bug. - - .. seealso:: - - :ref:`migration_3222` - - .. change:: - :tags: feature, sql - :tickets: 3184 - - The :class:`.UniqueConstraint` construct is now included when - reflecting a :class:`_schema.Table` object, for databases where this - is applicable. In order to achieve this - with sufficient accuracy, MySQL and PostgreSQL now contain features - that correct for the duplication of indexes and unique constraints - when reflecting tables, indexes, and constraints. - In the case of MySQL, there is not actually a "unique constraint" - concept independent of a "unique index", so for this backend - :class:`.UniqueConstraint` continues to remain non-present for a - reflected :class:`_schema.Table`. For PostgreSQL, the query used to - detect indexes against ``pg_index`` has been improved to check for - the same construct in ``pg_constraint``, and the implicitly - constructed unique index is not included with a - reflected :class:`_schema.Table`. - - In both cases, the :meth:`_reflection.Inspector.get_indexes` and the - :meth:`_reflection.Inspector.get_unique_constraints` methods return both - constructs individually, but include a new token - ``duplicates_constraint`` in the case of PostgreSQL or - ``duplicates_index`` in the case - of MySQL to indicate when this condition is detected. - Pull request courtesy Johannes Erdfelt. - - .. seealso:: - - :ref:`feature_3184` - - .. change:: - :tags: feature, postgresql - - Added support for the FILTER keyword as applied to aggregate - functions, supported by PostgreSQL 9.4. Pull request - courtesy Ilja Everilä. - - .. seealso:: - - :ref:`feature_gh134` - - .. change:: - :tags: bug, sql, engine - :tickets: 3215 - - Fixed bug where a "branched" connection, that is the kind you get - when you call :meth:`_engine.Connection.connect`, would not share invalidation - status with the parent. The architecture of branching has been tweaked - a bit so that the branched connection defers to the parent for - all invalidation status and operations. - - .. change:: - :tags: bug, sql, engine - :tickets: 3190 - - Fixed bug where a "branched" connection, that is the kind you get - when you call :meth:`_engine.Connection.connect`, would not share transaction - status with the parent. The architecture of branching has been tweaked - a bit so that the branched connection defers to the parent for - all transactional status and operations. - - .. change:: - :tags: bug, orm, declarative - :tickets: 2670 - - A relationship set up with :class:`.declared_attr` on - a :class:`.AbstractConcreteBase` base class will now be configured - on the abstract base mapping automatically, in addition to being - set up on descendant concrete classes as usual. - - .. seealso:: - - :ref:`feature_3150` - - .. change:: - :tags: feature, orm, declarative - :tickets: 3150 - - The :class:`.declared_attr` construct has newly improved - behaviors and features in conjunction with declarative. The - decorated function will now have access to the final column - copies present on the local mixin when invoked, and will also - be invoked exactly once for each mapped class, the returned result - being memoized. A new modifier :attr:`.declared_attr.cascading` - is added as well. - - .. seealso:: - - :ref:`feature_3150` - - .. change:: - :tags: feature, ext - :tickets: 3210 - - The :mod:`sqlalchemy.ext.automap` extension will now set - ``cascade="all, delete-orphan"`` automatically on a one-to-many - relationship/backref where the foreign key is detected as containing - one or more non-nullable columns. This argument is present in the - keywords passed to :func:`.automap.generate_relationship` in this - case and can still be overridden. Additionally, if the - :class:`_schema.ForeignKeyConstraint` specifies ``ondelete="CASCADE"`` - for a non-nullable or ``ondelete="SET NULL"`` for a nullable set - of columns, the argument ``passive_deletes=True`` is also added to the - relationship. Note that not all backends support reflection of - ondelete, but backends that do include PostgreSQL and MySQL. - - .. change:: - :tags: feature, sql - :tickets: 3206 - - Added new method :meth:`_expression.Select.with_statement_hint` and ORM - method :meth:`_query.Query.with_statement_hint` to support statement-level - hints that are not specific to a table. - - .. change:: - :tags: bug, sqlite - :tickets: 3203 - - SQLite now supports reflection of unique constraints from - temp tables; previously, this would fail with a TypeError. - Pull request courtesy Johannes Erdfelt. - - .. seealso:: - - :ref:`change_3204` - changes regarding SQLite temporary - table and view reflection. - - .. change:: - :tags: bug, sqlite - :tickets: 3204 - - Added :meth:`_reflection.Inspector.get_temp_table_names` and - :meth:`_reflection.Inspector.get_temp_view_names`; currently, only the - SQLite and Oracle dialects support these methods. The return of - temporary table and view names has been **removed** from SQLite and - Oracle's version of :meth:`_reflection.Inspector.get_table_names` and - :meth:`_reflection.Inspector.get_view_names`; other database backends cannot - support this information (such as MySQL), and the scope of operation - is different in that the tables can be local to a session and - typically aren't supported in remote schemas. - - .. seealso:: - - :ref:`change_3204` - - .. change:: - :tags: feature, postgresql - :tickets: 2891 - - Support has been added for reflection of materialized views - and foreign tables, as well as support for materialized views - within :meth:`_reflection.Inspector.get_view_names`, and a new method - :meth:`.PGInspector.get_foreign_table_names` available on the - PostgreSQL version of :class:`_reflection.Inspector`. Pull request courtesy - Rodrigo Menezes. - - .. seealso:: - - :ref:`feature_2891` - - - .. change:: - :tags: feature, orm - - Added new event handlers :meth:`.AttributeEvents.init_collection` - and :meth:`.AttributeEvents.dispose_collection`, which track when - a collection is first associated with an instance and when it is - replaced. These handlers supersede the :meth:`.collection.linker` - annotation. The old hook remains supported through an event adapter. - - .. change:: - :tags: bug, orm - :tickets: 3148, 3188 - - A major rework to the behavior of expression labels, most - specifically when used with ColumnProperty constructs with - custom SQL expressions and in conjunction with the "order by - labels" logic first introduced in 0.9. Fixes include that an - ``order_by(Entity.some_col_prop)`` will now make use of "order by - label" rules even if Entity has been subject to aliasing, - either via inheritance rendering or via the use of the - ``aliased()`` construct; rendering of the same column property - multiple times with aliasing (e.g. ``query(Entity.some_prop, - entity_alias.some_prop)``) will label each occurrence of the - entity with a distinct label, and additionally "order by - label" rules will work for both (e.g. - ``order_by(Entity.some_prop, entity_alias.some_prop)``). - Additional issues that could prevent the "order by label" - logic from working in 0.9, most notably that the state of a - Label could change such that "order by label" would stop - working depending on how things were called, has been fixed. - - .. seealso:: - - :ref:`bug_3188` - - - .. change:: - :tags: bug, mysql - :tickets: 3186 - - MySQL boolean symbols "true", "false" work again. 0.9's change - in :ticket:`2682` disallowed the MySQL dialect from making use of the - "true" and "false" symbols in the context of "IS" / "IS NOT", but - MySQL supports this syntax even though it has no boolean type. - MySQL remains "non native boolean", but the :func:`.true` - and :func:`.false` symbols again produce the - keywords "true" and "false", so that an expression like - ``column.is_(true())`` again works on MySQL. - - .. seealso:: - - :ref:`bug_3186` - - .. change:: - :tags: changed, mssql - :tickets: 3182 - - The hostname-based connection format for SQL Server when using - pyodbc will no longer specify a default "driver name", and a warning - is emitted if this is missing. The optimal driver name for SQL Server - changes frequently and is per-platform, so hostname based connections - need to specify this. DSN-based connections are preferred. - - .. seealso:: - - :ref:`change_3182` - - .. change:: - :tags: changed, sql - - The :func:`_expression.column` and :func:`_expression.table` - constructs are now importable from the "from sqlalchemy" namespace, - just like every other Core construct. - - .. change:: - :tags: changed, sql - :tickets: 2992 - - The implicit conversion of strings to :func:`_expression.text` constructs - when passed to most builder methods of :func:`_expression.select` as - well as :class:`_query.Query` now emits a warning with just the - plain string sent. The textual conversion still proceeds normally, - however. The only method that accepts a string without a warning - are the "label reference" methods like order_by(), group_by(); - these functions will now at compile time attempt to resolve a single - string argument to a column or label expression present in the - selectable; if none is located, the expression still renders, but - you get the warning again. The rationale here is that the implicit - conversion from string to text is more unexpected than not these days, - and it is better that the user send more direction to the Core / ORM - when passing a raw string as to what direction should be taken. - Core/ORM tutorials have been updated to go more in depth as to how text - is handled. - - .. seealso:: - - :ref:`migration_2992` - - - .. change:: - :tags: feature, engine - :tickets: 3178 - - A new style of warning can be emitted which will "filter" up to - N occurrences of a parameterized string. This allows parameterized - warnings that can refer to their arguments to be delivered a fixed - number of times until allowing Python warning filters to squelch them, - and prevents memory from growing unbounded within Python's - warning registries. - - .. seealso:: - - :ref:`feature_3178` - - .. change:: - :tags: feature, orm - - The :class:`_query.Query` will raise an exception when :meth:`_query.Query.yield_per` - is used with mappings or options where either - subquery eager loading, or joined eager loading with collections, - would take place. These loading strategies are - not currently compatible with yield_per, so by raising this error, - the method is safer to use. Eager loads can be disabled with - the ``lazyload('*')`` option or :meth:`_query.Query.enable_eagerloads`. - - .. seealso:: - - :ref:`migration_yield_per_eager_loading` - - .. change:: - :tags: bug, orm - :tickets: 3177 - - Changed the approach by which the "single inheritance criterion" - is applied, when using :meth:`_query.Query.from_self`, or its common - user :meth:`_query.Query.count`. The criteria to limit rows to those - with a certain type is now indicated on the inside subquery, - not the outside one, so that even if the "type" column is not - available in the columns clause, we can filter on it on the "inner" - query. - - .. seealso:: - - :ref:`migration_3177` - - .. change:: - :tags: changed, orm - - The ``proc()`` callable passed to the ``create_row_processor()`` - method of custom :class:`.Bundle` classes now accepts only a single - "row" argument. - - .. seealso:: - - :ref:`bundle_api_change` - - .. change:: - :tags: changed, orm - - Deprecated event hooks removed: ``populate_instance``, - ``create_instance``, ``translate_row``, ``append_result`` - - .. seealso:: - - :ref:`migration_deprecated_orm_events` - - .. change:: - :tags: bug, orm - :tickets: 3145 - - Made a small adjustment to the mechanics of lazy loading, - such that it has less chance of interfering with a joinload() in the - very rare circumstance that an object points to itself; in this - scenario, the object refers to itself while loading its attributes - which can cause a mixup between loaders. The use case of - "object points to itself" is not fully supported, but the fix also - removes some overhead so for now is part of testing. - - .. change:: - :tags: feature, orm - :tickets: 3176 - - A new implementation for :class:`.KeyedTuple` used by the - :class:`_query.Query` object offers dramatic speed improvements when - fetching large numbers of column-oriented rows. - - .. seealso:: - - :ref:`feature_3176` - - .. change:: - :tags: feature, orm - :tickets: 3008 - - The behavior of :paramref:`_orm.joinedload.innerjoin` as well as - :paramref:`_orm.relationship.innerjoin` is now to use "nested" - inner joins, that is, right-nested, as the default behavior when an - inner join joined eager load is chained to an outer join eager load. - - .. seealso:: - - :ref:`migration_3008` - - .. change:: - :tags: bug, orm - :tickets: 3171 - - The "resurrect" ORM event has been removed. This event hook had - no purpose since the old "mutable attribute" system was removed - in 0.8. - - .. change:: - :tags: bug, sql - :tickets: 3169 - - Using :meth:`_expression.Insert.from_select` now implies ``inline=True`` - on :func:`_expression.insert`. This helps to fix a bug where an - INSERT...FROM SELECT construct would inadvertently be compiled - as "implicit returning" on supporting backends, which would - cause breakage in the case of an INSERT that inserts zero rows - (as implicit returning expects a row), as well as arbitrary - return data in the case of an INSERT that inserts multiple - rows (e.g. only the first row of many). - A similar change is also applied to an INSERT..VALUES - with multiple parameter sets; implicit RETURNING will no longer emit - for this statement either. As both of these constructs deal - with variable numbers of rows, the - :attr:`_engine.ResultProxy.inserted_primary_key` accessor does not - apply. Previously, there was a documentation note that one - may prefer ``inline=True`` with INSERT..FROM SELECT as some databases - don't support returning and therefore can't do "implicit" returning, - but there's no reason an INSERT...FROM SELECT needs implicit returning - in any case. Regular explicit :meth:`_expression.Insert.returning` should - be used to return variable numbers of result rows if inserted - data is needed. - - .. change:: - :tags: bug, orm - :tickets: 3167 - - Fixed bug where attribute "set" events or columns with - ``@validates`` would have events triggered within the flush process, - when those columns were the targets of a "fetch and populate" - operation, such as an autoincremented primary key, a Python side - default, or a server-side default "eagerly" fetched via RETURNING. - - .. change:: - :tags: feature, oracle - - Added support for the Oracle table option ON COMMIT. - - .. change:: - :tags: feature, postgresql - :tickets: 2051 - - Added support for PG table options TABLESPACE, ON COMMIT, - WITH(OUT) OIDS, and INHERITS, when rendering DDL via - the :class:`_schema.Table` construct. Pull request courtesy - malikdiarra. - - .. seealso:: - - :ref:`postgresql_table_options` - - .. change:: - :tags: bug, orm, py3k - - The :class:`.IdentityMap` exposed from :attr:`.Session.identity_map` - now returns lists for ``items()`` and ``values()`` in Py3K. - Early porting to Py3K here had these returning iterators, when - they technically should be "iterable views"..for now, lists are OK. - - .. change:: - :tags: orm, feature - - UPDATE statements can now be batched within an ORM flush - into more performant executemany() call, similarly to how INSERT - statements can be batched; this will be invoked within flush - to the degree that subsequent UPDATE statements for the - same mapping and table involve the identical columns within the - VALUES clause, that no SET-level SQL expressions - are embedded, and that the versioning requirements for the mapping - are compatible with the backend dialect's ability to return - a correct rowcount for an executemany operation. - - .. change:: - :tags: engine, bug - :tickets: 3163 - - Removing (or adding) an event listener at the same time that the event - is being run itself, either from inside the listener or from a - concurrent thread, now raises a RuntimeError, as the collection used is - now an instance of ``collections.deque()`` and does not support changes - while being iterated. Previously, a plain Python list was used where - removal from inside the event itself would produce silent failures. - - .. change:: - :tags: orm, feature - :tickets: 2963 - - The ``info`` parameter has been added to the constructor for - :class:`.SynonymProperty` and :class:`.ComparableProperty`. - - .. change:: - :tags: sql, feature - :tickets: 2963 - - The ``info`` parameter has been added as a constructor argument - to all schema constructs including :class:`_schema.MetaData`, - :class:`.Index`, :class:`_schema.ForeignKey`, :class:`_schema.ForeignKeyConstraint`, - :class:`.UniqueConstraint`, :class:`.PrimaryKeyConstraint`, - :class:`.CheckConstraint`. - - .. change:: - :tags: orm, feature - :tickets: 2971 - - The :attr:`.InspectionAttr.info` collection is now moved down to - :class:`.InspectionAttr`, where in addition to being available - on all :class:`.MapperProperty` objects, it is also now available - on hybrid properties, association proxies, when accessed via - :attr:`_orm.Mapper.all_orm_descriptors`. - - .. change:: - :tags: sql, feature - :tickets: 3027 - - The :paramref:`_schema.Table.autoload_with` flag now implies that - :paramref:`_schema.Table.autoload` should be ``True``. Pull request - courtesy Malik Diarra. - - .. change:: - :tags: postgresql, feature - - Added new method :meth:`.PGInspector.get_enums`, when using the - inspector for PostgreSQL will provide a list of ENUM types. - Pull request courtesy Ilya Pekelny. - - .. change:: - :tags: mysql, bug - - The MySQL dialect will now disable :meth:`_events.ConnectionEvents.handle_error` - events from firing for those statements which it uses internally - to detect if a table exists or not. This is achieved using an - execution option ``skip_user_error_events`` that disables the handle - error event for the scope of that execution. In this way, user code - that rewrites exceptions doesn't need to worry about the MySQL - dialect or other dialects that occasionally need to catch - SQLAlchemy specific exceptions. - - .. change:: - :tags: mysql, bug - :tickets: 2515 - - Changed the default value of "raise_on_warnings" to False for - MySQLconnector. This was set at True for some reason. The "buffered" - flag unfortunately must stay at True as MySQLconnector does not allow - a cursor to be closed unless all results are fully fetched. - - .. change:: - :tags: bug, orm - :tickets: 3117 - - The "evaluator" for query.update()/delete() won't work with multi-table - updates, and needs to be set to `synchronize_session=False` or - `synchronize_session='fetch'`; this now raises an exception, with a - message to change the synchronize setting. - This is upgraded from a warning emitted as of 0.9.7. - - .. change:: - :tags: removed - - The Drizzle dialect has been removed from the Core; it is now - available as `sqlalchemy-drizzle `_, - an independent, third party dialect. The dialect is still based - almost entirely off of the MySQL dialect present in SQLAlchemy. - - .. seealso:: - - :ref:`change_2984` - - .. change:: - :tags: enhancement, orm - :tickets: 3061 - - Adjustment to attribute mechanics concerning when a value is - implicitly initialized to None via first access; this action, - which has always resulted in a population of the attribute, - no longer does so; the None value is returned but the underlying - attribute receives no set event. This is consistent with how collections - work and allows attribute mechanics to behave more consistently; - in particular, getting an attribute with no value does not squash - the event that should proceed if the value is actually set to None. - - .. seealso:: - - :ref:`migration_3061` - - .. change:: - :tags: feature, sql - :tickets: 3034 - - The :meth:`_expression.Select.limit` and :meth:`_expression.Select.offset` methods - now accept any SQL expression, in addition to integer values, as - arguments. Typically this is used to allow a bound parameter to be - passed, which can be substituted with a value later thus allowing - Python-side caching of the SQL query. The implementation - here is fully backwards compatible with existing third party dialects, - however those dialects which implement special LIMIT/OFFSET systems - will need modification in order to take advantage of the new - capabilities. Limit and offset also support "literal_binds" mode, - where bound parameters are rendered inline as strings based on - a compile-time option. - Work on this feature is courtesy of Dobes Vandermeer. - - - .. seealso:: - - :ref:`feature_3034`. diff --git a/doc/build/changelog/changelog_11.rst b/doc/build/changelog/changelog_11.rst deleted file mode 100644 index c84effc390..0000000000 --- a/doc/build/changelog/changelog_11.rst +++ /dev/null @@ -1,2739 +0,0 @@ -============= -1.1 Changelog -============= - -.. changelog_imports:: - - .. include:: changelog_10.rst - :start-line: 5 - - - .. include:: changelog_09.rst - :start-line: 5 - - - .. include:: changelog_08.rst - :start-line: 5 - - - .. include:: changelog_07.rst - :start-line: 5 - - -.. changelog:: - :version: 1.1.18 - :released: March 6, 2018 - - .. change:: - :tags: bug, mysql - :tickets: 4205 - :versions: 1.2.5 - - MySQL dialects now query the server version using ``SELECT @@version`` - explicitly to the server to ensure we are getting the correct version - information back. Proxy servers like MaxScale interfere with the value - that is passed to the DBAPI's connection.server_version value so this - is no longer reliable. - - .. change:: - :tags: bug, postgresql, py3k - :tickets: 4208 - :versions: 1.2.5 - - Fixed bug in PostgreSQL COLLATE / ARRAY adjustment first introduced - in :ticket:`4006` where new behaviors in Python 3.7 regular expressions - caused the fix to fail. - -.. changelog:: - :version: 1.1.17 - :released: February 22, 2018 - - .. change:: - :tags: bug, ext - :tickets: 4185 - - Repaired regression caused in 1.2.3 and 1.1.16 regarding association proxy - objects, revising the approach to :ticket:`4185` when calculating the - "owning class" of an association proxy to default to choosing the current - class if the proxy object is not directly associated with a mapped class, - such as a mixin. - -.. changelog:: - :version: 1.1.16 - :released: February 16, 2018 - - .. change:: - :tags: bug, postgresql - :versions: 1.2.3 - - Added "SSL SYSCALL error: Operation timed out" to the list - of messages that trigger a "disconnect" scenario for the - psycopg2 driver. Pull request courtesy André Cruz. - - .. change:: - :tags: bug, orm - :tickets: 4187 - :versions: 1.2.3 - - Fixed issue in post_update feature where an UPDATE is emitted - when the parent object has been deleted but the dependent object - is not. This issue has existed for a long time however - since 1.2 now asserts rows matched for post_update, this - was raising an error. - - .. change:: - :tags: bug, mysql - :tickets: 4136 - :versions: 1.2.0b4 - - Fixed bug where the MySQL "concat" and "match" operators failed to - propagate kwargs to the left and right expressions, causing compiler - options such as "literal_binds" to fail. - - .. change:: - :tags: bug, sql - :versions: 1.2.0b4 - - Added :func:`.nullsfirst` and :func:`.nullslast` as top level imports - in the ``sqlalchemy.`` and ``sqlalchemy.sql.`` namespace. Pull request - courtesy Lele Gaifax. - - .. change:: - :tags: bug, orm - :tickets: 4185 - :versions: 1.2.3 - - Fixed regression caused by fix for issue :ticket:`4116` affecting versions - 1.2.2 as well as 1.1.15, which had the effect of mis-calculation of the - "owning class" of an :class:`.AssociationProxy` as the ``NoneType`` class - in some declarative mixin/inheritance situations as well as if the - association proxy were accessed off of an un-mapped class. The "figure out - the owner" logic has been replaced by an in-depth routine that searches - through the complete mapper hierarchy assigned to the class or subclass to - determine the correct (we hope) match; will not assign the owner if no - match is found. An exception is now raised if the proxy is used - against an un-mapped instance. - - - .. change:: - :tags: bug, sql - :tickets: 4162 - :versions: 1.2.1 - - Fixed bug in :meth:`_expression.Insert.values` where using the "multi-values" - format in combination with :class:`_schema.Column` objects as keys rather - than strings would fail. Pull request courtesy Aubrey Stark-Toller. - - .. change:: - :tags: bug, postgresql - :versions: 1.2.3 - - Added "TRUNCATE" to the list of keywords accepted by the - PostgreSQL dialect as an "autocommit"-triggering keyword. - Pull request courtesy Jacob Hayes. - - .. change:: - :tags: bug, pool - :tickets: 4184 - :versions: 1.2.3 - - Fixed a fairly serious connection pool bug where a connection that is - acquired after being refreshed as a result of a user-defined - :class:`_exc.DisconnectionError` or due to the 1.2-released "pre_ping" feature - would not be correctly reset if the connection were returned to the pool by - weakref cleanup (e.g. the front-facing object is garbage collected); the - weakref would still refer to the previously invalidated DBAPI connection - which would have the reset operation erroneously called upon it instead. - This would lead to stack traces in the logs and a connection being checked - into the pool without being reset, which can cause locking issues. - - - .. change:: - :tags: bug, orm - :tickets: 4151 - :versions: 1.2.1 - - Fixed bug where an object that is expunged during a rollback of - a nested or subtransaction which also had its primary key mutated - would not be correctly removed from the session, causing subsequent - issues in using the session. - -.. changelog:: - :version: 1.1.15 - :released: November 3, 2017 - - .. change:: - :tags: bug, sqlite - :tickets: 4099 - :versions: 1.2.0b3 - - Fixed bug where SQLite CHECK constraint reflection would fail - if the referenced table were in a remote schema, e.g. on SQLite a - remote database referred to by ATTACH. - - .. change:: - :tags: bug, mysql - :tickets: 4097 - :versions: 1.2.0b3 - - Warning emitted when MariaDB 10.2.8 or earlier in the 10.2 - series is detected as there are major issues with CHECK - constraints within these versions that were resolved as of - 10.2.9. - - Note that this changelog message was NOT released with - SQLAlchemy 1.2.0b3 and was added retroactively. - - .. change:: - :tags: bug, mssql - :tickets: 4095 - :versions: 1.2.0b3 - - Added a full range of "connection closed" exception codes to the - PyODBC dialect for SQL Server, including '08S01', '01002', '08003', - '08007', '08S02', '08001', 'HYT00', 'HY010'. Previously, only '08S01' - was covered. - - .. change:: - :tags: bug, sql - :tickets: 4126 - :versions: 1.2.0 - - Fixed bug where ``__repr__`` of :class:`.ColumnDefault` would fail - if the argument were a tuple. Pull request courtesy Nicolas Caniart. - - .. change:: - :tags: bug, orm, declarative - :tickets: 4124 - :versions: 1.2.0 - - Fixed bug where a descriptor that is elsewhere a mapped column - or relationship within a hierarchy based on :class:`.AbstractConcreteBase` - would be referred towards during a refresh operation, causing an error - as the attribute is not mapped as a mapper property. - A similar issue can arise for other attributes like the "type" column - added by :class:`.AbstractConcreteBase` if the class fails to include - "concrete=True" in its mapper, however the check here should also - prevent that scenario from causing a problem. - - .. change:: 4006 - :tags: bug, postgresql - :tickets: 4006 - :versions: 1.2.0b3 - - Made further fixes to the :class:`_types.ARRAY` class in conjunction with - COLLATE, as the fix made in :ticket:`4006` failed to accommodate - for a multidimensional array. - - .. change:: - :tags: bug, orm, ext - :tickets: 4116 - :versions: 1.2.0 - - Fixed bug where the association proxy would inadvertently link itself - to an :class:`.AliasedClass` object if it were called first with - the :class:`.AliasedClass` as a parent, causing errors upon subsequent - usage. - - .. change:: - :tags: bug, mysql - :tickets: 4120 - :versions: 1.2.0 - - MySQL 5.7.20 now warns for use of the @tx_isolation variable; a version - check is now performed and uses @transaction_isolation instead - to prevent this warning. - - .. change:: - :tags: bug, postgresql - :tickets: 4107 - :versions: 1.2.0b3 - - Fixed bug in :obj:`_functions.array_agg` function where passing an argument - that is already of type :class:`_types.ARRAY`, such as a PostgreSQL - :obj:`_postgresql.array` construct, would produce a ``ValueError``, due - to the function attempting to nest the arrays. - - .. change:: - :tags: bug, orm - :tickets: 4078 - :versions: 1.2.0b3 - - Fixed bug where ORM relationship would warn against conflicting sync - targets (e.g. two relationships would both write to the same column) for - sibling classes in an inheritance hierarchy, where the two relationships - would never actually conflict during writes. - - .. change:: - :tags: bug, postgresql - :tickets: 4074 - :versions: 1.2.0b3 - - Fixed bug in PostgreSQL :meth:`.postgresql.dml.Insert.on_conflict_do_update` - which would prevent the insert statement from being used as a CTE, - e.g. via :meth:`_expression.Insert.cte`, within another statement. - - .. change:: - :tags: bug, orm - :tickets: 4103 - :versions: 1.2.0b3 - - Fixed bug where correlated select used against single-table inheritance - entity would fail to render correctly in the outer query, due to adjustment - for single inheritance discriminator criteria inappropriately re-applying - the criteria to the outer query. - - .. change:: - :tags: bug, mysql - :tickets: 4096 - :versions: 1.2.0b3 - - Fixed issue where CURRENT_TIMESTAMP would not reflect correctly - in the MariaDB 10.2 series due to a syntax change, where the function - is now represented as ``current_timestamp()``. - - .. change:: - :tags: bug, mysql - :tickets: 4098 - :versions: 1.2.0b3 - - MariaDB 10.2 now supports CHECK constraints (warning: use version 10.2.9 - or greater due to upstream issues noted in :ticket:`4097`). Reflection - now takes these CHECK constraints into account when they are present in - the ``SHOW CREATE TABLE`` output. - - .. change:: - :tags: bug, sql - :tickets: 4093 - :versions: 1.2.0b3 - - Fixed bug where the recently added :meth:`.ColumnOperators.any_` - and :meth:`.ColumnOperators.all_` methods didn't work when called - as methods, as opposed to using the standalone functions - :func:`_expression.any_` and :func:`_expression.all_`. Also - added documentation examples for these relatively unintuitive - SQL operators. - -.. changelog:: - :version: 1.1.14 - :released: September 5, 2017 - - .. change:: - :tags: bug, orm - :tickets: 4069 - :versions: 1.2.0b3 - - Fixed bug in :meth:`.Session.merge` following along similar lines as that - of :ticket:`4030`, where an internal check for a target object in - the identity map could lead to an error if it were to be garbage collected - immediately before the merge routine actually retrieves the object. - - .. change:: - :tags: bug, orm - :tickets: 4048 - :versions: 1.2.0b3 - - Fixed bug where an :func:`.undefer_group` option would not be recognized - if it extended from a relationship that was loading using joined eager - loading. Additionally, as the bug led to excess work being performed, - Python function call counts are also improved by 20% within the initial - calculation of result set columns, complementing the joined eager load - improvements of :ticket:`3915`. - - .. change:: - :tags: bug, orm - :tickets: 4068 - - Fixed race condition in ORM identity map which would cause objects - to be inappropriately removed during a load operation, causing - duplicate object identities to occur, particularly under joined eager - loading which involves deduplication of objects. The issue is specific - to garbage collection of weak references and is observed only under the - PyPy interpreter. - - .. change:: - :tags: bug, orm - :tickets: 4056 - :versions: 1.2.0b3 - - Fixed bug in :meth:`.Session.merge` where objects in a collection that had - the primary key attribute set to ``None`` for a key that is typically - autoincrementing would be considered to be a database-persisted key for - part of the internal deduplication process, causing only one object to - actually be inserted in the database. - - .. change:: - :tags: bug, sql - :tickets: 4053 - - Altered the range specification for window functions to allow - for two of the same PRECEDING or FOLLOWING keywords in a range - by allowing for the left side of the range to be positive - and for the right to be negative, e.g. (1, 3) is - "1 FOLLOWING AND 3 FOLLOWING". - - .. change:: - :tags: bug, orm - :tickets: 4067 - :versions: 1.2.0b3 - - An :class:`.InvalidRequestError` is raised when a :func:`.synonym` - is used against an attribute that is not against a :class:`.MapperProperty`, - such as an association proxy. Previously, a recursion overflow would - occur trying to locate non-existent attributes. - -.. changelog:: - :version: 1.1.13 - :released: August 3, 2017 - -.. changelog:: - :version: 1.1.12 - :released: July 24, 2017 - - .. change:: cache_order_sequence - :tags: feature, oracle, postgresql - :versions: 1.2.0b1 - - Added new keywords :paramref:`.Sequence.cache` and - :paramref:`.Sequence.order` to :class:`.Sequence`, to allow rendering - of the CACHE parameter understood by Oracle and PostgreSQL, and the - ORDER parameter understood by Oracle. Pull request - courtesy David Moore. - - - .. change:: 4033 - :tags: bug, orm - :tickets: 4033 - :versions: 1.2.0b2 - - Fixed regression from 1.1.11 where adding additional non-entity - columns to a query that includes an entity with subqueryload - relationships would fail, due to an inspection added in 1.1.11 as a - result of :ticket:`4011`. - - - .. change:: 4031 - :tags: bug, orm - :versions: 1.2.0b2 - :tickets: 4031 - - Fixed bug involving JSON NULL evaluation logic added in 1.1 as part - of :ticket:`3514` where the logic would not accommodate ORM - mapped attributes named differently from the :class:`_schema.Column` - that was mapped. - - .. change:: 4030 - :tags: bug, orm - :versions: 1.2.0b2 - :tickets: 4030 - - Added ``KeyError`` checks to all methods within - :class:`.WeakInstanceDict` where a check for ``key in dict`` is - followed by indexed access to that key, to guard against a race against - garbage collection that under load can remove the key from the dict - after the code assumes its present, leading to very infrequent - ``KeyError`` raises. - -.. changelog:: - :version: 1.1.11 - :released: Monday, June 19, 2017 - - .. change:: 4012 - :tags: bug, sql - :tickets: 4012 - :versions: 1.2.0b1 - - Fixed AttributeError which would occur in :class:`.WithinGroup` - construct during an iteration of the structure. - - .. change:: 4011 - :tags: bug, orm - :tickets: 4011 - :versions: 1.2.0b1 - - Fixed issue with subquery eagerloading which continues on from - the series of issues fixed in :ticket:`2699`, :ticket:`3106`, - :ticket:`3893` involving that the "subquery" contains the correct - FROM clause when beginning from a joined inheritance subclass - and then subquery eager loading onto a relationship from - the base class, while the query also includes criteria against - the subclass. The fix in the previous tickets did not accommodate - for additional subqueryload operations loading more deeply from - the first level, so the fix has been further generalized. - - .. change:: 4005 - :tags: bug, postgresql - :tickets: 4005 - :versions: 1.2.0b1 - - Continuing with the fix that correctly handles PostgreSQL - version string "10devel" released in 1.1.8, an additional regexp - bump to handle version strings of the form "10beta1". While - PostgreSQL now offers better ways to get this information, we - are sticking w/ the regexp at least through 1.1.x for the least - amount of risk to compatibility w/ older or alternate PostgreSQL - databases. - - .. change:: 4006 - :tags: bug, postgresql - :tickets: 4006 - :versions: 1.2.0b1 - - Fixed bug where using :class:`_types.ARRAY` with a string type that - features a collation would fail to produce the correct syntax - within CREATE TABLE. - - .. change:: 4007 - :tags: bug, mysql - :tickets: 4007 - :versions: 1.2.0b1 - - MySQL 5.7 has introduced permission limiting for the "SHOW VARIABLES" - command; the MySQL dialect will now handle when SHOW returns no - row, in particular for the initial fetch of SQL_MODE, and will - emit a warning that user permissions should be modified to allow the - row to be present. - - .. change:: 3994 - :tags: bug, mssql - :tickets: 3994 - :versions: 1.2.0b1 - - Fixed bug where SQL Server transaction isolation must be fetched - from a different view when using Azure data warehouse, the query - is now attempted against both views and then a NotImplemented - is raised unconditionally if failure continues to provide the - best resiliency against future arbitrary API changes in new - SQL Server versions. - - .. change:: 3997 - :tags: bug, oracle - :tickets: 3997 - :versions: 1.2.0b1 - - Support for two-phase transactions has been removed entirely for - cx_Oracle when version 6.0b1 or later of the DBAPI is in use. The two- - phase feature historically has never been usable under cx_Oracle 5.x in - any case, and cx_Oracle 6.x has removed the connection-level "twophase" - flag upon which this feature relied. - - .. change:: 3973 - :tags: bug, mssql - :tickets: 3973 - :versions: 1.2.0b1 - - Added a placeholder type :class:`_mssql.XML` to the SQL Server - dialect, so that a reflected table which includes this type can - be re-rendered as a CREATE TABLE. The type has no special round-trip - behavior nor does it currently support additional qualifying - arguments. - -.. changelog:: - :version: 1.1.10 - :released: Friday, May 19, 2017 - - .. change:: 3986 - :tags: bug, orm - :versions: 1.2.0b1 - :tickets: 3986 - - Fixed bug where a cascade such as "delete-orphan" (but others as well) - would fail to locate an object linked to a relationship that itself - is local to a subclass in an inheritance relationship, thus causing - the operation to not take place. - - .. change:: 3975 - :tags: bug, oracle - :versions: 1.2.0b1 - :tickets: 3975 - - Fixed bug in cx_Oracle dialect where version string parsing would - fail for cx_Oracle version 6.0b1 due to the "b" character. Version - string parsing is now via a regexp rather than a simple split. - - .. change:: 3949 - :tags: bug, schema - :versions: 1.2.0b1 - :tickets: 3949 - - An :class:`.ArgumentError` is now raised if a - :class:`_schema.ForeignKeyConstraint` object is created with a mismatched - number of "local" and "remote" columns, which otherwise causes the - internal state of the constraint to be incorrect. Note that this - also impacts the condition where a dialect's reflection process - produces a mismatched set of columns for a foreign key constraint. - - .. change:: 3980 - :tags: bug, ext - :versions: 1.2.0b1 - :tickets: 3980 - - Protected against testing "None" as a class in the case where - declarative classes are being garbage collected and new - automap prepare() operations are taking place concurrently, very - infrequently hitting a weakref that has not been fully acted upon - after gc. - - .. change:: - :tags: bug, postgresql - :versions: 1.2.0b1 - - Added "autocommit" support for GRANT, REVOKE keywords. Pull request - courtesy Jacob Hayes. - - .. change:: 3966 - :tags: bug, mysql - :versions: 1.2.0b1 - :tickets: 3966 - - Removed an ancient and unnecessary intercept of the UTC_TIMESTAMP - MySQL function, which was getting in the way of using it with a - parameter. - - .. change:: 3961 - :tags: bug, mysql - :versions: 1.2.0b1 - :tickets: 3961 - - Fixed bug in MySQL dialect regarding rendering of table options in - conjunction with PARTITION options when rendering CREATE TABLE. - The PARTITION related options need to follow the table options, - whereas previously this ordering was not enforced. - - -.. changelog:: - :version: 1.1.9 - :released: April 4, 2017 - - .. change:: 3956 - :tags: bug, ext - :tickets: 3956 - - Fixed regression released in 1.1.8 due to :ticket:`3950` where the - deeper search for information about column types in the case of a - "schema type" or a :class:`.TypeDecorator` would produce an attribute - error if the mapping also contained a :obj:`.column_property`. - - .. change:: 3952 - :tags: bug, sql - :versions: 1.2.0b1 - :tickets: 3952 - - Fixed regression released in 1.1.5 due to :ticket:`3859` where - adjustments to the "right-hand-side" evaluation of an expression - based on :class:`.Variant` to honor the underlying type's - "right-hand-side" rules caused the :class:`.Variant` type - to be inappropriately lost, in those cases when we *do* want the - left-hand side type to be transferred directly to the right hand side - so that bind-level rules can be applied to the expression's argument. - - .. change:: 3955 - :tags: bug, sql, postgresql - :versions: 1.2.0b1 - :tickets: 3955 - - Changed the mechanics of :class:`_engine.ResultProxy` to unconditionally - delay the "autoclose" step until the :class:`_engine.Connection` is done - with the object; in the case where PostgreSQL ON CONFLICT with - RETURNING returns no rows, autoclose was occurring in this previously - non-existent use case, causing the usual autocommit behavior that - occurs unconditionally upon INSERT/UPDATE/DELETE to fail. - -.. changelog:: - :version: 1.1.8 - :released: March 31, 2017 - - .. change:: 3950 - :tags: bug, ext - :versions: 1.2.0b1 - :tickets: 3950 - - Fixed bug in :mod:`sqlalchemy.ext.mutable` where the - :meth:`.Mutable.as_mutable` method would not track a type that had - been copied using :meth:`.TypeEngine.copy`. This became more of - a regression in 1.1 compared to 1.0 because the :class:`.TypeDecorator` - class is now a subclass of :class:`.SchemaEventTarget`, which among - other things indicates to the parent :class:`_schema.Column` that the type - should be copied when the :class:`_schema.Column` is. These copies are - common when using declarative with mixins or abstract classes. - - .. change:: - :tags: bug, ext - :versions: 1.2.0b1 - - Added support for bound parameters, e.g. those normally set up - via :meth:`_query.Query.params`, to the :meth:`.baked.Result.count` - method. Previously, support for parameters were omitted. Pull request - courtesy Pat Deegan. - - .. change:: - :tags: bug, postgresql - :versions: 1.2.0b1 - - Added support for parsing the PostgreSQL version string for - a development version like "PostgreSQL 10devel". Pull request - courtesy Sean McCully. - -.. changelog:: - :version: 1.1.7 - :released: March 27, 2017 - - .. change:: - :tags: feature, orm - :tickets: 3933 - :versions: 1.2.0b1 - - An :func:`.aliased()` construct can now be passed to the - :meth:`_query.Query.select_entity_from` method. Entities will be pulled - from the selectable represented by the :func:`.aliased` construct. - This allows special options for :func:`.aliased` such as - :paramref:`.aliased.adapt_on_names` to be used in conjunction with - :meth:`_query.Query.select_entity_from`. - - .. change:: - :tags: bug, engine - :tickets: 3946 - :versions: 1.2.0b1 - - Added an exception handler that will warn for the "cause" exception on - Py2K when the "autorollback" feature of :class:`_engine.Connection` itself - raises an exception. In Py3K, the two exceptions are naturally reported - by the interpreter as one occurring during the handling of the other. - This is continuing with the series of changes for rollback failure - handling that were last visited as part of :ticket:`2696` in 1.0.12. - - .. change:: - :tags: bug, orm - :tickets: 3947 - :versions: 1.2.0b1 - - Fixed a race condition which could occur under threaded environments - as a result of the caching added via :ticket:`3915`. An internal - collection of ``Column`` objects could be regenerated on an alias - object inappropriately, confusing a joined eager loader when it - attempts to render SQL and collect results and resulting in an - attribute error. The collection is now generated up front before - the alias object is cached and shared among threads. - - .. change:: - :tags: bug, sql, postgresql - :tickets: 2892 - - Added support for the :class:`.Variant` and the :class:`.SchemaType` - objects to be compatible with each other. That is, a variant - can be created against a type like :class:`.Enum`, and the instructions - to create constraints and/or database-specific type objects will - propagate correctly as per the variant's dialect mapping. - - .. change:: - :tags: bug, sql - :tickets: 3931 - - Fixed bug in compiler where the string identifier of a savepoint would - be cached in the identifier quoting dictionary; as these identifiers - are arbitrary, a small memory leak could occur if a single - :class:`_engine.Connection` had an unbounded number of savepoints used, - as well as if the savepoint clause constructs were used directly - with an unbounded umber of savepoint names. The memory leak does - **not** impact the vast majority of cases as normally the - :class:`_engine.Connection`, which renders savepoint names with a simple - counter starting at "1", is used on a per-transaction or - per-fixed-number-of-transactions basis before being discarded. - - .. change:: - :tags: bug, sql - :tickets: 3924 - - Fixed bug in new "schema translate" feature where the translated schema - name would be invoked in terms of an alias name when rendered along - with a column expression; occurred only when the source translate - name was "None". The "schema translate" feature now only takes - effect for :class:`.SchemaItem` and :class:`.SchemaType` subclasses, - that is, objects that correspond to a DDL-creatable structure in - a database. - -.. changelog:: - :version: 1.1.6 - :released: February 28, 2017 - - .. change:: - :tags: bug, mysql - - Added new MySQL 8.0 reserved words to the MySQL dialect for proper - quoting. Pull request courtesy Hanno Schlichting. - - .. change:: 3915 - :tags: bug, orm - :tickets: 3915 - - Addressed some long unattended performance concerns within the joined - eager loader query construction system that have accumulated since - earlier versions as a result of increased abstraction. The use of ad- - hoc :class:`.AliasedClass` objects per query, which produces lots of - column lookup overhead each time, has been replaced with a cached - approach that makes use of a small pool of :class:`.AliasedClass` - objects that are reused between invocations of joined eager loading. - Some mechanics involving eager join path construction have also been - optimized. Callcounts for an end-to-end query construction + single - row fetch test with a worst-case joined loader scenario have been - reduced by about 60% vs. 1.1.5 and 42% vs. that of 0.8.6. - - .. change:: 3804 - :tags: bug, postgresql - :tickets: 3804 - - Added regular expressions for the "IMPORT FOREIGN SCHEMA", - "REFRESH MATERIALIZED VIEW" PostgreSQL statements so that they - autocommit when invoked via a connection or engine without - an explicit transaction. Pull requests courtesy Frazer McLean - and Paweł Stiasny. - - .. change:: 3909 - :tags: bug, orm - :tickets: 3909 - - Fixed a major inefficiency in the "eager_defaults" feature whereby - an unnecessary SELECT would be emitted for column values where the - ORM had explicitly inserted NULL, corresponding to attributes that - were unset on the object but did not have any server default - specified, as well as expired attributes on update that nevertheless - had no server onupdate set up. As these columns are not part of the - RETURNING that eager_defaults tries to use, they should not be - post-SELECTed either. - - .. change:: 3908 - :tags: bug, orm - :tickets: 3908 - - Fixed two closely related bugs involving the mapper eager_defaults - flag in conjunction with single-table inheritance; one where the - eager defaults logic would inadvertently try to access a column - that's part of the mapper's "exclude_properties" list (used by - Declarative with single table inheritance) during the eager defaults - fetch, and the other where the full load of the row in order to - fetch the defaults would fail to use the correct inheriting mapper. - - - .. change:: 3905 - :tags: bug, sql - :tickets: 3905 - - Fixed bug whereby the :meth:`.DDLEvents.column_reflect` event would not - allow a non-textual expression to be passed as the value of the - "default" for the new column, such as a :class:`.FetchedValue` - object to indicate a generic triggered default or a - :func:`_expression.text` construct. Clarified the documentation - in this regard as well. - - .. change:: 3901 - :tags: bug, ext - :tickets: 3901 - - Fixed bug in new :mod:`sqlalchemy.ext.indexable` extension - where setting of a property that itself refers to another property - would fail. - - .. change:: 3900 - :tags: bug, postgresql - :tickets: 3900 - - Fixed bug in PostgreSQL :class:`.ExcludeConstraint` where the - "whereclause" and "using" parameters would not be copied during an - operation like :meth:`_schema.Table.tometadata`. - - .. change:: 3898 - :tags: bug, mssql - :tickets: 3898 - - Added a version check to the "get_isolation_level" feature, which is - invoked upon first connect, so that it skips for SQL Server version - 2000, as the necessary system view is not available prior to SQL Server - 2005. - - .. change:: 3897 - :tags: feature, ext - :tickets: 3896 - - Added :meth:`.baked.Result.scalar` and :meth:`.baked.Result.count` - to the "baked" query system. - - .. change:: 3895 - :tags: bug, orm, declarative - :tickets: 3895 - - Fixed bug where the "automatic exclude" feature of declarative that - ensures a column local to a single table inheritance subclass does - not appear as an attribute on other derivations of the base would - not take effect for multiple levels of subclassing from the base. - - .. change:: 3893 - :tags: bug, orm - :tickets: 3893 - - Fixed bug first introduced in 0.9.7 as a result of :ticket:`3106` - which would cause an incorrect query in some forms of multi-level - subqueryload against aliased entities, with an unnecessary extra - FROM entity in the innermost subquery. - -.. changelog:: - :version: 1.1.5 - :released: January 17, 2017 - - .. change:: mysql_index_prefix - :tags: feature, mysql - - Added a new parameter ``mysql_prefix`` supported by the :class:`.Index` - construct, allows specification of MySQL-specific prefixes such as - "FULLTEXT". Pull request courtesy Joseph Schorr. - - .. change:: 3854 - :tags: bug, orm - :tickets: 3854 - - Fixed bug in subquery loading where an object encountered as an - "existing" row, e.g. already loaded from a different path in the - same query, would not invoke subquery loaders for unloaded attributes - that specified this loading. This issue is in the same area - as that of :ticket:`3431`, :ticket:`3811` which involved - similar issues with joined loading. - - .. change:: 3888 - :tags: bug, postgresql - :tickets: 3888 - - Fixed bug in new "ON CONFLICT DO UPDATE" feature where the "set" - values for the UPDATE clause would not be subject to type-level - processing, as normally takes effect to handle both user-defined - type level conversions as well as dialect-required conversions, such - as those required for JSON datatypes. Additionally, clarified that - the keys in the ``set_`` dictionary should match the "key" of the - column, if distinct from the column name. A warning is emitted - for remaining column names that don't match column keys; for - compatibility reasons, these are emitted as they were previously. - - .. change:: 3872 - :tags: bug, examples - :tickets: 3872 - - Fixed two issues with the versioned_history example, one is that - the history table now gets autoincrement=False to avoid 1.1's new - errors regarding composite primary keys with autoincrement; the other - is that the sqlite_autoincrement flag is now used to ensure on SQLite, - unique identifiers are used for the lifespan of a table even if - some rows are deleted. Pull request courtesy Carlos García Montoro. - - .. change:: 3882 - :tags: bug, sql - :tickets: 3882 - - Fixed bug originally introduced in 0.9 via :ticket:`1068` where - order_by() would order by the label name based on name - alone, that is, even if the labeled expression were not at all the same - expression otherwise present, implicitly or explicitly, in the - selectable. The logic that orders by label now ensures that the - labeled expression is related to the one that resolves to that name - before ordering by the label name; additionally, the name has to - resolve to an actual label explicit in the expression elsewhere, not - just a column name. This logic is carefully kept separate from the - order by(textual name) feature that has a slightly different purpose. - - .. change:: try_finally_for_noautoflush - :tags: bug, orm - - The :attr:`.Session.no_autoflush` context manager now ensures that - the autoflush flag is reset within a "finally" block, so that if - an exception is raised within the block, the state still resets - appropriately. Pull request courtesy Emin Arakelian. - - .. change:: 3878 - :tags: bug, sql - :tickets: 3878 - - Fixed 1.1 regression where "import *" would not work for - sqlalchemy.sql.expression, due to mis-spelled ``any_`` and ``all_`` - functions. - - .. change:: 3880 - :tags: bg, sql - :tickets: 3880 - - Fixed bug where literal_binds compiler flag was not honored by the - :class:`_expression.Insert` construct for the "multiple values" feature; the - subsequent values are now rendered as literals. - - .. change:: 3877 - :tags: bug, oracle, postgresql - :tickets: 3877 - - Fixed bug where an INSERT from SELECT where the source table contains - an autoincrementing Sequence would fail to compile correctly. - - .. change:: 3876 - :tags: bug, mssql - :tickets: 3876 - - Fixed bug where SQL Server dialects would attempt to select the - last row identity for an INSERT from SELECT, failing in the case when - the SELECT has no rows. For such a statement, - the inline flag is set to True indicating no last primary key - should be fetched. - - .. change:: 3875 - :tags: bug, oracle - :tickets: 3875 - - Fixed bug where the "COMPRESSION" keyword was used in the ALL_TABLES - query on Oracle 9.2; even though Oracle docs state table compression - was introduced in 9i, the actual column is not present until - 10.1. - - .. change:: 3874 - :tags: bug, orm - :tickets: 3874 - - Fixed bug where the single-table inheritance query criteria would not - be inserted into the query in the case that the :class:`.Bundle` - construct were used as the selection criteria. - - .. change:: repr_for_url_reflect - :tags: bug, sql - - The engine URL embedded in the exception for "could not reflect" - in :meth:`_schema.MetaData.reflect` now conceals the password; also - the ``__repr__`` for :class:`.TLEngine` now acts like that of - :class:`_engine.Engine`, concealing the URL password. Pull request courtesy - Valery Yundin. - - .. change:: 3867 - :tags: bug, mysql - :tickets: 3867 - - The MySQL dialect now will not warn when a reflected column has a - "COMMENT" keyword on it, but note however the comment is not yet - reflected; this is on the roadmap for a future release. Pull request - courtesy Lele Long. - - .. change:: pg_timestamp_zero_prec - :tags: bug, postgresql - - The :class:`_postgresql.TIME` and :class:`_postgresql.TIMESTAMP` - datatypes now support a setting of zero for "precision"; previously - a zero would be ignored. Pull request courtesy Ionuț Ciocîrlan. - - .. change:: 3861 - :tags: bug, engine - :tickets: 3861 - - The "extend_existing" option of :class:`_schema.Table` reflection would - cause indexes and constraints to be doubled up in the case that the parameter - were used with :meth:`_schema.MetaData.reflect` (as the automap extension does) - due to tables being reflected both within the foreign key path as well - as directly. A new de-duplicating set is passed through within the - :meth:`_schema.MetaData.reflect` sequence to prevent double reflection in this - way. - - .. change:: 3859 - :tags: bug, sql - :tickets: 3859 - - Fixed issue in :class:`.Variant` where the "right hand coercion" logic, - inherited from :class:`.TypeDecorator`, would - coerce the right-hand side into the :class:`.Variant` itself, rather than - what the default type for the :class:`.Variant` would do. In the - case of :class:`.Variant`, we want the type to act mostly like the base - type so the default logic of :class:`.TypeDecorator` is now overridden - to fall back to the underlying wrapped type's logic. Is mostly relevant - for JSON at the moment. - - .. change:: 3856 - :tags: bug, orm - :tickets: 3856 - - Fixed bug related to :ticket:`3177`, where a UNION or other set operation - emitted by a :class:`_query.Query` would apply "single-inheritance" criteria - to the outside of the union (also referencing the wrong selectable), - even though this criteria is now expected to - be already present on the inside subqueries. The single-inheritance - criteria is now omitted once union() or another set operation is - called against :class:`_query.Query` in the same way as :meth:`_query.Query.from_self`. - - .. change:: 3548 - :tags: bug, firebird - :tickets: 3548 - - Ported the fix for Oracle quoted-lowercase names to Firebird, so that - a table name that is quoted as lower case can be reflected properly - including when the table name comes from the get_table_names() - inspection function. - -.. changelog:: - :version: 1.1.4 - :released: November 15, 2016 - - .. change:: 3842 - :tags: bug, sql - :tickets: 3842 - - Fixed bug where newly added warning for primary key on insert w/o - autoincrement setting (see :ticket:`3216`) would fail to emit - correctly when invoked upon a lower-case :func:`.table` construct. - - .. change:: 3852 - :tags: bug, orm - :tickets: 3852 - - Fixed regression in collections due to :ticket:`3457` whereby - deserialize during pickle or deepcopy would fail to establish all - attributes of an ORM collection, causing further mutation operations to - fail. - - .. change:: default_schema - :tags: bug, engine - - Removed long-broken "default_schema_name()" method from - :class:`_engine.Connection`. This method was left over from a very old - version and was non-working (e.g. would raise). Pull request - courtesy Benjamin Dopplinger. - - .. change:: pragma - :tags: bug, sqlite - - Added quotes to the PRAGMA directives in the pysqlcipher dialect - to support additional cipher arguments appropriately. Pull request - courtesy Kevin Jurczyk. - - .. change:: 3846 - :tags: bug, postgresql - :tickets: 3846, 3807 - - Fixed regression caused by the fix in :ticket:`3807` (version 1.1.0) - where we ensured that the tablename was qualified in the WHERE clause - of the DO UPDATE portion of PostgreSQL's ON CONFLICT, however you - *cannot* put the table name in the WHERE clause in the actual ON - CONFLICT itself. This was an incorrect assumption, so that portion - of the change in :ticket:`3807` is rolled back. - - .. change:: 3845 - :tags: bug, orm - :tickets: 3845 - - Fixed long-standing bug where the "noload" relationship loading - strategy would cause backrefs and/or back_populates options to be - ignored. - - .. change:: sscursor_mysql - :tags: feature, mysql - - Added support for server side cursors to the mysqlclient and - pymysql dialects. This feature is available via the - :paramref:`.Connection.execution_options.stream_results` flag as well - as the ``server_side_cursors=True`` dialect argument in the - same way that it has been for psycopg2 on PostgreSQL. Pull request - courtesy Roman Podoliaka. - - .. change:: - :tags: bug, mysql - :tickets: 3841 - - MySQL's native ENUM type supports any non-valid value being sent, and - in response will return a blank string. A hardcoded rule to check for - "is returning the blank string" has been added to the MySQL - implementation for ENUM so that this blank string is returned to the - application rather than being rejected as a non-valid value. Note that - if your MySQL enum is linking values to objects, you still get the - blank string back. - - .. change:: - :tags: bug, sqlite, py3k - - Added an optional import for the pysqlcipher3 DBAPI when using the - pysqlcipher dialect. This package will attempt to be imported - if the Python-2 only pysqlcipher DBAPI is non-present. - Pull request courtesy Kevin Jurczyk. - -.. changelog:: - :version: 1.1.3 - :released: October 27, 2016 - - .. change:: - :tags: bug, orm - :tickets: 3839 - - Fixed regression caused by :ticket:`2677` whereby calling - :meth:`.Session.delete` on an object that was already flushed as - deleted in that session would fail to set up the object in the - identity map (or reject the object), causing flush errors as the - object were in a state not accommodated by the unit of work. - The pre-1.1 behavior in this case has been restored, which is that - the object is put back into the identity map so that the DELETE - statement will be attempted again, which emits a warning that the number - of expected rows was not matched (unless the row were restored outside - of the session). - - .. change:: - :tags: bug, postgresql - :tickets: 3835 - - PostgreSQL table reflection will ensure that the - :paramref:`_schema.Column.autoincrement` flag is set to False when reflecting - a primary key column that is not of an :class:`.Integer` datatype, - even if the default is related to an integer-generating sequence. - This can happen if a column is created as SERIAL and the datatype - is changed. The autoincrement flag can only be True if the datatype - is of integer affinity in the 1.1 series. - - .. change:: - :tags: bug, orm - :tickets: 3836 - - Fixed regression where some :class:`_query.Query` methods like - :meth:`_query.Query.update` and others would fail if the :class:`_query.Query` - were against a series of mapped columns, rather than the mapped - entity as a whole. - - .. change:: - :tags: bug, sql - :tickets: 3833 - - Fixed bug involving new value translation and validation feature - in :class:`.Enum` whereby using the enum object in a string - concatenation would maintain the :class:`.Enum` type as the type - of the expression overall, producing missing lookups. A string - concatenation against an :class:`.Enum`-typed column now uses - :class:`.String` as the datatype of the expression itself. - - .. change:: - :tags: bug, sql - :tickets: 3832 - - Fixed regression which occurred as a side effect of :ticket:`2919`, - which in the less typical case of a user-defined - :class:`.TypeDecorator` that was also itself an instance of - :class:`.SchemaType` (rather than the implementation being such) - would cause the column attachment events to be skipped for the - type itself. - - -.. changelog:: - :version: 1.1.2 - :released: October 17, 2016 - - .. change:: - :tags: bug, sql - :tickets: 3823 - - Fixed a regression caused by a newly added function that performs the - "wrap callable" function of sql :class:`.DefaultGenerator` objects, - an attribute error raised for ``__module__`` when the default callable - was a ``functools.partial`` or other object that doesn't have a - ``__module__`` attribute. - - .. change:: - :tags: bug, orm - :tickets: 3824 - - Fixed bug involving the rule to disable a joined collection eager - loader on the other side of a many-to-one lazy loader, first added - in :ticket:`1495`, where the rule would fail if the parent object - had some other lazyloader-bound query options associated with it. - - .. change:: - :tags: bug, orm - :tickets: 3822 - - Fixed self-referential entity, deferred column loading issue in a - similar style as that of :ticket:`3431`, :ticket:`3811` where an entity - is present in multiple positions within the row due to self-referential - eager loading; when the deferred loader only applies to one of the - paths, the "present" column loader will now override the deferred non- - load for that entity regardless of row ordering. - - .. change:: - :tags: bug, sql, postgresql - :tickets: 3827 - - Fixed regression in :class:`.Enum` type where event handlers were not - transferred in the case of the type object being copied, due to a - conflicting copy() method added as part of :ticket:`3250`. This copy - occurs normally in situations when the column is copied, such as - in tometadata() or when using declarative mixins with columns. The - event handler not being present would impact the constraint being - created for a non-native enumerated type, but more critically the - ENUM object on the PostgreSQL backend. - - - .. change:: - :tags: bug, postgresql, sql - :tickets: 3828 - - Changed the naming convention used when generating bound parameters - for a multi-VALUES insert statement, so that the numbered parameter - names don't conflict with the anonymized parameters of a WHERE clause, - as is now common in a PostgreSQL ON CONFLICT construct. - -.. changelog:: - :version: 1.1.1 - :released: October 7, 2016 - - .. change:: - :tags: bug, mssql - :tickets: 3820 - - The "SELECT SERVERPROPERTY" - query added in :ticket:`3810` and :ticket:`3814` is failing on unknown - combinations of Pyodbc and SQL Server. While failure of this function - was anticipated, the exception catch was not broad enough so it now - catches all forms of pyodbc.Error. - - .. change:: - :tags: bug, core - :tickets: 3216 - - Changed the CompileError raised when various primary key missing - situations are detected to a warning. The statement is again - passed to the database where it will fail and the DBAPI error (usually - IntegrityError) raises as usual. - - .. seealso:: - - :ref:`change_3216` - -.. changelog:: - :version: 1.1.0 - :released: October 5, 2016 - - .. change:: - :tags: bug, sql - :tickets: 3805 - - Execution options can now be propagated from within a - statement at compile time to the outermost statement, so that - if an embedded element wants to set "autocommit" to be True for example, - it can propagate this to the enclosing statement. Currently, this - feature is enabled for a DML-oriented CTE embedded inside of a SELECT - statement, e.g. INSERT/UPDATE/DELETE inside of SELECT. - - .. change:: - :tags: bug, orm - :tickets: 3802 - - ORM attributes can now be assigned any object that is has a - ``__clause_element__()`` attribute, which will result in inline - SQL the way any :class:`_expression.ClauseElement` class does. This covers other - mapped attributes not otherwise transformed by further expression - constructs. - - .. change:: - :tags: feature, orm - :tickets: 3812 - - Enhanced the new "raise" lazy loader strategy to also include a - "raise_on_sql" variant, available both via :paramref:`.orm.relationship.lazy` - as well as :func:`_orm.raiseload`. This variant only raises if the - lazy load would actually emit SQL, vs. raising if the lazy loader - mechanism is invoked at all. - - .. change:: - :tags: bug, postgresql - :tickets: 3813 - - An adjustment to ON CONFLICT such that the "inserted_primary_key" - logic is able to accommodate the case where there's no INSERT or - UPDATE and there's no net change. The value comes out as None - in this case, rather than failing on an exception. - - .. change:: - :tags: bug, orm - :tickets: 3811 - - Made an adjustment to the bug fix first introduced in [ticket:3431] - that involves an object appearing in multiple contexts in a single - result set, such that an eager loader that would set the related - object value to be None will still fire off, thus satisfying the - load of that attribute. Previously, the adjustment only honored - a non-None value arriving for an eagerly loaded attribute in a - secondary row. - - .. change:: - :tags: bug, orm - :tickets: 3808 - - Fixed bug in new :meth:`.SessionEvents.persistent_to_deleted` event - where the target object could be garbage collected before the event - is fired off. - - .. change:: - :tags: bug, sql - :tickets: 3809 - - A string sent as a column default via the - :paramref:`_schema.Column.server_default` parameter is now escaped for quotes. - - .. seealso:: - - :ref:`change_3809` - - .. change:: - :tags: bug, postgresql - :tickets: 3807 - - Fixed issue in new PG "on conflict" construct where columns including - those of the "excluded" namespace would not be table-qualified - in the WHERE clauses in the statement. - - .. change:: - :tags: bug, sql, postgresql - :tickets: 3806 - - Added compiler-level flags used by PostgreSQL to place additional - parenthesis than would normally be generated by precedence rules - around operations involving JSON, HSTORE indexing operators as well as - within their operands since it has been observed that PostgreSQL's - precedence rules for at least the HSTORE indexing operator is not - consistent between 9.4 and 9.5. - - .. change:: - :tags: bug, sql, mysql - :tickets: 3803 - - The ``BaseException`` exception class is now intercepted by the - exception-handling routines of :class:`_engine.Connection`, and includes - handling by the :meth:`_events.ConnectionEvents.handle_error` - event. The :class:`_engine.Connection` is now **invalidated** by default in - the case of a system level exception that is not a subclass of - ``Exception``, including ``KeyboardInterrupt`` and the greenlet - ``GreenletExit`` class, to prevent further operations from occurring - upon a database connection that is in an unknown and possibly - corrupted state. The MySQL drivers are most targeted by this change - however the change is across all DBAPIs. - - .. seealso:: - - :ref:`change_3803` - - .. change:: - :tags: bug, sql - :tickets: 3799 - - The "eq" and "ne" operators are no longer part of the list of - "associative" operators, while they remain considered to be - "commutative". This allows an expression like ``(x == y) == z`` - to be maintained at the SQL level with parenthesis. Pull request - courtesy John Passaro. - - .. change:: - :tags: bug, orm - :tickets: 3767 - - The primaryjoin of a :func:`_orm.relationship` construct can now include - a :func:`.bindparam` object that includes a callable function to - generate values. Previously, the lazy loader strategy would - be incompatible with this use, and additionally would fail to correctly - detect if the "use_get" criteria should be used if the primary key - were involved with the bound parameter. - - .. change:: - :tags: bug, orm - :tickets: 3801 - - An UPDATE emitted from the ORM flush process can now accommodate a - SQL expression element for a column within the primary key of an - object, if the target database supports RETURNING in order to provide - the new value, or if the PK value is set "to itself" for the purposes - of bumping some other trigger / onupdate on the column. - - .. change:: - :tags: bug, orm - :tickets: 3788 - - Fixed bug where the "simple many-to-one" condition that allows lazy - loading to use get() from identity map would fail to be invoked if the - primaryjoin of the relationship had multiple clauses separated by AND - which were not in the same order as that of the primary key columns - being compared in each clause. This ordering - difference occurs for a composite foreign key where the table-bound - columns on the referencing side were not in the same order in the .c - collection as the primary key columns on the referenced side....which - in turn occurs a lot if one is using declarative mixins and/or - declared_attr to set up columns. - - .. change:: - :tags: bug, sql - :tickets: 3789 - - Stringify of expression with unnamed :class:`_schema.Column` objects, as - occurs in lots of situations including ORM error reporting, - will now render the name in string context as "" - rather than raising a compile error. - - .. change:: - :tags: bug, sql - :tickets: 3786 - - Raise a more descriptive exception / message when ClauseElement - or non-SQLAlchemy objects that are not "executable" are erroneously - passed to ``.execute()``; a new exception ObjectNotExecutableError - is raised consistently in all cases. - - .. change:: - :tags: bug, orm - :tickets: 3776 - - An exception is raised when two ``@validates`` decorators on a mapping - make use of the same name. Only one validator of a certain name - at a time is supported, there's no mechanism to chain these together, - as the order of the validators at the level of function decorator - can't be made deterministic. - - .. seealso:: - - :ref:`change_3776` - - .. change:: - :tags: bug, orm - - Mapper errors raised during :func:`.configure_mappers` now explicitly - include the name of the originating mapper in the exception message - to help in those situations where the wrapped exception does not - itself include the source mapper. Pull request courtesy - John Perkins. - - .. change:: - :tags: bug, mysql - :tickets: 3766 - - Fixed bug where the "literal_binds" flag would not be propagated - to a CAST expression under MySQL. - - .. change:: - :tags: bug, sql, postgresql, mysql - :tickets: 3765 - - Fixed regression in JSON datatypes where the "literal processor" for - a JSON index value would not be invoked. The native String and Integer - datatypes are now called upon from within the JSONIndexType - and JSONPathType. This is applied to the generic, PostgreSQL, and - MySQL JSON types and also has a dependency on :ticket:`3766`. - - .. change:: - :tags: change, orm - - Passing False to :meth:`_query.Query.order_by` in order to cancel - all order by's is deprecated; there is no longer any difference - between calling this method with False or with None. - - .. change:: - :tags: feature, orm - - The :meth:`_query.Query.group_by` method now resets the group by collection - if an argument of ``None`` is passed, in the same way that - :meth:`_query.Query.order_by` has worked for a long time. Pull request - courtesy Iuri Diniz. - - .. change:: - :tags: bug, sql - :tickets: 3763 - - Fixed bug where :class:`.Index` would fail to extract columns from - compound SQL expressions if those SQL expressions were wrapped inside - of an ORM-style ``__clause_element__()`` construct. This bug - exists in 1.0.x as well, however in 1.1 is more noticeable as - hybrid_property @expression now returns a wrapped element. - - .. change:: - :tags: change, orm, declarative - - Constructing a declarative base class that inherits from another class - will also inherit its docstring. This means - :func:`~.ext.declarative.as_declarative` acts more like a normal class - decorator. - -.. changelog:: - :version: 1.1.0b3 - :released: July 26, 2016 - - .. change:: - :tags: change, orm - :tickets: 3749 - - Removed a warning that dates back to 0.4 which emits when a same-named - relationship is placed on two mappers that inherits via joined or - single table inheritance. The warning does not apply to the - current unit of work implementation. - - .. seealso:: - - :ref:`change_3749` - - - .. change:: - :tags: bug, sql - :tickets: 3745 - - Fixed bug in new CTE feature for update/insert/delete stated - as a CTE inside of an enclosing statement (typically SELECT) whereby - oninsert and onupdate values weren't called upon for the embedded - statement. - - .. change:: - :tags: bug, sql - :tickets: 3744 - - Fixed bug in new CTE feature for update/insert/delete whereby - an anonymous (e.g. no name passed) :class:`_expression.CTE` construct around - the statement would fail. - - .. change:: - :tags: bug, ext - - sqlalchemy.ext.indexable will intercept IndexError as well - as KeyError when raising as AttributeError. - - .. change:: - :tags: feature, ext - - Added a "default" parameter to the new sqlalchemy.ext.indexable - extension. - -.. changelog:: - :version: 1.1.0b2 - :released: July 1, 2016 - - .. change:: - :tags: bug, ext, postgresql - :tickets: 3732 - - Made a slight behavioral change in the ``sqlalchemy.ext.compiler`` - extension, whereby the existing compilation schemes for an established - construct would be removed if that construct itself didn't already - have its own dedicated ``__visit_name__``. This was a - rare occurrence in 1.0, however in 1.1 :class:`_postgresql.ARRAY` - subclasses :class:`_types.ARRAY` and has this behavior. - As a result, setting up a compilation handler for another dialect - such as SQLite would render the main :class:`_postgresql.ARRAY` - object no longer compilable. - - .. change:: - :tags: bug, sql - :tickets: 3730 - - The processing performed by the :class:`.Boolean` datatype for backends - that only feature integer types has been made consistent between the - pure Python and C-extension versions, in that the C-extension version - will accept any integer value from the database as a boolean, not just - zero and one; additionally, non-boolean integer values being sent to - the database are coerced to exactly zero or one, instead of being - passed as the original integer value. - - .. seealso:: - - :ref:`change_3730` - - .. change:: - :tags: bug, sql - :tickets: 3725 - - Rolled back the validation rules a bit in :class:`.Enum` to allow - unknown string values to pass through, unless the flag - ``validate_string=True`` is passed to the Enum; any other kind of object is - still of course rejected. While the immediate use - is to allow comparisons to enums with LIKE, the fact that this - use exists indicates there may be more unknown-string-comparison use - cases than we expected, which hints that perhaps there are some - unknown string-INSERT cases too. - - .. change:: - :tags: bug, mysql - :tickets: 3726 - - Dialed back the "order the primary key columns per auto-increment" - described in :ref:`change_mysql_3216` a bit, so that if the - :class:`.PrimaryKeyConstraint` is explicitly defined, the order - of columns is maintained exactly, allowing control of this behavior - when necessary. - -.. changelog:: - :version: 1.1.0b1 - :released: June 16, 2016 - - .. change:: - :tags: feature, sql - :tickets: 3718 - - Added TABLESAMPLE support via the new :meth:`_expression.FromClause.tablesample` - method and standalone function. Pull request courtesy Ilja Everilä. - - .. seealso:: - - :ref:`change_3718` - - .. change:: - :tags: feature, orm, ext - - A new ORM extension :ref:`indexable_toplevel` is added, which allows - construction of Python attributes which refer to specific elements - of "indexed" structures such as arrays and JSON fields. Pull request - courtesy Jeong YunWon. - - .. seealso:: - - :ref:`feature_indexable` - - .. change:: - :tags: bug, sql - :tickets: 3724 - - :meth:`_expression.FromClause.count` is deprecated. This function makes use of - an arbitrary column in the table and is not reliable; for Core use, - ``func.count()`` should be preferred. - - .. change:: - :tags: feature, postgresql - :tickets: 3529 - - Added support for PostgreSQL's INSERT..ON CONFLICT using a new - PostgreSQL-specific :class:`.postgresql.dml.Insert` object. - Pull request and extensive efforts here by Robin Thomas. - - .. seealso:: - - :ref:`change_3529` - - .. change:: - :tags: feature, postgresql - - The DDL for DROP INDEX will emit "CONCURRENTLY" if the - ``postgresql_concurrently`` flag is set upon the - :class:`.Index` and if the database in use is detected as - PostgreSQL version 9.2 or greater. For CREATE INDEX, database - version detection is also added which will omit the clause if - PG version is less than 8.2. Pull request courtesy Iuri de Silvio. - - .. change:: - :tags: bug, orm - :tickets: 3708 - - Fixed an issue where a many-to-one change of an object from one - parent to another could work inconsistently when combined with - an un-flushed modification of the foreign key attribute. The attribute - move now considers the database-committed value of the foreign key - in order to locate the "previous" parent of the object being - moved. This allows events to fire off correctly including - backref events. Previously, these events would not always fire. - Applications which may have relied on the previously broken - behavior may be affected. - - .. seealso:: - - :ref:`change_3708` - - .. change:: - :tags: feature, sql - :tickets: 3049 - - Added support for ranges in window functions, using the - :paramref:`.expression.over.range_` and - :paramref:`.expression.over.rows` parameters. - - .. seealso:: - - :ref:`change_3049` - - .. change:: - :tags: feature, orm - - Added new flag :paramref:`.Session.bulk_insert_mappings.render_nulls` - which allows an ORM bulk INSERT to occur with NULL values rendered; - this bypasses server side defaults, however allows all statements - to be formed with the same set of columns, allowing them to be - batched. Pull request courtesy Tobias Sauerwein. - - .. change:: - :tags: feature, postgresql - :tickets: 3588 - - Added new parameter :paramref:`.PGInspector.get_view_names.include`, - allowing specification for what kinds of views should be returned. - Currently "plain" and "materialized" views are included. Pull - request courtesy Sebastian Bank. - - .. change:: - :tags: feature, mssql - - The ``mssql_clustered`` flag available on :class:`.UniqueConstraint`, - :class:`.PrimaryKeyConstraint`, :class:`.Index` now defaults to - ``None``, and can be set to False which will render the NONCLUSTERED - keyword in particular for a primary key, allowing a different index to - be used as "clustered". Pull request courtesy Saulius Žemaitaitis. - - .. change:: - :tags: feature, orm - :tickets: 1311 - - Added new event :meth:`.AttributeEvents.init_scalar`, as well - as a new example suite illustrating its use. This event can be used - to provide a Core-generated default value to a Python-side attribute - before the object is persisted. - - .. seealso:: - - :ref:`change_1311` - - .. change:: - :tags: feature, postgresql - :tickets: 3720 - - Added ``postgresql_tablespace`` as an argument to :class:`.Index` - to allow specification of TABLESPACE for an index in PostgreSQL. - Complements the same-named parameter on :class:`_schema.Table`. Pull - request courtesy Benjamin Bertrand. - - .. change:: - :tags: orm, feature - - Added :paramref:`.AutomapBase.prepare.schema` to the - :meth:`.AutomapBase.prepare` method, to indicate which schema - tables should be reflected from if not the default schema. - Pull request courtesy Josh Marlow. - - .. change:: - :tags: feature, sqlite - - The SQLite dialect now reflects ON UPDATE and ON DELETE phrases - within foreign key constraints. Pull request courtesy - Michal Petrucha. - - .. change:: - :tags: bug, mssql - - Adjustments to the mxODBC dialect to make use of the ``BinaryNull`` - symbol when appropriate in conjunction with the ``VARBINARY`` - data type. Pull request courtesy Sheila Allen. - - .. change:: - :tags: feature, sql - - Implemented reflection of CHECK constraints for SQLite and PostgreSQL. - This is available via the new inspector method - :meth:`_reflection.Inspector.get_check_constraints` as well as when reflecting - :class:`_schema.Table` objects in the form of :class:`.CheckConstraint` - objects present in the constraints collection. Pull request courtesy - Alex Grönholm. - - .. change:: - :tags: feature, postgresql - - Added new parameter - :paramref:`.GenerativeSelect.with_for_update.key_share`, which - will render the ``FOR NO KEY UPDATE`` version of ``FOR UPDATE`` - and ``FOR KEY SHARE`` instead of ``FOR SHARE`` - on the PostgreSQL backend. Pull request courtesy Sergey Skopin. - - .. change:: - :tags: feature, postgresql, oracle - - Added new parameter - :paramref:`.GenerativeSelect.with_for_update.skip_locked`, which - will render the ``SKIP LOCKED`` phrase for a ``FOR UPDATE`` or - ``FOR SHARE`` lock on the PostgreSQL and Oracle backends. Pull - request courtesy Jack Zhou. - - .. change:: - :tags: change, orm - :tickets: 3394 - - The :paramref:`_orm.Mapper.order_by` parameter is deprecated. - This is an old parameter no longer relevant to how SQLAlchemy - works, once the Query object was introduced. By deprecating it - we establish that we aren't supporting non-working use cases - and that we encourage applications to move off of the use of this - parameter. - - .. seealso:: - - :ref:`change_3394` - - .. change:: - :tags: feature, postgresql - - Added a new dialect for the PyGreSQL PostgreSQL dialect. Thanks - to Christoph Zwerschke and Kaolin Imago Fire for their efforts. - - .. change:: - :tags: bug, ext - :tickets: 3653 - - The docstring specified on a hybrid property or method is now honored - at the class level, allowing it to work with tools like Sphinx - autodoc. The mechanics here necessarily involve some wrapping of - expressions to occur for hybrid properties, which may cause them - to appear differently using introspection. - - .. seealso:: - - :ref:`change_3653` - - .. change:: - :tags: feature, sql - - New :meth:`.ColumnOperators.is_distinct_from` and - :meth:`.ColumnOperators.isnot_distinct_from` operators; pull request - courtesy Sebastian Bank. - - .. seealso:: - - :ref:`change_is_distinct_from` - - .. change:: - :tags: bug, orm - :tickets: 3488 - - Fixed bug where deferred columns would inadvertently be set up - for database load on the next object-wide unexpire, when the object - were merged into the session with ``session.merge(obj, load=False)``. - - .. change:: - :tags: feature, sql - - Added a hook in :meth:`.DDLCompiler.visit_create_table` called - :meth:`.DDLCompiler.create_table_suffix`, allowing custom dialects - to add keywords after the "CREATE TABLE" clause. Pull request - courtesy Mark Sandan. - - .. change:: - :tags: feature, sql - - Negative integer indexes are now accommodated by rows - returned from a :class:`_engine.ResultProxy`. Pull request courtesy - Emanuele Gaifas. - - .. seealso:: - - :ref:`change_gh_231` - - .. change:: - :tags: feature, sqlite - :tickets: 3629 - - The SQLite dialect now reflects the names of primary key constraints. - Pull request courtesy Diana Clarke. - - .. seealso:: - - :ref:`change_3629` - - .. change:: - :tags: feature, sql - :tickets: 2857 - - Added :meth:`_expression.Select.lateral` and related constructs to allow - for the SQL standard LATERAL keyword, currently only supported - by PostgreSQL. - - .. seealso:: - - :ref:`change_2857` - - .. change:: - :tags: feature, sql - :tickets: 1957 - - Added support for rendering "FULL OUTER JOIN" to both Core and ORM. - Pull request courtesy Stefan Urbanek. - - .. seealso:: - - :ref:`change_1957` - - .. change:: - :tags: feature, engine - - Added connection pool events :meth:`ConnectionEvents.close`, - :meth:`_events.ConnectionEvents.detach`, - :meth:`_events.ConnectionEvents.close_detached`. - - .. change:: - :tags: bug, orm, mysql - :tickets: 3680 - - Further continuing on the common MySQL exception case of - a savepoint being cancelled first covered in :ticket:`2696`, - the failure mode in which the :class:`.Session` is placed when a - SAVEPOINT vanishes before rollback has been improved to allow the - :class:`.Session` to still function outside of that savepoint. - It is assumed that the savepoint operation failed and was cancelled. - - .. seealso:: - - :ref:`change_3680` - - .. change:: - :tags: feature, mssql - :tickets: 3534 - - Added basic isolation level support to the SQL Server dialects - via :paramref:`_sa.create_engine.isolation_level` and - :paramref:`.Connection.execution_options.isolation_level` - parameters. - - .. seealso:: - - :ref:`change_3534` - - .. change:: - :tags: feature, mysql - :tickets: 3332 - - Added support for "autocommit" on MySQL drivers, via the - AUTOCOMMIT isolation level setting. Pull request courtesy - Roman Podoliaka. - - .. seealso:: - - :ref:`change_3332` - - .. change:: - :tags: bug, orm - :tickets: 3677 - - Fixed bug where a newly inserted instance that is rolled back - would still potentially cause persistence conflicts on the next - transaction, because the instance would not be checked that it - was expired. This fix will resolve a large class of cases that - erroneously cause the "New instance with identity X conflicts with - persistent instance Y" error. - - .. seealso:: - - :ref:`change_3677` - - .. change:: - :tags: bug, orm - :tickets: 3662 - - An improvement to the workings of :meth:`_query.Query.correlate` such - that when a "polymorphic" entity is used which represents a straight - join of several tables, the statement will ensure that all the - tables within the join are part of what's correlating. - - .. seealso:: - - :ref:`change_3662` - - .. change:: - :tags: bug, orm - :tickets: 3431 - - Fixed bug which would cause an eagerly loaded many-to-one attribute - to not be loaded, if the joined eager load were from a row where the - same entity were present multiple times, some calling for the attribute - to be eagerly loaded and others not. The logic here is revised to - take in the attribute even though a different loader path has - handled the parent entity already. - - .. seealso:: - - :ref:`change_3431` - - .. change:: - :tags: feature, engine - :tickets: 2837 - - All string formatting of bound parameter sets and result rows for - logging, exception, and ``repr()`` purposes now truncate very large - scalar values within each collection, including an - "N characters truncated" - notation, similar to how the display for large multiple-parameter sets - are themselves truncated. - - - .. seealso:: - - :ref:`change_2837` - - .. change:: - :tags: feature, ext - :tickets: 3297 - - Added :class:`.MutableSet` and :class:`.MutableList` helper classes - to the :ref:`mutable_toplevel` extension. Pull request courtesy - Jeong YunWon. - - .. change:: - :tags: feature, sql - :tickets: 2551 - - CTE functionality has been expanded to support all DML, allowing - INSERT, UPDATE, and DELETE statements to both specify their own - WITH clause, as well as for these statements themselves to be - CTE expressions when they include a RETURNING clause. - - .. seealso:: - - :ref:`change_2551` - - .. change:: - :tags: bug, orm - :tickets: 3641 - - A refinement to the logic which adds columns to the resulting SQL when - :meth:`_query.Query.distinct` is combined with :meth:`_query.Query.order_by` such - that columns which are already present will not be added - a second time, even if they are labeled with a different name. - Regardless of this change, the extra columns added to the SQL have - never been returned in the final result, so this change only impacts - the string form of the statement as well as its behavior when used in - a Core execution context. Additionally, columns are no longer added - when the DISTINCT ON format is used, provided the query is not - wrapped inside a subquery due to joined eager loading. - - .. seealso:: - - :ref:`change_3641` - - .. change:: - :tags: feature, sql - :tickets: 3292, 3095 - - Added support for PEP-435-style enumerated classes, namely - Python 3's ``enum.Enum`` class but also including compatible - enumeration libraries, to the :class:`_types.Enum` datatype. - The :class:`_types.Enum` datatype now also performs in-Python validation - of incoming values, and adds an option to forego creating the - CHECK constraint :paramref:`.Enum.create_constraint`. - Pull request courtesy Alex Grönholm. - - .. seealso:: - - :ref:`change_3292` - - :ref:`change_3095` - - .. change:: - :tags: change, postgresql - - The ``sqlalchemy.dialects.postgres`` module, long deprecated, is - removed; this has emitted a warning for many years and projects - should be calling upon ``sqlalchemy.dialects.postgresql``. - Engine URLs of the form ``postgres://`` will still continue to function, - however. - - .. change:: - :tags: bug, sqlite - :tickets: 3634 - - The workaround for right-nested joins on SQLite, where they are rewritten - as subqueries in order to work around SQLite's lack of support for this - syntax, is lifted when SQLite version 3.7.16 or greater is detected. - - .. seealso:: - - :ref:`change_3634` - - .. change:: - :tags: bug, sqlite - :tickets: 3633 - - The workaround for SQLite's unexpected delivery of column names as - ``tablename.columnname`` for some kinds of queries is now disabled - when SQLite version 3.10.0 or greater is detected. - - .. seealso:: - - :ref:`change_3633` - - .. change:: - :tags: feature, orm - :tickets: 2349 - - Added new parameter :paramref:`.orm.mapper.passive_deletes` to - available mapper options. This allows a DELETE to proceed - for a joined-table inheritance mapping against the base table only, - while allowing for ON DELETE CASCADE to handle deleting the row - from the subclass tables. - - .. seealso:: - - :ref:`change_2349` - - - .. change:: - :tags: bug, sybase - :tickets: 2278 - - The unsupported Sybase dialect now raises ``NotImplementedError`` - when attempting to compile a query that includes "offset"; Sybase - has no straightforward "offset" feature. - - .. change:: - :tags: feature, orm - :tickets: 3631 - - Calling str() on a core SQL construct has been made more "friendly", - when the construct contains non-standard SQL elements such as - RETURNING, array index operations, or dialect-specific or custom - datatypes. A string is now returned in these cases rendering an - approximation of the construct (typically the PostgreSQL-style - version of it) rather than raising an error. - - .. seealso:: - - :ref:`change_3631` - - .. change:: - :tags: bug, orm - :tickets: 3630 - - Fixed issue where two same-named relationships that refer to - a base class and a concrete-inherited subclass would raise an error - if those relationships were set up using "backref", while setting up the - identical configuration using relationship() instead with the conflicting - names would succeed, as is allowed in the case of a concrete mapping. - - .. seealso:: - - :ref:`change_3630` - - .. change:: - :tags: feature, orm - :tickets: 3081 - - The ``str()`` call for :class:`_query.Query` will now take into account - the :class:`_engine.Engine` to which the :class:`.Session` is bound, when - generating the string form of the SQL, so that the actual SQL - that would be emitted to the database is shown, if possible. Previously, - only the engine associated with the :class:`_schema.MetaData` to which the - mappings are associated would be used, if present. If - no bind can be located either on the :class:`.Session` or on - the :class:`_schema.MetaData` to which the mappings are associated, then - the "default" dialect is used to render the SQL, as was the case - previously. - - .. seealso:: - - :ref:`change_3081` - - .. change:: - :tags: feature, sql - :tickets: 3501 - - A deep improvement to the recently added :meth:`_expression.TextClause.columns` - method, and its interaction with result-row processing, now allows - the columns passed to the method to be positionally matched with the - result columns in the statement, rather than matching on name alone. - The advantage to this includes that when linking a textual SQL statement - to an ORM or Core table model, no system of labeling or de-duping of - common column names needs to occur, which also means there's no need - to worry about how label names match to ORM columns and so-forth. In - addition, the :class:`_engine.ResultProxy` has been further enhanced to - map column and string keys to a row with greater precision in some - cases. - - .. seealso:: - - :ref:`change_3501` - feature overview - - :ref:`behavior_change_3501` - backwards compatibility remarks - - .. change:: - :tags: feature, engine - :tickets: 2685 - - Multi-tenancy schema translation for :class:`_schema.Table` objects is added. - This supports the use case of an application that uses the same set of - :class:`_schema.Table` objects in many schemas, such as schema-per-user. - A new execution option - :paramref:`.Connection.execution_options.schema_translate_map` is - added. - - .. seealso:: - - :ref:`change_2685` - - .. change:: - :tags: feature, engine - :tickets: 3536 - - Added a new entrypoint system to the engine to allow "plugins" to - be stated in the query string for a URL. Custom plugins can - be written which will be given the chance up front to alter and/or - consume the engine's URL and keyword arguments, and then at engine - create time will be given the engine itself to allow additional - modifications or event registration. Plugins are written as a - subclass of :class:`.CreateEnginePlugin`; see that class for - details. - - .. change:: - :tags: feature, mysql - :tickets: 3547 - - Added :class:`.mysql.JSON` for MySQL 5.7. The JSON type provides - persistence of JSON values in MySQL as well as basic operator support - of "getitem" and "getpath", making use of the ``JSON_EXTRACT`` - function in order to refer to individual paths in a JSON structure. - - .. seealso:: - - :ref:`change_3547` - - .. change:: - :tags: feature, sql - :tickets: 3619 - - Added a new type to core :class:`_types.JSON`. This is the - base of the PostgreSQL :class:`_postgresql.JSON` type as well as that - of the new :class:`.mysql.JSON` type, so that a PG/MySQL-agnostic - JSON column may be used. The type features basic index and path - searching support. - - .. seealso:: - - :ref:`change_3619` - - .. change:: - :tags: bug, sql - :tickets: 3616 - - Fixed an assertion that would raise somewhat inappropriately - if a :class:`.Index` were associated with a :class:`_schema.Column` that - is associated with a lower-case-t :class:`_expression.TableClause`; the - association should be ignored for the purposes of associating - the index with a :class:`_schema.Table`. - - .. change:: - :tags: bug, orm - :tickets: 3601 - - The :meth:`.Session.merge` method now tracks pending objects by - primary key before emitting an INSERT, and merges distinct objects with - duplicate primary keys together as they are encountered, which is - essentially semi-deterministic at best. This behavior - matches what happens already with persistent objects. - - .. seealso:: - - :ref:`change_3601` - - .. change:: - :tags: bug, postgresql - :tickets: 3587 - - Added support for reflecting the source of materialized views - to the PostgreSQL version of the :meth:`_reflection.Inspector.get_view_definition` - method. - - .. change:: - :tags: bug, orm - :tickets: 3582 - - Fixed bug where the "single table inheritance" criteria would be - added onto the end of a query in some inappropriate situations, such - as when querying from an exists() of a single-inheritance subclass. - - .. seealso:: - - :ref:`change_3582` - - .. change:: - :tags: enhancement, schema - - The default generation functions passed to :class:`_schema.Column` objects - are now run through "update_wrapper", or an equivalent function - if a callable non-function is passed, so that introspection tools - preserve the name and docstring of the wrapped function. Pull - request courtesy hsum. - - .. change:: - :tags: change, sql, mysql - :tickets: 3216 - - The system by which a :class:`_schema.Column` considers itself to be an - "auto increment" column has been changed, such that autoincrement - is no longer implicitly enabled for a :class:`_schema.Table` that has a - composite primary key. In order to accommodate being able to enable - autoincrement for a composite PK member column while at the same time - maintaining SQLAlchemy's long standing behavior of enabling - implicit autoincrement for a single integer primary key, a third - state has been added to the :paramref:`_schema.Column.autoincrement` parameter - ``"auto"``, which is now the default. - - .. seealso:: - - :ref:`change_3216` - - :ref:`change_mysql_3216` - - .. change:: - :tags: change, mysql - :tickets: 3216 - - The MySQL dialect no longer generates an extra "KEY" directive when - generating CREATE TABLE DDL for a table using InnoDB with a - composite primary key with AUTO_INCREMENT on a column that isn't the - first column; to overcome InnoDB's limitation here, the PRIMARY KEY - constraint is now generated with the AUTO_INCREMENT column placed - first in the list of columns. - - .. seealso:: - - :ref:`change_mysql_3216` - - :ref:`change_3216` - - .. change:: - :tags: change, sqlite - - Added support to the SQLite dialect for the - :meth:`_reflection.Inspector.get_schema_names` method to work with SQLite; - pull request courtesy Brian Van Klaveren. Also repaired support - for creation of indexes with schemas as well as reflection of - foreign key constraints in schema-bound tables. - - .. seealso:: - - :ref:`change_sqlite_schemas` - - .. change:: - :tags: change, mssql - :tickets: 3434 - - The ``legacy_schema_aliasing`` flag, introduced in version 1.0.5 - as part of :ticket:`3424` to allow disabling of the MSSQL dialect's - attempts to create aliases for schema-qualified tables, now defaults - to False; the old behavior is now disabled unless explicitly turned on. - - .. seealso:: - - :ref:`change_3434` - - .. change:: - :tags: bug, orm - :tickets: 3250 - - Added a new type-level modifier :meth:`.TypeEngine.evaluates_none` - which indicates to the ORM that a positive set of None should be - persisted as the value NULL, instead of omitting the column from - the INSERT statement. This feature is used both as part of the - implementation for :ticket:`3514` as well as a standalone feature - available on any type. - - .. seealso:: - - :ref:`change_3250` - - .. change:: - :tags: bug, postgresql - :tickets: 2729 - - The use of a :class:`_postgresql.ARRAY` object that refers - to a :class:`_types.Enum` or :class:`_postgresql.ENUM` subtype - will now emit the expected "CREATE TYPE" and "DROP TYPE" DDL when - the type is used within a "CREATE TABLE" or "DROP TABLE". - - .. seealso:: - - :ref:`change_2729` - - .. change:: - :tags: bug, sql - :tickets: 3531 - - The :func:`.type_coerce` construct is now a fully fledged Core - expression element which is late-evaluated at compile time. Previously, - the function was only a conversion function which would handle different - expression inputs by returning either a :class:`.Label` of a column-oriented - expression or a copy of a given :class:`.BindParameter` object, - which in particular prevented the operation from being logically - maintained when an ORM-level expression transformation would convert - a column to a bound parameter (e.g. for lazy loading). - - .. seealso:: - - :ref:`change_3531` - - .. change:: - :tags: bug, orm - :tickets: 3526 - - Internal calls to "bookkeeping" functions within - :meth:`.Session.bulk_save_objects` and related bulk methods have - been scaled back to the extent that this functionality is not - currently used, e.g. checks for column default values to be - fetched after an INSERT or UPDATE statement. - - .. change:: - :tags: feature, orm - :tickets: 2677 - - The :class:`.SessionEvents` suite now includes events to allow - unambiguous tracking of all object lifecycle state transitions - in terms of the :class:`.Session` itself, e.g. pending, - transient, persistent, detached. The state of the object - within each event is also defined. - - .. seealso:: - - :ref:`change_2677` - - .. change:: - :tags: feature, orm - :tickets: 2677 - - Added a new session lifecycle state :term:`deleted`. This new state - represents an object that has been deleted from the :term:`persistent` - state and will move to the :term:`detached` state once the transaction - is committed. This resolves the long-standing issue that objects - which were deleted existed in a gray area between persistent and - detached. The :attr:`.InstanceState.persistent` accessor will - **no longer** report on a deleted object as persistent; the - :attr:`.InstanceState.deleted` accessor will instead be True for - these objects, until they become detached. - - .. seealso:: - - :ref:`change_2677` - - .. change:: - :tags: change, orm - :tickets: 2677 - - The :paramref:`.Session.weak_identity_map` parameter is deprecated. - See the new recipe at :ref:`session_referencing_behavior` for - an event-based approach to maintaining strong identity map behavior. - - .. seealso:: - - :ref:`change_2677` - - .. change:: - :tags: bug, sql - :tickets: 2919 - - The :class:`.TypeDecorator` type extender will now work in conjunction - with a :class:`.SchemaType` implementation, typically :class:`.Enum` - or :class:`.Boolean` with regards to ensuring that the per-table - events are propagated from the implementation type to the outer type. - These events are used - to ensure that the constraints or PostgreSQL types (e.g. ENUM) - are correctly created (and possibly dropped) along with the parent - table. - - .. seealso:: - - :ref:`change_2919` - - .. change:: - :tags: feature, sql - :tickets: 1370 - - Added support for "set-aggregate" functions of the form - `` WITHIN GROUP (ORDER BY )``, using the - method :meth:`.FunctionElement.within_group`. A series of common - set-aggregate functions with return types derived from the set have - been added. This includes functions like :class:`.percentile_cont`, - :class:`.dense_rank` and others. - - .. seealso:: - - :ref:`change_3132` - - .. change:: - :tags: feature, sql, postgresql - :tickets: 3132 - - Added support for the SQL-standard function :class:`_functions.array_agg`, - which automatically returns an :class:`_postgresql.ARRAY` of the correct type - and supports index / slice operations, as well as - :func:`_postgresql.array_agg`, which returns a :class:`_postgresql.ARRAY` - with additional comparison features. As arrays are only - supported on PostgreSQL at the moment, only actually works on - PostgreSQL. Also added a new construct - :class:`_postgresql.aggregate_order_by` in support of PG's - "ORDER BY" extension. - - .. seealso:: - - :ref:`change_3132` - - .. change:: - :tags: feature, sql - :tickets: 3516 - - Added a new type to core :class:`_types.ARRAY`. This is the - base of the PostgreSQL :class:`_postgresql.ARRAY` type, and is now part of Core - to begin supporting various SQL-standard array-supporting features - including some functions and eventual support for native arrays - on other databases that have an "array" concept, such as DB2 or Oracle. - Additionally, new operators :func:`_expression.any_` and - :func:`_expression.all_` have been added. These support not just - array constructs on PostgreSQL, but also subqueries that are usable - on MySQL (but sadly not on PostgreSQL). - - .. seealso:: - - :ref:`change_3516` - - .. change:: - :tags: feature, orm - :tickets: 3321 - - Added new checks for the common error case of passing mapped classes - or mapped instances into contexts where they are interpreted as - SQL bound parameters; a new exception is raised for this. - - .. seealso:: - - :ref:`change_3321` - - .. change:: - :tags: bug, postgresql - :tickets: 3499 - - The "hashable" flag on special datatypes such as :class:`_postgresql.ARRAY`, - :class:`_postgresql.JSON` and :class:`_postgresql.HSTORE` is now - set to False, which allows these types to be fetchable in ORM - queries that include entities within the row. - - .. seealso:: - - :ref:`change_3499` - - :ref:`change_3499_postgresql` - - .. change:: - :tags: bug, postgresql - :tickets: 3487 - - The PostgreSQL :class:`_postgresql.ARRAY` type now supports multidimensional - indexed access, e.g. expressions such as ``somecol[5][6]`` without - any need for explicit casts or type coercions, provided - that the :paramref:`.postgresql.ARRAY.dimensions` parameter is set to the - desired number of dimensions. - - .. seealso:: - - :ref:`change_3503` - - .. change:: - :tags: bug, postgresql - :tickets: 3503 - - The return type for the :class:`_postgresql.JSON` and :class:`_postgresql.JSONB` - when using indexed access has been fixed to work like PostgreSQL itself, - and returns an expression that itself is of type :class:`_postgresql.JSON` - or :class:`_postgresql.JSONB`. Previously, the accessor would return - :class:`.NullType` which disallowed subsequent JSON-like operators to be - used. - - .. seealso:: - - :ref:`change_3503` - - .. change:: - :tags: bug, postgresql - :tickets: 3503 - - The :class:`_postgresql.JSON`, :class:`_postgresql.JSONB` and - :class:`_postgresql.HSTORE` datatypes now allow full control over the - return type from an indexed textual access operation, either ``column[someindex].astext`` - for a JSON type or ``column[someindex]`` for an HSTORE type, - via the :paramref:`.postgresql.JSON.astext_type` and - :paramref:`.postgresql.HSTORE.text_type` parameters. - - .. seealso:: - - :ref:`change_3503` - - - .. change:: - :tags: bug, postgresql - :tickets: 3503 - - The :attr:`.postgresql.JSON.Comparator.astext` modifier no longer - calls upon :meth:`_expression.ColumnElement.cast` implicitly, as PG's JSON/JSONB - types allow cross-casting between each other as well. Code that - makes use of :meth:`_expression.ColumnElement.cast` on JSON indexed access, - e.g. ``col[someindex].cast(Integer)``, will need to be changed - to call :attr:`.postgresql.JSON.Comparator.astext` explicitly. - - .. seealso:: - - :ref:`change_3503_cast` - - - .. change:: - :tags: bug, orm, postgresql - :tickets: 3514 - - Additional fixes have been made regarding the value of ``None`` - in conjunction with the PostgreSQL :class:`_postgresql.JSON` type. When - the :paramref:`_types.JSON.none_as_null` flag is left at its default - value of ``False``, the ORM will now correctly insert the JSON - "'null'" string into the column whenever the value on the ORM - object is set to the value ``None`` or when the value ``None`` - is used with :meth:`.Session.bulk_insert_mappings`, - **including** if the column has a default or server default on it. - - .. seealso:: - - :ref:`change_3514` - - :ref:`change_3250` - - .. change:: - :tags: feature, postgresql - :tickets: 3514 - - Added a new constant :attr:`.postgresql.JSON.NULL`, indicating - that the JSON NULL value should be used for a value - regardless of other settings. - - .. seealso:: - - :ref:`change_3514_jsonnull` - - .. change:: - :tags: bug, sql - :tickets: 2528 - - The behavior of the :func:`_expression.union` construct and related constructs - such as :meth:`_query.Query.union` now handle the case where the embedded - SELECT statements need to be parenthesized due to the fact that they - include LIMIT, OFFSET and/or ORDER BY. These queries **do not work - on SQLite**, and will fail on that backend as they did before, but - should now work on all other backends. - - .. seealso:: - - :ref:`change_2528` - - .. change:: - :tags: feature, orm - :tickets: 3512 - - Added new relationship loading strategy :func:`_orm.raiseload` (also - accessible via ``lazy='raise'``). This strategy behaves almost like - :func:`_orm.noload` but instead of returning ``None`` it raises an - InvalidRequestError. Pull request courtesy Adrian Moennich. - - .. seealso:: - - :ref:`change_3512` - - .. change:: - :tags: bug, mssql - :tickets: 3504 - - Fixed issue where the SQL Server dialect would reflect a string- - or other variable-length column type with unbounded length - by assigning the token ``"max"`` to the - length attribute of the string. While using the ``"max"`` token - explicitly is supported by the SQL Server dialect, it isn't part - of the normal contract of the base string types, and instead the - length should just be left as None. The dialect now assigns the - length to None on reflection of the type so that the type behaves - normally in other contexts. - - .. seealso:: - - :ref:`change_3504` diff --git a/doc/build/changelog/changelog_12.rst b/doc/build/changelog/changelog_12.rst deleted file mode 100644 index a0187bc857..0000000000 --- a/doc/build/changelog/changelog_12.rst +++ /dev/null @@ -1,2945 +0,0 @@ -============= -1.2 Changelog -============= - -.. changelog_imports:: - - .. include:: changelog_11.rst - :start-line: 5 - - - .. include:: changelog_10.rst - :start-line: 5 - - -.. changelog:: - :version: 1.2.19 - :released: April 15, 2019 - - .. change:: - :tags: bug, orm - :tickets: 4507 - - Fixed a regression in 1.2 due to the introduction of baked queries for - relationship lazy loaders, where a race condition is created during the - generation of the "lazy clause" which occurs within a memoized attribute. If - two threads initialize the memoized attribute concurrently, the baked query - could be generated with bind parameter keys that are then replaced with new - keys by the next run, leading to a lazy load query that specifies the - related criteria as ``None``. The fix establishes that the parameter names - are fixed before the new clause and parameter objects are generated, so that - the names are the same every time. - - .. change:: - :tags: bug, oracle - :tickets: 4506 - - Added support for reflection of the :class:`_types.NCHAR` datatype to the Oracle - dialect, and added :class:`_types.NCHAR` to the list of types exported by the - Oracle dialect. - - - .. change:: - :tags: bug, examples - :tickets: 4528 - - Fixed bug in large_resultsets example case where a re-named "id" variable - due to code reformatting caused the test to fail. Pull request courtesy - Matt Schuchhardt. - - .. change:: - :tags: bug, mssql - :tickets: 4536 - :versions: 1.3.1 - - A commit() is emitted after an isolation level change to SNAPSHOT, as both - pyodbc and pymssql open an implicit transaction which blocks subsequent SQL - from being emitted in the current transaction. - - .. change:: - :tags: bug, engine - :tickets: 4406 - - Comparing two objects of :class:`.URL` using ``__eq__()`` did not take port - number into consideration, two objects differing only by port number were - considered equal. Port comparison is now added in ``__eq__()`` method of - :class:`.URL`, objects differing by port number are now not equal. - Additionally, ``__ne__()`` was not implemented for :class:`.URL` which - caused unexpected result when ``!=`` was used in Python2, since there are no - implied relationships among the comparison operators in Python2. - -.. changelog:: - :version: 1.2.18 - :released: February 15, 2019 - - .. change:: - :tags: bug, orm - :tickets: 4468 - - Fixed a regression in 1.2 where a wildcard/load_only loader option would - not work correctly against a loader path where of_type() were used to limit - to a particular subclass. The fix only works for of_type() of a simple - subclass so far, not a with_polymorphic entity which will be addressed in a - separate issue; it is unlikely this latter case was working previously. - - - .. change:: - :tags: bug, orm - :tickets: 4489 - - Fixed fairly simple but critical issue where the - :meth:`.SessionEvents.pending_to_persistent` event would be invoked for - objects not just when they move from pending to persistent, but when they - were also already persistent and just being updated, thus causing the event - to be invoked for all objects on every update. - - .. change:: - :tags: bug, sql - :tickets: 4485 - - Fixed issue where the :class:`_types.JSON` type had a read-only - :attr:`_types.JSON.should_evaluate_none` attribute, which would cause failures - when making use of the :meth:`.TypeEngine.evaluates_none` method in - conjunction with this type. Pull request courtesy Sanjana S. - - .. change:: - :tags: bug, mssql - :tickets: 4499 - - Fixed bug where the SQL Server "IDENTITY_INSERT" logic that allows an INSERT - to proceed with an explicit value on an IDENTITY column was not detecting - the case where :meth:`_expression.Insert.values` were used with a dictionary that - contained a :class:`_schema.Column` as key and a SQL expression as a value. - - .. change:: - :tags: bug, sqlite - :tickets: 4474 - - Fixed bug in SQLite DDL where using an expression as a server side default - required that it be contained within parenthesis to be accepted by the - sqlite parser. Pull request courtesy Bartlomiej Biernacki. - - .. change:: - :tags: bug, mysql - :tickets: 4492 - - Fixed a second regression caused by :ticket:`4344` (the first was - :ticket:`4361`), which works around MySQL issue 88718, where the lower - casing function used was not correct for Python 2 with OSX/Windows casing - conventions, which would then raise ``TypeError``. Full coverage has been - added to this logic so that every codepath is exercised in a mock style for - all three casing conventions on all versions of Python. MySQL 8.0 has - meanwhile fixed issue 88718 so the workaround is only applies to a - particular span of MySQL 8.0 versions. - -.. changelog:: - :version: 1.2.17 - :released: January 25, 2019 - - .. change:: - :tags: feature, orm - :tickets: 4461 - - Added new event hooks :meth:`.QueryEvents.before_compile_update` and - :meth:`.QueryEvents.before_compile_delete` which complement - :meth:`.QueryEvents.before_compile` in the case of the :meth:`_query.Query.update` - and :meth:`_query.Query.delete` methods. - - - .. change:: - :tags: bug, postgresql - :tickets: 4463 - - Revised the query used when reflecting CHECK constraints to make use of the - ``pg_get_constraintdef`` function, as the ``consrc`` column is being - deprecated in PG 12. Thanks to John A Stevenson for the tip. - - - .. change:: - :tags: bug, orm - :tickets: 4454 - - Fixed issue where when using single-table inheritance in conjunction with a - joined inheritance hierarchy that uses "with polymorphic" loading, the - "single table criteria" for that single-table entity could get confused for - that of other entities from the same hierarchy used in the same query.The - adaption of the "single table criteria" is made more specific to the target - entity to avoid it accidentally getting adapted to other tables in the - query. - - - .. change:: - :tags: bug, oracle - :tickets: 4457 - - Fixed regression in integer precision logic due to the refactor of the - cx_Oracle dialect in 1.2. We now no longer apply the cx_Oracle.NATIVE_INT - type to result columns sending integer values (detected as positive - precision with scale ==0) which encounters integer overflow issues with - values that go beyond the 32 bit boundary. Instead, the output variable - is left untyped so that cx_Oracle can choose the best option. - -.. changelog:: - :version: 1.2.16 - :released: January 11, 2019 - - .. change:: - :tag: bug, sql - :tickets: 4394 - - Fixed issue in "expanding IN" feature where using the same bound parameter - name more than once in a query would lead to a KeyError within the process - of rewriting the parameters in the query. - - .. change:: - :tags: bug, postgresql - :tickets: 4416 - - Fixed issue where a :class:`_postgresql.ENUM` or a custom domain present - in a remote schema would not be recognized within column reflection if - the name of the enum/domain or the name of the schema required quoting. - A new parsing scheme now fully parses out quoted or non-quoted tokens - including support for SQL-escaped quotes. - - .. change:: - :tags: bug, postgresql - - Fixed issue where multiple :class:`_postgresql.ENUM` objects referred to - by the same :class:`_schema.MetaData` object would fail to be created if - multiple objects had the same name under different schema names. The - internal memoization the PostgreSQL dialect uses to track if it has - created a particular :class:`_postgresql.ENUM` in the database during - a DDL creation sequence now takes schema name into account. - - .. change:: - :tags: bug, engine - :tickets: 4429 - - Fixed a regression introduced in version 1.2 where a refactor - of the :class:`.SQLAlchemyError` base exception class introduced an - inappropriate coercion of a plain string message into Unicode under - python 2k, which is not handled by the Python interpreter for characters - outside of the platform's encoding (typically ascii). The - :class:`.SQLAlchemyError` class now passes a bytestring through under - Py2K for ``__str__()`` as is the behavior of exception objects in general - under Py2K, does a safe coercion to unicode utf-8 with - backslash fallback for ``__unicode__()``. For Py3K the message is - typically unicode already, but if not is again safe-coerced with utf-8 - with backslash fallback for the ``__str__()`` method. - - .. change:: - :tags: bug, sql, oracle, mysql - :tickets: 4436 - - Fixed issue where the DDL emitted for :class:`.DropTableComment`, which - will be used by an upcoming version of Alembic, was incorrect for the MySQL - and Oracle databases. - - .. change:: - :tags: bug, sqlite - :tickets: 4431 - - Reflection of an index based on SQL expressions are now skipped with a - warning, in the same way as that of the Postgresql dialect, where we currently - do not support reflecting indexes that have SQL expressions within them. - Previously, an index with columns of None were produced which would break - tools like Alembic. - -.. changelog:: - :version: 1.2.15 - :released: December 11, 2018 - - .. change:: - :tags: bug, orm - :tickets: 4367 - - Fixed bug where the ORM annotations could be incorrect for the - primaryjoin/secondaryjoin a relationship if one used the pattern - ``ForeignKey(SomeClass.id)`` in the declarative mappings. This pattern - would leak undesired annotations into the join conditions which can break - aliasing operations done within :class:`_query.Query` that are not supposed to - impact elements in that join condition. These annotations are now removed - up front if present. - - .. change:: - :tags: bug, orm, declarative - :tickets: 4374 - - A warning is emitted in the case that a :func:`_expression.column` object is applied to - a declarative class, as it seems likely this intended to be a - :class:`_schema.Column` object. - - .. change:: - :tags: bug, orm - :tickets: 4366 - - In continuing with a similar theme as that of very recent :ticket:`4349`, - repaired issue with :meth:`.RelationshipProperty.Comparator.any` and - :meth:`.RelationshipProperty.Comparator.has` where the "secondary" - selectable needs to be explicitly part of the FROM clause in the - EXISTS subquery to suit the case where this "secondary" is a :class:`_expression.Join` - object. - - .. change:: - :tags: bug, orm - :tickets: 4363 - - Fixed regression caused by :ticket:`4349` where adding the "secondary" - table to the FROM clause for a dynamic loader would affect the ability of - the :class:`_query.Query` to make a subsequent join to another entity. The fix - adds the primary entity as the first element of the FROM list since - :meth:`_query.Query.join` wants to jump from that. Version 1.3 will have - a more comprehensive solution to this problem as well (:ticket:`4365`). - - - - - .. change:: - :tags: bug, orm - :tickets: 4400 - - Fixed bug where chaining of mapper options using - :meth:`.RelationshipProperty.of_type` in conjunction with a chained option - that refers to an attribute name by string only would fail to locate the - attribute. - - .. change:: - :tag: feature, mysql - :tickets: 4381 - - Added support for the ``write_timeout`` flag accepted by mysqlclient and - pymysql to be passed in the URL string. - - .. change:: - :tag: bug, postgresql - :tickets: 4377, 4380 - - Fixed issue where reflection of a PostgreSQL domain that is expressed as an - array would fail to be recognized. Pull request courtesy Jakub Synowiec. - - -.. changelog:: - :version: 1.2.14 - :released: November 10, 2018 - - .. change:: - :tags: bug, orm - :tickets: 4357 - - Fixed bug in :meth:`.Session.bulk_update_mappings` where alternate mapped - attribute names would result in the primary key column of the UPDATE - statement being included in the SET clause, as well as the WHERE clause; - while usually harmless, for SQL Server this can raise an error due to the - IDENTITY column. This is a continuation of the same bug that was fixed in - :ticket:`3849`, where testing was insufficient to catch this additional - flaw. - - .. change:: - :tags: bug, mysql - :tickets: 4361 - - Fixed regression caused by :ticket:`4344` released in 1.2.13, where the fix - for MySQL 8.0's case sensitivity problem with referenced column names when - reflecting foreign key referents is worked around using the - ``information_schema.columns`` view. The workaround was failing on OSX / - ``lower_case_table_names=2`` which produces non-matching casing for the - ``information_schema.columns`` vs. that of ``SHOW CREATE TABLE``, so in - case-insensitive SQL modes case-insensitive matching is now used. - - .. change:: - :tags: bug, orm - :tickets: 4347 - - Fixed a minor performance issue which could in some cases add unnecessary - overhead to result fetching, involving the use of ORM columns and entities - that include those same columns at the same time within a query. The issue - has to do with hash / eq overhead when referring to the column in different - ways. - -.. changelog:: - :version: 1.2.13 - :released: October 31, 2018 - - .. change:: - :tags: bug, postgresql - :tickets: 4337 - - Added support for the :class:`.aggregate_order_by` function to receive - multiple ORDER BY elements, previously only a single element was accepted. - - - .. change:: - :tags: bug, mysql - :tickets: 4348 - - Added word ``function`` to the list of reserved words for MySQL, which is - now a keyword in MySQL 8.0 - - .. change:: - :tags: feature, sql - :versions: 1.3.0b1 - - Refactored :class:`.SQLCompiler` to expose a - :meth:`.SQLCompiler.group_by_clause` method similar to the - :meth:`.SQLCompiler.order_by_clause` and :meth:`.SQLCompiler.limit_clause` - methods, which can be overridden by dialects to customize how GROUP BY - renders. Pull request courtesy Samuel Chou. - - .. change:: - :tags: bug, misc - - Fixed issue where part of the utility language helper internals was passing - the wrong kind of argument to the Python ``__import__`` builtin as the list - of modules to be imported. The issue produced no symptoms within the core - library but could cause issues with external applications that redefine the - ``__import__`` builtin or otherwise instrument it. Pull request courtesy Joe - Urciuoli. - - .. change:: - :tags: bug, orm - :tickets: 4349 - - Fixed bug where "dynamic" loader needs to explicitly set the "secondary" - table in the FROM clause of the query, to suit the case where the secondary - is a join object that is otherwise not pulled into the query from its - columns alone. - - - .. change:: - :tags: bug, orm, declarative - :tickets: 4350 - - Fixed regression caused by :ticket:`4326` in version 1.2.12 where using - :class:`.declared_attr` with a mixin in conjunction with - :func:`_orm.synonym` would fail to map the synonym properly to an inherited - subclass. - - .. change:: - :tags: bug, misc, py3k - :tickets: 4339 - - Fixed additional warnings generated by Python 3.7 due to changes in the - organization of the Python ``collections`` and ``collections.abc`` packages. - Previous ``collections`` warnings were fixed in version 1.2.11. Pull request - courtesy xtreak. - - .. change:: - :tags: bug, ext - - Added missing ``.index()`` method to list-based association collections - in the association proxy extension. - - .. change:: - :tags: bug, mysql - :tickets: 4344 - - Added a workaround for a MySQL bug #88718 introduced in the 8.0 series, - where the reflection of a foreign key constraint is not reporting the - correct case sensitivity for the referred column, leading to errors during - use of the reflected constraint such as when using the automap extension. - The workaround emits an additional query to the information_schema tables in - order to retrieve the correct case sensitive name. - - .. change:: - :tags: bug, sql - :tickets: 4341 - - Fixed bug where the :paramref:`.Enum.create_constraint` flag on the - :class:`.Enum` datatype would not be propagated to copies of the type, which - affects use cases such as declarative mixins and abstract bases. - - .. change:: - :tags: bug, orm, declarative - :tickets: 4352 - - The column conflict resolution technique discussed at - :ref:`orm_inheritance_column_conflicts` is now functional for a :class:`_schema.Column` - that is also a primary key column. Previously, a check for primary key - columns declared on a single-inheritance subclass would occur before the - column copy were allowed to pass. - - -.. changelog:: - :version: 1.2.12 - :released: September 19, 2018 - - .. change:: - :tags: bug, postgresql - :tickets: 4325 - - Fixed bug in PostgreSQL dialect where compiler keyword arguments such as - ``literal_binds=True`` were not being propagated to a DISTINCT ON - expression. - - .. change:: - :tags: bug, ext - :tickets: 4328 - - Fixed issue where :class:`.BakedQuery` did not include the specific query - class used by the :class:`.Session` as part of the cache key, leading to - incompatibilities when using custom query classes, in particular the - :class:`.ShardedQuery` which has some different argument signatures. - - .. change:: - :tags: bug, postgresql - :tickets: 4324 - - Fixed the :func:`_postgresql.array_agg` function, which is a slightly - altered version of the usual :func:`_functions.array_agg` function, to also - accept an incoming "type" argument without forcing an ARRAY around it, - essentially the same thing that was fixed for the generic function in 1.1 - in :ticket:`4107`. - - .. change:: - :tags: bug, postgresql - :tickets: 4323 - - Fixed bug in PostgreSQL ENUM reflection where a case-sensitive, quoted name - would be reported by the query including quotes, which would not match a - target column during table reflection as the quotes needed to be stripped - off. - - - .. change:: - :tags: bug, orm - - Added a check within the weakref cleanup for the :class:`.InstanceState` - object to check for the presence of the ``dict`` builtin, in an effort to - reduce error messages generated when these cleanups occur during interpreter - shutdown. Pull request courtesy Romuald Brunet. - - .. change:: - :tags: bug, orm, declarative - :tickets: 4326 - - Fixed bug where the declarative scan for attributes would receive the - expression proxy delivered by a hybrid attribute at the class level, and - not the hybrid attribute itself, when receiving the descriptor via the - ``@declared_attr`` callable on a subclass of an already-mapped class. This - would lead to an attribute that did not report itself as a hybrid when - viewed within :attr:`_orm.Mapper.all_orm_descriptors`. - - - .. change:: - :tags: bug, orm - :tickets: 4334 - :versions: 1.3.0b1 - - Fixed bug where use of :class:`_expression.Lateral` construct in conjunction with - :meth:`_query.Query.join` as well as :meth:`_query.Query.select_entity_from` would not - apply clause adaption to the right side of the join. "lateral" introduces - the use case of the right side of a join being correlatable. Previously, - adaptation of this clause wasn't considered. Note that in 1.2 only, - a selectable introduced by :meth:`_query.Query.subquery` is still not adapted - due to :ticket:`4304`; the selectable needs to be produced by the - :func:`_expression.select` function to be the right side of the "lateral" join. - - .. change:: - :tags: bug, oracle - :tickets: 4335 - - Fixed issue for cx_Oracle 7.0 where the behavior of Oracle param.getvalue() - now returns a list, rather than a single scalar value, breaking - autoincrement logic throughout the Core and ORM. The dml_ret_array_val - compatibility flag is used for cx_Oracle 6.3 and 6.4 to establish compatible - behavior with 7.0 and forward, for cx_Oracle 6.2.1 and prior a version - number check falls back to the old logic. - - - .. change:: - :tags: bug, orm - :tickets: 4327 - - Fixed 1.2 regression caused by :ticket:`3472` where the handling of an - "updated_at" style column within the context of a post-update operation - would also occur for a row that is to be deleted following the update, - meaning both that a column with a Python-side value generator would show - the now-deleted value that was emitted for the UPDATE before the DELETE - (which was not the previous behavior), as well as that a SQL- emitted value - generator would have the attribute expired, meaning the previous value - would be unreachable due to the row having been deleted and the object - detached from the session.The "postfetch" logic that was added as part of - :ticket:`3472` is now skipped entirely for an object that ultimately is to - be deleted. - -.. changelog:: - :version: 1.2.11 - :released: August 20, 2018 - - .. change:: - :tags: bug, py3k - - Started importing "collections" from "collections.abc" under Python 3.3 and - greater for Python 3.8 compatibility. Pull request courtesy Nathaniel - Knight. - - .. change:: - :tag: bug, sqlite - - Fixed issue where the "schema" name used for a SQLite database within table - reflection would not quote the schema name correctly. Pull request - courtesy Phillip Cloud. - - .. change:: - :tags: bug, sql - :tickets: 4320 - - Fixed issue that is closely related to :ticket:`3639` where an expression - rendered in a boolean context on a non-native boolean backend would - be compared to 1/0 even though it is already an implicitly boolean - expression, when :meth:`_expression.ColumnElement.self_group` were used. While this - does not affect the user-friendly backends (MySQL, SQLite) it was not - handled by Oracle (and possibly SQL Server). Whether or not the - expression is implicitly boolean on any database is now determined - up front as an additional check to not generate the integer comparison - within the compilation of the statement. - - .. change:: - :tags: bug, oracle - :tickets: 4309 - - For cx_Oracle, Integer datatypes will now be bound to "int", per advice - from the cx_Oracle developers. Previously, using cx_Oracle.NUMBER caused a - loss in precision within the cx_Oracle 6.x series. - - - .. change:: - :tags: bug, orm, declarative - :tickets: 4321 - - Fixed issue in previously untested use case, allowing a declarative mapped - class to inherit from a classically-mapped class outside of the declarative - base, including that it accommodates for unmapped intermediate classes. An - unmapped intermediate class may specify ``__abstract__``, which is now - interpreted correctly, or the intermediate class can remain unmarked, and - the classically mapped base class will be detected within the hierarchy - regardless. In order to anticipate existing scenarios which may be mixing - in classical mappings into existing declarative hierarchies, an error is - now raised if multiple mapped bases are detected for a given class. - - .. change:: - :tags: bug, sql - :tickets: 4322 - - Added missing window function parameters - :paramref:`.WithinGroup.over.range_` and :paramref:`.WithinGroup.over.rows` - parameters to the :meth:`.WithinGroup.over` and - :meth:`.FunctionFilter.over` methods, to correspond to the range/rows - feature added to the "over" method of SQL functions as part of - :ticket:`3049` in version 1.1. - - .. change:: - :tags: bug, sql - :tickets: 4313 - - Fixed bug where the multi-table support for UPDATE and DELETE statements - did not consider the additional FROM elements as targets for correlation, - when a correlated SELECT were also combined with the statement. This - change now includes that a SELECT statement in the WHERE clause for such a - statement will try to auto-correlate back to these additional tables in the - parent UPDATE/DELETE or unconditionally correlate if - :meth:`_expression.Select.correlate` is used. Note that auto-correlation raises an - error if the SELECT statement would have no FROM clauses as a result, which - can now occur if the parent UPDATE/DELETE specifies the same tables in its - additional set of tables; specify :meth:`_expression.Select.correlate` explicitly to - resolve. - -.. changelog:: - :version: 1.2.10 - :released: July 13, 2018 - - .. change:: - :tags: bug, sql - :tickets: 4300 - - Fixed bug where a :class:`.Sequence` would be dropped explicitly before any - :class:`_schema.Table` that refers to it, which breaks in the case when the - sequence is also involved in a server-side default for that table, when - using :meth:`_schema.MetaData.drop_all`. The step which processes sequences - to be dropped via non server-side column default functions is now invoked - after the table itself is dropped. - - .. change:: - :tags: bug, orm - :tickets: 4295 - - Fixed bug in :class:`.Bundle` construct where placing two columns of the - same name would be de-duplicated, when the :class:`.Bundle` were used as - part of the rendered SQL, such as in the ORDER BY or GROUP BY of the statement. - - - .. change:: - :tags: bug, orm - :tickets: 4298 - - Fixed regression in 1.2.9 due to :ticket:`4287` where using a - :class:`_orm.Load` option in conjunction with a string wildcard would result - in a TypeError. - -.. changelog:: - :version: 1.2.9 - :released: June 29, 2018 - - .. change:: - :tags: bug, mysql - - Fixed percent-sign doubling in mysql-connector-python dialect, which does - not require de-doubling of percent signs. Additionally, the mysql- - connector-python driver is inconsistent in how it passes the column names - in cursor.description, so a workaround decoder has been added to - conditionally decode these randomly-sometimes-bytes values to unicode only - if needed. Also improved test support for mysql-connector-python, however - it should be noted that this driver still has issues with unicode that - continue to be unresolved as of yet. - - - .. change:: - :tags: bug, mssql - :tickets: 4288 - - Fixed bug in MSSQL reflection where when two same-named tables in different - schemas had same-named primary key constraints, foreign key constraints - referring to one of the tables would have their columns doubled, causing - errors. Pull request courtesy Sean Dunn. - - .. change:: - :tags: bug, sql - :tickets: 4279 - - Fixed regression in 1.2 due to :ticket:`4147` where a :class:`_schema.Table` that - has had some of its indexed columns redefined with new ones, as would occur - when overriding columns during reflection or when using - :paramref:`_schema.Table.extend_existing`, such that the :meth:`_schema.Table.tometadata` - method would fail when attempting to copy those indexes as they still - referred to the replaced column. The copy logic now accommodates for this - condition. - - - .. change:: - :tags: bug, mysql - :tickets: 4293 - - Fixed bug in index reflection where on MySQL 8.0 an index that includes - ASC or DESC in an indexed column specification would not be correctly - reflected, as MySQL 8.0 introduces support for returning this information - in a table definition string. - - .. change:: - :tags: bug, orm - :tickets: 3505 - - Fixed issue where chaining multiple join elements inside of - :meth:`_query.Query.join` might not correctly adapt to the previous left-hand - side, when chaining joined inheritance classes that share the same base - class. - - .. change:: - :tags: bug, orm - :tickets: 4287 - - Fixed bug in cache key generation for baked queries which could cause a - too-short cache key to be generated for the case of eager loads across - subclasses. This could in turn cause the eagerload query to be cached in - place of a non-eagerload query, or vice versa, for a polymorphic "selectin" - load, or possibly for lazy loads or selectin loads as well. - - .. change:: - :tags: bug, sqlite - - Fixed issue in test suite where SQLite 3.24 added a new reserved word that - conflicted with a usage in TypeReflectionTest. Pull request courtesy Nils - Philippsen. - - .. change:: - :tags: feature, oracle - :tickets: 4290 - :versions: 1.3.0b1 - - Added a new event currently used only by the cx_Oracle dialect, - :meth:`.DialectEvents.setiputsizes`. The event passes a dictionary of - :class:`.BindParameter` objects to DBAPI-specific type objects that will be - passed, after conversion to parameter names, to the cx_Oracle - ``cursor.setinputsizes()`` method. This allows both visibility into the - setinputsizes process as well as the ability to alter the behavior of what - datatypes are passed to this method. - - .. seealso:: - - :ref:`cx_oracle_setinputsizes` - - .. change:: - :tags: bug, orm - :tickets: 4286 - - Fixed bug in new polymorphic selectin loading where the BakedQuery used - internally would be mutated by the given loader options, which would both - inappropriately mutate the subclass query as well as carry over the effect - to subsequent queries. - - .. change:: - :tags: bug, py3k - :tickets: 4291 - - Replaced the usage of inspect.formatargspec() with a vendored version - copied from the Python standard library, as inspect.formatargspec() - is deprecated and as of Python 3.7.0 is emitting a warning. - - .. change:: - :tags: feature, ext - :tickets: 4243 - :versions: 1.3.0b1 - - Added new attribute :attr:`_query.Query.lazy_loaded_from` which is populated - with an :class:`.InstanceState` that is using this :class:`_query.Query` in - order to lazy load a relationship. The rationale for this is that - it serves as a hint for the horizontal sharding feature to use, such that - the identity token of the state can be used as the default identity token - to use for the query within id_chooser(). - - .. change:: - :tags: bug, mysql - :tickets: 4283 - - Fixed bug in MySQLdb dialect and variants such as PyMySQL where an - additional "unicode returns" check upon connection makes explicit use of - the "utf8" character set, which in MySQL 8.0 emits a warning that utf8mb4 - should be used. This is now replaced with a utf8mb4 equivalent. - Documentation is also updated for the MySQL dialect to specify utf8mb4 in - all examples. Additional changes have been made to the test suite to use - utf8mb3 charsets and databases (there seem to be collation issues in some - edge cases with utf8mb4), and to support configuration default changes made - in MySQL 8.0 such as explicit_defaults_for_timestamp as well as new errors - raised for invalid MyISAM indexes. - - - - .. change:: - :tags: bug, mysql - :tickets: 3645 - - The :class:`_expression.Update` construct now accommodates a :class:`_expression.Join` object - as supported by MySQL for UPDATE..FROM. As the construct already - accepted an alias object for a similar purpose, the feature of UPDATE - against a non-table was already implied so this has been added. - - .. change:: - :tags: bug, mssql, py3k - :tickets: 4273 - - Fixed issue within the SQL Server dialect under Python 3 where when running - against a non-standard SQL server database that does not contain either the - "sys.dm_exec_sessions" or "sys.dm_pdw_nodes_exec_sessions" views, leading - to a failure to fetch the isolation level, the error raise would fail due - to an UnboundLocalError. - - - - .. change:: - :tags: bug, orm - :tickets: 4269 - - Fixed regression caused by :ticket:`4256` (itself a regression fix for - :ticket:`4228`) which breaks an undocumented behavior which converted for a - non-sequence of entities passed directly to the :class:`_query.Query` constructor - into a single-element sequence. While this behavior was never supported or - documented, it's already in use so has been added as a behavioral contract - to :class:`_query.Query`. - - .. change:: - :tags: bug, orm - :tickets: 4270 - - Fixed an issue that was both a performance regression in 1.2 as well as an - incorrect result regarding the "baked" lazy loader, involving the - generation of cache keys from the original :class:`_query.Query` object's loader - options. If the loader options were built up in a "branched" style using - common base elements for multiple options, the same options would be - rendered into the cache key repeatedly, causing both a performance issue as - well as generating the wrong cache key. This is fixed, along with a - performance improvement when such "branched" options are applied via - :meth:`_query.Query.options` to prevent the same option objects from being - applied repeatedly. - - .. change:: - :tags: bug, oracle, mysql - :tickets: 4275 - - Fixed INSERT FROM SELECT with CTEs for the Oracle and MySQL dialects, where - the CTE was being placed above the entire statement as is typical with - other databases, however Oracle and MariaDB 10.2 wants the CTE underneath - the "INSERT" segment. Note that the Oracle and MySQL dialects don't yet - work when a CTE is applied to a subquery inside of an UPDATE or DELETE - statement, as the CTE is still applied to the top rather than inside the - subquery. - - -.. changelog:: - :version: 1.2.8 - :released: May 28, 2018 - - .. change:: - :tags: bug, orm - :tickets: 4256 - - Fixed regression in 1.2.7 caused by :ticket:`4228`, which itself was fixing - a 1.2-level regression, where the ``query_cls`` callable passed to a - :class:`.Session` was assumed to be a subclass of :class:`_query.Query` with - class method availability, as opposed to an arbitrary callable. In - particular, the dogpile caching example illustrates ``query_cls`` as a - function and not a :class:`_query.Query` subclass. - - .. change:: - :tags: bug, engine - :tickets: 4252 - - Fixed connection pool issue whereby if a disconnection error were raised - during the connection pool's "reset on return" sequence in conjunction with - an explicit transaction opened against the enclosing :class:`_engine.Connection` - object (such as from calling :meth:`.Session.close` without a rollback or - commit, or calling :meth:`_engine.Connection.close` without first closing a - transaction declared with :meth:`_engine.Connection.begin`), a double-checkin would - result, which could then lead towards concurrent checkouts of the same - connection. The double-checkin condition is now prevented overall by an - assertion, as well as the specific double-checkin scenario has been - fixed. - - .. change:: - :tags: bug, oracle - :tickets: 4264 - - The Oracle BINARY_FLOAT and BINARY_DOUBLE datatypes now participate within - cx_Oracle.setinputsizes(), passing along NATIVE_FLOAT, so as to support the - NaN value. Additionally, :class:`_oracle.BINARY_FLOAT`, - :class:`_oracle.BINARY_DOUBLE` and :class:`_oracle.DOUBLE_PRECISION` now - subclass :class:`.Float`, since these are floating point datatypes, not - decimal. These datatypes were already defaulting the - :paramref:`.Float.asdecimal` flag to False in line with what - :class:`.Float` already does. - - .. change:: - :tags: bug, oracle - - Added reflection capabilities for the :class:`_oracle.BINARY_FLOAT`, - :class:`_oracle.BINARY_DOUBLE` datatypes. - - - .. change:: - :tags: bug, ext - :tickets: 4247 - - The horizontal sharding extension now makes use of the identity token - added to ORM identity keys as part of :ticket:`4137`, when an object - refresh or column-based deferred load or unexpiration operation occurs. - Since we know the "shard" that the object originated from, we make - use of this value when refreshing, thereby avoiding queries against - other shards that don't match this object's identity in any case. - - .. change:: - :tags: bug, sql - - Fixed issue where the "ambiguous literal" error message used when - interpreting literal values as SQL expression values would encounter a - tuple value, and fail to format the message properly. Pull request courtesy - Miguel Ventura. - - .. change:: - :tags: bug, mssql - :tickets: 4250 - - Fixed a 1.2 regression caused by :ticket:`4061` where the SQL Server - "BIT" type would be considered to be "native boolean". The goal here - was to avoid creating a CHECK constraint on the column, however the bigger - issue is that the BIT value does not behave like a true/false constant - and cannot be interpreted as a standalone expression, e.g. - "WHERE ". The SQL Server dialect now goes back to being - non-native boolean, but with an extra flag that still avoids creating - the CHECK constraint. - - .. change:: - :tags: bug, oracle - :tickets: 4259 - - Altered the Oracle dialect such that when an :class:`.Integer` type is in - use, the cx_Oracle.NUMERIC type is set up for setinputsizes(). In - SQLAlchemy 1.1 and earlier, cx_Oracle.NUMERIC was passed for all numeric - types unconditionally, and in 1.2 this was removed to allow for better - numeric precision. However, for integers, some database/client setups - will fail to coerce boolean values True/False into integers which introduces - regressive behavior when using SQLAlchemy 1.2. Overall, the setinputsizes - logic seems like it will need a lot more flexibility going forward so this - is a start for that. - - .. change:: - :tags: bug, engine - - Fixed a reference leak issue where the values of the parameter dictionary - used in a statement execution would remain referenced by the "compiled - cache", as a result of storing the key view used by Python 3 dictionary - keys(). Pull request courtesy Olivier Grisel. - - .. change:: - :tags: bug, orm - :tickets: 4128 - - Fixed a long-standing regression that occurred in version - 1.0, which prevented the use of a custom :class:`.MapperOption` - that alters the _params of a :class:`_query.Query` object for a - lazy load, since the lazy loader itself would overwrite those - parameters. This applies to the "temporal range" example - on the wiki. Note however that the - :meth:`_query.Query.populate_existing` method is now required in - order to rewrite the mapper options associated with an object - already loaded in the identity map. - - As part of this change, a custom defined - :class:`.MapperOption` will now cause lazy loaders related to - the target object to use a non-baked query by default unless - the :meth:`.MapperOption._generate_cache_key` method is implemented. - In particular, this repairs one regression which occurred when - using the dogpile.cache "advanced" example, which was not - returning cached results and instead emitting SQL due to an - incompatibility with the baked query loader; with the change, - the ``RelationshipCache`` option included for many releases - in the dogpile example will disable the "baked" query altogether. - Note that the dogpile example is also modernized to avoid both - of these issues as part of issue :ticket:`4258`. - - .. change:: - :tags: bug, ext - :tickets: 4266 - - Fixed a race condition which could occur if automap - :meth:`.AutomapBase.prepare` were used within a multi-threaded context - against other threads which may call :func:`.configure_mappers` as a - result of use of other mappers. The unfinished mapping work of automap - is particularly sensitive to being pulled in by a - :func:`.configure_mappers` step leading to errors. - - .. change:: - :tags: bug, orm - - Fixed bug where the new :meth:`.baked.Result.with_post_criteria` - method would not interact with a subquery-eager loader correctly, - in that the "post criteria" would not be applied to embedded - subquery eager loaders. This is related to :ticket:`4128` in that - the post criteria feature is now used by the lazy loader. - - .. change:: - :tags: bug, tests - :tickets: 4249 - - Fixed a bug in the test suite where if an external dialect returned - ``None`` for ``server_version_info``, the exclusion logic would raise an - ``AttributeError``. - - .. change:: - :tags: bug, orm - :tickets: 4258 - - Updated the dogpile.caching example to include new structures that - accommodate for the "baked" query system, which is used by default within - lazy loaders and some eager relationship loaders. The dogpile.caching - "relationship_caching" and "advanced" examples were also broken due to - :ticket:`4256`. The issue here is also worked-around by the fix in - :ticket:`4128`. - -.. changelog:: - :version: 1.2.7 - :released: April 20, 2018 - - .. change:: - :tags: bug, orm - :tickets: 4228 - - Fixed regression in 1.2 within sharded query feature where the - new "identity_token" element was not being correctly considered within - the scope of a lazy load operation, when searching the identity map - for a related many-to-one element. The new behavior will allow for - making use of the "id_chooser" in order to determine the best identity - key to retrieve from the identity map. In order to achieve this, some - refactoring of 1.2's "identity_token" approach has made some slight changes - to the implementation of ``ShardedQuery`` which should be noted for other - derivations of this class. - - .. change:: - :tags: bug, postgresql - :tickets: 4229 - - Fixed bug where the special "not equals" operator for the PostgreSQL - "range" datatypes such as DATERANGE would fail to render "IS NOT NULL" when - compared to the Python ``None`` value. - - - - .. change:: - :tags: bug, mssql - :tickets: 4234 - - Fixed 1.2 regression caused by :ticket:`4060` where the query used to - reflect SQL Server cross-schema foreign keys was limiting the criteria - incorrectly. - - - - .. change:: - :tags: bug, oracle - - The Oracle NUMBER datatype is reflected as INTEGER if the precision is NULL - and the scale is zero, as this is how INTEGER values come back when - reflected from Oracle's tables. Pull request courtesy Kent Bower. - - .. change:: - :tags: feature, postgresql - :tickets: 4160 - :versions: 1.3.0b1 - - Added new PG type :class:`_postgresql.REGCLASS` which assists in casting - table names to OID values. Pull request courtesy Sebastian Bank. - - .. change:: - :tags: bug, sql - :tickets: 4231 - - Fixed issue where the compilation of an INSERT statement with the - "literal_binds" option that also uses an explicit sequence and "inline" - generation, as on PostgreSQL and Oracle, would fail to accommodate the - extra keyword argument within the sequence processing routine. - - .. change:: - :tags: bug, orm - :tickets: 4241 - - Fixed issue in single-inheritance loading where the use of an aliased - entity against a single-inheritance subclass in conjunction with the - :meth:`_query.Query.select_from` method would cause the SQL to be rendered with - the unaliased table mixed in to the query, causing a cartesian product. In - particular this was affecting the new "selectin" loader when used against a - single-inheritance subclass. - -.. changelog:: - :version: 1.2.6 - :released: March 30, 2018 - - .. change:: - :tags: bug, mssql - :tickets: 4227 - - Adjusted the SQL Server version detection for pyodbc to only allow for - numeric tokens, filtering out non-integers, since the dialect does tuple- - numeric comparisons with this value. This is normally true for all known - SQL Server / pyodbc drivers in any case. - - .. change:: - :tags: feature, postgresql - - Added support for "PARTITION BY" in PostgreSQL table definitions, - using "postgresql_partition_by". Pull request courtesy - Vsevolod Solovyov. - - .. change:: - :tags: bug, sql - :tickets: 4204 - - Fixed a regression that occurred from the previous fix to :ticket:`4204` in - version 1.2.5, where a CTE that refers to itself after the - :meth:`_expression.CTE.alias` method has been called would not refer to itself - correctly. - - .. change:: - :tags: bug, engine - :tickets: 4225 - - Fixed bug in connection pool where a connection could be present in the - pool without all of its "connect" event handlers called, if a previous - "connect" handler threw an exception; note that the dialects themselves - have connect handlers that emit SQL, such as those which set transaction - isolation, which can fail if the database is in a non-available state, but - still allows a connection. The connection is now invalidated first if any - of the connect handlers fail. - - .. change:: - :tags: bug, oracle - :tickets: 4211 - - The minimum cx_Oracle version supported is 5.2 (June 2015). Previously, - the dialect asserted against version 5.0 but as of 1.2.2 we are using some - symbols that did not appear until 5.2. - - .. change:: - :tags: bug, declarative - :tickets: 4221 - - Removed a warning that would be emitted when calling upon - ``__table_args__``, ``__mapper_args__`` as named with a ``@declared_attr`` - method, when called from a non-mapped declarative mixin. Calling these - directly is documented as the approach to use when one is overriding one - of these methods on a mapped class. The warning still emits for regular - attribute names. - - .. change:: - :tags: bug, orm - :tickets: 4215 - - Fixed bug where using :meth:`.Mutable.associate_with` or - :meth:`.Mutable.as_mutable` in conjunction with a class that has non- - primary mappers set up with alternatively-named attributes would produce an - attribute error. Since non-primary mappers are not used for persistence, - the mutable extension now excludes non-primary mappers from its - instrumentation steps. - - -.. changelog:: - :version: 1.2.5 - :released: March 6, 2018 - - .. change:: - :tags: bug, sql - :tickets: 4210 - - Fixed bug in :class:.`CTE` construct along the same lines as that of - :ticket:`4204` where a :class:`_expression.CTE` that was aliased would not copy itself - correctly during a "clone" operation as is frequent within the ORM as well - as when using the :meth:`_expression.ClauseElement.params` method. - - .. change:: - :tags: bug, orm - :tickets: 4199 - - Fixed bug in new "polymorphic selectin" loading when a selection of - polymorphic objects were to be partially loaded from a relationship - lazy loader, leading to an "empty IN" condition within the load that - raises an error for the "inline" form of "IN". - - .. change:: - :tags: bug, sql - :tickets: 4204 - - Fixed bug in CTE rendering where a :class:`_expression.CTE` that was also turned into - an :class:`_expression.Alias` would not render its "ctename AS aliasname" clause - appropriately if there were more than one reference to the CTE in a FROM - clause. - - .. change:: - :tags: bug, orm - :tickets: 4209 - - Fixed 1.2 regression where a mapper option that contains an - :class:`.AliasedClass` object, as is typical when using the - :meth:`.QueryableAttribute.of_type` method, could not be pickled. 1.1's - behavior was to omit the aliased class objects from the path, so this - behavior is restored. - - .. change:: - :tags: feature, orm - :versions: 1.3.0b1 - - Added new feature :meth:`_query.Query.only_return_tuples`. Causes the - :class:`_query.Query` object to return keyed tuple objects unconditionally even - if the query is against a single entity. Pull request courtesy Eric - Atkin. - - - .. change:: - :tags: bug, sql - :tickets: 4198 - - Fixed bug in new "expanding IN parameter" feature where the bind parameter - processors for values wasn't working at all, tests failed to cover this - pretty basic case which includes that ENUM values weren't working. - -.. changelog:: - :version: 1.2.4 - :released: February 22, 2018 - - .. change:: - :tags: bug, orm - :tickets: 4193 - - Fixed 1.2 regression in ORM versioning feature where a mapping against a - :func:`_expression.select` or :func:`.alias` that also used a versioning column - against the underlying table would fail due to the check added as part of - :ticket:`3673`. - - .. change:: - :tags: bug, engine - :tickets: 4190 - - Fixed regression caused in 1.2.3 due to fix from :ticket:`4181` where - the changes to the event system involving :class:`_engine.Engine` and - :class:`.OptionEngine` did not accommodate for event removals, which - would raise an ``AttributeError`` when invoked at the class - level. - - .. change:: - :tags: bug, sql - :tickets: 4197 - - Fixed bug where CTE expressions would not have their name or alias name - quoted when the given name is case sensitive or otherwise requires quoting. - Pull request courtesy Eric Atkin. - -.. changelog:: - :version: 1.2.3 - :released: February 16, 2018 - - .. change:: - :tags: bug, oracle - :tickets: 4182 - - Fixed bug in cx_Oracle disconnect detection, used by pre_ping and other - features, where an error could be raised as DatabaseError which includes a - numeric error code; previously we weren't checking in this case for a - disconnect code. - - .. change:: - :tags: bug, sqlite - - Fixed the import error raised when a platform - has neither pysqlite2 nor sqlite3 installed, such - that the sqlite3-related import error is raised, - not the pysqlite2 one which is not the actual - failure mode. Pull request courtesy Robin. - - .. change:: - :tags: bug, orm - :tickets: 4175 - - Fixed bug where the :class:`.Bundle` object did not - correctly report upon the primary :class:`_orm.Mapper` object - represented by the bundle, if any. An immediate - side effect of this issue was that the new selectinload - loader strategy wouldn't work with the horizontal sharding - extension. - - .. change:: - :tags: bug, sql - :tickets: 4180 - - Fixed bug where the :class:`.Enum` type wouldn't handle - enum "aliases" correctly, when more than one key refers to the - same value. Pull request courtesy Daniel Knell. - - - .. change:: - :tags: bug, engine - :tickets: 4181 - - Fixed bug where events associated with an :class:`Engine` - at the class level would be doubled when the - :meth:`_engine.Engine.execution_options` method were used. To - achieve this, the semi-private class :class:`.OptionEngine` - no longer accepts events directly at the class level - and will raise an error; the class only propagates class-level - events from its parent :class:`_engine.Engine`. Instance-level - events continue to work as before. - - .. change:: - :tags: bug, tests - :tickets: 3265 - - A test added in 1.2 thought to confirm a Python 2.7 behavior turns out to - be confirming the behavior only as of Python 2.7.8. Python bug #8743 still - impacts set comparison in Python 2.7.7 and earlier, so the test in question - involving AssociationSet no longer runs for these older Python 2.7 - versions. - - .. change:: - :tags: feature, oracle - - The ON DELETE options for foreign keys are now part of - Oracle reflection. Oracle does not support ON UPDATE - cascades. Pull request courtesy Miroslav Shubernetskiy. - - - - .. change:: - :tags: bug, orm - :tickets: 4188 - - Fixed bug in concrete inheritance mapping where user-defined - attributes such as hybrid properties that mirror the names - of mapped attributes from sibling classes would be overwritten by - the mapper as non-accessible at the instance level. Additionally - ensured that user-bound descriptors are not implicitly invoked at the class - level during the mapper configuration stage. - - .. change:: - :tags: bug, orm - :tickets: 4178 - - Fixed bug where the :func:`_orm.reconstructor` event - helper would not be recognized if it were applied to the - ``__init__()`` method of the mapped class. - - .. change:: - :tags: bug, engine - :tickets: 4170 - - The :class:`.URL` object now allows query keys to be specified multiple - times where their values will be joined into a list. This is to support - the plugins feature documented at :class:`.CreateEnginePlugin` which - documents that "plugin" can be passed multiple times. Additionally, the - plugin names can be passed to :func:`_sa.create_engine` outside of the URL - using the new :paramref:`_sa.create_engine.plugins` parameter. - - .. change:: - :tags: feature, sql - :tickets: 3906 - - Added support for :class:`.Enum` to persist the values of the enumeration, - rather than the keys, when using a Python pep-435 style enumerated object. - The user supplies a callable function that will return the string values to - be persisted. This allows enumerations against non-string values to be - value-persistable as well. Pull request courtesy Jon Snyder. - - .. change:: - :tags: feature, orm - - Added new argument :paramref:`.attributes.set_attribute.inititator` - to the :func:`.attributes.set_attribute` function, allowing an - event token received from a listener function to be propagated - to subsequent set events. - -.. changelog:: - :version: 1.2.2 - :released: January 24, 2018 - - .. change:: - :tags: bug, mssql - :tickets: 4164 - - Added ODBC error code 10054 to the list of error - codes that count as a disconnect for ODBC / MSSQL server. - - - .. change:: - :tags: bug, orm - :tickets: 4171 - - Fixed 1.2 regression regarding new bulk_replace event - where a backref would fail to remove an object from the - previous owner when a bulk-assignment assigned the - object to a new owner. - - .. change:: - :tags: bug, oracle - :tickets: 4163 - - The cx_Oracle dialect now calls setinputsizes() with cx_Oracle.NCHAR - unconditionally when the NVARCHAR2 datatype, in SQLAlchemy corresponding - to sqltypes.Unicode(), is in use. Per cx_Oracle's author this allows - the correct conversions to occur within the Oracle client regardless - of the setting for NLS_NCHAR_CHARACTERSET. - - .. change:: - :tags: bug, mysql - - Added more MySQL 8.0 reserved words to the MySQL dialect - for quoting purposes. Pull request courtesy - Riccardo Magliocchetti. - -.. changelog:: - :version: 1.2.1 - :released: January 15, 2018 - - .. change:: - :tags: bug, orm - :tickets: 4159 - - Fixed regression where pickle format of a Load / _UnboundLoad object (e.g. - loader options) changed and ``__setstate__()`` was raising an - UnboundLocalError for an object received from the legacy format, even - though an attempt was made to do so. tests are now added to ensure this - works. - - .. change:: - :tags: bug, ext - :tickets: 4150 - - Fixed regression in association proxy due to :ticket:`3769` - (allow for chained any() / has()) where contains() against - an association proxy chained in the form - (o2m relationship, associationproxy(m2o relationship, m2o relationship)) - would raise an error regarding the re-application of contains() - on the final link of the chain. - - .. change:: - :tags: bug, orm - :tickets: 4153 - - Fixed regression caused by new lazyload caching scheme in :ticket:`3954` - where a query that makes use of loader options with of_type would cause - lazy loads of unrelated paths to fail with a TypeError. - - .. change:: - :tags: bug, oracle - :tickets: 4157 - - Fixed regression where the removal of most setinputsizes - rules from cx_Oracle dialect impacted the TIMESTAMP - datatype's ability to retrieve fractional seconds. - - - - .. change:: - :tags: bug, tests - - Removed an oracle-specific requirements rule from the public - test suite that was interfering with third party dialect - suites. - - .. change:: - :tags: bug, mssql - :tickets: 4154 - - Fixed regression in 1.2 where newly repaired quoting - of collation names in :ticket:`3785` breaks SQL Server, - which explicitly does not understand a quoted collation - name. Whether or not mixed-case collation names are - quoted or not is now deferred down to a dialect-level - decision so that each dialect can prepare these identifiers - directly. - - .. change:: - :tags: bug, orm - :tickets: 4156 - - Fixed bug in new "selectin" relationship loader where the loader could try - to load a non-existent relationship when loading a collection of - polymorphic objects, where only some of the mappers include that - relationship, typically when :meth:`.PropComparator.of_type` is being used. - - .. change:: - :tags: bug, tests - - Added a new exclusion rule group_by_complex_expression - which disables tests that use "GROUP BY ", which seems - to be not viable for at least two third party dialects. - - .. change:: - :tags: bug, oracle - - Fixed regression in Oracle imports where a missing comma caused - an undefined symbol to be present. Pull request courtesy - Miroslav Shubernetskiy. - -.. changelog:: - :version: 1.2.0 - :released: December 27, 2017 - - .. change:: - :tags: orm, feature - :tickets: 4137 - - Added a new data member to the identity key tuple - used by the ORM's identity map, known as the - "identity_token". This token defaults to None but - may be used by database sharding schemes to differentiate - objects in memory with the same primary key that come - from different databases. The horizontal sharding - extension integrates this token applying the shard - identifier to it, thus allowing primary keys to be - duplicated across horizontally sharded backends. - - .. seealso:: - - :ref:`change_4137` - - .. change:: - :tags: bug, mysql - :tickets: 4115 - - Fixed regression from issue 1.2.0b3 where "MariaDB" version comparison can - fail for some particular MariaDB version strings under Python 3. - - .. change:: - :tags: enhancement, sql - :tickets: 959 - - Implemented "DELETE..FROM" syntax for PostgreSQL, MySQL, MS SQL Server - (as well as within the unsupported Sybase dialect) in a manner similar - to how "UPDATE..FROM" works. A DELETE statement that refers to more than - one table will switch into "multi-table" mode and render the appropriate - "USING" or multi-table "FROM" clause as understood by the database. - Pull request courtesy Pieter Mulder. - - .. seealso:: - - :ref:`change_959` - - .. change:: - :tags: bug, sql - :tickets: 2694 - - Reworked the new "autoescape" feature introduced in - :ref:`change_2694` in 1.2.0b2 to be fully automatic; the escape - character now defaults to a forwards slash ``"/"`` and - is applied to percent, underscore, as well as the escape - character itself, for fully automatic escaping. The - character can also be changed using the "escape" parameter. - - .. seealso:: - - :ref:`change_2694` - - - .. change:: - :tags: bug, sql - :tickets: 4147 - - Fixed bug where the :meth:`_schema.Table.tometadata` method would not properly - accommodate :class:`.Index` objects that didn't consist of simple - column expressions, such as indexes against a :func:`_expression.text` construct, - indexes that used SQL expressions or :attr:`.func`, etc. The routine - now copies expressions fully to a new :class:`.Index` object while - substituting all table-bound :class:`_schema.Column` objects for those - of the target table. - - .. change:: - :tags: bug, sql - :tickets: 4142 - - Changed the "visit name" of :class:`_expression.ColumnElement` from "column" to - "column_element", so that when this element is used as the basis for a - user-defined SQL element, it is not assumed to behave like a table-bound - :class:`.ColumnClause` when processed by various SQL traversal utilities, - as are commonly used by the ORM. - - .. change:: - :tags: bug, sql, ext - :tickets: 4141 - - Fixed issue in :class:`_types.ARRAY` datatype which is essentially the same - issue as that of :ticket:`3832`, except not a regression, where - column attachment events on top of :class:`_types.ARRAY` would not fire - correctly, thus interfering with systems which rely upon this. A key - use case that was broken by this is the use of mixins to declare - columns that make use of :meth:`.MutableList.as_mutable`. - - .. change:: - :tags: feature, engine - :tickets: 4089 - - The "password" attribute of the :class:`.url.URL` object can now be - any user-defined or user-subclassed string object that responds to the - Python ``str()`` builtin. The object passed will be maintained as the - datamember :attr:`.url.URL.password_original` and will be consulted - when the :attr:`.url.URL.password` attribute is read to produce the - string value. - - .. change:: - :tags: bug, orm - :tickets: 4130 - - Fixed bug in :func:`.contains_eager` query option where making use of a - path that used :meth:`.PropComparator.of_type` to refer to a subclass - across more than one level of joins would also require that the "alias" - argument were provided with the same subtype in order to avoid adding - unwanted FROM clauses to the query; additionally, using - :func:`.contains_eager` across subclasses that use :func:`.aliased` objects - of subclasses as the :meth:`.PropComparator.of_type` argument will also - render correctly. - - - - - .. change:: - :tags: feature, postgresql - - Added new :class:`_postgresql.MONEY` datatype. Pull request courtesy - Cleber J Santos. - - .. change:: - :tags: bug, sql - :tickets: 4140 - - Fixed bug in new "expanding bind parameter" feature whereby if multiple - params were used in one statement, the regular expression would not - match the parameter name correctly. - - .. change:: - :tags: enhancement, ext - :tickets: 4135 - - Added new method :meth:`.baked.Result.with_post_criteria` to baked - query system, allowing non-SQL-modifying transformations to take place - after the query has been pulled from the cache. Among other things, - this method can be used with :class:`.horizontal_shard.ShardedQuery` - to set the shard identifier. :class:`.horizontal_shard.ShardedQuery` - has also been modified such that its :meth:`.ShardedQuery.get` method - interacts correctly with that of :class:`_baked.Result`. - - .. change:: - :tags: bug, oracle - :tickets: 4064 - - Added some additional rules to fully handle ``Decimal('Infinity')``, - ``Decimal('-Infinity')`` values with cx_Oracle numerics when using - ``asdecimal=True``. - - .. change:: - :tags: bug, mssql - :tickets: 4121 - - Fixed bug where sqltypes.BINARY and sqltypes.VARBINARY datatypes - would not include correct bound-value handlers for pyodbc, - which allows the pyodbc.NullParam value to be passed that - helps with FreeTDS. - - - - - .. change:: - :tags: feature, misc - - Added a new errors section to the documentation with background - about common error messages. Selected exceptions within SQLAlchemy - will include a link in their string output to the relevant section - within this page. - - .. change:: - :tags: bug, orm - :tickets: 4032 - - The :meth:`_query.Query.exists` method will now disable eager loaders for when - the query is rendered. Previously, joined-eager load joins would be rendered - unnecessarily as well as subquery eager load queries would be needlessly - generated. The new behavior matches that of the :meth:`_query.Query.subquery` - method. - -.. changelog:: - :version: 1.2.0b3 - :released: December 27, 2017 - :released: October 13, 2017 - - .. change:: - :tags: feature, postgresql - :tickets: 4109 - - Added a new flag ``use_batch_mode`` to the psycopg2 dialect. This flag - enables the use of psycopg2's ``psycopg2.extras.execute_batch`` - extension when the :class:`_engine.Engine` calls upon - ``cursor.executemany()``. This extension provides a critical - performance increase by over an order of magnitude when running INSERT - statements in batch. The flag is False by default as it is considered - to be experimental for now. - - .. seealso:: - - :ref:`change_4109` - - .. change:: - :tags: bug, mssql - :tickets: 4061 - - SQL Server supports what SQLAlchemy calls "native boolean" - with its BIT type, as this type only accepts 0 or 1 and the - DBAPIs return its value as True/False. So the SQL Server - dialects now enable "native boolean" support, in that a - CHECK constraint is not generated for a :class:`.Boolean` - datatype. The only difference vs. other native boolean - is that there are no "true" / "false" constants so "1" and - "0" are still rendered here. - - - .. change:: - :tags: bug, oracle - :tickets: 4064 - - Partial support for persisting and retrieving the Oracle value - "infinity" is implemented with cx_Oracle, using Python float values - only, e.g. ``float("inf")``. Decimal support is not yet fulfilled by - the cx_Oracle DBAPI driver. - - .. change:: - :tags: bug, oracle - - The cx_Oracle dialect has been reworked and modernized to take advantage of - new patterns that weren't present in the old 4.x series of cx_Oracle. This - includes that the minimum cx_Oracle version is the 5.x series and that - cx_Oracle 6.x is now fully tested. The most significant change involves - type conversions, primarily regarding the numeric / floating point and LOB - datatypes, making more effective use of cx_Oracle type handling hooks to - simplify how bind parameter and result data is processed. - - .. seealso:: - - :ref:`change_cxoracle_12` - - .. change:: - :tags: bug, oracle - :tickets: 3997 - - two phase support for cx_Oracle has been completely removed for all - versions of cx_Oracle, whereas in 1.2.0b1 this change only took effect for - the 6.x series of cx_Oracle. This feature never worked correctly - in any version of cx_Oracle and in cx_Oracle 6.x, the API which SQLAlchemy - relied upon was removed. - - .. seealso:: - - :ref:`change_cxoracle_12` - - .. change:: - :tags: bug, oracle - - The column keys present in a result set when using :meth:`_expression.Insert.returning` - with the cx_Oracle backend now use the correct column / label names - like that of all other dialects. Previously, these came out as - ``ret_nnn``. - - .. seealso:: - - :ref:`change_cxoracle_12` - - .. change:: - :tags: bug, oracle - - Several parameters to the cx_Oracle dialect are now deprecated and will - have no effect: ``auto_setinputsizes``, ``exclude_setinputsizes``, - ``allow_twophase``. - - .. seealso:: - - :ref:`change_cxoracle_12` - - - .. change:: - :tags: bug, sql - :tickets: 4075 - - Added a new method :meth:`.DefaultExecutionContext.get_current_parameters` - which is used within a function-based default value generator in - order to retrieve the current parameters being passed to the statement. - The new function differs from the - :attr:`.DefaultExecutionContext.current_parameters` attribute in - that it also provides for optional grouping of parameters that - correspond to a multi-valued "insert" construct. Previously it was not - possible to identify the subset of parameters that were relevant to - the function call. - - .. seealso:: - - :ref:`change_4075` - - :ref:`context_default_functions` - - .. change:: - :tags: bug, orm - :tickets: 4050 - - Fixed regression introduced in 1.2.0b1 due to :ticket:`3934` where the - :class:`.Session` would fail to "deactivate" the transaction, if a - rollback failed (the target issue is when MySQL loses track of a SAVEPOINT). - This would cause a subsequent call to :meth:`.Session.rollback` to raise - an error a second time, rather than completing and bringing the - :class:`.Session` back to ACTIVE. - - .. change:: - :tags: bug, postgresql - :tickets: 4041 - - Fixed bug where the pg8000 driver would fail if using - :meth:`_schema.MetaData.reflect` with a schema name, since the schema name would - be sent as a "quoted_name" object that's a string subclass, which pg8000 - doesn't recognize. The quoted_name type is added to pg8000's - py_types collection on connect. - - .. change:: - :tags: bug, postgresql - :tickets: 4016 - - Enabled UUID support for the pg8000 driver, which supports native Python - uuid round trips for this datatype. Arrays of UUID are still not supported, - however. - - .. change:: - :tags: mssql, bug - :tickets: 4057 - - Fixed the pymssql dialect so that percent signs in SQL text, such - as used in modulus expressions or literal textual values, are - **not** doubled up, as seems to be what pymssql expects. This is - despite the fact that the pymssql DBAPI uses the "pyformat" parameter - style which itself considers the percent sign to be significant. - - .. change:: - :tags: bug, orm, declarative - :tickets: 4091 - - A warning is emitted if a subclass attempts to override an attribute - that was declared on a superclass using ``@declared_attr.cascading`` - that the overridden attribute will be ignored. This use - case cannot be fully supported down to further subclasses without more - complex development efforts, so for consistency the "cascading" is - honored all the way down regardless of overriding attributes. - - .. change:: - :tags: bug, orm, declarative - :tickets: 4092 - - A warning is emitted if the ``@declared_attr.cascading`` attribute is - used with a special declarative name such as ``__tablename__``, as this - has no effect. - - .. change:: - :tags: feature, engine - :tickets: 4077 - - Added ``__next__()`` and ``next()`` methods to :class:`_engine.ResultProxy`, - so that the ``next()`` builtin function works on the object directly. - :class:`_engine.ResultProxy` has long had an ``__iter__()`` method which already - allows it to respond to the ``iter()`` builtin. The implementation - for ``__iter__()`` is unchanged, as performance testing has indicated - that iteration using a ``__next__()`` method with ``StopIteration`` - is about 20% slower in both Python 2.7 and 3.6. - - .. change:: - :tags: feature, mssql - :tickets: 4086 - - Added a new :class:`_mssql.TIMESTAMP` datatype, that - correctly acts like a binary datatype for SQL Server - rather than a datetime type, as SQL Server breaks the - SQL standard here. Also added :class:`_mssql.ROWVERSION`, - as the "TIMESTAMP" type in SQL Server is deprecated in - favor of ROWVERSION. - - .. change:: - :tags: bug, orm - :tickets: 4084 - - Fixed issue where the :func:`.make_transient_to_detached` function - would expire all attributes on the target object, including "deferred" - attributes, which has the effect of the attribute being undeferred - for the next refresh, causing an unexpected load of the attribute. - - .. change:: - :tags: bug, orm - :tickets: 4040 - - Fixed bug involving delete-orphan cascade where a related item - that becomes an orphan before the parent object is part of a - session is still tracked as moving into orphan status, which results - in it being expunged from the session rather than being flushed. - - .. note:: This fix was inadvertently merged during the 1.2.0b3 - release and was **not added to the changelog** at that time. - This changelog note was added to the release retroactively as of - version 1.2.13. - - .. change:: - :tags: bug, orm - :tickets: 4026 - - Fixed bug in :ref:`change_3948` which prevented "selectin" and - "inline" settings in a multi-level class hierarchy from interacting - together as expected. - - .. change:: - :tags: bug, oracle - :tickets: 4042 - - Fixed bug where an index reflected under Oracle with an expression like - "column DESC" would not be returned, if the table also had no primary - key, as a result of logic that attempts to filter out the - index implicitly added by Oracle onto the primary key columns. - - .. change:: - :tags: bug, orm - :tickets: 4071 - - Removed the warnings that are emitted when the LRU caches employed - by the mapper as well as loader strategies reach their threshold; the - purpose of this warning was at first a guard against excess cache keys - being generated but became basically a check on the "creating many - engines" antipattern. While this is still an antipattern, the presence - of test suites which both create an engine per test as well as raise - on all warnings will be an inconvenience; it should not be critical - that such test suites change their architecture just for this warning - (though engine-per-test suite is always better). - - .. change:: - :tags: bug, orm - :tickets: 4049 - - Fixed regression where the use of a :func:`.undefer_group` option - in conjunction with a lazy loaded relationship option would cause - an attribute error, due to a bug in the SQL cache key generation - added in 1.2 as part of :ticket:`3954`. - - .. change:: - :tags: bug, oracle - :tickets: 4045 - - Fixed more regressions caused by cx_Oracle 6.0; at the moment, the only - behavioral change for users is disconnect detection now detects for - cx_Oracle.DatabaseError in addition to cx_Oracle.InterfaceError, as - this behavior seems to have changed. Other issues regarding numeric - precision and uncloseable connections are pending with the upstream - cx_Oracle issue tracker. - - .. change:: - :tags: bug, mssql - :tickets: 4060 - - Fixed bug where the SQL Server dialect could pull columns from multiple - schemas when reflecting a self-referential foreign key constraint, if - multiple schemas contained a constraint of the same name against a - table of the same name. - - - .. change:: - :tags: feature, mssql - :tickets: 4058 - - Added support for "AUTOCOMMIT" isolation level, as established - via :meth:`_engine.Connection.execution_options`, to the - PyODBC and pymssql dialects. This isolation level sets the - appropriate DBAPI-specific flags on the underlying - connection object. - - .. change:: - :tags: bug, orm - :tickets: 4073 - - Modified the change made to the ORM update/delete evaluator in - :ticket:`3366` such that if an unmapped column expression is present - in the update or delete, if the evaluator can match its name to the - mapped columns of the target class, a warning is emitted, rather than - raising UnevaluatableError. This is essentially the pre-1.2 behavior, - and is to allow migration for applications that are currently relying - upon this pattern. However, if the given attribute name cannot be - matched to the columns of the mapper, the UnevaluatableError is - still raised, which is what was fixed in :ticket:`3366`. - - .. change:: - :tags: bug, sql - :tickets: 4087 - - Fixed bug in new SQL comments feature where table and column comment - would not be copied when using :meth:`_schema.Table.tometadata`. - - .. change:: - :tags: bug, sql - :tickets: 4102 - - In release 1.1, the :class:`.Boolean` type was broken in that - boolean coercion via ``bool()`` would occur for backends that did not - feature "native boolean", but would not occur for native boolean backends, - meaning the string ``"0"`` now behaved inconsistently. After a poll, a - consensus was reached that non-boolean values should be raising an error, - especially in the ambiguous case of string ``"0"``; so the :class:`.Boolean` - datatype will now raise ``ValueError`` if an incoming value is not - within the range ``None, True, False, 1, 0``. - - .. seealso:: - - :ref:`change_4102` - - .. change:: - :tags: bug, sql - :tickets: 4063 - - Refined the behavior of :meth:`.Operators.op` such that in all cases, - if the :paramref:`.Operators.op.is_comparison` flag is set to True, - the return type of the resulting expression will be - :class:`.Boolean`, and if the flag is False, the return type of the - resulting expression will be the same type as that of the left-hand - expression, which is the typical default behavior of other operators. - Also added a new parameter :paramref:`.Operators.op.return_type` as well - as a helper method :meth:`.Operators.bool_op`. - - .. seealso:: - - :ref:`change_4063` - - .. change:: - :tags: bug, mysql - :tickets: 4072 - - Changed the name of the ``.values`` attribute of the new MySQL - INSERT..ON DUPLICATE KEY UPDATE construct to ``.inserted``, as - :class:`_expression.Insert` already has a method called :meth:`_expression.Insert.values`. - The ``.inserted`` attribute ultimately renders the MySQL ``VALUES()`` - function. - - .. change:: - :tags: bug, mssql, orm - :tickets: 4062 - - Added a new class of "rowcount support" for dialects that is specific to - when "RETURNING", which on SQL Server looks like "OUTPUT inserted", is in - use, as the PyODBC backend isn't able to give us rowcount on an UPDATE or - DELETE statement when OUTPUT is in effect. This primarily affects the ORM - when a flush is updating a row that contains server-calculated values, - raising an error if the backend does not return the expected row count. - PyODBC now states that it supports rowcount except if OUTPUT.inserted is - present, which is taken into account by the ORM during a flush as to - whether it will look for a rowcount. - - .. change:: - :tags: bug, sql - :tickets: 4088 - - Internal refinements to the :class:`.Enum`, :class:`.Interval`, and - :class:`.Boolean` types, which now extend a common mixin - :class:`.Emulated` that indicates a type that provides Python-side - emulation of a DB native type, switching out to the DB native type when - a supporting backend is in use. The PostgreSQL - :class:`_postgresql.INTERVAL` type when used directly will now include - the correct type coercion rules for SQL expressions that also take - effect for :class:`_types.Interval` (such as adding a date to an - interval yields a datetime). - - - .. change:: - :tags: bug, mssql, orm - - Enabled the "sane_rowcount" flag for the pymssql dialect, indicating - that the DBAPI now reports the correct number of rows affected from - an UPDATE or DELETE statement. This impacts mostly the ORM versioning - feature in that it now can verify the number of rows affected on a - target version. - - .. change:: 4028 - :tags: bug, engine - :tickets: 4028 - - Made some adjustments to :class:`_pool.Pool` and :class:`_engine.Connection` such - that recovery logic is not run underneath exception catches for - ``pool.Empty``, ``AttributeError``, since when the recovery operation - itself fails, Python 3 creates a misleading stack trace referring to the - ``Empty`` / ``AttributeError`` as the cause, when in fact these exception - catches are part of control flow. - - - .. change:: - :tags: bug, oracle - :tickets: 4076 - - Fixed bug where Oracle 8 "non ansi" join mode would not add the - ``(+)`` operator to expressions that used an operator other than the - ``=`` operator. The ``(+)`` needs to be on all columns that are part - of the right-hand side. - - .. change:: - :tags: bug, mssql - :tickets: 4059 - - Added a rule to SQL Server index reflection to ignore the so-called - "heap" index that is implicitly present on a table that does not - specify a clustered index. - - -.. changelog:: - :version: 1.2.0b2 - :released: December 27, 2017 - :released: July 24, 2017 - - .. change:: 4033 - :tags: bug, orm - :tickets: 4033 - - Fixed regression from 1.1.11 where adding additional non-entity - columns to a query that includes an entity with subqueryload - relationships would fail, due to an inspection added in 1.1.11 as a - result of :ticket:`4011`. - - -.. changelog:: - :version: 1.2.0b1 - :released: December 27, 2017 - :released: July 10, 2017 - - .. change:: scoped_autocommit - :tags: feature, orm - - Added ``.autocommit`` attribute to :class:`.scoped_session`, proxying - the ``.autocommit`` attribute of the underling :class:`.Session` - currently assigned to the thread. Pull request courtesy - Ben Fagin. - - .. change:: 4009 - :tags: feature, mysql - :tickets: 4009 - - Added support for MySQL's ON DUPLICATE KEY UPDATE - MySQL-specific :class:`.mysql.dml.Insert` object. - Pull request courtesy Michael Doronin. - - .. seealso:: - - :ref:`change_4009` - - .. change:: 4018 - :tags: bug, sql - :tickets: 4018 - - The rules for type coercion between :class:`.Numeric`, :class:`.Integer`, - and date-related types now include additional logic that will attempt - to preserve the settings of the incoming type on the "resolved" type. - Currently the target for this is the ``asdecimal`` flag, so that - a math operation between :class:`.Numeric` or :class:`.Float` and - :class:`.Integer` will preserve the "asdecimal" flag as well as - if the type should be the :class:`.Float` subclass. - - .. seealso:: - - :ref:`change_floats_12` - - .. change:: 4020 - :tags: bug, sql, mysql - :tickets: 4020 - - The result processor for the :class:`.Float` type now unconditionally - runs values through the ``float()`` processor if the dialect - specifies that it also supports "native decimal" mode. While most - backends will deliver Python ``float`` objects for a floating point - datatype, the MySQL backends in some cases lack the typing information - in order to provide this and return ``Decimal`` unless the float - conversion is done. - - .. seealso:: - - :ref:`change_floats_12` - - .. change:: 4017 - :tags: bug, sql - :tickets: 4017 - - Added some extra strictness to the handling of Python "float" values - passed to SQL statements. A "float" value will be associated with the - :class:`.Float` datatype and not the Decimal-coercing :class:`.Numeric` - datatype as was the case before, eliminating a confusing warning - emitted on SQLite as well as unnecessary coercion to Decimal. - - .. seealso:: - - :ref:`change_floats_12` - - .. change:: 3058 - :tags: feature, orm - :tickets: 3058 - - Added a new feature :func:`_orm.with_expression` that allows an ad-hoc - SQL expression to be added to a specific entity in a query at result - time. This is an alternative to the SQL expression being delivered as - a separate element in the result tuple. - - .. seealso:: - - :ref:`change_3058` - - .. change:: 3496 - :tags: bug, orm - :tickets: 3496 - - An UPDATE emitted as a result of the - :paramref:`_orm.relationship.post_update` feature will now integrate with - the versioning feature to both bump the version id of the row as well - as assert that the existing version number was matched. - - .. seealso:: - - :ref:`change_3496` - - .. change:: 3769 - :tags: bug, ext - :tickets: 3769 - - The :meth:`.AssociationProxy.any`, :meth:`.AssociationProxy.has` - and :meth:`.AssociationProxy.contains` comparison methods now support - linkage to an attribute that is itself also an - :class:`.AssociationProxy`, recursively. - - .. seealso:: - - :ref:`change_3769` - - .. change:: 3853 - :tags: bug, ext - :tickets: 3853 - - Implemented in-place mutation operators ``__ior__``, ``__iand__``, - ``__ixor__`` and ``__isub__`` for :class:`.mutable.MutableSet` - and ``__iadd__`` for :class:`.mutable.MutableList` so that change - events are fired off when these mutator methods are used to alter the - collection. - - .. seealso:: - - :ref:`change_3853` - - .. change:: 3847 - :tags: bug, declarative - :tickets: 3847 - - A warning is emitted if the :attr:`.declared_attr.cascading` modifier - is used with a declarative attribute that is itself declared on - a class that is to be mapped, as opposed to a declarative mixin - class or ``__abstract__`` class. The :attr:`.declared_attr.cascading` - modifier currently only applies to mixin/abstract classes. - - .. change:: 4003 - :tags: feature, oracle - :tickets: 4003 - - The Oracle dialect now inspects unique and check constraints when using - :meth:`_reflection.Inspector.get_unique_constraints`, - :meth:`_reflection.Inspector.get_check_constraints`. - As Oracle does not have unique constraints that are separate from a unique - :class:`.Index`, a :class:`_schema.Table` that's reflected will still continue - to not have :class:`.UniqueConstraint` objects associated with it. - Pull requests courtesy Eloy Felix. - - .. seealso:: - - :ref:`change_4003` - - .. change:: 3948 - :tags: feature, orm - :tickets: 3948 - - Added a new style of mapper-level inheritance loading - "polymorphic selectin". This style of loading - emits queries for each subclass in an inheritance - hierarchy subsequent to the load of the base - object type, using IN to specify the desired - primary key values. - - .. seealso:: - - :ref:`change_3948` - - .. change:: 3472 - :tags: bug, orm - :tickets: 3471, 3472 - - Repaired several use cases involving the - :paramref:`_orm.relationship.post_update` feature when used in conjunction - with a column that has an "onupdate" value. When the UPDATE emits, - the corresponding object attribute is now expired or refreshed so that - the newly generated "onupdate" value can populate on the object; - previously the stale value would remain. Additionally, if the target - attribute is set in Python for the INSERT of the object, the value is - now re-sent during the UPDATE so that the "onupdate" does not overwrite - it (note this works just as well for server-generated onupdates). - Finally, the :meth:`.SessionEvents.refresh_flush` event is now emitted - for these attributes when refreshed within the flush. - - .. seealso:: - - :ref:`change_3471` - - .. change:: 3996 - :tags: bug, orm - :tickets: 3996 - - Fixed bug where programmatic version_id counter in conjunction with - joined table inheritance would fail if the version_id counter - were not actually incremented and no other values on the base table - were modified, as the UPDATE would have an empty SET clause. Since - programmatic version_id where version counter is not incremented - is a documented use case, this specific condition is now detected - and the UPDATE now sets the version_id value to itself, so that - concurrency checks still take place. - - .. change:: 3848 - :tags: bug, orm, declarative - :tickets: 3848 - - Fixed bug where using :class:`.declared_attr` on an - :class:`.AbstractConcreteBase` where a particular return value were some - non-mapped symbol, including ``None``, would cause the attribute - to hard-evaluate just once and store the value to the object - dictionary, not allowing it to invoke for subclasses. This behavior - is normal when :class:`.declared_attr` is on a mapped class, and - does not occur on a mixin or abstract class. Since - :class:`.AbstractConcreteBase` is both "abstract" and actually - "mapped", a special exception case is made here so that the - "abstract" behavior takes precedence for :class:`.declared_attr`. - - .. change:: 3673 - :tags: bug, orm - :tickets: 3673 - - The versioning feature does not support NULL for the version counter. - An exception is now raised if the version id is programmatic and - was set to NULL for an UPDATE. Pull request courtesy Diana Clarke. - - .. change:: 3999 - :tags: bug, sql - :tickets: 3999 - - The operator precedence for all comparison operators such as LIKE, IS, - IN, MATCH, equals, greater than, less than, etc. has all been merged - into one level, so that expressions which make use of these against - each other will produce parentheses between them. This suits the - stated operator precedence of databases like Oracle, MySQL and others - which place all of these operators as equal precedence, as well as - PostgreSQL as of 9.5 which has also flattened its operator precedence. - - .. seealso:: - - :ref:`change_3999` - - - .. change:: 3796 - :tags: bug, orm - :tickets: 3796 - - Removed a very old keyword argument from :class:`.scoped_session` - called ``scope``. This keyword was never documented and was an - early attempt at allowing for variable scopes. - - .. seealso:: - - :ref:`change_3796` - - .. change:: 3871 - :tags: bug, mysql - :tickets: 3871 - - Added support for views that are unreflectable due to stale - table definitions, when calling :meth:`_schema.MetaData.reflect`; a warning - is emitted for the table that cannot respond to ``DESCRIBE``, - but the operation succeeds. - - .. change:: baked_opts - :tags: feature, ext - - Added new flag :paramref:`.Session.enable_baked_queries` to the - :class:`.Session` to allow baked queries to be disabled - session-wide, reducing memory use. Also added new :class:`.Bakery` - wrapper so that the bakery returned by :paramref:`.BakedQuery.bakery` - can be inspected. - - .. change:: 3988 - :tags: bug, orm - :tickets: 3988 - - Fixed bug where combining a "with_polymorphic" load in conjunction - with subclass-linked relationships that specify joinedload with - innerjoin=True, would fail to demote those "innerjoins" to - "outerjoins" to suit the other polymorphic classes that don't - support that relationship. This applies to both a single and a - joined inheritance polymorphic load. - - .. change:: 3991 - :tags: bug, orm - :tickets: 3991 - - Added new argument :paramref:`.with_for_update` to the - :meth:`.Session.refresh` method. When the :meth:`_query.Query.with_lockmode` - method were deprecated in favor of :meth:`_query.Query.with_for_update`, - the :meth:`.Session.refresh` method was never updated to reflect - the new option. - - .. seealso:: - - :ref:`change_3991` - - .. change:: 3984 - :tags: bug, orm - :tickets: 3984 - - Fixed bug where a :func:`.column_property` that is also marked as - "deferred" would be marked as "expired" during a flush, causing it - to be loaded along with the unexpiry of regular attributes even - though this attribute was never accessed. - - .. change:: 3873 - :tags: bug, sql - :tickets: 3873 - - Repaired issue where the type of an expression that used - :meth:`.ColumnOperators.is_` or similar would not be a "boolean" type, - instead the type would be "nulltype", as well as when using custom - comparison operators against an untyped expression. This typing can - impact how the expression behaves in larger contexts as well as - in result-row-handling. - - .. change:: 3941 - :tags: bug, ext - :tickets: 3941 - - Improved the association proxy list collection so that premature - autoflush against a newly created association object can be prevented - in the case where ``list.append()`` is being used, and a lazy load - would be invoked when the association proxy accesses the endpoint - collection. The endpoint collection is now accessed first before - the creator is invoked to produce the association object. - - .. change:: 3969 - :tags: bug, sql - :tickets: 3969 - - Fixed the negation of a :class:`.Label` construct so that the - inner element is negated correctly, when the :func:`.not_` modifier - is applied to the labeled expression. - - .. change:: 3944 - :tags: feature, orm - :tickets: 3944 - - Added a new kind of eager loading called "selectin" loading. This - style of loading is very similar to "subquery" eager loading, - except that it uses an IN expression given a list of primary key - values from the loaded parent objects, rather than re-stating the - original query. This produces a more efficient query that is - "baked" (e.g. the SQL string is cached) and also works in the - context of :meth:`_query.Query.yield_per`. - - .. seealso:: - - :ref:`change_3944` - - .. change:: - :tags: bug, orm - :tickets: 3967 - - Fixed bug in subquery eager loading where the "join_depth" parameter - for self-referential relationships would not be correctly honored, - loading all available levels deep rather than correctly counting - the specified number of levels for eager loading. - - .. change:: - :tags: bug, orm - - Added warnings to the LRU "compiled cache" used by the :class:`_orm.Mapper` - (and ultimately will be for other ORM-based LRU caches) such that - when the cache starts hitting its size limits, the application will - emit a warning that this is a performance-degrading situation that - may require attention. The LRU caches can reach their size limits - primarily if an application is making use of an unbounded number - of :class:`_engine.Engine` objects, which is an antipattern. Otherwise, - this may suggest an issue that should be brought to the SQLAlchemy - developer's attention. - - .. change:: 3964 - :tags: bug, postgresql - :tickets: 3964 - - Fixed bug where the base :class:`_types.ARRAY` datatype would not - invoke the bind/result processors of :class:`_postgresql.ARRAY`. - - .. change:: 3963 - :tags: bug, orm - :tickets: 3963 - - Fixed bug to improve upon the specificity of loader options that - take effect subsequent to the lazy load of a related entity, so - that the loader options will match to an aliased or non-aliased - entity more specifically if those options include entity information. - - .. change:: 3954 - :tags: feature, orm - :tickets: 3954 - - The ``lazy="select"`` loader strategy now makes used of the - :class:`.BakedQuery` query caching system in all cases. This - removes most overhead of generating a :class:`_query.Query` object and - running it into a :func:`_expression.select` and then string SQL statement from - the process of lazy-loading related collections and objects. The - "baked" lazy loader has also been improved such that it can now - cache in most cases where query load options are used. - - .. seealso:: - - :ref:`change_3954` - - .. change:: 3740 - :tags: bug, sql - :tickets: 3740 - - The system by which percent signs in SQL statements are "doubled" - for escaping purposes has been refined. The "doubling" of percent - signs mostly associated with the :obj:`_expression.literal_column` construct - as well as operators like :meth:`.ColumnOperators.contains` now - occurs based on the stated paramstyle of the DBAPI in use; for - percent-sensitive paramstyles as are common with the PostgreSQL - and MySQL drivers the doubling will occur, for others like that - of SQLite it will not. This allows more database-agnostic use - of the :obj:`_expression.literal_column` construct to be possible. - - .. seealso:: - - :ref:`change_3740` - - .. change:: 3959 - :tags: bug, postgresql - :tickets: 3959 - - Added support for all possible "fields" identifiers when reflecting the - PostgreSQL ``INTERVAL`` datatype, e.g. "YEAR", "MONTH", "DAY TO - MINUTE", etc.. In addition, the :class:`_postgresql.INTERVAL` - datatype itself now includes a new parameter - :paramref:`.postgresql.INTERVAL.fields` where these qualifiers can be - specified; the qualifier is also reflected back into the resulting - datatype upon reflection / inspection. - - .. seealso:: - - :ref:`change_3959` - - .. change:: 3957 - :tags: bug, sql - :tickets: 3957 - - Fixed bug where a column-level :class:`.CheckConstraint` would fail - to compile the SQL expression using the underlying dialect compiler - as well as apply proper flags to generate literal values as - inline, in the case that the sqltext is a Core expression and - not just a plain string. This was long-ago fixed for table-level - check constraints in 0.9 as part of :ticket:`2742`, which more commonly - feature Core SQL expressions as opposed to plain string expressions. - - .. change:: 2626 - :tags: bug, mssql - :tickets: 2626 - - The SQL Server dialect now allows for a database and/or owner name - with a dot inside of it, using brackets explicitly in the string around - the owner and optionally the database name as well. In addition, - sending the :class:`.quoted_name` construct for the schema name will - not split on the dot and will deliver the full string as the "owner". - :class:`.quoted_name` is also now available from the ``sqlalchemy.sql`` - import space. - - .. seealso:: - - :ref:`change_2626` - - .. change:: 3953 - :tags: feature, sql - :tickets: 3953 - - Added a new kind of :func:`.bindparam` called "expanding". This is - for use in ``IN`` expressions where the list of elements is rendered - into individual bound parameters at statement execution time, rather - than at statement compilation time. This allows both a single bound - parameter name to be linked to an IN expression of multiple elements, - as well as allows query caching to be used with IN expressions. The - new feature allows the related features of "select in" loading and - "polymorphic in" loading to make use of the baked query extension - to reduce call overhead. This feature should be considered to be - **experimental** for 1.2. - - .. seealso:: - - :ref:`change_3953` - - .. change:: 3923 - :tags: bug, sql - :tickets: 3923 - - Fixed bug where a SQL-oriented Python-side column default could fail to - be executed properly upon INSERT in the "pre-execute" codepath, if the - SQL itself were an untyped expression, such as plain text. The "pre- - execute" codepath is fairly uncommon however can apply to non-integer - primary key columns with SQL defaults when RETURNING is not used. - - .. change:: 3785 - :tags: bug, sql - :tickets: 3785 - - The expression used for COLLATE as rendered by the column-level - :func:`_expression.collate` and :meth:`.ColumnOperators.collate` is now - quoted as an identifier when the name is case sensitive, e.g. has - uppercase characters. Note that this does not impact type-level - collation, which is already quoted. - - .. seealso:: - - :ref:`change_3785` - - .. change:: 3229 - :tags: feature, orm, ext - :tickets: 3229 - - The :meth:`_query.Query.update` method can now accommodate both - hybrid attributes as well as composite attributes as a source - of the key to be placed in the SET clause. For hybrids, an - additional decorator :meth:`.hybrid_property.update_expression` - is supplied for which the user supplies a tuple-returning function. - - .. seealso:: - - :ref:`change_3229` - - .. change:: 3753 - :tags: bug, orm - :tickets: 3753 - - The :func:`.attributes.flag_modified` function now raises - :class:`.InvalidRequestError` if the named attribute key is not - present within the object, as this is assumed to be present - in the flush process. To mark an object "dirty" for a flush - without referring to any specific attribute, the - :func:`.attributes.flag_dirty` function may be used. - - .. seealso:: - - :ref:`change_3753` - - .. change:: 3911_3912 - :tags: bug, ext - :tickets: 3911, 3912 - - The :class:`sqlalchemy.ext.hybrid.hybrid_property` class now supports - calling mutators like ``@setter``, ``@expression`` etc. multiple times - across subclasses, and now provides a ``@getter`` mutator, so that - a particular hybrid can be repurposed across subclasses or other - classes. This now matches the behavior of ``@property`` in standard - Python. - - .. seealso:: - - :ref:`change_3911_3912` - - - - .. change:: 1546 - :tags: feature, sql, postgresql, mysql, oracle - :tickets: 1546 - - Added support for SQL comments on :class:`_schema.Table` and :class:`_schema.Column` - objects, via the new :paramref:`_schema.Table.comment` and - :paramref:`_schema.Column.comment` arguments. The comments are included - as part of DDL on table creation, either inline or via an appropriate - ALTER statement, and are also reflected back within table reflection, - as well as via the :class:`_reflection.Inspector`. Supported backends currently - include MySQL, PostgreSQL, and Oracle. Many thanks to Frazer McLean - for a large amount of effort on this. - - .. seealso:: - - :ref:`change_1546` - - .. change:: 3919 - :tags: feature, engine - :tickets: 3919 - - Added native "pessimistic disconnection" handling to the :class:`_pool.Pool` - object. The new parameter :paramref:`_pool.Pool.pre_ping`, available from - the engine as :paramref:`_sa.create_engine.pool_pre_ping`, applies an - efficient form of the "pre-ping" recipe featured in the pooling - documentation, which upon each connection check out, emits a simple - statement, typically "SELECT 1", to test the connection for liveness. - If the existing connection is no longer able to respond to commands, - the connection is transparently recycled, and all other connections - made prior to the current timestamp are invalidated. - - .. seealso:: - - :ref:`pool_disconnects_pessimistic` - - :ref:`change_3919` - - .. change:: 3939 - :tags: bug, sql - :tickets: 3939 - - Fixed bug where the use of an :class:`_expression.Alias` object in a column - context would raise an argument error when it tried to group itself - into a parenthesized expression. Using :class:`_expression.Alias` in this way - is not yet a fully supported API, however it applies to some end-user - recipes and may have a more prominent role in support of some - future PostgreSQL features. - - .. change:: 3366 - :tags: bug, orm - :tickets: 3366 - - The "evaluate" strategy used by :meth:`_query.Query.update` and - :meth:`_query.Query.delete` can now accommodate a simple - object comparison from a many-to-one relationship to an instance, - when the attribute names of the primary key / foreign key columns - don't match the actual names of the columns. Previously this would - do a simple name-based match and fail with an AttributeError. - - .. change:: 3896_a - :tags: feature, orm - :tickets: 3896 - - Added new attribute event :meth:`.AttributeEvents.bulk_replace`. - This event is triggered when a collection is assigned to a - relationship, before the incoming collection is compared with the - existing one. This early event allows for conversion of incoming - non-ORM objects as well. The event is integrated with the - ``@validates`` decorator. - - .. seealso:: - - :ref:`change_3896_event` - - .. change:: 3896_b - :tags: bug, orm - :tickets: 3896 - - The ``@validates`` decorator now allows the decorated method to receive - objects from a "bulk collection set" operation that have not yet - been compared to the existing collection. This allows incoming values - to be converted to compatible ORM objects as is already allowed - from an "append" event. Note that this means that the - ``@validates`` method is called for **all** values during a collection - assignment, rather than just the ones that are new. - - .. seealso:: - - :ref:`change_3896_validates` - - .. change:: 3938 - :tags: bug, engine - :tickets: 3938 - - Fixed bug where in the unusual case of passing a - :class:`.Compiled` object directly to :meth:`_engine.Connection.execute`, - the dialect with which the :class:`.Compiled` object were generated - was not consulted for the paramstyle of the string statement, instead - assuming it would match the dialect-level paramstyle, causing - mismatches to occur. - - .. change:: 3303 - :tags: feature, orm - :tickets: 3303 - - Added new event handler :meth:`.AttributeEvents.modified` which is - triggered when the func:`.attributes.flag_modified` function is - invoked, which is common when using the :mod:`sqlalchemy.ext.mutable` - extension module. - - .. seealso:: - - :ref:`change_3303` - - .. change:: 3918 - :tags: bug, ext - :tickets: 3918 - - Fixed a bug in the ``sqlalchemy.ext.serializer`` extension whereby - an "annotated" SQL element (as produced by the ORM for many types - of SQL expressions) could not be reliably serialized. Also bumped - the default pickle level for the serializer to "HIGHEST_PROTOCOL". - - .. change:: 3891 - :tags: bug, orm - :tickets: 3891 - - Fixed bug in single-table inheritance where the select_from() - argument would not be taken into account when limiting rows - to a subclass. Previously, only expressions in the - columns requested would be taken into account. - - .. seealso:: - - :ref:`change_3891` - - .. change:: 3913 - :tags: bug, orm - :tickets: 3913 - - When assigning a collection to an attribute mapped by a relationship, - the previous collection is no longer mutated. Previously, the old - collection would be emptied out in conjunction with the "item remove" - events that fire off; the events now fire off without affecting - the old collection. - - .. seealso:: - - :ref:`change_3913` - - .. change:: 3932 - :tags: bug, oracle - :tickets: 3932 - - The cx_Oracle dialect now supports "sane multi rowcount", that is, - when a series of parameter sets are executed via DBAPI - ``cursor.executemany()``, we can make use of ``cursor.rowcount`` to - verify the number of rows matched. This has an impact within the - ORM when detecting concurrent modification scenarios, in that - some simple conditions can now be detected even when the ORM - is batching statements, as well as when the more strict versioning - feature is used, the ORM can still use statement batching. The - flag is enabled for cx_Oracle assuming at least version 5.0, which - is now commonplace. - - .. change:: 3907 - :tags: feature, sql - :tickets: 3907 - - The longstanding behavior of the :meth:`.ColumnOperators.in_` and - :meth:`.ColumnOperators.notin_` operators emitting a warning when - the right-hand condition is an empty sequence has been revised; - a simple "static" expression of "1 != 1" or "1 = 1" is now rendered - by default, rather than pulling in the original left-hand - expression. This causes the result for a NULL column comparison - against an empty set to change from NULL to true/false. The - behavior is configurable, and the old behavior can be enabled - using the :paramref:`_sa.create_engine.empty_in_strategy` parameter - to :func:`_sa.create_engine`. - - .. seealso:: - - :ref:`change_3907` - - .. change:: 3276 - :tags: bug, oracle - :tickets: 3276 - - Oracle reflection now "normalizes" the name given to a foreign key - constraint, that is, returns it as all lower case for a case - insensitive name. This was already the behavior for indexes - and primary key constraints as well as all table and column names. - This will allow Alembic autogenerate scripts to compare and render - foreign key constraint names correctly when initially specified - as case insensitive. - - .. seealso:: - - :ref:`change_3276` - - .. change:: 2694 - :tags: feature, sql - :tickets: 2694 - - Added a new option ``autoescape`` to the "startswith" and - "endswith" classes of comparators; this supplies an escape character - also applies it to all occurrences of the wildcard characters "%" - and "_" automatically. Pull request courtesy Diana Clarke. - - .. note:: This feature has been changed as of 1.2.0 from its initial - implementation in 1.2.0b2 such that autoescape is now passed as a - boolean value, rather than a specific character to use as the escape - character. - - .. seealso:: - - :ref:`change_2694` - - .. change:: 3934 - :tags: bug, orm - :tickets: 3934 - - The state of the :class:`.Session` is now present when the - :meth:`.SessionEvents.after_rollback` event is emitted, that is, the - attribute state of objects prior to their being expired. This is now - consistent with the behavior of the - :meth:`.SessionEvents.after_commit` event which also emits before the - attribute state of objects is expired. - - .. seealso:: - - :ref:`change_3934` - - .. change:: 3607 - :tags: bug, orm - :tickets: 3607 - - Fixed bug where :meth:`_query.Query.with_parent` would not work if the - :class:`_query.Query` were against an :func:`.aliased` construct rather than - a regular mapped class. Also adds a new parameter - :paramref:`.util.with_parent.from_entity` to the standalone - :func:`.util.with_parent` function as well as - :meth:`_query.Query.with_parent`. diff --git a/doc/build/changelog/changelog_13.rst b/doc/build/changelog/changelog_13.rst deleted file mode 100644 index 6039499692..0000000000 --- a/doc/build/changelog/changelog_13.rst +++ /dev/null @@ -1,4010 +0,0 @@ -============= -1.3 Changelog -============= - -.. changelog_imports:: - - .. include:: changelog_12.rst - :start-line: 5 - - .. include:: changelog_11.rst - :start-line: 5 - -.. changelog:: - :version: 1.3.25 - :include_notes_from: unreleased_13 - -.. changelog:: - :version: 1.3.24 - :released: March 30, 2021 - - .. change:: - :tags: bug, schema - :tickets: 6152 - - Fixed bug first introduced in as some combination of :ticket:`2892`, - :ticket:`2919` nnd :ticket:`3832` where the attachment events for a - :class:`_types.TypeDecorator` would be doubled up against the "impl" class, - if the "impl" were also a :class:`_types.SchemaType`. The real-world case - is any :class:`_types.TypeDecorator` against :class:`_types.Enum` or - :class:`_types.Boolean` would get a doubled - :class:`_schema.CheckConstraint` when the ``create_constraint=True`` flag - is set. - - - .. change:: - :tags: bug, schema, sqlite - :tickets: 6007 - :versions: 1.4.0 - - Fixed issue where the CHECK constraint generated by :class:`_types.Boolean` - or :class:`_types.Enum` would fail to render the naming convention - correctly after the first compilation, due to an unintended change of state - within the name given to the constraint. This issue was first introduced in - 0.9 in the fix for issue #3067, and the fix revises the approach taken at - that time which appears to have been more involved than what was needed. - - .. change:: - :tags: bug, orm - :tickets: 5983 - :versions: 1.4.0 - - Removed very old warning that states that passive_deletes is not intended - for many-to-one relationships. While it is likely that in many cases - placing this parameter on a many-to-one relationship is not what was - intended, there are use cases where delete cascade may want to be - disallowed following from such a relationship. - - - - .. change:: - :tags: bug, postgresql - :tickets: 5989 - :versions: 1.4.0 - - Fixed issue where using :class:`_postgresql.aggregate_order_by` would - return ARRAY(NullType) under certain conditions, interfering with - the ability of the result object to return data correctly. - - .. change:: - :tags: bug, schema - :tickets: 5919 - :versions: 1.4.0 - - Repaired / implemented support for primary key constraint naming - conventions that use column names/keys/etc as part of the convention. In - particular, this includes that the :class:`.PrimaryKeyConstraint` object - that's automatically associated with a :class:`.schema.Table` will update - its name as new primary key :class:`_schema.Column` objects are added to - the table and then to the constraint. Internal failure modes related to - this constraint construction process including no columns present, no name - present or blank name present are now accommodated. - - .. change:: - :tags: bug, schema - :tickets: 6071 - :versions: 1.4.3 - - Adjusted the logic that emits DROP statements for :class:`_schema.Sequence` - objects among the dropping of multiple tables, such that all - :class:`_schema.Sequence` objects are dropped after all tables, even if the - given :class:`_schema.Sequence` is related only to a :class:`_schema.Table` - object and not directly to the overall :class:`_schema.MetaData` object. - The use case supports the same :class:`_schema.Sequence` being associated - with more than one :class:`_schema.Table` at a time. - - .. change:: - :tags: bug, orm - :tickets: 5952 - :versions: 1.4.0 - - Fixed issue where the process of joining two tables could fail if one of - the tables had an unrelated, unresolvable foreign key constraint which - would raise :class:`_exc.NoReferenceError` within the join process, which - nonetheless could be bypassed to allow the join to complete. The logic - which tested the exception for significance within the process would make - assumptions about the construct which would fail. - - - .. change:: - :tags: bug, postgresql, reflection - :tickets: 6161 - :versions: 1.4.4 - - Fixed issue in PostgreSQL reflection where a column expressing "NOT NULL" - will supersede the nullability of a corresponding domain. - - .. change:: - :tags: bug, engine - :tickets: 5929 - :versions: 1.4.0 - - Fixed bug where the "schema_translate_map" feature failed to be taken into - account for the use case of direct execution of - :class:`_schema.DefaultGenerator` objects such as sequences, which included - the case where they were "pre-executed" in order to generate primary key - values when implicit_returning was disabled. - - .. change:: - :tags: bug, orm - :tickets: 6001 - :versions: 1.4.0 - - Fixed issue where the :class:`_mutable.MutableComposite` construct could be - placed into an invalid state when the parent object was already loaded, and - then covered by a subsequent query, due to the composite properties' - refresh handler replacing the object with a new one not handled by the - mutable extension. - - - .. change:: - :tags: bug, types, postgresql - :tickets: 6023 - :versions: 1.4.3 - - Adjusted the psycopg2 dialect to emit an explicit PostgreSQL-style cast for - bound parameters that contain ARRAY elements. This allows the full range of - datatypes to function correctly within arrays. The asyncpg dialect already - generated these internal casts in the final statement. This also includes - support for array slice updates as well as the PostgreSQL-specific - :meth:`_postgresql.ARRAY.contains` method. - - .. change:: - :tags: bug, mssql, reflection - :tickets: 5921 - - Fixed issue regarding SQL Server reflection for older SQL Server 2005 - version, a call to sp_columns would not proceed correctly without being - prefixed with the EXEC keyword. This method is not used in current 1.4 - series. - - -.. changelog:: - :version: 1.3.23 - :released: February 1, 2021 - - .. change:: - :tags: bug, ext - :tickets: 5836 - - Fixed issue where the stringification that is sometimes called when - attempting to generate the "key" for the ``.c`` collection on a selectable - would fail if the column were an unlabeled custom SQL construct using the - ``sqlalchemy.ext.compiler`` extension, and did not provide a default - compilation form; while this seems like an unusual case, it can get invoked - for some ORM scenarios such as when the expression is used in an "order by" - in combination with joined eager loading. The issue is that the lack of a - default compiler function was raising :class:`.CompileError` and not - :class:`.UnsupportedCompilationError`. - - .. change:: - :tags: bug, postgresql - :tickets: 5645 - - For SQLAlchemy 1.3 only, setup.py pins pg8000 to a version lower than - 1.16.6. Version 1.16.6 and above is supported by SQLAlchemy 1.4. Pull - request courtesy Giuseppe Lumia. - - .. change:: - :tags: bug, postgresql - :tickets: 5850 - - Fixed issue where using :meth:`_schema.Table.to_metadata` (called - :meth:`_schema.Table.tometadata` in 1.3) in conjunction with a PostgreSQL - :class:`_postgresql.ExcludeConstraint` that made use of ad-hoc column - expressions would fail to copy correctly. - - .. change:: - :tags: bug, sql - :tickets: 5816 - - Fixed bug where making use of the :meth:`.TypeEngine.with_variant` method - on a :class:`.TypeDecorator` type would fail to take into account the - dialect-specific mappings in use, due to a rule in :class:`.TypeDecorator` - that was instead attempting to check for chains of :class:`.TypeDecorator` - instances. - - - .. change:: - :tags: bug, mysql, reflection - :tickets: 5860 - - Fixed bug where MySQL server default reflection would fail for numeric - values with a negation symbol present. - - - .. change:: - :tags: bug, oracle - :tickets: 5813 - - Fixed regression in Oracle dialect introduced by :ticket:`4894` in - SQLAlchemy 1.3.11 where use of a SQL expression in RETURNING for an UPDATE - would fail to compile, due to a check for "server_default" when an - arbitrary SQL expression is not a column. - - - .. change:: - :tags: usecase, mysql - :tickets: 5808 - - Casting to ``FLOAT`` is now supported in MySQL >= (8, 0, 17) and - MariaDb >= (10, 4, 5). - - .. change:: - :tags: bug, mysql - :tickets: 5898 - - Fixed long-lived bug in MySQL dialect where the maximum identifier length - of 255 was too long for names of all types of constraints, not just - indexes, all of which have a size limit of 64. As metadata naming - conventions can create too-long names in this area, apply the limit to the - identifier generator within the DDL compiler. - - .. change:: - :tags: bug, oracle - :tickets: 5812 - - Fixed bug in Oracle dialect where retrieving a CLOB/BLOB column via - :meth:`_dml.Insert.returning` would fail as the LOB value would need to be - read when returned; additionally, repaired support for retrieval of Unicode - values via RETURNING under Python 2. - - .. change:: - :tags: bug, mysql - :tickets: 5821 - - Fixed deprecation warnings that arose as a result of the release of PyMySQL - 1.0, including deprecation warnings for the "db" and "passwd" parameters - now replaced with "database" and "password". - - - .. change:: - :tags: bug, mysql - :tickets: 5800 - - Fixed regression from SQLAlchemy 1.3.20 caused by the fix for - :ticket:`5462` which adds double-parenthesis for MySQL functional - expressions in indexes, as is required by the backend, this inadvertently - extended to include arbitrary :func:`_sql.text` expressions as well as - Alembic's internal textual component, which are required by Alembic for - arbitrary index expressions which don't imply double parenthesis. The - check has been narrowed to include only binary/ unary/functional - expressions directly. - -.. changelog:: - :version: 1.3.22 - :released: December 18, 2020 - - .. change:: - :tags: bug, oracle - :tickets: 5784 - :versions: 1.4.0b2 - - Fixed regression which occurred due to :ticket:`5755` which implemented - isolation level support for Oracle. It has been reported that many Oracle - accounts don't actually have permission to query the ``v$transaction`` - view so this feature has been altered to gracefully fallback when it fails - upon database connect, where the dialect will assume "READ COMMITTED" is - the default isolation level as was the case prior to SQLAlchemy 1.3.21. - However, explicit use of the :meth:`_engine.Connection.get_isolation_level` - method must now necessarily raise an exception, as Oracle databases with - this restriction explicitly disallow the user from reading the current - isolation level. - -.. changelog:: - :version: 1.3.21 - :released: December 17, 2020 - - .. change:: - :tags: bug, orm - :tickets: 5774 - :versions: 1.4.0b2 - - Added a comprehensive check and an informative error message for the case - where a mapped class, or a string mapped class name, is passed to - :paramref:`_orm.relationship.secondary`. This is an extremely common error - which warrants a clear message. - - Additionally, added a new rule to the class registry resolution such that - with regards to the :paramref:`_orm.relationship.secondary` parameter, if a - mapped class and its table are of the identical string name, the - :class:`.Table` will be favored when resolving this parameter. In all - other cases, the class continues to be favored if a class and table - share the identical name. - - .. change:: - :tags: sqlite, usecase - :tickets: 5685 - - Added ``sqlite_with_rowid=False`` dialect keyword to enable creating - tables as ``CREATE TABLE … WITHOUT ROWID``. Patch courtesy Sean Anderson. - - .. change:: - :tags: bug, sql - :tickets: 5691 - - A warning is emitted if a returning() method such as - :meth:`_sql.Insert.returning` is called multiple times, as this does not - yet support additive operation. Version 1.4 will support additive - operation for this. Additionally, any combination of the - :meth:`_sql.Insert.returning` and :meth:`_sql.ValuesBase.return_defaults` - methods now raises an error as these methods are mutually exclusive; - previously the operation would fail silently. - - - .. change:: - :tags: bug, mssql - :tickets: 5751 - - Fixed bug where a CREATE INDEX statement was rendered incorrectly when - both ``mssql-include`` and ``mssql_where`` were specified. Pull request - courtesy @Adiorz. - - .. change:: - :tags: bug, postgresql, mysql - :tickets: 5729 - :versions: 1.4.0b2 - - Fixed regression introduced in 1.3.2 for the PostgreSQL dialect, also - copied out to the MySQL dialect's feature in 1.3.18, where usage of a non - :class:`_schema.Table` construct such as :func:`_sql.text` as the argument - to :paramref:`_sql.Select.with_for_update.of` would fail to be accommodated - correctly within the PostgreSQL or MySQL compilers. - - - .. change:: - :tags: bug, mssql - :tickets: 5646 - - Added SQL Server code "01000" to the list of disconnect codes. - - - .. change:: - :tags: usecase, postgresql - :tickets: 5604 - :versions: 1.4.0b2 - - Added new parameter :paramref:`_postgresql.ExcludeConstraint.ops` to the - :class:`_postgresql.ExcludeConstraint` object, to support operator class - specification with this constraint. Pull request courtesy Alon Menczer. - - .. change:: - :tags: bug, mysql, reflection - :tickets: 5744 - :versions: 1.4.0b2 - - Fixed issue where reflecting a server default on MariaDB only that - contained a decimal point in the value would fail to be reflected - correctly, leading towards a reflected table that lacked any server - default. - - - .. change:: - :tags: bug, orm - :tickets: 5664 - - Fixed bug in :meth:`_query.Query.update` where objects in the - :class:`_ormsession.Session` that were already expired would be - unnecessarily SELECTed individually when they were refreshed by the - "evaluate"synchronize strategy. - - .. change:: - :tags: usecase, oracle - :tickets: 5755 - - Implemented support for the SERIALIZABLE isolation level for Oracle - databases, as well as a real implementation for - :meth:`_engine.Connection.get_isolation_level`. - - .. seealso:: - - :ref:`oracle_isolation_level` - - .. change:: - :tags: mysql, sql - :tickets: 5696 - - Added missing keywords to the ``RESERVED_WORDS`` list for the MySQL - dialect: ``action``, ``level``, ``mode``, ``status``, ``text``, ``time``. - Pull request courtesy Oscar Batori. - - .. change:: - :tags: bug, orm - :tickets: 5737 - :versions: 1.4.0b2 - - Fixed bug involving the ``restore_load_context`` option of ORM events such - as :meth:`_ormevent.InstanceEvents.load` such that the flag would not be - carried along to subclasses which were mapped after the event handler were - first established. - - - - .. change:: - :tags: bug, sql - :tickets: 5656 - - Fixed structural compiler issue where some constructs such as MySQL / - PostgreSQL "on conflict / on duplicate key" would rely upon the state of - the :class:`_sql.Compiler` object being fixed against their statement as - the top level statement, which would fail in cases where those statements - are branched from a different context, such as a DDL construct linked to a - SQL statement. - - - .. change:: - :tags: mssql, sqlite, reflection - :tickets: 5661 - - Fixed issue with composite primary key columns not being reported - in the correct order. Patch courtesy @fulpm. - -.. changelog:: - :version: 1.3.20 - :released: October 12, 2020 - - .. change:: - :tags: bug, orm - :tickets: 4428 - - An :class:`.ArgumentError` with more detail is now raised if the target - parameter for :meth:`_query.Query.join` is set to an unmapped object. - Prior to this change a less detailed ``AttributeError`` was raised. - Pull request courtesy Ramon Williams. - - .. change:: - :tags: bug, mysql - :tickets: 5568 - - The "skip_locked" keyword used with ``with_for_update()`` will emit a - warning when used on MariaDB backends, and will then be ignored. This is - a deprecated behavior that will raise in SQLAlchemy 1.4, as an application - that requests "skip locked" is looking for a non-blocking operation which - is not available on those backends. - - - - .. change:: - :tags: bug, engine - :tickets: 5599 - - Fixed issue where a non-string object sent to - :class:`_exc.SQLAlchemyError` or a subclass, as occurs with some third - party dialects, would fail to stringify correctly. Pull request - courtesy Andrzej Bartosiński. - - .. change:: - :tags: bug, sql - :tickets: 5644 - - Fixed issue where the ``pickle.dumps()`` operation against - :class:`_expression.Over` construct would produce a recursion overflow. - - .. change:: - :tags: postgresql, usecase - :tickets: 4392 - - The psycopg2 dialect now support PostgreSQL multiple host connections, by - passing host/port combinations to the query string. Pull request courtesy - Ramon Williams. - - .. seealso:: - - :ref:`psycopg2_multi_host` - - .. change:: - :tags: bug, mysql - :tickets: 5617 - - Fixed bug where an UPDATE statement against a JOIN using MySQL multi-table - format would fail to include the table prefix for the target table if the - statement had no WHERE clause, as only the WHERE clause were scanned to - detect a "multi table update" at that particular point. The target - is now also scanned if it's a JOIN to get the leftmost table as the - primary table and the additional entries as additional FROM entries. - - - .. change:: - :tags: bug, postgresql - :tickets: 5518 - - Adjusted the :meth:`_types.ARRAY.Comparator.any` and - :meth:`_types.ARRAY.Comparator.all` methods to implement a straight "NOT" - operation for negation, rather than negating the comparison operator. - - .. change:: - :tags: bug, pool - :tickets: 5582 - - Fixed issue where the following pool parameters were not being propagated - to the new pool created when :meth:`_engine.Engine.dispose` were called: - ``pre_ping``, ``use_lifo``. Additionally the ``recycle`` and - ``reset_on_return`` parameter is now propagated for the - :class:`_engine.AssertionPool` class. - - .. change:: - :tags: bug, ext, associationproxy - :tickets: 5541, 5542 - - An informative error is now raised when attempting to use an association - proxy element as a plain column expression to be SELECTed from or used in a - SQL function; this use case is not currently supported. - - - .. change:: - :tags: bug, sql - :tickets: 5618 - - Fixed bug where an error was not raised in the case where a - :func:`_sql.column` were added to more than one :func:`_sql.table` at a - time. This raised correctly for the :class:`_schema.Column` and - :class:`_schema.Table` objects. An :class:`_exc.ArgumentError` is now - raised when this occurs. - - .. change:: - :tags: bug, orm - :tickets: 4589 - - Fixed issue where using a loader option against a string attribute name - that is not actually a mapped attribute, such as a plain Python descriptor, - would raise an uninformative AttributeError; a descriptive error is now - raised. - - - - .. change:: - :tags: mysql, usecase - :tickets: 5462 - - Adjusted the MySQL dialect to correctly parenthesize functional index - expressions as accepted by MySQL 8. Pull request courtesy Ramon Williams. - - .. change:: - :tags: bug, engine - :tickets: 5632 - - Repaired a function-level import that was not using SQLAlchemy's standard - late-import system within the sqlalchemy.exc module. - - - .. change:: - :tags: change, mysql - :tickets: 5539 - - Add new MySQL reserved words: ``cube``, ``lateral`` added in MySQL 8.0.1 - and 8.0.14, respectively; this indicates that these terms will be quoted if - used as table or column identifier names. - - .. change:: - :tags: bug, mssql - :tickets: 5592 - - Fixed issue where a SQLAlchemy connection URI for Azure DW with - ``authentication=ActiveDirectoryIntegrated`` (and no username+password) - was not constructing the ODBC connection string in a way that was - acceptable to the Azure DW instance. - - .. change:: - :tags: bug, postgresql - :tickets: 5520 - - Fixed issue where the :class:`_postgresql.ENUM` type would not consult the - schema translate map when emitting a CREATE TYPE or DROP TYPE during the - test to see if the type exists or not. Additionally, repaired an issue - where if the same enum were encountered multiple times in a single DDL - sequence, the "check" query would run repeatedly rather than relying upon a - cached value. - - - .. change:: - :tags: bug, tests - :tickets: 5635 - - Fixed incompatibilities in the test suite when running against Pytest 6.x. - - -.. changelog:: - :version: 1.3.19 - :released: August 17, 2020 - - .. change:: - :tags: usecase, py3k - :tickets: #5357 - - Added a ``**kw`` argument to the :meth:`.DeclarativeMeta.__init__` method. - This allows a class to support the :pep:`487` metaclass hook - ``__init_subclass__``. Pull request courtesy Ewen Gillies. - - - .. change:: - :tags: bug, sql - :tickets: 5470 - - Repaired an issue where the "ORDER BY" clause rendering a label name rather - than a complete expression, which is particularly important for SQL Server, - would fail to occur if the expression were enclosed in a parenthesized - grouping in some cases. This case has been added to test support. The - change additionally adjusts the "automatically add ORDER BY columns when - DISTINCT is present" behavior of ORM query, deprecated in 1.4, to more - accurately detect column expressions that are already present. - - .. change:: - :tags: usecase, mysql - :tickets: 5481 - - The MySQL dialect will render FROM DUAL for a SELECT statement that has no - FROM clause but has a WHERE clause. This allows things like "SELECT 1 WHERE - EXISTS (subquery)" kinds of queries to be used as well as other use cases. - - - .. change:: - :tags: bug, mssql, sql - :tickets: 5467 - - Fixed bug where the mssql dialect incorrectly escaped object names that - contained ']' character(s). - - .. change:: - :tags: bug, reflection, sqlite, mssql - :tickets: 5456 - - Applied a sweep through all included dialects to ensure names that contain - single or double quotes are properly escaped when querying system tables, - for all :class:`.Inspector` methods that accept object names as an argument - (e.g. table names, view names, etc). SQLite and MSSQL contained two - quoting issues that were repaired. - - .. change:: - :tags: bug, mysql - :tickets: 5411 - - Fixed an issue where CREATE TABLE statements were not specifying the - COLLATE keyword correctly. - - .. change:: - :tags: bug, datatypes, sql - :tickets: 4733 - - The ``LookupError`` message will now provide the user with up to four - possible values that a column is constrained to via the :class:`.Enum`. - Values longer than 11 characters will be truncated and replaced with - ellipses. Pull request courtesy Ramon Williams. - - .. change:: - :tags: bug, postgresql - :tickets: 5476 - - Fixed issue where the return type for the various RANGE comparison - operators would itself be the same RANGE type rather than BOOLEAN, which - would cause an undesirable result in the case that a - :class:`.TypeDecorator` that defined result-processing behavior were in - use. Pull request courtesy Jim Bosch. - - - - .. change:: - :tags: bug, mysql - :tickets: 5493 - - Added MariaDB code 1927 to the list of "disconnect" codes, as recent - MariaDB versions apparently use this code when the database server was - stopped. - - .. change:: - :tags: usecase, declarative, orm - :tickets: 5513 - - The name of the virtual column used when using the - :class:`_declarative.AbstractConcreteBase` and - :class:`_declarative.ConcreteBase` classes can now be customized, to allow - for models that have a column that is actually named ``type``. Pull - request courtesy Jesse-Bakker. - - .. change:: - :tags: usecase, orm - :tickets: 5494 - - Adjusted the workings of the :meth:`_orm.Mapper.all_orm_descriptors` - accessor to represent the attributes in the order that they are located in - a deterministic way, assuming the use of Python 3.6 or higher which - maintains the sorting order of class attributes based on how they were - declared. This sorting is not guaranteed to match the declared order of - attributes in all cases however; see the method documentation for the exact - scheme. - - - - .. change:: - :tags: bug, sql - :tickets: 5500 - - Fixed issue where the - :paramref:`_engine.Connection.execution_options.schema_translate_map` - feature would not take effect when the :meth:`_schema.Sequence.next_value` - function function for a :class:`_schema.Sequence` were used in the - :paramref:`_schema.Column.server_default` parameter and the create table - DDL were emitted. - -.. changelog:: - :version: 1.3.18 - :released: June 25, 2020 - - .. change:: - :tags: bug, sqlite - :tickets: 5395 - - Added "exists" to the list of reserved words for SQLite so that this word - will be quoted when used as a label or column name. Pull request courtesy - Thodoris Sotiropoulos. - - .. change:: - :tags: bug, mssql - :tickets: 5366, 5364 - - Refined the logic used by the SQL Server dialect to interpret multi-part - schema names that contain many dots, to not actually lose any dots if the - name does not have bracking or quoting used, and additionally to support a - "dbname" token that has many parts including that it may have multiple, - independently-bracketed sections. - - - - .. change:: - :tags: bug, mssql, pyodbc - :tickets: 5346 - - Fixed an issue in the pyodbc connector such that a warning about pyodbc - "drivername" would be emitted when using a totally empty URL. Empty URLs - are normal when producing a non-connected dialect object or when using the - "creator" argument to create_engine(). The warning now only emits if the - driver name is missing but other parameters are still present. - - .. change:: - :tags: bug, mssql - :tickets: 5373 - - Fixed issue with assembling the ODBC connection string for the pyodbc - DBAPI. Tokens containing semicolons and/or braces "{}" were not being - correctly escaped, causing the ODBC driver to misinterpret the - connection string attributes. - - .. change:: - :tags: usecase, orm - :tickets: 5326 - - Improve error message when using :meth:`_query.Query.filter_by` in - a query where the first entity is not a mapped class. - - .. change:: - :tags: sql, schema - :tickets: 5324 - - Introduce :class:`.IdentityOptions` to store common parameters for - sequences and identity columns. - - .. change:: - :tags: usecase, sql - :tickets: 5309 - - Added a ".schema" parameter to the :func:`_expression.table` construct, - allowing ad-hoc table expressions to also include a schema name. - Pull request courtesy Dylan Modesitt. - - .. change:: - :tags: bug, mssql - :tickets: 5339 - - Fixed issue where ``datetime.time`` parameters were being converted to - ``datetime.datetime``, making them incompatible with comparisons like - ``>=`` against an actual :class:`_mssql.TIME` column. - - .. change:: - :tags: bug, mssql - :tickets: 5359 - - Fixed an issue where the ``is_disconnect`` function in the SQL Server - pyodbc dialect was incorrectly reporting the disconnect state when the - exception message had a substring that matched a SQL Server ODBC error - code. - - .. change:: - :tags: bug, engine - :tickets: 5326 - - Further refinements to the fixes to the "reset" agent fixed in - :ticket:`5326`, which now emits a warning when it is not being correctly - invoked and corrects for the behavior. Additional scenarios have been - identified and fixed where this warning was being emitted. - - - .. change:: - :tags: usecase, sqlite - :tickets: 5297 - - SQLite 3.31 added support for computed column. This change - enables their support in SQLAlchemy when targeting SQLite. - - .. change:: - :tags: bug, schema - :tickets: 5276 - - Fixed issue where ``dialect_options`` were omitted when a - database object (e.g., :class:`.Table`) was copied using - :func:`.tometadata`. - - .. change:: - :tags: bug, sql - :tickets: 5344 - - Correctly apply self_group in type_coerce element. - - The type coerce element did not correctly apply grouping rules when using - in an expression - - .. change:: - :tags: bug, oracle, reflection - :tickets: 5421 - - Fixed bug in Oracle dialect where indexes that contain the full set of - primary key columns would be mistaken as the primary key index itself, - which is omitted, even if there were multiples. The check has been refined - to compare the name of the primary key constraint against the index name - itself, rather than trying to guess based on the columns present in the - index. - - .. change:: - :tags: change, sql, sybase - :tickets: 5294 - - Added ``.offset`` support to sybase dialect. - Pull request courtesy Alan D. Snow. - - .. change:: - :tags: bug, engine - :tickets: 5341 - - Fixed issue in :class:`.URL` object where stringifying the object - would not URL encode special characters, preventing the URL from being - re-consumable as a real URL. Pull request courtesy Miguel Grinberg. - - .. change:: - :tags: usecase, mysql - :tickets: 4860 - - Implemented row-level locking support for mysql. Pull request courtesy - Quentin Somerville. - - .. change:: - :tags: change, mssql - :tickets: 5321 - - Moved the ``supports_sane_rowcount_returning = False`` requirement from - the ``PyODBCConnector`` level to the ``MSDialect_pyodbc`` since pyodbc - does work properly in some circumstances. - - .. change:: - :tags: change, examples - - Added new option ``--raw`` to the examples.performance suite - which will dump the raw profile test for consumption by any - number of profiling visualizer tools. Removed the "runsnake" - option as runsnake is very hard to build at this point; - - .. change:: - :tags: bug, sql - :tickets: 5353 - - Added :meth:`.Select.with_hint` output to the generic SQL string that is - produced when calling ``str()`` on a statement. Previously, this clause - would be omitted under the assumption that it was dialect specific. - The hint text is presented within brackets to indicate the rendering - of such hints varies among backends. - - - .. change:: - :tags: usecase, orm - :tickets: 5198 - - Added a new parameter :paramref:`_orm.query_expression.default_expr` to the - :func:`_orm.query_expression` construct, which will be appled to queries - automatically if the :func:`_orm.with_expression` option is not used. Pull - request courtesy Haoyu Sun. - -.. changelog:: - :version: 1.3.17 - :released: May 13, 2020 - - .. change:: - :tags: bug, oracle - :tickets: 5246 - - Some modifications to how the cx_oracle dialect sets up per-column - outputtype handlers for LOB and numeric datatypes to adjust for potential - changes coming in cx_Oracle 8. - - - .. change:: - :tags: bug, orm - :tickets: 5288 - - Fixed bug where using :func:`.with_polymorphic` as the target of a join via - :meth:`.RelationshipComparator.of_type` on a mapper that already has a - subquery-based with_polymorphic setting that's equivalent to the one - requested would not correctly alias the ON clause in the join. - - .. change:: - :tags: bug, oracle, performance - :tickets: 5314 - - Changed the implementation of fetching CLOB and BLOB objects to use - cx_Oracle's native implementation which fetches CLOB/BLOB objects inline - with other result columns, rather than performing a separate fetch. As - always, this can be disabled by setting auto_convert_lobs to False. - - As part of this change, the behavior of a CLOB that was given a blank - string on INSERT now returns None on SELECT, which is now consistent with - that of VARCHAR on Oracle. - - - .. change:: - :tags: usecase, postgresql - :tickets: 5265 - - Added support for columns or type :class:`_sqltypes.ARRAY` of :class:`.Enum`, - :class:`_postgresql.JSON` or :class:`_postgresql.JSONB` in PostgreSQL. - Previously a workaround was required in these use cases. - - - .. change:: - :tags: schema - :tickets: 4138 - - Add ``comment`` attribute to :class:`_schema.Column` ``__repr__`` method. - - .. change:: - :tags: bug, orm - :tickets: 5303 - - Fixed issue in the area of where loader options such as selectinload() - interact with the baked query system, such that the caching of a query is - not supposed to occur if the loader options themselves have elements such - as with_polymorphic() objects in them that currently are not - cache-compatible. The baked loader could sometimes not fully invalidate - itself in these some of these scenarios leading to missed eager loads. - - - .. change:: - :tags: bug, engine - :tickets: 5326 - - Fixed fairly critical issue where the DBAPI connection could be returned to - the connection pool while still in an un-rolled-back state. The reset agent - responsible for rolling back the connection could be corrupted in the case - that the transaction was "closed" without being rolled back or committed, - which can occur in some scenarios when using ORM sessions and emitting - .close() in a certain pattern involving savepoints. The fix ensures that - the reset agent is always active. - - - .. change:: - :tags: bug, orm - :tickets: 5304 - - Modified the internal "identity set" implementation, which is a set that - hashes objects on their id() rather than their hash values, to not actually - call the ``__hash__()`` method of the objects, which are typically - user-mapped objects. Some methods were calling this method as a side - effect of the implementation. - - - .. change:: - :tags: usecase, postgresql - :tickets: 5266 - - Raise an explicit :class:`.exc.CompileError` when adding a table with a - column of type :class:`_sqltypes.ARRAY` of :class:`.Enum` configured with - :paramref:`.Enum.native_enum` set to ``False`` when - :paramref:`.Enum.create_constraint` is not set to ``False`` - - .. change:: - :tags: bug, schema - :tickets: 5298 - - Fixed issue where an :class:`.Index` that is deferred in being associated - with a table, such as as when it contains a :class:`.Column` that is not - associated with any :class:`.Table` yet, would fail to attach correctly if - it also contained a non table-oriented expression. - - - .. change:: - :tags: change, firebird - :tickets: 5278 - - Adjusted dialect loading for ``firebird://`` URIs so the external - sqlalchemy-firebird dialect will be used if it has been installed, - otherwise fall back to the (now deprecated) internal Firebird dialect. - - .. change:: - :tags: bug, mssql, reflection - :tickets: 5255 - - Fix a regression introduced by the reflection of computed column in - MSSQL when using the legacy TDS version 4.2. The dialect will try - to detect the protocol version of first connect and run in compatibility - mode if it cannot detect it. - - .. change:: - :tags: bug, mssql, reflection - :tickets: 5271 - - Fix a regression introduced by the reflection of computed column in - MSSQL when using SQL server versions before 2012, which does not support - the ``concat`` function. - - .. change:: - :tags: bug, orm - :tickets: 5269 - - An informative error message is raised when an ORM many-to-one comparison - is attempted against an object that is not an actual mapped instance. - Comparisons such as those to scalar subqueries aren't supported; - generalized comparison with subqueries is better achieved using - :meth:`~.RelationshipProperty.Comparator.has`. - - - .. change:: - :tags: usecase, orm - :tickets: 5262 - - Added an accessor :attr:`.ColumnProperty.Comparator.expressions` which - provides access to the group of columns mapped under a multi-column - :class:`.ColumnProperty` attribute. - - - .. change:: - :tags: bug, schema - :tickets: 5316 - - A warning is emitted when making use of the :attr:`.MetaData.sorted_tables` - attribute as well as the :func:`_schema.sort_tables` function, and the - given tables cannot be correctly sorted due to a cyclic dependency between - foreign key constraints. In this case, the functions will no longer sort - the involved tables by foreign key, and a warning will be emitted. Other - tables that are not part of the cycle will still be returned in dependency - order. Previously, the sorted_table routines would return a collection that - would unconditionally omit all foreign keys when a cycle was detected, and - no warning was emitted. - - - .. change:: - :tags: orm, usecase - :tickets: 5237 - - Introduce :paramref:`_orm.relationship.sync_backref` flag in a relationship - to control if the synchronization events that mutate the in-Python - attributes are added. This supersedes the previous change :ticket:`5149`, - which warned that ``viewonly=True`` relationship target of a - back_populates or backref configuration would be disallowed. - -.. changelog:: - :version: 1.3.16 - :released: April 7, 2020 - - .. change:: - :tags: oracle, usecase - :tickets: 5200 - - Implemented AUTOCOMMIT isolation level for Oracle when using cx_Oracle. - Also added a fixed default isolation level of READ COMMITTED for Oracle. - - - .. change:: - :tags: bug, mysql - :tickets: 5239 - - Fixed issue in MySQL dialect when connecting to a pseudo-MySQL database - such as that provided by ProxySQL, the up front check for isolation level - when it returns no row will not prevent the dialect from continuing to - connect. A warning is emitted that the isolation level could not be - detected. - - - .. change:: - :tags: bug, tests - :tickets: 5201 - - Fixed an issue that prevented the test suite from running with the - recently released py.test 5.4.0. - - - .. change:: - :tags: bug, oracle, reflection - :tickets: 5146 - - Fixed regression / incorrect fix caused by fix for :ticket:`5146` where the - Oracle dialect reads from the "all_tab_comments" view to get table comments - but fails to accommodate for the current owner of the table being - requested, causing it to read the wrong comment if multiple tables of the - same name exist in multiple schemas. - - - .. change:: - :tags: types, enum - :tickets: 5183 - - The :class:`.Enum` type now supports the parameter :paramref:`.Enum.length` - to specify the length of the VARCHAR column to create when using - non native enums by setting :paramref:`.Enum.native_enum` to ``False`` - - .. change:: - :tags: bug, orm - :tickets: 5228 - - Fixed bug in :func:`_orm.selectinload` loading option where two or more - loaders that represent different relationships with the same string key - name as referenced from a single :func:`_orm.with_polymorphic` construct - with multiple subclass mappers would fail to invoke each subqueryload - separately, instead making use of a single string-based slot that would - prevent the other loaders from being invoked. - - - .. change:: - :tags: schema, reflection - :tickets: 5063 - - Added support for reflection of "computed" columns, which are now returned - as part of the structure returned by :meth:`_reflection.Inspector.get_columns`. - When reflecting full :class:`_schema.Table` objects, computed columns will - be represented using the :class:`.Computed` construct. - - .. change:: - :tags: orm, performance - :tickets: 5162 - - Modified the queries used by subqueryload and selectinload to no longer - ORDER BY the primary key of the parent entity; this ordering was there to - allow the rows as they come in to be copied into lists directly with a - minimal level of Python-side collation. However, these ORDER BY clauses - can negatively impact the performance of the query as in many scenarios - these columns are derived from a subquery or are otherwise not actual - primary key columns such that SQL planners cannot make use of indexes. The - Python-side collation uses the native itertools.group_by() to collate the - incoming rows, and has been modified to allow multiple - row-groups-per-parent to be assembled together using list.extend(), which - should still allow for relatively fast Python-side performance. There will - still be an ORDER BY present for a relationship that includes an explicit - order_by parameter, however this is the only ORDER BY that will be added to - the query for both kinds of loading. - - .. change:: - :tags: mssql, mysql, oracle, usecase - :tickets: 5137 - - Added support for :meth:`.ColumnOperators.is_distinct_from` and - :meth:`.ColumnOperators.isnot_distinct_from` to SQL Server, - MySQL, and Oracle. - - .. change:: - :tags: sqlite, usecase - :tickets: 5164 - - Implemented AUTOCOMMIT isolation level for SQLite when using pysqlite. - - .. change:: - :tags: bug, postgresql - :tickets: 5205 - - Fixed issue where a "covering" index, e.g. those which have an INCLUDE - clause, would be reflected including all the columns in INCLUDE clause as - regular columns. A warning is now emitted if these additional columns are - detected indicating that they are currently ignored. Note that full - support for "covering" indexes is part of :ticket:`4458`. Pull request - courtesy Marat Sharafutdinov. - - .. change:: - :tags: sql, types - :tickets: 5052 - - Add ability to literal compile a :class:`DateTime`, :class:`Date` - or :class:`Time` when using the string dialect for debugging purposes. - This change does not impact real dialect implementation that retain - their current behavior. - - .. change:: - :tags: installer - :tickets: 5207 - - Ensured that the "pyproject.toml" file is not included in builds, as the - presence of this file indicates to pip that a pep-517 installation process - should be used. As this mode of operation appears to be not well supported - by current tools / distros, these problems are avoided within the scope - of SQLAlchemy installation by omitting the file. - - - .. change:: - :tags: bug, orm - :tickets: 5210 - - Fixed issue where a lazyload that uses session-local "get" against a target - many-to-one relationship where an object with the correct primary key is - present, however it's an instance of a sibling class, does not correctly - return None as is the case when the lazy loader actually emits a load for - that row. - - .. change:: - :tags: bug, orm, declarative - :tickets: 5238 - - The string argument accepted as the first positional argument by the - :func:`_orm.relationship` function when using the Declarative API is no longer - interpreted using the Python ``eval()`` function; instead, the name is dot - separated and the names are looked up directly in the name resolution - dictionary without treating the value as a Python expression. However, - passing a string argument to the other :func:`_orm.relationship` parameters - that necessarily must accept Python expressions will still use ``eval()``; - the documentation has been clarified to ensure that there is no ambiguity - that this is in use. - - .. seealso:: - - :ref:`declarative_relationship_eval` - details on string evaluation - -.. changelog:: - :version: 1.3.15 - :released: March 11, 2020 - - .. change:: - :tags: bug, orm - :tickets: 5194 - - Adjusted the error message emitted by :meth:`_query.Query.join` when a left hand - side can't be located that the :meth:`_query.Query.select_from` method is the - best way to resolve the issue. Also, within the 1.3 series, used a - deterministic ordering when determining the FROM clause from a given column - entity passed to :class:`_query.Query` so that the same expression is determined - each time. - - - .. change:: - :tags: orm, bug - :tickets: 5196 - - Fixed regression in 1.3.14 due to :ticket:`4849` where a sys.exc_info() - call failed to be invoked correctly when a flush error would occur. Test - coverage has been added for this exception case. - - -.. changelog:: - :version: 1.3.14 - :released: March 10, 2020 - - .. change:: - :tags: bug, sql, postgresql - :tickets: 5181 - - Fixed bug where a CTE of an INSERT/UPDATE/DELETE that also uses RETURNING - could then not be SELECTed from directly, as the internal state of the - compiler would try to treat the outer SELECT as a DELETE statement itself - and access nonexistent state. - - - .. change:: - :tags: bug, orm - :tickets: 5110 - - Fixed regression caused in 1.3.13 by :ticket:`5056` where a refactor of the - ORM path registry system made it such that a path could no longer be - compared to an empty tuple, which can occur in a particular kind of joined - eager loading path. The "empty tuple" use case has been resolved so that - the path registry is compared to a path registry in all cases; the - :class:`.PathRegistry` object itself now implements ``__eq__()`` and - ``__ne__()`` methods which will take place for all equality comparisons and - continue to succeed in the not anticipated case that a non- - :class:`.PathRegistry` object is compared, while emitting a warning that - this object should not be the subject of the comparison. - - - - .. change:: - :tags: bug, orm - :tickets: 5149 - - Setting a relationship to viewonly=True which is also the target of a - back_populates or backref configuration will now emit a warning and - eventually be disallowed. back_populates refers specifically to mutation - of an attribute or collection, which is disallowed when the attribute is - subject to viewonly=True. The viewonly attribute is not subject to - persistence behaviors which means it will not reflect correct results - when it is locally mutated. - - .. change:: - :tags: bug, oracle - :tickets: 5146 - - Fixed a reflection bug where table comments could only be retrieved for - tables actually owned by the user but not for tables visible to the user - but owned by someone else. Pull request courtesy Dave Hirschfeld. - - .. change:: - :tags: bug, performance - :tickets: 5180 - - Revised an internal change to the test system added as a result of - :ticket:`5085` where a testing-related module per dialect would be loaded - unconditionally upon making use of that dialect, pulling in SQLAlchemy's - testing framework as well as the ORM into the module import space. This - would only impact initial startup time and memory to a modest extent, - however it's best that these additional modules aren't reverse-dependent on - straight Core usage. - - .. change:: - :tags: bug, installation - :tickets: 5138 - - Vendored the ``inspect.formatannotation`` function inside of - ``sqlalchemy.util.compat``, which is needed for the vendored version of - ``inspect.formatargspec``. The function is not documented in cPython and - is not guaranteed to be available in future Python versions. - - - .. change:: - :tags: bug, mssql - :tickets: 5132 - - Fixed issue where the :class:`_mssql.DATETIMEOFFSET` type would not - accommodate for the ``None`` value, introduced as part of the series of - fixes for this type first introduced in :ticket:`4983`, :ticket:`5045`. - Additionally, added support for passing a backend-specific date formatted - string through this type, as is typically allowed for date/time types on - most other DBAPIs. - - .. change:: - :tags: bug, engine - :tickets: 5182 - - Expanded the scope of cursor/connection cleanup when a statement is - executed to include when the result object fails to be constructed, or an - after_cursor_execute() event raises an error, or autocommit / autoclose - fails. This allows the DBAPI cursor to be cleaned up on failure and for - connectionless execution allows the connection to be closed out and - returned to the connection pool, where previously it waiting until garbage - collection would trigger a pool return. - - .. change:: - :tags: bug, postgresql - :tickets: 5158 - - Fixed issue where the "schema_translate_map" feature would not work with a - PostgreSQL native enumeration type (i.e. :class:`.Enum`, - :class:`_postgresql.ENUM`) in that while the "CREATE TYPE" statement would - be emitted with the correct schema, the schema would not be rendered in - the CREATE TABLE statement at the point at which the enumeration was - referenced. - - - .. change:: - :tags: usecase, ext - :tickets: 5114 - - Added keyword arguments to the :meth:`.MutableList.sort` function so that a - key function as well as the "reverse" keyword argument can be provided. - - - .. change:: - :tags: bug, general, py3k - :tickets: 4849 - - Applied an explicit "cause" to most if not all internally raised exceptions - that are raised from within an internal exception catch, to avoid - misleading stacktraces that suggest an error within the handling of an - exception. While it would be preferable to suppress the internally caught - exception in the way that the ``__suppress_context__`` attribute would, - there does not as yet seem to be a way to do this without suppressing an - enclosing user constructed context, so for now it exposes the internally - caught exception as the cause so that full information about the context - of the error is maintained. - - .. change:: - :tags: orm, bug - :tickets: 5121 - - Fixed an additional regression in the same area as that of :ticket:`5080` - introduced in 1.3.0b3 via :ticket:`4468` where the ability to create a - joined option across a :func:`.with_polymorphic` into a relationship - against the base class of that with_polymorphic, and then further into - regular mapped relationships would fail as the base class component would - not add itself to the load path in a way that could be located by the - loader strategy. The changes applied in :ticket:`5080` have been further - refined to also accommodate this scenario. - - .. change:: - :tags: bug, postgresql, reflection - :tickets: 5170 - - Fixed bug where PostgreSQL reflection of CHECK constraints would fail to - parse the constraint if the SQL text contained newline characters. The - regular expression has been adjusted to accommodate for this case. Pull - request courtesy Eric Borczuk. - - .. change:: - :tags: usecase, orm - :tickets: 5129 - - Added a new flag :paramref:`.InstanceEvents.restore_load_context` and - :paramref:`.SessionEvents.restore_load_context` which apply to the - :meth:`.InstanceEvents.load`, :meth:`.InstanceEvents.refresh`, and - :meth:`.SessionEvents.loaded_as_persistent` events, which when set will - restore the "load context" of the object after the event hook has been - called. This ensures that the object remains within the "loader context" - of the load operation that is already ongoing, rather than the object being - transferred to a new load context due to refresh operations which may have - occurred in the event. A warning is now emitted when this condition occurs, - which recommends use of the flag to resolve this case. The flag is - "opt-in" so that there is no risk introduced to existing applications. - - The change additionally adds support for the ``raw=True`` flag to - session lifecycle events. - - .. change:: - :tags: bug, mysql - :tickets: 5173 - - Fixed issue in MySQL :meth:`.mysql.Insert.on_duplicate_key_update` construct - where using a SQL function or other composed expression for a column argument - would not properly render the ``VALUES`` keyword surrounding the column - itself. - -.. changelog:: - :version: 1.3.13 - :released: January 22, 2020 - - .. change:: - :tags: bug, postgresql - :tickets: 5039 - - Fixed issue where the PostgreSQL dialect would fail to parse a reflected - CHECK constraint that was a boolean-valued function (as opposed to a - boolean-valued expression). - - .. change:: - :tags: bug, ext - :tickets: 5086 - - Fixed bug in sqlalchemy.ext.serializer where a unique - :class:`.BindParameter` object could conflict with itself if it were - present in the mapping itself, as well as the filter condition of the - query, as one side would be used against the non-deserialized version and - the other side would use the deserialized version. Logic is added to - :class:`.BindParameter` similar to its "clone" method which will uniquify - the parameter name upon deserialize so that it doesn't conflict with its - original. - - - .. change:: - :tags: usecase, sql - :tickets: 5079 - - A function created using :class:`.GenericFunction` can now specify that the - name of the function should be rendered with or without quotes by assigning - the :class:`.quoted_name` construct to the .name element of the object. - Prior to 1.3.4, quoting was never applied to function names, and some - quoting was introduced in :ticket:`4467` but no means to force quoting for - a mixed case name was available. Additionally, the :class:`.quoted_name` - construct when used as the name will properly register its lowercase name - in the function registry so that the name continues to be available via the - ``func.`` registry. - - .. seealso:: - - :class:`.GenericFunction` - - - .. change:: - :tags: bug, engine - :tickets: 5048 - - Fixed issue where the collection of value processors on a - :class:`.Compiled` object would be mutated when "expanding IN" parameters - were used with a datatype that has bind value processors; in particular, - this would mean that when using statement caching and/or baked queries, the - same compiled._bind_processors collection would be mutated concurrently. - Since these processors are the same function for a given bind parameter - namespace every time, there was no actual negative effect of this issue, - however, the execution of a :class:`.Compiled` object should never be - causing any changes in its state, especially given that they are intended - to be thread-safe and reusable once fully constructed. - - - .. change:: - :tags: tests, postgresql - :tickets: 5057 - - Improved detection of two phase transactions requirement for the PostgreSQL - database by testing that max_prepared_transactions is set to a value - greater than 0. Pull request courtesy Federico Caselli. - - - .. change:: - :tags: bug, orm, engine - :tickets: 5056, 5050, 5071 - - Added test support and repaired a wide variety of unnecessary reference - cycles created for short-lived objects, mostly in the area of ORM queries. - Thanks much to Carson Ip for the help on this. - - - .. change:: - :tags: orm, bug - :tickets: 5107 - - Fixed regression in loader options introduced in 1.3.0b3 via :ticket:`4468` - where the ability to create a loader option using - :meth:`.PropComparator.of_type` targeting an aliased entity that is an - inheriting subclass of the entity which the preceding relationship refers - to would fail to produce a matching path. See also :ticket:`5082` fixed - in this same release which involves a similar kind of issue. - - .. change:: - :tags: bug, tests - :tickets: 4946 - - Fixed a few test failures which would occur on Windows due to SQLite file - locking issues, as well as some timing issues in connection pool related - tests; pull request courtesy Federico Caselli. - - - .. change:: - :tags: orm, bug - :tickets: 5082 - - Fixed regression in joined eager loading introduced in 1.3.0b3 via - :ticket:`4468` where the ability to create a joined option across a - :func:`.with_polymorphic` into a polymorphic subclass using - :meth:`.RelationshipProperty.of_type` and then further along regular mapped - relationships would fail as the polymorphic subclass would not add itself - to the load path in a way that could be located by the loader strategy. A - tweak has been made to resolve this scenario. - - - .. change:: - :tags: performance, orm - - Identified a performance issue in the system by which a join is constructed - based on a mapped relationship. The clause adaption system would be used - for the majority of join expressions including in the common case where no - adaptation is needed. The conditions under which this adaptation occur - have been refined so that average non-aliased joins along a simple - relationship without a "secondary" table use about 70% less function calls. - - - .. change:: - :tags: usecase, postgresql - :tickets: 5040 - - Added support for prefixes to the :class:`_expression.CTE` construct, to allow - support for Postgresql 12 "MATERIALIZED" and "NOT MATERIALIZED" phrases. - Pull request courtesy Marat Sharafutdinov. - - .. seealso:: - - :meth:`_expression.HasCTE.cte` - - .. change:: - :tags: bug, mssql - :tickets: 5045 - - Fixed issue where a timezone-aware ``datetime`` value being converted to - string for use as a parameter value of a :class:`_mssql.DATETIMEOFFSET` - column was omitting the fractional seconds. - - .. change:: - :tags: bug, orm - :tickets: 5068 - - Repaired a warning in the ORM flush process that was not covered by test - coverage when deleting objects that use the "version_id" feature. This - warning is generally unreachable unless using a dialect that sets the - "supports_sane_rowcount" flag to False, which is not typically the case - however is possible for some MySQL configurations as well as older Firebird - drivers, and likely some third party dialects. - - .. change:: - :tags: bug, orm - :tickets: 5065 - - Fixed bug where usage of joined eager loading would not properly wrap the - query inside of a subquery when :meth:`_query.Query.group_by` were used against - the query. When any kind of result-limiting approach is used, such as - DISTINCT, LIMIT, OFFSET, joined eager loading embeds the row-limited query - inside of a subquery so that the collection results are not impacted. For - some reason, the presence of GROUP BY was never included in this criterion, - even though it has a similar effect as using DISTINCT. Additionally, the - bug would prevent using GROUP BY at all for a joined eager load query for - most database platforms which forbid non-aggregated, non-grouped columns - from being in the query, as the additional columns for the joined eager - load would not be accepted by the database. - - - -.. changelog:: - :version: 1.3.12 - :released: December 16, 2019 - - .. change:: - :tags: bug, sql - :tickets: 5028 - - Fixed bug where "distinct" keyword passed to :func:`_expression.select` would not - treat a string value as a "label reference" in the same way that the - :meth:`_expression.select.distinct` does; it would instead raise unconditionally. This - keyword argument and the others passed to :func:`_expression.select` will ultimately - be deprecated for SQLAlchemy 2.0. - - - .. change:: - :tags: bug, orm - :tickets: 4997 - - Fixed issue involving ``lazy="raise"`` strategy where an ORM delete of an - object would raise for a simple "use-get" style many-to-one relationship - that had lazy="raise" configured. This is inconsistent vs. the change - introduced in 1.3 as part of :ticket:`4353`, where it was established that - a history operation that does not expect emit SQL should bypass the - ``lazy="raise"`` check, and instead effectively treat it as - ``lazy="raise_on_sql"`` for this case. The fix adjusts the lazy loader - strategy to not raise for the case where the lazy load was instructed that - it should not emit SQL if the object were not present. - - .. change:: - :tags: bug, sql - - Changed the text of the exception for "Can't resolve label reference" to - include other kinds of label coercions, namely that "DISTINCT" is also in - this category under the PostgreSQL dialect. - - - .. change:: - :tags: bug, orm - :tickets: 5000 - - Fixed regression introduced in 1.3.0 related to the association proxy - refactor in :ticket:`4351` that prevented :func:`.composite` attributes - from working in terms of an association proxy that references them. - - .. change:: - :tags: bug, mssql - :tickets: 4983 - - Repaired support for the :class:`_mssql.DATETIMEOFFSET` datatype on PyODBC, - by adding PyODBC-level result handlers as it does not include native - support for this datatype. This includes usage of the Python 3 "timezone" - tzinfo subclass in order to set up a timezone, which on Python 2 makes - use of a minimal backport of "timezone" in sqlalchemy.util. - - - .. change:: - :tags: bug, orm - :tickets: 4993 - - Setting persistence-related flags on :func:`_orm.relationship` while also - setting viewonly=True will now emit a regular warning, as these flags do - not make sense for a viewonly=True relationship. In particular, the - "cascade" settings have their own warning that is generated based on the - individual values, such as "delete, delete-orphan", that should not apply - to a viewonly relationship. Note however that in the case of "cascade", - these settings are still erroneously taking effect even though the - relationship is set up as "viewonly". In 1.4, all persistence-related - cascade settings will be disallowed on a viewonly=True relationship in - order to resolve this issue. - - .. change:: - :tags: bug, sqlite - :tickets: 5014 - - Fixed issue to workaround SQLite's behavior of assigning "numeric" affinity - to JSON datatypes, first described at :ref:`change_3850`, which returns - scalar numeric JSON values as a number and not as a string that can be JSON - deserialized. The SQLite-specific JSON deserializer now gracefully - degrades for this case as an exception and bypasses deserialization for - single numeric values, as from a JSON perspective they are already - deserialized. - - - - .. change:: - :tags: bug, orm, py3k - :tickets: 4990 - - Fixed issue where when assigning a collection to itself as a slice, the - mutation operation would fail as it would first erase the assigned - collection inadvertently. As an assignment that does not change the - contents should not generate events, the operation is now a no-op. Note - that the fix only applies to Python 3; in Python 2, the ``__setitem__`` - hook isn't called in this case; ``__setslice__`` is used instead which - recreates the list item-by-item in all cases. - - .. change:: - :tags: bug, orm - :tickets: 5034 - - Fixed issue where by if the "begin" of a transaction failed at the Core - engine/connection level, such as due to network error or database is locked - for some transactional recipes, within the context of the :class:`.Session` - procuring that connection from the connection pool and then immediately - returning it, the ORM :class:`.Session` would not close the connection - despite this connection not being stored within the state of that - :class:`.Session`. This would lead to the connection being cleaned out by - the connection pool weakref handler within garbage collection which is an - unpreferred codepath that in some special configurations can emit errors in - standard error. - -.. changelog:: - :version: 1.3.11 - :released: November 11, 2019 - - .. change:: - :tags: bug, mssql - :tickets: 4973 - - Fixed issue in MSSQL dialect where an expression-based OFFSET value in a - SELECT would be rejected, even though the dialect can render this - expression inside of a ROW NUMBER-oriented LIMIT/OFFSET construct. - - - .. change:: - :tags: orm, usecase - :tickets: 4934 - - Added accessor :meth:`_query.Query.is_single_entity` to :class:`_query.Query`, which - will indicate if the results returned by this :class:`_query.Query` will be a - list of ORM entities, or a tuple of entities or column expressions. - SQLAlchemy hopes to improve upon the behavior of single entity / tuples in - future releases such that the behavior would be explicit up front, however - this attribute should be helpful with the current behavior. Pull request - courtesy Patrick Hayes. - - .. change:: - :tags: bug, mysql - :tickets: 4945 - - Added "Connection was killed" message interpreted from the base - pymysql.Error class in order to detect closed connection, based on reports - that this message is arriving via a pymysql.InternalError() object which - indicates pymysql is not handling it correctly. - - .. change:: - :tags: bug, orm - :tickets: 4954 - - The :paramref:`_orm.relationship.omit_join` flag was not intended to be - manually set to True, and will now emit a warning when this occurs. The - omit_join optimization is detected automatically, and the ``omit_join`` - flag was only intended to disable the optimization in the hypothetical case - that the optimization may have interfered with correct results, which has - not been observed with the modern version of this feature. Setting the - flag to True when it is not automatically detected may cause the selectin - load feature to not work correctly when a non-default primary join - condition is in use. - - - .. change:: - :tags: bug, orm - :tickets: 4915 - - A warning is emitted if a primary key value is passed to :meth:`_query.Query.get` - that consists of None for all primary key column positions. Previously, - passing a single None outside of a tuple would raise a ``TypeError`` and - passing a composite None (tuple of None values) would silently pass - through. The fix now coerces the single None into a tuple where it is - handled consistently with the other None conditions. Thanks to Lev - Izraelit for the help with this. - - - .. change:: - :tags: bug, orm - :tickets: 4947 - - The :class:`.BakedQuery` will not cache a query that was modified by a - :meth:`.QueryEvents.before_compile` event, so that compilation hooks that - may be applying ad-hoc modifications to queries will take effect on each - run. In particular this is helpful for events that modify queries used in - lazy loading as well as eager loading such as "select in" loading. In - order to re-enable caching for a query modified by this event, a new - flag ``bake_ok`` is added; see :ref:`baked_with_before_compile` for - details. - - A longer term plan to provide a new form of SQL caching should solve this - kind of issue more comprehensively. - - .. change:: - :tags: bug, tests - :tickets: 4920 - - Fixed test failures which would occur with newer SQLite as of version 3.30 - or greater, due to their addition of nulls ordering syntax as well as new - restrictions on aggregate functions. Pull request courtesy Nils Philippsen. - - - - .. change:: - :tags: bug, installation, windows - :tickets: 4967 - - Added a workaround for a setuptools-related failure that has been observed - as occurring on Windows installations, where setuptools is not correctly - reporting a build error when the MSVC build dependencies are not installed - and therefore not allowing graceful degradation into non C extensions - builds. - - .. change:: - :tags: bug, sql, py3k - :tickets: 4931 - - Changed the ``repr()`` of the :class:`.quoted_name` construct to use - regular string repr() under Python 3, rather than running it through - "backslashreplace" escaping, which can be misleading. - - .. change:: - :tags: bug, oracle, firebird - :tickets: 4931 - - Modified the approach of "name normalization" for the Oracle and Firebird - dialects, which converts from the UPPERCASE-as-case-insensitive convention - of these dialects into lowercase-as-case-insensitive for SQLAlchemy, to not - automatically apply the :class:`.quoted_name` construct to a name that - matches itself under upper or lower case conversion, as is the case for - many non-european characters. All names used within metadata structures - are converted to :class:`.quoted_name` objects in any case; the change - here would only affect the output of some inspection functions. - - .. change:: - :tags: bug, schema - :tickets: 4911 - - Fixed bug where a table that would have a column label overlap with a plain - column name, such as "foo.id AS foo_id" vs. "foo.foo_id", would prematurely - generate the ``._label`` attribute for a column before this overlap could - be detected due to the use of the ``index=True`` or ``unique=True`` flag on - the column in conjunction with the default naming convention of - ``"column_0_label"``. This would then lead to failures when ``._label`` - were used later to generate a bound parameter name, in particular those - used by the ORM when generating the WHERE clause for an UPDATE statement. - The issue has been fixed by using an alternate ``._label`` accessor for DDL - generation that does not affect the state of the :class:`_schema.Column`. The - accessor also bypasses the key-deduplication step as it is not necessary - for DDL, the naming is now consistently ``"_"`` - without any subsequent numeric symbols when used in DDL. - - - - .. change:: - :tags: bug, engine - :tickets: 4902 - - Fixed bug where parameter repr as used in logging and error reporting needs - additional context in order to distinguish between a list of parameters for - a single statement and a list of parameter lists, as the "list of lists" - structure could also indicate a single parameter list where the first - parameter itself is a list, such as for an array parameter. The - engine/connection now passes in an additional boolean indicating how the - parameters should be considered. The only SQLAlchemy backend that expects - arrays as parameters is that of psycopg2 which uses pyformat parameters, - so this issue has not been too apparent, however as other drivers that use - positional gain more features it is important that this be supported. It - also eliminates the need for the parameter repr function to guess based on - the parameter structure passed. - - .. change:: - :tags: usecase, schema - :tickets: 4894 - - Added DDL support for "computed columns"; these are DDL column - specifications for columns that have a server-computed value, either upon - SELECT (known as "virtual") or at the point of which they are INSERTed or - UPDATEd (known as "stored"). Support is established for Postgresql, MySQL, - Oracle SQL Server and Firebird. Thanks to Federico Caselli for lots of work - on this one. - - .. seealso:: - - :ref:`computed_ddl` - - - .. change:: - :tags: bug, engine, postgresql - :tickets: 4955 - - Fixed bug in :class:`_reflection.Inspector` where the cache key generation did not - take into account arguments passed in the form of tuples, such as the tuple - of view name styles to return for the PostgreSQL dialect. This would lead - the inspector to cache too generally for a more specific set of criteria. - The logic has been adjusted to include every keyword element in the cache, - as every argument is expected to be appropriate for a cache else the - caching decorator should be bypassed by the dialect. - - - .. change:: - :tags: bug, mssql - :tickets: 4923 - - Fixed an issue in the :meth:`_engine.Engine.table_names` method where it would - feed the dialect's default schema name back into the dialect level table - function, which in the case of SQL Server would interpret it as a - dot-tokenized schema name as viewed by the mssql dialect, which would - cause the method to fail in the case where the database username actually - had a dot inside of it. In 1.3, this method is still used by the - :meth:`_schema.MetaData.reflect` function so is a prominent codepath. In 1.4, - which is the current master development branch, this issue doesn't exist, - both because :meth:`_schema.MetaData.reflect` isn't using this method nor does the - method pass the default schema name explicitly. The fix nonetheless - guards against the default server name value returned by the dialect from - being interpreted as dot-tokenized name under any circumstances by - wrapping it in quoted_name(). - - .. change:: - :tags: bug, orm - :tickets: 4974 - - Fixed ORM bug where a "secondary" table that referred to a selectable which - in some way would refer to the local primary table would apply aliasing to - both sides of the join condition when a relationship-related join, either - via :meth:`_query.Query.join` or by :func:`_orm.joinedload`, were generated. The - "local" side is now excluded. - - .. change:: - :tags: usecase, sql - :tickets: 4276 - - Added new accessors to expressions of type :class:`_types.JSON` to allow for - specific datatype access and comparison, covering strings, integers, - numeric, boolean elements. This revises the documented approach of - CASTing to string when comparing values, instead adding specific - functionality into the PostgreSQL, SQlite, MySQL dialects to reliably - deliver these basic types in all cases. - - .. seealso:: - - :class:`_types.JSON` - - :meth:`_sqltypes.JSON.Comparator.as_string` - - :meth:`_sqltypes.JSON.Comparator.as_boolean` - - :meth:`_sqltypes.JSON.Comparator.as_float` - - :meth:`_sqltypes.JSON.Comparator.as_integer` - - .. change:: - :tags: usecase, oracle - :tickets: 4799 - - Added dialect-level flag ``encoding_errors`` to the cx_Oracle dialect, - which can be specified as part of :func:`_sa.create_engine`. This is passed - to SQLAlchemy's unicode decoding converter under Python 2, and to - cx_Oracle's ``cursor.var()`` object as the ``encodingErrors`` parameter - under Python 3, for the very unusual case that broken encodings are present - in the target database which cannot be fetched unless error handling is - relaxed. The value is ultimately one of the Python "encoding errors" - parameters passed to ``decode()``. - - .. change:: - :tags: usecase, sql - :tickets: 4933 - - The :func:`_expression.text` construct now supports "unique" bound parameters, which - will dynamically uniquify themselves on compilation thus allowing multiple - :func:`_expression.text` constructs with the same bound parameter names to be combined - together. - - - .. change:: - :tags: bug, oracle - :tickets: 4913 - - The :class:`_types.NCHAR` datatype will now bind to the - ``cx_Oracle.FIXED_NCHAR`` DBAPI data bindings when used in a bound - parameter, which supplies proper comparison behavior against a - variable-length string. Previously, the :class:`_types.NCHAR` datatype - would bind to ``cx_oracle.NCHAR`` which is not fixed length; the - :class:`_types.CHAR` datatype already binds to ``cx_Oracle.FIXED_CHAR`` - so it is now consistent that :class:`_types.NCHAR` binds to - ``cx_Oracle.FIXED_NCHAR``. - - - - .. change:: - :tags: bug, firebird - :tickets: 4903 - - Added additional "disconnect" message "Error writing data to the - connection" to Firebird disconnection detection. Pull request courtesy - lukens. - -.. changelog:: - :version: 1.3.10 - :released: October 9, 2019 - - .. change:: - :tags: bug, mssql - :tickets: 4857 - - Fixed bug in SQL Server dialect with new "max_identifier_length" feature - where the mssql dialect already featured this flag, and the implementation - did not accommodate for the new initialization hook correctly. - - - .. change:: - :tags: bug, oracle - :tickets: 4898, 4857 - - Fixed regression in Oracle dialect that was inadvertently using max - identifier length of 128 characters on Oracle server 12.2 and greater even - though the stated contract for the remainder of the 1.3 series is that - this value stays at 30 until version SQLAlchemy 1.4. Also repaired issues - with the retrieval of the "compatibility" version, and removed the warning - emitted when the "v$parameter" view was not accessible as this was causing - user confusion. - -.. changelog:: - :version: 1.3.9 - :released: October 4, 2019 - - .. change:: - :tags: usecase, engine - :tickets: 4857 - - Added new :func:`_sa.create_engine` parameter - :paramref:`_sa.create_engine.max_identifier_length`. This overrides the - dialect-coded "max identifier length" in order to accommodate for databases - that have recently changed this length and the SQLAlchemy dialect has - not yet been adjusted to detect for that version. This parameter interacts - with the existing :paramref:`_sa.create_engine.label_length` parameter in that - it establishes the maximum (and default) value for anonymously generated - labels. Additionally, post-connection detection of max identifier lengths - has been added to the dialect system. This feature is first being used - by the Oracle dialect. - - .. seealso:: - - :ref:`oracle_max_identifier_lengths` - in the Oracle dialect documentation - - .. change:: - :tags: usecase, oracle - :tickets: 4857 - - The Oracle dialect now emits a warning if Oracle version 12.2 or greater is - used, and the :paramref:`_sa.create_engine.max_identifier_length` parameter is - not set. The version in this specific case defaults to that of the - "compatibility" version set in the Oracle server configuration, not the - actual server version. In version 1.4, the default max_identifier_length - for 12.2 or greater will move to 128 characters. In order to maintain - forwards compatibility, applications should set - :paramref:`_sa.create_engine.max_identifier_length` to 30 in order to maintain - the same length behavior, or to 128 in order to test the upcoming behavior. - This length determines among other things how generated constraint names - are truncated for statements like ``CREATE CONSTRAINT`` and ``DROP - CONSTRAINT``, which means a the new length may produce a name-mismatch - against a name that was generated with the old length, impacting database - migrations. - - .. seealso:: - - :ref:`oracle_max_identifier_lengths` - in the Oracle dialect documentation - - .. change:: - :tags: usecase, sqlite - :tickets: 4863 - - Added support for sqlite "URI" connections, which allow for sqlite-specific - flags to be passed in the query string such as "read only" for Python - sqlite3 drivers that support this. - - .. seealso:: - - :ref:`pysqlite_uri_connections` - - .. change:: - :tags: bug, tests - :tickets: 4285 - - Fixed unit test regression released in 1.3.8 that would cause failure for - Oracle, SQL Server and other non-native ENUM platforms due to new - enumeration tests added as part of :ticket:`4285` enum sortability in the - unit of work; the enumerations created constraints that were duplicated on - name. - - .. change:: - :tags: bug, oracle - :tickets: 4886 - - Restored adding cx_Oracle.DATETIME to the setinputsizes() call when a - SQLAlchemy :class:`.Date`, :class:`.DateTime` or :class:`.Time` datatype is - used, as some complex queries require this to be present. This was removed - in the 1.2 series for arbitrary reasons. - - .. change:: - :tags: bug, mssql - :tickets: 4883 - - Added identifier quoting to the schema name applied to the "use" statement - which is invoked when a SQL Server multipart schema name is used within a - :class:`_schema.Table` that is being reflected, as well as for :class:`_reflection.Inspector` - methods such as :meth:`_reflection.Inspector.get_table_names`; this accommodates for - special characters or spaces in the database name. Additionally, the "use" - statement is not emitted if the current database matches the target owner - database name being passed. - - .. change:: - :tags: bug, orm - :tickets: 4872 - - Fixed regression in selectinload loader strategy caused by :ticket:`4775` - (released in version 1.3.6) where a many-to-one attribute of None would no - longer be populated by the loader. While this was usually not noticeable - due to the lazyloader populating None upon get, it would lead to a detached - instance error if the object were detached. - - .. change:: - :tags: bug, orm - :tickets: 4873 - - Passing a plain string expression to :meth:`.Session.query` is deprecated, - as all string coercions were removed in :ticket:`4481` and this one should - have been included. The :func:`_expression.literal_column` function may be used to - produce a textual column expression. - - .. change:: - :tags: usecase, sql - :tickets: 4847 - - Added an explicit error message for the case when objects passed to - :class:`_schema.Table` are not :class:`.SchemaItem` objects, rather than resolving - to an attribute error. - - - .. change:: - :tags: bug, orm - :tickets: 4890 - - A warning is emitted for a condition in which the :class:`.Session` may - implicitly swap an object out of the identity map for another one with the - same primary key, detaching the old one, which can be an observed result of - load operations which occur within the :meth:`.SessionEvents.after_flush` - hook. The warning is intended to notify the user that some special - condition has caused this to happen and that the previous object may not be - in the expected state. - - .. change:: - :tags: bug, sql - :tickets: 4837 - - Characters that interfere with "pyformat" or "named" formats in bound - parameters, namely ``%, (, )`` and the space character, as well as a few - other typically undesirable characters, are stripped early for a - :func:`.bindparam` that is using an anonymized name, which is typically - generated automatically from a named column which itself includes these - characters in its name and does not use a ``.key``, so that they do not - interfere either with the SQLAlchemy compiler's use of string formatting or - with the driver-level parsing of the parameter, both of which could be - demonstrated before the fix. The change only applies to anonymized - parameter names that are generated and consumed internally, not end-user - defined names, so the change should have no impact on any existing code. - Applies in particular to the psycopg2 driver which does not otherwise quote - special parameter names, but also strips leading underscores to suit Oracle - (but not yet leading numbers, as some anon parameters are currently - entirely numeric/underscore based); Oracle in any case continues to quote - parameter names that include special characters. - -.. changelog:: - :version: 1.3.8 - :released: August 27, 2019 - - .. change:: - :tags: bug, orm - :tickets: 4823 - - Fixed bug where :class:`_orm.Load` objects were not pickleable due to - mapper/relationship state in the internal context dictionary. These - objects are now converted to picklable using similar techniques as that of - other elements within the loader option system that have long been - serializable. - - .. change:: - :tags: bug, postgresql - :tickets: 4623 - - Revised the approach for the just added support for the psycopg2 - "execute_values()" feature added in 1.3.7 for :ticket:`4623`. The approach - relied upon a regular expression that would fail to match for a more - complex INSERT statement such as one which had subqueries involved. The - new approach matches exactly the string that was rendered as the VALUES - clause. - - .. change:: - :tags: usecase, orm - :tickets: 4285 - - Added support for the use of an :class:`.Enum` datatype using Python - pep-435 enumeration objects as values for use as a primary key column - mapped by the ORM. As these values are not inherently sortable, as - required by the ORM for primary keys, a new - :attr:`.TypeEngine.sort_key_function` attribute is added to the typing - system which allows any SQL type to implement a sorting for Python objects - of its type which is consulted by the unit of work. The :class:`.Enum` - type then defines this using the database value of a given enumeration. - The sorting scheme can be also be redefined by passing a callable to the - :paramref:`.Enum.sort_key_function` parameter. Pull request courtesy - Nicolas Caniart. - - .. change:: - :tags: bug, engine - :tickets: 4807 - - Fixed an issue whereby if the dialect "initialize" process which occurs on - first connect would encounter an unexpected exception, the initialize - process would fail to complete and then no longer attempt on subsequent - connection attempts, leaving the dialect in an un-initialized, or partially - initialized state, within the scope of parameters that need to be - established based on inspection of a live connection. The "invoke once" - logic in the event system has been reworked to accommodate for this - occurrence using new, private API features that establish an "exec once" - hook that will continue to allow the initializer to fire off on subsequent - connections, until it completes without raising an exception. This does not - impact the behavior of the existing ``once=True`` flag within the event - system. - - .. change:: - :tags: bug, sqlite, reflection - :tickets: 4810 - - Fixed bug where a FOREIGN KEY that was set up to refer to the parent table - by table name only without the column names would not correctly be - reflected as far as setting up the "referred columns", since SQLite's - PRAGMA does not report on these columns if they weren't given explicitly. - For some reason this was hardcoded to assume the name of the local column, - which might work for some cases but is not correct. The new approach - reflects the primary key of the referred table and uses the constraint - columns list as the referred columns list, if the remote column(s) aren't - present in the reflected pragma directly. - - - .. change:: - :tags: bug, postgresql - :tickets: 4822 - - Fixed bug where Postgresql operators such as - :meth:`.postgresql.ARRAY.Comparator.contains` and - :meth:`.postgresql.ARRAY.Comparator.contained_by` would fail to function - correctly for non-integer values when used against a - :class:`_postgresql.array` object, due to an erroneous assert statement. - - .. change:: - :tags: feature, engine - :tickets: 4815 - - Added new parameter :paramref:`_sa.create_engine.hide_parameters` which when - set to True will cause SQL parameters to no longer be logged, nor rendered - in the string representation of a :class:`.StatementError` object. - - - .. change:: - :tags: usecase, postgresql - :tickets: 4824 - - Added support for reflection of CHECK constraints that include the special - PostgreSQL qualifier "NOT VALID", which can be present for CHECK - constraints that were added to an existing table with the directive that - they not be applied to existing data in the table. The PostgreSQL - dictionary for CHECK constraints as returned by - :meth:`_reflection.Inspector.get_check_constraints` may include an additional entry - ``dialect_options`` which within will contain an entry ``"not_valid": - True`` if this symbol is detected. Pull request courtesy Bill Finn. - -.. changelog:: - :version: 1.3.7 - :released: August 14, 2019 - - .. change:: - :tags: bug, sql - :tickets: 4778 - - Fixed issue where :class:`.Index` object which contained a mixture of - functional expressions which were not resolvable to a particular column, - in combination with string-based column names, would fail to initialize - its internal state correctly leading to failures during DDL compilation. - - .. change:: - :tags: bug, sqlite - :tickets: 4798 - - The dialects that support json are supposed to take arguments - ``json_serializer`` and ``json_deserializer`` at the create_engine() level, - however the SQLite dialect calls them ``_json_serializer`` and - ``_json_deserilalizer``. The names have been corrected, the old names are - accepted with a change warning, and these parameters are now documented as - :paramref:`_sa.create_engine.json_serializer` and - :paramref:`_sa.create_engine.json_deserializer`. - - - .. change:: - :tags: bug, mysql - :tickets: 4804 - - The MySQL dialects will emit "SET NAMES" at the start of a connection when - charset is given to the MySQL driver, to appease an apparent behavior - observed in MySQL 8.0 that raises a collation error when a UNION includes - string columns unioned against columns of the form CAST(NULL AS CHAR(..)), - which is what SQLAlchemy's polymorphic_union function does. The issue - seems to have affected PyMySQL for at least a year, however has recently - appeared as of mysqlclient 1.4.4 based on changes in how this DBAPI creates - a connection. As the presence of this directive impacts three separate - MySQL charset settings which each have intricate effects based on their - presence, SQLAlchemy will now emit the directive on new connections to - ensure correct behavior. - - .. change:: - :tags: usecase, postgresql - :tickets: 4623 - - Added new dialect flag for the psycopg2 dialect, ``executemany_mode`` which - supersedes the previous experimental ``use_batch_mode`` flag. - ``executemany_mode`` supports both the "execute batch" and "execute values" - functions provided by psycopg2, the latter which is used for compiled - :func:`_expression.insert` constructs. Pull request courtesy Yuval Dinari. - - .. seealso:: - - :ref:`psycopg2_executemany_mode` - - - - - .. change:: - :tags: bug, sql - :tickets: 4787 - - Fixed bug where :meth:`.TypeEngine.column_expression` method would not be - applied to subsequent SELECT statements inside of a UNION or other - :class:`_selectable.CompoundSelect`, even though the SELECT statements are rendered at - the topmost level of the statement. New logic now differentiates between - rendering the column expression, which is needed for all SELECTs in the - list, vs. gathering the returned data type for the result row, which is - needed only for the first SELECT. - - .. change:: - :tags: bug, sqlite - :tickets: 4793 - - Fixed bug where usage of "PRAGMA table_info" in SQLite dialect meant that - reflection features to detect for table existence, list of table columns, - and list of foreign keys, would default to any table in any attached - database, when no schema name was given and the table did not exist in the - base schema. The fix explicitly runs PRAGMA for the 'main' schema and then - the 'temp' schema if the 'main' returned no rows, to maintain the behavior - of tables + temp tables in the "no schema" namespace, attached tables only - in the "schema" namespace. - - - .. change:: - :tags: bug, sql - :tickets: 4780 - - Fixed issue where internal cloning of SELECT constructs could lead to a key - error if the copy of the SELECT changed its state such that its list of - columns changed. This was observed to be occurring in some ORM scenarios - which may be unique to 1.3 and above, so is partially a regression fix. - - - - .. change:: - :tags: bug, orm - :tickets: 4777 - - Fixed regression caused by new selectinload for many-to-one logic where - a primaryjoin condition not based on real foreign keys would cause - KeyError if a related object did not exist for a given key value on the - parent object. - - .. change:: - :tags: usecase, mysql - :tickets: 4783 - - Added reserved words ARRAY and MEMBER to the MySQL reserved words list, as - MySQL 8.0 has now made these reserved. - - - .. change:: - :tags: bug, events - :tickets: 4794 - - Fixed issue in event system where using the ``once=True`` flag with - dynamically generated listener functions would cause event registration of - future events to fail if those listener functions were garbage collected - after they were used, due to an assumption that a listened function is - strongly referenced. The "once" wrapped is now modified to strongly - reference the inner function persistently, and documentation is updated - that using "once" does not imply automatic de-registration of listener - functions. - - .. change:: - :tags: bug, mysql - :tickets: 4751 - - Added another fix for an upstream MySQL 8 issue where a case sensitive - table name is reported incorrectly in foreign key constraint reflection, - this is an extension of the fix first added for :ticket:`4344` which - affects a case sensitive column name. The new issue occurs through MySQL - 8.0.17, so the general logic of the 88718 fix remains in place. - - .. seealso:: - - https://bugs.mysql.com/bug.php?id=96365 - upstream bug - - - .. change:: - :tags: usecase, mssql - :tickets: 4782 - - Added new :func:`_mssql.try_cast` construct for SQL Server which emits - "TRY_CAST" syntax. Pull request courtesy Leonel Atencio. - - .. change:: - :tags: bug, orm - :tickets: 4803 - - Fixed bug where using :meth:`_query.Query.first` or a slice expression in - conjunction with a query that has an expression based "offset" applied - would raise TypeError, due to an "or" conditional against "offset" that did - not expect it to be a SQL expression as opposed to an integer or None. - - -.. changelog:: - :version: 1.3.6 - :released: July 21, 2019 - - .. change:: - :tags: bug, engine - :tickets: 4754 - - Fixed bug where using reflection function such as :meth:`_schema.MetaData.reflect` - with an :class:`_engine.Engine` object that had execution options applied to it - would fail, as the resulting :class:`.OptionEngine` proxy object failed to - include a ``.engine`` attribute used within the reflection routines. - - .. change:: - :tags: bug, mysql - :tickets: 4743 - - Fixed bug where the special logic to render "NULL" for the - :class:`_types.TIMESTAMP` datatype when ``nullable=True`` would not work if the - column's datatype were a :class:`.TypeDecorator` or a :class:`.Variant`. - The logic now ensures that it unwraps down to the original - :class:`_types.TIMESTAMP` so that this special case NULL keyword is correctly - rendered when requested. - - .. change:: - :tags: performance, orm - :tickets: 4775 - - The optimization applied to selectin loading in :ticket:`4340` where a JOIN - is not needed to eagerly load related items is now applied to many-to-one - relationships as well, so that only the related table is queried for a - simple join condition. In this case, the related items are queried - based on the value of a foreign key column on the parent; if these columns - are deferred or otherwise not loaded on any of the parent objects in - the collection, the loader falls back to the JOIN method. - - - .. change:: - :tags: bug, orm - :tickets: 4773 - - Fixed regression caused by :ticket:`4365` where a join from an entity to - itself without using aliases no longer raises an informative error message, - instead failing on an assertion. The informative error condition has been - restored. - - - .. change:: - :tags: orm, feature - :tickets: 4736 - - Added new loader option method :meth:`_orm.Load.options` which allows loader - options to be constructed hierarchically, so that many sub-options can be - applied to a particular path without needing to call :func:`.defaultload` - many times. Thanks to Alessio Bogon for the idea. - - - .. change:: - :tags: usecase, postgresql - :tickets: 4771 - - Added support for reflection of indexes on PostgreSQL partitioned tables, - which was added to PostgreSQL as of version 11. - - .. change:: - :tags: bug, mysql - :tickets: 4624 - - Enhanced MySQL/MariaDB version string parsing to accommodate for exotic - MariaDB version strings where the "MariaDB" word is embedded among other - alphanumeric characters such as "MariaDBV1". This detection is critical in - order to correctly accommodate for API features that have split between MySQL - and MariaDB such as the "transaction_isolation" system variable. - - - .. change:: - :tags: bug, mssql - :tickets: 4745 - - Ensured that the queries used to reflect indexes and view definitions will - explicitly CAST string parameters into NVARCHAR, as many SQL Server drivers - frequently treat string values, particularly those with non-ascii - characters or larger string values, as TEXT which often don't compare - correctly against VARCHAR characters in SQL Server's information schema - tables for some reason. These CAST operations already take place for - reflection queries against SQL Server ``information_schema.`` tables but - were missing from three additional queries that are against ``sys.`` - tables. - - .. change:: - :tags: bug, orm - :tickets: 4713 - - Fixed an issue where the :meth:`.orm._ORMJoin.join` method, which is a - not-internally-used ORM-level method that exposes what is normally an - internal process of :meth:`_query.Query.join`, did not propagate the ``full`` and - ``outerjoin`` keyword arguments correctly. Pull request courtesy Denis - Kataev. - - .. change:: - :tags: bug, sql - :tickets: 4758 - - Adjusted the initialization for :class:`.Enum` to minimize how often it - invokes the ``.__members__`` attribute of a given PEP-435 enumeration - object, to suit the case where this attribute is expensive to invoke, as is - the case for some popular third party enumeration libraries. - - - .. change:: - :tags: bug, orm - :tickets: 4772 - - Fixed bug where a many-to-one relationship that specified ``uselist=True`` - would fail to update correctly during a primary key change where a related - column needs to change. - - - .. change:: - :tags: bug, orm - :tickets: 4772 - - Fixed bug where the detection for many-to-one or one-to-one use with a - "dynamic" relationship, which is an invalid configuration, would fail to - raise if the relationship were configured with ``uselist=True``. The - current fix is that it warns, instead of raises, as this would otherwise be - backwards incompatible, however in a future release it will be a raise. - - - .. change:: - :tags: bug, orm - :tickets: 4767 - - Fixed bug where a synonym created against a mapped attribute that does not - exist yet, as is the case when it refers to backref before mappers are - configured, would raise recursion errors when trying to test for attributes - on it which ultimately don't exist (as occurs when the classes are run - through Sphinx autodoc), as the unconfigured state of the synonym would put - it into an attribute not found loop. - - - .. change:: - :tags: usecase, postgresql - :tickets: 4756 - - Added support for multidimensional Postgresql array literals via nesting - the :class:`_postgresql.array` object within another one. The - multidimensional array type is detected automatically. - - .. seealso:: - - :class:`_postgresql.array` - - .. change:: - :tags: bug, sql, postgresql - :tickets: 4760 - - Fixed issue where the :class:`_functions.array_agg` construct in combination with - :meth:`.FunctionElement.filter` would not produce the correct operator - precedence in combination with the array index operator. - - - .. change:: - :tags: bug, sql - :tickets: 4747 - - Fixed an unlikely issue where the "corresponding column" routine for unions - and other :class:`_selectable.CompoundSelect` objects could return the wrong column in - some overlapping column situations, thus potentially impacting some ORM - operations when set operations are in use, if the underlying - :func:`_expression.select` constructs were used previously in other similar kinds of - routines, due to a cached value not being cleared. - - .. change:: - :tags: usecase, sqlite - :tickets: 4766 - - Added support for composite (tuple) IN operators with SQLite, by rendering - the VALUES keyword for this backend. As other backends such as DB2 are - known to use the same syntax, the syntax is enabled in the base compiler - using a dialect-level flag ``tuple_in_values``. The change also includes - support for "empty IN tuple" expressions for SQLite when using "in_()" - between a tuple value and an empty set. - - -.. changelog:: - :version: 1.3.5 - :released: June 17, 2019 - - .. change:: - :tags: bug, mysql - :tickets: 4715 - - Fixed bug where MySQL ON DUPLICATE KEY UPDATE would not accommodate setting - a column to the value NULL. Pull request courtesy Lukáš Banič. - - .. change:: - :tags: bug, orm - :tickets: 4723 - - Fixed a series of related bugs regarding joined table inheritance more than - two levels deep, in conjunction with modification to primary key values, - where those primary key columns are also linked together in a foreign key - relationship as is typical for joined table inheritance. The intermediary - table in a three-level inheritance hierarchy will now get its UPDATE if - only the primary key value has changed and passive_updates=False (e.g. - foreign key constraints not being enforced), whereas before it would be - skipped; similarly, with passive_updates=True (e.g. ON UPDATE CASCADE in - effect), the third-level table will not receive an UPDATE statement as was - the case earlier which would fail since CASCADE already modified it. In a - related issue, a relationship linked to a three-level inheritance hierarchy - on the primary key of an intermediary table of a joined-inheritance - hierarchy will also correctly have its foreign key column updated when the - parent object's primary key is modified, even if that parent object is a - subclass of the linked parent class, whereas before these classes would - not be counted. - - .. change:: - :tags: bug, orm - :tickets: 4729 - - Fixed bug where the :attr:`_orm.Mapper.all_orm_descriptors` accessor would - return an entry for the :class:`_orm.Mapper` itself under the declarative - ``__mapper__`` key, when this is not a descriptor. The ``.is_attribute`` - flag that's present on all :class:`.InspectionAttr` objects is now - consulted, which has also been modified to be ``True`` for an association - proxy, as it was erroneously set to False for this object. - - .. change:: - :tags: bug, orm - :tickets: 4704 - - Fixed regression in :meth:`_query.Query.join` where the ``aliased=True`` flag - would not properly apply clause adaptation to filter criteria, if a - previous join were made to the same entity. This is because the adapters - were placed in the wrong order. The order has been reversed so that the - adapter for the most recent ``aliased=True`` call takes precedence as was - the case in 1.2 and earlier. This broke the "elementtree" examples among - other things. - - .. change:: - :tags: bug, orm, py3k - :tickets: 4674 - - Replaced the Python compatibility routines for ``getfullargspec()`` with a - fully vendored version from Python 3.3. Originally, Python was emitting - deprecation warnings for this function in Python 3.8 alphas. While this - change was reverted, it was observed that Python 3 implementations for - ``getfullargspec()`` are an order of magnitude slower as of the 3.4 series - where it was rewritten against ``Signature``. While Python plans to - improve upon this situation, SQLAlchemy projects for now are using a simple - replacement to avoid any future issues. - - .. change:: - :tags: bug, orm - :tickets: 4694 - - Reworked the attribute mechanics used by :class:`.AliasedClass` to no - longer rely upon calling ``__getattribute__`` on the MRO of the wrapped - class, and to instead resolve the attribute normally on the wrapped class - using getattr(), and then unwrap/adapt that. This allows a greater range - of attribute styles on the mapped class including special ``__getattr__()`` - schemes; but it also makes the code simpler and more resilient in general. - - .. change:: - :tags: usecase, postgresql - :tickets: 4717 - - Added support for column sorting flags when reflecting indexes for - PostgreSQL, including ASC, DESC, NULLSFIRST, NULLSLAST. Also adds this - facility to the reflection system in general which can be applied to other - dialects in future releases. Pull request courtesy Eli Collins. - - .. change:: - :tags: bug, postgresql - :tickets: 4701 - - Fixed bug where PostgreSQL dialect could not correctly reflect an ENUM - datatype that has no members, returning a list with ``None`` for the - ``get_enums()`` call and raising a TypeError when reflecting a column which - has such a datatype. The inspection now returns an empty list. - - .. change:: - :tags: bug, sql - :tickets: 4730 - - Fixed a series of quoting issues which all stemmed from the concept of the - :func:`_expression.literal_column` construct, which when being "proxied" through a - subquery to be referred towards by a label that matches its text, the label - would not have quoting rules applied to it, even if the string in the - :class:`.Label` were set up as a :class:`.quoted_name` construct. Not - applying quoting to the text of the :class:`.Label` is a bug because this - text is strictly a SQL identifier name and not a SQL expression, and the - string should not have quotes embedded into it already unlike the - :func:`_expression.literal_column` which it may be applied towards. The existing - behavior of a non-labeled :func:`_expression.literal_column` being propagated as is on - the outside of a subquery is maintained in order to help with manual - quoting schemes, although it's not clear if valid SQL can be generated for - such a construct in any case. - -.. changelog:: - :version: 1.3.4 - :released: May 27, 2019 - - .. change:: - :tags: feature, mssql - :tickets: 4657 - - Added support for SQL Server filtered indexes, via the ``mssql_where`` - parameter which works similarly to that of the ``postgresql_where`` index - function in the PostgreSQL dialect. - - .. seealso:: - - :ref:`mssql_index_where` - - .. change:: - :tags: bug, misc - :tickets: 4625 - - Removed errant "sqla_nose.py" symbol from MANIFEST.in which created an - undesirable warning message. - - .. change:: - :tags: bug, sql - :tickets: 4653 - - Fixed that the :class:`.GenericFunction` class was inadvertently - registering itself as one of the named functions. Pull request courtesy - Adrien Berchet. - - .. change:: - :tags: bug, engine, postgresql - :tickets: 4663 - - Moved the "rollback" which occurs during dialect initialization so that it - occurs after additional dialect-specific initialize steps, in particular - those of the psycopg2 dialect which would inadvertently leave transactional - state on the first new connection, which could interfere with some - psycopg2-specific APIs which require that no transaction is started. Pull - request courtesy Matthew Wilkes. - - - .. change:: - :tags: bug, orm - :tickets: 4695 - - Fixed issue where the :paramref:`.AttributeEvents.active_history` flag - would not be set for an event listener that propagated to a subclass via the - :paramref:`.AttributeEvents.propagate` flag. This bug has been present - for the full span of the :class:`.AttributeEvents` system. - - - .. change:: - :tags: bug, orm - :tickets: 4690 - - Fixed regression where new association proxy system was still not proxying - hybrid attributes when they made use of the ``@hybrid_property.expression`` - decorator to return an alternate SQL expression, or when the hybrid - returned an arbitrary :class:`.PropComparator`, at the expression level. - This involved further generalization of the heuristics used to detect the - type of object being proxied at the level of :class:`.QueryableAttribute`, - to better detect if the descriptor ultimately serves mapped classes or - column expressions. - - .. change:: - :tags: bug, orm - :tickets: 4686 - - Applied the mapper "configure mutex" against the declarative class mapping - process, to guard against the race which can occur if mappers are used - while dynamic module import schemes are still in the process of configuring - mappers for related classes. This does not guard against all possible race - conditions, such as if the concurrent import has not yet encountered the - dependent classes as of yet, however it guards against as much as possible - within the SQLAlchemy declarative process. - - .. change:: - :tags: bug, mssql - :tickets: 4680 - - Added error code 20047 to "is_disconnect" for pymssql. Pull request - courtesy Jon Schuff. - - - .. change:: - :tags: bug, postgresql, orm - :tickets: 4661 - - Fixed an issue where the "number of rows matched" warning would emit even if - the dialect reported "supports_sane_multi_rowcount=False", as is the case - for psycogp2 with ``use_batch_mode=True`` and others. - - - .. change:: - :tags: bug, sql - :tickets: 4618 - - Fixed issue where double negation of a boolean column wouldn't reset - the "NOT" operator. - - .. change:: - :tags: mysql, bug - :tickets: 4650 - - Added support for DROP CHECK constraint which is required by MySQL 8.0.16 - to drop a CHECK constraint; MariaDB supports plain DROP CONSTRAINT. The - logic distinguishes between the two syntaxes by checking the server version - string for MariaDB presence. Alembic migrations has already worked - around this issue by implementing its own DROP for MySQL / MariaDB CHECK - constraints, however this change implements it straight in Core so that its - available for general use. Pull request courtesy Hannes Hansen. - - .. change:: - :tags: bug, orm - :tickets: 4647 - - A warning is now emitted for the case where a transient object is being - merged into the session with :meth:`.Session.merge` when that object is - already transient in the :class:`.Session`. This warns for the case where - the object would normally be double-inserted. - - - .. change:: - :tags: bug, orm - :tickets: 4676 - - Fixed regression in new relationship m2o comparison logic first introduced - at :ref:`change_4359` when comparing to an attribute that is persisted as - NULL and is in an un-fetched state in the mapped instance. Since the - attribute has no explicit default, it needs to default to NULL when - accessed in a persistent setting. - - - .. change:: - :tags: bug, sql - :tickets: 4569 - - The :class:`.GenericFunction` namespace is being migrated so that function - names are looked up in a case-insensitive manner, as SQL functions do not - collide on case sensitive differences nor is this something which would - occur with user-defined functions or stored procedures. Lookups for - functions declared with :class:`.GenericFunction` now use a case - insensitive scheme, however a deprecation case is supported which allows - two or more :class:`.GenericFunction` objects with the same name of - different cases to exist, which will cause case sensitive lookups to occur - for that particular name, while emitting a warning at function registration - time. Thanks to Adrien Berchet for a lot of work on this complicated - feature. - - -.. changelog:: - :version: 1.3.3 - :released: April 15, 2019 - - .. change:: - :tags: bug, postgresql - :tickets: 4601 - - Fixed regression from release 1.3.2 caused by :ticket:`4562` where a URL - that contained only a query string and no hostname, such as for the - purposes of specifying a service file with connection information, would no - longer be propagated to psycopg2 properly. The change in :ticket:`4562` - has been adjusted to further suit psycopg2's exact requirements, which is - that if there are any connection parameters whatsoever, the "dsn" parameter - is no longer required, so in this case the query string parameters are - passed alone. - - .. change:: - :tags: bug, pool - :tickets: 4585 - - Fixed behavioral regression as a result of deprecating the "use_threadlocal" - flag for :class:`_pool.Pool`, where the :class:`.SingletonThreadPool` no longer - makes use of this option which causes the "rollback on return" logic to take - place when the same :class:`_engine.Engine` is used multiple times in the context - of a transaction to connect or implicitly execute, thereby cancelling the - transaction. While this is not the recommended way to work with engines - and connections, it is nonetheless a confusing behavioral change as when - using :class:`.SingletonThreadPool`, the transaction should stay open - regardless of what else is done with the same engine in the same thread. - The ``use_threadlocal`` flag remains deprecated however the - :class:`.SingletonThreadPool` now implements its own version of the same - logic. - - - .. change:: - :tags: bug, orm - :tickets: 4584 - - Fixed 1.3 regression in new "ambiguous FROMs" query logic introduced in - :ref:`change_4365` where a :class:`_query.Query` that explicitly places an entity - in the FROM clause with :meth:`_query.Query.select_from` and also joins to it - using :meth:`_query.Query.join` would later cause an "ambiguous FROM" error if - that entity were used in additional joins, as the entity appears twice in - the "from" list of the :class:`_query.Query`. The fix resolves this ambiguity by - folding the standalone entity into the join that it's already a part of in - the same way that ultimately happens when the SELECT statement is rendered. - - .. change:: - :tags: bug, ext - :tickets: 4603 - - Fixed bug where using ``copy.copy()`` or ``copy.deepcopy()`` on - :class:`.MutableList` would cause the items within the list to be - duplicated, due to an inconsistency in how Python pickle and copy both make - use of ``__getstate__()`` and ``__setstate__()`` regarding lists. In order - to resolve, a ``__reduce_ex__`` method had to be added to - :class:`.MutableList`. In order to maintain backwards compatibility with - existing pickles based on ``__getstate__()``, the ``__setstate__()`` method - remains as well; the test suite asserts that pickles made against the old - version of the class can still be deserialized by the pickle module. - - .. change:: - :tags: bug, orm - :tickets: 4606 - - Adjusted the :meth:`_query.Query.filter_by` method to not call :func:`.and()` - internally against multiple criteria, instead passing it off to - :meth:`_query.Query.filter` as a series of criteria, instead of a single criteria. - This allows :meth:`_query.Query.filter_by` to defer to :meth:`_query.Query.filter`'s - treatment of variable numbers of clauses, including the case where the list - is empty. In this case, the :class:`_query.Query` object will not have a - ``.whereclause``, which allows subsequent "no whereclause" methods like - :meth:`_query.Query.select_from` to behave consistently. - - .. change:: - :tags: bug, mssql - :tickets: 4587 - - Fixed issue in SQL Server dialect where if a bound parameter were present in - an ORDER BY expression that would ultimately not be rendered in the SQL - Server version of the statement, the parameters would still be part of the - execution parameters, leading to DBAPI-level errors. Pull request courtesy - Matt Lewellyn. - -.. changelog:: - :version: 1.3.2 - :released: April 2, 2019 - - .. change:: - :tags: bug, documentation, sql - :tickets: 4580 - - Thanks to :ref:`change_3981`, we no longer need to rely on recipes that - subclass dialect-specific types directly, :class:`.TypeDecorator` can now - handle all cases. Additionally, the above change made it slightly less - likely that a direct subclass of a base SQLAlchemy type would work as - expected, which could be misleading. Documentation has been updated to use - :class:`.TypeDecorator` for these examples including the PostgreSQL - "ArrayOfEnum" example datatype and direct support for the "subclass a type - directly" has been removed. - - .. change:: - :tags: bug, postgresql - :tickets: 4550 - - Modified the :paramref:`.Select.with_for_update.of` parameter so that if a - join or other composed selectable is passed, the individual :class:`_schema.Table` - objects will be filtered from it, allowing one to pass a join() object to - the parameter, as occurs normally when using joined table inheritance with - the ORM. Pull request courtesy Raymond Lu. - - - .. change:: - :tags: feature, postgresql - :tickets: 4562 - - Added support for parameter-less connection URLs for the psycopg2 dialect, - meaning, the URL can be passed to :func:`_sa.create_engine` as - ``"postgresql+psycopg2://"`` with no additional arguments to indicate an - empty DSN passed to libpq, which indicates to connect to "localhost" with - no username, password, or database given. Pull request courtesy Julian - Mehnle. - - .. change:: - :tags: bug, orm, ext - :tickets: 4574, 4573 - - Restored instance-level support for plain Python descriptors, e.g. - ``@property`` objects, in conjunction with association proxies, in that if - the proxied object is not within ORM scope at all, it gets classified as - "ambiguous" but is proxed directly. For class level access, a basic class - level``__get__()`` now returns the - :class:`.AmbiguousAssociationProxyInstance` directly, rather than raising - its exception, which is the closest approximation to the previous behavior - that returned the :class:`.AssociationProxy` itself that's possible. Also - improved the stringification of these objects to be more descriptive of - current state. - - .. change:: - :tags: bug, orm - :tickets: 4537 - - Fixed bug where use of :func:`.with_polymorphic` or other aliased construct - would not properly adapt when the aliased target were used as the - :meth:`_expression.Select.correlate_except` target of a subquery used inside of a - :func:`.column_property`. This required a fix to the clause adaption - mechanics to properly handle a selectable that shows up in the "correlate - except" list, in a similar manner as which occurs for selectables that show - up in the "correlate" list. This is ultimately a fairly fundamental bug - that has lasted for a long time but it is hard to come across it. - - - .. change:: - :tags: bug, orm - :tickets: 4566 - - Fixed regression where a new error message that was supposed to raise when - attempting to link a relationship option to an AliasedClass without using - :meth:`.PropComparator.of_type` would instead raise an ``AttributeError``. - Note that in 1.3, it is no longer valid to create an option path from a - plain mapper relationship to an :class:`.AliasedClass` without using - :meth:`.PropComparator.of_type`. - -.. changelog:: - :version: 1.3.1 - :released: March 9, 2019 - - .. change:: - :tags: bug, mssql - :tickets: 4525 - - Fixed regression in SQL Server reflection due to :ticket:`4393` where the - removal of open-ended ``**kw`` from the :class:`.Float` datatype caused - reflection of this type to fail due to a "scale" argument being passed. - - .. change:: - :tags: bug, orm, ext - :tickets: 4522 - - Fixed regression where an association proxy linked to a synonym would no - longer work, both at instance level and at class level. - -.. changelog:: - :version: 1.3.0 - :released: March 4, 2019 - - .. change:: - :tags: feature, schema - :tickets: 4517 - - Added new parameters :paramref:`_schema.Table.resolve_fks` and - :paramref:`.MetaData.reflect.resolve_fks` which when set to False will - disable the automatic reflection of related tables encountered in - :class:`_schema.ForeignKey` objects, which can both reduce SQL overhead for omitted - tables as well as avoid tables that can't be reflected for database-specific - reasons. Two :class:`_schema.Table` objects present in the same :class:`_schema.MetaData` - collection can still refer to each other even if the reflection of the two - tables occurred separately. - - - .. change:: - :tags: feature, orm - :tickets: 4316 - - The :meth:`_query.Query.get` method can now accept a dictionary of attribute keys - and values as a means of indicating the primary key value to load; is - particularly useful for composite primary keys. Pull request courtesy - Sanjana S. - - .. change:: - :tags: feature, orm - :tickets: 3133 - - A SQL expression can now be assigned to a primary key attribute for an ORM - flush in the same manner as ordinary attributes as described in - :ref:`flush_embedded_sql_expressions` where the expression will be evaluated - and then returned to the ORM using RETURNING, or in the case of pysqlite, - works using the cursor.lastrowid attribute.Requires either a database that - supports RETURNING (e.g. Postgresql, Oracle, SQL Server) or pysqlite. - - .. change:: - :tags: bug, sql - :tickets: 4509 - - The :class:`_expression.Alias` class and related subclasses :class:`_expression.CTE`, - :class:`_expression.Lateral` and :class:`_expression.TableSample` have been reworked so that it is - not possible for a user to construct the objects directly. These constructs - require that the standalone construction function or selectable-bound method - be used to instantiate new objects. - - - .. change:: - :tags: feature, engine - :tickets: 4500 - - Revised the formatting for :class:`.StatementError` when stringified. Each - error detail is broken up over multiple newlines instead of spaced out on a - single line. Additionally, the SQL representation now stringifies the SQL - statement rather than using ``repr()``, so that newlines are rendered as is. - Pull request courtesy Nate Clark. - - .. seealso:: - - :ref:`change_4500` - -.. changelog:: - :version: 1.3.0b3 - :released: March 4, 2019 - :released: February 8, 2019 - - .. change:: - :tags: bug, ext - :tickets: 2642 - - Implemented a more comprehensive assignment operation (e.g. "bulk replace") - when using association proxy with sets or dictionaries. Fixes the problem - of redundant proxy objects being created to replace the old ones, which - leads to excessive events and SQL and in the case of unique constraints - will cause the flush to fail. - - .. seealso:: - - :ref:`change_2642` - - .. change:: - :tags: bug, postgresql - :tickets: 4473 - - Fixed issue where using an uppercase name for an index type (e.g. GIST, - BTREE, etc. ) or an EXCLUDE constraint would treat it as an identifier to - be quoted, rather than rendering it as is. The new behavior converts these - types to lowercase and ensures they contain only valid SQL characters. - - .. change:: - :tags: bug, orm - :tickets: 4469 - - Improved the behavior of :func:`_orm.with_polymorphic` in conjunction with - loader options, in particular wildcard operations as well as - :func:`_orm.load_only`. The polymorphic object will be more accurately - targeted so that column-level options on the entity will correctly take - effect.The issue is a continuation of the same kinds of things fixed in - :ticket:`4468`. - - - .. change:: - :tags: bug, sql - :tickets: 4481 - - Fully removed the behavior of strings passed directly as components of a - :func:`_expression.select` or :class:`_query.Query` object being coerced to :func:`_expression.text` - constructs automatically; the warning that has been emitted is now an - ArgumentError or in the case of order_by() / group_by() a CompileError. - This has emitted a warning since version 1.0 however its presence continues - to create concerns for the potential of mis-use of this behavior. - - Note that public CVEs have been posted for order_by() / group_by() which - are resolved by this commit: CVE-2019-7164 CVE-2019-7548 - - - .. seealso:: - - :ref:`change_4481` - - .. change:: - :tags: bug, sql - :tickets: 4467 - - Quoting is applied to :class:`.Function` names, those which are usually but - not necessarily generated from the :attr:`_expression.func` construct, at compile - time if they contain illegal characters, such as spaces or punctuation. The - names are as before treated as case insensitive however, meaning if the - names contain uppercase or mixed case characters, that alone does not - trigger quoting. The case insensitivity is currently maintained for - backwards compatibility. - - - .. change:: - :tags: bug, sql - :tickets: 4481 - - Added "SQL phrase validation" to key DDL phrases that are accepted as plain - strings, including :paramref:`_schema.ForeignKeyConstraint.on_delete`, - :paramref:`_schema.ForeignKeyConstraint.on_update`, - :paramref:`.ExcludeConstraint.using`, - :paramref:`_schema.ForeignKeyConstraint.initially`, for areas where a series of SQL - keywords only are expected.Any non-space characters that suggest the phrase - would need to be quoted will raise a :class:`.CompileError`. This change - is related to the series of changes committed as part of :ticket:`4481`. - - .. change:: - :tags: bug, orm, declarative - :tickets: 4470 - - Added some helper exceptions that invoke when a mapping based on - :class:`.AbstractConcreteBase`, :class:`.DeferredReflection`, or - :class:`.AutoMap` is used before the mapping is ready to be used, which - contain descriptive information on the class, rather than falling through - into other failure modes that are less informative. - - - .. change:: - :tags: change, tests - :tickets: 4460 - - The test system has removed support for Nose, which is unmaintained for - several years and is producing warnings under Python 3. The test suite is - currently standardized on Pytest. Pull request courtesy Parth Shandilya. - -.. changelog:: - :version: 1.3.0b2 - :released: March 4, 2019 - :released: January 25, 2019 - - .. change:: - :tags: bug, ext - :tickets: 4401 - - Fixed a regression in 1.3.0b1 caused by :ticket:`3423` where association - proxy objects that access an attribute that's only present on a polymorphic - subclass would raise an ``AttributeError`` even though the actual instance - being accessed was an instance of that subclass. - - .. change:: - :tags: bug, orm - :tickets: 1103 - - Fixed long-standing issue where duplicate collection members would cause a - backref to delete the association between the member and its parent object - when one of the duplicates were removed, as occurs as a side effect of - swapping two objects in one statement. - - .. seealso:: - - :ref:`change_1103` - - .. change:: - :tags: bug, mssql - :tickets: 4442 - - The ``literal_processor`` for the :class:`.Unicode` and - :class:`.UnicodeText` datatypes now render an ``N`` character in front of - the literal string expression as required by SQL Server for Unicode string - values rendered in SQL expressions. - - .. change:: - :tags: feature, orm - :tickets: 4423 - - Implemented a new feature whereby the :class:`.AliasedClass` construct can - now be used as the target of a :func:`_orm.relationship`. This allows the - concept of "non primary mappers" to no longer be necessary, as the - :class:`.AliasedClass` is much easier to configure and automatically inherits - all the relationships of the mapped class, as well as preserves the - ability for loader options to work normally. - - .. seealso:: - - :ref:`change_4423` - - .. change:: - :tags: bug, orm - :tickets: 4373 - - Extended the fix first made as part of :ticket:`3287`, where a loader option - made against a subclass using a wildcard would extend itself to include - application of the wildcard to attributes on the super classes as well, to a - "bound" loader option as well, e.g. in an expression like - ``Load(SomeSubClass).load_only('foo')``. Columns that are part of the - parent class of ``SomeSubClass`` will also be excluded in the same way as if - the unbound option ``load_only('foo')`` were used. - - .. change:: - :tags: bug, orm - :tickets: 4433 - - Improved error messages emitted by the ORM in the area of loader option - traversal. This includes early detection of mis-matched loader strategies - along with a clearer explanation why these strategies don't match. - - - .. change:: - :tags: change, orm - :tickets: 4412 - - Added a new function :func:`.close_all_sessions` which takes - over the task of the :meth:`.Session.close_all` method, which - is now deprecated as this is confusing as a classmethod. - Pull request courtesy Augustin Trancart. - - .. change:: - :tags: feature, orm - :tickets: 4397 - - Added new :meth:`.MapperEvents.before_mapper_configured` event. This - event complements the other "configure" stage mapper events with a per - mapper event that receives each :class:`_orm.Mapper` right before its - configure step, and additionally may be used to prevent or delay the - configuration of specific :class:`_orm.Mapper` objects using a new - return value :attr:`.orm.interfaces.EXT_SKIP`. See the - documentation link for an example. - - .. seealso:: - - :meth:`.MapperEvents.before_mapper_configured` - - - - .. change:: - :tags: bug, orm - - The "remove" event for collections is now called before the item is removed - in the case of the ``collection.remove()`` method, as is consistent with the - behavior for most other forms of collection item removal (such as - ``__delitem__``, replacement under ``__setitem__``). For ``pop()`` methods, - the remove event still fires after the operation. - - .. change:: - :tags: bug, orm declarative - :tickets: 4372 - - Added a ``__clause_element__()`` method to :class:`.ColumnProperty` which - can allow the usage of a not-fully-declared column or deferred attribute in - a declarative mapped class slightly more friendly when it's used in a - constraint or other column-oriented scenario within the class declaration, - though this still can't work in open-ended expressions; prefer to call the - :attr:`.ColumnProperty.expression` attribute if receiving ``TypeError``. - - .. change:: - :tags: bug, orm, engine - :tickets: 4464 - - Added accessors for execution options to Core and ORM, via - :meth:`_query.Query.get_execution_options`, - :meth:`_engine.Connection.get_execution_options`, - :meth:`_engine.Engine.get_execution_options`, and - :meth:`.Executable.get_execution_options`. PR courtesy Daniel Lister. - - .. change:: - :tags: bug, orm - :tickets: 4446 - - Fixed issue in association proxy due to :ticket:`3423` which caused the use - of custom :class:`.PropComparator` objects with hybrid attributes, such as - the one demonstrated in the ``dictlike-polymorphic`` example to not - function within an association proxy. The strictness that was added in - :ticket:`3423` has been relaxed, and additional logic to accommodate for - an association proxy that links to a custom hybrid have been added. - - .. change:: - :tags: change, general - :tickets: 4393 - - A large change throughout the library has ensured that all objects, - parameters, and behaviors which have been noted as deprecated or legacy now - emit ``DeprecationWarning`` warnings when invoked.As the Python 3 - interpreter now defaults to displaying deprecation warnings, as well as that - modern test suites based on tools like tox and pytest tend to display - deprecation warnings, this change should make it easier to note what API - features are obsolete. A major rationale for this change is so that long- - deprecated features that nonetheless still see continue to see real world - use can finally be removed in the near future; the biggest example of this - are the :class:`.SessionExtension` and :class:`.MapperExtension` classes as - well as a handful of other pre-event extension hooks, which have been - deprecated since version 0.7 but still remain in the library. Another is - that several major longstanding behaviors are to be deprecated as well, - including the threadlocal engine strategy, the convert_unicode flag, and non - primary mappers. - - .. seealso:: - - :ref:`change_4393_general` - - - .. change:: - :tags: change, engine - :tickets: 4393 - - The "threadlocal" engine strategy which has been a legacy feature of - SQLAlchemy since around version 0.2 is now deprecated, along with the - :paramref:`_pool.Pool.threadlocal` parameter of :class:`_pool.Pool` which has no - effect in most modern use cases. - - .. seealso:: - - :ref:`change_4393_threadlocal` - - .. change:: - :tags: change, sql - :tickets: 4393 - - The :paramref:`_sa.create_engine.convert_unicode` and - :paramref:`.String.convert_unicode` parameters have been deprecated. These - parameters were built back when most Python DBAPIs had little to no support - for Python Unicode objects, and SQLAlchemy needed to take on the very - complex task of marshalling data and SQL strings between Unicode and - bytestrings throughout the system in a performant way. Thanks to Python 3, - DBAPIs were compelled to adapt to Unicode-aware APIs and today all DBAPIs - supported by SQLAlchemy support Unicode natively, including on Python 2, - allowing this long-lived and very complicated feature to finally be (mostly) - removed. There are still of course a few Python 2 edge cases where - SQLAlchemy has to deal with Unicode however these are handled automatically; - in modern use, there should be no need for end-user interaction with these - flags. - - .. seealso:: - - :ref:`change_4393_convertunicode` - - .. change:: - :tags: bug, orm - :tickets: 3777 - - Implemented the ``.get_history()`` method, which also implies availability - of :attr:`.AttributeState.history`, for :func:`.synonym` attributes. - Previously, trying to access attribute history via a synonym would raise an - ``AttributeError``. - - .. change:: - :tags: feature, engine - :tickets: 3689 - - Added public accessor :meth:`.QueuePool.timeout` that returns the configured - timeout for a :class:`.QueuePool` object. Pull request courtesy Irina Delamare. - - .. change:: - :tags: feature, sql - :tickets: 4386 - - Amended the :class:`.AnsiFunction` class, the base of common SQL - functions like ``CURRENT_TIMESTAMP``, to accept positional arguments - like a regular ad-hoc function. This to suit the case that many of - these functions on specific backends accept arguments such as - "fractional seconds" precision and such. If the function is created - with arguments, it renders the parenthesis and the arguments. If - no arguments are present, the compiler generates the non-parenthesized form. - -.. changelog:: - :version: 1.3.0b1 - :released: March 4, 2019 - :released: November 16, 2018 - - .. change:: - :tags: bug, ext - :tickets: 3423 - - Reworked :class:`.AssociationProxy` to store state that's specific to a - parent class in a separate object, so that a single - :class:`.AssociationProxy` can serve for multiple parent classes, as is - intrinsic to inheritance, without any ambiguity in the state returned by it. - A new method :meth:`.AssociationProxy.for_class` is added to allow - inspection of class-specific state. - - .. seealso:: - - :ref:`change_3423` - - - .. change:: - :tags: bug, oracle - :tickets: 4369 - - Updated the parameters that can be sent to the cx_Oracle DBAPI to both allow - for all current parameters as well as for future parameters not added yet. - In addition, removed unused parameters that were deprecated in version 1.2, - and additionally we are now defaulting "threaded" to False. - - .. seealso:: - - :ref:`change_4369` - - .. change:: - :tags: bug, oracle - :tickets: 4242 - - The Oracle dialect will no longer use the NCHAR/NCLOB datatypes - represent generic unicode strings or clob fields in conjunction with - :class:`.Unicode` and :class:`.UnicodeText` unless the flag - ``use_nchar_for_unicode=True`` is passed to :func:`_sa.create_engine` - - this includes CREATE TABLE behavior as well as ``setinputsizes()`` for - bound parameters. On the read side, automatic Unicode conversion under - Python 2 has been added to CHAR/VARCHAR/CLOB result rows, to match the - behavior of cx_Oracle under Python 3. In order to mitigate the performance - hit under Python 2, SQLAlchemy's very performant (when C extensions - are built) native Unicode handlers are used under Python 2. - - .. seealso:: - - :ref:`change_4242` - - .. change:: - :tags: bug, orm - :tickets: 3844 - - Fixed issue regarding passive_deletes="all", where the foreign key - attribute of an object is maintained with its value even after the object - is removed from its parent collection. Previously, the unit of work would - set this to NULL even though passive_deletes indicated it should not be - modified. - - .. seealso:: - - :ref:`change_3844` - - .. change:: - :tags: bug, ext - :tickets: 4268 - - The long-standing behavior of the association proxy collection maintaining - only a weak reference to the parent object is reverted; the proxy will now - maintain a strong reference to the parent for as long as the proxy - collection itself is also in memory, eliminating the "stale association - proxy" error. This change is being made on an experimental basis to see if - any use cases arise where it causes side effects. - - .. seealso:: - - :ref:`change_4268` - - - .. change:: - :tags: bug, sql - :tickets: 4302 - - Added "like" based operators as "comparison" operators, including - :meth:`.ColumnOperators.startswith` :meth:`.ColumnOperators.endswith` - :meth:`.ColumnOperators.ilike` :meth:`.ColumnOperators.notilike` among many - others, so that all of these operators can be the basis for an ORM - "primaryjoin" condition. - - - .. change:: - :tags: feature, sqlite - :tickets: 3850 - - Added support for SQLite's json functionality via the new - SQLite implementation for :class:`_types.JSON`, :class:`_sqlite.JSON`. - The name used for the type is ``JSON``, following an example found at - SQLite's own documentation. Pull request courtesy Ilja Everilä. - - .. seealso:: - - :ref:`change_3850` - - .. change:: - :tags: feature, engine - - Added new "lifo" mode to :class:`.QueuePool`, typically enabled by setting - the flag :paramref:`_sa.create_engine.pool_use_lifo` to True. "lifo" mode - means the same connection just checked in will be the first to be checked - out again, allowing excess connections to be cleaned up from the server - side during periods of the pool being only partially utilized. Pull request - courtesy Taem Park. - - .. seealso:: - - :ref:`change_pr467` - - .. change:: - :tags: bug, orm - :tickets: 4359 - - Improved the behavior of a relationship-bound many-to-one object expression - such that the retrieval of column values on the related object are now - resilient against the object being detached from its parent - :class:`.Session`, even if the attribute has been expired. New features - within the :class:`.InstanceState` are used to memoize the last known value - of a particular column attribute before its expired, so that the expression - can still evaluate when the object is detached and expired at the same - time. Error conditions are also improved using modern attribute state - features to produce more specific messages as needed. - - .. seealso:: - - :ref:`change_4359` - - .. change:: - :tags: feature, mysql - :tickets: 4219 - - Support added for the "WITH PARSER" syntax of CREATE FULLTEXT INDEX - in MySQL, using the ``mysql_with_parser`` keyword argument. Reflection - is also supported, which accommodates MySQL's special comment format - for reporting on this option as well. Additionally, the "FULLTEXT" and - "SPATIAL" index prefixes are now reflected back into the ``mysql_prefix`` - index option. - - - - .. change:: - :tags: bug, orm, mysql, postgresql - :tickets: 4246 - - The ORM now doubles the "FOR UPDATE" clause within the subquery that - renders in conjunction with joined eager loading in some cases, as it has - been observed that MySQL does not lock the rows from a subquery. This - means the query renders with two FOR UPDATE clauses; note that on some - backends such as Oracle, FOR UPDATE clauses on subqueries are silently - ignored since they are unnecessary. Additionally, in the case of the "OF" - clause used primarily with PostgreSQL, the FOR UPDATE is rendered only on - the inner subquery when this is used so that the selectable can be targeted - to the table within the SELECT statement. - - .. seealso:: - - :ref:`change_4246` - - .. change:: - :tags: feature, mssql - :tickets: 4158 - - Added ``fast_executemany=True`` parameter to the SQL Server pyodbc dialect, - which enables use of pyodbc's new performance feature of the same name - when using Microsoft ODBC drivers. - - .. seealso:: - - :ref:`change_4158` - - .. change:: - :tags: bug, ext - :tickets: 4308 - - Fixed multiple issues regarding de-association of scalar objects with the - association proxy. ``del`` now works, and additionally a new flag - :paramref:`.AssociationProxy.cascade_scalar_deletes` is added, which when - set to True indicates that setting a scalar attribute to ``None`` or - deleting via ``del`` will also set the source association to ``None``. - - .. seealso:: - - :ref:`change_4308` - - - .. change:: - :tags: feature, ext - :tickets: 4318 - - Added new feature :meth:`.BakedQuery.to_query`, which allows for a - clean way of using one :class:`.BakedQuery` as a subquery inside of another - :class:`.BakedQuery` without needing to refer explicitly to a - :class:`.Session`. - - - .. change:: - :tags: feature, sqlite - :tickets: 4360 - - Implemented the SQLite ``ON CONFLICT`` clause as understood at the DDL - level, e.g. for primary key, unique, and CHECK constraints as well as - specified on a :class:`_schema.Column` to satisfy inline primary key and NOT NULL. - Pull request courtesy Denis Kataev. - - .. seealso:: - - :ref:`change_4360` - - .. change:: - :tags: feature, postgresql - :tickets: 4237 - - Added rudimental support for reflection of PostgreSQL - partitioned tables, e.g. that relkind='p' is added to reflection - queries that return table information. - - .. seealso:: - - :ref:`change_4237` - - .. change:: - :tags: feature, ext - :tickets: 4351 - - The :class:`.AssociationProxy` now has standard column comparison operations - such as :meth:`.ColumnOperators.like` and - :meth:`.ColumnOperators.startswith` available when the target attribute is a - plain column - the EXISTS expression that joins to the target table is - rendered as usual, but the column expression is then use within the WHERE - criteria of the EXISTS. Note that this alters the behavior of the - ``.contains()`` method on the association proxy to make use of - :meth:`.ColumnOperators.contains` when used on a column-based attribute. - - .. seealso:: - - :ref:`change_4351` - - - .. change:: - :tags: feature, orm - - Added new flag :paramref:`.Session.bulk_save_objects.preserve_order` to the - :meth:`.Session.bulk_save_objects` method, which defaults to True. When set - to False, the given mappings will be grouped into inserts and updates per - each object type, to allow for greater opportunities to batch common - operations together. Pull request courtesy Alessandro Cucci. - - .. change:: - :tags: bug, orm - :tickets: 4365 - - Refactored :meth:`_query.Query.join` to further clarify the individual components - of structuring the join. This refactor adds the ability for - :meth:`_query.Query.join` to determine the most appropriate "left" side of the - join when there is more than one element in the FROM list or the query is - against multiple entities. If more than one FROM/entity matches, an error - is raised that asks for an ON clause to be specified to resolve the - ambiguity. In particular this targets the regression we saw in - :ticket:`4363` but is also of general use. The codepaths within - :meth:`_query.Query.join` are now easier to follow and the error cases are - decided more specifically at an earlier point in the operation. - - .. seealso:: - - :ref:`change_4365` - - .. change:: - :tags: bug, sql - :tickets: 3981 - - Fixed issue with :meth:`.TypeEngine.bind_expression` and - :meth:`.TypeEngine.column_expression` methods where these methods would not - work if the target type were part of a :class:`.Variant`, or other target - type of a :class:`.TypeDecorator`. Additionally, the SQL compiler now - calls upon the dialect-level implementation when it renders these methods - so that dialects can now provide for SQL-level processing for built-in - types. - - .. seealso:: - - :ref:`change_3981` - - - .. change:: - :tags: bug, orm - :tickets: 4304 - - Fixed long-standing issue in :class:`_query.Query` where a scalar subquery such - as produced by :meth:`_query.Query.exists`, :meth:`_query.Query.as_scalar` and other - derivations from :attr:`_query.Query.statement` would not correctly be adapted - when used in a new :class:`_query.Query` that required entity adaptation, such as - when the query were turned into a union, or a from_self(), etc. The change - removes the "no adaptation" annotation from the :func:`_expression.select` object - produced by the :attr:`_query.Query.statement` accessor. - - .. change:: - :tags: bug, orm, declarative - :tickets: 4133 - - Fixed bug where declarative would not update the state of the - :class:`_orm.Mapper` as far as what attributes were present, when additional - attributes were added or removed after the mapper attribute collections had - already been called and memoized. Additionally, a ``NotImplementedError`` - is now raised if a fully mapped attribute (e.g. column, relationship, etc.) - is deleted from a class that is currently mapped, since the mapper will not - function correctly if the attribute has been removed. - - .. change:: - :tags: bug, mssql - :tickets: 4362 - - Deprecated the use of :class:`.Sequence` with SQL Server in order to affect - the "start" and "increment" of the IDENTITY value, in favor of new - parameters ``mssql_identity_start`` and ``mssql_identity_increment`` which - set these parameters directly. :class:`.Sequence` will be used to generate - real ``CREATE SEQUENCE`` DDL with SQL Server in a future release. - - .. seealso:: - - :ref:`change_4362` - - - .. change:: - :tags: feature, mysql - - Added support for the parameters in an ON DUPLICATE KEY UPDATE statement on - MySQL to be ordered, since parameter order in a MySQL UPDATE clause is - significant, in a similar manner as that described at - :ref:`tutorial_parameter_ordered_updates`. Pull request courtesy Maxim Bublis. - - .. seealso:: - - :ref:`change_mysql_ondupordering` - - .. change:: - :tags: feature, sql - :tickets: 4144 - - Added :class:`.Sequence` to the "string SQL" system that will render a - meaningful string expression (``""``) - when stringifying without a dialect a statement that includes a "sequence - nextvalue" expression, rather than raising a compilation error. - - - - .. change:: - :tags: bug, orm - :tickets: 4232 - - An informative exception is re-raised when a primary key value is not - sortable in Python during an ORM flush under Python 3, such as an ``Enum`` - that has no ``__lt__()`` method; normally Python 3 raises a ``TypeError`` - in this case. The flush process sorts persistent objects by primary key - in Python so the values must be sortable. - - - .. change:: - :tags: orm, bug - :tickets: 3604 - - Removed the collection converter used by the :class:`.MappedCollection` - class. This converter was used only to assert that the incoming dictionary - keys matched that of their corresponding objects, and only during a bulk set - operation. The converter can interfere with a custom validator or - :meth:`.AttributeEvents.bulk_replace` listener that wants to convert - incoming values further. The ``TypeError`` which would be raised by this - converter when an incoming key didn't match the value is removed; incoming - values during a bulk assignment will be keyed to their value-generated key, - and not the key that's explicitly present in the dictionary. - - Overall, @converter is superseded by the - :meth:`.AttributeEvents.bulk_replace` event handler added as part of - :ticket:`3896`. - - .. change:: - :tags: feature, sql - :tickets: 3989 - - Added new naming convention tokens ``column_0N_name``, ``column_0_N_name``, - etc., which will render the names / keys / labels for all columns referenced - by a particular constraint in a sequence. In order to accommodate for the - length of such a naming convention, the SQL compiler's auto-truncation - feature now applies itself to constraint names as well, which creates a - shortened, deterministically generated name for the constraint that will - apply to a target backend without going over the character limit of that - backend. - - The change also repairs two other issues. One is that the ``column_0_key`` - token wasn't available even though this token was documented, the other was - that the ``referred_column_0_name`` token would inadvertently render the - ``.key`` and not the ``.name`` of the column if these two values were - different. - - .. seealso:: - - :ref:`change_3989` - - - .. change:: - :tags: feature, ext - :tickets: 4196 - - Added support for bulk :meth:`_query.Query.update` and :meth:`_query.Query.delete` - to the :class:`.ShardedQuery` class within the horizontal sharding - extension. This also adds an additional expansion hook to the - bulk update/delete methods :meth:`_query.Query._execute_crud`. - - .. seealso:: - - :ref:`change_4196` - - .. change:: - :tags: feature, sql - :tickets: 4271 - - Added new logic to the "expanding IN" bound parameter feature whereby if - the given list is empty, a special "empty set" expression that is specific - to different backends is generated, thus allowing IN expressions to be - fully dynamic including empty IN expressions. - - .. seealso:: - - :ref:`change_4271` - - - - .. change:: - :tags: feature, mysql - - The "pre-ping" feature of the connection pool now uses - the ``ping()`` method of the DBAPI connection in the case of - mysqlclient, PyMySQL and mysql-connector-python. Pull request - courtesy Maxim Bublis. - - .. seealso:: - - :ref:`change_mysql_ping` - - .. change:: - :tags: feature, orm - :tickets: 4340 - - The "selectin" loader strategy now omits the JOIN in the case of a simple - one-to-many load, where it instead relies loads only from the related - table, relying upon the foreign key columns of the related table in order - to match up to primary keys in the parent table. This optimization can be - disabled by setting the :paramref:`_orm.relationship.omit_join` flag to False. - Many thanks to Jayson Reis for the efforts on this. - - .. seealso:: - - :ref:`change_4340` - - .. change:: - :tags: bug, orm - :tickets: 4353 - - Added new behavior to the lazy load that takes place when the "old" value of - a many-to-one is retrieved, such that exceptions which would be raised due - to either ``lazy="raise"`` or a detached session error are skipped. - - .. seealso:: - - :ref:`change_4353` - - .. change:: - :tags: feature, sql - - The Python builtin ``dir()`` is now supported for a SQLAlchemy "properties" - object, such as that of a Core columns collection (e.g. ``.c``), - ``mapper.attrs``, etc. Allows iPython autocompletion to work as well. - Pull request courtesy Uwe Korn. - - .. change:: - :tags: feature, orm - :tickets: 4257 - - Added ``.info`` dictionary to the :class:`.InstanceState` class, the object - that comes from calling :func:`_sa.inspect` on a mapped object. - - .. seealso:: - - :ref:`change_4257` - - .. change:: - :tags: feature, sql - :tickets: 3831 - - Added new feature :meth:`.FunctionElement.as_comparison` which allows a SQL - function to act as a binary comparison operation that can work within the - ORM. - - .. seealso:: - - :ref:`change_3831` - - .. change:: - :tags: bug, orm - :tickets: 4354 - - A long-standing oversight in the ORM, the ``__delete__`` method for a many- - to-one relationship was non-functional, e.g. for an operation such as ``del - a.b``. This is now implemented and is equivalent to setting the attribute - to ``None``. - - .. seealso:: - - :ref:`change_4354` diff --git a/doc/build/changelog/index.rst b/doc/build/changelog/index.rst index d6a0d26f65..136f012b7a 100644 --- a/doc/build/changelog/index.rst +++ b/doc/build/changelog/index.rst @@ -28,19 +28,6 @@ Change logs changelog_20 changelog_14 - changelog_13 - changelog_12 - changelog_11 - changelog_10 - changelog_09 - changelog_08 - changelog_07 - changelog_06 - changelog_05 - changelog_04 - changelog_03 - changelog_02 - changelog_01 Older Migration Guides @@ -51,12 +38,3 @@ Older Migration Guides migration_14 migration_13 - migration_12 - migration_11 - migration_10 - migration_09 - migration_08 - migration_07 - migration_06 - migration_05 - migration_04 diff --git a/doc/build/changelog/migration_04.rst b/doc/build/changelog/migration_04.rst deleted file mode 100644 index 2618c77e3a..0000000000 --- a/doc/build/changelog/migration_04.rst +++ /dev/null @@ -1,921 +0,0 @@ -============================= -What's new in SQLAlchemy 0.4? -============================= - -.. admonition:: About this Document - - This document describes changes between SQLAlchemy version 0.3, - last released October 14, 2007, and SQLAlchemy version 0.4, - last released October 12, 2008. - - Document date: March 21, 2008 - -First Things First -================== - -If you're using any ORM features, make sure you import from -``sqlalchemy.orm``: - -:: - - from sqlalchemy import * - from sqlalchemy.orm import * - -Secondly, anywhere you used to say ``engine=``, -``connectable=``, ``bind_to=``, ``something.engine``, -``metadata.connect()``, use ``bind``: - -:: - - myengine = create_engine("sqlite://") - - meta = MetaData(myengine) - - meta2 = MetaData() - meta2.bind = myengine - - session = create_session(bind=myengine) - - statement = select([table], bind=myengine) - -Got those ? Good! You're now (95%) 0.4 compatible. If -you're using 0.3.10, you can make these changes immediately; -they'll work there too. - -Module Imports -============== - -In 0.3, "``from sqlalchemy import *``" would import all of -sqlalchemy's sub-modules into your namespace. Version 0.4 no -longer imports sub-modules into the namespace. This may mean -you need to add extra imports into your code. - -In 0.3, this code worked: - -:: - - from sqlalchemy import * - - - class UTCDateTime(types.TypeDecorator): - pass - -In 0.4, one must do: - -:: - - from sqlalchemy import * - from sqlalchemy import types - - - class UTCDateTime(types.TypeDecorator): - pass - -Object Relational Mapping -========================= - -Querying --------- - -New Query API -^^^^^^^^^^^^^ - -Query is standardized on the generative interface (old -interface is still there, just deprecated). While most of -the generative interface is available in 0.3, the 0.4 Query -has the inner guts to match the generative outside, and has -a lot more tricks. All result narrowing is via ``filter()`` -and ``filter_by()``, limiting/offset is either through array -slices or ``limit()``/``offset()``, joining is via -``join()`` and ``outerjoin()`` (or more manually, through -``select_from()`` as well as manually-formed criteria). - -To avoid deprecation warnings, you must make some changes to -your 03 code - -User.query.get_by( \**kwargs ) - -:: - - User.query.filter_by(**kwargs).first() - -User.query.select_by( \**kwargs ) - -:: - - User.query.filter_by(**kwargs).all() - -User.query.select() - -:: - - User.query.filter(xxx).all() - -New Property-Based Expression Constructs -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -By far the most palpable difference within the ORM is that -you can now construct your query criterion using class-based -attributes directly. The ".c." prefix is no longer needed -when working with mapped classes: - -:: - - session.query(User).filter(and_(User.name == "fred", User.id > 17)) - -While simple column-based comparisons are no big deal, the -class attributes have some new "higher level" constructs -available, including what was previously only available in -``filter_by()``: - -:: - - # comparison of scalar relations to an instance - filter(Address.user == user) - - # return all users who contain a particular address - filter(User.addresses.contains(address)) - - # return all users who *dont* contain the address - filter(~User.address.contains(address)) - - # return all users who contain a particular address with - # the email_address like '%foo%' - filter(User.addresses.any(Address.email_address.like("%foo%"))) - - # same, email address equals 'foo@bar.com'. can fall back to keyword - # args for simple comparisons - filter(User.addresses.any(email_address="foo@bar.com")) - - # return all Addresses whose user attribute has the username 'ed' - filter(Address.user.has(name="ed")) - - # return all Addresses whose user attribute has the username 'ed' - # and an id > 5 (mixing clauses with kwargs) - filter(Address.user.has(User.id > 5, name="ed")) - -The ``Column`` collection remains available on mapped -classes in the ``.c`` attribute. Note that property-based -expressions are only available with mapped properties of -mapped classes. ``.c`` is still used to access columns in -regular tables and selectable objects produced from SQL -Expressions. - -Automatic Join Aliasing -^^^^^^^^^^^^^^^^^^^^^^^ - -We've had join() and outerjoin() for a while now: - -:: - - session.query(Order).join("items") - -Now you can alias them: - -:: - - session.query(Order).join("items", aliased=True).filter(Item.name="item 1").join( - "items", aliased=True - ).filter(Item.name == "item 3") - -The above will create two joins from orders->items using -aliases. the ``filter()`` call subsequent to each will -adjust its table criterion to that of the alias. To get at -the ``Item`` objects, use ``add_entity()`` and target each -join with an ``id``: - -:: - - session.query(Order).join("items", id="j1", aliased=True).filter( - Item.name == "item 1" - ).join("items", aliased=True, id="j2").filter(Item.name == "item 3").add_entity( - Item, id="j1" - ).add_entity( - Item, id="j2" - ) - -Returns tuples in the form: ``(Order, Item, Item)``. - -Self-referential Queries -^^^^^^^^^^^^^^^^^^^^^^^^ - -So query.join() can make aliases now. What does that give -us ? Self-referential queries ! Joins can be done without -any ``Alias`` objects: - -:: - - # standard self-referential TreeNode mapper with backref - mapper( - TreeNode, - tree_nodes, - properties={ - "children": relation( - TreeNode, backref=backref("parent", remote_side=tree_nodes.id) - ) - }, - ) - - # query for node with child containing "bar" two levels deep - session.query(TreeNode).join(["children", "children"], aliased=True).filter_by( - name="bar" - ) - -To add criterion for each table along the way in an aliased -join, you can use ``from_joinpoint`` to keep joining against -the same line of aliases: - -:: - - # search for the treenode along the path "n1/n12/n122" - - # first find a Node with name="n122" - q = sess.query(Node).filter_by(name="n122") - - # then join to parent with "n12" - q = q.join("parent", aliased=True).filter_by(name="n12") - - # join again to the next parent with 'n1'. use 'from_joinpoint' - # so we join from the previous point, instead of joining off the - # root table - q = q.join("parent", aliased=True, from_joinpoint=True).filter_by(name="n1") - - node = q.first() - -``query.populate_existing()`` -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The eager version of ``query.load()`` (or -``session.refresh()``). Every instance loaded from the -query, including all eagerly loaded items, get refreshed -immediately if already present in the session: - -:: - - session.query(Blah).populate_existing().all() - -Relations ---------- - -SQL Clauses Embedded in Updates/Inserts -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -For inline execution of SQL clauses, embedded right in the -UPDATE or INSERT, during a ``flush()``: - -:: - - - myobject.foo = mytable.c.value + 1 - - user.pwhash = func.md5(password) - - order.hash = text("select hash from hashing_table") - -The column-attribute is set up with a deferred loader after -the operation, so that it issues the SQL to load the new -value when you next access. - -Self-referential and Cyclical Eager Loading -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Since our alias-fu has improved, ``relation()`` can join -along the same table \*any number of times*; you tell it how -deep you want to go. Lets show the self-referential -``TreeNode`` more clearly: - -:: - - nodes = Table( - "nodes", - metadata, - Column("id", Integer, primary_key=True), - Column("parent_id", Integer, ForeignKey("nodes.id")), - Column("name", String(30)), - ) - - - class TreeNode(object): - pass - - - mapper( - TreeNode, - nodes, - properties={"children": relation(TreeNode, lazy=False, join_depth=3)}, - ) - -So what happens when we say: - -:: - - create_session().query(TreeNode).all() - -? A join along aliases, three levels deep off the parent: - -.. sourcecode:: sql - - SELECT - nodes_3.id AS nodes_3_id, nodes_3.parent_id AS nodes_3_parent_id, nodes_3.name AS nodes_3_name, - nodes_2.id AS nodes_2_id, nodes_2.parent_id AS nodes_2_parent_id, nodes_2.name AS nodes_2_name, - nodes_1.id AS nodes_1_id, nodes_1.parent_id AS nodes_1_parent_id, nodes_1.name AS nodes_1_name, - nodes.id AS nodes_id, nodes.parent_id AS nodes_parent_id, nodes.name AS nodes_name - FROM nodes LEFT OUTER JOIN nodes AS nodes_1 ON nodes.id = nodes_1.parent_id - LEFT OUTER JOIN nodes AS nodes_2 ON nodes_1.id = nodes_2.parent_id - LEFT OUTER JOIN nodes AS nodes_3 ON nodes_2.id = nodes_3.parent_id - ORDER BY nodes.oid, nodes_1.oid, nodes_2.oid, nodes_3.oid - -Notice the nice clean alias names too. The joining doesn't -care if it's against the same immediate table or some other -object which then cycles back to the beginning. Any kind -of chain of eager loads can cycle back onto itself when -``join_depth`` is specified. When not present, eager -loading automatically stops when it hits a cycle. - -Composite Types -^^^^^^^^^^^^^^^ - -This is one from the Hibernate camp. Composite Types let -you define a custom datatype that is composed of more than -one column (or one column, if you wanted). Lets define a -new type, ``Point``. Stores an x/y coordinate: - -:: - - class Point(object): - def __init__(self, x, y): - self.x = x - self.y = y - - def __composite_values__(self): - return self.x, self.y - - def __eq__(self, other): - return other.x == self.x and other.y == self.y - - def __ne__(self, other): - return not self.__eq__(other) - -The way the ``Point`` object is defined is specific to a -custom type; constructor takes a list of arguments, and the -``__composite_values__()`` method produces a sequence of -those arguments. The order will match up to our mapper, as -we'll see in a moment. - -Let's create a table of vertices storing two points per row: - -:: - - vertices = Table( - "vertices", - metadata, - Column("id", Integer, primary_key=True), - Column("x1", Integer), - Column("y1", Integer), - Column("x2", Integer), - Column("y2", Integer), - ) - -Then, map it ! We'll create a ``Vertex`` object which -stores two ``Point`` objects: - -:: - - class Vertex(object): - def __init__(self, start, end): - self.start = start - self.end = end - - - mapper( - Vertex, - vertices, - properties={ - "start": composite(Point, vertices.c.x1, vertices.c.y1), - "end": composite(Point, vertices.c.x2, vertices.c.y2), - }, - ) - -Once you've set up your composite type, it's usable just -like any other type: - -:: - - - v = Vertex(Point(3, 4), Point(26, 15)) - session.save(v) - session.flush() - - # works in queries too - q = session.query(Vertex).filter(Vertex.start == Point(3, 4)) - -If you'd like to define the way the mapped attributes -generate SQL clauses when used in expressions, create your -own ``sqlalchemy.orm.PropComparator`` subclass, defining any -of the common operators (like ``__eq__()``, ``__le__()``, -etc.), and send it in to ``composite()``. Composite types -work as primary keys too, and are usable in ``query.get()``: - -:: - - # a Document class which uses a composite Version - # object as primary key - document = query.get(Version(1, "a")) - -``dynamic_loader()`` relations -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -A ``relation()`` that returns a live ``Query`` object for -all read operations. Write operations are limited to just -``append()`` and ``remove()``, changes to the collection are -not visible until the session is flushed. This feature is -particularly handy with an "autoflushing" session which will -flush before each query. - -:: - - mapper( - Foo, - foo_table, - properties={ - "bars": dynamic_loader( - Bar, - backref="foo", - # - ) - }, - ) - - session = create_session(autoflush=True) - foo = session.query(Foo).first() - - foo.bars.append(Bar(name="lala")) - - for bar in foo.bars.filter(Bar.name == "lala"): - print(bar) - - session.commit() - -New Options: ``undefer_group()``, ``eagerload_all()`` -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -A couple of query options which are handy. -``undefer_group()`` marks a whole group of "deferred" -columns as undeferred: - -:: - - mapper( - Class, - table, - properties={ - "foo": deferred(table.c.foo, group="group1"), - "bar": deferred(table.c.bar, group="group1"), - "bat": deferred(table.c.bat, group="group1"), - }, - ) - - session.query(Class).options(undefer_group("group1")).filter(...).all() - -and ``eagerload_all()`` sets a chain of attributes to be -eager in one pass: - -:: - - mapper(Foo, foo_table, properties={"bar": relation(Bar)}) - mapper(Bar, bar_table, properties={"bat": relation(Bat)}) - mapper(Bat, bat_table) - - # eager load bar and bat - session.query(Foo).options(eagerload_all("bar.bat")).filter(...).all() - -New Collection API -^^^^^^^^^^^^^^^^^^ - -Collections are no longer proxied by an -{{{InstrumentedList}}} proxy, and access to members, methods -and attributes is direct. Decorators now intercept objects -entering and leaving the collection, and it is now possible -to easily write a custom collection class that manages its -own membership. Flexible decorators also replace the named -method interface of custom collections in 0.3, allowing any -class to be easily adapted to use as a collection container. - -Dictionary-based collections are now much easier to use and -fully ``dict``-like. Changing ``__iter__`` is no longer -needed for ``dict``s, and new built-in ``dict`` types cover -many needs: - -:: - - # use a dictionary relation keyed by a column - relation(Item, collection_class=column_mapped_collection(items.c.keyword)) - # or named attribute - relation(Item, collection_class=attribute_mapped_collection("keyword")) - # or any function you like - relation(Item, collection_class=mapped_collection(lambda entity: entity.a + entity.b)) - -Existing 0.3 ``dict``-like and freeform object derived -collection classes will need to be updated for the new API. -In most cases this is simply a matter of adding a couple -decorators to the class definition. - -Mapped Relations from External Tables/Subqueries -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -This feature quietly appeared in 0.3 but has been improved -in 0.4 thanks to better ability to convert subqueries -against a table into subqueries against an alias of that -table; this is key for eager loading, aliased joins in -queries, etc. It reduces the need to create mappers against -select statements when you just need to add some extra -columns or subqueries: - -:: - - mapper( - User, - users, - properties={ - "fullname": column_property( - (users.c.firstname + users.c.lastname).label("fullname") - ), - "numposts": column_property( - select([func.count(1)], users.c.id == posts.c.user_id) - .correlate(users) - .label("posts") - ), - }, - ) - -a typical query looks like: - -.. sourcecode:: sql - - SELECT (SELECT count(1) FROM posts WHERE users.id = posts.user_id) AS count, - users.firstname || users.lastname AS fullname, - users.id AS users_id, users.firstname AS users_firstname, users.lastname AS users_lastname - FROM users ORDER BY users.oid - -Horizontal Scaling (Sharding) API ---------------------------------- - -[browser:/sqlalchemy/trunk/examples/sharding/attribute_shard -.py] - -Sessions --------- - -New Session Create Paradigm; SessionContext, assignmapper Deprecated -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -That's right, the whole shebang is being replaced with two -configurational functions. Using both will produce the most -0.1-ish feel we've had since 0.1 (i.e., the least amount of -typing). - -Configure your own ``Session`` class right where you define -your ``engine`` (or anywhere): - -:: - - from sqlalchemy import create_engine - from sqlalchemy.orm import sessionmaker - - engine = create_engine("myengine://") - Session = sessionmaker(bind=engine, autoflush=True, transactional=True) - - # use the new Session() freely - sess = Session() - sess.save(someobject) - sess.flush() - -If you need to post-configure your Session, say with an -engine, add it later with ``configure()``: - -:: - - Session.configure(bind=create_engine(...)) - -All the behaviors of ``SessionContext`` and the ``query`` -and ``__init__`` methods of ``assignmapper`` are moved into -the new ``scoped_session()`` function, which is compatible -with both ``sessionmaker`` as well as ``create_session()``: - -:: - - from sqlalchemy.orm import scoped_session, sessionmaker - - Session = scoped_session(sessionmaker(autoflush=True, transactional=True)) - Session.configure(bind=engine) - - u = User(name="wendy") - - sess = Session() - sess.save(u) - sess.commit() - - # Session constructor is thread-locally scoped. Everyone gets the same - # Session in the thread when scope="thread". - sess2 = Session() - assert sess is sess2 - -When using a thread-local ``Session``, the returned class -has all of ``Session's`` interface implemented as -classmethods, and "assignmapper"'s functionality is -available using the ``mapper`` classmethod. Just like the -old ``objectstore`` days.... - -:: - - - # "assignmapper"-like functionality available via ScopedSession.mapper - Session.mapper(User, users_table) - - u = User(name="wendy") - - Session.commit() - -Sessions are again Weak Referencing By Default -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The weak_identity_map flag is now set to ``True`` by default -on Session. Instances which are externally deferenced and -fall out of scope are removed from the session -automatically. However, items which have "dirty" changes -present will remain strongly referenced until those changes -are flushed at which case the object reverts to being weakly -referenced (this works for 'mutable' types, like picklable -attributes, as well). Setting weak_identity_map to -``False`` restores the old strong-referencing behavior for -those of you using the session like a cache. - -Auto-Transactional Sessions -^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -As you might have noticed above, we are calling ``commit()`` -on ``Session``. The flag ``transactional=True`` means the -``Session`` is always in a transaction, ``commit()`` -persists permanently. - -Auto-Flushing Sessions -^^^^^^^^^^^^^^^^^^^^^^ - -Also, ``autoflush=True`` means the ``Session`` will -``flush()`` before each ``query`` as well as when you call -``flush()`` or ``commit()``. So now this will work: - -:: - - Session = sessionmaker(bind=engine, autoflush=True, transactional=True) - - u = User(name="wendy") - - sess = Session() - sess.save(u) - - # wendy is flushed, comes right back from a query - wendy = sess.query(User).filter_by(name="wendy").one() - -Transactional methods moved onto sessions -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -``commit()`` and ``rollback()``, as well as ``begin()`` are -now directly on ``Session``. No more need to use -``SessionTransaction`` for anything (it remains in the -background). - -:: - - Session = sessionmaker(autoflush=True, transactional=False) - - sess = Session() - sess.begin() - - # use the session - - sess.commit() # commit transaction - -Sharing a ``Session`` with an enclosing engine-level (i.e. -non-ORM) transaction is easy: - -:: - - Session = sessionmaker(autoflush=True, transactional=False) - - conn = engine.connect() - trans = conn.begin() - sess = Session(bind=conn) - - # ... session is transactional - - # commit the outermost transaction - trans.commit() - -Nested Session Transactions with SAVEPOINT -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Available at the Engine and ORM level. ORM docs so far: - -https://www.sqlalchemy.org/docs/04/session.html#unitofwork_managing - -Two-Phase Commit Sessions -^^^^^^^^^^^^^^^^^^^^^^^^^ - -Available at the Engine and ORM level. ORM docs so far: - -https://www.sqlalchemy.org/docs/04/session.html#unitofwork_managing - -Inheritance ------------ - -Polymorphic Inheritance with No Joins or Unions -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -New docs for inheritance: https://www.sqlalchemy.org/docs/04 -/mappers.html#advdatamapping_mapper_inheritance_joined - -Better Polymorphic Behavior with ``get()`` -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -All classes within a joined-table inheritance hierarchy get -an ``_instance_key`` using the base class, i.e. -``(BaseClass, (1, ), None)``. That way when you call -``get()`` a ``Query`` against the base class, it can locate -subclass instances in the current identity map without -querying the database. - -Types ------ - -Custom Subclasses of ``sqlalchemy.types.TypeDecorator`` -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -There is a `New API `_ for subclassing a TypeDecorator. -Using the 0.3 API causes compilation errors in some cases. - -SQL Expressions -=============== - -All New, Deterministic Label/Alias Generation ---------------------------------------------- - -All the "anonymous" labels and aliases use a simple -_ format now. SQL is much easier to read and -is compatible with plan optimizer caches. Just check out -some of the examples in the tutorials: -https://www.sqlalchemy.org/docs/04/ormtutorial.html -https://www.sqlalchemy.org/docs/04/sqlexpression.html - -Generative select() Constructs ------------------------------- - -This is definitely the way to go with ``select()``. See htt -p://www.sqlalchemy.org/docs/04/sqlexpression.html#sql_transf -orm . - -New Operator System -------------------- - -SQL operators and more or less every SQL keyword there is -are now abstracted into the compiler layer. They now act -intelligently and are type/backend aware, see: -https://www.sqlalchemy.org/docs/04/sqlexpression.html#sql_operators - -All ``type`` Keyword Arguments Renamed to ``type_`` ---------------------------------------------------- - -Just like it says: - -:: - - b = bindparam("foo", type_=String) - -in\_ Function Changed to Accept Sequence or Selectable ------------------------------------------------------- - -The in\_ function now takes a sequence of values or a -selectable as its sole argument. The previous API of passing -in values as positional arguments still works, but is now -deprecated. This means that - -:: - - my_table.select(my_table.c.id.in_(1, 2, 3)) - my_table.select(my_table.c.id.in_(*listOfIds)) - -should be changed to - -:: - - my_table.select(my_table.c.id.in_([1, 2, 3])) - my_table.select(my_table.c.id.in_(listOfIds)) - -Schema and Reflection -===================== - -``MetaData``, ``BoundMetaData``, ``DynamicMetaData``... -------------------------------------------------------- - -In the 0.3.x series, ``BoundMetaData`` and -``DynamicMetaData`` were deprecated in favor of ``MetaData`` -and ``ThreadLocalMetaData``. The older names have been -removed in 0.4. Updating is simple: - -.. sourcecode:: text - - +-------------------------------------+-------------------------+ - |If You Had | Now Use | - +=====================================+=========================+ - | ``MetaData`` | ``MetaData`` | - +-------------------------------------+-------------------------+ - | ``BoundMetaData`` | ``MetaData`` | - +-------------------------------------+-------------------------+ - | ``DynamicMetaData`` (with one | ``MetaData`` | - | engine or threadlocal=False) | | - +-------------------------------------+-------------------------+ - | ``DynamicMetaData`` | ``ThreadLocalMetaData`` | - | (with different engines per thread) | | - +-------------------------------------+-------------------------+ - -The seldom-used ``name`` parameter to ``MetaData`` types has -been removed. The ``ThreadLocalMetaData`` constructor now -takes no arguments. Both types can now be bound to an -``Engine`` or a single ``Connection``. - -One Step Multi-Table Reflection -------------------------------- - -You can now load table definitions and automatically create -``Table`` objects from an entire database or schema in one -pass: - -:: - - >>> metadata = MetaData(myengine, reflect=True) - >>> metadata.tables.keys() - ['table_a', 'table_b', 'table_c', '...'] - -``MetaData`` also gains a ``.reflect()`` method enabling -finer control over the loading process, including -specification of a subset of available tables to load. - -SQL Execution -============= - -``engine``, ``connectable``, and ``bind_to`` are all now ``bind`` ------------------------------------------------------------------ - -``Transactions``, ``NestedTransactions`` and ``TwoPhaseTransactions`` ---------------------------------------------------------------------- - -Connection Pool Events ----------------------- - -The connection pool now fires events when new DB-API -connections are created, checked out and checked back into -the pool. You can use these to execute session-scoped SQL -setup statements on fresh connections, for example. - -Oracle Engine Fixed -------------------- - -In 0.3.11, there were bugs in the Oracle Engine on how -Primary Keys are handled. These bugs could cause programs -that worked fine with other engines, such as sqlite, to fail -when using the Oracle Engine. In 0.4, the Oracle Engine has -been reworked, fixing these Primary Key problems. - -Out Parameters for Oracle -------------------------- - -:: - - result = engine.execute( - text( - "begin foo(:x, :y, :z); end;", - bindparams=[ - bindparam("x", Numeric), - outparam("y", Numeric), - outparam("z", Numeric), - ], - ), - x=5, - ) - assert result.out_parameters == {"y": 10, "z": 75} - -Connection-bound ``MetaData``, ``Sessions`` -------------------------------------------- - -``MetaData`` and ``Session`` can be explicitly bound to a -connection: - -:: - - conn = engine.connect() - sess = create_session(bind=conn) - -Faster, More Foolproof ``ResultProxy`` Objects ----------------------------------------------- - diff --git a/doc/build/changelog/migration_05.rst b/doc/build/changelog/migration_05.rst deleted file mode 100644 index d26a22c0d0..0000000000 --- a/doc/build/changelog/migration_05.rst +++ /dev/null @@ -1,770 +0,0 @@ -============================= -What's new in SQLAlchemy 0.5? -============================= - -.. admonition:: About this Document - - This document describes changes between SQLAlchemy version 0.4, - last released October 12, 2008, and SQLAlchemy version 0.5, - last released January 16, 2010. - - Document date: August 4, 2009 - - -This guide documents API changes which affect users -migrating their applications from the 0.4 series of -SQLAlchemy to 0.5. It's also recommended for those working -from `Essential SQLAlchemy -`_, which only -covers 0.4 and seems to even have some old 0.3isms in it. -Note that SQLAlchemy 0.5 removes many behaviors which were -deprecated throughout the span of the 0.4 series, and also -deprecates more behaviors specific to 0.4. - -Major Documentation Changes -=========================== - -Some sections of the documentation have been completely -rewritten and can serve as an introduction to new ORM -features. The ``Query`` and ``Session`` objects in -particular have some distinct differences in API and -behavior which fundamentally change many of the basic ways -things are done, particularly with regards to constructing -highly customized ORM queries and dealing with stale session -state, commits and rollbacks. - -* `ORM Tutorial - `_ - -* `Session Documentation - `_ - -Deprecations Source -=================== - -Another source of information is documented within a series -of unit tests illustrating up to date usages of some common -``Query`` patterns; this file can be viewed at -[source:sqlalchemy/trunk/test/orm/test_deprecations.py]. - -Requirements Changes -==================== - -* Python 2.4 or higher is required. The SQLAlchemy 0.4 line - is the last version with Python 2.3 support. - -Object Relational Mapping -========================= - -* **Column level expressions within Query.** - as detailed - in the `tutorial - `_, - ``Query`` has the capability to create specific SELECT - statements, not just those against full rows: - - :: - - session.query(User.name, func.count(Address.id).label("numaddresses")).join( - Address - ).group_by(User.name) - - The tuples returned by any multi-column/entity query are - *named*' tuples: - - :: - - for row in ( - session.query(User.name, func.count(Address.id).label("numaddresses")) - .join(Address) - .group_by(User.name) - ): - print("name", row.name, "number", row.numaddresses) - - ``Query`` has a ``statement`` accessor, as well as a - ``subquery()`` method which allow ``Query`` to be used to - create more complex combinations: - - :: - - subq = ( - session.query(Keyword.id.label("keyword_id")) - .filter(Keyword.name.in_(["beans", "carrots"])) - .subquery() - ) - recipes = session.query(Recipe).filter( - exists() - .where(Recipe.id == recipe_keywords.c.recipe_id) - .where(recipe_keywords.c.keyword_id == subq.c.keyword_id) - ) - -* **Explicit ORM aliases are recommended for aliased joins** - - The ``aliased()`` function produces an "alias" of a - class, which allows fine-grained control of aliases in - conjunction with ORM queries. While a table-level alias - (i.e. ``table.alias()``) is still usable, an ORM level - alias retains the semantics of the ORM mapped object which - is significant for inheritance mappings, options, and - other scenarios. E.g.: - - :: - - Friend = aliased(Person) - session.query(Person, Friend).join((Friend, Person.friends)).all() - -* **query.join() greatly enhanced.** - You can now specify - the target and ON clause for a join in multiple ways. A - target class alone can be provided where SQLA will attempt - to form a join to it via foreign key in the same way as - ``table.join(someothertable)``. A target and an explicit - ON condition can be provided, where the ON condition can - be a ``relation()`` name, an actual class descriptor, or a - SQL expression. Or the old way of just a ``relation()`` - name or class descriptor works too. See the ORM tutorial - which has several examples. - -* **Declarative is recommended for applications which don't - require (and don't prefer) abstraction between tables and - mappers** - The [/docs/05/reference/ext/declarative.html - Declarative] module, which is used to combine the - expression of ``Table``, ``mapper()``, and user defined - class objects together, is highly recommended as it - simplifies application configuration, ensures the "one - mapper per class" pattern, and allows the full range of - configuration available to distinct ``mapper()`` calls. - Separate ``mapper()`` and ``Table`` usage is now referred - to as "classical SQLAlchemy usage" and of course is freely - mixable with declarative. - -* **The .c. attribute has been removed** from classes (i.e. - ``MyClass.c.somecolumn``). As is the case in 0.4, class- - level properties are usable as query elements, i.e. - ``Class.c.propname`` is now superseded by - ``Class.propname``, and the ``c`` attribute continues to - remain on ``Table`` objects where they indicate the - namespace of ``Column`` objects present on the table. - - To get at the Table for a mapped class (if you didn't keep - it around already): - - :: - - table = class_mapper(someclass).mapped_table - - Iterate through columns: - - :: - - for col in table.c: - print(col) - - Work with a specific column: - - :: - - table.c.somecolumn - - The class-bound descriptors support the full set of Column - operators as well as the documented relation-oriented - operators like ``has()``, ``any()``, ``contains()``, etc. - - The reason for the hard removal of ``.c.`` is that in 0.5, - class-bound descriptors carry potentially different - meaning, as well as information regarding class mappings, - versus plain ``Column`` objects - and there are use cases - where you'd specifically want to use one or the other. - Generally, using class-bound descriptors invokes a set of - mapping/polymorphic aware translations, and using table- - bound columns does not. In 0.4, these translations were - applied across the board to all expressions, but 0.5 - differentiates completely between columns and mapped - descriptors, only applying translations to the latter. So - in many cases, particularly when dealing with joined table - inheritance configurations as well as when using - ``query()``, ``Class.propname`` and - ``table.c.colname`` are not interchangeable. - - For example, ``session.query(users.c.id, users.c.name)`` - is different versus ``session.query(User.id, User.name)``; - in the latter case, the ``Query`` is aware of the mapper - in use and further mapper-specific operations like - ``query.join()``, ``query.with_parent()`` etc. - may be used, but in the former case cannot. Additionally, - in polymorphic inheritance scenarios, the class-bound - descriptors refer to the columns present in the - polymorphic selectable in use, not necessarily the table - column which directly corresponds to the descriptor. For - example, a set of classes related by joined-table - inheritance to the ``person`` table along the - ``person_id`` column of each table will all have their - ``Class.person_id`` attribute mapped to the ``person_id`` - column in ``person``, and not their subclass table. - Version 0.4 would map this behavior onto table-bound - ``Column`` objects automatically. In 0.5, this automatic - conversion has been removed, so that you in fact *can* use - table-bound columns as a means to override the - translations which occur with polymorphic querying; this - allows ``Query`` to be able to create optimized selects - among joined-table or concrete-table inheritance setups, - as well as portable subqueries, etc. - -* **Session Now Synchronizes Automatically with - Transactions.** Session now synchronizes against the - transaction automatically by default, including autoflush - and autoexpire. A transaction is present at all times - unless disabled using the ``autocommit`` option. When all - three flags are set to their default, the Session recovers - gracefully after rollbacks and it's very difficult to get - stale data into the session. See the new Session - documentation for details. - -* **Implicit Order By Is Removed**. This will impact ORM - users who rely upon SA's "implicit ordering" behavior, - which states that all Query objects which don't have an - ``order_by()`` will ORDER BY the "id" or "oid" column of - the primary mapped table, and all lazy/eagerly loaded - collections apply a similar ordering. In 0.5, automatic - ordering must be explicitly configured on ``mapper()`` and - ``relation()`` objects (if desired), or otherwise when - using ``Query``. - - To convert an 0.4 mapping to 0.5, such that its ordering - behavior will be extremely similar to 0.4 or previous, use - the ``order_by`` setting on ``mapper()`` and - ``relation()``: - - :: - - mapper( - User, - users, - properties={"addresses": relation(Address, order_by=addresses.c.id)}, - order_by=users.c.id, - ) - - To set ordering on a backref, use the ``backref()`` - function: - - :: - - "keywords": relation( - Keyword, - secondary=item_keywords, - order_by=keywords.c.name, - backref=backref("items", order_by=items.c.id), - ) - - Using declarative ? To help with the new ``order_by`` - requirement, ``order_by`` and friends can now be set using - strings which are evaluated in Python later on (this works - **only** with declarative, not plain mappers): - - :: - - class MyClass(MyDeclarativeBase): - ... - "addresses": relation("Address", order_by="Address.id") - - It's generally a good idea to set ``order_by`` on - ``relation()s`` which load list-based collections of - items, since that ordering cannot otherwise be affected. - Other than that, the best practice is to use - ``Query.order_by()`` to control ordering of the primary - entities being loaded. - -* **Session is now - autoflush=True/autoexpire=True/autocommit=False.** - To - set it up, just call ``sessionmaker()`` with no arguments. - The name ``transactional=True`` is now - ``autocommit=False``. Flushes occur upon each query - issued (disable with ``autoflush=False``), within each - ``commit()`` (as always), and before each - ``begin_nested()`` (so rolling back to the SAVEPOINT is - meaningful). All objects are expired after each - ``commit()`` and after each ``rollback()``. After - rollback, pending objects are expunged, deleted objects - move back to persistent. These defaults work together - very nicely and there's really no more need for old - techniques like ``clear()`` (which is renamed to - ``expunge_all()`` as well). - - P.S.: sessions are now reusable after a ``rollback()``. - Scalar and collection attribute changes, adds and deletes - are all rolled back. - -* **session.add() replaces session.save(), session.update(), - session.save_or_update().** - the - ``session.add(someitem)`` and ``session.add_all([list of - items])`` methods replace ``save()``, ``update()``, and - ``save_or_update()``. Those methods will remain - deprecated throughout 0.5. - -* **backref configuration made less verbose.** - The - ``backref()`` function now uses the ``primaryjoin`` and - ``secondaryjoin`` arguments of the forwards-facing - ``relation()`` when they are not explicitly stated. It's - no longer necessary to specify - ``primaryjoin``/``secondaryjoin`` in both directions - separately. - -* **Simplified polymorphic options.** - The ORM's - "polymorphic load" behavior has been simplified. In 0.4, - mapper() had an argument called ``polymorphic_fetch`` - which could be configured as ``select`` or ``deferred``. - This option is removed; the mapper will now just defer any - columns which were not present in the SELECT statement. - The actual SELECT statement used is controlled by the - ``with_polymorphic`` mapper argument (which is also in 0.4 - and replaces ``select_table``), as well as the - ``with_polymorphic()`` method on ``Query`` (also in 0.4). - - An improvement to the deferred loading of inheriting - classes is that the mapper now produces the "optimized" - version of the SELECT statement in all cases; that is, if - class B inherits from A, and several attributes only - present on class B have been expired, the refresh - operation will only include B's table in the SELECT - statement and will not JOIN to A. - -* The ``execute()`` method on ``Session`` converts plain - strings into ``text()`` constructs, so that bind - parameters may all be specified as ":bindname" without - needing to call ``text()`` explicitly. If "raw" SQL is - desired here, use ``session.connection().execute("raw - text")``. - -* ``session.Query().iterate_instances()`` has been renamed - to just ``instances()``. The old ``instances()`` method - returning a list instead of an iterator no longer exists. - If you were relying on that behavior, you should use - ``list(your_query.instances())``. - -Extending the ORM -================= - -In 0.5 we're moving forward with more ways to modify and -extend the ORM. Heres a summary: - -* **MapperExtension.** - This is the classic extension - class, which remains. Methods which should rarely be - needed are ``create_instance()`` and - ``populate_instance()``. To control the initialization of - an object when it's loaded from the database, use the - ``reconstruct_instance()`` method, or more easily the - ``@reconstructor`` decorator described in the - documentation. - -* **SessionExtension.** - This is an easy to use extension - class for session events. In particular, it provides - ``before_flush()``, ``after_flush()`` and - ``after_flush_postexec()`` methods. This usage is - recommended over ``MapperExtension.before_XXX`` in many - cases since within ``before_flush()`` you can modify the - flush plan of the session freely, something which cannot - be done from within ``MapperExtension``. - -* **AttributeExtension.** - This class is now part of the - public API, and allows the interception of userland events - on attributes, including attribute set and delete - operations, and collection appends and removes. It also - allows the value to be set or appended to be modified. - The ``@validates`` decorator, described in the - documentation, provides a quick way to mark any mapped - attributes as being "validated" by a particular class - method. - -* **Attribute Instrumentation Customization.** - An API is - provided for ambitious efforts to entirely replace - SQLAlchemy's attribute instrumentation, or just to augment - it in some cases. This API was produced for the purposes - of the Trellis toolkit, but is available as a public API. - Some examples are provided in the distribution in the - ``/examples/custom_attributes`` directory. - -Schema/Types -============ - -* **String with no length no longer generates TEXT, it - generates VARCHAR** - The ``String`` type no longer - magically converts into a ``Text`` type when specified - with no length. This only has an effect when CREATE TABLE - is issued, as it will issue ``VARCHAR`` with no length - parameter, which is not valid on many (but not all) - databases. To create a TEXT (or CLOB, i.e. unbounded - string) column, use the ``Text`` type. - -* **PickleType() with mutable=True requires an __eq__() - method** - The ``PickleType`` type needs to compare values - when mutable=True. The method of comparing - ``pickle.dumps()`` is inefficient and unreliable. If an - incoming object does not implement ``__eq__()`` and is - also not ``None``, the ``dumps()`` comparison is used but - a warning is raised. For types which implement - ``__eq__()`` which includes all dictionaries, lists, etc., - comparison will use ``==`` and is now reliable by default. - -* **convert_bind_param() and convert_result_value() methods - of TypeEngine/TypeDecorator are removed.** - The O'Reilly - book unfortunately documented these methods even though - they were deprecated post 0.3. For a user-defined type - which subclasses ``TypeEngine``, the ``bind_processor()`` - and ``result_processor()`` methods should be used for - bind/result processing. Any user defined type, whether - extending ``TypeEngine`` or ``TypeDecorator``, which uses - the old 0.3 style can be easily adapted to the new style - using the following adapter: - - :: - - class AdaptOldConvertMethods(object): - """A mixin which adapts 0.3-style convert_bind_param and - convert_result_value methods - - """ - - def bind_processor(self, dialect): - def convert(value): - return self.convert_bind_param(value, dialect) - - return convert - - def result_processor(self, dialect): - def convert(value): - return self.convert_result_value(value, dialect) - - return convert - - def convert_result_value(self, value, dialect): - return value - - def convert_bind_param(self, value, dialect): - return value - - To use the above mixin: - - :: - - class MyType(AdaptOldConvertMethods, TypeEngine): - ... - -* The ``quote`` flag on ``Column`` and ``Table`` as well as - the ``quote_schema`` flag on ``Table`` now control quoting - both positively and negatively. The default is ``None``, - meaning let regular quoting rules take effect. When - ``True``, quoting is forced on. When ``False``, quoting - is forced off. - -* Column ``DEFAULT`` value DDL can now be more conveniently - specified with ``Column(..., server_default='val')``, - deprecating ``Column(..., PassiveDefault('val'))``. - ``default=`` is now exclusively for Python-initiated - default values, and can coexist with server_default. A - new ``server_default=FetchedValue()`` replaces the - ``PassiveDefault('')`` idiom for marking columns as - subject to influence from external triggers and has no DDL - side effects. - -* SQLite's ``DateTime``, ``Time`` and ``Date`` types now - **only accept datetime objects, not strings** as bind - parameter input. If you'd like to create your own - "hybrid" type which accepts strings and returns results as - date objects (from whatever format you'd like), create a - ``TypeDecorator`` that builds on ``String``. If you only - want string-based dates, just use ``String``. - -* Additionally, the ``DateTime`` and ``Time`` types, when - used with SQLite, now represent the "microseconds" field - of the Python ``datetime.datetime`` object in the same - manner as ``str(datetime)`` - as fractional seconds, not a - count of microseconds. That is: - - :: - - dt = datetime.datetime(2008, 6, 27, 12, 0, 0, 125) # 125 usec - - # old way - "2008-06-27 12:00:00.125" - - # new way - "2008-06-27 12:00:00.000125" - - So if an existing SQLite file-based database intends to be - used across 0.4 and 0.5, you either have to upgrade the - datetime columns to store the new format (NOTE: please - test this, I'm pretty sure its correct): - - .. sourcecode:: sql - - UPDATE mytable SET somedatecol = - substr(somedatecol, 0, 19) || '.' || substr((substr(somedatecol, 21, -1) / 1000000), 3, -1); - - or, enable "legacy" mode as follows: - - :: - - from sqlalchemy.databases.sqlite import DateTimeMixin - - DateTimeMixin.__legacy_microseconds__ = True - -Connection Pool no longer threadlocal by default -================================================ - -0.4 has an unfortunate default setting of -"pool_threadlocal=True", leading to surprise behavior when, -for example, using multiple Sessions within a single thread. -This flag is now off in 0.5. To re-enable 0.4's behavior, -specify ``pool_threadlocal=True`` to ``create_engine()``, or -alternatively use the "threadlocal" strategy via -``strategy="threadlocal"``. - -\*args Accepted, \*args No Longer Accepted -========================================== - -The policy with ``method(\*args)`` vs. ``method([args])`` -is, if the method accepts a variable-length set of items -which represent a fixed structure, it takes ``\*args``. If -the method accepts a variable-length set of items that are -data-driven, it takes ``[args]``. - -* The various Query.options() functions ``eagerload()``, - ``eagerload_all()``, ``lazyload()``, ``contains_eager()``, - ``defer()``, ``undefer()`` all accept variable-length - ``\*keys`` as their argument now, which allows a path to - be formulated using descriptors, ie.: - - :: - - query.options(eagerload_all(User.orders, Order.items, Item.keywords)) - - A single array argument is still accepted for backwards - compatibility. - -* Similarly, the ``Query.join()`` and ``Query.outerjoin()`` - methods accept a variable length \*args, with a single - array accepted for backwards compatibility: - - :: - - query.join("orders", "items") - query.join(User.orders, Order.items) - -* the ``in_()`` method on columns and similar only accepts a - list argument now. It no longer accepts ``\*args``. - -Removed -======= - -* **entity_name** - This feature was always problematic and - rarely used. 0.5's more deeply fleshed out use cases - revealed further issues with ``entity_name`` which led to - its removal. If different mappings are required for a - single class, break the class into separate subclasses and - map them separately. An example of this is at - [wiki:UsageRecipes/EntityName]. More information - regarding rationale is described at https://groups.google.c - om/group/sqlalchemy/browse_thread/thread/9e23a0641a88b96d? - hl=en . - -* **get()/load() cleanup** - - - The ``load()`` method has been removed. Its - functionality was kind of arbitrary and basically copied - from Hibernate, where it's also not a particularly - meaningful method. - - To get equivalent functionality: - - :: - - x = session.query(SomeClass).populate_existing().get(7) - - ``Session.get(cls, id)`` and ``Session.load(cls, id)`` - have been removed. ``Session.get()`` is redundant vs. - ``session.query(cls).get(id)``. - - ``MapperExtension.get()`` is also removed (as is - ``MapperExtension.load()``). To override the - functionality of ``Query.get()``, use a subclass: - - :: - - class MyQuery(Query): - def get(self, ident): - ... - - - session = sessionmaker(query_cls=MyQuery)() - - ad1 = session.query(Address).get(1) - -* ``sqlalchemy.orm.relation()`` - - - The following deprecated keyword arguments have been - removed: - - foreignkey, association, private, attributeext, is_backref - - In particular, ``attributeext`` is replaced with - ``extension`` - the ``AttributeExtension`` class is now in - the public API. - -* ``session.Query()`` - - - The following deprecated functions have been removed: - - list, scalar, count_by, select_whereclause, get_by, - select_by, join_by, selectfirst, selectone, select, - execute, select_statement, select_text, join_to, join_via, - selectfirst_by, selectone_by, apply_max, apply_min, - apply_avg, apply_sum - - Additionally, the ``id`` keyword argument to ``join()``, - ``outerjoin()``, ``add_entity()`` and ``add_column()`` has - been removed. To target table aliases in ``Query`` to - result columns, use the ``aliased`` construct: - - :: - - from sqlalchemy.orm import aliased - - address_alias = aliased(Address) - print(session.query(User, address_alias).join((address_alias, User.addresses)).all()) - -* ``sqlalchemy.orm.Mapper`` - - - * instances() - - - * get_session() - this method was not very noticeable, but - had the effect of associating lazy loads with a - particular session even if the parent object was - entirely detached, when an extension such as - ``scoped_session()`` or the old ``SessionContextExt`` - was used. It's possible that some applications which - relied upon this behavior will no longer work as - expected; but the better programming practice here is - to always ensure objects are present within sessions if - database access from their attributes are required. - -* ``mapper(MyClass, mytable)`` - - - Mapped classes no are longer instrumented with a "c" class - attribute; e.g. ``MyClass.c`` - -* ``sqlalchemy.orm.collections`` - - - The _prepare_instrumentation alias for - prepare_instrumentation has been removed. - -* ``sqlalchemy.orm`` - - - Removed the ``EXT_PASS`` alias of ``EXT_CONTINUE``. - -* ``sqlalchemy.engine`` - - - The alias from ``DefaultDialect.preexecute_sequences`` to - ``.preexecute_pk_sequences`` has been removed. - - The deprecated engine_descriptors() function has been - removed. - -* ``sqlalchemy.ext.activemapper`` - - - Module removed. - -* ``sqlalchemy.ext.assignmapper`` - - - Module removed. - -* ``sqlalchemy.ext.associationproxy`` - - - Pass-through of keyword args on the proxy's - ``.append(item, \**kw)`` has been removed and is now - simply ``.append(item)`` - -* ``sqlalchemy.ext.selectresults``, - ``sqlalchemy.mods.selectresults`` - - Modules removed. - -* ``sqlalchemy.ext.declarative`` - - - ``declared_synonym()`` removed. - -* ``sqlalchemy.ext.sessioncontext`` - - - Module removed. - -* ``sqlalchemy.log`` - - - The ``SADeprecationWarning`` alias to - ``sqlalchemy.exc.SADeprecationWarning`` has been removed. - -* ``sqlalchemy.exc`` - - - ``exc.AssertionError`` has been removed and usage replaced - by the Python built-in of the same name. - -* ``sqlalchemy.databases.mysql`` - - - The deprecated ``get_version_info`` dialect method has - been removed. - -Renamed or Moved -================ - -* ``sqlalchemy.exceptions`` is now ``sqlalchemy.exc`` - - - The module may still be imported under the old name until - 0.6. - -* ``FlushError``, ``ConcurrentModificationError``, - ``UnmappedColumnError`` -> sqlalchemy.orm.exc - - These exceptions moved to the orm package. Importing - 'sqlalchemy.orm' will install aliases in sqlalchemy.exc - for compatibility until 0.6. - -* ``sqlalchemy.logging`` -> ``sqlalchemy.log`` - - - This internal module was renamed. No longer needs to be - special cased when packaging SA with py2app and similar - tools that scan imports. - -* ``session.Query().iterate_instances()`` -> - ``session.Query().instances()``. - -Deprecated -========== - -* ``Session.save()``, ``Session.update()``, - ``Session.save_or_update()`` - - All three replaced by ``Session.add()`` - -* ``sqlalchemy.PassiveDefault`` - - - Use ``Column(server_default=...)`` Translates to - sqlalchemy.DefaultClause() under the hood. - -* ``session.Query().iterate_instances()``. It has been - renamed to ``instances()``. - diff --git a/doc/build/changelog/migration_06.rst b/doc/build/changelog/migration_06.rst deleted file mode 100644 index 0330ac5d4a..0000000000 --- a/doc/build/changelog/migration_06.rst +++ /dev/null @@ -1,1241 +0,0 @@ -============================= -What's New in SQLAlchemy 0.6? -============================= - -.. admonition:: About this Document - - This document describes changes between SQLAlchemy version 0.5, - last released January 16, 2010, and SQLAlchemy version 0.6, - last released May 5, 2012. - - Document date: June 6, 2010 - -This guide documents API changes which affect users -migrating their applications from the 0.5 series of -SQLAlchemy to 0.6. Note that SQLAlchemy 0.6 removes some -behaviors which were deprecated throughout the span of the -0.5 series, and also deprecates more behaviors specific to -0.5. - -Platform Support -================ - -* cPython versions 2.4 and upwards throughout the 2.xx - series - -* Jython 2.5.1 - using the zxJDBC DBAPI included with - Jython. - -* cPython 3.x - see [source:sqlalchemy/trunk/README.py3k] - for information on how to build for python3. - -New Dialect System -================== - -Dialect modules are now broken up into distinct -subcomponents, within the scope of a single database -backend. Dialect implementations are now in the -``sqlalchemy.dialects`` package. The -``sqlalchemy.databases`` package still exists as a -placeholder to provide some level of backwards compatibility -for simple imports. - -For each supported database, a sub-package exists within -``sqlalchemy.dialects`` where several files are contained. -Each package contains a module called ``base.py`` which -defines the specific SQL dialect used by that database. It -also contains one or more "driver" modules, each one -corresponding to a specific DBAPI - these files are named -corresponding to the DBAPI itself, such as ``pysqlite``, -``cx_oracle``, or ``pyodbc``. The classes used by -SQLAlchemy dialects are first declared in the ``base.py`` -module, defining all behavioral characteristics defined by -the database. These include capability mappings, such as -"supports sequences", "supports returning", etc., type -definitions, and SQL compilation rules. Each "driver" -module in turn provides subclasses of those classes as -needed which override the default behavior to accommodate -the additional features, behaviors, and quirks of that -DBAPI. For DBAPIs that support multiple backends (pyodbc, -zxJDBC, mxODBC), the dialect module will use mixins from the -``sqlalchemy.connectors`` package, which provide -functionality common to that DBAPI across all backends, most -typically dealing with connect arguments. This means that -connecting using pyodbc, zxJDBC or mxODBC (when implemented) -is extremely consistent across supported backends. - -The URL format used by ``create_engine()`` has been enhanced -to handle any number of DBAPIs for a particular backend, -using a scheme that is inspired by that of JDBC. The -previous format still works, and will select a "default" -DBAPI implementation, such as the PostgreSQL URL below that -will use psycopg2: - -:: - - create_engine("postgresql://scott:tiger@localhost/test") - -However to specify a specific DBAPI backend such as pg8000, -add it to the "protocol" section of the URL using a plus -sign "+": - -:: - - create_engine("postgresql+pg8000://scott:tiger@localhost/test") - -Important Dialect Links: - -* Documentation on connect arguments: - https://www.sqlalchemy.org/docs/06/dbengine.html#create- - engine-url-arguments. - -* Reference documentation for individual dialects: https://ww - w.sqlalchemy.org/docs/06/reference/dialects/index.html - -* The tips and tricks at DatabaseNotes. - - -Other notes regarding dialects: - -* the type system has been changed dramatically in - SQLAlchemy 0.6. This has an impact on all dialects - regarding naming conventions, behaviors, and - implementations. See the section on "Types" below. - -* the ``ResultProxy`` object now offers a 2x speed - improvement in some cases thanks to some refactorings. - -* the ``RowProxy``, i.e. individual result row object, is - now directly pickleable. - -* the setuptools entrypoint used to locate external dialects - is now called ``sqlalchemy.dialects``. An external - dialect written against 0.4 or 0.5 will need to be - modified to work with 0.6 in any case so this change does - not add any additional difficulties. - -* dialects now receive an initialize() event on initial - connection to determine connection properties. - -* Functions and operators generated by the compiler now use - (almost) regular dispatch functions of the form - "visit_" and "visit__fn" to provide - customed processing. This replaces the need to copy the - "functions" and "operators" dictionaries in compiler - subclasses with straightforward visitor methods, and also - allows compiler subclasses complete control over - rendering, as the full _Function or _BinaryExpression - object is passed in. - -Dialect Imports ---------------- - -The import structure of dialects has changed. Each dialect -now exports its base "dialect" class as well as the full set -of SQL types supported on that dialect via -``sqlalchemy.dialects.``. For example, to import a -set of PG types: - -:: - - from sqlalchemy.dialects.postgresql import ( - INTEGER, - BIGINT, - SMALLINT, - VARCHAR, - MACADDR, - DATE, - BYTEA, - ) - -Above, ``INTEGER`` is actually the plain ``INTEGER`` type -from ``sqlalchemy.types``, but the PG dialect makes it -available in the same way as those types which are specific -to PG, such as ``BYTEA`` and ``MACADDR``. - -Expression Language Changes -=========================== - -An Important Expression Language Gotcha ---------------------------------------- - -There's one quite significant behavioral change to the -expression language which may affect some applications. -The boolean value of Python boolean expressions, i.e. -``==``, ``!=``, and similar, now evaluates accurately with -regards to the two clause objects being compared. - -As we know, comparing a ``ClauseElement`` to any other -object returns another ``ClauseElement``: - -:: - - >>> from sqlalchemy.sql import column - >>> column("foo") == 5 - - -This so that Python expressions produce SQL expressions when -converted to strings: - -:: - - >>> str(column("foo") == 5) - 'foo = :foo_1' - -But what happens if we say this? - -:: - - >>> if column("foo") == 5: - ... print("yes") - -In previous versions of SQLAlchemy, the returned -``_BinaryExpression`` was a plain Python object which -evaluated to ``True``. Now it evaluates to whether or not -the actual ``ClauseElement`` should have the same hash value -as to that being compared. Meaning: - -:: - - >>> bool(column("foo") == 5) - False - >>> bool(column("foo") == column("foo")) - False - >>> c = column("foo") - >>> bool(c == c) - True - >>> - -That means code such as the following: - -:: - - if expression: - print("the expression is:", expression) - -Would not evaluate if ``expression`` was a binary clause. -Since the above pattern should never be used, the base -``ClauseElement`` now raises an exception if called in a -boolean context: - -:: - - >>> bool(c) - Traceback (most recent call last): - File "", line 1, in - ... - raise TypeError("Boolean value of this clause is not defined") - TypeError: Boolean value of this clause is not defined - -Code that wants to check for the presence of a -``ClauseElement`` expression should instead say: - -:: - - if expression is not None: - print("the expression is:", expression) - -Keep in mind, **this applies to Table and Column objects -too**. - -The rationale for the change is twofold: - -* Comparisons of the form ``if c1 == c2: `` - can actually be written now - -* Support for correct hashing of ``ClauseElement`` objects - now works on alternate platforms, namely Jython. Up until - this point SQLAlchemy relied heavily on the specific - behavior of cPython in this regard (and still had - occasional problems with it). - -Stricter "executemany" Behavior -------------------------------- - -An "executemany" in SQLAlchemy corresponds to a call to -``execute()``, passing along a collection of bind parameter -sets: - -:: - - connection.execute(table.insert(), {"data": "row1"}, {"data": "row2"}, {"data": "row3"}) - -When the ``Connection`` object sends off the given -``insert()`` construct for compilation, it passes to the -compiler the keynames present in the first set of binds -passed along to determine the construction of the -statement's VALUES clause. Users familiar with this -construct will know that additional keys present in the -remaining dictionaries don't have any impact. What's -different now is that all subsequent dictionaries need to -include at least *every* key that is present in the first -dictionary. This means that a call like this no longer -works: - -:: - - connection.execute( - table.insert(), - {"timestamp": today, "data": "row1"}, - {"timestamp": today, "data": "row2"}, - {"data": "row3"}, - ) - -Because the third row does not specify the 'timestamp' -column. Previous versions of SQLAlchemy would simply insert -NULL for these missing columns. However, if the -``timestamp`` column in the above example contained a -Python-side default value or function, it would *not* be -used. This because the "executemany" operation is optimized -for maximum performance across huge numbers of parameter -sets, and does not attempt to evaluate Python-side defaults -for those missing keys. Because defaults are often -implemented either as SQL expressions which are embedded -inline with the INSERT statement, or are server side -expressions which again are triggered based on the structure -of the INSERT string, which by definition cannot fire off -conditionally based on each parameter set, it would be -inconsistent for Python side defaults to behave differently -vs. SQL/server side defaults. (SQL expression based -defaults are embedded inline as of the 0.5 series, again to -minimize the impact of huge numbers of parameter sets). - -SQLAlchemy 0.6 therefore establishes predictable consistency -by forbidding any subsequent parameter sets from leaving any -fields blank. That way, there's no more silent failure of -Python side default values and functions, which additionally -are allowed to remain consistent in their behavior versus -SQL and server side defaults. - -UNION and other "compound" constructs parenthesize consistently ---------------------------------------------------------------- - -A rule that was designed to help SQLite has been removed, -that of the first compound element within another compound -(such as, a ``union()`` inside of an ``except_()``) wouldn't -be parenthesized. This is inconsistent and produces the -wrong results on PostgreSQL, which has precedence rules -regarding INTERSECTION, and its generally a surprise. When -using complex composites with SQLite, you now need to turn -the first element into a subquery (which is also compatible -on PG). A new example is in the SQL expression tutorial at -the end of -[https://www.sqlalchemy.org/docs/06/sqlexpression.html -#unions-and-other-set-operations]. See :ticket:`1665` and -r6690 for more background. - -C Extensions for Result Fetching -================================ - -The ``ResultProxy`` and related elements, including most -common "row processing" functions such as unicode -conversion, numerical/boolean conversions and date parsing, -have been re-implemented as optional C extensions for the -purposes of performance. This represents the beginning of -SQLAlchemy's path to the "dark side" where we hope to -continue improving performance by reimplementing critical -sections in C. The extensions can be built by specifying -``--with-cextensions``, i.e. ``python setup.py --with- -cextensions install``. - -The extensions have the most dramatic impact on result -fetching using direct ``ResultProxy`` access, i.e. that -which is returned by ``engine.execute()``, -``connection.execute()``, or ``session.execute()``. Within -results returned by an ORM ``Query`` object, result fetching -is not as high a percentage of overhead, so ORM performance -improves more modestly, and mostly in the realm of fetching -large result sets. The performance improvements highly -depend on the dbapi in use and on the syntax used to access -the columns of each row (eg ``row['name']`` is much faster -than ``row.name``). The current extensions have no impact -on the speed of inserts/updates/deletes, nor do they improve -the latency of SQL execution, that is, an application that -spends most of its time executing many statements with very -small result sets will not see much improvement. - -Performance has been improved in 0.6 versus 0.5 regardless -of the extensions. A quick overview of what connecting and -fetching 50,000 rows looks like with SQLite, using mostly -direct SQLite access, a ``ResultProxy``, and a simple mapped -ORM object: - -.. sourcecode:: text - - sqlite select/native: 0.260s - - 0.6 / C extension - - sqlalchemy.sql select: 0.360s - sqlalchemy.orm fetch: 2.500s - - 0.6 / Pure Python - - sqlalchemy.sql select: 0.600s - sqlalchemy.orm fetch: 3.000s - - 0.5 / Pure Python - - sqlalchemy.sql select: 0.790s - sqlalchemy.orm fetch: 4.030s - -Above, the ORM fetches the rows 33% faster than 0.5 due to -in-python performance enhancements. With the C extensions -we get another 20%. However, ``ResultProxy`` fetches -improve by 67% with the C extension versus not. Other -tests report as much as a 200% speed improvement for some -scenarios, such as those where lots of string conversions -are occurring. - -New Schema Capabilities -======================= - -The ``sqlalchemy.schema`` package has received some long- -needed attention. The most visible change is the newly -expanded DDL system. In SQLAlchemy, it was possible since -version 0.5 to create custom DDL strings and associate them -with tables or metadata objects: - -:: - - from sqlalchemy.schema import DDL - - DDL("CREATE TRIGGER users_trigger ...").execute_at("after-create", metadata) - -Now the full suite of DDL constructs are available under the -same system, including those for CREATE TABLE, ADD -CONSTRAINT, etc.: - -:: - - from sqlalchemy.schema import Constraint, AddConstraint - - AddContraint(CheckConstraint("value > 5")).execute_at("after-create", mytable) - -Additionally, all the DDL objects are now regular -``ClauseElement`` objects just like any other SQLAlchemy -expression object: - -:: - - from sqlalchemy.schema import CreateTable - - create = CreateTable(mytable) - - # dumps the CREATE TABLE as a string - print(create) - - # executes the CREATE TABLE statement - engine.execute(create) - -and using the ``sqlalchemy.ext.compiler`` extension you can -make your own: - -:: - - from sqlalchemy.schema import DDLElement - from sqlalchemy.ext.compiler import compiles - - - class AlterColumn(DDLElement): - def __init__(self, column, cmd): - self.column = column - self.cmd = cmd - - - @compiles(AlterColumn) - def visit_alter_column(element, compiler, **kw): - return "ALTER TABLE %s ALTER COLUMN %s %s ..." % ( - element.column.table.name, - element.column.name, - element.cmd, - ) - - - engine.execute(AlterColumn(table.c.mycolumn, "SET DEFAULT 'test'")) - -Deprecated/Removed Schema Elements ----------------------------------- - -The schema package has also been greatly streamlined. Many -options and methods which were deprecated throughout 0.5 -have been removed. Other little known accessors and methods -have also been removed. - -* the "owner" keyword argument is removed from ``Table``. - Use "schema" to represent any namespaces to be prepended - to the table name. - -* deprecated ``MetaData.connect()`` and - ``ThreadLocalMetaData.connect()`` have been removed - send - the "bind" attribute to bind a metadata. - -* deprecated metadata.table_iterator() method removed (use - sorted_tables) - -* the "metadata" argument is removed from - ``DefaultGenerator`` and subclasses, but remains locally - present on ``Sequence``, which is a standalone construct - in DDL. - -* deprecated ``PassiveDefault`` - use ``DefaultClause``. - - -* Removed public mutability from ``Index`` and - ``Constraint`` objects: - - * ``ForeignKeyConstraint.append_element()`` - - - * ``Index.append_column()`` - - - * ``UniqueConstraint.append_column()`` - - - * ``PrimaryKeyConstraint.add()`` - - - * ``PrimaryKeyConstraint.remove()`` - - -These should be constructed declaratively (i.e. in one -construction). - -* Other removed things: - - - * ``Table.key`` (no idea what this was for) - - - * ``Column.bind`` (get via column.table.bind) - - - * ``Column.metadata`` (get via column.table.metadata) - - - * ``Column.sequence`` (use column.default) - - -Other Behavioral Changes ------------------------- - -* ``UniqueConstraint``, ``Index``, ``PrimaryKeyConstraint`` - all accept lists of column names or column objects as - arguments. - -* The ``use_alter`` flag on ``ForeignKey`` is now a shortcut - option for operations that can be hand-constructed using - the ``DDL()`` event system. A side effect of this refactor - is that ``ForeignKeyConstraint`` objects with - ``use_alter=True`` will *not* be emitted on SQLite, which - does not support ALTER for foreign keys. This has no - effect on SQLite's behavior since SQLite does not actually - honor FOREIGN KEY constraints. - -* ``Table.primary_key`` is not assignable - use - ``table.append_constraint(PrimaryKeyConstraint(...))`` - -* A ``Column`` definition with a ``ForeignKey`` and no type, - e.g. ``Column(name, ForeignKey(sometable.c.somecol))`` - used to get the type of the referenced column. Now support - for that automatic type inference is partial and may not - work in all cases. - -Logging opened up -================= - -At the expense of a few extra method calls here and there, -you can set log levels for INFO and DEBUG after an engine, -pool, or mapper has been created, and logging will commence. -The ``isEnabledFor(INFO)`` method is now called -per-``Connection`` and ``isEnabledFor(DEBUG)`` -per-``ResultProxy`` if already enabled on the parent -connection. Pool logging sends to ``log.info()`` and -``log.debug()`` with no check - note that pool -checkout/checkin is typically once per transaction. - -Reflection/Inspector API -======================== - -The reflection system, which allows reflection of table -columns via ``Table('sometable', metadata, autoload=True)`` -has been opened up into its own fine-grained API, which -allows direct inspection of database elements such as -tables, columns, constraints, indexes, and more. This API -expresses return values as simple lists of strings, -dictionaries, and ``TypeEngine`` objects. The internals of -``autoload=True`` now build upon this system such that the -translation of raw database information into -``sqlalchemy.schema`` constructs is centralized and the -contract of individual dialects greatly simplified, vastly -reducing bugs and inconsistencies across different backends. - -To use an inspector: - -:: - - from sqlalchemy.engine.reflection import Inspector - - insp = Inspector.from_engine(my_engine) - - print(insp.get_schema_names()) - -the ``from_engine()`` method will in some cases provide a -backend-specific inspector with additional capabilities, -such as that of PostgreSQL which provides a -``get_table_oid()`` method: - -:: - - - my_engine = create_engine("postgresql://...") - pg_insp = Inspector.from_engine(my_engine) - - print(pg_insp.get_table_oid("my_table")) - -RETURNING Support -================= - -The ``insert()``, ``update()`` and ``delete()`` constructs -now support a ``returning()`` method, which corresponds to -the SQL RETURNING clause as supported by PostgreSQL, Oracle, -MS-SQL, and Firebird. It is not supported for any other -backend at this time. - -Given a list of column expressions in the same manner as -that of a ``select()`` construct, the values of these -columns will be returned as a regular result set: - -:: - - - result = connection.execute( - table.insert().values(data="some data").returning(table.c.id, table.c.timestamp) - ) - row = result.first() - print("ID:", row["id"], "Timestamp:", row["timestamp"]) - -The implementation of RETURNING across the four supported -backends varies wildly, in the case of Oracle requiring an -intricate usage of OUT parameters which are re-routed into a -"mock" result set, and in the case of MS-SQL using an -awkward SQL syntax. The usage of RETURNING is subject to -limitations: - -* it does not work for any "executemany()" style of - execution. This is a limitation of all supported DBAPIs. - -* Some backends, such as Oracle, only support RETURNING that - returns a single row - this includes UPDATE and DELETE - statements, meaning the update() or delete() construct - must match only a single row, or an error is raised (by - Oracle, not SQLAlchemy). - -RETURNING is also used automatically by SQLAlchemy, when -available and when not otherwise specified by an explicit -``returning()`` call, to fetch the value of newly generated -primary key values for single-row INSERT statements. This -means there's no more "SELECT nextval(sequence)" pre- -execution for insert statements where the primary key value -is required. Truth be told, implicit RETURNING feature -does incur more method overhead than the old "select -nextval()" system, which used a quick and dirty -cursor.execute() to get at the sequence value, and in the -case of Oracle requires additional binding of out -parameters. So if method/protocol overhead is proving to be -more expensive than additional database round trips, the -feature can be disabled by specifying -``implicit_returning=False`` to ``create_engine()``. - -Type System Changes -=================== - -New Architecture ----------------- - -The type system has been completely reworked behind the -scenes to provide two goals: - -* Separate the handling of bind parameters and result row - values, typically a DBAPI requirement, from the SQL - specification of the type itself, which is a database - requirement. This is consistent with the overall dialect - refactor that separates database SQL behavior from DBAPI. - -* Establish a clear and consistent contract for generating - DDL from a ``TypeEngine`` object and for constructing - ``TypeEngine`` objects based on column reflection. - -Highlights of these changes include: - -* The construction of types within dialects has been totally - overhauled. Dialects now define publicly available types - as UPPERCASE names exclusively, and internal - implementation types using underscore identifiers (i.e. - are private). The system by which types are expressed in - SQL and DDL has been moved to the compiler system. This - has the effect that there are much fewer type objects - within most dialects. A detailed document on this - architecture for dialect authors is in [source:/lib/sqlalc - hemy/dialects/type_migration_guidelines.txt]. - -* Reflection of types now returns the exact UPPERCASE type - within types.py, or the UPPERCASE type within the dialect - itself if the type is not a standard SQL type. This means - reflection now returns more accurate information about - reflected types. - -* User defined types that subclass ``TypeEngine`` and wish - to provide ``get_col_spec()`` should now subclass - ``UserDefinedType``. - -* The ``result_processor()`` method on all type classes now - accepts an additional argument ``coltype``. This is the - DBAPI type object attached to cursor.description, and - should be used when applicable to make better decisions on - what kind of result-processing callable should be - returned. Ideally result processor functions would never - need to use ``isinstance()``, which is an expensive call - at this level. - -Native Unicode Mode -------------------- - -As more DBAPIs support returning Python unicode objects -directly, the base dialect now performs a check upon the -first connection which establishes whether or not the DBAPI -returns a Python unicode object for a basic select of a -VARCHAR value. If so, the ``String`` type and all -subclasses (i.e. ``Text``, ``Unicode``, etc.) will skip the -"unicode" check/conversion step when result rows are -received. This offers a dramatic performance increase for -large result sets. The "unicode mode" currently is known to -work with: - -* sqlite3 / pysqlite - - -* psycopg2 - SQLA 0.6 now uses the "UNICODE" type extension - by default on each psycopg2 connection object - -* pg8000 - - -* cx_oracle (we use an output processor - nice feature !) - - -Other types may choose to disable unicode processing as -needed, such as the ``NVARCHAR`` type when used with MS-SQL. - -In particular, if porting an application based on a DBAPI -that formerly returned non-unicode strings, the "native -unicode" mode has a plainly different default behavior - -columns that are declared as ``String`` or ``VARCHAR`` now -return unicode by default whereas they would return strings -before. This can break code which expects non-unicode -strings. The psycopg2 "native unicode" mode can be -disabled by passing ``use_native_unicode=False`` to -``create_engine()``. - -A more general solution for string columns that explicitly -do not want a unicode object is to use a ``TypeDecorator`` -that converts unicode back to utf-8, or whatever is desired: - -:: - - class UTF8Encoded(TypeDecorator): - """Unicode type which coerces to utf-8.""" - - impl = sa.VARCHAR - - def process_result_value(self, value, dialect): - if isinstance(value, unicode): - value = value.encode("utf-8") - return value - -Note that the ``assert_unicode`` flag is now deprecated. -SQLAlchemy allows the DBAPI and backend database in use to -handle Unicode parameters when available, and does not add -operational overhead by checking the incoming type; modern -systems like sqlite and PostgreSQL will raise an encoding -error on their end if invalid data is passed. In those -cases where SQLAlchemy does need to coerce a bind parameter -from Python Unicode to an encoded string, or when the -Unicode type is used explicitly, a warning is raised if the -object is a bytestring. This warning can be suppressed or -converted to an exception using the Python warnings filter -documented at: https://docs.python.org/library/warnings.html - -Generic Enum Type ------------------ - -We now have an ``Enum`` in the ``types`` module. This is a -string type that is given a collection of "labels" which -constrain the possible values given to those labels. By -default, this type generates a ``VARCHAR`` using the size of -the largest label, and applies a CHECK constraint to the -table within the CREATE TABLE statement. When using MySQL, -the type by default uses MySQL's ENUM type, and when using -PostgreSQL the type will generate a user defined type using -``CREATE TYPE AS ENUM``. In order to create the -type using PostgreSQL, the ``name`` parameter must be -specified to the constructor. The type also accepts a -``native_enum=False`` option which will issue the -VARCHAR/CHECK strategy for all databases. Note that -PostgreSQL ENUM types currently don't work with pg8000 or -zxjdbc. - -Reflection Returns Dialect-Specific Types ------------------------------------------ - -Reflection now returns the most specific type possible from -the database. That is, if you create a table using -``String``, then reflect it back, the reflected column will -likely be ``VARCHAR``. For dialects that support a more -specific form of the type, that's what you'll get. So a -``Text`` type would come back as ``oracle.CLOB`` on Oracle, -a ``LargeBinary`` might be an ``mysql.MEDIUMBLOB`` etc. The -obvious advantage here is that reflection preserves as much -information possible from what the database had to say. - -Some applications that deal heavily in table metadata may -wish to compare types across reflected tables and/or non- -reflected tables. There's a semi-private accessor available -on ``TypeEngine`` called ``_type_affinity`` and an -associated comparison helper ``_compare_type_affinity``. -This accessor returns the "generic" ``types`` class which -the type corresponds to: - -:: - - >>> String(50)._compare_type_affinity(postgresql.VARCHAR(50)) - True - >>> Integer()._compare_type_affinity(mysql.REAL) - False - -Miscellaneous API Changes -------------------------- - -The usual "generic" types are still the general system in -use, i.e. ``String``, ``Float``, ``DateTime``. There's a -few changes there: - -* Types no longer make any guesses as to default parameters. - In particular, ``Numeric``, ``Float``, as well as - subclasses NUMERIC, FLOAT, DECIMAL don't generate any - length or scale unless specified. This also continues to - include the controversial ``String`` and ``VARCHAR`` types - (although MySQL dialect will pre-emptively raise when - asked to render VARCHAR with no length). No defaults are - assumed, and if they are used in a CREATE TABLE statement, - an error will be raised if the underlying database does - not allow non-lengthed versions of these types. - -* the ``Binary`` type has been renamed to ``LargeBinary``, - for BLOB/BYTEA/similar types. For ``BINARY`` and - ``VARBINARY``, those are present directly as - ``types.BINARY``, ``types.VARBINARY``, as well as in the - MySQL and MS-SQL dialects. - -* ``PickleType`` now uses == for comparison of values when - mutable=True, unless the "comparator" argument with a - comparison function is specified to the type. If you are - pickling a custom object you should implement an - ``__eq__()`` method so that value-based comparisons are - accurate. - -* The default "precision" and "scale" arguments of Numeric - and Float have been removed and now default to None. - NUMERIC and FLOAT will be rendered with no numeric - arguments by default unless these values are provided. - -* DATE, TIME and DATETIME types on SQLite can now take - optional "storage_format" and "regexp" argument. - "storage_format" can be used to store those types using a - custom string format. "regexp" allows to use a custom - regular expression to match string values from the - database. - -* ``__legacy_microseconds__`` on SQLite ``Time`` and - ``DateTime`` types is not supported anymore. You should - use the new "storage_format" argument instead. - -* ``DateTime`` types on SQLite now use by a default a - stricter regular expression to match strings from the - database. Use the new "regexp" argument if you are using - data stored in a legacy format. - -ORM Changes -=========== - -Upgrading an ORM application from 0.5 to 0.6 should require -little to no changes, as the ORM's behavior remains almost -identical. There are some default argument and name -changes, and some loading behaviors have been improved. - -New Unit of Work ----------------- - -The internals for the unit of work, primarily -``topological.py`` and ``unitofwork.py``, have been -completely rewritten and are vastly simplified. This -should have no impact on usage, as all existing behavior -during flush has been maintained exactly (or at least, as -far as it is exercised by our testsuite and the handful of -production environments which have tested it heavily). The -performance of flush() now uses 20-30% fewer method calls -and should also use less memory. The intent and flow of the -source code should now be reasonably easy to follow, and the -architecture of the flush is fairly open-ended at this -point, creating room for potential new areas of -sophistication. The flush process no longer has any -reliance on recursion so flush plans of arbitrary size and -complexity can be flushed. Additionally, the mapper's -"save" process, which issues INSERT and UPDATE statements, -now caches the "compiled" form of the two statements so that -callcounts are further dramatically reduced with very large -flushes. - -Any changes in behavior observed with flush versus earlier -versions of 0.6 or 0.5 should be reported to us ASAP - we'll -make sure no functionality is lost. - -Changes to ``query.update()`` and ``query.delete()`` ----------------------------------------------------- - -* the 'expire' option on query.update() has been renamed to - 'fetch', thus matching that of query.delete() - -* ``query.update()`` and ``query.delete()`` both default to - 'evaluate' for the synchronize strategy. - -* the 'synchronize' strategy for update() and delete() - raises an error on failure. There is no implicit fallback - onto "fetch". Failure of evaluation is based on the - structure of criteria, so success/failure is deterministic - based on code structure. - -``relation()`` is officially named ``relationship()`` ------------------------------------------------------ - -This to solve the long running issue that "relation" means a -"table or derived table" in relational algebra terms. The -``relation()`` name, which is less typing, will hang around -for the foreseeable future so this change should be entirely -painless. - -Subquery eager loading ----------------------- - -A new kind of eager loading is added called "subquery" -loading. This is a load that emits a second SQL query -immediately after the first which loads full collections for -all the parents in the first query, joining upwards to the -parent using INNER JOIN. Subquery loading is used similarly -to the current joined-eager loading, using the -```subqueryload()```` and ````subqueryload_all()```` options -as well as the ````lazy='subquery'```` setting on -````relationship()```. The subquery load is usually much -more efficient for loading many larger collections as it -uses INNER JOIN unconditionally and also doesn't re-load -parent rows. - -```eagerload()````, ````eagerload_all()```` is now ````joinedload()````, ````joinedload_all()``` ------------------------------------------------------------------------------------------------- - -To make room for the new subquery load feature, the existing -```eagerload()````/````eagerload_all()```` options are now -superseded by ````joinedload()```` and -````joinedload_all()````. The old names will hang around -for the foreseeable future just like ````relation()```. - -```lazy=False|None|True|'dynamic'```` now accepts ````lazy='noload'|'joined'|'subquery'|'select'|'dynamic'``` -------------------------------------------------------------------------------------------------------------- - -Continuing on the theme of loader strategies opened up, the -standard keywords for the ```lazy```` option on -````relationship()```` are now ````select```` for lazy -loading (via a SELECT issued on attribute access), -````joined```` for joined-eager loading, ````subquery```` -for subquery-eager loading, ````noload```` for no loading -should occur, and ````dynamic```` for a "dynamic" -relationship. The old ````True````, ````False````, -````None``` arguments are still accepted with the identical -behavior as before. - -innerjoin=True on relation, joinedload --------------------------------------- - -Joined-eagerly loaded scalars and collections can now be -instructed to use INNER JOIN instead of OUTER JOIN. On -PostgreSQL this is observed to provide a 300-600% speedup on -some queries. Set this flag for any many-to-one which is -on a NOT NULLable foreign key, and similarly for any -collection where related items are guaranteed to exist. - -At mapper level: - -:: - - mapper(Child, child) - mapper( - Parent, - parent, - properties={"child": relationship(Child, lazy="joined", innerjoin=True)}, - ) - -At query time level: - -:: - - session.query(Parent).options(joinedload(Parent.child, innerjoin=True)).all() - -The ``innerjoin=True`` flag at the ``relationship()`` level -will also take effect for any ``joinedload()`` option which -does not override the value. - -Many-to-one Enhancements ------------------------- - -* many-to-one relations now fire off a lazyload in fewer - cases, including in most cases will not fetch the "old" - value when a new one is replaced. - -* many-to-one relation to a joined-table subclass now uses - get() for a simple load (known as the "use_get" - condition), i.e. ``Related``->``Sub(Base)``, without the - need to redefine the primaryjoin condition in terms of the - base table. [ticket:1186] - -* specifying a foreign key with a declarative column, i.e. - ``ForeignKey(MyRelatedClass.id)`` doesn't break the - "use_get" condition from taking place [ticket:1492] - -* relationship(), joinedload(), and joinedload_all() now - feature an option called "innerjoin". Specify ``True`` or - ``False`` to control whether an eager join is constructed - as an INNER or OUTER join. Default is ``False`` as always. - The mapper options will override whichever setting is - specified on relationship(). Should generally be set for - many-to-one, not nullable foreign key relations to allow - improved join performance. [ticket:1544] - -* the behavior of joined eager loading such that the main - query is wrapped in a subquery when LIMIT/OFFSET are - present now makes an exception for the case when all eager - loads are many-to-one joins. In those cases, the eager - joins are against the parent table directly along with the - limit/offset without the extra overhead of a subquery, - since a many-to-one join does not add rows to the result. - - For example, in 0.5 this query: - - :: - - session.query(Address).options(eagerload(Address.user)).limit(10) - - would produce SQL like: - - .. sourcecode:: sql - - SELECT * FROM - (SELECT * FROM addresses LIMIT 10) AS anon_1 - LEFT OUTER JOIN users AS users_1 ON users_1.id = anon_1.addresses_user_id - - This because the presence of any eager loaders suggests - that some or all of them may relate to multi-row - collections, which would necessitate wrapping any kind of - rowcount-sensitive modifiers like LIMIT inside of a - subquery. - - In 0.6, that logic is more sensitive and can detect if all - eager loaders represent many-to-ones, in which case the - eager joins don't affect the rowcount: - - .. sourcecode:: sql - - SELECT * FROM addresses LEFT OUTER JOIN users AS users_1 ON users_1.id = addresses.user_id LIMIT 10 - -Mutable Primary Keys with Joined Table Inheritance --------------------------------------------------- - -A joined table inheritance config where the child table has -a PK that foreign keys to the parent PK can now be updated -on a CASCADE-capable database like PostgreSQL. -``mapper()`` now has an option ``passive_updates=True`` -which indicates this foreign key is updated automatically. -If on a non-cascading database like SQLite or MySQL/MyISAM, -set this flag to ``False``. A future feature enhancement -will try to get this flag to be auto-configuring based on -dialect/table style in use. - -Beaker Caching --------------- - -A promising new example of Beaker integration is in -``examples/beaker_caching``. This is a straightforward -recipe which applies a Beaker cache within the result- -generation engine of ``Query``. Cache parameters are -provided via ``query.options()``, and allows full control -over the contents of the cache. SQLAlchemy 0.6 includes -improvements to the ``Session.merge()`` method to support -this and similar recipes, as well as to provide -significantly improved performance in most scenarios. - -Other Changes -------------- - -* the "row tuple" object returned by ``Query`` when multiple - column/entities are selected is now picklable as well as - higher performing. - -* ``query.join()`` has been reworked to provide more - consistent behavior and more flexibility (includes - [ticket:1537]) - -* ``query.select_from()`` accepts multiple clauses to - produce multiple comma separated entries within the FROM - clause. Useful when selecting from multiple-homed join() - clauses. - -* the "dont_load=True" flag on ``Session.merge()`` is - deprecated and is now "load=False". - -* added "make_transient()" helper function which transforms - a persistent/ detached instance into a transient one (i.e. - deletes the instance_key and removes from any session.) - [ticket:1052] - -* the allow_null_pks flag on mapper() is deprecated and has - been renamed to allow_partial_pks. It is turned "on" by - default. This means that a row which has a non-null value - for any of its primary key columns will be considered an - identity. The need for this scenario typically only occurs - when mapping to an outer join. When set to False, a PK - that has NULLs in it will not be considered a primary key - - in particular this means a result row will come back as - None (or not be filled into a collection), and new in 0.6 - also indicates that session.merge() won't issue a round - trip to the database for such a PK value. [ticket:1680] - -* the mechanics of "backref" have been fully merged into the - finer grained "back_populates" system, and take place - entirely within the ``_generate_backref()`` method of - ``RelationProperty``. This makes the initialization - procedure of ``RelationProperty`` simpler and allows - easier propagation of settings (such as from subclasses of - ``RelationProperty``) into the reverse reference. The - internal ``BackRef()`` is gone and ``backref()`` returns a - plain tuple that is understood by ``RelationProperty``. - -* the keys attribute of ``ResultProxy`` is now a method, so - references to it (``result.keys``) must be changed to - method invocations (``result.keys()``) - -* ``ResultProxy.last_inserted_ids`` is now deprecated, use - ``ResultProxy.inserted_primary_key`` instead. - -Deprecated/Removed ORM Elements -------------------------------- - -Most elements that were deprecated throughout 0.5 and raised -deprecation warnings have been removed (with a few -exceptions). All elements that were marked "pending -deprecation" are now deprecated and will raise a warning -upon use. - -* 'transactional' flag on sessionmaker() and others is - removed. Use 'autocommit=True' to indicate - 'transactional=False'. - -* 'polymorphic_fetch' argument on mapper() is removed. - Loading can be controlled using the 'with_polymorphic' - option. - -* 'select_table' argument on mapper() is removed. Use - 'with_polymorphic=("*", )' for this - functionality. - -* 'proxy' argument on synonym() is removed. This flag did - nothing throughout 0.5, as the "proxy generation" - behavior is now automatic. - -* Passing a single list of elements to joinedload(), - joinedload_all(), contains_eager(), lazyload(), defer(), - and undefer() instead of multiple positional \*args is - deprecated. - -* Passing a single list of elements to query.order_by(), - query.group_by(), query.join(), or query.outerjoin() - instead of multiple positional \*args is deprecated. - -* ``query.iterate_instances()`` is removed. Use - ``query.instances()``. - -* ``Query.query_from_parent()`` is removed. Use the - sqlalchemy.orm.with_parent() function to produce a - "parent" clause, or alternatively ``query.with_parent()``. - -* ``query._from_self()`` is removed, use - ``query.from_self()`` instead. - -* the "comparator" argument to composite() is removed. Use - "comparator_factory". - -* ``RelationProperty._get_join()`` is removed. - - -* the 'echo_uow' flag on Session is removed. Use logging - on the "sqlalchemy.orm.unitofwork" name. - -* ``session.clear()`` is removed. use - ``session.expunge_all()``. - -* ``session.save()``, ``session.update()``, - ``session.save_or_update()`` are removed. Use - ``session.add()`` and ``session.add_all()``. - -* the "objects" flag on session.flush() remains deprecated. - - -* the "dont_load=True" flag on session.merge() is deprecated - in favor of "load=False". - -* ``ScopedSession.mapper`` remains deprecated. See the - usage recipe at https://www.sqlalchemy.org/trac/wiki/Usag - eRecipes/SessionAwareMapper - -* passing an ``InstanceState`` (internal SQLAlchemy state - object) to ``attributes.init_collection()`` or - ``attributes.get_history()`` is deprecated. These - functions are public API and normally expect a regular - mapped object instance. - -* the 'engine' parameter to ``declarative_base()`` is - removed. Use the 'bind' keyword argument. - -Extensions -========== - -SQLSoup -------- - -SQLSoup has been modernized and updated to reflect common -0.5/0.6 capabilities, including well defined session -integration. Please read the new docs at [https://www.sqlalc -hemy.org/docs/06/reference/ext/sqlsoup.html]. - -Declarative ------------ - -The ``DeclarativeMeta`` (default metaclass for -``declarative_base``) previously allowed subclasses to -modify ``dict_`` to add class attributes (e.g. columns). -This no longer works, the ``DeclarativeMeta`` constructor -now ignores ``dict_``. Instead, the class attributes should -be assigned directly, e.g. ``cls.id=Column(...)``, or the -`MixIn class `_ approach should be used -instead of the metaclass approach. - diff --git a/doc/build/changelog/migration_07.rst b/doc/build/changelog/migration_07.rst deleted file mode 100644 index 19716ad3c4..0000000000 --- a/doc/build/changelog/migration_07.rst +++ /dev/null @@ -1,1369 +0,0 @@ -============================= -What's New in SQLAlchemy 0.7? -============================= - -.. admonition:: About this Document - - This document describes changes between SQLAlchemy version 0.6, - last released May 5, 2012, and SQLAlchemy version 0.7, - undergoing maintenance releases as of October, 2012. - - Document date: July 27, 2011 - -Introduction -============ - -This guide introduces what's new in SQLAlchemy version 0.7, -and also documents changes which affect users migrating -their applications from the 0.6 series of SQLAlchemy to 0.7. - -To as great a degree as possible, changes are made in such a -way as to not break compatibility with applications built -for 0.6. The changes that are necessarily not backwards -compatible are very few, and all but one, the change to -mutable attribute defaults, should affect an exceedingly -small portion of applications - many of the changes regard -non-public APIs and undocumented hacks some users may have -been attempting to use. - -A second, even smaller class of non-backwards-compatible -changes is also documented. This class of change regards -those features and behaviors that have been deprecated at -least since version 0.5 and have been raising warnings since -their deprecation. These changes would only affect -applications that are still using 0.4- or early 0.5-style -APIs. As the project matures, we have fewer and fewer of -these kinds of changes with 0.x level releases, which is a -product of our API having ever fewer features that are less -than ideal for the use cases they were meant to solve. - -An array of existing functionalities have been superseded in -SQLAlchemy 0.7. There's not much difference between the -terms "superseded" and "deprecated", except that the former -has a much weaker suggestion of the old feature would ever -be removed. In 0.7, features like ``synonym`` and -``comparable_property``, as well as all the ``Extension`` -and other event classes, have been superseded. But these -"superseded" features have been re-implemented such that -their implementations live mostly outside of core ORM code, -so their continued "hanging around" doesn't impact -SQLAlchemy's ability to further streamline and refine its -internals, and we expect them to remain within the API for -the foreseeable future. - -New Features -============ - -New Event System ----------------- - -SQLAlchemy started early with the ``MapperExtension`` class, -which provided hooks into the persistence cycle of mappers. -As SQLAlchemy quickly became more componentized, pushing -mappers into a more focused configurational role, many more -"extension", "listener", and "proxy" classes popped up to -solve various activity-interception use cases in an ad-hoc -fashion. Part of this was driven by the divergence of -activities; ``ConnectionProxy`` objects wanted to provide a -system of rewriting statements and parameters; -``AttributeExtension`` provided a system of replacing -incoming values, and ``DDL`` objects had events that could -be switched off of dialect-sensitive callables. - -0.7 re-implements virtually all of these plugin points with -a new, unified approach, which retains all the -functionalities of the different systems, provides more -flexibility and less boilerplate, performs better, and -eliminates the need to learn radically different APIs for -each event subsystem. The pre-existing classes -``MapperExtension``, ``SessionExtension``, -``AttributeExtension``, ``ConnectionProxy``, -``PoolListener`` as well as the ``DDLElement.execute_at`` -method are deprecated and now implemented in terms of the -new system - these APIs remain fully functional and are -expected to remain in place for the foreseeable future. - -The new approach uses named events and user-defined -callables to associate activities with events. The API's -look and feel was driven by such diverse sources as JQuery, -Blinker, and Hibernate, and was also modified further on -several occasions during conferences with dozens of users on -Twitter, which appears to have a much higher response rate -than the mailing list for such questions. - -It also features an open-ended system of target -specification that allows events to be associated with API -classes, such as for all ``Session`` or ``Engine`` objects, -with specific instances of API classes, such as for a -specific ``Pool`` or ``Mapper``, as well as for related -objects like a user- defined class that's mapped, or -something as specific as a certain attribute on instances of -a particular subclass of a mapped parent class. Individual -listener subsystems can apply wrappers to incoming user- -defined listener functions which modify how they are called -- an mapper event can receive either the instance of the -object being operated upon, or its underlying -``InstanceState`` object. An attribute event can opt whether -or not to have the responsibility of returning a new value. - -Several systems now build upon the new event API, including -the new "mutable attributes" API as well as composite -attributes. The greater emphasis on events has also led to -the introduction of a handful of new events, including -attribute expiration and refresh operations, pickle -loads/dumps operations, completed mapper construction -operations. - -.. seealso:: - - :ref:`event_toplevel` - -:ticket:`1902` - -Hybrid Attributes, implements/supersedes synonym(), comparable_property() -------------------------------------------------------------------------- - -The "derived attributes" example has now been turned into an -official extension. The typical use case for ``synonym()`` -is to provide descriptor access to a mapped column; the use -case for ``comparable_property()`` is to be able to return a -``PropComparator`` from any descriptor. In practice, the -approach of "derived" is easier to use, more extensible, is -implemented in a few dozen lines of pure Python with almost -no imports, and doesn't require the ORM core to even be -aware of it. The feature is now known as the "Hybrid -Attributes" extension. - -``synonym()`` and ``comparable_property()`` are still part -of the ORM, though their implementations have been moved -outwards, building on an approach that is similar to that of -the hybrid extension, so that the core ORM -mapper/query/property modules aren't really aware of them -otherwise. - -.. seealso:: - - :ref:`hybrids_toplevel` - -:ticket:`1903` - -Speed Enhancements ------------------- - -As is customary with all major SQLA releases, a wide pass -through the internals to reduce overhead and callcounts has -been made which further reduces the work needed in common -scenarios. Highlights of this release include: - -* The flush process will now bundle INSERT statements into - batches fed to ``cursor.executemany()``, for rows where - the primary key is already present. In particular this - usually applies to the "child" table on a joined table - inheritance configuration, meaning the number of calls to - ``cursor.execute`` for a large bulk insert of joined- - table objects can be cut in half, allowing native DBAPI - optimizations to take place for those statements passed - to ``cursor.executemany()`` (such as re-using a prepared - statement). - -* The codepath invoked when accessing a many-to-one - reference to a related object that's already loaded has - been greatly simplified. The identity map is checked - directly without the need to generate a new ``Query`` - object first, which is expensive in the context of - thousands of in-memory many-to-ones being accessed. The - usage of constructed-per-call "loader" objects is also no - longer used for the majority of lazy attribute loads. - -* The rewrite of composites allows a shorter codepath when - mapper internals access mapped attributes within a - flush. - -* New inlined attribute access functions replace the - previous usage of "history" when the "save-update" and - other cascade operations need to cascade among the full - scope of datamembers associated with an attribute. This - reduces the overhead of generating a new ``History`` - object for this speed-critical operation. - -* The internals of the ``ExecutionContext``, the object - corresponding to a statement execution, have been - inlined and simplified. - -* The ``bind_processor()`` and ``result_processor()`` - callables generated by types for each statement - execution are now cached (carefully, so as to avoid memory - leaks for ad-hoc types and dialects) for the lifespan of - that type, further reducing per-statement call overhead. - -* The collection of "bind processors" for a particular - ``Compiled`` instance of a statement is also cached on - the ``Compiled`` object, taking further advantage of the - "compiled cache" used by the flush process to re-use the - same compiled form of INSERT, UPDATE, DELETE statements. - -A demonstration of callcount reduction including a sample -benchmark script is at -https://techspot.zzzeek.org/2010/12/12/a-tale-of-three- -profiles/ - -Composites Rewritten --------------------- - -The "composite" feature has been rewritten, like -``synonym()`` and ``comparable_property()``, to use a -lighter weight implementation based on descriptors and -events, rather than building into the ORM internals. This -allowed the removal of some latency from the mapper/unit of -work internals, and simplifies the workings of composite. -The composite attribute now no longer conceals the -underlying columns it builds upon, which now remain as -regular attributes. Composites can also act as a proxy for -``relationship()`` as well as ``Column()`` attributes. - -The major backwards-incompatible change of composites is -that they no longer use the ``mutable=True`` system to -detect in-place mutations. Please use the `Mutation -Tracking `_ extension to establish in-place change events -to existing composite usage. - -.. seealso:: - - :ref:`mapper_composite` - - :ref:`mutable_toplevel` - -:ticket:`2008` :ticket:`2024` - -More succinct form of query.join(target, onclause) --------------------------------------------------- - -The default method of issuing ``query.join()`` to a target -with an explicit onclause is now: - -:: - - query.join(SomeClass, SomeClass.id == ParentClass.some_id) - -In 0.6, this usage was considered to be an error, because -``join()`` accepts multiple arguments corresponding to -multiple JOIN clauses - the two-argument form needed to be -in a tuple to disambiguate between single-argument and two- -argument join targets. In the middle of 0.6 we added -detection and an error message for this specific calling -style, since it was so common. In 0.7, since we are -detecting the exact pattern anyway, and since having to type -out a tuple for no reason is extremely annoying, the non- -tuple method now becomes the "normal" way to do it. The -"multiple JOIN" use case is exceedingly rare compared to the -single join case, and multiple joins these days are more -clearly represented by multiple calls to ``join()``. - -The tuple form will remain for backwards compatibility. - -Note that all the other forms of ``query.join()`` remain -unchanged: - -:: - - query.join(MyClass.somerelation) - query.join("somerelation") - query.join(MyTarget) - # ... etc - -`Querying with Joins -`_ - -:ticket:`1923` - -.. _07_migration_mutation_extension: - -Mutation event extension, supersedes "mutable=True" ---------------------------------------------------- - -A new extension, :ref:`mutable_toplevel`, provides a -mechanism by which user-defined datatypes can provide change -events back to the owning parent or parents. The extension -includes an approach for scalar database values, such as -those managed by :class:`.PickleType`, ``postgresql.ARRAY``, or -other custom ``MutableType`` classes, as well as an approach -for ORM "composites", those configured using :func:`~.sqlalchemy.orm.composite`. - -.. seealso:: - - :ref:`mutable_toplevel` - -NULLS FIRST / NULLS LAST operators ----------------------------------- - -These are implemented as an extension to the ``asc()`` and -``desc()`` operators, called ``nullsfirst()`` and -``nullslast()``. - -.. seealso:: - - :func:`.nullsfirst` - - :func:`.nullslast` - -:ticket:`723` - -select.distinct(), query.distinct() accepts \*args for PostgreSQL DISTINCT ON ------------------------------------------------------------------------------ - -This was already available by passing a list of expressions -to the ``distinct`` keyword argument of ``select()``, the -``distinct()`` method of ``select()`` and ``Query`` now -accept positional arguments which are rendered as DISTINCT -ON when a PostgreSQL backend is used. - -`distinct() `_ - -`Query.distinct() `_ - -:ticket:`1069` - -``Index()`` can be placed inline inside of ``Table``, ``__table_args__`` ------------------------------------------------------------------------- - -The Index() construct can be created inline with a Table -definition, using strings as column names, as an alternative -to the creation of the index outside of the Table. That is: - -:: - - Table( - "mytable", - metadata, - Column("id", Integer, primary_key=True), - Column("name", String(50), nullable=False), - Index("idx_name", "name"), - ) - -The primary rationale here is for the benefit of declarative -``__table_args__``, particularly when used with mixins: - -:: - - class HasNameMixin(object): - name = Column("name", String(50), nullable=False) - - @declared_attr - def __table_args__(cls): - return (Index("name"), {}) - - - class User(HasNameMixin, Base): - __tablename__ = "user" - id = Column("id", Integer, primary_key=True) - -`Indexes `_ - -Window Function SQL Construct ------------------------------ - -A "window function" provides to a statement information -about the result set as it's produced. This allows criteria -against various things like "row number", "rank" and so -forth. They are known to be supported at least by -PostgreSQL, SQL Server and Oracle, possibly others. - -The best introduction to window functions is on PostgreSQL's -site, where window functions have been supported since -version 8.4: - -https://www.postgresql.org/docs/current/static/tutorial-window.html - -SQLAlchemy provides a simple construct typically invoked via -an existing function clause, using the ``over()`` method, -which accepts ``order_by`` and ``partition_by`` keyword -arguments. Below we replicate the first example in PG's -tutorial: - -:: - - from sqlalchemy.sql import table, column, select, func - - empsalary = table("empsalary", column("depname"), column("empno"), column("salary")) - - s = select( - [ - empsalary, - func.avg(empsalary.c.salary) - .over(partition_by=empsalary.c.depname) - .label("avg"), - ] - ) - - print(s) - -SQL: - -.. sourcecode:: sql - - SELECT empsalary.depname, empsalary.empno, empsalary.salary, - avg(empsalary.salary) OVER (PARTITION BY empsalary.depname) AS avg - FROM empsalary - -`sqlalchemy.sql.expression.over `_ - -:ticket:`1844` - -execution_options() on Connection accepts "isolation_level" argument --------------------------------------------------------------------- - -This sets the transaction isolation level for a single -``Connection``, until that ``Connection`` is closed and its -underlying DBAPI resource returned to the connection pool, -upon which the isolation level is reset back to the default. -The default isolation level is set using the -``isolation_level`` argument to ``create_engine()``. - -Transaction isolation support is currently only supported by -the PostgreSQL and SQLite backends. - -`execution_options() `_ - -:ticket:`2001` - -``TypeDecorator`` works with integer primary key columns --------------------------------------------------------- - -A ``TypeDecorator`` which extends the behavior of -``Integer`` can be used with a primary key column. The -"autoincrement" feature of ``Column`` will now recognize -that the underlying database column is still an integer so -that lastrowid mechanisms continue to function. The -``TypeDecorator`` itself will have its result value -processor applied to newly generated primary keys, including -those received by the DBAPI ``cursor.lastrowid`` accessor. - -:ticket:`2005` :ticket:`2006` - -``TypeDecorator`` is present in the "sqlalchemy" import space -------------------------------------------------------------- - -No longer need to import this from ``sqlalchemy.types``, -it's now mirrored in ``sqlalchemy``. - -New Dialects ------------- - -Dialects have been added: - -* a MySQLdb driver for the Drizzle database: - - - `Drizzle `_ - -* support for the pymysql DBAPI: - - - `pymsql Notes - `_ - -* psycopg2 now works with Python 3 - - -Behavioral Changes (Backwards Compatible) -========================================= - -C Extensions Build by Default ------------------------------ - -This is as of 0.7b4. The exts will build if cPython 2.xx -is detected. If the build fails, such as on a windows -install, that condition is caught and the non-C install -proceeds. The C exts won't build if Python 3 or PyPy is -used. - -Query.count() simplified, should work virtually always ------------------------------------------------------- - -The very old guesswork which occurred within -``Query.count()`` has been modernized to use -``.from_self()``. That is, ``query.count()`` is now -equivalent to: - -:: - - query.from_self(func.count(literal_column("1"))).scalar() - -Previously, internal logic attempted to rewrite the columns -clause of the query itself, and upon detection of a -"subquery" condition, such as a column-based query that -might have aggregates in it, or a query with DISTINCT, would -go through a convoluted process of rewriting the columns -clause. This logic failed in complex conditions, -particularly those involving joined table inheritance, and -was long obsolete by the more comprehensive ``.from_self()`` -call. - -The SQL emitted by ``query.count()`` is now always of the -form: - -.. sourcecode:: sql - - SELECT count(1) AS count_1 FROM ( - SELECT user.id AS user_id, user.name AS user_name from user - ) AS anon_1 - -that is, the original query is preserved entirely inside of -a subquery, with no more guessing as to how count should be -applied. - -:ticket:`2093` - -To emit a non-subquery form of count() -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -MySQL users have already reported that the MyISAM engine not -surprisingly falls over completely with this simple change. -Note that for a simple ``count()`` that optimizes for DBs -that can't handle simple subqueries, ``func.count()`` should -be used: - -:: - - from sqlalchemy import func - - session.query(func.count(MyClass.id)).scalar() - -or for ``count(*)``: - -:: - - from sqlalchemy import func, literal_column - - session.query(func.count(literal_column("*"))).select_from(MyClass).scalar() - -LIMIT/OFFSET clauses now use bind parameters --------------------------------------------- - -The LIMIT and OFFSET clauses, or their backend equivalents -(i.e. TOP, ROW NUMBER OVER, etc.), use bind parameters for -the actual values, for all backends which support it (most -except for Sybase). This allows better query optimizer -performance as the textual string for multiple statements -with differing LIMIT/OFFSET are now identical. - -:ticket:`805` - -Logging enhancements --------------------- - -Vinay Sajip has provided a patch to our logging system such -that the "hex string" embedded in logging statements for -engines and pools is no longer needed to allow the ``echo`` -flag to work correctly. A new system that uses filtered -logging objects allows us to maintain our current behavior -of ``echo`` being local to individual engines without the -need for additional identifying strings local to those -engines. - -:ticket:`1926` - -Simplified polymorphic_on assignment ------------------------------------- - -The population of the ``polymorphic_on`` column-mapped -attribute, when used in an inheritance scenario, now occurs -when the object is constructed, i.e. its ``__init__`` method -is called, using the init event. The attribute then behaves -the same as any other column-mapped attribute. Previously, -special logic would fire off during flush to populate this -column, which prevented any user code from modifying its -behavior. The new approach improves upon this in three -ways: 1. the polymorphic identity is now present on the -object as soon as its constructed; 2. the polymorphic -identity can be changed by user code without any difference -in behavior from any other column-mapped attribute; 3. the -internals of the mapper during flush are simplified and no -longer need to make special checks for this column. - -:ticket:`1895` - -contains_eager() chains across multiple paths (i.e. "all()") ------------------------------------------------------------- - -The ```contains_eager()```` modifier now will chain itself -for a longer path without the need to emit individual -````contains_eager()``` calls. Instead of: - -:: - - session.query(A).options(contains_eager(A.b), contains_eager(A.b, B.c)) - -you can say: - -:: - - session.query(A).options(contains_eager(A.b, B.c)) - -:ticket:`2032` - -Flushing of orphans that have no parent is allowed --------------------------------------------------- - -We've had a long standing behavior that checks for a so- -called "orphan" during flush, that is, an object which is -associated with a ``relationship()`` that specifies "delete- -orphan" cascade, has been newly added to the session for an -INSERT, and no parent relationship has been established. -This check was added years ago to accommodate some test -cases which tested the orphan behavior for consistency. In -modern SQLA, this check is no longer needed on the Python -side. The equivalent behavior of the "orphan check" is -accomplished by making the foreign key reference to the -object's parent row NOT NULL, where the database does its -job of establishing data consistency in the same way SQLA -allows most other operations to do. If the object's parent -foreign key is nullable, then the row can be inserted. The -"orphan" behavior runs when the object was persisted with a -particular parent, and is then disassociated with that -parent, leading to a DELETE statement emitted for it. - -:ticket:`1912` - -Warnings generated when collection members, scalar referents not part of the flush ----------------------------------------------------------------------------------- - -Warnings are now emitted when related objects referenced via -a loaded ``relationship()`` on a parent object marked as -"dirty" are not present in the current ``Session``. - -The ``save-update`` cascade takes effect when objects are -added to the ``Session``, or when objects are first -associated with a parent, so that an object and everything -related to it are usually all present in the same -``Session``. However, if ``save-update`` cascade is -disabled for a particular ``relationship()``, then this -behavior does not occur, and the flush process does not try -to correct for it, instead staying consistent to the -configured cascade behavior. Previously, when such objects -were detected during the flush, they were silently skipped. -The new behavior is that a warning is emitted, for the -purposes of alerting to a situation that more often than not -is the source of unexpected behavior. - -:ticket:`1973` - -Setup no longer installs a Nose plugin --------------------------------------- - -Since we moved to nose we've used a plugin that installs via -setuptools, so that the ``nosetests`` script would -automatically run SQLA's plugin code, necessary for our -tests to have a full environment. In the middle of 0.6, we -realized that the import pattern here meant that Nose's -"coverage" plugin would break, since "coverage" requires -that it be started before any modules to be covered are -imported; so in the middle of 0.6 we made the situation -worse by adding a separate ``sqlalchemy-nose`` package to -the build to overcome this. - -In 0.7 we've done away with trying to get ``nosetests`` to -work automatically, since the SQLAlchemy module would -produce a large number of nose configuration options for all -usages of ``nosetests``, not just the SQLAlchemy unit tests -themselves, and the additional ``sqlalchemy-nose`` install -was an even worse idea, producing an extra package in Python -environments. The ``sqla_nose.py`` script in 0.7 is now -the only way to run the tests with nose. - -:ticket:`1949` - -Non-``Table``-derived constructs can be mapped ----------------------------------------------- - -A construct that isn't against any ``Table`` at all, like a -function, can be mapped. - -:: - - from sqlalchemy import select, func - from sqlalchemy.orm import mapper - - - class Subset(object): - pass - - - selectable = select(["x", "y", "z"]).select_from(func.some_db_function()).alias() - mapper(Subset, selectable, primary_key=[selectable.c.x]) - -:ticket:`1876` - -aliased() accepts ``FromClause`` elements ------------------------------------------ - -This is a convenience helper such that in the case a plain -``FromClause``, such as a ``select``, ``Table`` or ``join`` -is passed to the ``orm.aliased()`` construct, it passes -through to the ``.alias()`` method of that from construct -rather than constructing an ORM level ``AliasedClass``. - -:ticket:`2018` - -Session.connection(), Session.execute() accept 'bind' ------------------------------------------------------ - -This is to allow execute/connection operations to -participate in the open transaction of an engine explicitly. -It also allows custom subclasses of ``Session`` that -implement their own ``get_bind()`` method and arguments to -use those custom arguments with both the ``execute()`` and -``connection()`` methods equally. - -`Session.connection `_ -`Session.execute `_ - -:ticket:`1996` - -Standalone bind parameters in columns clause auto-labeled. ----------------------------------------------------------- - -Bind parameters present in the "columns clause" of a select -are now auto-labeled like other "anonymous" clauses, which -among other things allows their "type" to be meaningful when -the row is fetched, as in result row processors. - -SQLite - relative file paths are normalized through os.path.abspath() ---------------------------------------------------------------------- - -This so that a script that changes the current directory -will continue to target the same location as subsequent -SQLite connections are established. - -:ticket:`2036` - -MS-SQL - ``String``/``Unicode``/``VARCHAR``/``NVARCHAR``/``VARBINARY`` emit "max" for no length ------------------------------------------------------------------------------------------------ - -On the MS-SQL backend, the String/Unicode types, and their -counterparts VARCHAR/ NVARCHAR, as well as VARBINARY -(:ticket:`1833`) emit "max" as the length when no length is -specified. This makes it more compatible with PostgreSQL's -VARCHAR type which is similarly unbounded when no length -specified. SQL Server defaults the length on these types -to '1' when no length is specified. - -Behavioral Changes (Backwards Incompatible) -=========================================== - -Note again, aside from the default mutability change, most -of these changes are \*extremely minor* and will not affect -most users. - -``PickleType`` and ARRAY mutability turned off by default ---------------------------------------------------------- - -This change refers to the default behavior of the ORM when -mapping columns that have either the ``PickleType`` or -``postgresql.ARRAY`` datatypes. The ``mutable`` flag is now -set to ``False`` by default. If an existing application uses -these types and depends upon detection of in-place -mutations, the type object must be constructed with -``mutable=True`` to restore the 0.6 behavior: - -:: - - Table( - "mytable", - metadata, - # .... - Column("pickled_data", PickleType(mutable=True)), - ) - -The ``mutable=True`` flag is being phased out, in favor of -the new `Mutation Tracking `_ extension. This extension -provides a mechanism by which user-defined datatypes can -provide change events back to the owning parent or parents. - -The previous approach of using ``mutable=True`` does not -provide for change events - instead, the ORM must scan -through all mutable values present in a session and compare -them against their original value for changes every time -``flush()`` is called, which is a very time consuming event. -This is a holdover from the very early days of SQLAlchemy -when ``flush()`` was not automatic and the history tracking -system was not nearly as sophisticated as it is now. - -Existing applications which use ``PickleType``, -``postgresql.ARRAY`` or other ``MutableType`` subclasses, -and require in-place mutation detection, should migrate to -the new mutation tracking system, as ``mutable=True`` is -likely to be deprecated in the future. - -:ticket:`1980` - -Mutability detection of ``composite()`` requires the Mutation Tracking Extension --------------------------------------------------------------------------------- - -So-called "composite" mapped attributes, those configured -using the technique described at `Composite Column Types -`_, have been re-implemented such -that the ORM internals are no longer aware of them (leading -to shorter and more efficient codepaths in critical -sections). While composite types are generally intended to -be treated as immutable value objects, this was never -enforced. For applications that use composites with -mutability, the `Mutation Tracking `_ extension offers a -base class which establishes a mechanism for user-defined -composite types to send change event messages back to the -owning parent or parents of each object. - -Applications which use composite types and rely upon in- -place mutation detection of these objects should either -migrate to the "mutation tracking" extension, or change the -usage of the composite types such that in-place changes are -no longer needed (i.e., treat them as immutable value -objects). - -SQLite - the SQLite dialect now uses ``NullPool`` for file-based databases --------------------------------------------------------------------------- - -This change is **99.999% backwards compatible**, unless you -are using temporary tables across connection pool -connections. - -A file-based SQLite connection is blazingly fast, and using -``NullPool`` means that each call to ``Engine.connect`` -creates a new pysqlite connection. - -Previously, the ``SingletonThreadPool`` was used, which -meant that all connections to a certain engine in a thread -would be the same connection. It's intended that the new -approach is more intuitive, particularly when multiple -connections are used. - -``SingletonThreadPool`` is still the default engine when a -``:memory:`` database is used. - -Note that this change **breaks temporary tables used across -Session commits**, due to the way SQLite handles temp -tables. See the note at -https://www.sqlalchemy.org/docs/dialects/sqlite.html#using- -temporary-tables-with-sqlite if temporary tables beyond the -scope of one pool connection are desired. - -:ticket:`1921` - -``Session.merge()`` checks version ids for versioned mappers ------------------------------------------------------------- - -Session.merge() will check the version id of the incoming -state against that of the database, assuming the mapping -uses version ids and incoming state has a version_id -assigned, and raise StaleDataError if they don't match. -This is the correct behavior, in that if incoming state -contains a stale version id, it should be assumed the state -is stale. - -If merging data into a versioned state, the version id -attribute can be left undefined, and no version check will -take place. - -This check was confirmed by examining what Hibernate does - -both the ``merge()`` and the versioning features were -originally adapted from Hibernate. - -:ticket:`2027` - -Tuple label names in Query Improved ------------------------------------ - -This improvement is potentially slightly backwards -incompatible for an application that relied upon the old -behavior. - -Given two mapped classes ``Foo`` and ``Bar`` each with a -column ``spam``: - -:: - - - qa = session.query(Foo.spam) - qb = session.query(Bar.spam) - - qu = qa.union(qb) - -The name given to the single column yielded by ``qu`` will -be ``spam``. Previously it would be something like -``foo_spam`` due to the way the ``union`` would combine -things, which is inconsistent with the name ``spam`` in the -case of a non-unioned query. - -:ticket:`1942` - -Mapped column attributes reference the most specific column first ------------------------------------------------------------------ - -This is a change to the behavior involved when a mapped -column attribute references multiple columns, specifically -when dealing with an attribute on a joined-table subclass -that has the same name as that of an attribute on the -superclass. - -Using declarative, the scenario is this: - -:: - - class Parent(Base): - __tablename__ = "parent" - id = Column(Integer, primary_key=True) - - - class Child(Parent): - __tablename__ = "child" - id = Column(Integer, ForeignKey("parent.id"), primary_key=True) - -Above, the attribute ``Child.id`` refers to both the -``child.id`` column as well as ``parent.id`` - this due to -the name of the attribute. If it were named differently on -the class, such as ``Child.child_id``, it then maps -distinctly to ``child.id``, with ``Child.id`` being the same -attribute as ``Parent.id``. - -When the ``id`` attribute is made to reference both -``parent.id`` and ``child.id``, it stores them in an ordered -list. An expression such as ``Child.id`` then refers to -just *one* of those columns when rendered. Up until 0.6, -this column would be ``parent.id``. In 0.7, it is the less -surprising ``child.id``. - -The legacy of this behavior deals with behaviors and -restrictions of the ORM that don't really apply anymore; all -that was needed was to reverse the order. - -A primary advantage of this approach is that it's now easier -to construct ``primaryjoin`` expressions that refer to the -local column: - -:: - - class Child(Parent): - __tablename__ = "child" - id = Column(Integer, ForeignKey("parent.id"), primary_key=True) - some_related = relationship( - "SomeRelated", primaryjoin="Child.id==SomeRelated.child_id" - ) - - - class SomeRelated(Base): - __tablename__ = "some_related" - id = Column(Integer, primary_key=True) - child_id = Column(Integer, ForeignKey("child.id")) - -Prior to 0.7 the ``Child.id`` expression would reference -``Parent.id``, and it would be necessary to map ``child.id`` -to a distinct attribute. - -It also means that a query like this one changes its -behavior: - -:: - - session.query(Parent).filter(Child.id > 7) - -In 0.6, this would render: - -.. sourcecode:: sql - - SELECT parent.id AS parent_id - FROM parent - WHERE parent.id > :id_1 - -in 0.7, you get: - -.. sourcecode:: sql - - SELECT parent.id AS parent_id - FROM parent, child - WHERE child.id > :id_1 - -which you'll note is a cartesian product - this behavior is -now equivalent to that of any other attribute that is local -to ``Child``. The ``with_polymorphic()`` method, or a -similar strategy of explicitly joining the underlying -``Table`` objects, is used to render a query against all -``Parent`` objects with criteria against ``Child``, in the -same manner as that of 0.5 and 0.6: - -:: - - print(s.query(Parent).with_polymorphic([Child]).filter(Child.id > 7)) - -Which on both 0.6 and 0.7 renders: - -.. sourcecode:: sql - - SELECT parent.id AS parent_id, child.id AS child_id - FROM parent LEFT OUTER JOIN child ON parent.id = child.id - WHERE child.id > :id_1 - -Another effect of this change is that a joined-inheritance -load across two tables will populate from the child table's -value, not that of the parent table. An unusual case is that -a query against "Parent" using ``with_polymorphic="*"`` -issues a query against "parent", with a LEFT OUTER JOIN to -"child". The row is located in "Parent", sees the -polymorphic identity corresponds to "Child", but suppose the -actual row in "child" has been *deleted*. Due to this -corruption, the row comes in with all the columns -corresponding to "child" set to NULL - this is now the value -that gets populated, not the one in the parent table. - -:ticket:`1892` - -Mapping to joins with two or more same-named columns requires explicit declaration ----------------------------------------------------------------------------------- - -This is somewhat related to the previous change in -:ticket:`1892`. When mapping to a join, same-named columns -must be explicitly linked to mapped attributes, i.e. as -described in `Mapping a Class Against Multiple Tables `_. - -Given two tables ``foo`` and ``bar``, each with a primary -key column ``id``, the following now produces an error: - -:: - - - foobar = foo.join(bar, foo.c.id == bar.c.foo_id) - mapper(FooBar, foobar) - -This because the ``mapper()`` refuses to guess what column -is the primary representation of ``FooBar.id`` - is it -``foo.c.id`` or is it ``bar.c.id`` ? The attribute must be -explicit: - -:: - - - foobar = foo.join(bar, foo.c.id == bar.c.foo_id) - mapper(FooBar, foobar, properties={"id": [foo.c.id, bar.c.id]}) - -:ticket:`1896` - -Mapper requires that polymorphic_on column be present in the mapped selectable ------------------------------------------------------------------------------- - -This is a warning in 0.6, now an error in 0.7. The column -given for ``polymorphic_on`` must be in the mapped -selectable. This to prevent some occasional user errors -such as: - -:: - - mapper(SomeClass, sometable, polymorphic_on=some_lookup_table.c.id) - -where above the polymorphic_on needs to be on a -``sometable`` column, in this case perhaps -``sometable.c.some_lookup_id``. There are also some -"polymorphic union" scenarios where similar mistakes -sometimes occur. - -Such a configuration error has always been "wrong", and the -above mapping doesn't work as specified - the column would -be ignored. It is however potentially backwards -incompatible in the rare case that an application has been -unknowingly relying upon this behavior. - -:ticket:`1875` - -``DDL()`` constructs now escape percent signs ---------------------------------------------- - -Previously, percent signs in ``DDL()`` strings would have to -be escaped, i.e. ``%%`` depending on DBAPI, for those DBAPIs -that accept ``pyformat`` or ``format`` binds (i.e. psycopg2, -mysql-python), which was inconsistent versus ``text()`` -constructs which did this automatically. The same escaping -now occurs for ``DDL()`` as for ``text()``. - -:ticket:`1897` - -``Table.c`` / ``MetaData.tables`` refined a bit, don't allow direct mutation ----------------------------------------------------------------------------- - -Another area where some users were tinkering around in such -a way that doesn't actually work as expected, but still left -an exceedingly small chance that some application was -relying upon this behavior, the construct returned by the -``.c`` attribute on ``Table`` and the ``.tables`` attribute -on ``MetaData`` is explicitly non-mutable. The "mutable" -version of the construct is now private. Adding columns to -``.c`` involves using the ``append_column()`` method of -``Table``, which ensures things are associated with the -parent ``Table`` in the appropriate way; similarly, -``MetaData.tables`` has a contract with the ``Table`` -objects stored in this dictionary, as well as a little bit -of new bookkeeping in that a ``set()`` of all schema names -is tracked, which is satisfied only by using the public -``Table`` constructor as well as ``Table.tometadata()``. - -It is of course possible that the ``ColumnCollection`` and -``dict`` collections consulted by these attributes could -someday implement events on all of their mutational methods -such that the appropriate bookkeeping occurred upon direct -mutation of the collections, but until someone has the -motivation to implement all that along with dozens of new -unit tests, narrowing the paths to mutation of these -collections will ensure no application is attempting to rely -upon usages that are currently not supported. - -:ticket:`1893` :ticket:`1917` - -server_default consistently returns None for all inserted_primary_key values ----------------------------------------------------------------------------- - -Established consistency when server_default is present on an -Integer PK column. SQLA doesn't pre-fetch these, nor do they -come back in cursor.lastrowid (DBAPI). Ensured all backends -consistently return None in result.inserted_primary_key for -these - some backends may have returned a value previously. -Using a server_default on a primary key column is extremely -unusual. If a special function or SQL expression is used -to generate primary key defaults, this should be established -as a Python-side "default" instead of server_default. - -Regarding reflection for this case, reflection of an int PK -col with a server_default sets the "autoincrement" flag to -False, except in the case of a PG SERIAL col where we -detected a sequence default. - -:ticket:`2020` :ticket:`2021` - -The ``sqlalchemy.exceptions`` alias in sys.modules is removed -------------------------------------------------------------- - -For a few years we've added the string -``sqlalchemy.exceptions`` to ``sys.modules``, so that a -statement like "``import sqlalchemy.exceptions``" would -work. The name of the core exceptions module has been -``exc`` for a long time now, so the recommended import for -this module is: - -:: - - from sqlalchemy import exc - -The ``exceptions`` name is still present in "``sqlalchemy``" -for applications which might have said ``from sqlalchemy -import exceptions``, but they should also start using the -``exc`` name. - -Query Timing Recipe Changes ---------------------------- - -While not part of SQLAlchemy itself, it's worth mentioning -that the rework of the ``ConnectionProxy`` into the new -event system means it is no longer appropriate for the -"Timing all Queries" recipe. Please adjust query-timers to -use the ``before_cursor_execute()`` and -``after_cursor_execute()`` events, demonstrated in the -updated recipe UsageRecipes/Profiling. - -Deprecated API -============== - -Default constructor on types will not accept arguments ------------------------------------------------------- - -Simple types like ``Integer``, ``Date`` etc. in the core -types module don't accept arguments. The default -constructor that accepts/ignores a catchall ``\*args, -\**kwargs`` is restored as of 0.7b4/0.7.0, but emits a -deprecation warning. - -If arguments are being used with a core type like -``Integer``, it may be that you intended to use a dialect -specific type, such as ``sqlalchemy.dialects.mysql.INTEGER`` -which does accept a "display_width" argument for example. - -compile_mappers() renamed configure_mappers(), simplified configuration internals ---------------------------------------------------------------------------------- - -This system slowly morphed from something small, implemented -local to an individual mapper, and poorly named into -something that's more of a global "registry-" level function -and poorly named, so we've fixed both by moving the -implementation out of ``Mapper`` altogether and renaming it -to ``configure_mappers()``. It is of course normally not -needed for an application to call ``configure_mappers()`` as -this process occurs on an as-needed basis, as soon as the -mappings are needed via attribute or query access. - -:ticket:`1966` - -Core listener/proxy superseded by event listeners -------------------------------------------------- - -``PoolListener``, ``ConnectionProxy``, -``DDLElement.execute_at`` are superseded by -``event.listen()``, using the ``PoolEvents``, -``EngineEvents``, ``DDLEvents`` dispatch targets, -respectively. - -ORM extensions superseded by event listeners --------------------------------------------- - -``MapperExtension``, ``AttributeExtension``, -``SessionExtension`` are superseded by ``event.listen()``, -using the ``MapperEvents``/``InstanceEvents``, -``AttributeEvents``, ``SessionEvents``, dispatch targets, -respectively. - -Sending a string to 'distinct' in select() for MySQL should be done via prefixes --------------------------------------------------------------------------------- - -This obscure feature allows this pattern with the MySQL -backend: - -:: - - select([mytable], distinct="ALL", prefixes=["HIGH_PRIORITY"]) - -The ``prefixes`` keyword or ``prefix_with()`` method should -be used for non-standard or unusual prefixes: - -:: - - select([mytable]).prefix_with("HIGH_PRIORITY", "ALL") - -``useexisting`` superseded by ``extend_existing`` and ``keep_existing`` ------------------------------------------------------------------------ - -The ``useexisting`` flag on Table has been superseded by a -new pair of flags ``keep_existing`` and ``extend_existing``. -``extend_existing`` is equivalent to ``useexisting`` - the -existing Table is returned, and additional constructor -elements are added. With ``keep_existing``, the existing -Table is returned, but additional constructor elements are -not added - these elements are only applied when the Table -is newly created. - -Backwards Incompatible API Changes -================================== - -Callables passed to ``bindparam()`` don't get evaluated - affects the Beaker example ------------------------------------------------------------------------------------- - -:ticket:`1950` - -Note this affects the Beaker caching example, where the -workings of the ``_params_from_query()`` function needed a -slight adjustment. If you're using code from the Beaker -example, this change should be applied. - -types.type_map is now private, types._type_map ----------------------------------------------- - -We noticed some users tapping into this dictionary inside of -``sqlalchemy.types`` as a shortcut to associating Python -types with SQL types. We can't guarantee the contents or -format of this dictionary, and additionally the business of -associating Python types in a one-to-one fashion has some -grey areas that should are best decided by individual -applications, so we've underscored this attribute. - -:ticket:`1870` - -Renamed the ``alias`` keyword arg of standalone ``alias()`` function to ``name`` --------------------------------------------------------------------------------- - -This so that the keyword argument ``name`` matches that of -the ``alias()`` methods on all ``FromClause`` objects as -well as the ``name`` argument on ``Query.subquery()``. - -Only code that uses the standalone ``alias()`` function, and -not the method bound functions, and passes the alias name -using the explicit keyword name ``alias``, and not -positionally, would need modification here. - -Non-public ``Pool`` methods underscored ---------------------------------------- - -All methods of ``Pool`` and subclasses which are not -intended for public use have been renamed with underscores. -That they were not named this way previously was a bug. - -Pooling methods now underscored or removed: - -``Pool.create_connection()`` -> -``Pool._create_connection()`` - -``Pool.do_get()`` -> ``Pool._do_get()`` - -``Pool.do_return_conn()`` -> ``Pool._do_return_conn()`` - -``Pool.do_return_invalid()`` -> removed, was not used - -``Pool.return_conn()`` -> ``Pool._return_conn()`` - -``Pool.get()`` -> ``Pool._get()``, public API is -``Pool.connect()`` - -``SingletonThreadPool.cleanup()`` -> ``_cleanup()`` - -``SingletonThreadPool.dispose_local()`` -> removed, use -``conn.invalidate()`` - -:ticket:`1982` - -Previously Deprecated, Now Removed -================================== - -Query.join(), Query.outerjoin(), eagerload(), eagerload_all(), others no longer allow lists of attributes as arguments ----------------------------------------------------------------------------------------------------------------------- - -Passing a list of attributes or attribute names to -``Query.join``, ``eagerload()``, and similar has been -deprecated since 0.5: - -:: - - # old way, deprecated since 0.5 - session.query(Houses).join([Houses.rooms, Room.closets]) - session.query(Houses).options(eagerload_all([Houses.rooms, Room.closets])) - -These methods all accept \*args as of the 0.5 series: - -:: - - # current way, in place since 0.5 - session.query(Houses).join(Houses.rooms, Room.closets) - session.query(Houses).options(eagerload_all(Houses.rooms, Room.closets)) - -``ScopedSession.mapper`` is removed ------------------------------------ - -This feature provided a mapper extension which linked class- -based functionality with a particular ``ScopedSession``, in -particular providing the behavior such that new object -instances would be automatically associated with that -session. The feature was overused by tutorials and -frameworks which led to great user confusion due to its -implicit behavior, and was deprecated in 0.5.5. Techniques -for replicating its functionality are at -[wiki:UsageRecipes/SessionAwareMapper] - diff --git a/doc/build/changelog/migration_08.rst b/doc/build/changelog/migration_08.rst deleted file mode 100644 index 0f661cca79..0000000000 --- a/doc/build/changelog/migration_08.rst +++ /dev/null @@ -1,1539 +0,0 @@ -============================= -What's New in SQLAlchemy 0.8? -============================= - -.. admonition:: About this Document - - This document describes changes between SQLAlchemy version 0.7, - undergoing maintenance releases as of October, 2012, - and SQLAlchemy version 0.8, which is expected for release - in early 2013. - - Document date: October 25, 2012 - Updated: March 9, 2013 - -Introduction -============ - -This guide introduces what's new in SQLAlchemy version 0.8, -and also documents changes which affect users migrating -their applications from the 0.7 series of SQLAlchemy to 0.8. - -SQLAlchemy releases are closing in on 1.0, and each new -version since 0.5 features fewer major usage changes. Most -applications that are settled into modern 0.7 patterns -should be movable to 0.8 with no changes. Applications that -use 0.6 and even 0.5 patterns should be directly migratable -to 0.8 as well, though larger applications may want to test -with each interim version. - -Platform Support -================ - -Targeting Python 2.5 and Up Now -------------------------------- - -SQLAlchemy 0.8 will target Python 2.5 and forward; -compatibility for Python 2.4 is being dropped. - -The internals will be able to make usage of Python ternaries -(that is, ``x if y else z``) which will improve things -versus the usage of ``y and x or z``, which naturally has -been the source of some bugs, as well as context managers -(that is, ``with:``) and perhaps in some cases -``try:/except:/else:`` blocks which will help with code -readability. - -SQLAlchemy will eventually drop 2.5 support as well - when -2.6 is reached as the baseline, SQLAlchemy will move to use -2.6/3.3 in-place compatibility, removing the usage of the -``2to3`` tool and maintaining a source base that works with -Python 2 and 3 at the same time. - -New ORM Features -================ - -.. _feature_relationship_08: - -Rewritten :func:`_orm.relationship` mechanics ----------------------------------------------- -0.8 features a much improved and capable system regarding -how :func:`_orm.relationship` determines how to join between two -entities. The new system includes these features: - -* The ``primaryjoin`` argument is **no longer needed** when - constructing a :func:`_orm.relationship` against a class that - has multiple foreign key paths to the target. Only the - ``foreign_keys`` argument is needed to specify those - columns which should be included: - - :: - - - class Parent(Base): - __tablename__ = "parent" - id = Column(Integer, primary_key=True) - child_id_one = Column(Integer, ForeignKey("child.id")) - child_id_two = Column(Integer, ForeignKey("child.id")) - - child_one = relationship("Child", foreign_keys=child_id_one) - child_two = relationship("Child", foreign_keys=child_id_two) - - - class Child(Base): - __tablename__ = "child" - id = Column(Integer, primary_key=True) - -* relationships against self-referential, composite foreign - keys where **a column points to itself** are now - supported. The canonical case is as follows: - - :: - - class Folder(Base): - __tablename__ = "folder" - __table_args__ = ( - ForeignKeyConstraint( - ["account_id", "parent_id"], ["folder.account_id", "folder.folder_id"] - ), - ) - - account_id = Column(Integer, primary_key=True) - folder_id = Column(Integer, primary_key=True) - parent_id = Column(Integer) - name = Column(String) - - parent_folder = relationship( - "Folder", backref="child_folders", remote_side=[account_id, folder_id] - ) - - Above, the ``Folder`` refers to its parent ``Folder`` - joining from ``account_id`` to itself, and ``parent_id`` - to ``folder_id``. When SQLAlchemy constructs an auto- - join, no longer can it assume all columns on the "remote" - side are aliased, and all columns on the "local" side are - not - the ``account_id`` column is **on both sides**. So - the internal relationship mechanics were totally rewritten - to support an entirely different system whereby two copies - of ``account_id`` are generated, each containing different - *annotations* to determine their role within the - statement. Note the join condition within a basic eager - load: - - .. sourcecode:: sql - - SELECT - folder.account_id AS folder_account_id, - folder.folder_id AS folder_folder_id, - folder.parent_id AS folder_parent_id, - folder.name AS folder_name, - folder_1.account_id AS folder_1_account_id, - folder_1.folder_id AS folder_1_folder_id, - folder_1.parent_id AS folder_1_parent_id, - folder_1.name AS folder_1_name - FROM folder - LEFT OUTER JOIN folder AS folder_1 - ON - folder_1.account_id = folder.account_id - AND folder.folder_id = folder_1.parent_id - - WHERE folder.folder_id = ? AND folder.account_id = ? - -* Previously difficult custom join conditions, like those involving - functions and/or CASTing of types, will now function as - expected in most cases:: - - class HostEntry(Base): - __tablename__ = "host_entry" - - id = Column(Integer, primary_key=True) - ip_address = Column(INET) - content = Column(String(50)) - - # relationship() using explicit foreign_keys, remote_side - parent_host = relationship( - "HostEntry", - primaryjoin=ip_address == cast(content, INET), - foreign_keys=content, - remote_side=ip_address, - ) - - The new :func:`_orm.relationship` mechanics make use of a - SQLAlchemy concept known as :term:`annotations`. These annotations - are also available to application code explicitly via - the :func:`.foreign` and :func:`.remote` functions, either - as a means to improve readability for advanced configurations - or to directly inject an exact configuration, bypassing - the usual join-inspection heuristics:: - - from sqlalchemy.orm import foreign, remote - - - class HostEntry(Base): - __tablename__ = "host_entry" - - id = Column(Integer, primary_key=True) - ip_address = Column(INET) - content = Column(String(50)) - - # relationship() using explicit foreign() and remote() annotations - # in lieu of separate arguments - parent_host = relationship( - "HostEntry", - primaryjoin=remote(ip_address) == cast(foreign(content), INET), - ) - -.. seealso:: - - :ref:`relationship_configure_joins` - a newly revised section on :func:`_orm.relationship` - detailing the latest techniques for customizing related attributes and collection - access. - -:ticket:`1401` :ticket:`610` - -.. _feature_orminspection_08: - -New Class/Object Inspection System ----------------------------------- - -Lots of SQLAlchemy users are writing systems that require -the ability to inspect the attributes of a mapped class, -including being able to get at the primary key columns, -object relationships, plain attributes, and so forth, -typically for the purpose of building data-marshalling -systems, like JSON/XML conversion schemes and of course form -libraries galore. - -Originally, the :class:`_schema.Table` and :class:`_schema.Column` model were the -original inspection points, which have a well-documented -system. While SQLAlchemy ORM models are also fully -introspectable, this has never been a fully stable and -supported feature, and users tended to not have a clear idea -how to get at this information. - -0.8 now provides a consistent, stable and fully -documented API for this purpose, including an inspection -system which works on mapped classes, instances, attributes, -and other Core and ORM constructs. The entrypoint to this -system is the core-level :func:`_sa.inspect` function. -In most cases, the object being inspected -is one already part of SQLAlchemy's system, -such as :class:`_orm.Mapper`, :class:`.InstanceState`, -:class:`_reflection.Inspector`. In some cases, new objects have been -added with the job of providing the inspection API in -certain contexts, such as :class:`.AliasedInsp` and -:class:`.AttributeState`. - -A walkthrough of some key capabilities follows: - -.. sourcecode:: pycon+sql - - >>> class User(Base): - ... __tablename__ = "user" - ... id = Column(Integer, primary_key=True) - ... name = Column(String) - ... name_syn = synonym(name) - ... addresses = relationship("Address") - - >>> # universal entry point is inspect() - >>> b = inspect(User) - - >>> # b in this case is the Mapper - >>> b - - - >>> # Column namespace - >>> b.columns.id - Column('id', Integer(), table=, primary_key=True, nullable=False) - - >>> # mapper's perspective of the primary key - >>> b.primary_key - (Column('id', Integer(), table=, primary_key=True, nullable=False),) - - >>> # MapperProperties available from .attrs - >>> b.attrs.keys() - ['name_syn', 'addresses', 'id', 'name'] - - >>> # .column_attrs, .relationships, etc. filter this collection - >>> b.column_attrs.keys() - ['id', 'name'] - - >>> list(b.relationships) - [] - - >>> # they are also namespaces - >>> b.column_attrs.id - - - >>> b.relationships.addresses - - - >>> # point inspect() at a mapped, class level attribute, - >>> # returns the attribute itself - >>> b = inspect(User.addresses) - >>> b - - - >>> # From here we can get the mapper: - >>> b.mapper - - - >>> # the parent inspector, in this case a mapper - >>> b.parent - - - >>> # an expression - >>> print(b.expression) - {printsql}"user".id = address.user_id{stop} - - >>> # inspect works on instances - >>> u1 = User(id=3, name="x") - >>> b = inspect(u1) - - >>> # it returns the InstanceState - >>> b - - - >>> # similar attrs accessor refers to the - >>> b.attrs.keys() - ['id', 'name_syn', 'addresses', 'name'] - - >>> # attribute interface - from attrs, you get a state object - >>> b.attrs.id - - - >>> # this object can give you, current value... - >>> b.attrs.id.value - 3 - - >>> # ... current history - >>> b.attrs.id.history - History(added=[3], unchanged=(), deleted=()) - - >>> # InstanceState can also provide session state information - >>> # lets assume the object is persistent - >>> s = Session() - >>> s.add(u1) - >>> s.commit() - - >>> # now we can get primary key identity, always - >>> # works in query.get() - >>> b.identity - (3,) - - >>> # the mapper level key - >>> b.identity_key - (, (3,)) - - >>> # state within the session - >>> b.persistent, b.transient, b.deleted, b.detached - (True, False, False, False) - - >>> # owning session - >>> b.session - - -.. seealso:: - - :ref:`core_inspection_toplevel` - -:ticket:`2208` - -New with_polymorphic() feature, can be used anywhere ----------------------------------------------------- - -The :meth:`_query.Query.with_polymorphic` method allows the user to -specify which tables should be present when querying against -a joined-table entity. Unfortunately the method is awkward -and only applies to the first entity in the list, and -otherwise has awkward behaviors both in usage as well as -within the internals. A new enhancement to the -:func:`.aliased` construct has been added called -:func:`.with_polymorphic` which allows any entity to be -"aliased" into a "polymorphic" version of itself, freely -usable anywhere: - -:: - - from sqlalchemy.orm import with_polymorphic - - palias = with_polymorphic(Person, [Engineer, Manager]) - session.query(Company).join(palias, Company.employees).filter( - or_(Engineer.language == "java", Manager.hair == "pointy") - ) - -.. seealso:: - - :ref:`with_polymorphic` - newly updated documentation for polymorphic - loading control. - -:ticket:`2333` - -of_type() works with alias(), with_polymorphic(), any(), has(), joinedload(), subqueryload(), contains_eager() --------------------------------------------------------------------------------------------------------------- - -The :meth:`.PropComparator.of_type` method is used to specify -a specific subtype to use when constructing SQL expressions along -a :func:`_orm.relationship` that has a :term:`polymorphic` mapping as its target. -This method can now be used to target *any number* of target subtypes, -by combining it with the new :func:`.with_polymorphic` function:: - - # use eager loading in conjunction with with_polymorphic targets - Job_P = with_polymorphic(Job, [SubJob, ExtraJob], aliased=True) - q = ( - s.query(DataContainer) - .join(DataContainer.jobs.of_type(Job_P)) - .options(contains_eager(DataContainer.jobs.of_type(Job_P))) - ) - -The method now works equally well in most places a regular relationship -attribute is accepted, including with loader functions like -:func:`_orm.joinedload`, :func:`.subqueryload`, :func:`.contains_eager`, -and comparison methods like :meth:`.PropComparator.any` -and :meth:`.PropComparator.has`:: - - # use eager loading in conjunction with with_polymorphic targets - Job_P = with_polymorphic(Job, [SubJob, ExtraJob], aliased=True) - q = ( - s.query(DataContainer) - .join(DataContainer.jobs.of_type(Job_P)) - .options(contains_eager(DataContainer.jobs.of_type(Job_P))) - ) - - # pass subclasses to eager loads (implicitly applies with_polymorphic) - q = s.query(ParentThing).options( - joinedload_all(ParentThing.container, DataContainer.jobs.of_type(SubJob)) - ) - - # control self-referential aliasing with any()/has() - Job_A = aliased(Job) - q = ( - s.query(Job) - .join(DataContainer.jobs) - .filter( - DataContainer.jobs.of_type(Job_A).any( - and_(Job_A.id < Job.id, Job_A.type == "fred") - ) - ) - ) - -.. seealso:: - - :ref:`inheritance_of_type` - -:ticket:`2438` :ticket:`1106` - -Events Can Be Applied to Unmapped Superclasses ----------------------------------------------- - -Mapper and instance events can now be associated with an unmapped -superclass, where those events will be propagated to subclasses -as those subclasses are mapped. The ``propagate=True`` flag -should be used. This feature allows events to be associated -with a declarative base class:: - - from sqlalchemy.ext.declarative import declarative_base - - Base = declarative_base() - - - @event.listens_for("load", Base, propagate=True) - def on_load(target, context): - print("New instance loaded:", target) - - - # on_load() will be applied to SomeClass - class SomeClass(Base): - __tablename__ = "sometable" - - # ... - -:ticket:`2585` - -Declarative Distinguishes Between Modules/Packages --------------------------------------------------- - -A key feature of Declarative is the ability to refer -to other mapped classes using their string name. The -registry of class names is now sensitive to the owning -module and package of a given class. The classes -can be referred to via dotted name in expressions:: - - class Snack(Base): - # ... - - peanuts = relationship( - "nuts.Peanut", primaryjoin="nuts.Peanut.snack_id == Snack.id" - ) - -The resolution allows that any full or partial -disambiguating package name can be used. If the -path to a particular class is still ambiguous, -an error is raised. - -:ticket:`2338` - - -New DeferredReflection Feature in Declarative ---------------------------------------------- - -The "deferred reflection" example has been moved to a -supported feature within Declarative. This feature allows -the construction of declarative mapped classes with only -placeholder ``Table`` metadata, until a ``prepare()`` step -is called, given an ``Engine`` with which to reflect fully -all tables and establish actual mappings. The system -supports overriding of columns, single and joined -inheritance, as well as distinct bases-per-engine. A full -declarative configuration can now be created against an -existing table that is assembled upon engine creation time -in one step: - -:: - - class ReflectedOne(DeferredReflection, Base): - __abstract__ = True - - - class ReflectedTwo(DeferredReflection, Base): - __abstract__ = True - - - class MyClass(ReflectedOne): - __tablename__ = "mytable" - - - class MyOtherClass(ReflectedOne): - __tablename__ = "myothertable" - - - class YetAnotherClass(ReflectedTwo): - __tablename__ = "yetanothertable" - - - ReflectedOne.prepare(engine_one) - ReflectedTwo.prepare(engine_two) - -.. seealso:: - - :class:`.DeferredReflection` - -:ticket:`2485` - -ORM Classes Now Accepted by Core Constructs -------------------------------------------- - -While the SQL expressions used with :meth:`_query.Query.filter`, -such as ``User.id == 5``, have always been compatible for -use with core constructs such as :func:`_expression.select`, the mapped -class itself would not be recognized when passed to :func:`_expression.select`, -:meth:`_expression.Select.select_from`, or :meth:`_expression.Select.correlate`. -A new SQL registration system allows a mapped class to be -accepted as a FROM clause within the core:: - - from sqlalchemy import select - - stmt = select([User]).where(User.id == 5) - -Above, the mapped ``User`` class will expand into -the :class:`_schema.Table` to which ``User`` is mapped. - -:ticket:`2245` - -.. _change_orm_2365: - -Query.update() supports UPDATE..FROM ------------------------------------- - -The new UPDATE..FROM mechanics work in query.update(). -Below, we emit an UPDATE against ``SomeEntity``, adding -a FROM clause (or equivalent, depending on backend) -against ``SomeOtherEntity``:: - - query(SomeEntity).filter(SomeEntity.id == SomeOtherEntity.id).filter( - SomeOtherEntity.foo == "bar" - ).update({"data": "x"}) - -In particular, updates to joined-inheritance -entities are supported, provided the target of the UPDATE is local to the -table being filtered on, or if the parent and child tables -are mixed, they are joined explicitly in the query. Below, -given ``Engineer`` as a joined subclass of ``Person``: - -:: - - query(Engineer).filter(Person.id == Engineer.id).filter( - Person.name == "dilbert" - ).update({"engineer_data": "java"}) - -would produce: - -.. sourcecode:: sql - - UPDATE engineer SET engineer_data='java' FROM person - WHERE person.id=engineer.id AND person.name='dilbert' - -:ticket:`2365` - -rollback() will only roll back "dirty" objects from a begin_nested() --------------------------------------------------------------------- - -A behavioral change that should improve efficiency for those -users using SAVEPOINT via ``Session.begin_nested()`` - upon -``rollback()``, only those objects that were made dirty -since the last flush will be expired, the rest of the -``Session`` remains intact. This because a ROLLBACK to a -SAVEPOINT does not terminate the containing transaction's -isolation, so no expiry is needed except for those changes -that were not flushed in the current transaction. - -:ticket:`2452` - -Caching Example now uses dogpile.cache --------------------------------------- - -The caching example now uses `dogpile.cache `_. -Dogpile.cache is a rewrite of the caching portion -of Beaker, featuring vastly simpler and faster operation, -as well as support for distributed locking. - -Note that the SQLAlchemy APIs used by the Dogpile example as well -as the previous Beaker example have changed slightly, in particular -this change is needed as illustrated in the Beaker example: - -.. sourcecode:: diff - - --- examples/beaker_caching/caching_query.py - +++ examples/beaker_caching/caching_query.py - @@ -222,7 +222,8 @@ - - """ - if query._current_path: - - mapper, key = query._current_path[-2:] - + mapper, prop = query._current_path[-2:] - + key = prop.key - - for cls in mapper.class_.__mro__: - if (cls, key) in self._relationship_options: - -.. seealso:: - - :ref:`examples_caching` - -:ticket:`2589` - -New Core Features -================= - -Fully extensible, type-level operator support in Core ------------------------------------------------------ - -The Core has to date never had any system of adding support -for new SQL operators to Column and other expression -constructs, other than the :meth:`.ColumnOperators.op` method -which is "just enough" to make things work. There has also -never been any system in place for Core which allows the -behavior of existing operators to be overridden. Up until -now, the only way operators could be flexibly redefined was -in the ORM layer, using :func:`.column_property` given a -``comparator_factory`` argument. Third party libraries -like GeoAlchemy therefore were forced to be ORM-centric and -rely upon an array of hacks to apply new operations as well -as to get them to propagate correctly. - -The new operator system in Core adds the one hook that's -been missing all along, which is to associate new and -overridden operators with *types*. Since after all, it's -not really a column, CAST operator, or SQL function that -really drives what kinds of operations are present, it's the -*type* of the expression. The implementation details are -minimal - only a few extra methods are added to the core -:class:`_expression.ColumnElement` type so that it consults its -:class:`.TypeEngine` object for an optional set of operators. -New or revised operations can be associated with any type, -either via subclassing of an existing type, by using -:class:`.TypeDecorator`, or "globally across-the-board" by -attaching a new :class:`.TypeEngine.Comparator` object to an existing type -class. - -For example, to add logarithm support to :class:`.Numeric` types: - -:: - - - from sqlalchemy.types import Numeric - from sqlalchemy.sql import func - - - class CustomNumeric(Numeric): - class comparator_factory(Numeric.Comparator): - def log(self, other): - return func.log(self.expr, other) - -The new type is usable like any other type: - -:: - - - data = Table( - "data", - metadata, - Column("id", Integer, primary_key=True), - Column("x", CustomNumeric(10, 5)), - Column("y", CustomNumeric(10, 5)), - ) - - stmt = select([data.c.x.log(data.c.y)]).where(data.c.x.log(2) < value) - print(conn.execute(stmt).fetchall()) - -New features which have come from this immediately include -support for PostgreSQL's HSTORE type, as well as new -operations associated with PostgreSQL's ARRAY -type. It also paves the way for existing types to acquire -lots more operators that are specific to those types, such -as more string, integer and date operators. - -.. seealso:: - - :ref:`types_operators` - - :class:`.HSTORE` - -:ticket:`2547` - -.. _feature_2623: - -Multiple-VALUES support for Insert ----------------------------------- - -The :meth:`_expression.Insert.values` method now supports a list of dictionaries, -which will render a multi-VALUES statement such as -``VALUES (), (), ...``. This is only relevant to backends which -support this syntax, including PostgreSQL, SQLite, and MySQL. It is -not the same thing as the usual ``executemany()`` style of INSERT which -remains unchanged:: - - users.insert().values( - [ - {"name": "some name"}, - {"name": "some other name"}, - {"name": "yet another name"}, - ] - ) - -.. seealso:: - - :meth:`_expression.Insert.values` - -:ticket:`2623` - -Type Expressions ----------------- - -SQL expressions can now be associated with types. Historically, -:class:`.TypeEngine` has always allowed Python-side functions which -receive both bound parameters as well as result row values, passing -them through a Python side conversion function on the way to/back from -the database. The new feature allows similar -functionality, except on the database side:: - - from sqlalchemy.types import String - from sqlalchemy import func, Table, Column, MetaData - - - class LowerString(String): - def bind_expression(self, bindvalue): - return func.lower(bindvalue) - - def column_expression(self, col): - return func.lower(col) - - - metadata = MetaData() - test_table = Table("test_table", metadata, Column("data", LowerString)) - -Above, the ``LowerString`` type defines a SQL expression that will be emitted -whenever the ``test_table.c.data`` column is rendered in the columns -clause of a SELECT statement: - -.. sourcecode:: pycon+sql - - >>> print(select([test_table]).where(test_table.c.data == "HI")) - {printsql}SELECT lower(test_table.data) AS data - FROM test_table - WHERE test_table.data = lower(:data_1) - -This feature is also used heavily by the new release of GeoAlchemy, -to embed PostGIS expressions inline in SQL based on type rules. - -.. seealso:: - - :ref:`types_sql_value_processing` - -:ticket:`1534` - -Core Inspection System ----------------------- - -The :func:`_sa.inspect` function introduced in :ref:`feature_orminspection_08` -also applies to the core. Applied to an :class:`_engine.Engine` it produces -an :class:`_reflection.Inspector` object:: - - from sqlalchemy import inspect - from sqlalchemy import create_engine - - engine = create_engine("postgresql://scott:tiger@localhost/test") - insp = inspect(engine) - print(insp.get_table_names()) - -It can also be applied to any :class:`_expression.ClauseElement`, which returns -the :class:`_expression.ClauseElement` itself, such as :class:`_schema.Table`, :class:`_schema.Column`, -:class:`_expression.Select`, etc. This allows it to work fluently between Core -and ORM constructs. - - -New Method :meth:`_expression.Select.correlate_except` -------------------------------------------------------- -:func:`_expression.select` now has a method :meth:`_expression.Select.correlate_except` -which specifies "correlate on all FROM clauses except those -specified". It can be used for mapping scenarios where -a related subquery should correlate normally, except -against a particular target selectable:: - - class SnortEvent(Base): - __tablename__ = "event" - - id = Column(Integer, primary_key=True) - signature = Column(Integer, ForeignKey("signature.id")) - - signatures = relationship("Signature", lazy=False) - - - class Signature(Base): - __tablename__ = "signature" - - id = Column(Integer, primary_key=True) - - sig_count = column_property( - select([func.count("*")]) - .where(SnortEvent.signature == id) - .correlate_except(SnortEvent) - ) - -.. seealso:: - - :meth:`_expression.Select.correlate_except` - -PostgreSQL HSTORE type ----------------------- - -Support for PostgreSQL's ``HSTORE`` type is now available as -:class:`_postgresql.HSTORE`. This type makes great usage -of the new operator system to provide a full range of operators -for HSTORE types, including index access, concatenation, -and containment methods such as -:meth:`~.HSTORE.comparator_factory.has_key`, -:meth:`~.HSTORE.comparator_factory.has_any`, and -:meth:`~.HSTORE.comparator_factory.matrix`:: - - from sqlalchemy.dialects.postgresql import HSTORE - - data = Table( - "data_table", - metadata, - Column("id", Integer, primary_key=True), - Column("hstore_data", HSTORE), - ) - - engine.execute(select([data.c.hstore_data["some_key"]])).scalar() - - engine.execute(select([data.c.hstore_data.matrix()])).scalar() - -.. seealso:: - - :class:`_postgresql.HSTORE` - - :class:`_postgresql.hstore` - -:ticket:`2606` - -Enhanced PostgreSQL ARRAY type ------------------------------- - -The :class:`_postgresql.ARRAY` type will accept an optional -"dimension" argument, pinning it to a fixed number of -dimensions and greatly improving efficiency when retrieving -results: - -:: - - # old way, still works since PG supports N-dimensions per row: - Column("my_array", postgresql.ARRAY(Integer)) - - # new way, will render ARRAY with correct number of [] in DDL, - # will process binds and results more efficiently as we don't need - # to guess how many levels deep to go - Column("my_array", postgresql.ARRAY(Integer, dimensions=2)) - -The type also introduces new operators, using the new type-specific -operator framework. New operations include indexed access:: - - result = conn.execute(select([mytable.c.arraycol[2]])) - -slice access in SELECT:: - - result = conn.execute(select([mytable.c.arraycol[2:4]])) - -slice updates in UPDATE:: - - conn.execute(mytable.update().values({mytable.c.arraycol[2:3]: [7, 8]})) - -freestanding array literals:: - - >>> from sqlalchemy.dialects import postgresql - >>> conn.scalar(select([postgresql.array([1, 2]) + postgresql.array([3, 4, 5])])) - [1, 2, 3, 4, 5] - -array concatenation, where below, the right side ``[4, 5, 6]`` is coerced into an array literal:: - - select([mytable.c.arraycol + [4, 5, 6]]) - -.. seealso:: - - :class:`_postgresql.ARRAY` - - :class:`_postgresql.array` - -:ticket:`2441` - -New, configurable DATE, TIME types for SQLite ---------------------------------------------- - -SQLite has no built-in DATE, TIME, or DATETIME types, and -instead provides some support for storage of date and time -values either as strings or integers. The date and time -types for SQLite are enhanced in 0.8 to be much more -configurable as to the specific format, including that the -"microseconds" portion is optional, as well as pretty much -everything else. - -:: - - Column("sometimestamp", sqlite.DATETIME(truncate_microseconds=True)) - Column( - "sometimestamp", - sqlite.DATETIME( - storage_format=( - "%(year)04d%(month)02d%(day)02d" - "%(hour)02d%(minute)02d%(second)02d%(microsecond)06d" - ), - regexp="(\d{4})(\d{2})(\d{2})(\d{2})(\d{2})(\d{2})(\d{6})", - ), - ) - Column( - "somedate", - sqlite.DATE( - storage_format="%(month)02d/%(day)02d/%(year)04d", - regexp="(?P\d+)/(?P\d+)/(?P\d+)", - ), - ) - -Huge thanks to Nate Dub for the sprinting on this at Pycon 2012. - -.. seealso:: - - :class:`_sqlite.DATETIME` - - :class:`_sqlite.DATE` - - :class:`_sqlite.TIME` - -:ticket:`2363` - -"COLLATE" supported across all dialects; in particular MySQL, PostgreSQL, SQLite --------------------------------------------------------------------------------- - -The "collate" keyword, long accepted by the MySQL dialect, is now established -on all :class:`.String` types and will render on any backend, including -when features such as :meth:`_schema.MetaData.create_all` and :func:`.cast` is used: - -.. sourcecode:: pycon+sql - - >>> stmt = select([cast(sometable.c.somechar, String(20, collation="utf8"))]) - >>> print(stmt) - {printsql}SELECT CAST(sometable.somechar AS VARCHAR(20) COLLATE "utf8") AS anon_1 - FROM sometable - -.. seealso:: - - :class:`.String` - -:ticket:`2276` - -"Prefixes" now supported for :func:`_expression.update`, :func:`_expression.delete` ------------------------------------------------------------------------------------- -Geared towards MySQL, a "prefix" can be rendered within any of -these constructs. E.g.:: - - stmt = table.delete().prefix_with("LOW_PRIORITY", dialect="mysql") - - - stmt = table.update().prefix_with("LOW_PRIORITY", dialect="mysql") - -The method is new in addition to those which already existed -on :func:`_expression.insert`, :func:`_expression.select` and :class:`_query.Query`. - -.. seealso:: - - :meth:`_expression.Update.prefix_with` - - :meth:`_expression.Delete.prefix_with` - - :meth:`_expression.Insert.prefix_with` - - :meth:`_expression.Select.prefix_with` - - :meth:`_query.Query.prefix_with` - -:ticket:`2431` - - -Behavioral Changes -================== - -.. _legacy_is_orphan_addition: - -The consideration of a "pending" object as an "orphan" has been made more aggressive ------------------------------------------------------------------------------------- - -This is a late add to the 0.8 series, however it is hoped that the new behavior -is generally more consistent and intuitive in a wider variety of -situations. The ORM has since at least version 0.4 included behavior -such that an object that's "pending", meaning that it's -associated with a :class:`.Session` but hasn't been inserted into the database -yet, is automatically expunged from the :class:`.Session` when it becomes an "orphan", -which means it has been de-associated with a parent object that refers to it -with ``delete-orphan`` cascade on the configured :func:`_orm.relationship`. This -behavior is intended to approximately mirror the behavior of a persistent -(that is, already inserted) object, where the ORM will emit a DELETE for such -objects that become orphans based on the interception of detachment events. - -The behavioral change comes into play for objects that -are referred to by multiple kinds of parents that each specify ``delete-orphan``; the -typical example is an :ref:`association object ` that bridges two other kinds of objects -in a many-to-many pattern. Previously, the behavior was such that the -pending object would be expunged only when de-associated with *all* of its parents. -With the behavioral change, the pending object -is expunged as soon as it is de-associated from *any* of the parents that it was -previously associated with. This behavior is intended to more closely -match that of persistent objects, which are deleted as soon -as they are de-associated from any parent. - -The rationale for the older behavior dates back -at least to version 0.4, and was basically a defensive decision to try to alleviate -confusion when an object was still being constructed for INSERT. But the reality -is that the object is re-associated with the :class:`.Session` as soon as it is -attached to any new parent in any case. - -It's still possible to flush an object -that is not associated with all of its required parents, if the object was either -not associated with those parents in the first place, or if it was expunged, but then -re-associated with a :class:`.Session` via a subsequent attachment event but still -not fully associated. In this situation, it is expected that the database -would emit an integrity error, as there are likely NOT NULL foreign key columns -that are unpopulated. The ORM makes the decision to let these INSERT attempts -occur, based on the judgment that an object that is only partially associated with -its required parents but has been actively associated with some of them, -is more often than not a user error, rather than an intentional -omission which should be silently skipped - silently skipping the INSERT here would -make user errors of this nature very hard to debug. - -The old behavior, for applications that might have been relying upon it, can be re-enabled for -any :class:`_orm.Mapper` by specifying the flag ``legacy_is_orphan`` as a mapper -option. - -The new behavior allows the following test case to work:: - - from sqlalchemy import Column, Integer, String, ForeignKey - from sqlalchemy.orm import relationship, backref - from sqlalchemy.ext.declarative import declarative_base - - Base = declarative_base() - - - class User(Base): - __tablename__ = "user" - id = Column(Integer, primary_key=True) - name = Column(String(64)) - - - class UserKeyword(Base): - __tablename__ = "user_keyword" - user_id = Column(Integer, ForeignKey("user.id"), primary_key=True) - keyword_id = Column(Integer, ForeignKey("keyword.id"), primary_key=True) - - user = relationship( - User, backref=backref("user_keywords", cascade="all, delete-orphan") - ) - - keyword = relationship( - "Keyword", backref=backref("user_keywords", cascade="all, delete-orphan") - ) - - # uncomment this to enable the old behavior - # __mapper_args__ = {"legacy_is_orphan": True} - - - class Keyword(Base): - __tablename__ = "keyword" - id = Column(Integer, primary_key=True) - keyword = Column("keyword", String(64)) - - - from sqlalchemy import create_engine - from sqlalchemy.orm import Session - - # note we're using PostgreSQL to ensure that referential integrity - # is enforced, for demonstration purposes. - e = create_engine("postgresql://scott:tiger@localhost/test", echo=True) - - Base.metadata.drop_all(e) - Base.metadata.create_all(e) - - session = Session(e) - - u1 = User(name="u1") - k1 = Keyword(keyword="k1") - - session.add_all([u1, k1]) - - uk1 = UserKeyword(keyword=k1, user=u1) - - # previously, if session.flush() were called here, - # this operation would succeed, but if session.flush() - # were not called here, the operation fails with an - # integrity error. - # session.flush() - del u1.user_keywords[0] - - session.commit() - -:ticket:`2655` - -The after_attach event fires after the item is associated with the Session instead of before; before_attach added ------------------------------------------------------------------------------------------------------------------ - -Event handlers which use after_attach can now assume the -given instance is associated with the given session: - -:: - - @event.listens_for(Session, "after_attach") - def after_attach(session, instance): - assert instance in session - -Some use cases require that it work this way. However, -other use cases require that the item is *not* yet part of -the session, such as when a query, intended to load some -state required for an instance, emits autoflush first and -would otherwise prematurely flush the target object. Those -use cases should use the new "before_attach" event: - -:: - - @event.listens_for(Session, "before_attach") - def before_attach(session, instance): - instance.some_necessary_attribute = ( - session.query(Widget).filter_by(instance.widget_name).first() - ) - -:ticket:`2464` - - - -Query now auto-correlates like a select() does ----------------------------------------------- - -Previously it was necessary to call :meth:`_query.Query.correlate` in -order to have a column- or WHERE-subquery correlate to the -parent: - -:: - - subq = ( - session.query(Entity.value) - .filter(Entity.id == Parent.entity_id) - .correlate(Parent) - .as_scalar() - ) - session.query(Parent).filter(subq == "some value") - -This was the opposite behavior of a plain ``select()`` -construct which would assume auto-correlation by default. -The above statement in 0.8 will correlate automatically: - -:: - - subq = session.query(Entity.value).filter(Entity.id == Parent.entity_id).as_scalar() - session.query(Parent).filter(subq == "some value") - -like in ``select()``, correlation can be disabled by calling -``query.correlate(None)`` or manually set by passing an -entity, ``query.correlate(someentity)``. - -:ticket:`2179` - -.. _correlation_context_specific: - -Correlation is now always context-specific ------------------------------------------- - -To allow a wider variety of correlation scenarios, the behavior of -:meth:`_expression.Select.correlate` and :meth:`_query.Query.correlate` has changed slightly -such that the SELECT statement will omit the "correlated" target from the -FROM clause only if the statement is actually used in that context. Additionally, -it's no longer possible for a SELECT statement that's placed as a FROM -in an enclosing SELECT statement to "correlate" (i.e. omit) a FROM clause. - -This change only makes things better as far as rendering SQL, in that it's no -longer possible to render illegal SQL where there are insufficient FROM -objects relative to what's being selected:: - - from sqlalchemy.sql import table, column, select - - t1 = table("t1", column("x")) - t2 = table("t2", column("y")) - s = select([t1, t2]).correlate(t1) - - print(s) - -Prior to this change, the above would return: - -.. sourcecode:: sql - - SELECT t1.x, t2.y FROM t2 - -which is invalid SQL as "t1" is not referred to in any FROM clause. - -Now, in the absence of an enclosing SELECT, it returns: - -.. sourcecode:: sql - - SELECT t1.x, t2.y FROM t1, t2 - -Within a SELECT, the correlation takes effect as expected: - -.. sourcecode:: python - - s2 = select([t1, t2]).where(t1.c.x == t2.c.y).where(t1.c.x == s) - print(s2) - -.. sourcecode:: sql - - SELECT t1.x, t2.y FROM t1, t2 - WHERE t1.x = t2.y AND t1.x = - (SELECT t1.x, t2.y FROM t2) - -This change is not expected to impact any existing applications, as -the correlation behavior remains identical for properly constructed -expressions. Only an application that relies, most likely within a -testing scenario, on the invalid string output of a correlated -SELECT used in a non-correlating context would see any change. - -:ticket:`2668` - - -.. _metadata_create_drop_tables: - -create_all() and drop_all() will now honor an empty list as such ----------------------------------------------------------------- - -The methods :meth:`_schema.MetaData.create_all` and :meth:`_schema.MetaData.drop_all` -will now accept a list of :class:`_schema.Table` objects that is empty, -and will not emit any CREATE or DROP statements. Previously, -an empty list was interpreted the same as passing ``None`` -for a collection, and CREATE/DROP would be emitted for all -items unconditionally. - -This is a bug fix but some applications may have been relying upon -the previous behavior. - -:ticket:`2664` - -Repaired the Event Targeting of :class:`.InstrumentationEvents` ---------------------------------------------------------------- - -The :class:`.InstrumentationEvents` series of event targets have -documented that the events will only be fired off according to -the actual class passed as a target. Through 0.7, this wasn't the -case, and any event listener applied to :class:`.InstrumentationEvents` -would be invoked for all classes mapped. In 0.8, additional -logic has been added so that the events will only invoke for those -classes sent in. The ``propagate`` flag here is set to ``True`` -by default as class instrumentation events are typically used to -intercept classes that aren't yet created. - -:ticket:`2590` - -No more magic coercion of "=" to IN when comparing to subquery in MS-SQL ------------------------------------------------------------------------- - -We found a very old behavior in the MSSQL dialect which -would attempt to rescue users from themselves when -doing something like this: - -:: - - scalar_subq = select([someothertable.c.id]).where(someothertable.c.data == "foo") - select([sometable]).where(sometable.c.id == scalar_subq) - -SQL Server doesn't allow an equality comparison to a scalar -SELECT, that is, "x = (SELECT something)". The MSSQL dialect -would convert this to an IN. The same thing would happen -however upon a comparison like "(SELECT something) = x", and -overall this level of guessing is outside of SQLAlchemy's -usual scope so the behavior is removed. - -:ticket:`2277` - -Fixed the behavior of :meth:`.Session.is_modified` --------------------------------------------------- - -The :meth:`.Session.is_modified` method accepts an argument -``passive`` which basically should not be necessary, the -argument in all cases should be the value ``True`` - when -left at its default of ``False`` it would have the effect of -hitting the database, and often triggering autoflush which -would itself change the results. In 0.8 the ``passive`` -argument will have no effect, and unloaded attributes will -never be checked for history since by definition there can -be no pending state change on an unloaded attribute. - -.. seealso:: - - :meth:`.Session.is_modified` - -:ticket:`2320` - -:attr:`_schema.Column.key` is honored in the :attr:`_expression.Select.c` attribute of :func:`_expression.select` with :meth:`_expression.Select.apply_labels` ---------------------------------------------------------------------------------------------------------------------------------------------------------------- -Users of the expression system know that :meth:`_expression.Select.apply_labels` -prepends the table name to each column name, affecting the -names that are available from :attr:`_expression.Select.c`: - -:: - - s = select([table1]).apply_labels() - s.c.table1_col1 - s.c.table1_col2 - -Before 0.8, if the :class:`_schema.Column` had a different :attr:`_schema.Column.key`, this -key would be ignored, inconsistently versus when -:meth:`_expression.Select.apply_labels` were not used: - -:: - - # before 0.8 - table1 = Table("t1", metadata, Column("col1", Integer, key="column_one")) - s = select([table1]) - s.c.column_one # would be accessible like this - s.c.col1 # would raise AttributeError - - s = select([table1]).apply_labels() - s.c.table1_column_one # would raise AttributeError - s.c.table1_col1 # would be accessible like this - -In 0.8, :attr:`_schema.Column.key` is honored in both cases: - -:: - - # with 0.8 - table1 = Table("t1", metadata, Column("col1", Integer, key="column_one")) - s = select([table1]) - s.c.column_one # works - s.c.col1 # AttributeError - - s = select([table1]).apply_labels() - s.c.table1_column_one # works - s.c.table1_col1 # AttributeError - -All other behavior regarding "name" and "key" are the same, -including that the rendered SQL will still use the form -``_`` - the emphasis here was on -preventing the :attr:`_schema.Column.key` contents from being rendered into the -``SELECT`` statement so that there are no issues with -special/ non-ascii characters used in the :attr:`_schema.Column.key`. - -:ticket:`2397` - -single_parent warning is now an error -------------------------------------- - -A :func:`_orm.relationship` that is many-to-one or many-to-many and -specifies "cascade='all, delete-orphan'", which is an -awkward but nonetheless supported use case (with -restrictions) will now raise an error if the relationship -does not specify the ``single_parent=True`` option. -Previously it would only emit a warning, but a failure would -follow almost immediately within the attribute system in any -case. - -:ticket:`2405` - -Adding the ``inspector`` argument to the ``column_reflect`` event ------------------------------------------------------------------ - -0.7 added a new event called ``column_reflect``, provided so -that the reflection of columns could be augmented as each -one were reflected. We got this event slightly wrong in -that the event gave no way to get at the current -``Inspector`` and ``Connection`` being used for the -reflection, in the case that additional information from the -database is needed. As this is a new event not widely used -yet, we'll be adding the ``inspector`` argument into it -directly:: - - @event.listens_for(Table, "column_reflect") - def listen_for_col(inspector, table, column_info): - ... - -:ticket:`2418` - -Disabling auto-detect of collations, casing for MySQL ------------------------------------------------------ - -The MySQL dialect does two calls, one very expensive, to -load all possible collations from the database as well as -information on casing, the first time an ``Engine`` -connects. Neither of these collections are used for any -SQLAlchemy functions, so these calls will be changed to no -longer be emitted automatically. Applications that might -have relied on these collections being present on -``engine.dialect`` will need to call upon -``_detect_collations()`` and ``_detect_casing()`` directly. - -:ticket:`2404` - -"Unconsumed column names" warning becomes an exception ------------------------------------------------------- - -Referring to a non-existent column in an ``insert()`` or -``update()`` construct will raise an error instead of a -warning: - -:: - - t1 = table("t1", column("x")) - t1.insert().values(x=5, z=5) # raises "Unconsumed column names: z" - -:ticket:`2415` - -Inspector.get_primary_keys() is deprecated, use Inspector.get_pk_constraint ---------------------------------------------------------------------------- - -These two methods on ``Inspector`` were redundant, where -``get_primary_keys()`` would return the same information as -``get_pk_constraint()`` minus the name of the constraint: - -:: - - >>> insp.get_primary_keys() - ["a", "b"] - - >>> insp.get_pk_constraint() - {"name":"pk_constraint", "constrained_columns":["a", "b"]} - -:ticket:`2422` - -Case-insensitive result row names will be disabled in most cases ----------------------------------------------------------------- - -A very old behavior, the column names in ``RowProxy`` were -always compared case-insensitively: - -:: - - >>> row = result.fetchone() - >>> row["foo"] == row["FOO"] == row["Foo"] - True - -This was for the benefit of a few dialects which in the -early days needed this, like Oracle and Firebird, but in -modern usage we have more accurate ways of dealing with the -case-insensitive behavior of these two platforms. - -Going forward, this behavior will be available only -optionally, by passing the flag ```case_sensitive=False``` -to ```create_engine()```, but otherwise column names -requested from the row must match as far as casing. - -:ticket:`2423` - -``InstrumentationManager`` and alternate class instrumentation is now an extension ----------------------------------------------------------------------------------- - -The ``sqlalchemy.orm.interfaces.InstrumentationManager`` -class is moved to -``sqlalchemy.ext.instrumentation.InstrumentationManager``. -The "alternate instrumentation" system was built for the -benefit of a very small number of installations that needed -to work with existing or unusual class instrumentation -systems, and generally is very seldom used. The complexity -of this system has been exported to an ``ext.`` module. It -remains unused until once imported, typically when a third -party library imports ``InstrumentationManager``, at which -point it is injected back into ``sqlalchemy.orm`` by -replacing the default ``InstrumentationFactory`` with -``ExtendedInstrumentationRegistry``. - -Removed -======= - -SQLSoup -------- - -SQLSoup is a handy package that presents an alternative -interface on top of the SQLAlchemy ORM. SQLSoup is now -moved into its own project and documented/released -separately; see https://bitbucket.org/zzzeek/sqlsoup. - -SQLSoup is a very simple tool that could also benefit from -contributors who are interested in its style of usage. - -:ticket:`2262` - -MutableType ------------ - -The older "mutable" system within the SQLAlchemy ORM has -been removed. This refers to the ``MutableType`` interface -which was applied to types such as ``PickleType`` and -conditionally to ``TypeDecorator``, and since very early -SQLAlchemy versions has provided a way for the ORM to detect -changes in so-called "mutable" data structures such as JSON -structures and pickled objects. However, the -implementation was never reasonable and forced a very -inefficient mode of usage on the unit-of-work which caused -an expensive scan of all objects to take place during flush. -In 0.7, the `sqlalchemy.ext.mutable `_ extension was -introduced so that user-defined datatypes can appropriately -send events to the unit of work as changes occur. - -Today, usage of ``MutableType`` is expected to be low, as -warnings have been in place for some years now regarding its -inefficiency. - -:ticket:`2442` - -sqlalchemy.exceptions (has been sqlalchemy.exc for years) ---------------------------------------------------------- - -We had left in an alias ``sqlalchemy.exceptions`` to attempt -to make it slightly easier for some very old libraries that -hadn't yet been upgraded to use ``sqlalchemy.exc``. Some -users are still being confused by it however so in 0.8 we're -taking it out entirely to eliminate any of that confusion. - -:ticket:`2433` - diff --git a/doc/build/changelog/migration_09.rst b/doc/build/changelog/migration_09.rst deleted file mode 100644 index 287fc2c933..0000000000 --- a/doc/build/changelog/migration_09.rst +++ /dev/null @@ -1,1922 +0,0 @@ -============================== -What's New in SQLAlchemy 0.9? -============================== - -.. admonition:: About this Document - - This document describes changes between SQLAlchemy version 0.8, - undergoing maintenance releases as of May, 2013, - and SQLAlchemy version 0.9, which had its first production - release on December 30, 2013. - - Document last updated: June 10, 2015 - -Introduction -============ - -This guide introduces what's new in SQLAlchemy version 0.9, -and also documents changes which affect users migrating -their applications from the 0.8 series of SQLAlchemy to 0.9. - -Please carefully review -:ref:`behavioral_changes_orm_09` and :ref:`behavioral_changes_core_09` for -potentially backwards-incompatible changes. - -Platform Support -================ - -Targeting Python 2.6 and Up Now, Python 3 without 2to3 -------------------------------------------------------- - -The first achievement of the 0.9 release is to remove the dependency -on the 2to3 tool for Python 3 compatibility. To make this -more straightforward, the lowest Python release targeted now -is 2.6, which features a wide degree of cross-compatibility with -Python 3. All SQLAlchemy modules and unit tests are now interpreted -equally well with any Python interpreter from 2.6 forward, including -the 3.1 and 3.2 interpreters. - -:ticket:`2671` - -C Extensions Supported on Python 3 ------------------------------------ - -The C extensions have been ported to support Python 3 and now build -in both Python 2 and Python 3 environments. - -:ticket:`2161` - -.. _behavioral_changes_orm_09: - -Behavioral Changes - ORM -======================== - -.. _migration_2824: - -Composite attributes are now returned as their object form when queried on a per-attribute basis ------------------------------------------------------------------------------------------------- - -Using a :class:`_query.Query` in conjunction with a composite attribute now returns the object -type maintained by that composite, rather than being broken out into individual -columns. Using the mapping setup at :ref:`mapper_composite`:: - - >>> session.query(Vertex.start, Vertex.end).filter(Vertex.start == Point(3, 4)).all() - [(Point(x=3, y=4), Point(x=5, y=6))] - -This change is backwards-incompatible with code that expects the individual attribute -to be expanded into individual columns. To get that behavior, use the ``.clauses`` -accessor:: - - - >>> session.query(Vertex.start.clauses, Vertex.end.clauses).filter( - ... Vertex.start == Point(3, 4) - ... ).all() - [(3, 4, 5, 6)] - -.. seealso:: - - :ref:`change_2824` - -:ticket:`2824` - - -.. _migration_2736: - -:meth:`_query.Query.select_from` no longer applies the clause to corresponding entities ----------------------------------------------------------------------------------------- -The :meth:`_query.Query.select_from` method has been popularized in recent versions -as a means of controlling the first thing that a :class:`_query.Query` object -"selects from", typically for the purposes of controlling how a JOIN will -render. - -Consider the following example against the usual ``User`` mapping:: - - select_stmt = select([User]).where(User.id == 7).alias() - - q = ( - session.query(User) - .join(select_stmt, User.id == select_stmt.c.id) - .filter(User.name == "ed") - ) - -The above statement predictably renders SQL like the following: - -.. sourcecode:: sql - - SELECT "user".id AS user_id, "user".name AS user_name - FROM "user" JOIN (SELECT "user".id AS id, "user".name AS name - FROM "user" - WHERE "user".id = :id_1) AS anon_1 ON "user".id = anon_1.id - WHERE "user".name = :name_1 - -If we wanted to reverse the order of the left and right elements of the -JOIN, the documentation would lead us to believe we could use -:meth:`_query.Query.select_from` to do so:: - - q = ( - session.query(User) - .select_from(select_stmt) - .join(User, User.id == select_stmt.c.id) - .filter(User.name == "ed") - ) - -However, in version 0.8 and earlier, the above use of :meth:`_query.Query.select_from` -would apply the ``select_stmt`` to **replace** the ``User`` entity, as it -selects from the ``user`` table which is compatible with ``User``: - -.. sourcecode:: sql - - -- SQLAlchemy 0.8 and earlier... - SELECT anon_1.id AS anon_1_id, anon_1.name AS anon_1_name - FROM (SELECT "user".id AS id, "user".name AS name - FROM "user" - WHERE "user".id = :id_1) AS anon_1 JOIN "user" ON anon_1.id = anon_1.id - WHERE anon_1.name = :name_1 - -The above statement is a mess, the ON clause refers ``anon_1.id = anon_1.id``, -our WHERE clause has been replaced with ``anon_1`` as well. - -This behavior is quite intentional, but has a different use case from that -which has become popular for :meth:`_query.Query.select_from`. The above behavior -is now available by a new method known as :meth:`_query.Query.select_entity_from`. -This is a lesser used behavior that in modern SQLAlchemy is roughly equivalent -to selecting from a customized :func:`.aliased` construct:: - - select_stmt = select([User]).where(User.id == 7) - user_from_stmt = aliased(User, select_stmt.alias()) - - q = session.query(user_from_stmt).filter(user_from_stmt.name == "ed") - -So with SQLAlchemy 0.9, our query that selects from ``select_stmt`` produces -the SQL we expect: - -.. sourcecode:: sql - - -- SQLAlchemy 0.9 - SELECT "user".id AS user_id, "user".name AS user_name - FROM (SELECT "user".id AS id, "user".name AS name - FROM "user" - WHERE "user".id = :id_1) AS anon_1 JOIN "user" ON "user".id = id - WHERE "user".name = :name_1 - -The :meth:`_query.Query.select_entity_from` method will be available in SQLAlchemy -**0.8.2**, so applications which rely on the old behavior can transition -to this method first, ensure all tests continue to function, then upgrade -to 0.9 without issue. - -:ticket:`2736` - - -.. _migration_2833: - -``viewonly=True`` on ``relationship()`` prevents history from taking effect ---------------------------------------------------------------------------- - -The ``viewonly`` flag on :func:`_orm.relationship` is applied to prevent changes -to the target attribute from having any effect within the flush process. -This is achieved by eliminating the attribute from being considered during -the flush. However, up until now, changes to the attribute would still -register the parent object as "dirty" and trigger a potential flush. The change -is that the ``viewonly`` flag now prevents history from being set for the -target attribute as well. Attribute events like backrefs and user-defined events -still continue to function normally. - -The change is illustrated as follows:: - - from sqlalchemy import Column, Integer, ForeignKey, create_engine - from sqlalchemy.orm import backref, relationship, Session - from sqlalchemy.ext.declarative import declarative_base - from sqlalchemy import inspect - - Base = declarative_base() - - - class A(Base): - __tablename__ = "a" - id = Column(Integer, primary_key=True) - - - class B(Base): - __tablename__ = "b" - - id = Column(Integer, primary_key=True) - a_id = Column(Integer, ForeignKey("a.id")) - a = relationship("A", backref=backref("bs", viewonly=True)) - - - e = create_engine("sqlite://") - Base.metadata.create_all(e) - - a = A() - b = B() - - sess = Session(e) - sess.add_all([a, b]) - sess.commit() - - b.a = a - - assert b in sess.dirty - - # before 0.9.0 - # assert a in sess.dirty - # assert inspect(a).attrs.bs.history.has_changes() - - # after 0.9.0 - assert a not in sess.dirty - assert not inspect(a).attrs.bs.history.has_changes() - -:ticket:`2833` - -.. _migration_2751: - -Association Proxy SQL Expression Improvements and Fixes -------------------------------------------------------- - -The ``==`` and ``!=`` operators as implemented by an association proxy -that refers to a scalar value on a scalar relationship now produces -a more complete SQL expression, intended to take into account -the "association" row being present or not when the comparison is against -``None``. - -Consider this mapping:: - - class A(Base): - __tablename__ = "a" - - id = Column(Integer, primary_key=True) - - b_id = Column(Integer, ForeignKey("b.id"), primary_key=True) - b = relationship("B") - b_value = association_proxy("b", "value") - - - class B(Base): - __tablename__ = "b" - id = Column(Integer, primary_key=True) - value = Column(String) - -Up through 0.8, a query like the following:: - - s.query(A).filter(A.b_value == None).all() - -would produce: - -.. sourcecode:: sql - - SELECT a.id AS a_id, a.b_id AS a_b_id - FROM a - WHERE EXISTS (SELECT 1 - FROM b - WHERE b.id = a.b_id AND b.value IS NULL) - -In 0.9, it now produces: - -.. sourcecode:: sql - - SELECT a.id AS a_id, a.b_id AS a_b_id - FROM a - WHERE (EXISTS (SELECT 1 - FROM b - WHERE b.id = a.b_id AND b.value IS NULL)) OR a.b_id IS NULL - -The difference being, it not only checks ``b.value``, it also checks -if ``a`` refers to no ``b`` row at all. This will return different -results versus prior versions, for a system that uses this type of -comparison where some parent rows have no association row. - -More critically, a correct expression is emitted for ``A.b_value != None``. -In 0.8, this would return ``True`` for ``A`` rows that had no ``b``: - -.. sourcecode:: sql - - SELECT a.id AS a_id, a.b_id AS a_b_id - FROM a - WHERE NOT (EXISTS (SELECT 1 - FROM b - WHERE b.id = a.b_id AND b.value IS NULL)) - -Now in 0.9, the check has been reworked so that it ensures -the A.b_id row is present, in addition to ``B.value`` being -non-NULL: - -.. sourcecode:: sql - - SELECT a.id AS a_id, a.b_id AS a_b_id - FROM a - WHERE EXISTS (SELECT 1 - FROM b - WHERE b.id = a.b_id AND b.value IS NOT NULL) - -In addition, the ``has()`` operator is enhanced such that you can -call it against a scalar column value with no criterion only, -and it will produce criteria that checks for the association row -being present or not:: - - s.query(A).filter(A.b_value.has()).all() - -output: - -.. sourcecode:: sql - - SELECT a.id AS a_id, a.b_id AS a_b_id - FROM a - WHERE EXISTS (SELECT 1 - FROM b - WHERE b.id = a.b_id) - -This is equivalent to ``A.b.has()``, but allows one to query -against ``b_value`` directly. - -:ticket:`2751` - -.. _migration_2810: - -Association Proxy Missing Scalar returns None ---------------------------------------------- - -An association proxy from a scalar attribute to a scalar will now return -``None`` if the proxied object isn't present. This is consistent with the -fact that missing many-to-ones return None in SQLAlchemy, so should the -proxied value. E.g.:: - - from sqlalchemy import * - from sqlalchemy.orm import * - from sqlalchemy.ext.declarative import declarative_base - from sqlalchemy.ext.associationproxy import association_proxy - - Base = declarative_base() - - - class A(Base): - __tablename__ = "a" - - id = Column(Integer, primary_key=True) - b = relationship("B", uselist=False) - - bname = association_proxy("b", "name") - - - class B(Base): - __tablename__ = "b" - - id = Column(Integer, primary_key=True) - a_id = Column(Integer, ForeignKey("a.id")) - name = Column(String) - - - a1 = A() - - # this is how m2o's always have worked - assert a1.b is None - - # but prior to 0.9, this would raise AttributeError, - # now returns None just like the proxied value. - assert a1.bname is None - -:ticket:`2810` - - -.. _change_2787: - -attributes.get_history() will query from the DB by default if value not present -------------------------------------------------------------------------------- - -A bugfix regarding :func:`.attributes.get_history` allows a column-based attribute -to query out to the database for an unloaded value, assuming the ``passive`` -flag is left at its default of ``PASSIVE_OFF``. Previously, this flag would -not be honored. Additionally, a new method :meth:`.AttributeState.load_history` -is added to complement the :attr:`.AttributeState.history` attribute, which -will emit loader callables for an unloaded attribute. - -This is a small change demonstrated as follows:: - - from sqlalchemy import Column, Integer, String, create_engine, inspect - from sqlalchemy.orm import Session, attributes - from sqlalchemy.ext.declarative import declarative_base - - Base = declarative_base() - - - class A(Base): - __tablename__ = "a" - id = Column(Integer, primary_key=True) - data = Column(String) - - - e = create_engine("sqlite://", echo=True) - Base.metadata.create_all(e) - - sess = Session(e) - - a1 = A(data="a1") - sess.add(a1) - sess.commit() # a1 is now expired - - # history doesn't emit loader callables - assert inspect(a1).attrs.data.history == (None, None, None) - - # in 0.8, this would fail to load the unloaded state. - assert attributes.get_history(a1, "data") == ( - (), - [ - "a1", - ], - (), - ) - - # load_history() is now equivalent to get_history() with - # passive=PASSIVE_OFF ^ INIT_OK - assert inspect(a1).attrs.data.load_history() == ( - (), - [ - "a1", - ], - (), - ) - -:ticket:`2787` - -.. _behavioral_changes_core_09: - -Behavioral Changes - Core -========================= - -Type objects no longer accept ignored keyword arguments -------------------------------------------------------- - -Up through the 0.8 series, most type objects accepted arbitrary keyword -arguments which were silently ignored:: - - from sqlalchemy import Date, Integer - - # storage_format argument here has no effect on any backend; - # it needs to be on the SQLite-specific type - d = Date(storage_format="%(day)02d.%(month)02d.%(year)04d") - - # display_width argument here has no effect on any backend; - # it needs to be on the MySQL-specific type - i = Integer(display_width=5) - -This was a very old bug for which a deprecation warning was added to the -0.8 series, but because nobody ever runs Python with the "-W" flag, it -was mostly never seen: - -.. sourcecode:: text - - $ python -W always::DeprecationWarning ~/dev/sqlalchemy/test.py - /Users/classic/dev/sqlalchemy/test.py:5: SADeprecationWarning: Passing arguments to - type object constructor is deprecated - d = Date(storage_format="%(day)02d.%(month)02d.%(year)04d") - /Users/classic/dev/sqlalchemy/test.py:9: SADeprecationWarning: Passing arguments to - type object constructor is deprecated - i = Integer(display_width=5) - -As of the 0.9 series the "catch all" constructor is removed from -:class:`.TypeEngine`, and these meaningless arguments are no longer accepted. - -The correct way to make use of dialect-specific arguments such as -``storage_format`` and ``display_width`` is to use the appropriate -dialect-specific types:: - - from sqlalchemy.dialects.sqlite import DATE - from sqlalchemy.dialects.mysql import INTEGER - - d = DATE(storage_format="%(day)02d.%(month)02d.%(year)04d") - - i = INTEGER(display_width=5) - -What about the case where we want the dialect-agnostic type also? We -use the :meth:`.TypeEngine.with_variant` method:: - - from sqlalchemy import Date, Integer - from sqlalchemy.dialects.sqlite import DATE - from sqlalchemy.dialects.mysql import INTEGER - - d = Date().with_variant( - DATE(storage_format="%(day)02d.%(month)02d.%(year)04d"), "sqlite" - ) - - i = Integer().with_variant(INTEGER(display_width=5), "mysql") - -:meth:`.TypeEngine.with_variant` isn't new, it was added in SQLAlchemy -0.7.2. So code that is running on the 0.8 series can be corrected to use -this approach and tested before upgrading to 0.9. - -``None`` can no longer be used as a "partial AND" constructor --------------------------------------------------------------- - -``None`` can no longer be used as the "backstop" to form an AND condition piecemeal. -This pattern was not a documented pattern even though some SQLAlchemy internals -made use of it:: - - condition = None - - for cond in conditions: - condition = condition & cond - - if condition is not None: - stmt = stmt.where(condition) - -The above sequence, when ``conditions`` is non-empty, will on 0.9 produce -``SELECT .. WHERE AND NULL``. The ``None`` is no longer implicitly -ignored, and is instead consistent with when ``None`` is interpreted in other -contexts besides that of a conjunction. - -The correct code for both 0.8 and 0.9 should read:: - - from sqlalchemy.sql import and_ - - if conditions: - stmt = stmt.where(and_(*conditions)) - -Another variant that works on all backends on 0.9, but on 0.8 only works on -backends that support boolean constants:: - - from sqlalchemy.sql import true - - condition = true() - - for cond in conditions: - condition = cond & condition - - stmt = stmt.where(condition) - -On 0.8, this will produce a SELECT statement that always has ``AND true`` -in the WHERE clause, which is not accepted by backends that don't support -boolean constants (MySQL, MSSQL). On 0.9, the ``true`` constant will be dropped -within an ``and_()`` conjunction. - -.. seealso:: - - :ref:`migration_2804` - -.. _migration_2873: - -The "password" portion of a ``create_engine()`` no longer considers the ``+`` sign as an encoded space ------------------------------------------------------------------------------------------------------- - -For whatever reason, the Python function ``unquote_plus()`` was applied to the -"password" field of a URL, which is an incorrect application of the -encoding rules described in `RFC 1738 `_ -in that it escaped spaces as plus signs. The stringification of a URL -now only encodes ":", "@", or "/" and nothing else, and is now applied to both the -``username`` and ``password`` fields (previously it only applied to the -password). On parsing, encoded characters are converted, but plus signs and -spaces are passed through as is: - -.. sourcecode:: text - - # password: "pass word + other:words" - dbtype://user:pass word + other%3Awords@host/dbname - - # password: "apples/oranges" - dbtype://username:apples%2Foranges@hostspec/database - - # password: "apples@oranges@@" - dbtype://username:apples%40oranges%40%40@hostspec/database - - # password: '', username is "username@" - dbtype://username%40:@hostspec/database - - -:ticket:`2873` - -.. _migration_2879: - -The precedence rules for COLLATE have been changed --------------------------------------------------- - -Previously, an expression like the following:: - - print((column("x") == "somevalue").collate("en_EN")) - -would produce an expression like this: - -.. sourcecode:: sql - - -- 0.8 behavior - (x = :x_1) COLLATE en_EN - -The above is misunderstood by MSSQL and is generally not the syntax suggested -for any database. The expression will now produce the syntax illustrated -by that of most database documentation: - -.. sourcecode:: sql - - -- 0.9 behavior - x = :x_1 COLLATE en_EN - -The potentially backwards incompatible change arises if the -:meth:`.ColumnOperators.collate` operator is being applied to the right-hand -column, as follows:: - - print(column("x") == literal("somevalue").collate("en_EN")) - -In 0.8, this produces: - -.. sourcecode:: sql - - x = :param_1 COLLATE en_EN - -However in 0.9, will now produce the more accurate, but probably not what you -want, form of: - -.. sourcecode:: sql - - x = (:param_1 COLLATE en_EN) - -The :meth:`.ColumnOperators.collate` operator now works more appropriately within an -``ORDER BY`` expression as well, as a specific precedence has been given to the -``ASC`` and ``DESC`` operators which will again ensure no parentheses are -generated: - -.. sourcecode:: pycon+sql - - >>> # 0.8 - >>> print(column("x").collate("en_EN").desc()) - {printsql}(x COLLATE en_EN) DESC{stop} - - >>> # 0.9 - >>> print(column("x").collate("en_EN").desc()) - {printsql}x COLLATE en_EN DESC{stop} - -:ticket:`2879` - - - -.. _migration_2878: - -PostgreSQL CREATE TYPE AS ENUM now applies quoting to values ----------------------------------------------------------------- - -The :class:`_postgresql.ENUM` type will now apply escaping to single quote -signs within the enumerated values: - -.. sourcecode:: pycon+sql - - >>> from sqlalchemy.dialects import postgresql - >>> type = postgresql.ENUM("one", "two", "three's", name="myenum") - >>> from sqlalchemy.dialects.postgresql import base - >>> print(base.CreateEnumType(type).compile(dialect=postgresql.dialect())) - {printsql}CREATE TYPE myenum AS ENUM ('one','two','three''s') - -Existing workarounds which already escape single quote signs will need to be -modified, else they will now double-escape. - -:ticket:`2878` - -New Features -============ - -.. _feature_2268: - -Event Removal API ------------------ - -Events established using :func:`.event.listen` or :func:`.event.listens_for` -can now be removed using the new :func:`.event.remove` function. The ``target``, -``identifier`` and ``fn`` arguments sent to :func:`.event.remove` need to match -exactly those which were sent for listening, and the event will be removed -from all locations in which it had been established:: - - @event.listens_for(MyClass, "before_insert", propagate=True) - def my_before_insert(mapper, connection, target): - """listen for before_insert""" - # ... - - - event.remove(MyClass, "before_insert", my_before_insert) - -In the example above, the ``propagate=True`` flag is set. This -means ``my_before_insert()`` is established as a listener for ``MyClass`` -as well as all subclasses of ``MyClass``. -The system tracks everywhere that the ``my_before_insert()`` -listener function had been placed as a result of this call and removes it as -a result of calling :func:`.event.remove`. - -The removal system uses a registry to associate arguments passed to -:func:`.event.listen` with collections of event listeners, which are in many -cases wrapped versions of the original user-supplied function. This registry -makes heavy use of weak references in order to allow all the contained contents, -such as listener targets, to be garbage collected when they go out of scope. - -:ticket:`2268` - -.. _feature_1418: - -New Query Options API; ``load_only()`` option ---------------------------------------------- - -The system of loader options such as :func:`_orm.joinedload`, -:func:`_orm.subqueryload`, :func:`_orm.lazyload`, :func:`_orm.defer`, etc. -all build upon a new system known as :class:`_orm.Load`. :class:`_orm.Load` provides -a "method chained" (a.k.a. :term:`generative`) approach to loader options, so that -instead of joining together long paths using dots or multiple attribute names, -an explicit loader style is given for each path. - -While the new way is slightly more verbose, it is simpler to understand -in that there is no ambiguity in what options are being applied to which paths; -it simplifies the method signatures of the options and provides greater flexibility -particularly for column-based options. The old systems are to remain functional -indefinitely as well and all styles can be mixed. - -**Old Way** - -To set a certain style of loading along every link in a multi-element path, the ``_all()`` -option has to be used:: - - query(User).options(joinedload_all("orders.items.keywords")) - -**New Way** - -Loader options are now chainable, so the same ``joinedload(x)`` method is applied -equally to each link, without the need to keep straight between -:func:`_orm.joinedload` and :func:`_orm.joinedload_all`:: - - query(User).options(joinedload("orders").joinedload("items").joinedload("keywords")) - -**Old Way** - -Setting an option on path that is based on a subclass requires that all -links in the path be spelled out as class bound attributes, since the -:meth:`.PropComparator.of_type` method needs to be called:: - - session.query(Company).options( - subqueryload_all(Company.employees.of_type(Engineer), Engineer.machines) - ) - -**New Way** - -Only those elements in the path that actually need :meth:`.PropComparator.of_type` -need to be set as a class-bound attribute, string-based names can be resumed -afterwards:: - - session.query(Company).options( - subqueryload(Company.employees.of_type(Engineer)).subqueryload("machines") - ) - -**Old Way** - -Setting the loader option on the last link in a long path uses a syntax -that looks a lot like it should be setting the option for all links in the -path, causing confusion:: - - query(User).options(subqueryload("orders.items.keywords")) - -**New Way** - -A path can now be spelled out using :func:`.defaultload` for entries in the -path where the existing loader style should be unchanged. More verbose -but the intent is clearer:: - - query(User).options(defaultload("orders").defaultload("items").subqueryload("keywords")) - -The dotted style can still be taken advantage of, particularly in the case -of skipping over several path elements:: - - query(User).options(defaultload("orders.items").subqueryload("keywords")) - -**Old Way** - -The :func:`.defer` option on a path needed to be spelled out with the full -path for each column:: - - query(User).options(defer("orders.description"), defer("orders.isopen")) - -**New Way** - -A single :class:`_orm.Load` object that arrives at the target path can have -:meth:`_orm.Load.defer` called upon it repeatedly:: - - query(User).options(defaultload("orders").defer("description").defer("isopen")) - -The Load Class -^^^^^^^^^^^^^^^ - -The :class:`_orm.Load` class can be used directly to provide a "bound" target, -especially when multiple parent entities are present:: - - from sqlalchemy.orm import Load - - query(User, Address).options(Load(Address).joinedload("entries")) - -Load Only -^^^^^^^^^ - -A new option :func:`.load_only` achieves a "defer everything but" style of load, -loading only the given columns and deferring the rest:: - - from sqlalchemy.orm import load_only - - query(User).options(load_only("name", "fullname")) - - # specify explicit parent entity - query(User, Address).options(Load(User).load_only("name", "fullname")) - - # specify path - query(User).options(joinedload(User.addresses).load_only("email_address")) - -Class-specific Wildcards -^^^^^^^^^^^^^^^^^^^^^^^^^ - -Using :class:`_orm.Load`, a wildcard may be used to set the loading for all -relationships (or perhaps columns) on a given entity, without affecting any -others:: - - # lazyload all User relationships - query(User).options(Load(User).lazyload("*")) - - # undefer all User columns - query(User).options(Load(User).undefer("*")) - - # lazyload all Address relationships - query(User).options(defaultload(User.addresses).lazyload("*")) - - # undefer all Address columns - query(User).options(defaultload(User.addresses).undefer("*")) - -:ticket:`1418` - - -.. _feature_2877: - -New ``text()`` Capabilities ---------------------------- - -The :func:`_expression.text` construct gains new methods: - -* :meth:`_expression.TextClause.bindparams` allows bound parameter types and values - to be set flexibly:: - - # setup values - stmt = text( - "SELECT id, name FROM user WHERE name=:name AND timestamp=:timestamp" - ).bindparams(name="ed", timestamp=datetime(2012, 11, 10, 15, 12, 35)) - - # setup types and/or values - stmt = ( - text("SELECT id, name FROM user WHERE name=:name AND timestamp=:timestamp") - .bindparams(bindparam("name", value="ed"), bindparam("timestamp", type_=DateTime())) - .bindparam(timestamp=datetime(2012, 11, 10, 15, 12, 35)) - ) - -* :meth:`_expression.TextClause.columns` supersedes the ``typemap`` option - of :func:`_expression.text`, returning a new construct :class:`.TextAsFrom`:: - - # turn a text() into an alias(), with a .c. collection: - stmt = text("SELECT id, name FROM user").columns(id=Integer, name=String) - stmt = stmt.alias() - - stmt = select([addresses]).select_from( - addresses.join(stmt), addresses.c.user_id == stmt.c.id - ) - - - # or into a cte(): - stmt = text("SELECT id, name FROM user").columns(id=Integer, name=String) - stmt = stmt.cte("x") - - stmt = select([addresses]).select_from( - addresses.join(stmt), addresses.c.user_id == stmt.c.id - ) - -:ticket:`2877` - -.. _feature_722: - -INSERT from SELECT ------------------- - -After literally years of pointless procrastination this relatively minor -syntactical feature has been added, and is also backported to 0.8.3, -so technically isn't "new" in 0.9. A :func:`_expression.select` construct or other -compatible construct can be passed to the new method :meth:`_expression.Insert.from_select` -where it will be used to render an ``INSERT .. SELECT`` construct: - -.. sourcecode:: pycon+sql - - >>> from sqlalchemy.sql import table, column - >>> t1 = table("t1", column("a"), column("b")) - >>> t2 = table("t2", column("x"), column("y")) - >>> print(t1.insert().from_select(["a", "b"], t2.select().where(t2.c.y == 5))) - {printsql}INSERT INTO t1 (a, b) SELECT t2.x, t2.y - FROM t2 - WHERE t2.y = :y_1 - -The construct is smart enough to also accommodate ORM objects such as classes -and :class:`_query.Query` objects:: - - s = Session() - q = s.query(User.id, User.name).filter_by(name="ed") - ins = insert(Address).from_select((Address.id, Address.email_address), q) - -rendering: - -.. sourcecode:: sql - - INSERT INTO addresses (id, email_address) - SELECT users.id AS users_id, users.name AS users_name - FROM users WHERE users.name = :name_1 - -:ticket:`722` - -.. _feature_github_42: - -New FOR UPDATE support on ``select()``, ``Query()`` ---------------------------------------------------- - -An attempt is made to simplify the specification of the ``FOR UPDATE`` -clause on ``SELECT`` statements made within Core and ORM, and support is added -for the ``FOR UPDATE OF`` SQL supported by PostgreSQL and Oracle. - -Using the core :meth:`_expression.GenerativeSelect.with_for_update`, options like ``FOR SHARE`` and -``NOWAIT`` can be specified individually, rather than linking to arbitrary -string codes:: - - stmt = select([table]).with_for_update(read=True, nowait=True, of=table) - -On Posgtresql the above statement might render like: - -.. sourcecode:: sql - - SELECT table.a, table.b FROM table FOR SHARE OF table NOWAIT - -The :class:`_query.Query` object gains a similar method :meth:`_query.Query.with_for_update` -which behaves in the same way. This method supersedes the existing -:meth:`_query.Query.with_lockmode` method, which translated ``FOR UPDATE`` clauses -using a different system. At the moment, the "lockmode" string argument is still -accepted by the :meth:`.Session.refresh` method. - - -.. _feature_2867: - -Floating Point String-Conversion Precision Configurable for Native Floating Point Types ---------------------------------------------------------------------------------------- - -The conversion which SQLAlchemy does whenever a DBAPI returns a Python -floating point type which is to be converted into a Python ``Decimal()`` -necessarily involves an intermediary step which converts the floating point -value to a string. The scale used for this string conversion was previously -hardcoded to 10, and is now configurable. The setting is available on -both the :class:`.Numeric` as well as the :class:`.Float` -type, as well as all SQL- and dialect-specific descendant types, using the -parameter ``decimal_return_scale``. If the type supports a ``.scale`` parameter, -as is the case with :class:`.Numeric` and some float types such as -:class:`.mysql.DOUBLE`, the value of ``.scale`` is used as the default -for ``.decimal_return_scale`` if it is not otherwise specified. If both -``.scale`` and ``.decimal_return_scale`` are absent, then the default of -10 takes place. E.g.:: - - from sqlalchemy.dialects.mysql import DOUBLE - import decimal - - data = Table( - "data", - metadata, - Column("double_value", mysql.DOUBLE(decimal_return_scale=12, asdecimal=True)), - ) - - conn.execute( - data.insert(), - double_value=45.768392065789, - ) - result = conn.scalar(select([data.c.double_value])) - - # previously, this would typically be Decimal("45.7683920658"), - # e.g. trimmed to 10 decimal places - - # now we get 12, as requested, as MySQL can support this - # much precision for DOUBLE - assert result == decimal.Decimal("45.768392065789") - -:ticket:`2867` - - -.. _change_2824: - -Column Bundles for ORM queries ------------------------------- - -The :class:`.Bundle` allows for querying of sets of columns, which are then -grouped into one name under the tuple returned by the query. The initial -purposes of :class:`.Bundle` are 1. to allow "composite" ORM columns to be -returned as a single value in a column-based result set, rather than expanding -them out into individual columns and 2. to allow the creation of custom result-set -constructs within the ORM, using ad-hoc columns and return types, without involving -the more heavyweight mechanics of mapped classes. - -.. seealso:: - - :ref:`migration_2824` - - :ref:`bundles` - -:ticket:`2824` - - -Server Side Version Counting ------------------------------ - -The versioning feature of the ORM (now also documented at :ref:`mapper_version_counter`) -can now make use of server-side version counting schemes, such as those produced -by triggers or database system columns, as well as conditional programmatic schemes outside -of the version_id_counter function itself. By providing the value ``False`` -to the ``version_id_generator`` parameter, the ORM will use the already-set version -identifier, or alternatively fetch the version identifier -from each row at the same time the INSERT or UPDATE is emitted. When using a -server-generated version identifier, it is strongly -recommended that this feature be used only on a backend with strong RETURNING -support (PostgreSQL, SQL Server; Oracle also supports RETURNING but the cx_oracle -driver has only limited support), else the additional SELECT statements will -add significant performance -overhead. The example provided at :ref:`server_side_version_counter` illustrates -the usage of the PostgreSQL ``xmin`` system column in order to integrate it with -the ORM's versioning feature. - -.. seealso:: - - :ref:`server_side_version_counter` - -:ticket:`2793` - -.. _feature_1535: - -``include_backrefs=False`` option for ``@validates`` ----------------------------------------------------- - -The :func:`.validates` function now accepts an option ``include_backrefs=True``, -which will bypass firing the validator for the case where the event initiated -from a backref:: - - from sqlalchemy import Column, Integer, ForeignKey - from sqlalchemy.orm import relationship, validates - from sqlalchemy.ext.declarative import declarative_base - - Base = declarative_base() - - - class A(Base): - __tablename__ = "a" - - id = Column(Integer, primary_key=True) - bs = relationship("B", backref="a") - - @validates("bs") - def validate_bs(self, key, item): - print("A.bs validator") - return item - - - class B(Base): - __tablename__ = "b" - - id = Column(Integer, primary_key=True) - a_id = Column(Integer, ForeignKey("a.id")) - - @validates("a", include_backrefs=False) - def validate_a(self, key, item): - print("B.a validator") - return item - - - a1 = A() - a1.bs.append(B()) # prints only "A.bs validator" - -:ticket:`1535` - - -PostgreSQL JSON Type --------------------- - -The PostgreSQL dialect now features a :class:`_postgresql.JSON` type to -complement the :class:`_postgresql.HSTORE` type. - -.. seealso:: - - :class:`_postgresql.JSON` - -:ticket:`2581` - -.. _feature_automap: - -Automap Extension ------------------ - -A new extension is added in **0.9.1** known as :mod:`sqlalchemy.ext.automap`. This is an -**experimental** extension which expands upon the functionality of Declarative -as well as the :class:`.DeferredReflection` class. Essentially, the extension -provides a base class :class:`.AutomapBase` which automatically generates -mapped classes and relationships between them based on given table metadata. - -The :class:`_schema.MetaData` in use normally might be produced via reflection, but -there is no requirement that reflection is used. The most basic usage -illustrates how :mod:`sqlalchemy.ext.automap` is able to deliver mapped -classes, including relationships, based on a reflected schema:: - - from sqlalchemy.ext.automap import automap_base - from sqlalchemy.orm import Session - from sqlalchemy import create_engine - - Base = automap_base() - - # engine, suppose it has two tables 'user' and 'address' set up - engine = create_engine("sqlite:///mydatabase.db") - - # reflect the tables - Base.prepare(engine, reflect=True) - - # mapped classes are now created with names matching that of the table - # name. - User = Base.classes.user - Address = Base.classes.address - - session = Session(engine) - - # rudimentary relationships are produced - session.add(Address(email_address="foo@bar.com", user=User(name="foo"))) - session.commit() - - # collection-based relationships are by default named "_collection" - print(u1.address_collection) - -Beyond that, the :class:`.AutomapBase` class is a declarative base, and supports -all the features that declarative does. The "automapping" feature can be used -with an existing, explicitly declared schema to generate relationships and -missing classes only. Naming schemes and relationship-production routines -can be dropped in using callable functions. - -It is hoped that the :class:`.AutomapBase` system provides a quick -and modernized solution to the problem that the very famous -`SQLSoup `_ -also tries to solve, that of generating a quick and rudimentary object -model from an existing database on the fly. By addressing the issue strictly -at the mapper configuration level, and integrating fully with existing -Declarative class techniques, :class:`.AutomapBase` seeks to provide -a well-integrated approach to the issue of expediently auto-generating ad-hoc -mappings. - -.. seealso:: - - :ref:`automap_toplevel` - -Behavioral Improvements -======================= - -Improvements that should produce no compatibility issues except in exceedingly -rare and unusual hypothetical cases, but are good to be aware of in case there are -unexpected issues. - -.. _feature_joins_09: - -Many JOIN and LEFT OUTER JOIN expressions will no longer be wrapped in (SELECT * FROM ..) AS ANON_1 ---------------------------------------------------------------------------------------------------- - -For many years, the SQLAlchemy ORM has been held back from being able to nest -a JOIN inside the right side of an existing JOIN (typically a LEFT OUTER JOIN, -as INNER JOINs could always be flattened): - -.. sourcecode:: sql - - SELECT a.*, b.*, c.* FROM a LEFT OUTER JOIN (b JOIN c ON b.id = c.id) ON a.id - -This was due to the fact that SQLite up until version **3.7.16** cannot parse a statement of the above format: - -.. sourcecode:: text - - SQLite version 3.7.15.2 2013-01-09 11:53:05 - Enter ".help" for instructions - Enter SQL statements terminated with a ";" - sqlite> create table a(id integer); - sqlite> create table b(id integer); - sqlite> create table c(id integer); - sqlite> select a.id, b.id, c.id from a left outer join (b join c on b.id=c.id) on b.id=a.id; - Error: no such column: b.id - -Right-outer-joins are of course another way to work around right-side -parenthesization; this would be significantly complicated and visually unpleasant -to implement, but fortunately SQLite doesn't support RIGHT OUTER JOIN either :): - -.. sourcecode:: sql - - sqlite> select a.id, b.id, c.id from b join c on b.id=c.id - ...> right outer join a on b.id=a.id; - Error: RIGHT and FULL OUTER JOINs are not currently supported - -Back in 2005, it wasn't clear if other databases had trouble with this form, -but today it seems clear every database tested except SQLite now supports it -(Oracle 8, a very old database, doesn't support the JOIN keyword at all, -but SQLAlchemy has always had a simple rewriting scheme in place for Oracle's syntax). -To make matters worse, SQLAlchemy's usual workaround of applying a -SELECT often degrades performance on platforms like PostgreSQL and MySQL: - -.. sourcecode:: sql - - SELECT a.*, anon_1.* FROM a LEFT OUTER JOIN ( - SELECT b.id AS b_id, c.id AS c_id - FROM b JOIN c ON b.id = c.id - ) AS anon_1 ON a.id=anon_1.b_id - -A JOIN like the above form is commonplace when working with joined-table inheritance structures; -any time :meth:`_query.Query.join` is used to join from some parent to a joined-table subclass, or -when :func:`_orm.joinedload` is used similarly, SQLAlchemy's ORM would always make sure a nested -JOIN was never rendered, lest the query wouldn't be able to run on SQLite. Even though -the Core has always supported a JOIN of the more compact form, the ORM had to avoid it. - -An additional issue would arise when producing joins across many-to-many relationships -where special criteria is present in the ON clause. Consider an eager load join like the following:: - - session.query(Order).outerjoin(Order.items) - -Assuming a many-to-many from ``Order`` to ``Item`` which actually refers to a subclass -like ``Subitem``, the SQL for the above would look like: - -.. sourcecode:: sql - - SELECT order.id, order.name - FROM order LEFT OUTER JOIN order_item ON order.id = order_item.order_id - LEFT OUTER JOIN item ON order_item.item_id = item.id AND item.type = 'subitem' - -What's wrong with the above query? Basically, that it will load many ``order`` / -``order_item`` rows where the criteria of ``item.type == 'subitem'`` is not true. - -As of SQLAlchemy 0.9, an entirely new approach has been taken. The ORM no longer -worries about nesting JOINs in the right side of an enclosing JOIN, and it now will -render these as often as possible while still returning the correct results. When -the SQL statement is passed to be compiled, the **dialect compiler** will **rewrite the join** -to suit the target backend, if that backend is known to not support a right-nested -JOIN (which currently is only SQLite - if other backends have this issue please -let us know!). - -So a regular ``query(Parent).join(Subclass)`` will now usually produce a simpler -expression: - -.. sourcecode:: sql - - SELECT parent.id AS parent_id - FROM parent JOIN ( - base_table JOIN subclass_table - ON base_table.id = subclass_table.id) ON parent.id = base_table.parent_id - -Joined eager loads like ``query(Parent).options(joinedload(Parent.subclasses))`` -will alias the individual tables instead of wrapping in an ``ANON_1``: - -.. sourcecode:: sql - - SELECT parent.*, base_table_1.*, subclass_table_1.* FROM parent - LEFT OUTER JOIN ( - base_table AS base_table_1 JOIN subclass_table AS subclass_table_1 - ON base_table_1.id = subclass_table_1.id) - ON parent.id = base_table_1.parent_id - -Many-to-many joins and eagerloads will right nest the "secondary" and "right" tables: - -.. sourcecode:: sql - - SELECT order.id, order.name - FROM order LEFT OUTER JOIN - (order_item JOIN item ON order_item.item_id = item.id AND item.type = 'subitem') - ON order_item.order_id = order.id - -All of these joins, when rendered with a :class:`_expression.Select` statement that specifically -specifies ``use_labels=True``, which is true for all the queries the ORM emits, -are candidates for "join rewriting", which is the process of rewriting all those right-nested -joins into nested SELECT statements, while maintaining the identical labeling used by -the :class:`_expression.Select`. So SQLite, the one database that won't support this very -common SQL syntax even in 2013, shoulders the extra complexity itself, -with the above queries rewritten as: - -.. sourcecode:: sql - - -- sqlite only! - SELECT parent.id AS parent_id - FROM parent JOIN ( - SELECT base_table.id AS base_table_id, - base_table.parent_id AS base_table_parent_id, - subclass_table.id AS subclass_table_id - FROM base_table JOIN subclass_table ON base_table.id = subclass_table.id - ) AS anon_1 ON parent.id = anon_1.base_table_parent_id - - -- sqlite only! - SELECT parent.id AS parent_id, anon_1.subclass_table_1_id AS subclass_table_1_id, - anon_1.base_table_1_id AS base_table_1_id, - anon_1.base_table_1_parent_id AS base_table_1_parent_id - FROM parent LEFT OUTER JOIN ( - SELECT base_table_1.id AS base_table_1_id, - base_table_1.parent_id AS base_table_1_parent_id, - subclass_table_1.id AS subclass_table_1_id - FROM base_table AS base_table_1 - JOIN subclass_table AS subclass_table_1 ON base_table_1.id = subclass_table_1.id - ) AS anon_1 ON parent.id = anon_1.base_table_1_parent_id - - -- sqlite only! - SELECT "order".id AS order_id - FROM "order" LEFT OUTER JOIN ( - SELECT order_item_1.order_id AS order_item_1_order_id, - order_item_1.item_id AS order_item_1_item_id, - item.id AS item_id, item.type AS item_type - FROM order_item AS order_item_1 - JOIN item ON item.id = order_item_1.item_id AND item.type IN (?) - ) AS anon_1 ON "order".id = anon_1.order_item_1_order_id - -.. note:: - - As of SQLAlchemy 1.1, the workarounds present in this feature for SQLite - will automatically disable themselves when SQLite version **3.7.16** - or greater is detected, as SQLite has repaired support for right-nested joins. - -The :meth:`_expression.Join.alias`, :func:`.aliased` and :func:`.with_polymorphic` functions now -support a new argument, ``flat=True``, which is used to construct aliases of joined-table -entities without embedding into a SELECT. This flag is not on by default, to help with -backwards compatibility - but now a "polymorphic" selectable can be joined as a target -without any subqueries generated:: - - employee_alias = with_polymorphic(Person, [Engineer, Manager], flat=True) - - session.query(Company).join(Company.employees.of_type(employee_alias)).filter( - or_(Engineer.primary_language == "python", Manager.manager_name == "dilbert") - ) - -Generates (everywhere except SQLite): - -.. sourcecode:: sql - - SELECT companies.company_id AS companies_company_id, companies.name AS companies_name - FROM companies JOIN ( - people AS people_1 - LEFT OUTER JOIN engineers AS engineers_1 ON people_1.person_id = engineers_1.person_id - LEFT OUTER JOIN managers AS managers_1 ON people_1.person_id = managers_1.person_id - ) ON companies.company_id = people_1.company_id - WHERE engineers.primary_language = %(primary_language_1)s - OR managers.manager_name = %(manager_name_1)s - -:ticket:`2369` :ticket:`2587` - -.. _feature_2976: - -Right-nested inner joins available in joined eager loads ---------------------------------------------------------- - -As of version 0.9.4, the above mentioned right-nested joining can be enabled -in the case of a joined eager load where an "outer" join is linked to an "inner" -on the right side. - -Normally, a joined eager load chain like the following:: - - query(User).options( - joinedload("orders", innerjoin=False).joinedload("items", innerjoin=True) - ) - -Would not produce an inner join; because of the LEFT OUTER JOIN from user->order, -joined eager loading could not use an INNER join from order->items without changing -the user rows that are returned, and would instead ignore the "chained" ``innerjoin=True`` -directive. How 0.9.0 should have delivered this would be that instead of: - -.. sourcecode:: sql - - FROM users LEFT OUTER JOIN orders ON LEFT OUTER JOIN items ON - -the new "right-nested joins are OK" logic would kick in, and we'd get: - -.. sourcecode:: sql - - FROM users LEFT OUTER JOIN (orders JOIN items ON ) ON - -Since we missed the boat on that, to avoid further regressions we've added the above -functionality by specifying the string ``"nested"`` to :paramref:`_orm.joinedload.innerjoin`:: - - query(User).options( - joinedload("orders", innerjoin=False).joinedload("items", innerjoin="nested") - ) - -This feature is new in 0.9.4. - -:ticket:`2976` - - - -ORM can efficiently fetch just-generated INSERT/UPDATE defaults using RETURNING -------------------------------------------------------------------------------- - -The :class:`_orm.Mapper` has long supported an undocumented flag known as -``eager_defaults=True``. The effect of this flag is that when an INSERT or UPDATE -proceeds, and the row is known to have server-generated default values, -a SELECT would immediately follow it in order to "eagerly" load those new values. -Normally, the server-generated columns are marked as "expired" on the object, -so that no overhead is incurred unless the application actually accesses these -columns soon after the flush. The ``eager_defaults`` flag was therefore not -of much use as it could only decrease performance, and was present only to support -exotic event schemes where users needed default values to be available -immediately within the flush process. - -In 0.9, as a result of the version id enhancements, ``eager_defaults`` can now -emit a RETURNING clause for these values, so on a backend with strong RETURNING -support in particular PostgreSQL, the ORM can fetch newly generated default -and SQL expression values inline with the INSERT or UPDATE. ``eager_defaults``, -when enabled, makes use of RETURNING automatically when the target backend -and :class:`_schema.Table` supports "implicit returning". - -.. _change_2836: - -Subquery Eager Loading will apply DISTINCT to the innermost SELECT for some queries ------------------------------------------------------------------------------------- - -In an effort to reduce the number of duplicate rows that can be generated -by subquery eager loading when a many-to-one relationship is involved, a -DISTINCT keyword will be applied to the innermost SELECT when the join is -targeting columns that do not comprise the primary key, as in when loading -along a many to one. - -That is, when subquery loading on a many-to-one from A->B: - -.. sourcecode:: sql - - SELECT b.id AS b_id, b.name AS b_name, anon_1.b_id AS a_b_id - FROM (SELECT DISTINCT a_b_id FROM a) AS anon_1 - JOIN b ON b.id = anon_1.a_b_id - -Since ``a.b_id`` is a non-distinct foreign key, DISTINCT is applied so that -redundant ``a.b_id`` are eliminated. The behavior can be turned on or off -unconditionally for a particular :func:`_orm.relationship` using the flag -``distinct_target_key``, setting the value to ``True`` for unconditionally -on, ``False`` for unconditionally off, and ``None`` for the feature to take -effect when the target SELECT is against columns that do not comprise a full -primary key. In 0.9, ``None`` is the default. - -The option is also backported to 0.8 where the ``distinct_target_key`` -option defaults to ``False``. - -While the feature here is designed to help performance by eliminating -duplicate rows, the ``DISTINCT`` keyword in SQL itself can have a negative -performance impact. If columns in the SELECT are not indexed, ``DISTINCT`` -will likely perform an ``ORDER BY`` on the rowset which can be expensive. -By keeping the feature limited just to foreign keys which are hopefully -indexed in any case, it's expected that the new defaults are reasonable. - -The feature also does not eliminate every possible dupe-row scenario; if -a many-to-one is present elsewhere in the chain of joins, dupe rows may still -be present. - -:ticket:`2836` - -.. _migration_2789: - -Backref handlers can now propagate more than one level deep ------------------------------------------------------------ - -The mechanism by which attribute events pass along their "initiator", that is -the object associated with the start of the event, has been changed; instead -of a :class:`.AttributeImpl` being passed, a new object :class:`.attributes.Event` -is passed instead; this object refers to the :class:`.AttributeImpl` as well as -to an "operation token", representing if the operation is an append, remove, -or replace operation. - -The attribute event system no longer looks at this "initiator" object in order to halt a -recursive series of attribute events. Instead, the system of preventing endless -recursion due to mutually-dependent backref handlers has been moved -to the ORM backref event handlers specifically, which now take over the role -of ensuring that a chain of mutually-dependent events (such as append to collection -A.bs, set many-to-one attribute B.a in response) doesn't go into an endless recursion -stream. The rationale here is that the backref system, given more detail and control -over event propagation, can finally allow operations more than one level deep -to occur; the typical scenario is when a collection append results in a many-to-one -replacement operation, which in turn should cause the item to be removed from a -previous collection:: - - class Parent(Base): - __tablename__ = "parent" - - id = Column(Integer, primary_key=True) - children = relationship("Child", backref="parent") - - - class Child(Base): - __tablename__ = "child" - - id = Column(Integer, primary_key=True) - parent_id = Column(ForeignKey("parent.id")) - - - p1 = Parent() - p2 = Parent() - c1 = Child() - - p1.children.append(c1) - - assert c1.parent is p1 # backref event establishes c1.parent as p1 - - p2.children.append(c1) - - assert c1.parent is p2 # backref event establishes c1.parent as p2 - assert c1 not in p1.children # second backref event removes c1 from p1.children - -Above, prior to this change, the ``c1`` object would still have been present -in ``p1.children``, even though it is also present in ``p2.children`` at the -same time; the backref handlers would have stopped at replacing ``c1.parent`` with -``p2`` instead of ``p1``. In 0.9, using the more detailed :class:`.Event` -object as well as letting the backref handlers make more detailed decisions about -these objects, the propagation can continue onto removing ``c1`` from ``p1.children`` -while maintaining a check against the propagation from going into an endless -recursive loop. - -End-user code which a. makes use of the :meth:`.AttributeEvents.set`, -:meth:`.AttributeEvents.append`, or :meth:`.AttributeEvents.remove` events, -and b. initiates further attribute modification operations as a result of these -events may need to be modified to prevent recursive loops, as the attribute system -no longer stops a chain of events from propagating endlessly in the absence of the backref -event handlers. Additionally, code which depends upon the value of the ``initiator`` -will need to be adjusted to the new API, and furthermore must be ready for the -value of ``initiator`` to change from its original value within a string of -backref-initiated events, as the backref handlers may now swap in a -new ``initiator`` value for some operations. - -:ticket:`2789` - -.. _change_2838: - -The typing system now handles the task of rendering "literal bind" values -------------------------------------------------------------------------- - -A new method is added to :class:`.TypeEngine` :meth:`.TypeEngine.literal_processor` -as well as :meth:`.TypeDecorator.process_literal_param` for :class:`.TypeDecorator` -which take on the task of rendering so-called "inline literal parameters" - parameters -that normally render as "bound" values, but are instead being rendered inline -into the SQL statement due to the compiler configuration. This feature is used -when generating DDL for constructs such as :class:`.CheckConstraint`, as well -as by Alembic when using constructs such as ``op.inline_literal()``. Previously, -a simple "isinstance" check checked for a few basic types, and the "bind processor" -was used unconditionally, leading to such issues as strings being encoded into utf-8 -prematurely. - -Custom types written with :class:`.TypeDecorator` should continue to work in -"inline literal" scenarios, as the :meth:`.TypeDecorator.process_literal_param` -falls back to :meth:`.TypeDecorator.process_bind_param` by default, as these methods -usually handle a data manipulation, not as much how the data is presented to the -database. :meth:`.TypeDecorator.process_literal_param` can be specified to -specifically produce a string representing how a value should be rendered -into an inline DDL statement. - -:ticket:`2838` - - -.. _change_2812: - -Schema identifiers now carry along their own quoting information ---------------------------------------------------------------------- - -This change simplifies the Core's usage of so-called "quote" flags, such -as the ``quote`` flag passed to :class:`_schema.Table` and :class:`_schema.Column`. The flag -is now internalized within the string name itself, which is now represented -as an instance of :class:`.quoted_name`, a string subclass. The -:class:`.IdentifierPreparer` now relies solely on the quoting preferences -reported by the :class:`.quoted_name` object rather than checking for any -explicit ``quote`` flags in most cases. The issue resolved here includes -that various case-sensitive methods such as :meth:`_engine.Engine.has_table` as well -as similar methods within dialects now function with explicitly quoted names, -without the need to complicate or introduce backwards-incompatible changes -to those APIs (many of which are 3rd party) with the details of quoting flags - -in particular, a wider range of identifiers now function correctly with the -so-called "uppercase" backends like Oracle, Firebird, and DB2 (backends that -store and report upon table and column names using all uppercase for case -insensitive names). - -The :class:`.quoted_name` object is used internally as needed; however if -other keywords require fixed quoting preferences, the class is available -publicly. - -:ticket:`2812` - -.. _migration_2804: - -Improved rendering of Boolean constants, NULL constants, conjunctions ----------------------------------------------------------------------- - -New capabilities have been added to the :func:`.true` and :func:`.false` -constants, in particular in conjunction with :func:`.and_` and :func:`.or_` -functions as well as the behavior of the WHERE/HAVING clauses in conjunction -with these types, boolean types overall, and the :func:`.null` constant. - -Starting with a table such as this:: - - from sqlalchemy import Table, Boolean, Integer, Column, MetaData - - t1 = Table("t", MetaData(), Column("x", Boolean()), Column("y", Integer)) - -A select construct will now render the boolean column as a binary expression -on backends that don't feature ``true``/``false`` constant behavior: - -.. sourcecode:: pycon+sql - - >>> from sqlalchemy import select, and_, false, true - >>> from sqlalchemy.dialects import mysql, postgresql - - >>> print(select([t1]).where(t1.c.x).compile(dialect=mysql.dialect())) - {printsql}SELECT t.x, t.y FROM t WHERE t.x = 1 - -The :func:`.and_` and :func:`.or_` constructs will now exhibit quasi -"short circuit" behavior, that is truncating a rendered expression, when a -:func:`.true` or :func:`.false` constant is present: - -.. sourcecode:: pycon+sql - - >>> print( - ... select([t1]).where(and_(t1.c.y > 5, false())).compile(dialect=postgresql.dialect()) - ... ) - {printsql}SELECT t.x, t.y FROM t WHERE false - -:func:`.true` can be used as the base to build up an expression: - -.. sourcecode:: pycon+sql - - >>> expr = true() - >>> expr = expr & (t1.c.y > 5) - >>> print(select([t1]).where(expr)) - {printsql}SELECT t.x, t.y FROM t WHERE t.y > :y_1 - -The boolean constants :func:`.true` and :func:`.false` themselves render as -``0 = 1`` and ``1 = 1`` for a backend with no boolean constants: - -.. sourcecode:: pycon+sql - - >>> print(select([t1]).where(and_(t1.c.y > 5, false())).compile(dialect=mysql.dialect())) - {printsql}SELECT t.x, t.y FROM t WHERE 0 = 1 - -Interpretation of ``None``, while not particularly valid SQL, is at least -now consistent: - -.. sourcecode:: pycon+sql - - >>> print(select([t1.c.x]).where(None)) - {printsql}SELECT t.x FROM t WHERE NULL{stop} - - >>> print(select([t1.c.x]).where(None).where(None)) - {printsql}SELECT t.x FROM t WHERE NULL AND NULL{stop} - - >>> print(select([t1.c.x]).where(and_(None, None))) - {printsql}SELECT t.x FROM t WHERE NULL AND NULL{stop} - -:ticket:`2804` - -.. _migration_1068: - -Label constructs can now render as their name alone in an ORDER BY ------------------------------------------------------------------- - -For the case where a :class:`.Label` is used in both the columns clause -as well as the ORDER BY clause of a SELECT, the label will render as -just its name in the ORDER BY clause, assuming the underlying dialect -reports support of this feature. - -E.g. an example like:: - - from sqlalchemy.sql import table, column, select, func - - t = table("t", column("c1"), column("c2")) - expr = (func.foo(t.c.c1) + t.c.c2).label("expr") - - stmt = select([expr]).order_by(expr) - - print(stmt) - -Prior to 0.9 would render as: - -.. sourcecode:: sql - - SELECT foo(t.c1) + t.c2 AS expr - FROM t ORDER BY foo(t.c1) + t.c2 - -And now renders as: - -.. sourcecode:: sql - - SELECT foo(t.c1) + t.c2 AS expr - FROM t ORDER BY expr - -The ORDER BY only renders the label if the label isn't further -embedded into an expression within the ORDER BY, other than a simple -``ASC`` or ``DESC``. - -The above format works on all databases tested, but might have -compatibility issues with older database versions (MySQL 4? Oracle 8? -etc.). Based on user reports we can add rules that will disable the -feature based on database version detection. - -:ticket:`1068` - -.. _migration_2848: - -``RowProxy`` now has tuple-sorting behavior -------------------------------------------- - -The :class:`.RowProxy` object acts much like a tuple, but up until now -would not sort as a tuple if a list of them were sorted using ``sorted()``. -The ``__eq__()`` method now compares both sides as a tuple and also -an ``__lt__()`` method has been added:: - - users.insert().execute( - dict(user_id=1, user_name="foo"), - dict(user_id=2, user_name="bar"), - dict(user_id=3, user_name="def"), - ) - - rows = users.select().order_by(users.c.user_name).execute().fetchall() - - eq_(rows, [(2, "bar"), (3, "def"), (1, "foo")]) - - eq_(sorted(rows), [(1, "foo"), (2, "bar"), (3, "def")]) - -:ticket:`2848` - -.. _migration_2850: - -A bindparam() construct with no type gets upgraded via copy when a type is available ------------------------------------------------------------------------------------- - -The logic which "upgrades" a :func:`.bindparam` construct to take on the -type of the enclosing expression has been improved in two ways. First, the -:func:`.bindparam` object is **copied** before the new type is assigned, so that -the given :func:`.bindparam` is not mutated in place. Secondly, this same -operation occurs when an :class:`_expression.Insert` or :class:`_expression.Update` construct is compiled, -regarding the "values" that were set in the statement via the :meth:`.ValuesBase.values` -method. - -If given an untyped :func:`.bindparam`:: - - bp = bindparam("some_col") - -If we use this parameter as follows:: - - expr = mytable.c.col == bp - -The type for ``bp`` remains as ``NullType``, however if ``mytable.c.col`` -is of type ``String``, then ``expr.right``, that is the right side of the -binary expression, will take on the ``String`` type. Previously, ``bp`` itself -would have been changed in place to have ``String`` as its type. - -Similarly, this operation occurs in an :class:`_expression.Insert` or :class:`_expression.Update`:: - - stmt = mytable.update().values(col=bp) - -Above, ``bp`` remains unchanged, but the ``String`` type will be used when -the statement is executed, which we can see by examining the ``binds`` dictionary:: - - >>> compiled = stmt.compile() - >>> compiled.binds["some_col"].type - String - -The feature allows custom types to take their expected effect within INSERT/UPDATE -statements without needing to explicitly specify those types within every -:func:`.bindparam` expression. - -The potentially backwards-compatible changes involve two unlikely -scenarios. Since the bound parameter is -**cloned**, users should not be relying upon making in-place changes to a -:func:`.bindparam` construct once created. Additionally, code which uses -:func:`.bindparam` within an :class:`_expression.Insert` or :class:`_expression.Update` statement -which is relying on the fact that the :func:`.bindparam` is not typed according -to the column being assigned towards will no longer function in that way. - -:ticket:`2850` - - -.. _migration_1765: - -Columns can reliably get their type from a column referred to via ForeignKey ----------------------------------------------------------------------------- - -There's a long standing behavior which says that a :class:`_schema.Column` can be -declared without a type, as long as that :class:`_schema.Column` is referred to -by a :class:`_schema.ForeignKeyConstraint`, and the type from the referenced column -will be copied into this one. The problem has been that this feature never -worked very well and wasn't maintained. The core issue was that the -:class:`_schema.ForeignKey` object doesn't know what target :class:`_schema.Column` it -refers to until it is asked, typically the first time the foreign key is used -to construct a :class:`_expression.Join`. So until that time, the parent :class:`_schema.Column` -would not have a type, or more specifically, it would have a default type -of :class:`.NullType`. - -While it's taken a long time, the work to reorganize the initialization of -:class:`_schema.ForeignKey` objects has been completed such that this feature can -finally work acceptably. At the core of the change is that the :attr:`_schema.ForeignKey.column` -attribute no longer lazily initializes the location of the target :class:`_schema.Column`; -the issue with this system was that the owning :class:`_schema.Column` would be stuck -with :class:`.NullType` as its type until the :class:`_schema.ForeignKey` happened to -be used. - -In the new version, the :class:`_schema.ForeignKey` coordinates with the eventual -:class:`_schema.Column` it will refer to using internal attachment events, so that the -moment the referencing :class:`_schema.Column` is associated with the -:class:`_schema.MetaData`, all :class:`_schema.ForeignKey` objects that -refer to it will be sent a message that they need to initialize their parent -column. This system is more complicated but works more solidly; as a bonus, -there are now tests in place for a wide variety of :class:`_schema.Column` / -:class:`_schema.ForeignKey` configuration scenarios and error messages have been -improved to be very specific to no less than seven different error conditions. - -Scenarios which now work correctly include: - -1. The type on a :class:`_schema.Column` is immediately present as soon as the - target :class:`_schema.Column` becomes associated with the same :class:`_schema.MetaData`; - this works no matter which side is configured first:: - - >>> from sqlalchemy import Table, MetaData, Column, Integer, ForeignKey - >>> metadata = MetaData() - >>> t2 = Table("t2", metadata, Column("t1id", ForeignKey("t1.id"))) - >>> t2.c.t1id.type - NullType() - >>> t1 = Table("t1", metadata, Column("id", Integer, primary_key=True)) - >>> t2.c.t1id.type - Integer() - -2. The system now works with :class:`_schema.ForeignKeyConstraint` as well:: - - >>> from sqlalchemy import Table, MetaData, Column, Integer, ForeignKeyConstraint - >>> metadata = MetaData() - >>> t2 = Table( - ... "t2", - ... metadata, - ... Column("t1a"), - ... Column("t1b"), - ... ForeignKeyConstraint(["t1a", "t1b"], ["t1.a", "t1.b"]), - ... ) - >>> t2.c.t1a.type - NullType() - >>> t2.c.t1b.type - NullType() - >>> t1 = Table( - ... "t1", - ... metadata, - ... Column("a", Integer, primary_key=True), - ... Column("b", Integer, primary_key=True), - ... ) - >>> t2.c.t1a.type - Integer() - >>> t2.c.t1b.type - Integer() - -3. It even works for "multiple hops" - that is, a :class:`_schema.ForeignKey` that refers to a - :class:`_schema.Column` that refers to another :class:`_schema.Column`:: - - >>> from sqlalchemy import Table, MetaData, Column, Integer, ForeignKey - >>> metadata = MetaData() - >>> t2 = Table("t2", metadata, Column("t1id", ForeignKey("t1.id"))) - >>> t3 = Table("t3", metadata, Column("t2t1id", ForeignKey("t2.t1id"))) - >>> t2.c.t1id.type - NullType() - >>> t3.c.t2t1id.type - NullType() - >>> t1 = Table("t1", metadata, Column("id", Integer, primary_key=True)) - >>> t2.c.t1id.type - Integer() - >>> t3.c.t2t1id.type - Integer() - -:ticket:`1765` - - -Dialect Changes -=============== - -Firebird ``fdb`` is now the default Firebird dialect. ------------------------------------------------------ - -The ``fdb`` dialect is now used if an engine is created without a dialect -specifier, i.e. ``firebird://``. ``fdb`` is a ``kinterbasdb`` compatible -DBAPI which per the Firebird project is now their official Python driver. - -:ticket:`2504` - -Firebird ``fdb`` and ``kinterbasdb`` set ``retaining=False`` by default ------------------------------------------------------------------------ - -Both the ``fdb`` and ``kinterbasdb`` DBAPIs support a flag ``retaining=True`` -which can be passed to the ``commit()`` and ``rollback()`` methods of its -connection. The documented rationale for this flag is so that the DBAPI -can re-use internal transaction state for subsequent transactions, for the -purposes of improving performance. However, newer documentation refers -to analyses of Firebird's "garbage collection" which expresses that this flag -can have a negative effect on the database's ability to process cleanup -tasks, and has been reported as *lowering* performance as a result. - -It's not clear how this flag is actually usable given this information, -and as it appears to be only a performance enhancing feature, it now defaults -to ``False``. The value can be controlled by passing the flag ``retaining=True`` -to the :func:`_sa.create_engine` call. This is a new flag which is added as of -0.8.2, so applications on 0.8.2 can begin setting this to ``True`` or ``False`` -as desired. - -.. seealso:: - - :mod:`sqlalchemy.dialects.firebird.fdb` - - :mod:`sqlalchemy.dialects.firebird.kinterbasdb` - - https://pythonhosted.org/fdb/usage-guide.html#retaining-transactions - information - on the "retaining" flag. - -:ticket:`2763` - - - - - diff --git a/doc/build/changelog/migration_10.rst b/doc/build/changelog/migration_10.rst deleted file mode 100644 index c7988b3cd5..0000000000 --- a/doc/build/changelog/migration_10.rst +++ /dev/null @@ -1,2778 +0,0 @@ -============================= -What's New in SQLAlchemy 1.0? -============================= - -.. admonition:: About this Document - - This document describes changes between SQLAlchemy version 0.9, - undergoing maintenance releases as of May, 2014, - and SQLAlchemy version 1.0, released in April, 2015. - - Document last updated: June 9, 2015 - -Introduction -============ - -This guide introduces what's new in SQLAlchemy version 1.0, -and also documents changes which affect users migrating -their applications from the 0.9 series of SQLAlchemy to 1.0. - -Please carefully review the sections on behavioral changes for -potentially backwards-incompatible changes in behavior. - - -New Features and Improvements - ORM -=================================== - -New Session Bulk INSERT/UPDATE API ----------------------------------- - -A new series of :class:`.Session` methods which provide hooks directly -into the unit of work's facility for emitting INSERT and UPDATE -statements has been created. When used correctly, this expert-oriented system -can allow ORM-mappings to be used to generate bulk insert and update -statements batched into executemany groups, allowing the statements -to proceed at speeds that rival direct use of the Core. - -.. seealso:: - - :ref:`bulk_operations` - introduction and full documentation - -:ticket:`3100` - -New Performance Example Suite ------------------------------ - -Inspired by the benchmarking done for the :ref:`bulk_operations` feature -as well as for the :ref:`faq_how_to_profile` section of the FAQ, a new -example section has been added which features several scripts designed -to illustrate the relative performance profile of various Core and ORM -techniques. The scripts are organized into use cases, and are packaged -under a single console interface such that any combination of demonstrations -can be run, dumping out timings, Python profile results and/or RunSnake profile -displays. - -.. seealso:: - - :ref:`examples_performance` - -"Baked" Queries ---------------- - -The "baked" query feature is an unusual new approach which allows for -straightforward construction an invocation of :class:`_query.Query` objects -using caching, which upon successive calls features vastly reduced -Python function call overhead (over 75%). By specifying a -:class:`_query.Query` object as a series of lambdas which are only invoked -once, a query as a pre-compiled unit begins to be feasible:: - - from sqlalchemy.ext import baked - from sqlalchemy import bindparam - - bakery = baked.bakery() - - - def search_for_user(session, username, email=None): - - baked_query = bakery(lambda session: session.query(User)) - baked_query += lambda q: q.filter(User.name == bindparam("username")) - - baked_query += lambda q: q.order_by(User.id) - - if email: - baked_query += lambda q: q.filter(User.email == bindparam("email")) - - result = baked_query(session).params(username=username, email=email).all() - - return result - -.. seealso:: - - :ref:`baked_toplevel` - -:ticket:`3054` - -.. _feature_3150: - -Improvements to declarative mixins, ``@declared_attr`` and related features ---------------------------------------------------------------------------- - -The declarative system in conjunction with :class:`.declared_attr` has been -overhauled to support new capabilities. - -A function decorated with :class:`.declared_attr` is now called only **after** -any mixin-based column copies are generated. This means the function can -call upon mixin-established columns and will receive a reference to the correct -:class:`_schema.Column` object:: - - class HasFooBar(object): - foobar = Column(Integer) - - @declared_attr - def foobar_prop(cls): - return column_property("foobar: " + cls.foobar) - - - class SomeClass(HasFooBar, Base): - __tablename__ = "some_table" - id = Column(Integer, primary_key=True) - -Above, ``SomeClass.foobar_prop`` will be invoked against ``SomeClass``, -and ``SomeClass.foobar`` will be the final :class:`_schema.Column` object that is -to be mapped to ``SomeClass``, as opposed to the non-copied object present -directly on ``HasFooBar``, even though the columns aren't mapped yet. - -The :class:`.declared_attr` function now **memoizes** the value -that's returned on a per-class basis, so that repeated calls to the same -attribute will return the same value. We can alter the example to illustrate -this:: - - class HasFooBar(object): - @declared_attr - def foobar(cls): - return Column(Integer) - - @declared_attr - def foobar_prop(cls): - return column_property("foobar: " + cls.foobar) - - - class SomeClass(HasFooBar, Base): - __tablename__ = "some_table" - id = Column(Integer, primary_key=True) - -Previously, ``SomeClass`` would be mapped with one particular copy of -the ``foobar`` column, but the ``foobar_prop`` by calling upon ``foobar`` -a second time would produce a different column. The value of -``SomeClass.foobar`` is now memoized during declarative setup time, so that -even before the attribute is mapped by the mapper, the interim column -value will remain consistent no matter how many times the -:class:`.declared_attr` is called upon. - -The two behaviors above should help considerably with declarative definition -of many types of mapper properties that derive from other attributes, where -the :class:`.declared_attr` function is called upon from other -:class:`.declared_attr` functions locally present before the class is -actually mapped. - -For a pretty slim edge case where one wishes to build a declarative mixin -that establishes distinct columns per subclass, a new modifier -:attr:`.declared_attr.cascading` is added. With this modifier, the -decorated function will be invoked individually for each class in the -mapped inheritance hierarchy. While this is already the behavior for -special attributes such as ``__table_args__`` and ``__mapper_args__``, -for columns and other properties the behavior by default assumes that attribute -is affixed to the base class only, and just inherited from subclasses. -With :attr:`.declared_attr.cascading`, individual behaviors can be -applied:: - - class HasIdMixin(object): - @declared_attr.cascading - def id(cls): - if has_inherited_table(cls): - return Column(ForeignKey("myclass.id"), primary_key=True) - else: - return Column(Integer, primary_key=True) - - - class MyClass(HasIdMixin, Base): - __tablename__ = "myclass" - # ... - - - class MySubClass(MyClass): - """ """ - - # ... - -.. seealso:: - - :ref:`mixin_inheritance_columns` - -Finally, the :class:`.AbstractConcreteBase` class has been reworked -so that a relationship or other mapper property can be set up inline -on the abstract base:: - - from sqlalchemy import Column, Integer, ForeignKey - from sqlalchemy.orm import relationship - from sqlalchemy.ext.declarative import ( - declarative_base, - declared_attr, - AbstractConcreteBase, - ) - - Base = declarative_base() - - - class Something(Base): - __tablename__ = "something" - id = Column(Integer, primary_key=True) - - - class Abstract(AbstractConcreteBase, Base): - id = Column(Integer, primary_key=True) - - @declared_attr - def something_id(cls): - return Column(ForeignKey(Something.id)) - - @declared_attr - def something(cls): - return relationship(Something) - - - class Concrete(Abstract): - __tablename__ = "cca" - __mapper_args__ = {"polymorphic_identity": "cca", "concrete": True} - -The above mapping will set up a table ``cca`` with both an ``id`` and -a ``something_id`` column, and ``Concrete`` will also have a relationship -``something``. The new feature is that ``Abstract`` will also have an -independently configured relationship ``something`` that builds against -the polymorphic union of the base. - -:ticket:`3150` :ticket:`2670` :ticket:`3149` :ticket:`2952` :ticket:`3050` - -ORM full object fetches 25% faster ----------------------------------- - -The mechanics of the ``loading.py`` module as well as the identity map -have undergone several passes of inlining, refactoring, and pruning, so -that a raw load of rows now populates ORM-based objects around 25% faster. -Assuming a 1M row table, a script like the following illustrates the type -of load that's improved the most:: - - import time - from sqlalchemy import Integer, Column, create_engine, Table - from sqlalchemy.orm import Session - from sqlalchemy.ext.declarative import declarative_base - - Base = declarative_base() - - - class Foo(Base): - __table__ = Table( - "foo", - Base.metadata, - Column("id", Integer, primary_key=True), - Column("a", Integer(), nullable=False), - Column("b", Integer(), nullable=False), - Column("c", Integer(), nullable=False), - ) - - - engine = create_engine("mysql+mysqldb://scott:tiger@localhost/test", echo=True) - - sess = Session(engine) - - now = time.time() - - # avoid using all() so that we don't have the overhead of building - # a large list of full objects in memory - for obj in sess.query(Foo).yield_per(100).limit(1000000): - pass - - print("Total time: %d" % (time.time() - now)) - -Local MacBookPro results bench from 19 seconds for 0.9 down to 14 seconds for -1.0. The :meth:`_query.Query.yield_per` call is always a good idea when batching -huge numbers of rows, as it prevents the Python interpreter from having -to allocate a huge amount of memory for all objects and their instrumentation -at once. Without the :meth:`_query.Query.yield_per`, the above script on the -MacBookPro is 31 seconds on 0.9 and 26 seconds on 1.0, the extra time spent -setting up very large memory buffers. - -.. _feature_3176: - -New KeyedTuple implementation dramatically faster -------------------------------------------------- - -We took a look into the :class:`.KeyedTuple` implementation in the hopes -of improving queries like this:: - - rows = sess.query(Foo.a, Foo.b, Foo.c).all() - -The :class:`.KeyedTuple` class is used rather than Python's -``collections.namedtuple()``, because the latter has a very complex -type-creation routine that benchmarks much slower than :class:`.KeyedTuple`. -However, when fetching hundreds of thousands of rows, -``collections.namedtuple()`` quickly overtakes :class:`.KeyedTuple` which -becomes dramatically slower as instance invocation goes up. What to do? -A new type that hedges between the approaches of both. Benching -all three types for "size" (number of rows returned) and "num" -(number of distinct queries), the new "lightweight keyed tuple" either -outperforms both, or lags very slightly behind the faster object, based on -which scenario. In the "sweet spot", where we are both creating a good number -of new types as well as fetching a good number of rows, the lightweight -object totally smokes both namedtuple and KeyedTuple: - -.. sourcecode:: text - - ----------------- - size=10 num=10000 # few rows, lots of queries - namedtuple: 3.60302400589 # namedtuple falls over - keyedtuple: 0.255059957504 # KeyedTuple very fast - lw keyed tuple: 0.582715034485 # lw keyed trails right on KeyedTuple - ----------------- - size=100 num=1000 # <--- sweet spot - namedtuple: 0.365247011185 - keyedtuple: 0.24896979332 - lw keyed tuple: 0.0889317989349 # lw keyed blows both away! - ----------------- - size=10000 num=100 - namedtuple: 0.572599887848 - keyedtuple: 2.54251694679 - lw keyed tuple: 0.613876104355 - ----------------- - size=1000000 num=10 # few queries, lots of rows - namedtuple: 5.79669594765 # namedtuple very fast - keyedtuple: 28.856498003 # KeyedTuple falls over - lw keyed tuple: 6.74346804619 # lw keyed trails right on namedtuple - - -:ticket:`3176` - -.. _feature_slots: - -Significant Improvements in Structural Memory Use -------------------------------------------------- - -Structural memory use has been improved via much more significant use -of ``__slots__`` for many internal objects. This optimization is -particularly geared towards the base memory size of large applications -that have lots of tables and columns, and reduces memory -size for a variety of high-volume objects including event listening -internals, comparator objects and parts of the ORM attribute and -loader strategy system. - -A bench that makes use of heapy measure the startup size of Nova -illustrates a difference of about 3.7 fewer megs, or 46%, -taken up by SQLAlchemy's objects, associated dictionaries, as -well as weakrefs, within a basic import of "nova.db.sqlalchemy.models": - -.. sourcecode:: text - - # reported by heapy, summation of SQLAlchemy objects + - # associated dicts + weakref-related objects with core of Nova imported: - - Before: total count 26477 total bytes 7975712 - After: total count 18181 total bytes 4236456 - - # reported for the Python module space overall with the - # core of Nova imported: - - Before: Partition of a set of 355558 objects. Total size = 61661760 bytes. - After: Partition of a set of 346034 objects. Total size = 57808016 bytes. - - -.. _feature_updatemany: - -UPDATE statements are now batched with executemany() in a flush ---------------------------------------------------------------- - -UPDATE statements can now be batched within an ORM flush -into more performant executemany() call, similarly to how INSERT -statements can be batched; this will be invoked within flush -based on the following criteria: - -* two or more UPDATE statements in sequence involve the identical set of - columns to be modified. - -* The statement has no embedded SQL expressions in the SET clause. - -* The mapping does not use a :paramref:`~.orm.mapper.version_id_col`, or - the backend dialect supports a "sane" rowcount for an executemany() - operation; most DBAPIs support this correctly now. - -.. _feature_3178: - - -.. _bug_3035: - -Session.get_bind() handles a wider variety of inheritance scenarios -------------------------------------------------------------------- - -The :meth:`.Session.get_bind` method is invoked whenever a query or unit -of work flush process seeks to locate the database engine that corresponds -to a particular class. The method has been improved to handle a variety -of inheritance-oriented scenarios, including: - -* Binding to a Mixin or Abstract Class:: - - class MyClass(SomeMixin, Base): - __tablename__ = "my_table" - # ... - - - session = Session(binds={SomeMixin: some_engine}) - -* Binding to inherited concrete subclasses individually based on table:: - - class BaseClass(Base): - __tablename__ = "base" - - # ... - - - class ConcreteSubClass(BaseClass): - __tablename__ = "concrete" - - # ... - - __mapper_args__ = {"concrete": True} - - - session = Session(binds={base_table: some_engine, concrete_table: some_other_engine}) - -:ticket:`3035` - - -.. _bug_3227: - -Session.get_bind() will receive the Mapper in all relevant Query cases ----------------------------------------------------------------------- - -A series of issues were repaired where the :meth:`.Session.get_bind` -would not receive the primary :class:`_orm.Mapper` of the :class:`_query.Query`, -even though this mapper was readily available (the primary mapper is the -single mapper, or alternatively the first mapper, that is associated with -a :class:`_query.Query` object). - -The :class:`_orm.Mapper` object, when passed to :meth:`.Session.get_bind`, -is typically used by sessions that make use of the -:paramref:`.Session.binds` parameter to associate mappers with a -series of engines (although in this use case, things frequently -"worked" in most cases anyway as the bind would be located via the -mapped table object), or more specifically implement a user-defined -:meth:`.Session.get_bind` method that provides some pattern of -selecting engines based on mappers, such as horizontal sharding or a -so-called "routing" session that routes queries to different backends. - -These scenarios include: - -* :meth:`_query.Query.count`:: - - session.query(User).count() - -* :meth:`_query.Query.update` and :meth:`_query.Query.delete`, both for the UPDATE/DELETE - statement as well as for the SELECT used by the "fetch" strategy:: - - session.query(User).filter(User.id == 15).update( - {"name": "foob"}, synchronize_session="fetch" - ) - - session.query(User).filter(User.id == 15).delete(synchronize_session="fetch") - -* Queries against individual columns:: - - session.query(User.id, User.name).all() - -* SQL functions and other expressions against indirect mappings such as - :obj:`.column_property`:: - - class User(Base): - ... - - score = column_property(func.coalesce(self.tables.users.c.name, None)) - - - session.query(func.max(User.score)).scalar() - -:ticket:`3227` :ticket:`3242` :ticket:`1326` - -.. _feature_2963: - -.info dictionary improvements ------------------------------ - -The :attr:`.InspectionAttr.info` collection is now available on every kind -of object that one would retrieve from the :attr:`_orm.Mapper.all_orm_descriptors` -collection. This includes :class:`.hybrid_property` and :func:`.association_proxy`. -However, as these objects are class-bound descriptors, they must be accessed -**separately** from the class to which they are attached in order to get -at the attribute. Below this is illustrated using the -:attr:`_orm.Mapper.all_orm_descriptors` namespace:: - - class SomeObject(Base): - # ... - - @hybrid_property - def some_prop(self): - return self.value + 5 - - - inspect(SomeObject).all_orm_descriptors.some_prop.info["foo"] = "bar" - -It is also available as a constructor argument for all :class:`.SchemaItem` -objects (e.g. :class:`_schema.ForeignKey`, :class:`.UniqueConstraint` etc.) as well -as remaining ORM constructs such as :func:`_orm.synonym`. - -:ticket:`2971` - -:ticket:`2963` - -.. _bug_3188: - -ColumnProperty constructs work a lot better with aliases, order_by ------------------------------------------------------------------- - -A variety of issues regarding :func:`.column_property` have been fixed, -most specifically with regards to the :func:`.aliased` construct as well -as the "order by label" logic introduced in 0.9 (see :ref:`migration_1068`). - -Given a mapping like the following:: - - class A(Base): - __tablename__ = "a" - - id = Column(Integer, primary_key=True) - - - class B(Base): - __tablename__ = "b" - - id = Column(Integer, primary_key=True) - a_id = Column(ForeignKey("a.id")) - - - A.b = column_property(select([func.max(B.id)]).where(B.a_id == A.id).correlate(A)) - -A simple scenario that included "A.b" twice would fail to render -correctly:: - - print(sess.query(A, a1).order_by(a1.b)) - -This would order by the wrong column: - -.. sourcecode:: sql - - SELECT a.id AS a_id, (SELECT max(b.id) AS max_1 FROM b - WHERE b.a_id = a.id) AS anon_1, a_1.id AS a_1_id, - (SELECT max(b.id) AS max_2 - FROM b WHERE b.a_id = a_1.id) AS anon_2 - FROM a, a AS a_1 ORDER BY anon_1 - -New output: - -.. sourcecode:: sql - - SELECT a.id AS a_id, (SELECT max(b.id) AS max_1 - FROM b WHERE b.a_id = a.id) AS anon_1, a_1.id AS a_1_id, - (SELECT max(b.id) AS max_2 - FROM b WHERE b.a_id = a_1.id) AS anon_2 - FROM a, a AS a_1 ORDER BY anon_2 - -There were also many scenarios where the "order by" logic would fail -to order by label, for example if the mapping were "polymorphic":: - - class A(Base): - __tablename__ = "a" - - id = Column(Integer, primary_key=True) - type = Column(String) - - __mapper_args__ = {"polymorphic_on": type, "with_polymorphic": "*"} - -The order_by would fail to use the label, as it would be anonymized due -to the polymorphic loading: - -.. sourcecode:: sql - - SELECT a.id AS a_id, a.type AS a_type, (SELECT max(b.id) AS max_1 - FROM b WHERE b.a_id = a.id) AS anon_1 - FROM a ORDER BY (SELECT max(b.id) AS max_2 - FROM b WHERE b.a_id = a.id) - -Now that the order by label tracks the anonymized label, this now works: - -.. sourcecode:: sql - - SELECT a.id AS a_id, a.type AS a_type, (SELECT max(b.id) AS max_1 - FROM b WHERE b.a_id = a.id) AS anon_1 - FROM a ORDER BY anon_1 - -Included in these fixes are a variety of heisenbugs that could corrupt -the state of an ``aliased()`` construct such that the labeling logic -would again fail; these have also been fixed. - -:ticket:`3148` :ticket:`3188` - -New Features and Improvements - Core -==================================== - -.. _feature_3034: - -Select/Query LIMIT / OFFSET may be specified as an arbitrary SQL expression ---------------------------------------------------------------------------- - -The :meth:`_expression.Select.limit` and :meth:`_expression.Select.offset` methods now accept -any SQL expression, in addition to integer values, as arguments. The ORM -:class:`_query.Query` object also passes through any expression to the underlying -:class:`_expression.Select` object. Typically -this is used to allow a bound parameter to be passed, which can be substituted -with a value later:: - - sel = select([table]).limit(bindparam("mylimit")).offset(bindparam("myoffset")) - -Dialects which don't support non-integer LIMIT or OFFSET expressions may continue -to not support this behavior; third party dialects may also need modification -in order to take advantage of the new behavior. A dialect which currently -uses the ``._limit`` or ``._offset`` attributes will continue to function -for those cases where the limit/offset was specified as a simple integer value. -However, when a SQL expression is specified, these two attributes will -instead raise a :class:`.CompileError` on access. A third-party dialect which -wishes to support the new feature should now call upon the ``._limit_clause`` -and ``._offset_clause`` attributes to receive the full SQL expression, rather -than the integer value. - -.. _feature_3282: - -The ``use_alter`` flag on ``ForeignKeyConstraint`` is (usually) no longer needed --------------------------------------------------------------------------------- - -The :meth:`_schema.MetaData.create_all` and :meth:`_schema.MetaData.drop_all` methods will -now make use of a system that automatically renders an ALTER statement -for foreign key constraints that are involved in mutually-dependent cycles -between tables, without the -need to specify :paramref:`_schema.ForeignKeyConstraint.use_alter`. Additionally, -the foreign key constraints no longer need to have a name in order to be -created via ALTER; only the DROP operation requires a name. In the case -of a DROP, the feature will ensure that only constraints which have -explicit names are actually included as ALTER statements. In the -case of an unresolvable cycle within a DROP, the system emits -a succinct and clear error message now if the DROP cannot proceed. - -The :paramref:`_schema.ForeignKeyConstraint.use_alter` and -:paramref:`_schema.ForeignKey.use_alter` flags remain in place, and continue to have -the same effect of establishing those constraints for which ALTER is -required during a CREATE/DROP scenario. - -As of version 1.0.1, special logic takes over in the case of SQLite, which -does not support ALTER, in the case that during a DROP, the given tables have -an unresolvable cycle; in this case a warning is emitted, and the tables -are dropped with **no** ordering, which is usually fine on SQLite unless -constraints are enabled. To resolve the warning and proceed with at least -a partial ordering on a SQLite database, particularly one where constraints -are enabled, re-apply "use_alter" flags to those -:class:`_schema.ForeignKey` and :class:`_schema.ForeignKeyConstraint` objects which should -be explicitly omitted from the sort. - -.. seealso:: - - :ref:`use_alter` - full description of the new behavior. - -:ticket:`3282` - -.. _change_3330: - -ResultProxy "auto close" is now a "soft" close ----------------------------------------------- - -For many releases, the :class:`_engine.ResultProxy` object has always been -automatically closed out at the point at which all result rows have been -fetched. This was to allow usage of the object without the need to call -upon :meth:`_engine.ResultProxy.close` explicitly; as all DBAPI resources had been -freed, the object was safe to discard. However, the object maintained -a strict "closed" behavior, which meant that any subsequent calls to -:meth:`_engine.ResultProxy.fetchone`, :meth:`_engine.ResultProxy.fetchmany` or -:meth:`_engine.ResultProxy.fetchall` would now raise a :class:`.ResourceClosedError`:: - - >>> result = connection.execute(stmt) - >>> result.fetchone() - (1, 'x') - >>> result.fetchone() - None # indicates no more rows - >>> result.fetchone() - exception: ResourceClosedError - -This behavior is inconsistent vs. what pep-249 states, which is -that you can call upon the fetch methods repeatedly even after results -are exhausted. It also interferes with behavior for some implementations of -result proxy, such as the :class:`.BufferedColumnResultProxy` used by the -cx_oracle dialect for certain datatypes. - -To solve this, the "closed" state of the :class:`_engine.ResultProxy` has been -broken into two states; a "soft close" which does the majority of what -"close" does, in that it releases the DBAPI cursor and in the case of a -"close with result" object will also release the connection, and a -"closed" state which is everything included by "soft close" as well as -establishing the fetch methods as "closed". The :meth:`_engine.ResultProxy.close` -method is now never called implicitly, only the :meth:`_engine.ResultProxy._soft_close` -method which is non-public:: - - >>> result = connection.execute(stmt) - >>> result.fetchone() - (1, 'x') - >>> result.fetchone() - None # indicates no more rows - >>> result.fetchone() - None # still None - >>> result.fetchall() - [] - >>> result.close() - >>> result.fetchone() - exception: ResourceClosedError # *now* it raises - -:ticket:`3330` -:ticket:`3329` - -CHECK Constraints now support the ``%(column_0_name)s`` token in naming conventions ------------------------------------------------------------------------------------ - -The ``%(column_0_name)s`` will derive from the first column found in the -expression of a :class:`.CheckConstraint`:: - - metadata = MetaData(naming_convention={"ck": "ck_%(table_name)s_%(column_0_name)s"}) - - foo = Table("foo", metadata, Column("value", Integer)) - - CheckConstraint(foo.c.value > 5) - -Will render: - -.. sourcecode:: sql - - CREATE TABLE foo ( - value INTEGER, - CONSTRAINT ck_foo_value CHECK (value > 5) - ) - -The combination of naming conventions with the constraint produced by a -:class:`.SchemaType` such as :class:`.Boolean` or :class:`.Enum` will also -now make use of all CHECK constraint conventions. - -.. seealso:: - - :ref:`naming_check_constraints` - - :ref:`naming_schematypes` - -:ticket:`3299` - -.. _change_3341: - -Constraints referring to unattached Columns can auto-attach to the Table when their referred columns are attached ------------------------------------------------------------------------------------------------------------------ - -Since at least version 0.8, a :class:`.Constraint` has had the ability to -"auto-attach" itself to a :class:`_schema.Table` based on being passed table-attached columns:: - - from sqlalchemy import Table, Column, MetaData, Integer, UniqueConstraint - - m = MetaData() - - t = Table("t", m, Column("a", Integer), Column("b", Integer)) - - uq = UniqueConstraint(t.c.a, t.c.b) # will auto-attach to Table - - assert uq in t.constraints - -In order to assist with some cases that tend to come up with declarative, -this same auto-attachment logic can now function even if the :class:`_schema.Column` -objects are not yet associated with the :class:`_schema.Table`; additional events -are established such that when those :class:`_schema.Column` objects are associated, -the :class:`.Constraint` is also added:: - - from sqlalchemy import Table, Column, MetaData, Integer, UniqueConstraint - - m = MetaData() - - a = Column("a", Integer) - b = Column("b", Integer) - - uq = UniqueConstraint(a, b) - - t = Table("t", m, a, b) - - assert uq in t.constraints # constraint auto-attached - -The above feature was a late add as of version 1.0.0b3. A fix as of -version 1.0.4 for :ticket:`3411` ensures that this logic -does not occur if the :class:`.Constraint` refers to a mixture of -:class:`_schema.Column` objects and string column names; as we do not yet have -tracking for the addition of names to a :class:`_schema.Table`:: - - from sqlalchemy import Table, Column, MetaData, Integer, UniqueConstraint - - m = MetaData() - - a = Column("a", Integer) - b = Column("b", Integer) - - uq = UniqueConstraint(a, "b") - - t = Table("t", m, a, b) - - # constraint *not* auto-attached, as we do not have tracking - # to locate when a name 'b' becomes available on the table - assert uq not in t.constraints - -Above, the attachment event for column "a" to table "t" will fire off before -column "b" is attached (as "a" is stated in the :class:`_schema.Table` constructor -before "b"), and the constraint will fail to locate "b" if it were to attempt -an attachment. For consistency, if the constraint refers to any string names, -the autoattach-on-column-attach logic is skipped. - -The original auto-attach logic of course remains in place, if the :class:`_schema.Table` -already contains all the target :class:`_schema.Column` objects at the time -the :class:`.Constraint` is constructed:: - - from sqlalchemy import Table, Column, MetaData, Integer, UniqueConstraint - - m = MetaData() - - a = Column("a", Integer) - b = Column("b", Integer) - - - t = Table("t", m, a, b) - - uq = UniqueConstraint(a, "b") - - # constraint auto-attached normally as in older versions - assert uq in t.constraints - -:ticket:`3341` -:ticket:`3411` - -.. _change_2051: - -.. _feature_insert_from_select_defaults: - -INSERT FROM SELECT now includes Python and SQL-expression defaults ------------------------------------------------------------------- - -:meth:`_expression.Insert.from_select` now includes Python and SQL-expression defaults if -otherwise unspecified; the limitation where non-server column defaults -aren't included in an INSERT FROM SELECT is now lifted and these -expressions are rendered as constants into the SELECT statement:: - - from sqlalchemy import Table, Column, MetaData, Integer, select, func - - m = MetaData() - - t = Table( - "t", m, Column("x", Integer), Column("y", Integer, default=func.somefunction()) - ) - - stmt = select([t.c.x]) - print(t.insert().from_select(["x"], stmt)) - -Will render: - -.. sourcecode:: sql - - INSERT INTO t (x, y) SELECT t.x, somefunction() AS somefunction_1 - FROM t - -The feature can be disabled using -:paramref:`.Insert.from_select.include_defaults`. - -.. _change_3087: - -Column server defaults now render literal values ------------------------------------------------- - -The "literal binds" compiler flag is switched on when a -:class:`.DefaultClause`, set up by :paramref:`_schema.Column.server_default` -is present as a SQL expression to be compiled. This allows literals -embedded in SQL to render correctly, such as:: - - from sqlalchemy import Table, Column, MetaData, Text - from sqlalchemy.schema import CreateTable - from sqlalchemy.dialects.postgresql import ARRAY, array - from sqlalchemy.dialects import postgresql - - metadata = MetaData() - - tbl = Table( - "derp", - metadata, - Column("arr", ARRAY(Text), server_default=array(["foo", "bar", "baz"])), - ) - - print(CreateTable(tbl).compile(dialect=postgresql.dialect())) - -Now renders: - -.. sourcecode:: sql - - CREATE TABLE derp ( - arr TEXT[] DEFAULT ARRAY['foo', 'bar', 'baz'] - ) - -Previously, the literal values ``"foo", "bar", "baz"`` would render as -bound parameters, which are useless in DDL. - -:ticket:`3087` - -.. _feature_3184: - -UniqueConstraint is now part of the Table reflection process ------------------------------------------------------------- - -A :class:`_schema.Table` object populated using ``autoload=True`` will now -include :class:`.UniqueConstraint` constructs as well as -:class:`.Index` constructs. This logic has a few caveats for -PostgreSQL and MySQL: - -PostgreSQL -^^^^^^^^^^ - -PostgreSQL has the behavior such that when a UNIQUE constraint is -created, it implicitly creates a UNIQUE INDEX corresponding to that -constraint as well. The :meth:`_reflection.Inspector.get_indexes` and the -:meth:`_reflection.Inspector.get_unique_constraints` methods will continue to -**both** return these entries distinctly, where -:meth:`_reflection.Inspector.get_indexes` now features a token -``duplicates_constraint`` within the index entry indicating the -corresponding constraint when detected. However, when performing -full table reflection using ``Table(..., autoload=True)``, the -:class:`.Index` construct is detected as being linked to the -:class:`.UniqueConstraint`, and is **not** present within the -:attr:`_schema.Table.indexes` collection; only the :class:`.UniqueConstraint` -will be present in the :attr:`_schema.Table.constraints` collection. This -deduplication logic works by joining to the ``pg_constraint`` table -when querying ``pg_index`` to see if the two constructs are linked. - -MySQL -^^^^^ - -MySQL does not have separate concepts for a UNIQUE INDEX and a UNIQUE -constraint. While it supports both syntaxes when creating tables and indexes, -it does not store them any differently. The -:meth:`_reflection.Inspector.get_indexes` -and the :meth:`_reflection.Inspector.get_unique_constraints` methods will continue to -**both** return an entry for a UNIQUE index in MySQL, -where :meth:`_reflection.Inspector.get_unique_constraints` features a new token -``duplicates_index`` within the constraint entry indicating that this is a -dupe entry corresponding to that index. However, when performing -full table reflection using ``Table(..., autoload=True)``, -the :class:`.UniqueConstraint` construct is -**not** part of the fully reflected :class:`_schema.Table` construct under any -circumstances; this construct is always represented by a :class:`.Index` -with the ``unique=True`` setting present in the :attr:`_schema.Table.indexes` -collection. - -.. seealso:: - - :ref:`postgresql_index_reflection` - - :ref:`mysql_unique_constraints` - -:ticket:`3184` - - -New systems to safely emit parameterized warnings -------------------------------------------------- - -For a long time, there has been a restriction that warning messages could not -refer to data elements, such that a particular function might emit an -infinite number of unique warnings. The key place this occurs is in the -``Unicode type received non-unicode bind param value`` warning. Placing -the data value in this message would mean that the Python ``__warningregistry__`` -for that module, or in some cases the Python-global ``warnings.onceregistry``, -would grow unbounded, as in most warning scenarios, one of these two collections -is populated with every distinct warning message. - -The change here is that by using a special ``string`` type that purposely -changes how the string is hashed, we can control that a large number of -parameterized messages are hashed only on a small set of possible hash -values, such that a warning such as ``Unicode type received non-unicode -bind param value`` can be tailored to be emitted only a specific number -of times; beyond that, the Python warnings registry will begin recording -them as duplicates. - -To illustrate, the following test script will show only ten warnings being -emitted for ten of the parameter sets, out of a total of 1000:: - - from sqlalchemy import create_engine, Unicode, select, cast - import random - import warnings - - e = create_engine("sqlite://") - - # Use the "once" filter (which is also the default for Python - # warnings). Exactly ten of these warnings will - # be emitted; beyond that, the Python warnings registry will accumulate - # new values as dupes of one of the ten existing. - warnings.filterwarnings("once") - - for i in range(1000): - e.execute( - select([cast(("foo_%d" % random.randint(0, 1000000)).encode("ascii"), Unicode)]) - ) - -The format of the warning here is: - -.. sourcecode:: text - - /path/lib/sqlalchemy/sql/sqltypes.py:186: SAWarning: Unicode type received - non-unicode bind param value 'foo_4852'. (this warning may be - suppressed after 10 occurrences) - - -:ticket:`3178` - -Key Behavioral Changes - ORM -============================ - -.. _bug_3228: - -query.update() now resolves string names into mapped attribute names --------------------------------------------------------------------- - -The documentation for :meth:`_query.Query.update` states that the given -``values`` dictionary is "a dictionary with attributes names as keys", -implying that these are mapped attribute names. Unfortunately, the function -was designed more in mind to receive attributes and SQL expressions and -not as much strings; when strings -were passed, these strings would be passed through straight to the core -update statement without any resolution as far as how these names are -represented on the mapped class, meaning the name would have to match that -of a table column exactly, not how an attribute of that name was mapped -onto the class. - -The string names are now resolved as attribute names in earnest:: - - class User(Base): - __tablename__ = "user" - - id = Column(Integer, primary_key=True) - name = Column("user_name", String(50)) - -Above, the column ``user_name`` is mapped as ``name``. Previously, -a call to :meth:`_query.Query.update` that was passed strings would have to -have been called as follows:: - - session.query(User).update({"user_name": "moonbeam"}) - -The given string is now resolved against the entity:: - - session.query(User).update({"name": "moonbeam"}) - -It is typically preferable to use the attribute directly, to avoid any -ambiguity:: - - session.query(User).update({User.name: "moonbeam"}) - -The change also indicates that synonyms and hybrid attributes can be referred -to by string name as well:: - - class User(Base): - __tablename__ = "user" - - id = Column(Integer, primary_key=True) - name = Column("user_name", String(50)) - - @hybrid_property - def fullname(self): - return self.name - - - session.query(User).update({"fullname": "moonbeam"}) - -:ticket:`3228` - -.. _bug_3371: - -Warnings emitted when comparing objects with None values to relationships -------------------------------------------------------------------------- - -This change is new as of 1.0.1. Some users are performing -queries that are essentially of this form:: - - session.query(Address).filter(Address.user == User(id=None)) - -This pattern is not currently supported in SQLAlchemy. For all versions, -it emits SQL resembling: - -.. sourcecode:: sql - - SELECT address.id AS address_id, address.user_id AS address_user_id, - address.email_address AS address_email_address - FROM address WHERE ? = address.user_id - (None,) - -Note above, there is a comparison ``WHERE ? = address.user_id`` where the -bound value ``?`` is receiving ``None``, or ``NULL`` in SQL. **This will -always return False in SQL**. The comparison here would in theory -generate SQL as follows: - -.. sourcecode:: sql - - SELECT address.id AS address_id, address.user_id AS address_user_id, - address.email_address AS address_email_address - FROM address WHERE address.user_id IS NULL - -But right now, **it does not**. Applications which are relying upon the -fact that "NULL = NULL" produces False in all cases run the risk that -someday, SQLAlchemy might fix this issue to generate "IS NULL", and the queries -will then produce different results. Therefore with this kind of operation, -you will see a warning: - -.. sourcecode:: text - - SAWarning: Got None for value of column user.id; this is unsupported - for a relationship comparison and will not currently produce an - IS comparison (but may in a future release) - -Note that this pattern was broken in most cases for release 1.0.0 including -all of the betas; a value like ``SYMBOL('NEVER_SET')`` would be generated. -This issue has been fixed, but as a result of identifying this pattern, -the warning is now there so that we can more safely repair this broken -behavior (now captured in :ticket:`3373`) in a future release. - -:ticket:`3371` - -.. _bug_3374: - -A "negated contains or equals" relationship comparison will use the current value of attributes, not the database value -------------------------------------------------------------------------------------------------------------------------- - -This change is new as of 1.0.1; while we would have preferred for this to be in 1.0.0, -it only became apparent as a result of :ticket:`3371`. - -Given a mapping:: - - class A(Base): - __tablename__ = "a" - id = Column(Integer, primary_key=True) - - - class B(Base): - __tablename__ = "b" - id = Column(Integer, primary_key=True) - a_id = Column(ForeignKey("a.id")) - a = relationship("A") - -Given ``A``, with primary key of 7, but which we changed to be 10 -without flushing:: - - s = Session(autoflush=False) - a1 = A(id=7) - s.add(a1) - s.commit() - - a1.id = 10 - -A query against a many-to-one relationship with this object as the target -will use the value 10 in the bound parameters:: - - s.query(B).filter(B.a == a1) - -Produces: - -.. sourcecode:: sql - - SELECT b.id AS b_id, b.a_id AS b_a_id - FROM b - WHERE ? = b.a_id - (10,) - -However, before this change, the negation of this criteria would **not** use -10, it would use 7, unless the object were flushed first:: - - s.query(B).filter(B.a != a1) - -Produces (in 0.9 and all versions prior to 1.0.1): - -.. sourcecode:: sql - - SELECT b.id AS b_id, b.a_id AS b_a_id - FROM b - WHERE b.a_id != ? OR b.a_id IS NULL - (7,) - -For a transient object, it would produce a broken query: - -.. sourcecode:: sql - - SELECT b.id, b.a_id - FROM b - WHERE b.a_id != :a_id_1 OR b.a_id IS NULL - -- {u'a_id_1': symbol('NEVER_SET')} - -This inconsistency has been repaired, and in all queries the current attribute -value, in this example ``10``, will now be used. - -:ticket:`3374` - -.. _migration_3061: - -Changes to attribute events and other operations regarding attributes that have no pre-existing value ------------------------------------------------------------------------------------------------------- - -In this change, the default return value of ``None`` when accessing an object -is now returned dynamically on each access, rather than implicitly setting the -attribute's state with a special "set" operation when it is first accessed. -The visible result of this change is that ``obj.__dict__`` is not implicitly -modified on get, and there are also some minor behavioral changes -for :func:`.attributes.get_history` and related functions. - -Given an object with no state:: - - >>> obj = Foo() - -It has always been SQLAlchemy's behavior such that if we access a scalar -or many-to-one attribute that was never set, it is returned as ``None``:: - - >>> obj.someattr - None - -This value of ``None`` is in fact now part of the state of ``obj``, and is -not unlike as though we had set the attribute explicitly, e.g. -``obj.someattr = None``. However, the "set on get" here would behave -differently as far as history and events. It would not emit any attribute -event, and additionally if we view history, we see this:: - - >>> inspect(obj).attrs.someattr.history - History(added=(), unchanged=[None], deleted=()) # 0.9 and below - -That is, it's as though the attribute were always ``None`` and were -never changed. This is explicitly different from if we had set the -attribute first instead:: - - >>> obj = Foo() - >>> obj.someattr = None - >>> inspect(obj).attrs.someattr.history - History(added=[None], unchanged=(), deleted=()) # all versions - -The above means that the behavior of our "set" operation can be corrupted -by the fact that the value was accessed via "get" earlier. In 1.0, this -inconsistency has been resolved, by no longer actually setting anything -when the default "getter" is used. - - >>> obj = Foo() - >>> obj.someattr - None - >>> inspect(obj).attrs.someattr.history - History(added=(), unchanged=(), deleted=()) # 1.0 - >>> obj.someattr = None - >>> inspect(obj).attrs.someattr.history - History(added=[None], unchanged=(), deleted=()) - -The reason the above behavior hasn't had much impact is because the -INSERT statement in relational databases considers a missing value to be -the same as NULL in most cases. Whether SQLAlchemy received a history -event for a particular attribute set to None or not would usually not matter; -as the difference between sending None/NULL or not wouldn't have an impact. -However, as :ticket:`3060` (described here in :ref:`migration_3060`) -illustrates, there are some seldom edge cases -where we do in fact want to positively have ``None`` set. Also, allowing -the attribute event here means it's now possible to create "default value" -functions for ORM mapped attributes. - -As part of this change, the generation of the implicit "None" is now disabled -for other situations where this used to occur; this includes when an -attribute set operation on a many-to-one is received; previously, the "old" value -would be "None" if it had been not set otherwise; it now will send the -value :data:`.orm.attributes.NEVER_SET`, which is a value that may be sent -to an attribute listener now. This symbol may also be received when -calling on mapper utility functions such as :meth:`_orm.Mapper.primary_key_from_instance`; -if the primary key attributes have no setting at all, whereas the value -would be ``None`` before, it will now be the :data:`.orm.attributes.NEVER_SET` -symbol, and no change to the object's state occurs. - -:ticket:`3061` - -.. _migration_3060: - -Priority of attribute changes on relationship-bound attributes vs. FK-bound may appear to change ------------------------------------------------------------------------------------------------- - -As a side effect of :ticket:`3060`, setting a relationship-bound attribute to ``None`` -is now a tracked history event which refers to the intention of persisting -``None`` to that attribute. As it has always been the case that setting a -relationship-bound attribute will trump direct assignment to the foreign key -attributes, a change in behavior can be seen here when assigning None. -Given a mapping:: - - class A(Base): - __tablename__ = "table_a" - - id = Column(Integer, primary_key=True) - - - class B(Base): - __tablename__ = "table_b" - - id = Column(Integer, primary_key=True) - a_id = Column(ForeignKey("table_a.id")) - a = relationship(A) - -In 1.0, the relationship-bound attribute takes precedence over the FK-bound -attribute in all cases, whether or not -the value we assign is a reference to an ``A`` object or is ``None``. -In 0.9, the behavior is inconsistent and -only takes effect if a value is assigned; the None is not considered:: - - a1 = A(id=1) - a2 = A(id=2) - session.add_all([a1, a2]) - session.flush() - - b1 = B() - b1.a = a1 # we expect a_id to be '1'; takes precedence in 0.9 and 1.0 - - b2 = B() - b2.a = None # we expect a_id to be None; takes precedence only in 1.0 - - b1.a_id = 2 - b2.a_id = 2 - - session.add_all([b1, b2]) - session.commit() - - assert b1.a is a1 # passes in both 0.9 and 1.0 - assert b2.a is None # passes in 1.0, in 0.9 it's a2 - -:ticket:`3060` - -.. _bug_3139: - -session.expunge() will fully detach an object that's been deleted ------------------------------------------------------------------ - -The behavior of :meth:`.Session.expunge` had a bug that caused an -inconsistency in behavior regarding deleted objects. The -:func:`.object_session` function as well as the :attr:`.InstanceState.session` -attribute would still report object as belonging to the :class:`.Session` -subsequent to the expunge:: - - u1 = sess.query(User).first() - sess.delete(u1) - - sess.flush() - - assert u1 not in sess - assert inspect(u1).session is sess # this is normal before commit - - sess.expunge(u1) - - assert u1 not in sess - assert inspect(u1).session is None # would fail - -Note that it is normal for ``u1 not in sess`` to be True while -``inspect(u1).session`` still refers to the session, while the transaction -is ongoing subsequent to the delete operation and :meth:`.Session.expunge` -has not been called; the full detachment normally completes once the -transaction is committed. This issue would also impact functions -that rely on :meth:`.Session.expunge` such as :func:`.make_transient`. - -:ticket:`3139` - -.. _migration_yield_per_eager_loading: - -Joined/Subquery eager loading explicitly disallowed with yield_per ------------------------------------------------------------------- - -In order to make the :meth:`_query.Query.yield_per` method easier to use, -an exception is raised if any subquery eager loaders, or joined -eager loaders that would use collections, are -to take effect when yield_per is used, as these are currently not compatible -with yield-per (subquery loading could be in theory, however). -When this error is raised, the :func:`.lazyload` option can be sent with -an asterisk:: - - q = sess.query(Object).options(lazyload("*")).yield_per(100) - -or use :meth:`_query.Query.enable_eagerloads`:: - - q = sess.query(Object).enable_eagerloads(False).yield_per(100) - -The :func:`.lazyload` option has the advantage that additional many-to-one -joined loader options can still be used:: - - q = ( - sess.query(Object) - .options(lazyload("*"), joinedload("some_manytoone")) - .yield_per(100) - ) - -.. _bug_3233: - -Changes and fixes in handling of duplicate join targets -------------------------------------------------------- - -Changes here encompass bugs where an unexpected and inconsistent -behavior would occur in some scenarios when joining to an entity -twice, or to multiple single-table entities against the same table, -without using a relationship-based ON clause, as well as when joining -multiple times to the same target relationship. - -Starting with a mapping as:: - - from sqlalchemy import Integer, Column, String, ForeignKey - from sqlalchemy.orm import Session, relationship - from sqlalchemy.ext.declarative import declarative_base - - Base = declarative_base() - - - class A(Base): - __tablename__ = "a" - id = Column(Integer, primary_key=True) - bs = relationship("B") - - - class B(Base): - __tablename__ = "b" - id = Column(Integer, primary_key=True) - a_id = Column(ForeignKey("a.id")) - -A query that joins to ``A.bs`` twice:: - - print(s.query(A).join(A.bs).join(A.bs)) - -Will render: - -.. sourcecode:: sql - - SELECT a.id AS a_id - FROM a JOIN b ON a.id = b.a_id - -The query deduplicates the redundant ``A.bs`` because it is attempting -to support a case like the following:: - - s.query(A).join(A.bs).filter(B.foo == "bar").reset_joinpoint().join(A.bs, B.cs).filter( - C.bar == "bat" - ) - -That is, the ``A.bs`` is part of a "path". As part of :ticket:`3367`, -arriving at the same endpoint twice without it being part of a -larger path will now emit a warning: - -.. sourcecode:: text - - SAWarning: Pathed join target A.bs has already been joined to; skipping - -The bigger change involves when joining to an entity without using a -relationship-bound path. If we join to ``B`` twice:: - - print(s.query(A).join(B, B.a_id == A.id).join(B, B.a_id == A.id)) - -In 0.9, this would render as follows: - -.. sourcecode:: sql - - SELECT a.id AS a_id - FROM a JOIN b ON b.a_id = a.id JOIN b AS b_1 ON b_1.a_id = a.id - -This is problematic since the aliasing is implicit and in the case of different -ON clauses can lead to unpredictable results. - -In 1.0, no automatic aliasing is applied and we get: - -.. sourcecode:: sql - - SELECT a.id AS a_id - FROM a JOIN b ON b.a_id = a.id JOIN b ON b.a_id = a.id - -This will raise an error from the database. While it might be nice if -the "duplicate join target" acted identically if we joined both from -redundant relationships vs. redundant non-relationship based targets, -for now we are only changing the behavior in the more serious case where -implicit aliasing would have occurred previously, and only emitting a warning -in the relationship case. Ultimately, joining to the same thing twice without -any aliasing to disambiguate should raise an error in all cases. - -The change also has an impact on single-table inheritance targets. Using -a mapping as follows:: - - from sqlalchemy import Integer, Column, String, ForeignKey - from sqlalchemy.orm import Session, relationship - from sqlalchemy.ext.declarative import declarative_base - - Base = declarative_base() - - - class A(Base): - __tablename__ = "a" - - id = Column(Integer, primary_key=True) - type = Column(String) - - __mapper_args__ = {"polymorphic_on": type, "polymorphic_identity": "a"} - - - class ASub1(A): - __mapper_args__ = {"polymorphic_identity": "asub1"} - - - class ASub2(A): - __mapper_args__ = {"polymorphic_identity": "asub2"} - - - class B(Base): - __tablename__ = "b" - - id = Column(Integer, primary_key=True) - - a_id = Column(Integer, ForeignKey("a.id")) - - a = relationship("A", primaryjoin="B.a_id == A.id", backref="b") - - - s = Session() - - print(s.query(ASub1).join(B, ASub1.b).join(ASub2, B.a)) - - print(s.query(ASub1).join(B, ASub1.b).join(ASub2, ASub2.id == B.a_id)) - -The two queries at the bottom are equivalent, and should both render -the identical SQL: - -.. sourcecode:: sql - - SELECT a.id AS a_id, a.type AS a_type - FROM a JOIN b ON b.a_id = a.id JOIN a ON b.a_id = a.id AND a.type IN (:type_1) - WHERE a.type IN (:type_2) - -The above SQL is invalid, as it renders "a" within the FROM list twice. -However, the implicit aliasing bug would occur with the second query only -and render this instead: - -.. sourcecode:: sql - - SELECT a.id AS a_id, a.type AS a_type - FROM a JOIN b ON b.a_id = a.id JOIN a AS a_1 - ON a_1.id = b.a_id AND a_1.type IN (:type_1) - WHERE a_1.type IN (:type_2) - -Where above, the second join to "a" is aliased. While this seems convenient, -it's not how single-inheritance queries work in general and is misleading -and inconsistent. - -The net effect is that applications which were relying on this bug will now -have an error raised by the database. The solution is to use the expected -form. When referring to multiple subclasses of a single-inheritance -entity in a query, you must manually use aliases to disambiguate the table, -as all the subclasses normally refer to the same table:: - - asub2_alias = aliased(ASub2) - - print(s.query(ASub1).join(B, ASub1.b).join(asub2_alias, B.a.of_type(asub2_alias))) - -:ticket:`3233` -:ticket:`3367` - - -Deferred Columns No Longer Implicitly Undefer ---------------------------------------------- - -Mapped attributes marked as deferred without explicit undeferral -will now remain "deferred" even if their column is otherwise -present in the result set in some way. This is a performance -enhancement in that an ORM load no longer spends time searching -for each deferred column when the result set is obtained. However, -for an application that has been relying upon this, an explicit -:func:`.undefer` or similar option should now be used, in order -to prevent a SELECT from being emitted when the attribute is accessed. - - -.. _migration_deprecated_orm_events: - -Deprecated ORM Event Hooks Removed ----------------------------------- - -The following ORM event hooks, some of which have been deprecated since -0.5, have been removed: ``translate_row``, ``populate_instance``, -``append_result``, ``create_instance``. The use cases for these hooks -originated in the very early 0.1 / 0.2 series of SQLAlchemy and have long -since been unnecessary. In particular, the hooks were largely unusable -as the behavioral contracts within these events was strongly linked to -the surrounding internals, such as how an instance needs to be created -and initialized as well as how columns are located within an ORM-generated -row. The removal of these hooks greatly simplifies the mechanics of ORM -object loading. - -.. _bundle_api_change: - -API Change for new Bundle feature when custom row loaders are used ------------------------------------------------------------------- - -The new :class:`.Bundle` object of 0.9 has a small change in API, -when the ``create_row_processor()`` method is overridden on a custom class. -Previously, the sample code looked like:: - - from sqlalchemy.orm import Bundle - - - class DictBundle(Bundle): - def create_row_processor(self, query, procs, labels): - """Override create_row_processor to return values as dictionaries""" - - def proc(row, result): - return dict(zip(labels, (proc(row, result) for proc in procs))) - - return proc - -The unused ``result`` member is now removed:: - - from sqlalchemy.orm import Bundle - - - class DictBundle(Bundle): - def create_row_processor(self, query, procs, labels): - """Override create_row_processor to return values as dictionaries""" - - def proc(row): - return dict(zip(labels, (proc(row) for proc in procs))) - - return proc - -.. seealso:: - - :ref:`bundles` - -.. _migration_3008: - -Right inner join nesting now the default for joinedload with innerjoin=True ---------------------------------------------------------------------------- - -The behavior of :paramref:`_orm.joinedload.innerjoin` as well as -:paramref:`_orm.relationship.innerjoin` is now to use "nested" -inner joins, that is, right-nested, as the default behavior when an -inner join joined eager load is chained to an outer join eager load. In -order to get the old behavior of chaining all joined eager loads as -outer join when an outer join is present, use ``innerjoin="unnested"``. - -As introduced in :ref:`feature_2976` from version 0.9, the behavior of -``innerjoin="nested"`` is that an inner join eager load chained to an outer -join eager load will use a right-nested join. ``"nested"`` is now implied -when using ``innerjoin=True``:: - - query(User).options( - joinedload("orders", innerjoin=False).joinedload("items", innerjoin=True) - ) - -With the new default, this will render the FROM clause in the form:\ - -.. sourcecode:: text - - FROM users LEFT OUTER JOIN (orders JOIN items ON ) ON - -That is, using a right-nested join for the INNER join so that the full -result of ``users`` can be returned. The use of an INNER join is more efficient -than using an OUTER join, and allows the :paramref:`_orm.joinedload.innerjoin` -optimization parameter to take effect in all cases. - -To get the older behavior, use ``innerjoin="unnested"``:: - - query(User).options( - joinedload("orders", innerjoin=False).joinedload("items", innerjoin="unnested") - ) - -This will avoid right-nested joins and chain the joins together using all -OUTER joins despite the innerjoin directive: - -.. sourcecode:: text - - FROM users LEFT OUTER JOIN orders ON LEFT OUTER JOIN items ON - -As noted in the 0.9 notes, the only database backend that has difficulty -with right-nested joins is SQLite; SQLAlchemy as of 0.9 converts a right-nested -join into a subquery as a join target on SQLite. - -.. seealso:: - - :ref:`feature_2976` - description of the feature as introduced in 0.9.4. - -:ticket:`3008` - -.. _change_3249: - -Subqueries no longer applied to uselist=False joined eager loads ----------------------------------------------------------------- - -Given a joined eager load like the following:: - - class A(Base): - __tablename__ = "a" - id = Column(Integer, primary_key=True) - b = relationship("B", uselist=False) - - - class B(Base): - __tablename__ = "b" - id = Column(Integer, primary_key=True) - a_id = Column(ForeignKey("a.id")) - - - s = Session() - print(s.query(A).options(joinedload(A.b)).limit(5)) - -SQLAlchemy considers the relationship ``A.b`` to be a "one to many, -loaded as a single value", which is essentially a "one to one" -relationship. However, joined eager loading has always treated the -above as a situation where the main query needs to be inside a -subquery, as would normally be needed for a collection of B objects -where the main query has a LIMIT applied: - -.. sourcecode:: sql - - SELECT anon_1.a_id AS anon_1_a_id, b_1.id AS b_1_id, b_1.a_id AS b_1_a_id - FROM (SELECT a.id AS a_id - FROM a LIMIT :param_1) AS anon_1 - LEFT OUTER JOIN b AS b_1 ON anon_1.a_id = b_1.a_id - -However, since the relationship of the inner query to the outer one is -that at most only one row is shared in the case of ``uselist=False`` -(in the same way as a many-to-one), the "subquery" used with LIMIT + -joined eager loading is now dropped in this case: - -.. sourcecode:: sql - - SELECT a.id AS a_id, b_1.id AS b_1_id, b_1.a_id AS b_1_a_id - FROM a LEFT OUTER JOIN b AS b_1 ON a.id = b_1.a_id - LIMIT :param_1 - -In the case that the LEFT OUTER JOIN returns more than one row, the ORM -has always emitted a warning here and ignored additional results for -``uselist=False``, so the results in that error situation should not change. - -:ticket:`3249` - - -query.update() / query.delete() raises if used with join(), select_from(), from_self() --------------------------------------------------------------------------------------- - -A warning is emitted in SQLAlchemy 0.9.10 (not yet released as of -June 9, 2015) when the :meth:`_query.Query.update` or :meth:`_query.Query.delete` methods -are invoked against a query which has also called upon :meth:`_query.Query.join`, -:meth:`_query.Query.outerjoin`, -:meth:`_query.Query.select_from` or :meth:`_query.Query.from_self`. These are unsupported -use cases which silently fail in the 0.9 series up until 0.9.10 where it emits -a warning. In 1.0, these cases raise an exception. - -:ticket:`3349` - - -query.update() with ``synchronize_session='evaluate'`` raises on multi-table update ------------------------------------------------------------------------------------ - -The "evaluator" for :meth:`_query.Query.update` won't work with multi-table -updates, and needs to be set to ``synchronize_session=False`` or -``synchronize_session='fetch'`` when multiple tables are present. -The new behavior is that an explicit exception is now raised, with a message -to change the synchronize setting. -This is upgraded from a warning emitted as of 0.9.7. - -:ticket:`3117` - -Resurrect Event has been Removed --------------------------------- - -The "resurrect" ORM event has been removed entirely. This event ceased to -have any function since version 0.8 removed the older "mutable" system -from the unit of work. - - -.. _migration_3177: - -Change to single-table-inheritance criteria when using from_self(), count() ---------------------------------------------------------------------------- - -Given a single-table inheritance mapping, such as:: - - class Widget(Base): - __table__ = "widget_table" - - - class FooWidget(Widget): - pass - -Using :meth:`_query.Query.from_self` or :meth:`_query.Query.count` against a subclass -would produce a subquery, but then add the "WHERE" criteria for subtypes -to the outside:: - - sess.query(FooWidget).from_self().all() - -rendering: - -.. sourcecode:: sql - - SELECT - anon_1.widgets_id AS anon_1_widgets_id, - anon_1.widgets_type AS anon_1_widgets_type - FROM (SELECT widgets.id AS widgets_id, widgets.type AS widgets_type, - FROM widgets) AS anon_1 - WHERE anon_1.widgets_type IN (?) - -The issue with this is that if the inner query does not specify all -columns, then we can't add the WHERE clause on the outside (it actually tries, -and produces a bad query). This decision -apparently goes way back to 0.6.5 with the note "may need to make more -adjustments to this". Well, those adjustments have arrived! So now the -above query will render: - -.. sourcecode:: sql - - SELECT - anon_1.widgets_id AS anon_1_widgets_id, - anon_1.widgets_type AS anon_1_widgets_type - FROM (SELECT widgets.id AS widgets_id, widgets.type AS widgets_type, - FROM widgets - WHERE widgets.type IN (?)) AS anon_1 - -So that queries that don't include "type" will still work!:: - - sess.query(FooWidget.id).count() - -Renders: - -.. sourcecode:: sql - - SELECT count(*) AS count_1 - FROM (SELECT widgets.id AS widgets_id - FROM widgets - WHERE widgets.type IN (?)) AS anon_1 - - -:ticket:`3177` - - -.. _migration_3222: - - -single-table-inheritance criteria added to all ON clauses unconditionally -------------------------------------------------------------------------- - -When joining to a single-table inheritance subclass target, the ORM always adds -the "single table criteria" when joining on a relationship. Given a -mapping as:: - - class Widget(Base): - __tablename__ = "widget" - id = Column(Integer, primary_key=True) - type = Column(String) - related_id = Column(ForeignKey("related.id")) - related = relationship("Related", backref="widget") - __mapper_args__ = {"polymorphic_on": type} - - - class FooWidget(Widget): - __mapper_args__ = {"polymorphic_identity": "foo"} - - - class Related(Base): - __tablename__ = "related" - id = Column(Integer, primary_key=True) - -It's been the behavior for quite some time that a JOIN on the relationship -will render a "single inheritance" clause for the type:: - - s.query(Related).join(FooWidget, Related.widget).all() - -SQL output: - -.. sourcecode:: sql - - SELECT related.id AS related_id - FROM related JOIN widget ON related.id = widget.related_id AND widget.type IN (:type_1) - -Above, because we joined to a subclass ``FooWidget``, :meth:`_query.Query.join` -knew to add the ``AND widget.type IN ('foo')`` criteria to the ON clause. - -The change here is that the ``AND widget.type IN()`` criteria is now appended -to *any* ON clause, not just those generated from a relationship, -including one that is explicitly stated:: - - # ON clause will now render as - # related.id = widget.related_id AND widget.type IN (:type_1) - s.query(Related).join(FooWidget, FooWidget.related_id == Related.id).all() - -As well as the "implicit" join when no ON clause of any kind is stated:: - - # ON clause will now render as - # related.id = widget.related_id AND widget.type IN (:type_1) - s.query(Related).join(FooWidget).all() - -Previously, the ON clause for these would not include the single-inheritance -criteria. Applications that are already adding this criteria to work around -this will want to remove its explicit use, though it should continue to work -fine if the criteria happens to be rendered twice in the meantime. - -.. seealso:: - - :ref:`bug_3233` - -:ticket:`3222` - -Key Behavioral Changes - Core -============================= - -.. _migration_2992: - -Warnings emitted when coercing full SQL fragments into text() -------------------------------------------------------------- - -Since SQLAlchemy's inception, there has always been an emphasis on not getting -in the way of the usage of plain text. The Core and ORM expression systems -were intended to allow any number of points at which the user can just -use plain text SQL expressions, not just in the sense that you can send a -full SQL string to :meth:`_engine.Connection.execute`, but that you can send strings -with SQL expressions into many functions, such as :meth:`_expression.Select.where`, -:meth:`_query.Query.filter`, and :meth:`_expression.Select.order_by`. - -Note that by "SQL expressions" we mean a **full fragment of a SQL string**, -such as:: - - # the argument sent to where() is a full SQL expression - stmt = select([sometable]).where("somecolumn = 'value'") - -and we are **not talking about string arguments**, that is, the normal -behavior of passing string values that become parameterized:: - - # This is a normal Core expression with a string argument - - # we aren't talking about this!! - stmt = select([sometable]).where(sometable.c.somecolumn == "value") - -The Core tutorial has long featured an example of the use of this technique, -using a :func:`_expression.select` construct where virtually all components of it -are specified as straight strings. However, despite this long-standing -behavior and example, users are apparently surprised that this behavior -exists, and when asking around the community, I was unable to find any user -that was in fact *not* surprised that you can send a full string into a method -like :meth:`_query.Query.filter`. - -So the change here is to encourage the user to qualify textual strings when -composing SQL that is partially or fully composed from textual fragments. -When composing a select as below:: - - stmt = select(["a", "b"]).where("a = b").select_from("sometable") - -The statement is built up normally, with all the same coercions as before. -However, one will see the following warnings emitted: - -.. sourcecode:: text - - SAWarning: Textual column expression 'a' should be explicitly declared - with text('a'), or use column('a') for more specificity - (this warning may be suppressed after 10 occurrences) - - SAWarning: Textual column expression 'b' should be explicitly declared - with text('b'), or use column('b') for more specificity - (this warning may be suppressed after 10 occurrences) - - SAWarning: Textual SQL expression 'a = b' should be explicitly declared - as text('a = b') (this warning may be suppressed after 10 occurrences) - - SAWarning: Textual SQL FROM expression 'sometable' should be explicitly - declared as text('sometable'), or use table('sometable') for more - specificity (this warning may be suppressed after 10 occurrences) - -These warnings attempt to show exactly where the issue is by displaying -the parameters as well as where the string was received. -The warnings make use of the :ref:`feature_3178` so that parameterized warnings -can be emitted safely without running out of memory, and as always, if -one wishes the warnings to be exceptions, the -`Python Warnings Filter `_ -should be used:: - - import warnings - - warnings.simplefilter("error") # all warnings raise an exception - -Given the above warnings, our statement works just fine, but -to get rid of the warnings we would rewrite our statement as follows:: - - from sqlalchemy import select, text - - stmt = ( - select([text("a"), text("b")]).where(text("a = b")).select_from(text("sometable")) - ) - -and as the warnings suggest, we can give our statement more specificity -about the text if we use :func:`_expression.column` and :func:`.table`:: - - from sqlalchemy import select, text, column, table - - stmt = ( - select([column("a"), column("b")]) - .where(text("a = b")) - .select_from(table("sometable")) - ) - -Where note also that :func:`.table` and :func:`_expression.column` can now -be imported from "sqlalchemy" without the "sql" part. - -The behavior here applies to :func:`_expression.select` as well as to key methods -on :class:`_query.Query`, including :meth:`_query.Query.filter`, -:meth:`_query.Query.from_statement` and :meth:`_query.Query.having`. - -ORDER BY and GROUP BY are special cases -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -There is one case where usage of a string has special meaning, and as part -of this change we have enhanced its functionality. When we have a -:func:`_expression.select` or :class:`_query.Query` that refers to some column name or named -label, we might want to GROUP BY and/or ORDER BY known columns or labels:: - - stmt = ( - select([user.c.name, func.count(user.c.id).label("id_count")]) - .group_by("name") - .order_by("id_count") - ) - -In the above statement we expect to see "ORDER BY id_count", as opposed to a -re-statement of the function. The string argument given is actively -matched to an entry in the columns clause during compilation, so the above -statement would produce as we expect, without warnings (though note that -the ``"name"`` expression has been resolved to ``users.name``!): - -.. sourcecode:: sql - - SELECT users.name, count(users.id) AS id_count - FROM users GROUP BY users.name ORDER BY id_count - -However, if we refer to a name that cannot be located, then we get -the warning again, as below:: - - stmt = select([user.c.name, func.count(user.c.id).label("id_count")]).order_by( - "some_label" - ) - -The output does what we say, but again it warns us: - -.. sourcecode:: text - - SAWarning: Can't resolve label reference 'some_label'; converting to - text() (this warning may be suppressed after 10 occurrences) - -.. sourcecode:: sql - - SELECT users.name, count(users.id) AS id_count - FROM users ORDER BY some_label - -The above behavior applies to all those places where we might want to refer -to a so-called "label reference"; ORDER BY and GROUP BY, but also within an -OVER clause as well as a DISTINCT ON clause that refers to columns (e.g. the -PostgreSQL syntax). - -We can still specify any arbitrary expression for ORDER BY or others using -:func:`_expression.text`:: - - stmt = select([users]).order_by(text("some special expression")) - -The upshot of the whole change is that SQLAlchemy now would like us -to tell it when a string is sent that this string is explicitly -a :func:`_expression.text` construct, or a column, table, etc., and if we use it as a -label name in an order by, group by, or other expression, SQLAlchemy expects -that the string resolves to something known, else it should again -be qualified with :func:`_expression.text` or similar. - -:ticket:`2992` - -.. _bug_3288: - -Python-side defaults invoked for each row individually when using a multivalued insert --------------------------------------------------------------------------------------- - -Support for Python-side column defaults when using the multi-valued -version of :meth:`_expression.Insert.values` were essentially not implemented, and -would only work "by accident" in specific situations, when the dialect in -use was using a non-positional (e.g. named) style of bound parameter, and -when it was not necessary that a Python-side callable be invoked for each -row. - -The feature has been overhauled so that it works more similarly to -that of an "executemany" style of invocation:: - - import itertools - - counter = itertools.count(1) - t = Table( - "my_table", - metadata, - Column("id", Integer, default=lambda: next(counter)), - Column("data", String), - ) - - conn.execute( - t.insert().values( - [ - {"data": "d1"}, - {"data": "d2"}, - {"data": "d3"}, - ] - ) - ) - -The above example will invoke ``next(counter)`` for each row individually -as would be expected: - -.. sourcecode:: sql - - INSERT INTO my_table (id, data) VALUES (?, ?), (?, ?), (?, ?) - (1, 'd1', 2, 'd2', 3, 'd3') - -Previously, a positional dialect would fail as a bind would not be generated -for additional positions: - -.. sourcecode:: text - - Incorrect number of bindings supplied. The current statement uses 6, - and there are 4 supplied. - [SQL: u'INSERT INTO my_table (id, data) VALUES (?, ?), (?, ?), (?, ?)'] - [parameters: (1, 'd1', 'd2', 'd3')] - -And with a "named" dialect, the same value for "id" would be re-used in -each row (hence this change is backwards-incompatible with a system that -relied on this): - -.. sourcecode:: sql - - INSERT INTO my_table (id, data) VALUES (:id, :data_0), (:id, :data_1), (:id, :data_2) - -- {u'data_2': 'd3', u'data_1': 'd2', u'data_0': 'd1', 'id': 1} - -The system will also refuse to invoke a "server side" default as inline-rendered -SQL, since it cannot be guaranteed that a server side default is compatible -with this. If the VALUES clause renders for a specific column, then a Python-side -value is required; if an omitted value only refers to a server-side default, -an exception is raised:: - - t = Table( - "my_table", - metadata, - Column("id", Integer, primary_key=True), - Column("data", String, server_default="some default"), - ) - - conn.execute( - t.insert().values( - [ - {"data": "d1"}, - {"data": "d2"}, - {}, - ] - ) - ) - -will raise: - -.. sourcecode:: text - - sqlalchemy.exc.CompileError: INSERT value for column my_table.data is - explicitly rendered as a boundparameter in the VALUES clause; a - Python-side value or SQL expression is required - -Previously, the value "d1" would be copied into that of the third -row (but again, only with named format!): - -.. sourcecode:: sql - - INSERT INTO my_table (data) VALUES (:data_0), (:data_1), (:data_0) - -- {u'data_1': 'd2', u'data_0': 'd1'} - -:ticket:`3288` - -.. _change_3163: - -Event listeners can not be added or removed from within that event's runner ---------------------------------------------------------------------------- - -Removal of an event listener from inside that same event itself would -modify the elements of a list during iteration, which would cause -still-attached event listeners to silently fail to fire. To prevent -this while still maintaining performance, the lists have been replaced -with ``collections.deque()``, which does not allow any additions or -removals during iteration, and instead raises ``RuntimeError``. - -:ticket:`3163` - -.. _change_3169: - -The INSERT...FROM SELECT construct now implies ``inline=True`` --------------------------------------------------------------- - -Using :meth:`_expression.Insert.from_select` now implies ``inline=True`` -on :func:`_expression.insert`. This helps to fix a bug where an -INSERT...FROM SELECT construct would inadvertently be compiled -as "implicit returning" on supporting backends, which would -cause breakage in the case of an INSERT that inserts zero rows -(as implicit returning expects a row), as well as arbitrary -return data in the case of an INSERT that inserts multiple -rows (e.g. only the first row of many). -A similar change is also applied to an INSERT..VALUES -with multiple parameter sets; implicit RETURNING will no longer emit -for this statement either. As both of these constructs deal -with variable numbers of rows, the -:attr:`_engine.ResultProxy.inserted_primary_key` accessor does not -apply. Previously, there was a documentation note that one -may prefer ``inline=True`` with INSERT..FROM SELECT as some databases -don't support returning and therefore can't do "implicit" returning, -but there's no reason an INSERT...FROM SELECT needs implicit returning -in any case. Regular explicit :meth:`_expression.Insert.returning` should -be used to return variable numbers of result rows if inserted -data is needed. - -:ticket:`3169` - -.. _change_3027: - -``autoload_with`` now implies ``autoload=True`` ------------------------------------------------ - -A :class:`_schema.Table` can be set up for reflection by passing -:paramref:`_schema.Table.autoload_with` alone:: - - my_table = Table("my_table", metadata, autoload_with=some_engine) - -:ticket:`3027` - -.. _change_3266: - -DBAPI exception wrapping and handle_error() event improvements --------------------------------------------------------------- - -SQLAlchemy's wrapping of DBAPI exceptions was not taking place in the -case where a :class:`_engine.Connection` object was invalidated, and then tried -to reconnect and encountered an error; this has been resolved. - -Additionally, the recently added :meth:`_events.ConnectionEvents.handle_error` -event is now invoked for errors that occur upon initial connect, upon -reconnect, and when :func:`_sa.create_engine` is used given a custom connection -function via :paramref:`_sa.create_engine.creator`. - -The :class:`.ExceptionContext` object has a new datamember -:attr:`.ExceptionContext.engine` that will always refer to the :class:`_engine.Engine` -in use, in those cases when the :class:`_engine.Connection` object is not available -(e.g. on initial connect). - - -:ticket:`3266` - -.. _change_3243: - -ForeignKeyConstraint.columns is now a ColumnCollection ------------------------------------------------------- - -:attr:`_schema.ForeignKeyConstraint.columns` was previously a plain list -containing either strings or :class:`_schema.Column` objects, depending on -how the :class:`_schema.ForeignKeyConstraint` was constructed and whether it was -associated with a table. The collection is now a :class:`_expression.ColumnCollection`, -and is only initialized after the :class:`_schema.ForeignKeyConstraint` is -associated with a :class:`_schema.Table`. A new accessor -:attr:`_schema.ForeignKeyConstraint.column_keys` -is added to unconditionally return string keys for the local set of -columns regardless of how the object was constructed or its current -state. - - -.. _feature_3084: - -MetaData.sorted_tables accessor is "deterministic" --------------------------------------------------- - -The sorting of tables resulting from the :attr:`_schema.MetaData.sorted_tables` -accessor is "deterministic"; the ordering should be the same in all cases -regardless of Python hashing. This is done by first sorting the tables -by name before passing them to the topological algorithm, which maintains -that ordering as it iterates. - -Note that this change does **not** yet apply to the ordering applied -when emitting :meth:`_schema.MetaData.create_all` or :meth:`_schema.MetaData.drop_all`. - -:ticket:`3084` - -.. _bug_3170: - -null(), false() and true() constants are no longer singletons -------------------------------------------------------------- - -These three constants were changed to return a "singleton" value -in 0.9; unfortunately, that would lead to a query like the following -to not render as expected:: - - select([null(), null()]) - -rendering only ``SELECT NULL AS anon_1``, because the two :func:`.null` -constructs would come out as the same ``NULL`` object, and -SQLAlchemy's Core model is based on object identity in order to -determine lexical significance. The change in 0.9 had no -importance other than the desire to save on object overhead; in general, -an unnamed construct needs to stay lexically unique so that it gets -labeled uniquely. - -:ticket:`3170` - -.. _change_3204: - -SQLite/Oracle have distinct methods for temporary table/view name reporting ---------------------------------------------------------------------------- - -The :meth:`_reflection.Inspector.get_table_names` and :meth:`_reflection.Inspector.get_view_names` -methods in the case of SQLite/Oracle would also return the names of temporary -tables and views, which is not provided by any other dialect (in the case -of MySQL at least it is not even possible). This logic has been moved -out to two new methods :meth:`_reflection.Inspector.get_temp_table_names` and -:meth:`_reflection.Inspector.get_temp_view_names`. - -Note that reflection of a specific named temporary table or temporary view, -either by ``Table('name', autoload=True)`` or via methods like -:meth:`_reflection.Inspector.get_columns` continues to function for most if not all -dialects. For SQLite specifically, there is a bug fix for UNIQUE constraint -reflection from temp tables as well, which is :ticket:`3203`. - -:ticket:`3204` - -Dialect Improvements and Changes - PostgreSQL -============================================= - -.. _change_3319: - -Overhaul of ENUM type create/drop rules ---------------------------------------- - -The rules for PostgreSQL :class:`_postgresql.ENUM` have been made more strict -with regards to creating and dropping of the TYPE. - -An :class:`_postgresql.ENUM` that is created **without** being explicitly -associated with a :class:`_schema.MetaData` object will be created *and* dropped -corresponding to :meth:`_schema.Table.create` and :meth:`_schema.Table.drop`:: - - table = Table( - "sometable", metadata, Column("some_enum", ENUM("a", "b", "c", name="myenum")) - ) - - table.create(engine) # will emit CREATE TYPE and CREATE TABLE - table.drop(engine) # will emit DROP TABLE and DROP TYPE - new for 1.0 - -This means that if a second table also has an enum named 'myenum', the -above DROP operation will now fail. In order to accommodate the use case -of a common shared enumerated type, the behavior of a metadata-associated -enumeration has been enhanced. - -An :class:`_postgresql.ENUM` that is created **with** being explicitly -associated with a :class:`_schema.MetaData` object will *not* be created *or* dropped -corresponding to :meth:`_schema.Table.create` and :meth:`_schema.Table.drop`, with -the exception of :meth:`_schema.Table.create` called with the ``checkfirst=True`` -flag:: - - my_enum = ENUM("a", "b", "c", name="myenum", metadata=metadata) - - table = Table("sometable", metadata, Column("some_enum", my_enum)) - - # will fail: ENUM 'my_enum' does not exist - table.create(engine) - - # will check for enum and emit CREATE TYPE - table.create(engine, checkfirst=True) - - table.drop(engine) # will emit DROP TABLE, *not* DROP TYPE - - metadata.drop_all(engine) # will emit DROP TYPE - - metadata.create_all(engine) # will emit CREATE TYPE - -:ticket:`3319` - -New PostgreSQL Table options ----------------------------- - -Added support for PG table options TABLESPACE, ON COMMIT, -WITH(OUT) OIDS, and INHERITS, when rendering DDL via -the :class:`_schema.Table` construct. - -.. seealso:: - - :ref:`postgresql_table_options` - -:ticket:`2051` - -.. _feature_get_enums: - -New get_enums() method with PostgreSQL Dialect ----------------------------------------------- - -The :func:`_sa.inspect` method returns a :class:`.PGInspector` object in the -case of PostgreSQL, which includes a new :meth:`.PGInspector.get_enums` -method that returns information on all available ``ENUM`` types:: - - from sqlalchemy import inspect, create_engine - - engine = create_engine("postgresql+psycopg2://host/dbname") - insp = inspect(engine) - print(insp.get_enums()) - -.. seealso:: - - :meth:`.PGInspector.get_enums` - -.. _feature_2891: - -PostgreSQL Dialect reflects Materialized Views, Foreign Tables --------------------------------------------------------------- - -Changes are as follows: - -* the :class:`Table` construct with ``autoload=True`` will now match a name - that exists in the database as a materialized view or foreign table. - -* :meth:`_reflection.Inspector.get_view_names` will return plain and materialized view - names. - -* :meth:`_reflection.Inspector.get_table_names` does **not** change for PostgreSQL, it - continues to return only the names of plain tables. - -* A new method :meth:`.PGInspector.get_foreign_table_names` is added which - will return the names of tables that are specifically marked as "foreign" - in the PostgreSQL schema tables. - -The change to reflection involves adding ``'m'`` and ``'f'`` to the list -of qualifiers we use when querying ``pg_class.relkind``, but this change -is new in 1.0.0 to avoid any backwards-incompatible surprises for those -running 0.9 in production. - -:ticket:`2891` - -.. _change_3264: - -PostgreSQL ``has_table()`` now works for temporary tables ---------------------------------------------------------- - -This is a simple fix such that "has table" for temporary tables now works, -so that code like the following may proceed:: - - from sqlalchemy import * - - metadata = MetaData() - user_tmp = Table( - "user_tmp", - metadata, - Column("id", INT, primary_key=True), - Column("name", VARCHAR(50)), - prefixes=["TEMPORARY"], - ) - - e = create_engine("postgresql://scott:tiger@localhost/test", echo="debug") - with e.begin() as conn: - user_tmp.create(conn, checkfirst=True) - - # checkfirst will succeed - user_tmp.create(conn, checkfirst=True) - -The very unlikely case that this behavior will cause a non-failing application -to behave differently, is because PostgreSQL allows a non-temporary table -to silently overwrite a temporary table. So code like the following will -now act completely differently, no longer creating the real table following -the temporary table:: - - from sqlalchemy import * - - metadata = MetaData() - user_tmp = Table( - "user_tmp", - metadata, - Column("id", INT, primary_key=True), - Column("name", VARCHAR(50)), - prefixes=["TEMPORARY"], - ) - - e = create_engine("postgresql://scott:tiger@localhost/test", echo="debug") - with e.begin() as conn: - user_tmp.create(conn, checkfirst=True) - - m2 = MetaData() - user = Table( - "user_tmp", - m2, - Column("id", INT, primary_key=True), - Column("name", VARCHAR(50)), - ) - - # in 0.9, *will create* the new table, overwriting the old one. - # in 1.0, *will not create* the new table - user.create(conn, checkfirst=True) - -:ticket:`3264` - -.. _feature_gh134: - -PostgreSQL FILTER keyword -------------------------- - -The SQL standard FILTER keyword for aggregate functions is now supported -by PostgreSQL as of 9.4. SQLAlchemy allows this using -:meth:`.FunctionElement.filter`:: - - func.count(1).filter(True) - -.. seealso:: - - :meth:`.FunctionElement.filter` - - :class:`.FunctionFilter` - -PG8000 dialect supports client side encoding --------------------------------------------- - -The :paramref:`_sa.create_engine.encoding` parameter is now honored -by the pg8000 dialect, using on connect handler which -emits ``SET CLIENT_ENCODING`` matching the selected encoding. - -PG8000 native JSONB support ---------------------------- - -Support for PG8000 versions greater than 1.10.1 has been added, where -JSONB is supported natively. - - -Support for psycopg2cffi Dialect on PyPy ----------------------------------------- - -Support for the pypy psycopg2cffi dialect is added. - -.. seealso:: - - :mod:`sqlalchemy.dialects.postgresql.psycopg2cffi` - -Dialect Improvements and Changes - MySQL -======================================== - -.. _change_3155: - -MySQL TIMESTAMP Type now renders NULL / NOT NULL in all cases -------------------------------------------------------------- - -The MySQL dialect has always worked around MySQL's implicit NOT NULL -default associated with TIMESTAMP columns by emitting NULL for -such a type, if the column is set up with ``nullable=True``. However, -MySQL 5.6.6 and above features a new flag -`explicit_defaults_for_timestamp `_ which repairs MySQL's non-standard -behavior to make it behave like any other type; to accommodate this, -SQLAlchemy now emits NULL/NOT NULL unconditionally for all TIMESTAMP -columns. - -.. seealso:: - - :ref:`mysql_timestamp_null` - -:ticket:`3155` - - -.. _change_3283: - -MySQL SET Type Overhauled to support empty sets, unicode, blank value handling ------------------------------------------------------------------------------- - -The :class:`.mysql.SET` type historically not included a system of handling -blank sets and empty values separately; as different drivers had different -behaviors for treatment of empty strings and empty-string-set representations, -the SET type tried only to hedge between these behaviors, opting to treat the -empty set as ``set([''])`` as is still the current behavior for the -MySQL-Connector-Python DBAPI. -Part of the rationale here was that it was otherwise impossible to actually -store a blank string within a MySQL SET, as the driver gives us back strings -with no way to discern between ``set([''])`` and ``set()``. It was left -to the user to determine if ``set([''])`` actually meant "empty set" or not. - -The new behavior moves the use case for the blank string, which is an unusual -case that isn't even documented in MySQL's documentation, into a special -case, and the default behavior of :class:`.mysql.SET` is now: - -* to treat the empty string ``''`` as returned by MySQL-python into the empty - set ``set()``; - -* to convert the single-blank value set ``set([''])`` returned by - MySQL-Connector-Python into the empty set ``set()``; - -* To handle the case of a set type that actually wishes includes the blank - value ``''`` in its list of possible values, - a new feature (required in this use case) is implemented whereby the set - value is persisted and loaded as a bitwise integer value; the - flag :paramref:`.mysql.SET.retrieve_as_bitwise` is added in order to - enable this. - -Using the :paramref:`.mysql.SET.retrieve_as_bitwise` flag allows the set -to be persisted and retrieved with no ambiguity of values. Theoretically -this flag can be turned on in all cases, as long as the given list of -values to the type matches the ordering exactly as declared in the -database; it only makes the SQL echo output a bit more unusual. - -The default behavior of :class:`.mysql.SET` otherwise remains the same, -roundtripping values using strings. The string-based behavior now -supports unicode fully including MySQL-python with use_unicode=0. - -:ticket:`3283` - - -MySQL internal "no such table" exceptions not passed to event handlers ----------------------------------------------------------------------- - -The MySQL dialect will now disable :meth:`_events.ConnectionEvents.handle_error` -events from firing for those statements which it uses internally -to detect if a table exists or not. This is achieved using an -execution option ``skip_user_error_events`` that disables the handle -error event for the scope of that execution. In this way, user code -that rewrites exceptions doesn't need to worry about the MySQL -dialect or other dialects that occasionally need to catch -SQLAlchemy specific exceptions. - - -Changed the default value of ``raise_on_warnings`` for MySQL-Connector ----------------------------------------------------------------------- - -Changed the default value of "raise_on_warnings" to False for -MySQL-Connector. This was set at True for some reason. The "buffered" -flag unfortunately must stay at True as MySQLconnector does not allow -a cursor to be closed unless all results are fully fetched. - -:ticket:`2515` - -.. _bug_3186: - -MySQL boolean symbols "true", "false" work again ------------------------------------------------- - -0.9's overhaul of the IS/IS NOT operators as well as boolean types in -:ticket:`2682` disallowed the MySQL dialect from making use of the -"true" and "false" symbols in the context of "IS" / "IS NOT". Apparently, -even though MySQL has no "boolean" type, it supports IS / IS NOT when the -special "true" and "false" symbols are used, even though these are otherwise -synonymous with "1" and "0" (and IS/IS NOT don't work with the numerics). - -So the change here is that the MySQL dialect remains "non native boolean", -but the :func:`.true` and :func:`.false` symbols again produce the -keywords "true" and "false", so that an expression like ``column.is_(true())`` -again works on MySQL. - -:ticket:`3186` - -.. _change_3263: - -The match() operator now returns an agnostic MatchType compatible with MySQL's floating point return value ----------------------------------------------------------------------------------------------------------- - -The return type of a :meth:`.ColumnOperators.match` expression is now a new type -called :class:`.MatchType`. This is a subclass of :class:`.Boolean`, -that can be intercepted by the dialect in order to produce a different -result type at SQL execution time. - -Code like the following will now function correctly and return floating points -on MySQL:: - - >>> connection.execute( - ... select( - ... [ - ... matchtable.c.title.match("Agile Ruby Programming").label("ruby"), - ... matchtable.c.title.match("Dive Python").label("python"), - ... matchtable.c.title, - ... ] - ... ).order_by(matchtable.c.id) - ... ) - [ - (2.0, 0.0, 'Agile Web Development with Ruby On Rails'), - (0.0, 2.0, 'Dive Into Python'), - (2.0, 0.0, "Programming Matz's Ruby"), - (0.0, 0.0, 'The Definitive Guide to Django'), - (0.0, 1.0, 'Python in a Nutshell') - ] - - -:ticket:`3263` - -.. _change_2984: - -Drizzle Dialect is now an External Dialect ------------------------------------------- - -The dialect for `Drizzle `_ is now an external -dialect, available at https://bitbucket.org/zzzeek/sqlalchemy-drizzle. -This dialect was added to SQLAlchemy right before SQLAlchemy was able to -accommodate third party dialects well; going forward, all databases that aren't -within the "ubiquitous use" category are third party dialects. -The dialect's implementation hasn't changed and is still based on the -MySQL + MySQLdb dialects within SQLAlchemy. The dialect is as of yet -unreleased and in "attic" status; however it passes the majority of tests -and is generally in decent working order, if someone wants to pick up -on polishing it. - -Dialect Improvements and Changes - SQLite -========================================= - -SQLite named and unnamed UNIQUE and FOREIGN KEY constraints will inspect and reflect -------------------------------------------------------------------------------------- - -UNIQUE and FOREIGN KEY constraints are now fully reflected on -SQLite both with and without names. Previously, foreign key -names were ignored and unnamed unique constraints were skipped. In particular -this will help with Alembic's new SQLite migration features. - -To achieve this, for both foreign keys and unique constraints, the result -of PRAGMA foreign_keys, index_list, and index_info is combined with regular -expression parsing of the CREATE TABLE statement overall to form a complete -picture of the names of constraints, as well as differentiating UNIQUE -constraints that were created as UNIQUE vs. unnamed INDEXes. - -:ticket:`3244` - -:ticket:`3261` - -Dialect Improvements and Changes - SQL Server -============================================= - -.. _change_3182: - -PyODBC driver name is required with hostname-based SQL Server connections -------------------------------------------------------------------------- - -Connecting to SQL Server with PyODBC using a DSN-less connection, e.g. -with an explicit hostname, now requires a driver name - SQLAlchemy will no -longer attempt to guess a default:: - - engine = create_engine( - "mssql+pyodbc://scott:tiger@myhost:port/databasename?driver=SQL+Server+Native+Client+10.0" - ) - -SQLAlchemy's previously hardcoded default of "SQL Server" is obsolete on -Windows, and SQLAlchemy cannot be tasked with guessing the best driver -based on operation system/driver detection. Using a DSN is always preferred -when using ODBC to avoid this issue entirely. - -:ticket:`3182` - -SQL Server 2012 large text / binary types render as VARCHAR, NVARCHAR, VARBINARY --------------------------------------------------------------------------------- - -The rendering of the :class:`_expression.TextClause`, :class:`.UnicodeText`, and :class:`.LargeBinary` -types has been changed for SQL Server 2012 and greater, with options -to control the behavior completely, based on deprecation guidelines from -Microsoft. See :ref:`mssql_large_type_deprecation` for details. - -Dialect Improvements and Changes - Oracle -========================================= - -.. _change_3220: - -Improved support for CTEs in Oracle ------------------------------------ - -CTE support has been fixed up for Oracle, and there is also a new feature -:meth:`_expression.CTE.with_suffixes` that can assist with Oracle's special directives:: - - included_parts = ( - select([part.c.sub_part, part.c.part, part.c.quantity]) - .where(part.c.part == "p1") - .cte(name="included_parts", recursive=True) - .suffix_with( - "search depth first by part set ord1", - "cycle part set y_cycle to 1 default 0", - dialect="oracle", - ) - ) - -:ticket:`3220` - -New Oracle Keywords for DDL ---------------------------- - -Keywords such as COMPRESS, ON COMMIT, BITMAP: - -:ref:`oracle_table_options` - -:ref:`oracle_index_options` diff --git a/doc/build/changelog/migration_11.rst b/doc/build/changelog/migration_11.rst deleted file mode 100644 index 8a1ba3ba0e..0000000000 --- a/doc/build/changelog/migration_11.rst +++ /dev/null @@ -1,3036 +0,0 @@ -============================= -What's New in SQLAlchemy 1.1? -============================= - -.. admonition:: About this Document - - This document describes changes between SQLAlchemy version 1.0 - and SQLAlchemy version 1.1. - -Introduction -============ - -This guide introduces what's new in SQLAlchemy version 1.1, -and also documents changes which affect users migrating -their applications from the 1.0 series of SQLAlchemy to 1.1. - -Please carefully review the sections on behavioral changes for -potentially backwards-incompatible changes in behavior. - -Platform / Installer Changes -============================ - -Setuptools is now required for install --------------------------------------- - -SQLAlchemy's ``setup.py`` file has for many years supported operation -both with Setuptools installed and without; supporting a "fallback" mode -that uses straight Distutils. As a Setuptools-less Python environment is -now unheard of, and in order to support the featureset of Setuptools -more fully, in particular to support py.test's integration with it as well -as things like "extras", ``setup.py`` now depends on Setuptools fully. - -.. seealso:: - - :ref:`installation` - -:ticket:`3489` - -Enabling / Disabling C Extension builds is only via environment variable ------------------------------------------------------------------------- - -The C Extensions build by default during install as long as it is possible. -To disable C extension builds, the ``DISABLE_SQLALCHEMY_CEXT`` environment -variable was made available as of SQLAlchemy 0.8.6 / 0.9.4. The previous -approach of using the ``--without-cextensions`` argument has been removed, -as it relies on deprecated features of setuptools. - -.. seealso:: - - :ref:`c_extensions` - -:ticket:`3500` - - -New Features and Improvements - ORM -=================================== - -.. _change_2677: - -New Session lifecycle events ----------------------------- - -The :class:`.Session` has long supported events that allow some degree -of tracking of state changes to objects, including -:meth:`.SessionEvents.before_attach`, :meth:`.SessionEvents.after_attach`, -and :meth:`.SessionEvents.before_flush`. The Session documentation also -documents major object states at :ref:`session_object_states`. However, -there has never been system of tracking objects specifically as they -pass through these transitions. Additionally, the status of "deleted" objects -has historically been murky as the objects act somewhere between -the "persistent" and "detached" states. - -To clean up this area and allow the realm of session state transition -to be fully transparent, a new series of events have been added that -are intended to cover every possible way that an object might transition -between states, and additionally the "deleted" status has been given -its own official state name within the realm of session object states. - -New State Transition Events -^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Transitions between all states of an object such as :term:`persistent`, -:term:`pending` and others can now be intercepted in terms of a -session-level event intended to cover a specific transition. -Transitions as objects move into a :class:`.Session`, move out of a -:class:`.Session`, and even all the transitions which occur when the -transaction is rolled back using :meth:`.Session.rollback` -are explicitly present in the interface of :class:`.SessionEvents`. - -In total, there are **ten new events**. A summary of these events is in a -newly written documentation section :ref:`session_lifecycle_events`. - - -New Object State "deleted" is added, deleted objects no longer "persistent" -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The :term:`persistent` state of an object in the :class:`.Session` has -always been documented as an object that has a valid database identity; -however in the case of objects that were deleted within a flush, they -have always been in a grey area where they are not really "detached" -from the :class:`.Session` yet, because they can still be restored -within a rollback, but are not really "persistent" because their database -identity has been deleted and they aren't present in the identity map. - -To resolve this grey area given the new events, a new object state -:term:`deleted` is introduced. This state exists between the "persistent" and -"detached" states. An object that is marked for deletion via -:meth:`.Session.delete` remains in the "persistent" state until a flush -proceeds; at that point, it is removed from the identity map, moves -to the "deleted" state, and the :meth:`.SessionEvents.persistent_to_deleted` -hook is invoked. If the :class:`.Session` object's transaction is rolled -back, the object is restored as persistent; the -:meth:`.SessionEvents.deleted_to_persistent` transition is called. Otherwise -if the :class:`.Session` object's transaction is committed, -the :meth:`.SessionEvents.deleted_to_detached` transition is invoked. - -Additionally, the :attr:`.InstanceState.persistent` accessor **no longer returns -True** for an object that is in the new "deleted" state; instead, the -:attr:`.InstanceState.deleted` accessor has been enhanced to reliably -report on this new state. When the object is detached, the :attr:`.InstanceState.deleted` -returns False and the :attr:`.InstanceState.detached` accessor is True -instead. To determine if an object was deleted either in the current -transaction or in a previous transaction, use the -:attr:`.InstanceState.was_deleted` accessor. - -Strong Identity Map is Deprecated -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -One of the inspirations for the new series of transition events was to enable -leak-proof tracking of objects as they move in and out of the identity map, -so that a "strong reference" may be maintained mirroring the object -moving in and out of this map. With this new capability, there is no longer -any need for the :paramref:`.Session.weak_identity_map` parameter and the -corresponding :class:`.StrongIdentityMap` object. This option has remained -in SQLAlchemy for many years as the "strong-referencing" behavior used to be -the only behavior available, and many applications were written to assume -this behavior. It has long been recommended that strong-reference tracking -of objects not be an intrinsic job of the :class:`.Session` and instead -be an application-level construct built as needed by the application; the -new event model allows even the exact behavior of the strong identity map -to be replicated. See :ref:`session_referencing_behavior` for a new -recipe illustrating how to replace the strong identity map. - -:ticket:`2677` - -.. _change_1311: - -New init_scalar() event intercepts default values at ORM level --------------------------------------------------------------- - -The ORM produces a value of ``None`` when an attribute that has not been -set is first accessed, for a non-persistent object:: - - >>> obj = MyObj() - >>> obj.some_value - None - -There's a use case for this in-Python value to correspond to that of a -Core-generated default value, even before the object is persisted. -To suit this use case a new event :meth:`.AttributeEvents.init_scalar` -is added. The new example ``active_column_defaults.py`` at -:ref:`examples_instrumentation` illustrates a sample use, so the effect -can instead be:: - - >>> obj = MyObj() - >>> obj.some_value - "my default" - -:ticket:`1311` - -.. _change_3499: - -Changes regarding "unhashable" types, impacts deduping of ORM rows ------------------------------------------------------------------- - -The :class:`_query.Query` object has a well-known behavior of "deduping" -returned rows that contain at least one ORM-mapped entity (e.g., a -full mapped object, as opposed to individual column values). The -primary purpose of this is so that the handling of entities works -smoothly in conjunction with the identity map, including to -accommodate for the duplicate entities normally represented within -joined eager loading, as well as when joins are used for the purposes -of filtering on additional columns. - -This deduplication relies upon the hashability of the elements within -the row. With the introduction of PostgreSQL's special types like -:class:`_postgresql.ARRAY`, :class:`_postgresql.HSTORE` and -:class:`_postgresql.JSON`, the experience of types within rows being -unhashable and encountering problems here is more prevalent than -it was previously. - -In fact, SQLAlchemy has since version 0.8 included a flag on datatypes that -are noted as "unhashable", however this flag was not used consistently -on built in types. As described in :ref:`change_3499_postgresql`, this -flag is now set consistently for all of PostgreSQL's "structural" types. - -The "unhashable" flag is also set on the :class:`.NullType` type, -as :class:`.NullType` is used to refer to any expression of unknown -type. - -Since :class:`.NullType` is applied to most -usages of :attr:`.func`, as :attr:`.func` doesn't actually know anything -about the function names given in most cases, **using func() will -often disable row deduping unless explicit typing is applied**. -The following examples illustrate ``func.substr()`` applied to a string -expression, and ``func.date()`` applied to a datetime expression; both -examples will return duplicate rows due to the joined eager load unless -explicit typing is applied:: - - result = ( - session.query(func.substr(A.some_thing, 0, 4), A).options(joinedload(A.bs)).all() - ) - - users = ( - session.query( - func.date(User.date_created, "start of month").label("month"), - User, - ) - .options(joinedload(User.orders)) - .all() - ) - -The above examples, in order to retain deduping, should be specified as:: - - result = ( - session.query(func.substr(A.some_thing, 0, 4, type_=String), A) - .options(joinedload(A.bs)) - .all() - ) - - users = ( - session.query( - func.date(User.date_created, "start of month", type_=DateTime).label("month"), - User, - ) - .options(joinedload(User.orders)) - .all() - ) - -Additionally, the treatment of a so-called "unhashable" type is slightly -different than its been in previous releases; internally we are using -the ``id()`` function to get a "hash value" from these structures, just -as we would any ordinary mapped object. This replaces the previous -approach which applied a counter to the object. - -:ticket:`3499` - -.. _change_3321: - -Specific checks added for passing mapped classes, instances as SQL literals ---------------------------------------------------------------------------- - -The typing system now has specific checks for passing of SQLAlchemy -"inspectable" objects in contexts where they would otherwise be handled as -literal values. Any SQLAlchemy built-in object that is legal to pass as a -SQL value (which is not already a :class:`_expression.ClauseElement` instance) -includes a method ``__clause_element__()`` which provides a -valid SQL expression for that object. For SQLAlchemy objects that -don't provide this, such as mapped classes, mappers, and mapped -instances, a more informative error message is emitted rather than -allowing the DBAPI to receive the object and fail later. An example -is illustrated below, where a string-based attribute ``User.name`` is -compared to a full instance of ``User()``, rather than against a -string value:: - - >>> some_user = User() - >>> q = s.query(User).filter(User.name == some_user) - sqlalchemy.exc.ArgumentError: Object <__main__.User object at 0x103167e90> is not legal as a SQL literal value - -The exception is now immediate when the comparison is made between -``User.name == some_user``. Previously, a comparison like the above -would produce a SQL expression that would only fail once resolved -into a DBAPI execution call; the mapped ``User`` object would -ultimately become a bound parameter that would be rejected by the -DBAPI. - -Note that in the above example, the expression fails because -``User.name`` is a string-based (e.g. column oriented) attribute. -The change does *not* impact the usual case of comparing a many-to-one -relationship attribute to an object, which is handled distinctly:: - - >>> # Address.user refers to the User mapper, so - >>> # this is of course still OK! - >>> q = s.query(Address).filter(Address.user == some_user) - - -:ticket:`3321` - -.. _feature_indexable: - -New Indexable ORM extension ---------------------------- - -The :ref:`indexable_toplevel` extension is an extension to the hybrid -attribute feature which allows the construction of attributes which -refer to specific elements of an "indexable" data type, such as an array -or JSON field:: - - class Person(Base): - __tablename__ = "person" - - id = Column(Integer, primary_key=True) - data = Column(JSON) - - name = index_property("data", "name") - -Above, the ``name`` attribute will read/write the field ``"name"`` -from the JSON column ``data``, after initializing it to an -empty dictionary:: - - >>> person = Person(name="foobar") - >>> person.name - foobar - -The extension also triggers a change event when the attribute is modified, -so that there's no need to use :class:`~.mutable.MutableDict` in order -to track this change. - -.. seealso:: - - :ref:`indexable_toplevel` - -.. _change_3250: - -New options allowing explicit persistence of NULL over a default ----------------------------------------------------------------- - -Related to the new JSON-NULL support added to PostgreSQL as part of -:ref:`change_3514`, the base :class:`.TypeEngine` class now supports -a method :meth:`.TypeEngine.evaluates_none` which allows a positive set -of the ``None`` value on an attribute to be persisted as NULL, rather than -omitting the column from the INSERT statement, which has the effect of using -the column-level default. This allows a mapper-level -configuration of the existing object-level technique of assigning -:func:`_expression.null` to the attribute. - -.. seealso:: - - :ref:`session_forcing_null` - -:ticket:`3250` - - -.. _change_3582: - -Further Fixes to single-table inheritance querying --------------------------------------------------- - -Continuing from 1.0's :ref:`migration_3177`, the :class:`_query.Query` should -no longer inappropriately add the "single inheritance" criteria when the -query is against a subquery expression such as an exists:: - - class Widget(Base): - __tablename__ = "widget" - id = Column(Integer, primary_key=True) - type = Column(String) - data = Column(String) - __mapper_args__ = {"polymorphic_on": type} - - - class FooWidget(Widget): - __mapper_args__ = {"polymorphic_identity": "foo"} - - - q = session.query(FooWidget).filter(FooWidget.data == "bar").exists() - - session.query(q).all() - -Produces: - -.. sourcecode:: sql - - SELECT EXISTS (SELECT 1 - FROM widget - WHERE widget.data = :data_1 AND widget.type IN (:type_1)) AS anon_1 - -The IN clause on the inside is appropriate, in order to limit to FooWidget -objects, however previously the IN clause would also be generated a second -time on the outside of the subquery. - -:ticket:`3582` - -.. _change_3680: - -Improved Session state when a SAVEPOINT is cancelled by the database --------------------------------------------------------------------- - -A common case with MySQL is that a SAVEPOINT is cancelled when a deadlock -occurs within the transaction. The :class:`.Session` has been modified -to deal with this failure mode slightly more gracefully, such that the -outer, non-savepoint transaction still remains usable:: - - s = Session() - s.begin_nested() - - s.add(SomeObject()) - - try: - # assume the flush fails, flush goes to rollback to the - # savepoint and that also fails - s.flush() - except Exception as err: - print("Something broke, and our SAVEPOINT vanished too") - - # this is the SAVEPOINT transaction, marked as - # DEACTIVE so the rollback() call succeeds - s.rollback() - - # this is the outermost transaction, remains ACTIVE - # so rollback() or commit() can succeed - s.rollback() - -This issue is a continuation of :ticket:`2696` where we emit a warning -so that the original error can be seen when running on Python 2, even though -the SAVEPOINT exception takes precedence. On Python 3, exceptions are chained -so both failures are reported individually. - - -:ticket:`3680` - -.. _change_3677: - -Erroneous "new instance X conflicts with persistent instance Y" flush errors fixed ----------------------------------------------------------------------------------- - -The :meth:`.Session.rollback` method is responsible for removing objects -that were INSERTed into the database, e.g. moved from pending to persistent, -within that now rolled-back transaction. Objects that make this state -change are tracked in a weak-referencing collection, and if an object is -garbage collected from that collection, the :class:`.Session` no longer worries -about it (it would otherwise not scale for operations that insert many new -objects within a transaction). However, an issue arises if the application -re-loads that same garbage-collected row within the transaction, before the -rollback occurs; if a strong reference to this object remains into the next -transaction, the fact that this object was not inserted and should be -removed would be lost, and the flush would incorrectly raise an error:: - - from sqlalchemy import Column, create_engine - from sqlalchemy.orm import Session - from sqlalchemy.ext.declarative import declarative_base - - Base = declarative_base() - - - class A(Base): - __tablename__ = "a" - id = Column(Integer, primary_key=True) - - - e = create_engine("sqlite://", echo=True) - Base.metadata.create_all(e) - - s = Session(e) - - # persist an object - s.add(A(id=1)) - s.flush() - - # rollback buffer loses reference to A - - # load it again, rollback buffer knows nothing - # about it - a1 = s.query(A).first() - - # roll back the transaction; all state is expired but the - # "a1" reference remains - s.rollback() - - # previous "a1" conflicts with the new one because we aren't - # checking that it never got committed - s.add(A(id=1)) - s.commit() - -The above program would raise: - -.. sourcecode:: text - - FlushError: New instance with identity key - (, ('u1',)) conflicts - with persistent instance - -The bug is that when the above exception is raised, the unit of work -is operating upon the original object assuming it's a live row, when in -fact the object is expired and upon testing reveals that it's gone. The -fix tests this condition now, so in the SQL log we see: - -.. sourcecode:: sql - - BEGIN (implicit) - - INSERT INTO a (id) VALUES (?) - (1,) - - SELECT a.id AS a_id FROM a LIMIT ? OFFSET ? - (1, 0) - - ROLLBACK - - BEGIN (implicit) - - SELECT a.id AS a_id FROM a WHERE a.id = ? - (1,) - - INSERT INTO a (id) VALUES (?) - (1,) - - COMMIT - -Above, the unit of work now does a SELECT for the row we're about to report -as a conflict for, sees that it doesn't exist, and proceeds normally. -The expense of this SELECT is only incurred in the case when we would have -erroneously raised an exception in any case. - - -:ticket:`3677` - -.. _change_2349: - -passive_deletes feature for joined-inheritance mappings -------------------------------------------------------- - -A joined-table inheritance mapping may now allow a DELETE to proceed -as a result of :meth:`.Session.delete`, which only emits DELETE for the -base table, and not the subclass table, allowing configured ON DELETE CASCADE -to take place for the configured foreign keys. This is configured using -the :paramref:`.orm.mapper.passive_deletes` option:: - - from sqlalchemy import Column, Integer, String, ForeignKey, create_engine - from sqlalchemy.orm import Session - from sqlalchemy.ext.declarative import declarative_base - - Base = declarative_base() - - - class A(Base): - __tablename__ = "a" - id = Column("id", Integer, primary_key=True) - type = Column(String) - - __mapper_args__ = { - "polymorphic_on": type, - "polymorphic_identity": "a", - "passive_deletes": True, - } - - - class B(A): - __tablename__ = "b" - b_table_id = Column("b_table_id", Integer, primary_key=True) - bid = Column("bid", Integer, ForeignKey("a.id", ondelete="CASCADE")) - data = Column("data", String) - - __mapper_args__ = {"polymorphic_identity": "b"} - -With the above mapping, the :paramref:`.orm.mapper.passive_deletes` option -is configured on the base mapper; it takes effect for all non-base mappers -that are descendants of the mapper with the option set. A DELETE for -an object of type ``B`` no longer needs to retrieve the primary key value -of ``b_table_id`` if unloaded, nor does it need to emit a DELETE statement -for the table itself:: - - session.delete(some_b) - session.commit() - -Will emit SQL as: - -.. sourcecode:: sql - - DELETE FROM a WHERE a.id = %(id)s - -- {'id': 1} - COMMIT - -As always, the target database must have foreign key support with -ON DELETE CASCADE enabled. - -:ticket:`2349` - -.. _change_3630: - -Same-named backrefs will not raise an error when applied to concrete inheritance subclasses -------------------------------------------------------------------------------------------- - -The following mapping has always been possible without issue:: - - class A(Base): - __tablename__ = "a" - id = Column(Integer, primary_key=True) - b = relationship("B", foreign_keys="B.a_id", backref="a") - - - class A1(A): - __tablename__ = "a1" - id = Column(Integer, primary_key=True) - b = relationship("B", foreign_keys="B.a1_id", backref="a1") - __mapper_args__ = {"concrete": True} - - - class B(Base): - __tablename__ = "b" - id = Column(Integer, primary_key=True) - - a_id = Column(ForeignKey("a.id")) - a1_id = Column(ForeignKey("a1.id")) - -Above, even though class ``A`` and class ``A1`` have a relationship -named ``b``, no conflict warning or error occurs because class ``A1`` is -marked as "concrete". - -However, if the relationships were configured the other way, an error -would occur:: - - class A(Base): - __tablename__ = "a" - id = Column(Integer, primary_key=True) - - - class A1(A): - __tablename__ = "a1" - id = Column(Integer, primary_key=True) - __mapper_args__ = {"concrete": True} - - - class B(Base): - __tablename__ = "b" - id = Column(Integer, primary_key=True) - - a_id = Column(ForeignKey("a.id")) - a1_id = Column(ForeignKey("a1.id")) - - a = relationship("A", backref="b") - a1 = relationship("A1", backref="b") - -The fix enhances the backref feature so that an error is not emitted, -as well as an additional check within the mapper logic to bypass warning -for an attribute being replaced. - -:ticket:`3630` - -.. _change_3749: - -Same-named relationships on inheriting mappers no longer warn -------------------------------------------------------------- - -When creating two mappers in an inheritance scenario, placing a relationship -on both with the same name would emit the warning -"relationship '' on mapper supersedes the same relationship -on inherited mapper ''; this can cause dependency issues during flush". -An example is as follows:: - - class A(Base): - __tablename__ = "a" - id = Column(Integer, primary_key=True) - bs = relationship("B") - - - class ASub(A): - __tablename__ = "a_sub" - id = Column(Integer, ForeignKey("a.id"), primary_key=True) - bs = relationship("B") - - - class B(Base): - __tablename__ = "b" - id = Column(Integer, primary_key=True) - a_id = Column(ForeignKey("a.id")) - -This warning dates back to the 0.4 series in 2007 and is based on a version of -the unit of work code that has since been entirely rewritten. Currently, there -is no known issue with the same-named relationships being placed on a base -class and a descendant class, so the warning is lifted. However, note that -this use case is likely not prevalent in real world use due to the warning. -While rudimentary test support is added for this use case, it is possible that -some new issue with this pattern may be identified. - -.. versionadded:: 1.1.0b3 - -:ticket:`3749` - -.. _change_3653: - -Hybrid properties and methods now propagate the docstring as well as .info --------------------------------------------------------------------------- - -A hybrid method or property will now reflect the ``__doc__`` value -present in the original docstring:: - - class A(Base): - __tablename__ = "a" - id = Column(Integer, primary_key=True) - - name = Column(String) - - @hybrid_property - def some_name(self): - """The name field""" - return self.name - -The above value of ``A.some_name.__doc__`` is now honored:: - - >>> A.some_name.__doc__ - The name field - -However, to accomplish this, the mechanics of hybrid properties necessarily -becomes more complex. Previously, the class-level accessor for a hybrid -would be a simple pass-through, that is, this test would succeed:: - - >>> assert A.name is A.some_name - -With the change, the expression returned by ``A.some_name`` is wrapped inside -of its own ``QueryableAttribute`` wrapper:: - - >>> A.some_name - - -A lot of testing went into making sure this wrapper works correctly, including -for elaborate schemes like that of the -`Custom Value Object `_ -recipe, however we'll be looking to see that no other regressions occur for -users. - -As part of this change, the :attr:`.hybrid_property.info` collection is now -also propagated from the hybrid descriptor itself, rather than from the underlying -expression. That is, accessing ``A.some_name.info`` now returns the same -dictionary that you'd get from ``inspect(A).all_orm_descriptors['some_name'].info``:: - - >>> A.some_name.info["foo"] = "bar" - >>> from sqlalchemy import inspect - >>> inspect(A).all_orm_descriptors["some_name"].info - {'foo': 'bar'} - -Note that this ``.info`` dictionary is **separate** from that of a mapped attribute -which the hybrid descriptor may be proxying directly; this is a behavioral -change from 1.0. The wrapper will still proxy other useful attributes -of a mirrored attribute such as :attr:`.QueryableAttribute.property` and -:attr:`.QueryableAttribute.class_`. - -:ticket:`3653` - -.. _change_3601: - -Session.merge resolves pending conflicts the same as persistent ---------------------------------------------------------------- - -The :meth:`.Session.merge` method will now track the identities of objects given -within a graph to maintain primary key uniqueness before emitting an INSERT. -When duplicate objects of the same identity are encountered, non-primary-key -attributes are **overwritten** as the objects are encountered, which is -essentially non-deterministic. This behavior matches that of how persistent -objects, that is objects that are already located in the database via -primary key, are already treated, so this behavior is more internally -consistent. - -Given:: - - u1 = User(id=7, name="x") - u1.orders = [ - Order(description="o1", address=Address(id=1, email_address="a")), - Order(description="o2", address=Address(id=1, email_address="b")), - Order(description="o3", address=Address(id=1, email_address="c")), - ] - - sess = Session() - sess.merge(u1) - -Above, we merge a ``User`` object with three new ``Order`` objects, each referring to -a distinct ``Address`` object, however each is given the same primary key. -The current behavior of :meth:`.Session.merge` is to look in the identity -map for this ``Address`` object, and use that as the target. If the object -is present, meaning that the database already has a row for ``Address`` with -primary key "1", we can see that the ``email_address`` field of the ``Address`` -will be overwritten three times, in this case with the values a, b and finally -c. - -However, if the ``Address`` row for primary key "1" were not present, :meth:`.Session.merge` -would instead create three separate ``Address`` instances, and we'd then get -a primary key conflict upon INSERT. The new behavior is that the proposed -primary key for these ``Address`` objects are tracked in a separate dictionary -so that we merge the state of the three proposed ``Address`` objects onto -one ``Address`` object to be inserted. - -It may have been preferable if the original case emitted some kind of warning -that conflicting data were present in a single merge-tree, however the -non-deterministic merging of values has been the behavior for many -years for the persistent case; it now matches for the pending case. A -feature that warns for conflicting values could still be feasible for both -cases but would add considerable performance overhead as each column value -would have to be compared during the merge. - - -:ticket:`3601` - -.. _change_3708: - -Fix involving many-to-one object moves with user-initiated foreign key manipulations ------------------------------------------------------------------------------------- - -A bug has been fixed involving the mechanics of replacing a many-to-one -reference to an object with another object. During the attribute operation, -the location of the object that was previously referred to now makes use of the -database-committed foreign key value, rather than the current foreign key -value. The main effect of the fix is that a backref event towards a collection -will fire off more accurately when a many-to-one change is made, even if the -foreign key attribute was manually moved to the new value beforehand. Assume a -mapping of the classes ``Parent`` and ``SomeClass``, where ``SomeClass.parent`` -refers to ``Parent`` and ``Parent.items`` refers to the collection of -``SomeClass`` objects:: - - some_object = SomeClass() - session.add(some_object) - some_object.parent_id = some_parent.id - some_object.parent = some_parent - -Above, we've made a pending object ``some_object``, manipulated its foreign key -towards ``Parent`` to refer to it, *then* we actually set up the relationship. -Before the bug fix, the backref would not have fired off:: - - # before the fix - assert some_object not in some_parent.items - -The fix now is that when we seek to locate the previous value of -``some_object.parent``, we disregard the parent id that's been manually set, -and we look for the database-committed value. In this case, it's None because -the object is pending, so the event system logs ``some_object.parent`` -as a net change:: - - # after the fix, backref fired off for some_object.parent = some_parent - assert some_object in some_parent.items - -While it is discouraged to manipulate foreign key attributes that are managed -by relationships, there is limited support for this use case. Applications -that manipulate foreign keys in order to allow loads to proceed will often make -use of the :meth:`.Session.enable_relationship_loading` and -:attr:`.RelationshipProperty.load_on_pending` features, which cause -relationships to emit lazy loads based on in-memory foreign key values that -aren't persisted. Whether or not these features are in use, this behavioral -improvement will now be apparent. - -:ticket:`3708` - -.. _change_3662: - -Improvements to the Query.correlate method with polymorphic entities --------------------------------------------------------------------- - -In recent SQLAlchemy versions, the SQL generated by many forms of -"polymorphic" queries has a more "flat" form than it used to, where -a JOIN of several tables is no longer bundled into a subquery unconditionally. -To accommodate this, the :meth:`_query.Query.correlate` method now extracts the -individual tables from such a polymorphic selectable and ensures that all -are part of the "correlate" for the subquery. Assuming the -``Person/Manager/Engineer->Company`` setup from the mapping documentation, -using with_polymorphic:: - - sess.query(Person.name).filter( - sess.query(Company.name) - .filter(Company.company_id == Person.company_id) - .correlate(Person) - .as_scalar() - == "Elbonia, Inc." - ) - -The above query now produces: - -.. sourcecode:: sql - - SELECT people.name AS people_name - FROM people - LEFT OUTER JOIN engineers ON people.person_id = engineers.person_id - LEFT OUTER JOIN managers ON people.person_id = managers.person_id - WHERE (SELECT companies.name - FROM companies - WHERE companies.company_id = people.company_id) = ? - -Before the fix, the call to ``correlate(Person)`` would inadvertently -attempt to correlate to the join of ``Person``, ``Engineer`` and ``Manager`` -as a single unit, so ``Person`` wouldn't be correlated: - -.. sourcecode:: sql - - -- old, incorrect query - SELECT people.name AS people_name - FROM people - LEFT OUTER JOIN engineers ON people.person_id = engineers.person_id - LEFT OUTER JOIN managers ON people.person_id = managers.person_id - WHERE (SELECT companies.name - FROM companies, people - WHERE companies.company_id = people.company_id) = ? - -Using correlated subqueries against polymorphic mappings still has some -unpolished edges. If for example ``Person`` is polymorphically linked -to a so-called "concrete polymorphic union" query, the above subquery -may not correctly refer to this subquery. In all cases, a way to refer -to the "polymorphic" entity fully is to create an :func:`.aliased` object -from it first:: - - # works with all SQLAlchemy versions and all types of polymorphic - # aliasing. - - paliased = aliased(Person) - sess.query(paliased.name).filter( - sess.query(Company.name) - .filter(Company.company_id == paliased.company_id) - .correlate(paliased) - .as_scalar() - == "Elbonia, Inc." - ) - -The :func:`.aliased` construct guarantees that the "polymorphic selectable" -is wrapped in a subquery. By referring to it explicitly in the correlated -subquery, the polymorphic form is correctly used. - -:ticket:`3662` - -.. _change_3081: - -Stringify of Query will consult the Session for the correct dialect -------------------------------------------------------------------- - -Calling ``str()`` on a :class:`_query.Query` object will consult the :class:`.Session` -for the correct "bind" to use, in order to render the SQL that would be -passed to the database. In particular this allows a :class:`_query.Query` that -refers to dialect-specific SQL constructs to be renderable, assuming the -:class:`_query.Query` is associated with an appropriate :class:`.Session`. -Previously, this behavior would only take effect if the :class:`_schema.MetaData` -to which the mappings were associated were itself bound to the target -:class:`_engine.Engine`. - -If neither the underlying :class:`_schema.MetaData` nor the :class:`.Session` are -associated with any bound :class:`_engine.Engine`, then the fallback to the -"default" dialect is used to generate the SQL string. - -.. seealso:: - - :ref:`change_3631` - -:ticket:`3081` - -.. _change_3431: - -Joined eager loading where the same entity is present multiple times in one row -------------------------------------------------------------------------------- - -A fix has been made to the case has been made whereby an attribute will be -loaded via joined eager loading, even if the entity was already loaded from the -row on a different "path" that doesn't include the attribute. This is a -deep use case that's hard to reproduce, but the general idea is as follows:: - - class A(Base): - __tablename__ = "a" - id = Column(Integer, primary_key=True) - b_id = Column(ForeignKey("b.id")) - c_id = Column(ForeignKey("c.id")) - - b = relationship("B") - c = relationship("C") - - - class B(Base): - __tablename__ = "b" - id = Column(Integer, primary_key=True) - c_id = Column(ForeignKey("c.id")) - - c = relationship("C") - - - class C(Base): - __tablename__ = "c" - id = Column(Integer, primary_key=True) - d_id = Column(ForeignKey("d.id")) - d = relationship("D") - - - class D(Base): - __tablename__ = "d" - id = Column(Integer, primary_key=True) - - - c_alias_1 = aliased(C) - c_alias_2 = aliased(C) - - q = s.query(A) - q = q.join(A.b).join(c_alias_1, B.c).join(c_alias_1.d) - q = q.options( - contains_eager(A.b).contains_eager(B.c, alias=c_alias_1).contains_eager(C.d) - ) - q = q.join(c_alias_2, A.c) - q = q.options(contains_eager(A.c, alias=c_alias_2)) - -The above query emits SQL like this: - -.. sourcecode:: sql - - SELECT - d.id AS d_id, - c_1.id AS c_1_id, c_1.d_id AS c_1_d_id, - b.id AS b_id, b.c_id AS b_c_id, - c_2.id AS c_2_id, c_2.d_id AS c_2_d_id, - a.id AS a_id, a.b_id AS a_b_id, a.c_id AS a_c_id - FROM - a - JOIN b ON b.id = a.b_id - JOIN c AS c_1 ON c_1.id = b.c_id - JOIN d ON d.id = c_1.d_id - JOIN c AS c_2 ON c_2.id = a.c_id - -We can see that the ``c`` table is selected from twice; once in the context -of ``A.b.c -> c_alias_1`` and another in the context of ``A.c -> c_alias_2``. -Also, we can see that it is quite possible that the ``C`` identity for a -single row is the **same** for both ``c_alias_1`` and ``c_alias_2``, meaning -two sets of columns in one row result in only one new object being added -to the identity map. - -The query options above only call for the attribute ``C.d`` to be loaded -in the context of ``c_alias_1``, and not ``c_alias_2``. So whether or not -the final ``C`` object we get in the identity map has the ``C.d`` attribute -loaded depends on how the mappings are traversed, which while not completely -random, is essentially non-deterministic. The fix is that even if the -loader for ``c_alias_1`` is processed after that of ``c_alias_2`` for a -single row where they both refer to the same identity, the ``C.d`` -element will still be loaded. Previously, the loader did not seek to -modify the load of an entity that was already loaded via a different path. -The loader that reaches the entity first has always been non-deterministic, -so this fix may be detectable as a behavioral change in some situations and -not others. - -The fix includes tests for two variants of the "multiple paths to one entity" -case, and the fix should hopefully cover all other scenarios of this nature. - -:ticket:`3431` - - -New MutableList and MutableSet helpers added to the mutation tracking extension -------------------------------------------------------------------------------- - -New helper classes :class:`.MutableList` and :class:`.MutableSet` have been -added to the :ref:`mutable_toplevel` extension, to complement the existing -:class:`.MutableDict` helper. - -:ticket:`3297` - -.. _change_3512: - -New "raise" / "raise_on_sql" loader strategies ----------------------------------------------- - -To assist with the use case of preventing unwanted lazy loads from occurring -after a series of objects are loaded, the new "lazy='raise'" and -"lazy='raise_on_sql'" strategies and -corresponding loader option :func:`_orm.raiseload` may be applied to a -relationship attribute which will cause it to raise ``InvalidRequestError`` -when a non-eagerly-loaded attribute is accessed for read. The two variants -test for either a lazy load of any variety, including those that would -only return None or retrieve from the identity map:: - - >>> from sqlalchemy.orm import raiseload - >>> a1 = s.query(A).options(raiseload(A.some_b)).first() - >>> a1.some_b - Traceback (most recent call last): - ... - sqlalchemy.exc.InvalidRequestError: 'A.some_b' is not available due to lazy='raise' - -Or a lazy load only where SQL would be emitted:: - - >>> from sqlalchemy.orm import raiseload - >>> a1 = s.query(A).options(raiseload(A.some_b, sql_only=True)).first() - >>> a1.some_b - Traceback (most recent call last): - ... - sqlalchemy.exc.InvalidRequestError: 'A.bs' is not available due to lazy='raise_on_sql' - -:ticket:`3512` - -.. _change_3394: - -Mapper.order_by is deprecated ------------------------------ - -This old parameter from the very first versions of SQLAlchemy was part of -the original design of the ORM which featured the :class:`_orm.Mapper` object -as a public-facing query structure. This role has long since been replaced -by the :class:`_query.Query` object, where we use :meth:`_query.Query.order_by` to -indicate the ordering of results in a way that works consistently for any -combination of SELECT statements, entities and SQL expressions. There are -many areas in which :paramref:`_orm.Mapper.order_by` doesn't work as expected -(or what would be expected is not clear), such as when queries are combined -into unions; these cases are not supported. - - -:ticket:`3394` - -New Features and Improvements - Core -==================================== - -.. _change_3803: - -Engines now invalidate connections, run error handlers for BaseException ------------------------------------------------------------------------- - -.. versionadded:: 1.1 this change is a late add to the 1.1 series just - prior to 1.1 final, and is not present in the 1.1 beta releases. - -The Python ``BaseException`` class is below that of ``Exception`` but is the -identifiable base for system-level exceptions such as ``KeyboardInterrupt``, -``SystemExit``, and notably the ``GreenletExit`` exception that's used by -eventlet and gevent. This exception class is now intercepted by the exception- -handling routines of :class:`_engine.Connection`, and includes handling by the -:meth:`_events.ConnectionEvents.handle_error` event. The :class:`_engine.Connection` is now -**invalidated** by default in the case of a system level exception that is not -a subclass of ``Exception``, as it is assumed an operation was interrupted and -the connection may be in an unusable state. The MySQL drivers are most -targeted by this change however the change is across all DBAPIs. - -Note that upon invalidation, the immediate DBAPI connection used by -:class:`_engine.Connection` is disposed, and the :class:`_engine.Connection`, if still -being used subsequent to the exception raise, will use a new -DBAPI connection for subsequent operations upon next use; however, the state of -any transaction in progress is lost and the appropriate ``.rollback()`` method -must be called if applicable before this re-use can proceed. - -In order to identify this change, it was straightforward to demonstrate a pymysql or -mysqlclient / MySQL-Python connection moving into a corrupted state when -these exceptions occur in the middle of the connection doing its work; -the connection would then be returned to the connection pool where subsequent -uses would fail, or even before returning to the pool would cause secondary -failures in context managers that call ``.rollback()`` upon the exception -catch. The behavior here is expected to reduce -the incidence of the MySQL error "commands out of sync", as well as the -``ResourceClosedError`` which can occur when the MySQL driver fails to -report ``cursor.description`` correctly, when running under greenlet -conditions where greenlets are killed, or where ``KeyboardInterrupt`` exceptions -are handled without exiting the program entirely. - -The behavior is distinct from the usual auto-invalidation feature, in that it -does not assume that the backend database itself has been shut down or -restarted; it does not recycle the entire connection pool as is the case -for usual DBAPI disconnect exceptions. - -This change should be a net improvement for all users with the exception -of **any application that currently intercepts ``KeyboardInterrupt`` or -``GreenletExit`` and wishes to continue working within the same transaction**. -Such an operation is theoretically possible with other DBAPIs that do not appear to be -impacted by ``KeyboardInterrupt`` such as psycopg2. For these DBAPIs, -the following workaround will disable the connection from being recycled -for specific exceptions:: - - - engine = create_engine("postgresql+psycopg2://") - - - @event.listens_for(engine, "handle_error") - def cancel_disconnect(ctx): - if isinstance(ctx.original_exception, KeyboardInterrupt): - ctx.is_disconnect = False - -:ticket:`3803` - - -.. _change_2551: - -CTE Support for INSERT, UPDATE, DELETE --------------------------------------- - -One of the most widely requested features is support for common table -expressions (CTE) that work with INSERT, UPDATE, DELETE, and is now implemented. -An INSERT/UPDATE/DELETE can both draw from a WITH clause that's stated at the -top of the SQL, as well as can be used as a CTE itself in the context of -a larger statement. - -As part of this change, an INSERT from SELECT that includes a CTE will now -render the CTE at the top of the entire statement, rather than nested -in the SELECT statement as was the case in 1.0. - -Below is an example that renders UPDATE, INSERT and SELECT all in one -statement: - -.. sourcecode:: pycon+sql - - >>> from sqlalchemy import table, column, select, literal, exists - >>> orders = table( - ... "orders", - ... column("region"), - ... column("amount"), - ... column("product"), - ... column("quantity"), - ... ) - >>> - >>> upsert = ( - ... orders.update() - ... .where(orders.c.region == "Region1") - ... .values(amount=1.0, product="Product1", quantity=1) - ... .returning(*(orders.c._all_columns)) - ... .cte("upsert") - ... ) - >>> - >>> insert = orders.insert().from_select( - ... orders.c.keys(), - ... select([literal("Region1"), literal(1.0), literal("Product1"), literal(1)]).where( - ... ~exists(upsert.select()) - ... ), - ... ) - >>> - >>> print(insert) # Note: formatting added for clarity - {printsql}WITH upsert AS - (UPDATE orders SET amount=:amount, product=:product, quantity=:quantity - WHERE orders.region = :region_1 - RETURNING orders.region, orders.amount, orders.product, orders.quantity - ) - INSERT INTO orders (region, amount, product, quantity) - SELECT - :param_1 AS anon_1, :param_2 AS anon_2, - :param_3 AS anon_3, :param_4 AS anon_4 - WHERE NOT ( - EXISTS ( - SELECT upsert.region, upsert.amount, - upsert.product, upsert.quantity - FROM upsert)) - -:ticket:`2551` - -.. _change_3049: - -Support for RANGE and ROWS specification within window functions ----------------------------------------------------------------- - -New :paramref:`.expression.over.range_` and :paramref:`.expression.over.rows` parameters allow -RANGE and ROWS expressions for window functions: - -.. sourcecode:: pycon+sql - - >>> from sqlalchemy import func - - >>> print(func.row_number().over(order_by="x", range_=(-5, 10))) - {printsql}row_number() OVER (ORDER BY x RANGE BETWEEN :param_1 PRECEDING AND :param_2 FOLLOWING){stop} - - >>> print(func.row_number().over(order_by="x", rows=(None, 0))) - {printsql}row_number() OVER (ORDER BY x ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW){stop} - - >>> print(func.row_number().over(order_by="x", range_=(-2, None))) - {printsql}row_number() OVER (ORDER BY x RANGE BETWEEN :param_1 PRECEDING AND UNBOUNDED FOLLOWING){stop} - -:paramref:`.expression.over.range_` and :paramref:`.expression.over.rows` are specified as -2-tuples and indicate negative and positive values for specific ranges, -0 for "CURRENT ROW", and None for UNBOUNDED. - -.. seealso:: - - :ref:`tutorial_window_functions` - -:ticket:`3049` - -.. _change_2857: - -Support for the SQL LATERAL keyword ------------------------------------ - -The LATERAL keyword is currently known to only be supported by PostgreSQL 9.3 -and greater, however as it is part of the SQL standard support for this keyword -is added to Core. The implementation of :meth:`_expression.Select.lateral` employs -special logic beyond just rendering the LATERAL keyword to allow for -correlation of tables that are derived from the same FROM clause as the -selectable, e.g. lateral correlation: - -.. sourcecode:: pycon+sql - - >>> from sqlalchemy import table, column, select, true - >>> people = table("people", column("people_id"), column("age"), column("name")) - >>> books = table("books", column("book_id"), column("owner_id")) - >>> subq = ( - ... select([books.c.book_id]) - ... .where(books.c.owner_id == people.c.people_id) - ... .lateral("book_subq") - ... ) - >>> print(select([people]).select_from(people.join(subq, true()))) - {printsql}SELECT people.people_id, people.age, people.name - FROM people JOIN LATERAL (SELECT books.book_id AS book_id - FROM books WHERE books.owner_id = people.people_id) - AS book_subq ON true - -.. seealso:: - - :ref:`tutorial_lateral_correlation` - - :class:`_expression.Lateral` - - :meth:`_expression.Select.lateral` - - -:ticket:`2857` - -.. _change_3718: - -Support for TABLESAMPLE ------------------------ - -The SQL standard TABLESAMPLE can be rendered using the -:meth:`_expression.FromClause.tablesample` method, which returns a :class:`_expression.TableSample` -construct similar to an alias:: - - from sqlalchemy import func - - selectable = people.tablesample(func.bernoulli(1), name="alias", seed=func.random()) - stmt = select([selectable.c.people_id]) - -Assuming ``people`` with a column ``people_id``, the above -statement would render as: - -.. sourcecode:: sql - - SELECT alias.people_id FROM - people AS alias TABLESAMPLE bernoulli(:bernoulli_1) - REPEATABLE (random()) - -:ticket:`3718` - -.. _change_3216: - -The ``.autoincrement`` directive is no longer implicitly enabled for a composite primary key column ---------------------------------------------------------------------------------------------------- - -SQLAlchemy has always had the convenience feature of enabling the backend database's -"autoincrement" feature for a single-column integer primary key; by "autoincrement" -we mean that the database column will include whatever DDL directives the -database provides in order to indicate an auto-incrementing integer identifier, -such as the SERIAL keyword on PostgreSQL or AUTO_INCREMENT on MySQL, and additionally -that the dialect will receive these generated values from the execution -of a :meth:`_schema.Table.insert` construct using techniques appropriate to that -backend. - -What's changed is that this feature no longer turns on automatically for a -*composite* primary key; previously, a table definition such as:: - - Table( - "some_table", - metadata, - Column("x", Integer, primary_key=True), - Column("y", Integer, primary_key=True), - ) - -Would have "autoincrement" semantics applied to the ``'x'`` column, only -because it's first in the list of primary key columns. In order to -disable this, one would have to turn off ``autoincrement`` on all columns:: - - # old way - Table( - "some_table", - metadata, - Column("x", Integer, primary_key=True, autoincrement=False), - Column("y", Integer, primary_key=True, autoincrement=False), - ) - -With the new behavior, the composite primary key will not have autoincrement -semantics unless a column is marked explicitly with ``autoincrement=True``:: - - # column 'y' will be SERIAL/AUTO_INCREMENT/ auto-generating - Table( - "some_table", - metadata, - Column("x", Integer, primary_key=True), - Column("y", Integer, primary_key=True, autoincrement=True), - ) - -In order to anticipate some potential backwards-incompatible scenarios, -the :meth:`_schema.Table.insert` construct will perform more thorough checks -for missing primary key values on composite primary key columns that don't -have autoincrement set up; given a table such as:: - - Table( - "b", - metadata, - Column("x", Integer, primary_key=True), - Column("y", Integer, primary_key=True), - ) - -An INSERT emitted with no values for this table will produce this warning: - -.. sourcecode:: text - - SAWarning: Column 'b.x' is marked as a member of the primary - key for table 'b', but has no Python-side or server-side default - generator indicated, nor does it indicate 'autoincrement=True', - and no explicit value is passed. Primary key columns may not - store NULL. Note that as of SQLAlchemy 1.1, 'autoincrement=True' - must be indicated explicitly for composite (e.g. multicolumn) - primary keys if AUTO_INCREMENT/SERIAL/IDENTITY behavior is - expected for one of the columns in the primary key. CREATE TABLE - statements are impacted by this change as well on most backends. - -For a column that is receiving primary key values from a server-side -default or something less common such as a trigger, the presence of a -value generator can be indicated using :class:`.FetchedValue`:: - - Table( - "b", - metadata, - Column("x", Integer, primary_key=True, server_default=FetchedValue()), - Column("y", Integer, primary_key=True, server_default=FetchedValue()), - ) - -For the very unlikely case where a composite primary key is actually intended -to store NULL in one or more of its columns (only supported on SQLite and MySQL), -specify the column with ``nullable=True``:: - - Table( - "b", - metadata, - Column("x", Integer, primary_key=True), - Column("y", Integer, primary_key=True, nullable=True), - ) - -In a related change, the ``autoincrement`` flag may be set to True -on a column that has a client-side or server-side default. This typically -will not have much impact on the behavior of the column during an INSERT. - - -.. seealso:: - - :ref:`change_mysql_3216` - -:ticket:`3216` - -.. _change_is_distinct_from: - -Support for IS DISTINCT FROM and IS NOT DISTINCT FROM ------------------------------------------------------ - -New operators :meth:`.ColumnOperators.is_distinct_from` and -:meth:`.ColumnOperators.isnot_distinct_from` allow the IS DISTINCT -FROM and IS NOT DISTINCT FROM sql operation: - -.. sourcecode:: pycon+sql - - >>> print(column("x").is_distinct_from(None)) - {printsql}x IS DISTINCT FROM NULL{stop} - -Handling is provided for NULL, True and False: - -.. sourcecode:: pycon+sql - - >>> print(column("x").isnot_distinct_from(False)) - {printsql}x IS NOT DISTINCT FROM false{stop} - -For SQLite, which doesn't have this operator, "IS" / "IS NOT" is rendered, -which on SQLite works for NULL unlike other backends: - -.. sourcecode:: pycon+sql - - >>> from sqlalchemy.dialects import sqlite - >>> print(column("x").is_distinct_from(None).compile(dialect=sqlite.dialect())) - {printsql}x IS NOT NULL{stop} - -.. _change_1957: - -Core and ORM support for FULL OUTER JOIN ----------------------------------------- - -The new flag :paramref:`.FromClause.outerjoin.full`, available at the Core -and ORM level, instructs the compiler to render ``FULL OUTER JOIN`` -where it would normally render ``LEFT OUTER JOIN``:: - - stmt = select([t1]).select_from(t1.outerjoin(t2, full=True)) - -The flag also works at the ORM level:: - - q = session.query(MyClass).outerjoin(MyOtherClass, full=True) - -:ticket:`1957` - -.. _change_3501: - -ResultSet column matching enhancements; positional column setup for textual SQL -------------------------------------------------------------------------------- - -A series of improvements were made to the :class:`_engine.ResultProxy` system -in the 1.0 series as part of :ticket:`918`, which reorganizes the internals -to match cursor-bound result columns with table/ORM metadata positionally, -rather than by matching names, for compiled SQL constructs that contain full -information about the result rows to be returned. This allows a dramatic savings -on Python overhead as well as much greater accuracy in linking ORM and Core -SQL expressions to result rows. In 1.1, this reorganization has been taken -further internally, and also has been made available to pure-text SQL -constructs via the use of the recently added :meth:`_expression.TextClause.columns` method. - -TextAsFrom.columns() now works positionally -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The :meth:`_expression.TextClause.columns` method, added in 0.9, accepts column-based arguments -positionally; in 1.1, when all columns are passed positionally, the correlation -of these columns to the ultimate result set is also performed positionally. -The key advantage here is that textual SQL can now be linked to an ORM- -level result set without the need to deal with ambiguous or duplicate column -names, or with having to match labeling schemes to ORM-level labeling schemes. All -that's needed now is the same ordering of columns within the textual SQL -and the column arguments passed to :meth:`_expression.TextClause.columns`:: - - - from sqlalchemy import text - - stmt = text( - "SELECT users.id, addresses.id, users.id, " - "users.name, addresses.email_address AS email " - "FROM users JOIN addresses ON users.id=addresses.user_id " - "WHERE users.id = 1" - ).columns(User.id, Address.id, Address.user_id, User.name, Address.email_address) - - query = session.query(User).from_statement(stmt).options(contains_eager(User.addresses)) - result = query.all() - -Above, the textual SQL contains the column "id" three times, which would -normally be ambiguous. Using the new feature, we can apply the mapped -columns from the ``User`` and ``Address`` class directly, even linking -the ``Address.user_id`` column to the ``users.id`` column in textual SQL -for fun, and the :class:`_query.Query` object will receive rows that are correctly -targetable as needed, including for an eager load. - -This change is **backwards incompatible** with code that passes the columns -to the method with a different ordering than is present in the textual statement. -It is hoped that this impact will be low due to the fact that this -method has always been documented illustrating the columns being passed in the same order as that of the -textual SQL statement, as would seem intuitive, even though the internals -weren't checking for this. The method itself was only added as of 0.9 in -any case and may not yet have widespread use. Notes on exactly how to handle -this behavioral change for applications using it are at :ref:`behavior_change_3501`. - -.. seealso:: - - :ref:`tutorial_select_arbitrary_text` - - :ref:`behavior_change_3501` - backwards compatibility remarks - -Positional matching is trusted over name-based matching for Core/ORM SQL constructs -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Another aspect of this change is that the rules for matching columns have also been modified -to rely upon "positional" matching more fully for compiled SQL constructs -as well. Given a statement like the following:: - - ua = users.alias("ua") - stmt = select([users.c.user_id, ua.c.user_id]) - -The above statement will compile to: - -.. sourcecode:: sql - - SELECT users.user_id, ua.user_id FROM users, users AS ua - -In 1.0, the above statement when executed would be matched to its original -compiled construct using positional matching, however because the statement -contains the ``'user_id'`` label duplicated, the "ambiguous column" rule -would still get involved and prevent the columns from being fetched from a row. -As of 1.1, the "ambiguous column" rule does not affect an exact match from -a column construct to the SQL column, which is what the ORM uses to -fetch columns:: - - result = conn.execute(stmt) - row = result.first() - - # these both match positionally, so no error - user_id = row[users.c.user_id] - ua_id = row[ua.c.user_id] - - # this still raises, however - user_id = row["user_id"] - -Much less likely to get an "ambiguous column" error message -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -As part of this change, the wording of the error message ``Ambiguous column -name '' in result set! try 'use_labels' option on select statement.`` -has been dialed back; as this message should now be extremely rare when using -the ORM or Core compiled SQL constructs, it merely states -``Ambiguous column name '' in result set column descriptions``, and -only when a result column is retrieved using the string name that is actually -ambiguous, e.g. ``row['user_id']`` in the above example. It also now refers -to the actual ambiguous name from the rendered SQL statement itself, -rather than indicating the key or name that was local to the construct being -used for the fetch. - -:ticket:`3501` - -.. _change_3292: - -Support for Python's native ``enum`` type and compatible forms --------------------------------------------------------------- - -The :class:`.Enum` type can now be constructed using any -PEP-435 compliant enumerated type. When using this mode, input values -and return values are the actual enumerated objects, not the -string/integer/etc values:: - - import enum - from sqlalchemy import Table, MetaData, Column, Enum, create_engine - - - class MyEnum(enum.Enum): - one = 1 - two = 2 - three = 3 - - - t = Table("data", MetaData(), Column("value", Enum(MyEnum))) - - e = create_engine("sqlite://") - t.create(e) - - e.execute(t.insert(), {"value": MyEnum.two}) - assert e.scalar(t.select()) is MyEnum.two - -The ``Enum.enums`` collection is now a list instead of a tuple -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -As part of the changes to :class:`.Enum`, the :attr:`.Enum.enums` collection -of elements is now a list instead of a tuple. This because lists -are appropriate for variable length sequences of homogeneous items where -the position of the element is not semantically significant. - -:ticket:`3292` - -.. _change_gh_231: - -Negative integer indexes accommodated by Core result rows ---------------------------------------------------------- - -The :class:`.RowProxy` object now accommodates single negative integer indexes -like a regular Python sequence, both in the pure Python and C-extension -version. Previously, negative values would only work in slices:: - - >>> from sqlalchemy import create_engine - >>> e = create_engine("sqlite://") - >>> row = e.execute("select 1, 2, 3").first() - >>> row[-1], row[-2], row[1], row[-2:2] - 3 2 2 (2,) - -.. _change_3095: - -The ``Enum`` type now does in-Python validation of values ---------------------------------------------------------- - -To accommodate for Python native enumerated objects, as well as for edge -cases such as that of where a non-native ENUM type is used within an ARRAY -and a CHECK constraint is infeasible, the :class:`.Enum` datatype now adds -in-Python validation of input values when the :paramref:`.Enum.validate_strings` -flag is used (1.1.0b2):: - - - >>> from sqlalchemy import Table, MetaData, Column, Enum, create_engine - >>> t = Table( - ... "data", - ... MetaData(), - ... Column("value", Enum("one", "two", "three", validate_strings=True)), - ... ) - >>> e = create_engine("sqlite://") - >>> t.create(e) - >>> e.execute(t.insert(), {"value": "four"}) - Traceback (most recent call last): - ... - sqlalchemy.exc.StatementError: (exceptions.LookupError) - "four" is not among the defined enum values - [SQL: u'INSERT INTO data (value) VALUES (?)'] - [parameters: [{'value': 'four'}]] - -This validation is turned off by default as there are already use cases -identified where users don't want such validation (such as string comparisons). -For non-string types, it necessarily takes place in all cases. The -check also occurs unconditionally on the result-handling side as well, when -values coming from the database are returned. - -This validation is in addition to the existing behavior of creating a -CHECK constraint when a non-native enumerated type is used. The creation of -this CHECK constraint can now be disabled using the new -:paramref:`.Enum.create_constraint` flag. - -:ticket:`3095` - -.. _change_3730: - -Non-native boolean integer values coerced to zero/one/None in all cases ------------------------------------------------------------------------ - -The :class:`.Boolean` datatype coerces Python booleans to integer values -for backends that don't have a native boolean type, such as SQLite and -MySQL. On these backends, a CHECK constraint is normally set up which -ensures the values in the database are in fact one of these two values. -However, MySQL ignores CHECK constraints, the constraint is optional, and -an existing database might not have this constraint. The :class:`.Boolean` -datatype has been repaired such that an incoming Python-side value that is -already an integer value is coerced to zero or one, not just passed as-is; -additionally, the C-extension version of the int-to-boolean processor for -results now uses the same Python boolean interpretation of the value, -rather than asserting an exact one or zero value. This is now consistent -with the pure-Python int-to-boolean processor and is more forgiving of -existing data already within the database. Values of None/NULL are as before -retained as None/NULL. - -.. note:: - - this change had an unintended side effect that the interpretation of non- - integer values, such as strings, also changed in behavior such that the - string value ``"0"`` would be interpreted as "true", but only on backends - that don't have a native boolean datatype - on "native boolean" backends - like PostgreSQL, the string value ``"0"`` is passed directly to the driver - and is interpreted as "false". This is an inconsistency that did not occur - with the previous implementation. It should be noted that passing strings or - any other value outside of ``None``, ``True``, ``False``, ``1``, ``0`` to - the :class:`.Boolean` datatype is **not supported** and version 1.2 will - raise an error for this scenario (or possibly just emit a warning, TBD). - See also :ticket:`4102`. - - -:ticket:`3730` - -.. _change_2837: - -Large parameter and row values are now truncated in logging and exception displays ----------------------------------------------------------------------------------- - -A large value present as a bound parameter for a SQL statement, as well as a -large value present in a result row, will now be truncated during display -within logging, exception reporting, as well as ``repr()`` of the row itself:: - - >>> from sqlalchemy import create_engine - >>> import random - >>> e = create_engine("sqlite://", echo="debug") - >>> some_value = "".join(chr(random.randint(52, 85)) for i in range(5000)) - >>> row = e.execute("select ?", [some_value]).first() - ... # (lines are wrapped for clarity) ... - 2016-02-17 13:23:03,027 INFO sqlalchemy.engine.base.Engine select ? - 2016-02-17 13:23:03,027 INFO sqlalchemy.engine.base.Engine - ('E6@?>9HPOJB<:=TSTLA;9K;9FPM4M8M@;NM6GU - LUAEBT9QGHNHTHR5EP75@OER4?SKC;D:TFUMD:M>;C6U:JLM6R67GEK4=4:P - GJ7HQ6 ... (4702 characters truncated) ... J6IK546AJMB4N6S9L;;9AKI;=RJP - HDSSOTNBUEEC9@Q:RCL:I@5?FO<9K>KJAGAO@E6@A7JI8O:J7B69T6<8;F:S;4BEIJS9HM - K:;5OLPM@JR;R:J6Q>7T@I::OTDC:CC<=NGP6C>BC8N',) - 2016-02-17 13:23:03,027 DEBUG sqlalchemy.engine.base.Engine Col ('?',) - 2016-02-17 13:23:03,027 DEBUG sqlalchemy.engine.base.Engine - Row (u'E6@?>9HPOJB<:=TSTLA;9K;9FPM4M8M@; - NM6GULUAEBT9QGHNHTHR5EP75@OER4?SKC;D:TFUMD:M>;C6U:JLM6R67GEK4=4:PGJ7HQ ... (4703 characters truncated) ... J6IK546AJMB4N6S9L;;9AKI;= - RJPHDSSOTNBUEEC9@Q:RCL:I@5?FO<9K>KJAGAO@E6@A7JI8O:J7B69T6<8;F:S;4BEIJS9HM - K:;5OLPM@JR;R:J6Q>7T@I::OTDC:CC<=NGP6C>BC8N',) - >>> print(row) - (u'E6@?>9HPOJB<:=TSTLA;9K;9FPM4M8M@;NM6 - GULUAEBT9QGHNHTHR5EP75@OER4?SKC;D:TFUMD:M>;C6U:JLM6R67GEK4 - =4:PGJ7HQ ... (4703 characters truncated) ... J6IK546AJMB4N6S9L;;9AKI; - =RJPHDSSOTNBUEEC9@Q:RCL:I@5?FO<9K>KJAGAO@E6@A7JI8O:J7B69T6<8;F:S;4BEIJS9H - MK:;5OLPM@JR;R:J6Q>7T@I::OTDC:CC<=NGP6C>BC8N',) - - -:ticket:`2837` - - -.. _change_3619: - -JSON support added to Core --------------------------- - -As MySQL now has a JSON datatype in addition to the PostgreSQL JSON datatype, -the core now gains a :class:`sqlalchemy.types.JSON` datatype that is the basis -for both of these. Using this type allows access to the "getitem" operator -as well as the "getpath" operator in a way that is agnostic across PostgreSQL -and MySQL. - -The new datatype also has a series of improvements to the handling of -NULL values as well as expression handling. - -.. seealso:: - - :ref:`change_3547` - - :class:`_types.JSON` - - :class:`_postgresql.JSON` - - :class:`.mysql.JSON` - -:ticket:`3619` - -.. _change_3514: - -JSON "null" is inserted as expected with ORM operations, omitted when not present -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The :class:`_types.JSON` type and its descendant types :class:`_postgresql.JSON` -and :class:`.mysql.JSON` have a flag :paramref:`.types.JSON.none_as_null` which -when set to True indicates that the Python value ``None`` should translate -into a SQL NULL rather than a JSON NULL value. This flag defaults to False, -which means that the Python value ``None`` should result in a JSON NULL value. - -This logic would fail, and is now corrected, in the following circumstances: - -1. When the column also contained a default or server_default value, -a positive value of ``None`` on the mapped attribute that expects to persist -JSON "null" would still result in the column-level default being triggered, -replacing the ``None`` value:: - - class MyObject(Base): - # ... - - json_value = Column(JSON(none_as_null=False), default="some default") - - - # would insert "some default" instead of "'null'", - # now will insert "'null'" - obj = MyObject(json_value=None) - session.add(obj) - session.commit() - -2. When the column *did not* contain a default or server_default value, a missing -value on a JSON column configured with none_as_null=False would still render -JSON NULL rather than falling back to not inserting any value, behaving -inconsistently vs. all other datatypes:: - - class MyObject(Base): - # ... - - some_other_value = Column(String(50)) - json_value = Column(JSON(none_as_null=False)) - - - # would result in NULL for some_other_value, - # but json "'null'" for json_value. Now results in NULL for both - # (the json_value is omitted from the INSERT) - obj = MyObject() - session.add(obj) - session.commit() - -This is a behavioral change that is backwards incompatible for an application -that was relying upon this to default a missing value as JSON null. This -essentially establishes that a **missing value is distinguished from a present -value of None**. See :ref:`behavior_change_3514` for further detail. - -3. When the :meth:`.Session.bulk_insert_mappings` method were used, ``None`` -would be ignored in all cases:: - - # would insert SQL NULL and/or trigger defaults, - # now inserts "'null'" - session.bulk_insert_mappings(MyObject, [{"json_value": None}]) - -The :class:`_types.JSON` type now implements the -:attr:`.TypeEngine.should_evaluate_none` flag, -indicating that ``None`` should not be ignored here; it is configured -automatically based on the value of :paramref:`.types.JSON.none_as_null`. -Thanks to :ticket:`3061`, we can differentiate when the value ``None`` is actively -set by the user versus when it was never set at all. - -The feature applies as well to the new base :class:`_types.JSON` type -and its descendant types. - -:ticket:`3514` - -.. _change_3514_jsonnull: - -New JSON.NULL Constant Added -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -To ensure that an application can always have full control at the value level -of whether a :class:`_types.JSON`, :class:`_postgresql.JSON`, :class:`.mysql.JSON`, -or :class:`_postgresql.JSONB` column -should receive a SQL NULL or JSON ``"null"`` value, the constant -:attr:`.types.JSON.NULL` has been added, which in conjunction with -:func:`.null` can be used to determine fully between SQL NULL and -JSON ``"null"``, regardless of what :paramref:`.types.JSON.none_as_null` is set -to:: - - from sqlalchemy import null - from sqlalchemy.dialects.postgresql import JSON - - obj1 = MyObject(json_value=null()) # will *always* insert SQL NULL - obj2 = MyObject(json_value=JSON.NULL) # will *always* insert JSON string "null" - - session.add_all([obj1, obj2]) - session.commit() - -The feature applies as well to the new base :class:`_types.JSON` type -and its descendant types. - -:ticket:`3514` - -.. _change_3516: - -Array support added to Core; new ANY and ALL operators ------------------------------------------------------- - -Along with the enhancements made to the PostgreSQL :class:`_postgresql.ARRAY` -type described in :ref:`change_3503`, the base class of :class:`_postgresql.ARRAY` -itself has been moved to Core in a new class :class:`_types.ARRAY`. - -Arrays are part of the SQL standard, as are several array-oriented functions -such as ``array_agg()`` and ``unnest()``. In support of these constructs -for not just PostgreSQL but also potentially for other array-capable backends -in the future such as DB2, the majority of array logic for SQL expressions -is now in Core. The :class:`_types.ARRAY` type still **only works on -PostgreSQL**, however it can be used directly, supporting special array -use cases such as indexed access, as well as support for the ANY and ALL:: - - mytable = Table("mytable", metadata, Column("data", ARRAY(Integer, dimensions=2))) - - expr = mytable.c.data[5][6] - - expr = mytable.c.data[5].any(12) - -In support of ANY and ALL, the :class:`_types.ARRAY` type retains the same -:meth:`.types.ARRAY.Comparator.any` and :meth:`.types.ARRAY.Comparator.all` methods -from the PostgreSQL type, but also exports these operations to new -standalone operator functions :func:`_expression.any_` and -:func:`_expression.all_`. These two functions work in more -of the traditional SQL way, allowing a right-side expression form such -as:: - - from sqlalchemy import any_, all_ - - select([mytable]).where(12 == any_(mytable.c.data[5])) - -For the PostgreSQL-specific operators "contains", "contained_by", and -"overlaps", one should continue to use the :class:`_postgresql.ARRAY` -type directly, which provides all functionality of the :class:`_types.ARRAY` -type as well. - -The :func:`_expression.any_` and :func:`_expression.all_` operators -are open-ended at the Core level, however their interpretation by backend -databases is limited. On the PostgreSQL backend, the two operators -**only accept array values**. Whereas on the MySQL backend, they -**only accept subquery values**. On MySQL, one can use an expression -such as:: - - from sqlalchemy import any_, all_ - - subq = select([mytable.c.value]) - select([mytable]).where(12 > any_(subq)) - -:ticket:`3516` - -.. _change_3132: - -New Function features, "WITHIN GROUP", array_agg and set aggregate functions ----------------------------------------------------------------------------- - -With the new :class:`_types.ARRAY` type we can also implement a pre-typed -function for the ``array_agg()`` SQL function that returns an array, -which is now available using :class:`_functions.array_agg`:: - - from sqlalchemy import func - - stmt = select([func.array_agg(table.c.value)]) - -A PostgreSQL element for an aggregate ORDER BY is also added via -:class:`_postgresql.aggregate_order_by`:: - - from sqlalchemy.dialects.postgresql import aggregate_order_by - - expr = func.array_agg(aggregate_order_by(table.c.a, table.c.b.desc())) - stmt = select([expr]) - -Producing: - -.. sourcecode:: sql - - SELECT array_agg(table1.a ORDER BY table1.b DESC) AS array_agg_1 FROM table1 - -The PG dialect itself also provides an :func:`_postgresql.array_agg` wrapper to -ensure the :class:`_postgresql.ARRAY` type:: - - from sqlalchemy.dialects.postgresql import array_agg - - stmt = select([array_agg(table.c.value).contains("foo")]) - -Additionally, functions like ``percentile_cont()``, ``percentile_disc()``, -``rank()``, ``dense_rank()`` and others that require an ordering via -``WITHIN GROUP (ORDER BY )`` are now available via the -:meth:`.FunctionElement.within_group` modifier:: - - from sqlalchemy import func - - stmt = select( - [ - department.c.id, - func.percentile_cont(0.5).within_group(department.c.salary.desc()), - ] - ) - -The above statement would produce SQL similar to: - -.. sourcecode:: sql - - SELECT department.id, percentile_cont(0.5) - WITHIN GROUP (ORDER BY department.salary DESC) - -Placeholders with correct return types are now provided for these functions, -and include :class:`.percentile_cont`, :class:`.percentile_disc`, -:class:`.rank`, :class:`.dense_rank`, :class:`.mode`, :class:`.percent_rank`, -and :class:`.cume_dist`. - -:ticket:`3132` :ticket:`1370` - -.. _change_2919: - -TypeDecorator now works with Enum, Boolean, "schema" types automatically ------------------------------------------------------------------------- - -The :class:`.SchemaType` types include types such as :class:`.Enum` -and :class:`.Boolean` which, in addition to corresponding to a database -type, also generate either a CHECK constraint or in the case of PostgreSQL -ENUM a new CREATE TYPE statement, will now work automatically with -:class:`.TypeDecorator` recipes. Previously, a :class:`.TypeDecorator` for -an :class:`_postgresql.ENUM` had to look like this:: - - # old way - class MyEnum(TypeDecorator, SchemaType): - impl = postgresql.ENUM("one", "two", "three", name="myenum") - - def _set_table(self, table): - self.impl._set_table(table) - -The :class:`.TypeDecorator` now propagates those additional events so it -can be done like any other type:: - - # new way - class MyEnum(TypeDecorator): - impl = postgresql.ENUM("one", "two", "three", name="myenum") - -:ticket:`2919` - -.. _change_2685: - -Multi-Tenancy Schema Translation for Table objects --------------------------------------------------- - -To support the use case of an application that uses the same set of -:class:`_schema.Table` objects in many schemas, such as schema-per-user, a new -execution option :paramref:`.Connection.execution_options.schema_translate_map` -is added. Using this mapping, a set of :class:`_schema.Table` -objects can be made on a per-connection basis to refer to any set of schemas -instead of the :paramref:`_schema.Table.schema` to which they were assigned. The -translation works for DDL and SQL generation, as well as with the ORM. - -For example, if the ``User`` class were assigned the schema "per_user":: - - class User(Base): - __tablename__ = "user" - id = Column(Integer, primary_key=True) - - __table_args__ = {"schema": "per_user"} - -On each request, the :class:`.Session` can be set up to refer to a -different schema each time:: - - session = Session() - session.connection( - execution_options={"schema_translate_map": {"per_user": "account_one"}} - ) - - # will query from the ``account_one.user`` table - session.query(User).get(5) - -.. seealso:: - - :ref:`schema_translating` - -:ticket:`2685` - -.. _change_3631: - -"Friendly" stringification of Core SQL constructs without a dialect -------------------------------------------------------------------- - -Calling ``str()`` on a Core SQL construct will now produce a string -in more cases than before, supporting various SQL constructs not normally -present in default SQL such as RETURNING, array indexes, and non-standard -datatypes: - -.. sourcecode:: pycon+sql - - >>> from sqlalchemy import table, column - t>>> t = table('x', column('a'), column('b')) - >>> print(t.insert().returning(t.c.a, t.c.b)) - {printsql}INSERT INTO x (a, b) VALUES (:a, :b) RETURNING x.a, x.b - -The ``str()`` function now calls upon an entirely separate dialect / compiler -intended just for plain string printing without a specific dialect set up, -so as more "just show me a string!" cases come up, these can be added -to this dialect/compiler without impacting behaviors on real dialects. - -.. seealso:: - - :ref:`change_3081` - -:ticket:`3631` - -.. _change_3531: - -The type_coerce function is now a persistent SQL element --------------------------------------------------------- - -The :func:`_expression.type_coerce` function previously would return -an object either of type :class:`.BindParameter` or :class:`.Label`, depending -on the input. An effect this would have was that in the case where expression -transformations were used, such as the conversion of an element from a -:class:`_schema.Column` to a :class:`.BindParameter` that's critical to ORM-level -lazy loading, the type coercion information would not be used since it would -have been lost already. - -To improve this behavior, the function now returns a persistent -:class:`.TypeCoerce` container around the given expression, which itself -remains unaffected; this construct is evaluated explicitly by the -SQL compiler. This allows for the coercion of the inner expression -to be maintained no matter how the statement is modified, including if -the contained element is replaced with a different one, as is common -within the ORM's lazy loading feature. - -The test case illustrating the effect makes use of a heterogeneous -primaryjoin condition in conjunction with custom types and lazy loading. -Given a custom type that applies a CAST as a "bind expression":: - - class StringAsInt(TypeDecorator): - impl = String - - def column_expression(self, col): - return cast(col, Integer) - - def bind_expression(self, value): - return cast(value, String) - -Then, a mapping where we are equating a string "id" column on one -table to an integer "id" column on the other:: - - class Person(Base): - __tablename__ = "person" - id = Column(StringAsInt, primary_key=True) - - pets = relationship( - "Pets", - primaryjoin=( - "foreign(Pets.person_id)" "==cast(type_coerce(Person.id, Integer), Integer)" - ), - ) - - - class Pets(Base): - __tablename__ = "pets" - id = Column("id", Integer, primary_key=True) - person_id = Column("person_id", Integer) - -Above, in the :paramref:`_orm.relationship.primaryjoin` expression, we are -using :func:`.type_coerce` to handle bound parameters passed via -lazyloading as integers, since we already know these will come from -our ``StringAsInt`` type which maintains the value as an integer in -Python. We are then using :func:`.cast` so that as a SQL expression, -the VARCHAR "id" column will be CAST to an integer for a regular non- -converted join as with :meth:`_query.Query.join` or :func:`_orm.joinedload`. -That is, a joinedload of ``.pets`` looks like: - -.. sourcecode:: sql - - SELECT person.id AS person_id, pets_1.id AS pets_1_id, - pets_1.person_id AS pets_1_person_id - FROM person - LEFT OUTER JOIN pets AS pets_1 - ON pets_1.person_id = CAST(person.id AS INTEGER) - -Without the CAST in the ON clause of the join, strongly-typed databases -such as PostgreSQL will refuse to implicitly compare the integer and fail. - -The lazyload case of ``.pets`` relies upon replacing -the ``Person.id`` column at load time with a bound parameter, which receives -a Python-loaded value. This replacement is specifically where the intent -of our :func:`.type_coerce` function would be lost. Prior to the change, -this lazy load comes out as: - -.. sourcecode:: sql - - SELECT pets.id AS pets_id, pets.person_id AS pets_person_id - FROM pets - WHERE pets.person_id = CAST(CAST(%(param_1)s AS VARCHAR) AS INTEGER) - -- {'param_1': 5} - -Where above, we see that our in-Python value of ``5`` is CAST first -to a VARCHAR, then back to an INTEGER in SQL; a double CAST which works, -but is nevertheless not what we asked for. - -With the change, the :func:`.type_coerce` function maintains a wrapper -even after the column is swapped out for a bound parameter, and the query now -looks like: - -.. sourcecode:: sql - - SELECT pets.id AS pets_id, pets.person_id AS pets_person_id - FROM pets - WHERE pets.person_id = CAST(%(param_1)s AS INTEGER) - -- {'param_1': 5} - -Where our outer CAST that's in our primaryjoin still takes effect, but the -needless CAST that's in part of the ``StringAsInt`` custom type is removed -as intended by the :func:`.type_coerce` function. - - -:ticket:`3531` - -Key Behavioral Changes - ORM -============================ - -.. _behavior_change_3514: - -JSON Columns will not insert JSON NULL if no value is supplied and no default is established --------------------------------------------------------------------------------------------- - -As detailed in :ref:`change_3514`, :class:`_types.JSON` will not render -a JSON "null" value if the value is missing entirely. To prevent SQL NULL, -a default should be set up. Given the following mapping:: - - class MyObject(Base): - # ... - - json_value = Column(JSON(none_as_null=False), nullable=False) - -The following flush operation will fail with an integrity error:: - - obj = MyObject() # note no json_value - session.add(obj) - session.commit() # will fail with integrity error - -If the default for the column should be JSON NULL, set this on the -Column:: - - class MyObject(Base): - # ... - - json_value = Column(JSON(none_as_null=False), nullable=False, default=JSON.NULL) - -Or, ensure the value is present on the object:: - - obj = MyObject(json_value=None) - session.add(obj) - session.commit() # will insert JSON NULL - -Note that setting ``None`` for the default is the same as omitting it entirely; -the :paramref:`.types.JSON.none_as_null` flag does not impact the value of ``None`` -passed to :paramref:`_schema.Column.default` or :paramref:`_schema.Column.server_default`:: - - # default=None is the same as omitting it entirely, does not apply JSON NULL - json_value = Column(JSON(none_as_null=False), nullable=False, default=None) - -.. seealso:: - - :ref:`change_3514` - -.. _change_3641: - -Columns no longer added redundantly with DISTINCT + ORDER BY ------------------------------------------------------------- - -A query such as the following will now augment only those columns -that are missing from the SELECT list, without duplicates:: - - q = ( - session.query(User.id, User.name.label("name")) - .distinct() - .order_by(User.id, User.name, User.fullname) - ) - -Produces: - -.. sourcecode:: sql - - SELECT DISTINCT user.id AS a_id, user.name AS name, - user.fullname AS a_fullname - FROM a ORDER BY user.id, user.name, user.fullname - -Previously, it would produce: - -.. sourcecode:: sql - - SELECT DISTINCT user.id AS a_id, user.name AS name, user.name AS a_name, - user.fullname AS a_fullname - FROM a ORDER BY user.id, user.name, user.fullname - -Where above, the ``user.name`` column is added unnecessarily. The results -would not be affected, as the additional columns are not included in the -result in any case, but the columns are unnecessary. - -Additionally, when the PostgreSQL DISTINCT ON format is used by passing -expressions to :meth:`_query.Query.distinct`, the above "column adding" logic -is disabled entirely. - -When the query is being bundled into a subquery for the purposes of -joined eager loading, the "augment column list" rules are necessarily -more aggressive so that the ORDER BY can still be satisfied, so this case -remains unchanged. - -:ticket:`3641` - -.. _change_3776: - -Same-named @validates decorators will now raise an exception ------------------------------------------------------------- - -The :func:`_orm.validates` decorator is only intended to be created once -per class for a particular attribute name. Creating more than one -now raises an error, whereas previously it would silently pick only the -last defined validator:: - - class A(Base): - __tablename__ = "a" - id = Column(Integer, primary_key=True) - - data = Column(String) - - @validates("data") - def _validate_data_one(self): - assert "x" in data - - @validates("data") - def _validate_data_two(self): - assert "y" in data - - - configure_mappers() - -Will raise: - -.. sourcecode:: text - - sqlalchemy.exc.InvalidRequestError: A validation function for mapped attribute 'data' - on mapper Mapper|A|a already exists. - -:ticket:`3776` - -Key Behavioral Changes - Core -============================= - -.. _behavior_change_3501: - -TextClause.columns() will match columns positionally, not by name, when passed positionally -------------------------------------------------------------------------------------------- - -The new behavior of the :meth:`_expression.TextClause.columns` method, which itself -was recently added as of the 0.9 series, is that when -columns are passed positionally without any additional keyword arguments, -they are linked to the ultimate result set -columns positionally, and no longer on name. It is hoped that the impact -of this change will be low due to the fact that the method has always been documented -illustrating the columns being passed in the same order as that of the -textual SQL statement, as would seem intuitive, even though the internals -weren't checking for this. - -An application that is using this method by passing :class:`_schema.Column` objects -to it positionally must ensure that the position of those :class:`_schema.Column` -objects matches the position in which these columns are stated in the -textual SQL. - -E.g., code like the following:: - - stmt = text("SELECT id, name, description FROM table") - - # no longer matches by name - stmt = stmt.columns(my_table.c.name, my_table.c.description, my_table.c.id) - -Would no longer work as expected; the order of the columns given is now -significant:: - - # correct version - stmt = stmt.columns(my_table.c.id, my_table.c.name, my_table.c.description) - -Possibly more likely, a statement that worked like this:: - - stmt = text("SELECT * FROM table") - stmt = stmt.columns(my_table.c.id, my_table.c.name, my_table.c.description) - -is now slightly risky, as the "*" specification will generally deliver columns -in the order in which they are present in the table itself. If the structure -of the table changes due to schema changes, this ordering may no longer be the same. -Therefore when using :meth:`_expression.TextClause.columns`, it's advised to list out -the desired columns explicitly in the textual SQL, though it's no longer -necessary to worry about the names themselves in the textual SQL. - -.. seealso:: - - :ref:`change_3501` - -.. _change_3809: - -String server_default now literal quoted ----------------------------------------- - -A server default passed to :paramref:`_schema.Column.server_default` as a plain -Python string that has quotes embedded is now -passed through the literal quoting system: - -.. sourcecode:: pycon+sql - - >>> from sqlalchemy.schema import MetaData, Table, Column, CreateTable - >>> from sqlalchemy.types import String - >>> t = Table("t", MetaData(), Column("x", String(), server_default="hi ' there")) - >>> print(CreateTable(t)) - {printsql}CREATE TABLE t ( - x VARCHAR DEFAULT 'hi '' there' - ) - -Previously the quote would render directly. This change may be backwards -incompatible for applications with such a use case who were working around -the issue. - - -:ticket:`3809` - -.. _change_2528: - -A UNION or similar of SELECTs with LIMIT/OFFSET/ORDER BY now parenthesizes the embedded selects ------------------------------------------------------------------------------------------------ - -An issue that, like others, was long driven by SQLite's lack of capabilities -has now been enhanced to work on all supporting backends. We refer to a query that -is a UNION of SELECT statements that themselves contain row-limiting or ordering -features which include LIMIT, OFFSET, and/or ORDER BY: - -.. sourcecode:: sql - - (SELECT x FROM table1 ORDER BY y LIMIT 1) UNION - (SELECT x FROM table2 ORDER BY y LIMIT 2) - -The above query requires parenthesis within each sub-select in order to -group the sub-results correctly. Production of the above statement in -SQLAlchemy Core looks like:: - - stmt1 = select([table1.c.x]).order_by(table1.c.y).limit(1) - stmt2 = select([table1.c.x]).order_by(table2.c.y).limit(2) - - stmt = union(stmt1, stmt2) - -Previously, the above construct would not produce parenthesization for the -inner SELECT statements, producing a query that fails on all backends. - -The above formats will **continue to fail on SQLite**; additionally, the format -that includes ORDER BY but no LIMIT/SELECT will **continue to fail on Oracle**. -This is not a backwards-incompatible change, because the queries fail without -the parentheses as well; with the fix, the queries at least work on all other -databases. - -In all cases, in order to produce a UNION of limited SELECT statements that -also works on SQLite and in all cases on Oracle, the -subqueries must be a SELECT of an ALIAS:: - - stmt1 = select([table1.c.x]).order_by(table1.c.y).limit(1).alias().select() - stmt2 = select([table2.c.x]).order_by(table2.c.y).limit(2).alias().select() - - stmt = union(stmt1, stmt2) - -This workaround works on all SQLAlchemy versions. In the ORM, it looks like:: - - stmt1 = session.query(Model1).order_by(Model1.y).limit(1).subquery().select() - stmt2 = session.query(Model2).order_by(Model2.y).limit(1).subquery().select() - - stmt = session.query(Model1).from_statement(stmt1.union(stmt2)) - -The behavior here has many parallels to the "join rewriting" behavior -introduced in SQLAlchemy 0.9 in :ref:`feature_joins_09`; however in this case -we have opted not to add new rewriting behavior to accommodate this -case for SQLite. -The existing rewriting behavior is very complicated already, and the case of -UNIONs with parenthesized SELECT statements is much less common than the -"right-nested-join" use case of that feature. - -:ticket:`2528` - - -Dialect Improvements and Changes - PostgreSQL -============================================= - -.. _change_3529: - -Support for INSERT..ON CONFLICT (DO UPDATE | DO NOTHING) --------------------------------------------------------- - -The ``ON CONFLICT`` clause of ``INSERT`` added to PostgreSQL as of -version 9.5 is now supported using a PostgreSQL-specific version of the -:class:`_expression.Insert` object, via :func:`sqlalchemy.dialects.postgresql.dml.insert`. -This :class:`_expression.Insert` subclass adds two new methods :meth:`_expression.Insert.on_conflict_do_update` -and :meth:`_expression.Insert.on_conflict_do_nothing` which implement the full syntax -supported by PostgreSQL 9.5 in this area:: - - from sqlalchemy.dialects.postgresql import insert - - insert_stmt = insert(my_table).values(id="some_id", data="some data to insert") - - do_update_stmt = insert_stmt.on_conflict_do_update( - index_elements=[my_table.c.id], set_=dict(data="some data to update") - ) - - conn.execute(do_update_stmt) - -The above will render: - -.. sourcecode:: sql - - INSERT INTO my_table (id, data) - VALUES (:id, :data) - ON CONFLICT id DO UPDATE SET data=:data_2 - -.. seealso:: - - :ref:`postgresql_insert_on_conflict` - -:ticket:`3529` - -.. _change_3499_postgresql: - -ARRAY and JSON types now correctly specify "unhashable" -------------------------------------------------------- - -As described in :ref:`change_3499`, the ORM relies upon being able to -produce a hash function for column values when a query's selected entities -mixes full ORM entities with column expressions. The ``hashable=False`` -flag is now correctly set on all of PG's "data structure" types, including -:class:`_postgresql.ARRAY` and :class:`_postgresql.JSON`. -The :class:`_postgresql.JSONB` and :class:`.HSTORE` -types already included this flag. For :class:`_postgresql.ARRAY`, -this is conditional based on the :paramref:`.postgresql.ARRAY.as_tuple` -flag, however it should no longer be necessary to set this flag -in order to have an array value present in a composed ORM row. - -.. seealso:: - - :ref:`change_3499` - - :ref:`change_3503` - -:ticket:`3499` - -.. _change_3503: - -Correct SQL Types are Established from Indexed Access of ARRAY, JSON, HSTORE ----------------------------------------------------------------------------- - -For all three of :class:`_postgresql.ARRAY`, :class:`_postgresql.JSON` and :class:`.HSTORE`, -the SQL type assigned to the expression returned by indexed access, e.g. -``col[someindex]``, should be correct in all cases. - -This includes: - -* The SQL type assigned to indexed access of an :class:`_postgresql.ARRAY` takes into - account the number of dimensions configured. An :class:`_postgresql.ARRAY` with three - dimensions will return a SQL expression with a type of :class:`_postgresql.ARRAY` of - one less dimension. Given a column with type ``ARRAY(Integer, dimensions=3)``, - we can now perform this expression:: - - int_expr = col[5][6][7] # returns an Integer expression object - - Previously, the indexed access to ``col[5]`` would return an expression of - type :class:`.Integer` where we could no longer perform indexed access - for the remaining dimensions, unless we used :func:`.cast` or :func:`.type_coerce`. - -* The :class:`_postgresql.JSON` and :class:`_postgresql.JSONB` types now mirror what PostgreSQL - itself does for indexed access. This means that all indexed access for - a :class:`_postgresql.JSON` or :class:`_postgresql.JSONB` type returns an expression that itself - is *always* :class:`_postgresql.JSON` or :class:`_postgresql.JSONB` itself, unless the - :attr:`~.postgresql.JSON.Comparator.astext` modifier is used. This means that whether - the indexed access of the JSON structure ultimately refers to a string, - list, number, or other JSON structure, PostgreSQL always considers it - to be JSON itself unless it is explicitly cast differently. Like - the :class:`_postgresql.ARRAY` type, this means that it is now straightforward - to produce JSON expressions with multiple levels of indexed access:: - - json_expr = json_col["key1"]["attr1"][5] - -* The "textual" type that is returned by indexed access of :class:`.HSTORE` - as well as the "textual" type that is returned by indexed access of - :class:`_postgresql.JSON` and :class:`_postgresql.JSONB` in conjunction with the - :attr:`~.postgresql.JSON.Comparator.astext` modifier is now configurable; it defaults - to :class:`_expression.TextClause` in both cases but can be set to a user-defined - type using the :paramref:`.postgresql.JSON.astext_type` or - :paramref:`.postgresql.HSTORE.text_type` parameters. - -.. seealso:: - - :ref:`change_3503_cast` - -:ticket:`3499` -:ticket:`3487` - -.. _change_3503_cast: - -The JSON cast() operation now requires ``.astext`` is called explicitly ------------------------------------------------------------------------ - -As part of the changes in :ref:`change_3503`, the workings of the -:meth:`_expression.ColumnElement.cast` operator on :class:`_postgresql.JSON` and -:class:`_postgresql.JSONB` no longer implicitly invoke the -:attr:`.postgresql.JSON.Comparator.astext` modifier; PostgreSQL's JSON/JSONB types -support CAST operations to each other without the "astext" aspect. - -This means that in most cases, an application that was doing this:: - - expr = json_col["somekey"].cast(Integer) - -Will now need to change to this:: - - expr = json_col["somekey"].astext.cast(Integer) - -.. _change_2729: - -ARRAY with ENUM will now emit CREATE TYPE for the ENUM ------------------------------------------------------- - -A table definition like the following will now emit CREATE TYPE -as expected:: - - enum = Enum( - "manager", - "place_admin", - "carwash_admin", - "parking_admin", - "service_admin", - "tire_admin", - "mechanic", - "carwasher", - "tire_mechanic", - name="work_place_roles", - ) - - - class WorkPlacement(Base): - __tablename__ = "work_placement" - id = Column(Integer, primary_key=True) - roles = Column(ARRAY(enum)) - - - e = create_engine("postgresql://scott:tiger@localhost/test", echo=True) - Base.metadata.create_all(e) - -emits: - -.. sourcecode:: sql - - CREATE TYPE work_place_roles AS ENUM ( - 'manager', 'place_admin', 'carwash_admin', 'parking_admin', - 'service_admin', 'tire_admin', 'mechanic', 'carwasher', - 'tire_mechanic') - - CREATE TABLE work_placement ( - id SERIAL NOT NULL, - roles work_place_roles[], - PRIMARY KEY (id) - ) - - -:ticket:`2729` - -Check constraints now reflect ------------------------------ - -The PostgreSQL dialect now supports reflection of CHECK constraints -both within the method :meth:`_reflection.Inspector.get_check_constraints` as well -as within :class:`_schema.Table` reflection within the :attr:`_schema.Table.constraints` -collection. - -"Plain" and "Materialized" views can be inspected separately ------------------------------------------------------------- - -The new argument :paramref:`.PGInspector.get_view_names.include` -allows specification of which sub-types of views should be returned:: - - from sqlalchemy import inspect - - insp = inspect(engine) - - plain_views = insp.get_view_names(include="plain") - all_views = insp.get_view_names(include=("plain", "materialized")) - -:ticket:`3588` - - -Added tablespace option to Index --------------------------------- - -The :class:`.Index` object now accepts the argument ``postgresql_tablespace`` -in order to specify TABLESPACE, the same way as accepted by the -:class:`_schema.Table` object. - -.. seealso:: - - :ref:`postgresql_index_storage` - -:ticket:`3720` - -Support for PyGreSQL --------------------- - -The `PyGreSQL `_ DBAPI is now supported. - - -The "postgres" module is removed --------------------------------- - -The ``sqlalchemy.dialects.postgres`` module, long deprecated, is -removed; this has emitted a warning for many years and projects -should be calling upon ``sqlalchemy.dialects.postgresql``. -Engine URLs of the form ``postgres://`` will still continue to function, -however. - -Support for FOR UPDATE SKIP LOCKED / FOR NO KEY UPDATE / FOR KEY SHARE ------------------------------------------------------------------------ - -The new parameters :paramref:`.GenerativeSelect.with_for_update.skip_locked` -and :paramref:`.GenerativeSelect.with_for_update.key_share` -in both Core and ORM apply a modification to a "SELECT...FOR UPDATE" -or "SELECT...FOR SHARE" query on the PostgreSQL backend: - -* SELECT FOR NO KEY UPDATE:: - - stmt = select([table]).with_for_update(key_share=True) - -* SELECT FOR UPDATE SKIP LOCKED:: - - stmt = select([table]).with_for_update(skip_locked=True) - -* SELECT FOR KEY SHARE:: - - stmt = select([table]).with_for_update(read=True, key_share=True) - -Dialect Improvements and Changes - MySQL -======================================== - -.. _change_3547: - -MySQL JSON Support ------------------- - -A new type :class:`.mysql.JSON` is added to the MySQL dialect supporting -the JSON type newly added to MySQL 5.7. This type provides both persistence -of JSON as well as rudimentary indexed-access using the ``JSON_EXTRACT`` -function internally. An indexable JSON column that works across MySQL -and PostgreSQL can be achieved by using the :class:`_types.JSON` datatype -common to both MySQL and PostgreSQL. - -.. seealso:: - - :ref:`change_3619` - -:ticket:`3547` - -.. _change_3332: - -Added support for AUTOCOMMIT "isolation level" ----------------------------------------------- - -The MySQL dialect now accepts the value "AUTOCOMMIT" for the -:paramref:`_sa.create_engine.isolation_level` and -:paramref:`.Connection.execution_options.isolation_level` -parameters:: - - connection = engine.connect() - connection = connection.execution_options(isolation_level="AUTOCOMMIT") - -The isolation level makes use of the various "autocommit" attributes -provided by most MySQL DBAPIs. - -:ticket:`3332` - -.. _change_mysql_3216: - -No more generation of an implicit KEY for composite primary key w/ AUTO_INCREMENT ---------------------------------------------------------------------------------- - -The MySQL dialect had the behavior such that if a composite primary key -on an InnoDB table featured AUTO_INCREMENT on one of its columns which was -not the first column, e.g.:: - - t = Table( - "some_table", - metadata, - Column("x", Integer, primary_key=True, autoincrement=False), - Column("y", Integer, primary_key=True, autoincrement=True), - mysql_engine="InnoDB", - ) - -DDL such as the following would be generated: - -.. sourcecode:: sql - - CREATE TABLE some_table ( - x INTEGER NOT NULL, - y INTEGER NOT NULL AUTO_INCREMENT, - PRIMARY KEY (x, y), - KEY idx_autoinc_y (y) - )ENGINE=InnoDB - -Note the above "KEY" with an auto-generated name; this is a change that -found its way into the dialect many years ago in response to the issue that -the AUTO_INCREMENT would otherwise fail on InnoDB without this additional KEY. - -This workaround has been removed and replaced with the much better system -of just stating the AUTO_INCREMENT column *first* within the primary key: - -.. sourcecode:: sql - - CREATE TABLE some_table ( - x INTEGER NOT NULL, - y INTEGER NOT NULL AUTO_INCREMENT, - PRIMARY KEY (y, x) - )ENGINE=InnoDB - -To maintain explicit control of the ordering of primary key columns, -use the :class:`.PrimaryKeyConstraint` construct explicitly (1.1.0b2) -(along with a KEY for the autoincrement column as required by MySQL), e.g.:: - - t = Table( - "some_table", - metadata, - Column("x", Integer, primary_key=True), - Column("y", Integer, primary_key=True, autoincrement=True), - PrimaryKeyConstraint("x", "y"), - UniqueConstraint("y"), - mysql_engine="InnoDB", - ) - -Along with the change :ref:`change_3216`, composite primary keys with -or without auto increment are now easier to specify; -:paramref:`_schema.Column.autoincrement` -now defaults to the value ``"auto"`` and the ``autoincrement=False`` -directives are no longer needed:: - - t = Table( - "some_table", - metadata, - Column("x", Integer, primary_key=True), - Column("y", Integer, primary_key=True, autoincrement=True), - mysql_engine="InnoDB", - ) - -Dialect Improvements and Changes - SQLite -========================================= - -.. _change_3634: - -Right-nested join workaround lifted for SQLite version 3.7.16 -------------------------------------------------------------- - -In version 0.9, the feature introduced by :ref:`feature_joins_09` went -through lots of effort to support rewriting of joins on SQLite to always -use subqueries in order to achieve a "right-nested-join" effect, as -SQLite has not supported this syntax for many years. Ironically, -the version of SQLite noted in that migration note, 3.7.15.2, was the *last* -version of SQLite to actually have this limitation! The next release was -3.7.16 and support for right nested joins was quietly added. In 1.1, the work -to identify the specific SQLite version and source commit where this change -was made was done (SQLite's changelog refers to it with the cryptic phrase "Enhance -the query optimizer to exploit transitive join constraints" without linking -to any issue number, change number, or further explanation), and the workarounds -present in this change are now lifted for SQLite when the DBAPI reports -that version 3.7.16 or greater is in effect. - -:ticket:`3634` - -.. _change_3633: - -Dotted column names workaround lifted for SQLite version 3.10.0 ---------------------------------------------------------------- - -The SQLite dialect has long had a workaround for an issue where the database -driver does not report the correct column names for some SQL result sets, in -particular when UNION is used. The workaround is detailed at -:ref:`sqlite_dotted_column_names`, and requires that SQLAlchemy assume that any -column name with a dot in it is actually a ``tablename.columnname`` combination -delivered via this buggy behavior, with an option to turn it off via the -``sqlite_raw_colnames`` execution option. - -As of SQLite version 3.10.0, the bug in UNION and other queries has been fixed; -like the change described in :ref:`change_3634`, SQLite's changelog only -identifies it cryptically as "Added the colUsed field to sqlite3_index_info for -use by the sqlite3_module.xBestIndex method", however SQLAlchemy's translation -of these dotted column names is no longer required with this version, so is -turned off when version 3.10.0 or greater is detected. - -Overall, the SQLAlchemy :class:`_engine.ResultProxy` as of the 1.0 series relies much -less on column names in result sets when delivering results for Core and ORM -SQL constructs, so the importance of this issue was already lessened in any -case. - -:ticket:`3633` - -.. _change_sqlite_schemas: - -Improved Support for Remote Schemas ------------------------------------ -The SQLite dialect now implements :meth:`_reflection.Inspector.get_schema_names` -and additionally has improved support for tables and indexes that are -created and reflected from a remote schema, which in SQLite is a -database that is assigned a name via the ``ATTACH`` statement; previously, -the``CREATE INDEX`` DDL didn't work correctly for a schema-bound table -and the :meth:`_reflection.Inspector.get_foreign_keys` method will now indicate the -given schema in the results. Cross-schema foreign keys aren't supported. - -.. _change_3629: - -Reflection of the name of PRIMARY KEY constraints -------------------------------------------------- - -The SQLite backend now takes advantage of the "sqlite_master" view -of SQLite in order to extract the name of the primary key constraint -of a table from the original DDL, in the same way that is achieved for -foreign key constraints in recent SQLAlchemy versions. - -:ticket:`3629` - -Check constraints now reflect ------------------------------ - -The SQLite dialect now supports reflection of CHECK constraints -both within the method :meth:`_reflection.Inspector.get_check_constraints` as well -as within :class:`_schema.Table` reflection within the :attr:`_schema.Table.constraints` -collection. - -ON DELETE and ON UPDATE foreign key phrases now reflect -------------------------------------------------------- - -The :class:`_reflection.Inspector` will now include ON DELETE and ON UPDATE -phrases from foreign key constraints on the SQLite dialect, and the -:class:`_schema.ForeignKeyConstraint` object as reflected as part of a -:class:`_schema.Table` will also indicate these phrases. - -Dialect Improvements and Changes - SQL Server -============================================= - -.. _change_3534: - -Added transaction isolation level support for SQL Server --------------------------------------------------------- - -All SQL Server dialects support transaction isolation level settings -via the :paramref:`_sa.create_engine.isolation_level` and -:paramref:`.Connection.execution_options.isolation_level` -parameters. The four standard levels are supported as well as -``SNAPSHOT``:: - - engine = create_engine( - "mssql+pyodbc://scott:tiger@ms_2008", isolation_level="REPEATABLE READ" - ) - -.. seealso:: - - :ref:`mssql_isolation_level` - -:ticket:`3534` - -.. _change_3504: - -String / varlength types no longer represent "max" explicitly on reflection ---------------------------------------------------------------------------- - -When reflecting a type such as :class:`.String`, :class:`_expression.TextClause`, etc. -which includes a length, an "un-lengthed" type under SQL Server would -copy the "length" parameter as the value ``"max"``:: - - >>> from sqlalchemy import create_engine, inspect - >>> engine = create_engine("mssql+pyodbc://scott:tiger@ms_2008", echo=True) - >>> engine.execute("create table s (x varchar(max), y varbinary(max))") - >>> insp = inspect(engine) - >>> for col in insp.get_columns("s"): - ... print(col["type"].__class__, col["type"].length) - max - max - -The "length" parameter in the base types is expected to be an integer value -or None only; None indicates unbounded length which the SQL Server dialect -interprets as "max". The fix then is so that these lengths come -out as None, so that the type objects work in non-SQL Server contexts:: - - >>> for col in insp.get_columns("s"): - ... print(col["type"].__class__, col["type"].length) - None - None - -Applications which may have been relying on a direct comparison of the "length" -value to the string "max" should consider the value of ``None`` to mean -the same thing. - -:ticket:`3504` - -Support for "non clustered" on primary key to allow clustered elsewhere ------------------------------------------------------------------------ - -The ``mssql_clustered`` flag available on :class:`.UniqueConstraint`, -:class:`.PrimaryKeyConstraint`, :class:`.Index` now defaults to ``None``, and -can be set to False which will render the NONCLUSTERED keyword in particular -for a primary key, allowing a different index to be used as "clustered". - -.. seealso:: - - :ref:`mssql_indexes` - -.. _change_3434: - -The legacy_schema_aliasing flag is now set to False ---------------------------------------------------- - -SQLAlchemy 1.0.5 introduced the ``legacy_schema_aliasing`` flag to the -MSSQL dialect, allowing so-called "legacy mode" aliasing to be turned off. -This aliasing attempts to turn schema-qualified tables into aliases; -given a table such as:: - - account_table = Table( - "account", - metadata, - Column("id", Integer, primary_key=True), - Column("info", String(100)), - schema="customer_schema", - ) - -The legacy mode of behavior will attempt to turn a schema-qualified table -name into an alias: - -.. sourcecode:: pycon+sql - - >>> eng = create_engine("mssql+pymssql://mydsn", legacy_schema_aliasing=True) - >>> print(account_table.select().compile(eng)) - {printsql}SELECT account_1.id, account_1.info - FROM customer_schema.account AS account_1 - -However, this aliasing has been shown to be unnecessary and in many cases -produces incorrect SQL. - -In SQLAlchemy 1.1, the ``legacy_schema_aliasing`` flag now defaults to -False, disabling this mode of behavior and allowing the MSSQL dialect to behave -normally with schema-qualified tables. For applications which may rely -on this behavior, set the flag back to True. - - -:ticket:`3434` - -Dialect Improvements and Changes - Oracle -========================================= - -Support for SKIP LOCKED ------------------------ - -The new parameter :paramref:`.GenerativeSelect.with_for_update.skip_locked` -in both Core and ORM will generate the "SKIP LOCKED" suffix for a -"SELECT...FOR UPDATE" or "SELECT.. FOR SHARE" query. diff --git a/doc/build/changelog/migration_12.rst b/doc/build/changelog/migration_12.rst deleted file mode 100644 index 454b17f12a..0000000000 --- a/doc/build/changelog/migration_12.rst +++ /dev/null @@ -1,1851 +0,0 @@ -============================= -What's New in SQLAlchemy 1.2? -============================= - -.. admonition:: About this Document - - This document describes changes between SQLAlchemy version 1.1 - and SQLAlchemy version 1.2. - - -Introduction -============ - -This guide introduces what's new in SQLAlchemy version 1.2, -and also documents changes which affect users migrating -their applications from the 1.1 series of SQLAlchemy to 1.2. - -Please carefully review the sections on behavioral changes for -potentially backwards-incompatible changes in behavior. - -Platform Support -================ - -Targeting Python 2.7 and Up ---------------------------- - -SQLAlchemy 1.2 now moves the minimum Python version to 2.7, no longer -supporting 2.6. New language features are expected to be merged -into the 1.2 series that were not supported in Python 2.6. For Python 3 support, -SQLAlchemy is currently tested on versions 3.5 and 3.6. - - -New Features and Improvements - ORM -=================================== - -.. _change_3954: - -"Baked" loading now the default for lazy loads ----------------------------------------------- - -The :mod:`sqlalchemy.ext.baked` extension, first introduced in the 1.0 series, -allows for the construction of a so-called :class:`.BakedQuery` object, -which is an object that generates a :class:`_query.Query` object in conjunction -with a cache key representing the structure of the query; this cache key -is then linked to the resulting string SQL statement so that subsequent use -of another :class:`.BakedQuery` with the same structure will bypass all the -overhead of building the :class:`_query.Query` object, building the core -:func:`_expression.select` object within, as well as the compilation of the :func:`_expression.select` -into a string, cutting out well the majority of function call overhead normally -associated with constructing and emitting an ORM :class:`_query.Query` object. - -The :class:`.BakedQuery` is now used by default by the ORM when it generates -a "lazy" query for the lazy load of a :func:`_orm.relationship` construct, e.g. -that of the default ``lazy="select"`` relationship loader strategy. This -will allow for a significant reduction in function calls within the scope -of an application's use of lazy load queries to load collections and related -objects. Previously, this feature was available -in 1.0 and 1.1 through the use of a global API method or by using the -``baked_select`` strategy, it's now the only implementation for this behavior. -The feature has also been improved such that the caching can still take place -for objects that have additional loader options in effect subsequent -to the lazy load. - -The caching behavior can be disabled on a per-relationship basis using the -:paramref:`_orm.relationship.bake_queries` flag, which is available for -very unusual cases, such as a relationship that uses a custom -:class:`_query.Query` implementation that's not compatible with caching. - - -:ticket:`3954` - -.. _change_3944: - -New "selectin" eager loading, loads all collections at once using IN --------------------------------------------------------------------- - -A new eager loader called "selectin" loading is added, which in many ways -is similar to "subquery" loading, however produces a simpler SQL statement -that is cacheable as well as more efficient. - -Given a query as below:: - - q = ( - session.query(User) - .filter(User.name.like("%ed%")) - .options(subqueryload(User.addresses)) - ) - -The SQL produced would be the query against ``User`` followed by the -subqueryload for ``User.addresses`` (note the parameters are also listed): - -.. sourcecode:: sql - - SELECT users.id AS users_id, users.name AS users_name - FROM users - WHERE users.name LIKE ? - ('%ed%',) - - SELECT addresses.id AS addresses_id, - addresses.user_id AS addresses_user_id, - addresses.email_address AS addresses_email_address, - anon_1.users_id AS anon_1_users_id - FROM (SELECT users.id AS users_id - FROM users - WHERE users.name LIKE ?) AS anon_1 - JOIN addresses ON anon_1.users_id = addresses.user_id - ORDER BY anon_1.users_id - ('%ed%',) - -With "selectin" loading, we instead get a SELECT that refers to the -actual primary key values loaded in the parent query:: - - q = ( - session.query(User) - .filter(User.name.like("%ed%")) - .options(selectinload(User.addresses)) - ) - -Produces: - -.. sourcecode:: sql - - SELECT users.id AS users_id, users.name AS users_name - FROM users - WHERE users.name LIKE ? - ('%ed%',) - - SELECT users_1.id AS users_1_id, - addresses.id AS addresses_id, - addresses.user_id AS addresses_user_id, - addresses.email_address AS addresses_email_address - FROM users AS users_1 - JOIN addresses ON users_1.id = addresses.user_id - WHERE users_1.id IN (?, ?) - ORDER BY users_1.id - (1, 3) - -The above SELECT statement includes these advantages: - -* It doesn't use a subquery, just an INNER JOIN, meaning it will perform - much better on a database like MySQL that doesn't like subqueries - -* Its structure is independent of the original query; in conjunction with the - new :ref:`expanding IN parameter system ` we can in most cases - use the "baked" query to cache the string SQL, reducing per-query overhead - significantly - -* Because the query only fetches for a given list of primary key identifiers, - "selectin" loading is potentially compatible with :meth:`_query.Query.yield_per` to - operate on chunks of a SELECT result at a time, provided that the - database driver allows for multiple, simultaneous cursors (SQLite, PostgreSQL; - **not** MySQL drivers or SQL Server ODBC drivers). Neither joined eager - loading nor subquery eager loading are compatible with :meth:`_query.Query.yield_per`. - -The disadvantages of selectin eager loading are potentially large SQL -queries, with large lists of IN parameters. The list of IN parameters themselves -are chunked in groups of 500, so a result set of more than 500 lead objects -will have more additional "SELECT IN" queries following. Also, support -for composite primary keys depends on the database's ability to use -tuples with IN, e.g. -``(table.column_one, table_column_two) IN ((?, ?), (?, ?) (?, ?))``. -Currently, PostgreSQL and MySQL are known to be compatible with this syntax, -SQLite is not. - -.. seealso:: - - :ref:`selectin_eager_loading` - -:ticket:`3944` - -.. _change_3948: - -"selectin" polymorphic loading, loads subclasses using separate IN queries --------------------------------------------------------------------------- - -Along similar lines as the "selectin" relationship loading feature just -described at :ref:`change_3944` is "selectin" polymorphic loading. This -is a polymorphic loading feature tailored primarily towards joined eager -loading that allows the loading of the base entity to proceed with a simple -SELECT statement, but then the attributes of the additional subclasses -are loaded with additional SELECT statements: - -.. sourcecode:: pycon+sql - - >>> from sqlalchemy.orm import selectin_polymorphic - - >>> query = session.query(Employee).options( - ... selectin_polymorphic(Employee, [Manager, Engineer]) - ... ) - - >>> query.all() - {execsql}SELECT - employee.id AS employee_id, - employee.name AS employee_name, - employee.type AS employee_type - FROM employee - () - - SELECT - engineer.id AS engineer_id, - employee.id AS employee_id, - employee.type AS employee_type, - engineer.engineer_name AS engineer_engineer_name - FROM employee JOIN engineer ON employee.id = engineer.id - WHERE employee.id IN (?, ?) ORDER BY employee.id - (1, 2) - - SELECT - manager.id AS manager_id, - employee.id AS employee_id, - employee.type AS employee_type, - manager.manager_name AS manager_manager_name - FROM employee JOIN manager ON employee.id = manager.id - WHERE employee.id IN (?) ORDER BY employee.id - (3,) - -.. seealso:: - - :ref:`polymorphic_selectin` - -:ticket:`3948` - -.. _change_3058: - -ORM attributes that can receive ad-hoc SQL expressions ------------------------------------------------------- - -A new ORM attribute type :func:`_orm.query_expression` is added which -is similar to :func:`_orm.deferred`, except its SQL expression -is determined at query time using a new option :func:`_orm.with_expression`; -if not specified, the attribute defaults to ``None``:: - - from sqlalchemy.orm import query_expression - from sqlalchemy.orm import with_expression - - - class A(Base): - __tablename__ = "a" - id = Column(Integer, primary_key=True) - x = Column(Integer) - y = Column(Integer) - - # will be None normally... - expr = query_expression() - - - # but let's give it x + y - a1 = session.query(A).options(with_expression(A.expr, A.x + A.y)).first() - print(a1.expr) - -.. seealso:: - - :ref:`mapper_querytime_expression` - -:ticket:`3058` - -.. _change_orm_959: - -ORM Support of multiple-table deletes -------------------------------------- - -The ORM :meth:`_query.Query.delete` method supports multiple-table criteria -for DELETE, as introduced in :ref:`change_959`. The feature works -in the same manner as multiple-table criteria for UPDATE, first -introduced in 0.8 and described at :ref:`change_orm_2365`. - -Below, we emit a DELETE against ``SomeEntity``, adding -a FROM clause (or equivalent, depending on backend) -against ``SomeOtherEntity``:: - - query(SomeEntity).filter(SomeEntity.id == SomeOtherEntity.id).filter( - SomeOtherEntity.foo == "bar" - ).delete() - -.. seealso:: - - :ref:`change_959` - -:ticket:`959` - -.. _change_3229: - -Support for bulk updates of hybrids, composites ------------------------------------------------ - -Both hybrid attributes (e.g. :mod:`sqlalchemy.ext.hybrid`) as well as composite -attributes (:ref:`mapper_composite`) now support being used in the -SET clause of an UPDATE statement when using :meth:`_query.Query.update`. - -For hybrids, simple expressions can be used directly, or the new decorator -:meth:`.hybrid_property.update_expression` can be used to break a value -into multiple columns/expressions:: - - class Person(Base): - # ... - - first_name = Column(String(10)) - last_name = Column(String(10)) - - @hybrid.hybrid_property - def name(self): - return self.first_name + " " + self.last_name - - @name.expression - def name(cls): - return func.concat(cls.first_name, " ", cls.last_name) - - @name.update_expression - def name(cls, value): - f, l = value.split(" ", 1) - return [(cls.first_name, f), (cls.last_name, l)] - -Above, an UPDATE can be rendered using:: - - session.query(Person).filter(Person.id == 5).update({Person.name: "Dr. No"}) - -Similar functionality is available for composites, where composite values -will be broken out into their individual columns for bulk UPDATE:: - - session.query(Vertex).update({Edge.start: Point(3, 4)}) - -.. seealso:: - - :ref:`hybrid_bulk_update` - -.. _change_3911_3912: - -Hybrid attributes support reuse among subclasses, redefinition of @getter -------------------------------------------------------------------------- - -The :class:`sqlalchemy.ext.hybrid.hybrid_property` class now supports -calling mutators like ``@setter``, ``@expression`` etc. multiple times -across subclasses, and now provides a ``@getter`` mutator, so that -a particular hybrid can be repurposed across subclasses or other -classes. This now is similar to the behavior of ``@property`` in standard -Python:: - - class FirstNameOnly(Base): - # ... - - first_name = Column(String) - - @hybrid_property - def name(self): - return self.first_name - - @name.setter - def name(self, value): - self.first_name = value - - - class FirstNameLastName(FirstNameOnly): - # ... - - last_name = Column(String) - - @FirstNameOnly.name.getter - def name(self): - return self.first_name + " " + self.last_name - - @name.setter - def name(self, value): - self.first_name, self.last_name = value.split(" ", maxsplit=1) - - @name.expression - def name(cls): - return func.concat(cls.first_name, " ", cls.last_name) - -Above, the ``FirstNameOnly.name`` hybrid is referenced by the -``FirstNameLastName`` subclass in order to repurpose it specifically to the -new subclass. This is achieved by copying the hybrid object to a new one -within each call to ``@getter``, ``@setter``, as well as in all other -mutator methods like ``@expression``, leaving the previous hybrid's definition -intact. Previously, methods like ``@setter`` would modify the existing -hybrid in-place, interfering with the definition on the superclass. - -.. note:: Be sure to read the documentation at :ref:`hybrid_reuse_subclass` - for important notes regarding how to override - :meth:`.hybrid_property.expression` - and :meth:`.hybrid_property.comparator`, as a special qualifier - :attr:`.hybrid_property.overrides` may be necessary to avoid name - conflicts with :class:`.QueryableAttribute` in some cases. - -.. note:: This change in ``@hybrid_property`` implies that when adding setters and - other state to a ``@hybrid_property``, the **methods must retain the name - of the original hybrid**, else the new hybrid with the additional state will - be present on the class as the non-matching name. This is the same behavior - as that of the ``@property`` construct that is part of standard Python:: - - class FirstNameOnly(Base): - @hybrid_property - def name(self): - return self.first_name - - # WRONG - will raise AttributeError: can't set attribute when - # assigning to .name - @name.setter - def _set_name(self, value): - self.first_name = value - - - class FirstNameOnly(Base): - @hybrid_property - def name(self): - return self.first_name - - # CORRECT - note regular Python @property works the same way - @name.setter - def name(self, value): - self.first_name = value - -:ticket:`3911` - -:ticket:`3912` - -.. _change_3896_event: - -New bulk_replace event ----------------------- - -To suit the validation use case described in :ref:`change_3896_validates`, -a new :meth:`.AttributeEvents.bulk_replace` method is added, which is -called in conjunction with the :meth:`.AttributeEvents.append` and -:meth:`.AttributeEvents.remove` events. "bulk_replace" is called before -"append" and "remove" so that the collection can be modified ahead of comparison -to the existing collection. After that, individual items -are appended to a new target collection, firing off the "append" -event for items new to the collection, as was the previous behavior. -Below illustrates both "bulk_replace" and -"append" at the same time, including that "append" will receive an object -already handled by "bulk_replace" if collection assignment is used. -A new symbol :attr:`~.attributes.OP_BULK_REPLACE` may be used to determine -if this "append" event is the second part of a bulk replace:: - - from sqlalchemy.orm.attributes import OP_BULK_REPLACE - - - @event.listens_for(SomeObject.collection, "bulk_replace") - def process_collection(target, values, initiator): - values[:] = [_make_value(value) for value in values] - - - @event.listens_for(SomeObject.collection, "append", retval=True) - def process_collection(target, value, initiator): - # make sure bulk_replace didn't already do it - if initiator is None or initiator.op is not OP_BULK_REPLACE: - return _make_value(value) - else: - return value - -:ticket:`3896` - -.. _change_3303: - -New "modified" event handler for sqlalchemy.ext.mutable -------------------------------------------------------- - -A new event handler :meth:`.AttributeEvents.modified` is added, which is -triggered corresponding to calls to the :func:`.attributes.flag_modified` -method, which is normally called from the :mod:`sqlalchemy.ext.mutable` -extension:: - - from sqlalchemy.ext.declarative import declarative_base - from sqlalchemy.ext.mutable import MutableDict - from sqlalchemy import event - - Base = declarative_base() - - - class MyDataClass(Base): - __tablename__ = "my_data" - id = Column(Integer, primary_key=True) - data = Column(MutableDict.as_mutable(JSONEncodedDict)) - - - @event.listens_for(MyDataClass.data, "modified") - def modified_json(instance): - print("json value modified:", instance.data) - -Above, the event handler will be triggered when an in-place change to the -``.data`` dictionary occurs. - -:ticket:`3303` - -.. _change_3991: - -Added "for update" arguments to Session.refresh ------------------------------------------------- - -Added new argument :paramref:`.Session.refresh.with_for_update` to the -:meth:`.Session.refresh` method. When the :meth:`_query.Query.with_lockmode` -method were deprecated in favor of :meth:`_query.Query.with_for_update`, -the :meth:`.Session.refresh` method was never updated to reflect -the new option:: - - session.refresh(some_object, with_for_update=True) - -The :paramref:`.Session.refresh.with_for_update` argument accepts a dictionary -of options that will be passed as the same arguments which are sent to -:meth:`_query.Query.with_for_update`:: - - session.refresh(some_objects, with_for_update={"read": True}) - -The new parameter supersedes the :paramref:`.Session.refresh.lockmode` -parameter. - -:ticket:`3991` - -.. _change_3853: - -In-place mutation operators work for MutableSet, MutableList ------------------------------------------------------------- - -Implemented the in-place mutation operators ``__ior__``, ``__iand__``, -``__ixor__`` and ``__isub__`` for :class:`.mutable.MutableSet` and ``__iadd__`` -for :class:`.mutable.MutableList`. While these -methods would successfully update the collection previously, they would -not correctly fire off change events. The operators mutate the collection -as before but additionally emit the correct change event so that the change -becomes part of the next flush process:: - - model = session.query(MyModel).first() - model.json_set &= {1, 3} - -:ticket:`3853` - -.. _change_3769: - -AssociationProxy any(), has(), contains() work with chained association proxies -------------------------------------------------------------------------------- - -The :meth:`.AssociationProxy.any`, :meth:`.AssociationProxy.has` -and :meth:`.AssociationProxy.contains` comparison methods now support -linkage to an attribute that is -itself also an :class:`.AssociationProxy`, recursively. Below, ``A.b_values`` -is an association proxy that links to ``AtoB.bvalue``, which is -itself an association proxy onto ``B``:: - - class A(Base): - __tablename__ = "a" - id = Column(Integer, primary_key=True) - - b_values = association_proxy("atob", "b_value") - c_values = association_proxy("atob", "c_value") - - - class B(Base): - __tablename__ = "b" - id = Column(Integer, primary_key=True) - a_id = Column(ForeignKey("a.id")) - value = Column(String) - - c = relationship("C") - - - class C(Base): - __tablename__ = "c" - id = Column(Integer, primary_key=True) - b_id = Column(ForeignKey("b.id")) - value = Column(String) - - - class AtoB(Base): - __tablename__ = "atob" - - a_id = Column(ForeignKey("a.id"), primary_key=True) - b_id = Column(ForeignKey("b.id"), primary_key=True) - - a = relationship("A", backref="atob") - b = relationship("B", backref="atob") - - b_value = association_proxy("b", "value") - c_value = association_proxy("b", "c") - -We can query on ``A.b_values`` using :meth:`.AssociationProxy.contains` to -query across the two proxies ``A.b_values``, ``AtoB.b_value``: - -.. sourcecode:: pycon+sql - - >>> s.query(A).filter(A.b_values.contains("hi")).all() - {execsql}SELECT a.id AS a_id - FROM a - WHERE EXISTS (SELECT 1 - FROM atob - WHERE a.id = atob.a_id AND (EXISTS (SELECT 1 - FROM b - WHERE b.id = atob.b_id AND b.value = :value_1))) - -Similarly, we can query on ``A.c_values`` using :meth:`.AssociationProxy.any` -to query across the two proxies ``A.c_values``, ``AtoB.c_value``: - -.. sourcecode:: pycon+sql - - >>> s.query(A).filter(A.c_values.any(value="x")).all() - {execsql}SELECT a.id AS a_id - FROM a - WHERE EXISTS (SELECT 1 - FROM atob - WHERE a.id = atob.a_id AND (EXISTS (SELECT 1 - FROM b - WHERE b.id = atob.b_id AND (EXISTS (SELECT 1 - FROM c - WHERE b.id = c.b_id AND c.value = :value_1))))) - -:ticket:`3769` - -.. _change_4137: - -Identity key enhancements to support sharding ---------------------------------------------- - -The identity key structure used by the ORM now contains an additional -member, so that two identical primary keys that originate from different -contexts can co-exist within the same identity map. - -The example at :ref:`examples_sharding` has been updated to illustrate this -behavior. The example shows a sharded class ``WeatherLocation`` that -refers to a dependent ``WeatherReport`` object, where the ``WeatherReport`` -class is mapped to a table that stores a simple integer primary key. Two -``WeatherReport`` objects from different databases may have the same -primary key value. The example now illustrates that a new ``identity_token`` -field tracks this difference so that the two objects can co-exist in the -same identity map:: - - tokyo = WeatherLocation("Asia", "Tokyo") - newyork = WeatherLocation("North America", "New York") - - tokyo.reports.append(Report(80.0)) - newyork.reports.append(Report(75)) - - sess = create_session() - - sess.add_all([tokyo, newyork, quito]) - - sess.commit() - - # the Report class uses a simple integer primary key. So across two - # databases, a primary key will be repeated. The "identity_token" tracks - # in memory that these two identical primary keys are local to different - # databases. - - newyork_report = newyork.reports[0] - tokyo_report = tokyo.reports[0] - - assert inspect(newyork_report).identity_key == (Report, (1,), "north_america") - assert inspect(tokyo_report).identity_key == (Report, (1,), "asia") - - # the token representing the originating shard is also available directly - - assert inspect(newyork_report).identity_token == "north_america" - assert inspect(tokyo_report).identity_token == "asia" - -:ticket:`4137` - -New Features and Improvements - Core -==================================== - -.. _change_4102: - -Boolean datatype now enforces strict True/False/None values ------------------------------------------------------------ - -In version 1.1, the change described in :ref:`change_3730` produced an -unintended side effect of altering the way :class:`.Boolean` behaves when -presented with a non-integer value, such as a string. In particular, the -string value ``"0"``, which would previously result in the value ``False`` -being generated, would now produce ``True``. Making matters worse, the change -in behavior was only for some backends and not others, meaning code that sends -string ``"0"`` values to :class:`.Boolean` would break inconsistently across -backends. - -The ultimate solution to this problem is that **string values are not supported -with Boolean**, so in 1.2 a hard ``TypeError`` is raised if a non-integer / -True/False/None value is passed. Additionally, only the integer values -0 and 1 are accepted. - -To accommodate for applications that wish to have more liberal interpretation -of boolean values, the :class:`.TypeDecorator` should be used. Below -illustrates a recipe that will allow for the "liberal" behavior of the pre-1.1 -:class:`.Boolean` datatype:: - - from sqlalchemy import Boolean - from sqlalchemy import TypeDecorator - - - class LiberalBoolean(TypeDecorator): - impl = Boolean - - def process_bind_param(self, value, dialect): - if value is not None: - value = bool(int(value)) - return value - -:ticket:`4102` - -.. _change_3919: - -Pessimistic disconnection detection added to the connection pool ----------------------------------------------------------------- - -The connection pool documentation has long featured a recipe for using -the :meth:`_events.ConnectionEvents.engine_connect` engine event to emit a simple -statement on a checked-out connection to test it for liveness. The -functionality of this recipe has now been added into the connection pool -itself, when used in conjunction with an appropriate dialect. Using -the new parameter :paramref:`_sa.create_engine.pool_pre_ping`, each connection -checked out will be tested for freshness before being returned:: - - engine = create_engine("mysql+pymysql://", pool_pre_ping=True) - -While the "pre-ping" approach adds a small amount of latency to the connection -pool checkout, for a typical application that is transactionally-oriented -(which includes most ORM applications), this overhead is minimal, and -eliminates the problem of acquiring a stale connection that will raise -an error, requiring that the application either abandon or retry the operation. - -The feature does **not** accommodate for connections dropped within -an ongoing transaction or SQL operation. If an application must recover -from these as well, it would need to employ its own operation retry logic -to anticipate these errors. - - -.. seealso:: - - :ref:`pool_disconnects_pessimistic` - - -:ticket:`3919` - -.. _change_3907: - -The IN / NOT IN operator's empty collection behavior is now configurable; default expression simplified -------------------------------------------------------------------------------------------------------- - -An expression such as ``column.in_([])``, which is assumed to be false, -now produces the expression ``1 != 1`` -by default, instead of ``column != column``. This will **change the result** -of a query that is comparing a SQL expression or column that evaluates to -NULL when compared to an empty set, producing a boolean value false or true -(for NOT IN) rather than NULL. The warning that would emit under -this condition is also removed. The old behavior is available using the -:paramref:`_sa.create_engine.empty_in_strategy` parameter to -:func:`_sa.create_engine`. - -In SQL, the IN and NOT IN operators do not support comparison to a -collection of values that is explicitly empty; meaning, this syntax is -illegal: - -.. sourcecode:: sql - - mycolumn IN () - -To work around this, SQLAlchemy and other database libraries detect this -condition and render an alternative expression that evaluates to false, or -in the case of NOT IN, to true, based on the theory that "col IN ()" is always -false since nothing is in "the empty set". Typically, in order to -produce a false/true constant that is portable across databases and works -in the context of the WHERE clause, a simple tautology such as ``1 != 1`` is -used to evaluate to false and ``1 = 1`` to evaluate to true (a simple constant -"0" or "1" often does not work as the target of a WHERE clause). - -SQLAlchemy in its early days began with this approach as well, but soon it -was theorized that the SQL expression ``column IN ()`` would not evaluate to -false if the "column" were NULL; instead, the expression would produce NULL, -since "NULL" means "unknown", and comparisons to NULL in SQL usually produce -NULL. - -To simulate this result, SQLAlchemy changed from using ``1 != 1`` to -instead use th expression ``expr != expr`` for empty "IN" and ``expr = expr`` -for empty "NOT IN"; that is, instead of using a fixed value we use the -actual left-hand side of the expression. If the left-hand side of -the expression passed evaluates to NULL, then the comparison overall -also gets the NULL result instead of false or true. - -Unfortunately, users eventually complained that this expression had a very -severe performance impact on some query planners. At that point, a warning -was added when an empty IN expression was encountered, favoring that SQLAlchemy -continues to be "correct" and urging users to avoid code that generates empty -IN predicates in general, since typically they can be safely omitted. However, -this is of course burdensome in the case of queries that are built up dynamically -from input variables, where an incoming set of values might be empty. - -In recent months, the original assumptions of this decision have been -questioned. The notion that the expression "NULL IN ()" should return NULL was -only theoretical, and could not be tested since databases don't support that -syntax. However, as it turns out, you can in fact ask a relational database -what value it would return for "NULL IN ()" by simulating the empty set as -follows: - -.. sourcecode:: sql - - SELECT NULL IN (SELECT 1 WHERE 1 != 1) - -With the above test, we see that the databases themselves can't agree on -the answer. PostgreSQL, considered by most to be the most "correct" database, -returns False; because even though "NULL" represents "unknown", the "empty set" -means nothing is present, including all unknown values. On the -other hand, MySQL and MariaDB return NULL for the above expression, defaulting -to the more common behavior of "all comparisons to NULL return NULL". - -SQLAlchemy's SQL architecture is more sophisticated than it was when this -design decision was first made, so we can now allow either behavior to -be invoked at SQL string compilation time. Previously, the conversion to a -comparison expression were done at construction time, that is, the moment -the :meth:`.ColumnOperators.in_` or :meth:`.ColumnOperators.notin_` operators were invoked. -With the compilation-time behavior, the dialect itself can be instructed -to invoke either approach, that is, the "static" ``1 != 1`` comparison or the -"dynamic" ``expr != expr`` comparison. The default has been **changed** -to be the "static" comparison, since this agrees with the behavior that -PostgreSQL would have in any case and this is also what the vast majority -of users prefer. This will **change the result** of a query that is comparing -a null expression to the empty set, particularly one that is querying -for the negation ``where(~null_expr.in_([]))``, since this now evaluates to true -and not NULL. - -The behavior can now be controlled using the flag -:paramref:`_sa.create_engine.empty_in_strategy`, which defaults to the -``"static"`` setting, but may also be set to ``"dynamic"`` or -``"dynamic_warn"``, where the ``"dynamic_warn"`` setting is equivalent to the -previous behavior of emitting ``expr != expr`` as well as a performance -warning. However, it is anticipated that most users will appreciate the -"static" default. - -:ticket:`3907` - -.. _change_3953: - -Late-expanded IN parameter sets allow IN expressions with cached statements ---------------------------------------------------------------------------- - -Added a new kind of :func:`.bindparam` called "expanding". This is -for use in ``IN`` expressions where the list of elements is rendered -into individual bound parameters at statement execution time, rather -than at statement compilation time. This allows both a single bound -parameter name to be linked to an IN expression of multiple elements, -as well as allows query caching to be used with IN expressions. The -new feature allows the related features of "select in" loading and -"polymorphic in" loading to make use of the baked query extension -to reduce call overhead:: - - stmt = select([table]).where(table.c.col.in_(bindparam("foo", expanding=True))) - conn.execute(stmt, {"foo": [1, 2, 3]}) - -The feature should be regarded as **experimental** within the 1.2 series. - - -:ticket:`3953` - -.. _change_3999: - -Flattened operator precedence for comparison operators -------------------------------------------------------- - -The operator precedence for operators like IN, LIKE, equals, IS, MATCH, and -other comparison operators has been flattened into one level. This will -have the effect of more parenthesization being generated when comparison -operators are combined together, such as:: - - (column("q") == null()) != (column("y") == null()) - -Will now generate ``(q IS NULL) != (y IS NULL)`` rather than -``q IS NULL != y IS NULL``. - - -:ticket:`3999` - -.. _change_1546: - -Support for SQL Comments on Table, Column, includes DDL, reflection -------------------------------------------------------------------- - -The Core receives support for string comments associated with tables -and columns. These are specified via the :paramref:`_schema.Table.comment` and -:paramref:`_schema.Column.comment` arguments:: - - Table( - "my_table", - metadata, - Column("q", Integer, comment="the Q value"), - comment="my Q table", - ) - -Above, DDL will be rendered appropriately upon table create to associate -the above comments with the table/ column within the schema. When -the above table is autoloaded or inspected with :meth:`_reflection.Inspector.get_columns`, -the comments are included. The table comment is also available independently -using the :meth:`_reflection.Inspector.get_table_comment` method. - -Current backend support includes MySQL, PostgreSQL, and Oracle. - -:ticket:`1546` - -.. _change_959: - -Multiple-table criteria support for DELETE ------------------------------------------- - -The :class:`_expression.Delete` construct now supports multiple-table criteria, -implemented for those backends which support it, currently these are -PostgreSQL, MySQL and Microsoft SQL Server (support is also added to the -currently non-working Sybase dialect). The feature works in the same -was as that of multiple-table criteria for UPDATE, first introduced in -the 0.7 and 0.8 series. - -Given a statement as:: - - stmt = ( - users.delete() - .where(users.c.id == addresses.c.id) - .where(addresses.c.email_address.startswith("ed%")) - ) - conn.execute(stmt) - -The resulting SQL from the above statement on a PostgreSQL backend -would render as: - -.. sourcecode:: sql - - DELETE FROM users USING addresses - WHERE users.id = addresses.id - AND (addresses.email_address LIKE %(email_address_1)s || '%%') - -.. seealso:: - - :ref:`tutorial_multi_table_deletes` - -:ticket:`959` - -.. _change_2694: - -New "autoescape" option for startswith(), endswith() ----------------------------------------------------- - -The "autoescape" parameter is added to :meth:`.ColumnOperators.startswith`, -:meth:`.ColumnOperators.endswith`, :meth:`.ColumnOperators.contains`. -This parameter when set to ``True`` will automatically escape all occurrences -of ``%``, ``_`` with an escape character, which defaults to a forwards slash ``/``; -occurrences of the escape character itself are also escaped. The forwards slash -is used to avoid conflicts with settings like PostgreSQL's -``standard_confirming_strings``, whose default value changed as of PostgreSQL -9.1, and MySQL's ``NO_BACKSLASH_ESCAPES`` settings. The existing "escape" parameter -can now be used to change the autoescape character, if desired. - -.. note:: This feature has been changed as of 1.2.0 from its initial - implementation in 1.2.0b2 such that autoescape is now passed as a boolean - value, rather than a specific character to use as the escape character. - -An expression such as:: - - >>> column("x").startswith("total%score", autoescape=True) - -Renders as: - -.. sourcecode:: sql - - x LIKE :x_1 || '%' ESCAPE '/' - -Where the value of the parameter "x_1" is ``'total/%score'``. - -Similarly, an expression that has backslashes:: - - >>> column("x").startswith("total/score", autoescape=True) - -Will render the same way, with the value of the parameter "x_1" as -``'total//score'``. - - -:ticket:`2694` - -.. _change_floats_12: - -Stronger typing added to "float" datatypes ------------------------------------------- - -A series of changes allow for use of the :class:`.Float` datatype to more -strongly link itself to Python floating point values, instead of the more -generic :class:`.Numeric`. The changes are mostly related to ensuring -that Python floating point values are not erroneously coerced to -``Decimal()``, and are coerced to ``float`` if needed, on the result side, -if the application is working with plain floats. - -* A plain Python "float" value passed to a SQL expression will now be - pulled into a literal parameter with the type :class:`.Float`; previously, - the type was :class:`.Numeric`, with the default "asdecimal=True" flag, which - meant the result type would coerce to ``Decimal()``. In particular, - this would emit a confusing warning on SQLite:: - - - float_value = connection.scalar( - select([literal(4.56)]) # the "BindParameter" will now be - # Float, not Numeric(asdecimal=True) - ) - -* Math operations between :class:`.Numeric`, :class:`.Float`, and - :class:`.Integer` will now preserve the :class:`.Numeric` or :class:`.Float` - type in the resulting expression's type, including the ``asdecimal`` flag - as well as if the type should be :class:`.Float`:: - - # asdecimal flag is maintained - expr = column("a", Integer) * column("b", Numeric(asdecimal=False)) - assert expr.type.asdecimal == False - - # Float subclass of Numeric is maintained - expr = column("a", Integer) * column("b", Float()) - assert isinstance(expr.type, Float) - -* The :class:`.Float` datatype will apply the ``float()`` processor to - result values unconditionally if the DBAPI is known to support native - ``Decimal()`` mode. Some backends do not always guarantee that a floating - point number comes back as plain float and not precision numeric such - as MySQL. - -:ticket:`4017` - -:ticket:`4018` - -:ticket:`4020` - -.. change_3249: - -Support for GROUPING SETS, CUBE, ROLLUP ---------------------------------------- - -All three of GROUPING SETS, CUBE, ROLLUP are available via the -:attr:`.func` namespace. In the case of CUBE and ROLLUP, these functions -already work in previous versions, however for GROUPING SETS, a placeholder -is added to the compiler to allow for the space. All three functions -are named in the documentation now: - -.. sourcecode:: pycon+sql - - >>> from sqlalchemy import select, table, column, func, tuple_ - >>> t = table("t", column("value"), column("x"), column("y"), column("z"), column("q")) - >>> stmt = select([func.sum(t.c.value)]).group_by( - ... func.grouping_sets( - ... tuple_(t.c.x, t.c.y), - ... tuple_(t.c.z, t.c.q), - ... ) - ... ) - >>> print(stmt) - {printsql}SELECT sum(t.value) AS sum_1 - FROM t GROUP BY GROUPING SETS((t.x, t.y), (t.z, t.q)) - -:ticket:`3429` - -.. _change_4075: - -Parameter helper for multi-valued INSERT with contextual default generator --------------------------------------------------------------------------- - -A default generation function, e.g. that described at -:ref:`context_default_functions`, can look at the current parameters relevant -to the statement via the :attr:`.DefaultExecutionContext.current_parameters` -attribute. However, in the case of a :class:`_expression.Insert` construct that specifies -multiple VALUES clauses via the :meth:`_expression.Insert.values` method, the user-defined -function is called multiple times, once for each parameter set, however there -was no way to know which subset of keys in -:attr:`.DefaultExecutionContext.current_parameters` apply to that column. A -new function :meth:`.DefaultExecutionContext.get_current_parameters` is added, -which includes a keyword argument -:paramref:`.DefaultExecutionContext.get_current_parameters.isolate_multiinsert_groups` -defaulting to ``True``, which performs the extra work of delivering a sub-dictionary of -:attr:`.DefaultExecutionContext.current_parameters` which has the names -localized to the current VALUES clause being processed:: - - - def mydefault(context): - return context.get_current_parameters()["counter"] + 12 - - - mytable = Table( - "mytable", - metadata_obj, - Column("counter", Integer), - Column("counter_plus_twelve", Integer, default=mydefault, onupdate=mydefault), - ) - - stmt = mytable.insert().values([{"counter": 5}, {"counter": 18}, {"counter": 20}]) - - conn.execute(stmt) - -:ticket:`4075` - -Key Behavioral Changes - ORM -============================ - -.. _change_3934: - -The after_rollback() Session event now emits before the expiration of objects ------------------------------------------------------------------------------ - -The :meth:`.SessionEvents.after_rollback` event now has access to the attribute -state of objects before their state has been expired (e.g. the "snapshot -removal"). This allows the event to be consistent with the behavior -of the :meth:`.SessionEvents.after_commit` event which also emits before the -"snapshot" has been removed:: - - sess = Session() - - user = sess.query(User).filter_by(name="x").first() - - - @event.listens_for(sess, "after_rollback") - def after_rollback(session): - # 'user.name' is now present, assuming it was already - # loaded. previously this would raise upon trying - # to emit a lazy load. - print("user name: %s" % user.name) - - - @event.listens_for(sess, "after_commit") - def after_commit(session): - # 'user.name' is present, assuming it was already - # loaded. this is the existing behavior. - print("user name: %s" % user.name) - - - if should_rollback: - sess.rollback() - else: - sess.commit() - -Note that the :class:`.Session` will still disallow SQL from being emitted -within this event; meaning that unloaded attributes will still not be -able to load within the scope of the event. - -:ticket:`3934` - -.. _change_3891: - -Fixed issue involving single-table inheritance with ``select_from()`` ---------------------------------------------------------------------- - -The :meth:`_query.Query.select_from` method now honors the single-table inheritance -column discriminator when generating SQL; previously, only the expressions -in the query column list would be taken into account. - -Supposing ``Manager`` is a subclass of ``Employee``. A query like the following:: - - sess.query(Manager.id) - -Would generate SQL as: - -.. sourcecode:: sql - - SELECT employee.id FROM employee WHERE employee.type IN ('manager') - -However, if ``Manager`` were only specified by :meth:`_query.Query.select_from` -and not in the columns list, the discriminator would not be added:: - - sess.query(func.count(1)).select_from(Manager) - -would generate: - -.. sourcecode:: sql - - SELECT count(1) FROM employee - -With the fix, :meth:`_query.Query.select_from` now works correctly and we get: - -.. sourcecode:: sql - - SELECT count(1) FROM employee WHERE employee.type IN ('manager') - -Applications that may have been working around this by supplying the -WHERE clause manually may need to be adjusted. - -:ticket:`3891` - -.. _change_3913: - -Previous collection is no longer mutated upon replacement ---------------------------------------------------------- - -The ORM emits events whenever the members of a mapped collection change. -In the case of assigning a collection to an attribute that would replace -the previous collection, a side effect of this was that the collection -being replaced would also be mutated, which is misleading and unnecessary:: - - >>> a1, a2, a3 = Address("a1"), Address("a2"), Address("a3") - >>> user.addresses = [a1, a2] - - >>> previous_collection = user.addresses - - # replace the collection with a new one - >>> user.addresses = [a2, a3] - - >>> previous_collection - [Address('a1'), Address('a2')] - -Above, prior to the change, the ``previous_collection`` would have had the -"a1" member removed, corresponding to the member that's no longer in the -new collection. - -:ticket:`3913` - -.. _change_3896_validates: - -A @validates method receives all values on bulk-collection set before comparison --------------------------------------------------------------------------------- - -A method that uses ``@validates`` will now receive all members of a collection -during a "bulk set" operation, before comparison is applied against the -existing collection. - -Given a mapping as:: - - class A(Base): - __tablename__ = "a" - id = Column(Integer, primary_key=True) - bs = relationship("B") - - @validates("bs") - def convert_dict_to_b(self, key, value): - return B(data=value["data"]) - - - class B(Base): - __tablename__ = "b" - id = Column(Integer, primary_key=True) - a_id = Column(ForeignKey("a.id")) - data = Column(String) - -Above, we could use the validator as follows, to convert from an incoming -dictionary to an instance of ``B`` upon collection append:: - - a1 = A() - a1.bs.append({"data": "b1"}) - -However, a collection assignment would fail, since the ORM would assume -incoming objects are already instances of ``B`` as it attempts to compare them -to the existing members of the collection, before doing collection appends -which actually invoke the validator. This would make it impossible for bulk -set operations to accommodate non-ORM objects like dictionaries that needed -up-front modification:: - - a1 = A() - a1.bs = [{"data": "b1"}] - -The new logic uses the new :meth:`.AttributeEvents.bulk_replace` event to ensure -that all values are sent to the ``@validates`` function up front. - -As part of this change, this means that validators will now receive -**all** members of a collection upon bulk set, not just the members that -are new. Supposing a simple validator such as:: - - class A(Base): - # ... - - @validates("bs") - def validate_b(self, key, value): - assert value.data is not None - return value - -Above, if we began with a collection as:: - - a1 = A() - - b1, b2 = B(data="one"), B(data="two") - a1.bs = [b1, b2] - -And then, replaced the collection with one that overlaps the first:: - - b3 = B(data="three") - a1.bs = [b2, b3] - -Previously, the second assignment would trigger the ``A.validate_b`` -method only once, for the ``b3`` object. The ``b2`` object would be seen -as being already present in the collection and not validated. With the new -behavior, both ``b2`` and ``b3`` are passed to ``A.validate_b`` before passing -onto the collection. It is thus important that validation methods employ -idempotent behavior to suit such a case. - -.. seealso:: - - :ref:`change_3896_event` - -:ticket:`3896` - -.. _change_3753: - -Use flag_dirty() to mark an object as "dirty" without any attribute changing ----------------------------------------------------------------------------- - -An exception is now raised if the :func:`.attributes.flag_modified` function -is used to mark an attribute as modified that isn't actually loaded:: - - a1 = A(data="adf") - s.add(a1) - - s.flush() - - # expire, similarly as though we said s.commit() - s.expire(a1, "data") - - # will raise InvalidRequestError - attributes.flag_modified(a1, "data") - -This because the flush process will most likely fail in any case if the -attribute remains un-present by the time flush occurs. To mark an object -as "modified" without referring to any attribute specifically, so that it -is considered within the flush process for the purpose of custom event handlers -such as :meth:`.SessionEvents.before_flush`, use the new -:func:`.attributes.flag_dirty` function:: - - from sqlalchemy.orm import attributes - - attributes.flag_dirty(a1) - -:ticket:`3753` - -.. _change_3796: - -"scope" keyword removed from scoped_session -------------------------------------------- - -A very old and undocumented keyword argument ``scope`` has been removed:: - - from sqlalchemy.orm import scoped_session - - Session = scoped_session(sessionmaker()) - - session = Session(scope=None) - -The purpose of this keyword was an attempt to allow for variable -"scopes", where ``None`` indicated "no scope" and would therefore return -a new :class:`.Session`. The keyword has never been documented and will -now raise ``TypeError`` if encountered. It is not anticipated that this -keyword is in use, however if users report issues related to this during -beta testing, it can be restored with a deprecation. - -:ticket:`3796` - -.. _change_3471: - -Refinements to post_update in conjunction with onupdate -------------------------------------------------------- - -A relationship that uses the :paramref:`_orm.relationship.post_update` feature -will now interact better with a column that has an :paramref:`_schema.Column.onupdate` -value set. If an object is inserted with an explicit value for the column, -it is re-stated during the UPDATE so that the "onupdate" rule does not -overwrite it:: - - class A(Base): - __tablename__ = "a" - id = Column(Integer, primary_key=True) - favorite_b_id = Column(ForeignKey("b.id", name="favorite_b_fk")) - bs = relationship("B", primaryjoin="A.id == B.a_id") - favorite_b = relationship( - "B", primaryjoin="A.favorite_b_id == B.id", post_update=True - ) - updated = Column(Integer, onupdate=my_onupdate_function) - - - class B(Base): - __tablename__ = "b" - id = Column(Integer, primary_key=True) - a_id = Column(ForeignKey("a.id", name="a_fk")) - - - a1 = A() - b1 = B() - - a1.bs.append(b1) - a1.favorite_b = b1 - a1.updated = 5 - s.add(a1) - s.flush() - -Above, the previous behavior would be that an UPDATE would emit after the -INSERT, thus triggering the "onupdate" and overwriting the value -"5". The SQL now looks like: - -.. sourcecode:: sql - - INSERT INTO a (favorite_b_id, updated) VALUES (?, ?) - (None, 5) - INSERT INTO b (a_id) VALUES (?) - (1,) - UPDATE a SET favorite_b_id=?, updated=? WHERE a.id = ? - (1, 5, 1) - -Additionally, if the value of "updated" is *not* set, then we correctly -get back the newly generated value on ``a1.updated``; previously, the logic -that refreshes or expires the attribute to allow the generated value -to be present would not fire off for a post-update. The -:meth:`.InstanceEvents.refresh_flush` event is also emitted when a refresh -within flush occurs in this case. - -:ticket:`3471` - -:ticket:`3472` - -.. _change_3496: - -post_update integrates with ORM versioning ------------------------------------------- - -The post_update feature, documented at :ref:`post_update`, involves that an -UPDATE statement is emitted in response to changes to a particular -relationship-bound foreign key, in addition to the INSERT/UPDATE/DELETE that -would normally be emitted for the target row. This UPDATE statement -now participates in the versioning feature, documented at -:ref:`mapper_version_counter`. - -Given a mapping:: - - class Node(Base): - __tablename__ = "node" - id = Column(Integer, primary_key=True) - version_id = Column(Integer, default=0) - parent_id = Column(ForeignKey("node.id")) - favorite_node_id = Column(ForeignKey("node.id")) - - nodes = relationship("Node", primaryjoin=remote(parent_id) == id) - favorite_node = relationship( - "Node", primaryjoin=favorite_node_id == remote(id), post_update=True - ) - - __mapper_args__ = {"version_id_col": version_id} - -An UPDATE of a node that associates another node as "favorite" will -now increment the version counter as well as match the current version:: - - node = Node() - session.add(node) - session.commit() # node is now version #1 - - node = session.query(Node).get(node.id) - node.favorite_node = Node() - session.commit() # node is now version #2 - -Note that this means an object that receives an UPDATE in response to -other attributes changing, and a second UPDATE due to a post_update -relationship change, will now receive -**two version counter updates for one flush**. However, if the object -is subject to an INSERT within the current flush, the version counter -**will not** be incremented an additional time, unless a server-side -versioning scheme is in place. - -The reason post_update emits an UPDATE even for an UPDATE is now discussed at -:ref:`faq_post_update_update`. - -.. seealso:: - - :ref:`post_update` - - :ref:`faq_post_update_update` - - -:ticket:`3496` - -Key Behavioral Changes - Core -============================= - -.. _change_4063: - -The typing behavior of custom operators has been made consistent ----------------------------------------------------------------- - -User defined operators can be made on the fly using the -:meth:`.Operators.op` function. Previously, the typing behavior of -an expression against such an operator was inconsistent and also not -controllable. - -Whereas in 1.1, an expression such as the following would produce -a result with no return type (assume ``-%>`` is some special operator -supported by the database):: - - >>> column("x", types.DateTime).op("-%>")(None).type - NullType() - -Other types would use the default behavior of using the left-hand type -as the return type:: - - >>> column("x", types.String(50)).op("-%>")(None).type - String(length=50) - -These behaviors were mostly by accident, so the behavior has been made -consistent with the second form, that is the default return type is the -same as the left-hand expression:: - - >>> column("x", types.DateTime).op("-%>")(None).type - DateTime() - -As most user-defined operators tend to be "comparison" operators, often -one of the many special operators defined by PostgreSQL, the -:paramref:`.Operators.op.is_comparison` flag has been repaired to follow -its documented behavior of allowing the return type to be :class:`.Boolean` -in all cases, including for :class:`_types.ARRAY` and :class:`_types.JSON`:: - - >>> column("x", types.String(50)).op("-%>", is_comparison=True)(None).type - Boolean() - >>> column("x", types.ARRAY(types.Integer)).op("-%>", is_comparison=True)(None).type - Boolean() - >>> column("x", types.JSON()).op("-%>", is_comparison=True)(None).type - Boolean() - -To assist with boolean comparison operators, a new shorthand method -:meth:`.Operators.bool_op` has been added. This method should be preferred -for on-the-fly boolean operators: - -.. sourcecode:: pycon+sql - - >>> print(column("x", types.Integer).bool_op("-%>")(5)) - {printsql}x -%> :x_1 - - -.. _change_3740: - -Percent signs in literal_column() now conditionally escaped ------------------------------------------------------------ - -The :obj:`_expression.literal_column` construct now escapes percent sign characters -conditionally, based on whether or not the DBAPI in use makes use of a -percent-sign-sensitive paramstyle or not (e.g. 'format' or 'pyformat'). - -Previously, it was not possible to produce a :obj:`_expression.literal_column` -construct that stated a single percent sign: - -.. sourcecode:: pycon+sql - - >>> from sqlalchemy import literal_column - >>> print(literal_column("some%symbol")) - {printsql}some%%symbol - -The percent sign is now unaffected for dialects that are not set to -use the 'format' or 'pyformat' paramstyles; dialects such most MySQL -dialects which do state one of these paramstyles will continue to escape -as is appropriate: - -.. sourcecode:: pycon+sql - - >>> from sqlalchemy import literal_column - >>> print(literal_column("some%symbol")) - {printsql}some%symbol{stop} - >>> from sqlalchemy.dialects import mysql - >>> print(literal_column("some%symbol").compile(dialect=mysql.dialect())) - {printsql}some%%symbol{stop} - -As part of this change, the doubling that has been present when using -operators like :meth:`.ColumnOperators.contains`, -:meth:`.ColumnOperators.startswith` and :meth:`.ColumnOperators.endswith` -is also refined to only occur when appropriate. - -:ticket:`3740` - - -.. _change_3785: - -The column-level COLLATE keyword now quotes the collation name --------------------------------------------------------------- - -A bug in the :func:`_expression.collate` and :meth:`.ColumnOperators.collate` -functions, used to supply ad-hoc column collations at the statement level, -is fixed, where a case sensitive name would not be quoted:: - - stmt = select([mytable.c.x, mytable.c.y]).order_by( - mytable.c.somecolumn.collate("fr_FR") - ) - -now renders: - -.. sourcecode:: sql - - SELECT mytable.x, mytable.y, - FROM mytable ORDER BY mytable.somecolumn COLLATE "fr_FR" - -Previously, the case sensitive name `"fr_FR"` would not be quoted. Currently, -manual quoting of the "fr_FR" name is **not** detected, so applications that -are manually quoting the identifier should be adjusted. Note that this change -does not impact the use of collations at the type level (e.g. specified -on the datatype like :class:`.String` at the table level), where quoting -is already applied. - -:ticket:`3785` - -Dialect Improvements and Changes - PostgreSQL -============================================= - -.. _change_4109: - -Support for Batch Mode / Fast Execution Helpers ------------------------------------------------- - -The psycopg2 ``cursor.executemany()`` method has been identified as performing -poorly, particularly with INSERT statements. To alleviate this, psycopg2 -has added `Fast Execution Helpers `_ -which rework statements into fewer server round trips by sending multiple -DML statements in batch. SQLAlchemy 1.2 now includes support for these -helpers to be used transparently whenever the :class:`_engine.Engine` makes use -of ``cursor.executemany()`` to invoke a statement against multiple parameter -sets. The feature is off by default and can be enabled using the -``use_batch_mode`` argument on :func:`_sa.create_engine`:: - - engine = create_engine( - "postgresql+psycopg2://scott:tiger@host/dbname", use_batch_mode=True - ) - -The feature is considered to be experimental for the moment but may become -on by default in a future release. - -.. seealso:: - - :ref:`psycopg2_batch_mode` - -:ticket:`4109` - -.. _change_3959: - -Support for fields specification in INTERVAL, including full reflection ------------------------------------------------------------------------ - -The "fields" specifier in PostgreSQL's INTERVAL datatype allows specification -of which fields of the interval to store, including such values as "YEAR", -"MONTH", "YEAR TO MONTH", etc. The :class:`_postgresql.INTERVAL` datatype -now allows these values to be specified:: - - from sqlalchemy.dialects.postgresql import INTERVAL - - Table("my_table", metadata, Column("some_interval", INTERVAL(fields="DAY TO SECOND"))) - -Additionally, all INTERVAL datatypes can now be reflected independently -of the "fields" specifier present; the "fields" parameter in the datatype -itself will also be present:: - - >>> inspect(engine).get_columns("my_table") - [{'comment': None, - 'name': u'some_interval', 'nullable': True, - 'default': None, 'autoincrement': False, - 'type': INTERVAL(fields=u'day to second')}] - -:ticket:`3959` - -Dialect Improvements and Changes - MySQL -======================================== - -.. _change_4009: - -Support for INSERT..ON DUPLICATE KEY UPDATE -------------------------------------------- - -The ``ON DUPLICATE KEY UPDATE`` clause of ``INSERT`` supported by MySQL -is now supported using a MySQL-specific version of the -:class:`_expression.Insert` object, via :func:`sqlalchemy.dialects.mysql.dml.insert`. -This :class:`_expression.Insert` subclass adds a new method -:meth:`~.mysql.dml.Insert.on_duplicate_key_update` that implements MySQL's syntax:: - - from sqlalchemy.dialects.mysql import insert - - insert_stmt = insert(my_table).values(id="some_id", data="some data to insert") - - on_conflict_stmt = insert_stmt.on_duplicate_key_update( - data=insert_stmt.inserted.data, status="U" - ) - - conn.execute(on_conflict_stmt) - -The above will render: - -.. sourcecode:: sql - - INSERT INTO my_table (id, data) - VALUES (:id, :data) - ON DUPLICATE KEY UPDATE data=VALUES(data), status=:status_1 - -.. seealso:: - - :ref:`mysql_insert_on_duplicate_key_update` - -:ticket:`4009` - - -Dialect Improvements and Changes - Oracle -========================================= - -.. _change_cxoracle_12: - -Major Refactor to cx_Oracle Dialect, Typing System --------------------------------------------------- - -With the introduction of the 6.x series of the cx_Oracle DBAPI, SQLAlchemy's -cx_Oracle dialect has been reworked and simplified to take advantage of recent -improvements in cx_Oracle as well as dropping support for patterns that were -more relevant before the 5.x series of cx_Oracle. - -* The minimum cx_Oracle version supported is now 5.1.3; 5.3 or the most recent - 6.x series are recommended. - -* The handling of datatypes has been refactored. The ``cursor.setinputsizes()`` - method is no longer used for any datatype except LOB types, per advice from - cx_Oracle's developers. As a result, the parameters ``auto_setinputsizes`` - and ``exclude_setinputsizes`` are deprecated and no longer have any effect. - -* The ``coerce_to_decimal`` flag, when set to False to indicate that coercion - of numeric types with precision and scale to ``Decimal`` should not occur, - only impacts untyped (e.g. plain string with no :class:`.TypeEngine` objects) - statements. A Core expression that includes a :class:`.Numeric` type or - subtype will now follow the decimal coercion rules of that type. - -* The "two phase" transaction support in the dialect, already dropped for the - 6.x series of cx_Oracle, has now been removed entirely as this feature has - never worked correctly and is unlikely to have been in production use. - As a result, the ``allow_twophase`` dialect flag is deprecated and also has no - effect. - -* Fixed a bug involving the column keys present with RETURNING. Given - a statement as follows:: - - result = conn.execute(table.insert().values(x=5).returning(table.c.a, table.c.b)) - - Previously, the keys in each row of the result would be ``ret_0`` and ``ret_1``, - which are identifiers internal to the cx_Oracle RETURNING implementation. - The keys will now be ``a`` and ``b`` as is expected for other dialects. - -* cx_Oracle's LOB datatype represents return values as a ``cx_Oracle.LOB`` - object, which is a cursor-associated proxy that returns the ultimate data - value via a ``.read()`` method. Historically, if more rows were read before - these LOB objects were consumed (specifically, more rows than the value of - cursor.arraysize which causes a new batch of rows to be read), these LOB - objects would raise the error "LOB variable no longer valid after subsequent - fetch". SQLAlchemy worked around this by both automatically calling - ``.read()`` upon these LOBs within its typing system, as well as using a - special ``BufferedColumnResultSet`` which would ensure this data was buffered - in case a call like ``cursor.fetchmany()`` or ``cursor.fetchall()`` were - used. - - The dialect now makes use of a cx_Oracle outputtypehandler to handle these - ``.read()`` calls, so that they are always called up front regardless of how - many rows are being fetched, so that this error can no longer occur. As a - result, the use of the ``BufferedColumnResultSet``, as well as some other - internals to the Core ``ResultSet`` that were specific to this use case, - have been removed. The type objects are also simplified as they no longer - need to process a binary column result. - - Additionally, cx_Oracle 6.x has removed the conditions under which this error - occurs in any case, so the error is no longer possible. The error - can occur on SQLAlchemy in the case that the seldom (if ever) used - ``auto_convert_lobs=False`` option is in use, in conjunction with the - previous 5.x series of cx_Oracle, and more rows are read before the LOB - objects can be consumed. Upgrading to cx_Oracle 6.x will resolve that issue. - -.. _change_4003: - -Oracle Unique, Check constraints now reflected ----------------------------------------------- - -UNIQUE and CHECK constraints now reflect via -:meth:`_reflection.Inspector.get_unique_constraints` and -:meth:`_reflection.Inspector.get_check_constraints`. A :class:`_schema.Table` object that's -reflected will now include :class:`.CheckConstraint` objects as well. -See the notes at :ref:`oracle_constraint_reflection` for information -on behavioral quirks here, including that most :class:`_schema.Table` objects -will still not include any :class:`.UniqueConstraint` objects as these -usually represent via :class:`.Index`. - -.. seealso:: - - :ref:`oracle_constraint_reflection` - - -:ticket:`4003` - -.. _change_3276: - -Oracle foreign key constraint names are now "name normalized" -------------------------------------------------------------- - -The names of foreign key constraints as delivered to a -:class:`_schema.ForeignKeyConstraint` object during table reflection as well as -within the :meth:`_reflection.Inspector.get_foreign_keys` method will now be -"name normalized", that is, expressed as lower case for a case insensitive -name, rather than the raw UPPERCASE format that Oracle uses:: - - >>> insp.get_indexes("addresses") - [{'unique': False, 'column_names': [u'user_id'], - 'name': u'address_idx', 'dialect_options': {}}] - - >>> insp.get_pk_constraint("addresses") - {'name': u'pk_cons', 'constrained_columns': [u'id']} - - >>> insp.get_foreign_keys("addresses") - [{'referred_table': u'users', 'referred_columns': [u'id'], - 'referred_schema': None, 'name': u'user_id_fk', - 'constrained_columns': [u'user_id']}] - -Previously, the foreign keys result would look like:: - - [ - { - "referred_table": "users", - "referred_columns": ["id"], - "referred_schema": None, - "name": "USER_ID_FK", - "constrained_columns": ["user_id"], - } - ] - -Where the above could create problems particularly with Alembic autogenerate. - -:ticket:`3276` - - -Dialect Improvements and Changes - SQL Server -============================================= - -.. _change_2626: - -SQL Server schema names with embedded dots supported ----------------------------------------------------- - -The SQL Server dialect has a behavior such that a schema name with a dot inside -of it is assumed to be a "database"."owner" identifier pair, which is -necessarily split up into these separate components during table and component -reflection operations, as well as when rendering quoting for the schema name so -that the two symbols are quoted separately. The schema argument can -now be passed using brackets to manually specify where this split -occurs, allowing database and/or owner names that themselves contain one -or more dots:: - - Table("some_table", metadata, Column("q", String(50)), schema="[MyDataBase.dbo]") - -The above table will consider the "owner" to be ``MyDataBase.dbo``, which -will also be quoted upon render, and the "database" as None. To individually -refer to database name and owner, use two pairs of brackets:: - - Table( - "some_table", - metadata, - Column("q", String(50)), - schema="[MyDataBase.SomeDB].[MyDB.owner]", - ) - -Additionally, the :class:`.quoted_name` construct is now honored when -passed to "schema" by the SQL Server dialect; the given symbol will -not be split on the dot if the quote flag is True and will be interpreted -as the "owner". - -.. seealso:: - - :ref:`multipart_schema_names` - -:ticket:`2626` - -AUTOCOMMIT isolation level support ----------------------------------- - -Both the PyODBC and pymssql dialects now support the "AUTOCOMMIT" isolation -level as set by :meth:`_engine.Connection.execution_options` which will establish -the correct flags on the DBAPI connection object. diff --git a/doc/build/changelog/unreleased_11/README.txt b/doc/build/changelog/unreleased_11/README.txt deleted file mode 100644 index 068264c720..0000000000 --- a/doc/build/changelog/unreleased_11/README.txt +++ /dev/null @@ -1,8 +0,0 @@ -See doc/build/changelog/README.txt for -information on the doc build system. - - -DO NOT REMOVE THIS FILE!!!! THIS DIRECTORY MUST BE PRESENT -FOR DOCS TO BUILD. - - diff --git a/doc/build/changelog/unreleased_12/README.txt b/doc/build/changelog/unreleased_12/README.txt deleted file mode 100644 index 068264c720..0000000000 --- a/doc/build/changelog/unreleased_12/README.txt +++ /dev/null @@ -1,8 +0,0 @@ -See doc/build/changelog/README.txt for -information on the doc build system. - - -DO NOT REMOVE THIS FILE!!!! THIS DIRECTORY MUST BE PRESENT -FOR DOCS TO BUILD. - - diff --git a/doc/build/changelog/unreleased_13/6135.rst b/doc/build/changelog/unreleased_13/6135.rst deleted file mode 100644 index 942b04edf9..0000000000 --- a/doc/build/changelog/unreleased_13/6135.rst +++ /dev/null @@ -1,10 +0,0 @@ -.. change:: - :tags: schema, bug - :tickets: 6135 - :versions: 1.4.6 - - The :class:`_schema.Table` object now raises an informative error message if - it is instantiated without passing at least the :paramref:`_schema.Table.name` - and :paramref:`_schema.Table.metadata` arguments positionally. Previously, if - these were passed as keyword arguments, the object would silently fail to - initialize correctly. diff --git a/doc/build/changelog/unreleased_13/6182.rst b/doc/build/changelog/unreleased_13/6182.rst deleted file mode 100644 index 3228b4f2b4..0000000000 --- a/doc/build/changelog/unreleased_13/6182.rst +++ /dev/null @@ -1,13 +0,0 @@ -.. change:: - :tags: bug, postgresql, regression - :tickets: 6182 - :versions: 1.4.5 - - Fixed regression caused by :ticket:`6023` where the PostgreSQL cast - operator applied to elements within an :class:`_types.ARRAY` when using - psycopg2 would fail to use the correct type in the case that the datatype - were also embedded within an instance of the :class:`_types.Variant` - adapter. - - Additionally, repairs support for the correct CREATE TYPE to be emitted - when using a ``Variant(ARRAY(some_schema_type))``. diff --git a/doc/build/changelog/unreleased_13/6392.rst b/doc/build/changelog/unreleased_13/6392.rst deleted file mode 100644 index e7cda565a5..0000000000 --- a/doc/build/changelog/unreleased_13/6392.rst +++ /dev/null @@ -1,9 +0,0 @@ -.. change:: - :tags: bug, orm - :tickets: 6392 - :versions: 1.4.12 - - Fixed issue in :meth:`_orm.Session.bulk_save_objects` when used with persistent - objects which would fail to track the primary key of mappings where the - column name of the primary key were different than the attribute name. - diff --git a/doc/build/changelog/unreleased_13/6589.rst b/doc/build/changelog/unreleased_13/6589.rst deleted file mode 100644 index b6f5fc6063..0000000000 --- a/doc/build/changelog/unreleased_13/6589.rst +++ /dev/null @@ -1,7 +0,0 @@ -.. change:: - :tags: bug, sqlite - :tickets: 6589 - :versions: 1.4.18 - - Add note regarding encryption-related pragmas for pysqlcipher passed in the - url. diff --git a/doc/build/changelog/unreleased_13/7115.rst b/doc/build/changelog/unreleased_13/7115.rst deleted file mode 100644 index 1f2c7fcf86..0000000000 --- a/doc/build/changelog/unreleased_13/7115.rst +++ /dev/null @@ -1,16 +0,0 @@ -.. change:: - :tags: bug, mysql, mariadb - :tickets: 7115, 7136 - :versions: 1.4.26 - - Fixes to accommodate for the MariaDB 10.6 series, including backwards - incompatible changes in both the mariadb-connector Python driver (supported - on SQLAlchemy 1.4 only) as well as the native 10.6 client libraries that - are used automatically by the mysqlclient DBAPI (applies to both 1.3 and - 1.4). The "utf8mb3" encoding symbol is now reported by these client - libraries when the encoding is stated as "utf8", leading to lookup and - encoding errors within the MySQL dialect that does not expect this symbol. - Updates to both the MySQL base library to accommodate for this utf8mb3 - symbol being reported as well as to the test suite. Thanks to Georg Richter - for support. - diff --git a/doc/build/changelog/unreleased_13/README.txt b/doc/build/changelog/unreleased_13/README.txt deleted file mode 100644 index 068264c720..0000000000 --- a/doc/build/changelog/unreleased_13/README.txt +++ /dev/null @@ -1,8 +0,0 @@ -See doc/build/changelog/README.txt for -information on the doc build system. - - -DO NOT REMOVE THIS FILE!!!! THIS DIRECTORY MUST BE PRESENT -FOR DOCS TO BUILD. - - diff --git a/doc/build/core/api_basics.rst b/doc/build/core/api_basics.rst index a2b7060d3a..640b4fd446 100644 --- a/doc/build/core/api_basics.rst +++ b/doc/build/core/api_basics.rst @@ -6,6 +6,5 @@ Core API Basics :maxdepth: 2 event - inspection exceptions internals diff --git a/doc/build/core/defaults.rst b/doc/build/core/defaults.rst deleted file mode 100644 index ef5ad20815..0000000000 --- a/doc/build/core/defaults.rst +++ /dev/null @@ -1,838 +0,0 @@ -.. currentmodule:: sqlalchemy.schema - -.. _metadata_defaults_toplevel: - -.. _metadata_defaults: - -Column INSERT/UPDATE Defaults -============================= - -Column INSERT and UPDATE defaults refer to functions that create a **default -value** for a particular column in a row as an INSERT or UPDATE statement is -proceeding against that row, in the case where **no value was provided to the -INSERT or UPDATE statement for that column**. That is, if a table has a column -called "timestamp", and an INSERT statement proceeds which does not include a -value for this column, an INSERT default would create a new value, such as -the current time, that is used as the value to be INSERTed into the "timestamp" -column. If the statement *does* include a value for this column, then the -default does *not* take place. - -Column defaults can be server-side functions or constant values which are -defined in the database along with the schema in :term:`DDL`, or as SQL -expressions which are rendered directly within an INSERT or UPDATE statement -emitted by SQLAlchemy; they may also be client-side Python functions or -constant values which are invoked by SQLAlchemy before data is passed to the -database. - -.. note:: - - A column default handler should not be confused with a construct that - intercepts and modifies incoming values for INSERT and UPDATE statements - which *are* provided to the statement as it is invoked. This is known - as :term:`data marshalling`, where a column value is modified in some way - by the application before being sent to the database. SQLAlchemy provides - a few means of achieving this which include using :ref:`custom datatypes - `, :ref:`SQL execution events ` and - in the ORM :ref:`custom validators ` as well as - :ref:`attribute events `. Column defaults are only - invoked when there is **no value present** for a column in a SQL - :term:`DML` statement. - - -SQLAlchemy provides an array of features regarding default generation -functions which take place for non-present values during INSERT and UPDATE -statements. Options include: - -* Scalar values used as defaults during INSERT and UPDATE operations -* Python functions which execute upon INSERT and UPDATE operations -* SQL expressions which are embedded in INSERT statements (or in some cases execute beforehand) -* SQL expressions which are embedded in UPDATE statements -* Server side default values used during INSERT -* Markers for server-side triggers used during UPDATE - -The general rule for all insert/update defaults is that they only take effect -if no value for a particular column is passed as an ``execute()`` parameter; -otherwise, the given value is used. - -Scalar Defaults ---------------- - -The simplest kind of default is a scalar value used as the default value of a column:: - - Table("mytable", metadata_obj, Column("somecolumn", Integer, default=12)) - -Above, the value "12" will be bound as the column value during an INSERT if no -other value is supplied. - -A scalar value may also be associated with an UPDATE statement, though this is -not very common (as UPDATE statements are usually looking for dynamic -defaults):: - - Table("mytable", metadata_obj, Column("somecolumn", Integer, onupdate=25)) - -Python-Executed Functions -------------------------- - -The :paramref:`_schema.Column.default` and :paramref:`_schema.Column.onupdate` keyword arguments also accept Python -functions. These functions are invoked at the time of insert or update if no -other value for that column is supplied, and the value returned is used for -the column's value. Below illustrates a crude "sequence" that assigns an -incrementing counter to a primary key column:: - - # a function which counts upwards - i = 0 - - - def mydefault(): - global i - i += 1 - return i - - - t = Table( - "mytable", - metadata_obj, - Column("id", Integer, primary_key=True, default=mydefault), - ) - -It should be noted that for real "incrementing sequence" behavior, the -built-in capabilities of the database should normally be used, which may -include sequence objects or other autoincrementing capabilities. For primary -key columns, SQLAlchemy will in most cases use these capabilities -automatically. See the API documentation for -:class:`~sqlalchemy.schema.Column` including the :paramref:`_schema.Column.autoincrement` flag, as -well as the section on :class:`~sqlalchemy.schema.Sequence` later in this -chapter for background on standard primary key generation techniques. - -To illustrate onupdate, we assign the Python ``datetime`` function ``now`` to -the :paramref:`_schema.Column.onupdate` attribute:: - - import datetime - - t = Table( - "mytable", - metadata_obj, - Column("id", Integer, primary_key=True), - # define 'last_updated' to be populated with datetime.now() - Column("last_updated", DateTime, onupdate=datetime.datetime.now), - ) - -When an update statement executes and no value is passed for ``last_updated``, -the ``datetime.datetime.now()`` Python function is executed and its return -value used as the value for ``last_updated``. Notice that we provide ``now`` -as the function itself without calling it (i.e. there are no parenthesis -following) - SQLAlchemy will execute the function at the time the statement -executes. - -.. _context_default_functions: - -Context-Sensitive Default Functions -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The Python functions used by :paramref:`_schema.Column.default` and -:paramref:`_schema.Column.onupdate` may also make use of the current statement's -context in order to determine a value. The `context` of a statement is an -internal SQLAlchemy object which contains all information about the statement -being executed, including its source expression, the parameters associated with -it and the cursor. The typical use case for this context with regards to -default generation is to have access to the other values being inserted or -updated on the row. To access the context, provide a function that accepts a -single ``context`` argument:: - - def mydefault(context): - return context.get_current_parameters()["counter"] + 12 - - - t = Table( - "mytable", - metadata_obj, - Column("counter", Integer), - Column("counter_plus_twelve", Integer, default=mydefault, onupdate=mydefault), - ) - -The above default generation function is applied so that it will execute for -all INSERT and UPDATE statements where a value for ``counter_plus_twelve`` was -otherwise not provided, and the value will be that of whatever value is present -in the execution for the ``counter`` column, plus the number 12. - -For a single statement that is being executed using "executemany" style, e.g. -with multiple parameter sets passed to :meth:`_engine.Connection.execute`, the -user-defined function is called once for each set of parameters. For the use case of -a multi-valued :class:`_expression.Insert` construct (e.g. with more than one VALUES -clause set up via the :meth:`_expression.Insert.values` method), the user-defined function -is also called once for each set of parameters. - -When the function is invoked, the special method -:meth:`.DefaultExecutionContext.get_current_parameters` is available from -the context object (an subclass of :class:`.DefaultExecutionContext`). This -method returns a dictionary of column-key to values that represents the -full set of values for the INSERT or UPDATE statement. In the case of a -multi-valued INSERT construct, the subset of parameters that corresponds to -the individual VALUES clause is isolated from the full parameter dictionary -and returned alone. - -.. versionadded:: 1.2 - - Added :meth:`.DefaultExecutionContext.get_current_parameters` method, - which improves upon the still-present - :attr:`.DefaultExecutionContext.current_parameters` attribute - by offering the service of organizing multiple VALUES clauses - into individual parameter dictionaries. - -.. _defaults_client_invoked_sql: - -Client-Invoked SQL Expressions ------------------------------- - -The :paramref:`_schema.Column.default` and :paramref:`_schema.Column.onupdate` keywords may -also be passed SQL expressions, which are in most cases rendered inline within the -INSERT or UPDATE statement:: - - t = Table( - "mytable", - metadata_obj, - Column("id", Integer, primary_key=True), - # define 'create_date' to default to now() - Column("create_date", DateTime, default=func.now()), - # define 'key' to pull its default from the 'keyvalues' table - Column( - "key", - String(20), - default=select(keyvalues.c.key).where(keyvalues.c.type="type1"), - ), - # define 'last_modified' to use the current_timestamp SQL function on update - Column("last_modified", DateTime, onupdate=func.utc_timestamp()), - ) - -Above, the ``create_date`` column will be populated with the result of the -``now()`` SQL function (which, depending on backend, compiles into ``NOW()`` -or ``CURRENT_TIMESTAMP`` in most cases) during an INSERT statement, and the -``key`` column with the result of a SELECT subquery from another table. The -``last_modified`` column will be populated with the value of -the SQL ``UTC_TIMESTAMP()`` MySQL function when an UPDATE statement is -emitted for this table. - -.. note:: - - When using SQL functions with the :attr:`.func` construct, we "call" the - named function, e.g. with parenthesis as in ``func.now()``. This differs - from when we specify a Python callable as a default such as - ``datetime.datetime``, where we pass the function itself, but we don't - invoke it ourselves. In the case of a SQL function, invoking - ``func.now()`` returns the SQL expression object that will render the - "NOW" function into the SQL being emitted. - -Default and update SQL expressions specified by :paramref:`_schema.Column.default` and -:paramref:`_schema.Column.onupdate` are invoked explicitly by SQLAlchemy when an -INSERT or UPDATE statement occurs, typically rendered inline within the DML -statement except in certain cases listed below. This is different than a -"server side" default, which is part of the table's DDL definition, e.g. as -part of the "CREATE TABLE" statement, which are likely more common. For -server side defaults, see the next section :ref:`server_defaults`. - -When a SQL expression indicated by :paramref:`_schema.Column.default` is used with -primary key columns, there are some cases where SQLAlchemy must "pre-execute" -the default generation SQL function, meaning it is invoked in a separate SELECT -statement, and the resulting value is passed as a parameter to the INSERT. -This only occurs for primary key columns for an INSERT statement that is being -asked to return this primary key value, where RETURNING or ``cursor.lastrowid`` -may not be used. An :class:`_expression.Insert` construct that specifies the -:paramref:`~.expression.insert.inline` flag will always render default expressions -inline. - -When the statement is executed with a single set of parameters (that is, it is -not an "executemany" style execution), the returned -:class:`~sqlalchemy.engine.CursorResult` will contain a collection accessible -via :meth:`_engine.CursorResult.postfetch_cols` which contains a list of all -:class:`~sqlalchemy.schema.Column` objects which had an inline-executed -default. Similarly, all parameters which were bound to the statement, including -all Python and SQL expressions which were pre-executed, are present in the -:meth:`_engine.CursorResult.last_inserted_params` or -:meth:`_engine.CursorResult.last_updated_params` collections on -:class:`~sqlalchemy.engine.CursorResult`. The -:attr:`_engine.CursorResult.inserted_primary_key` collection contains a list of primary -key values for the row inserted (a list so that single-column and -composite-column primary keys are represented in the same format). - -.. _server_defaults: - -Server-invoked DDL-Explicit Default Expressions ------------------------------------------------ - -A variant on the SQL expression default is the :paramref:`_schema.Column.server_default`, which gets -placed in the CREATE TABLE statement during a :meth:`_schema.Table.create` operation: - -.. sourcecode:: python+sql - - t = Table( - "test", - metadata_obj, - Column("abc", String(20), server_default="abc"), - Column("created_at", DateTime, server_default=func.sysdate()), - Column("index_value", Integer, server_default=text("0")), - ) - -A create call for the above table will produce: - -.. sourcecode:: sql - - CREATE TABLE test ( - abc varchar(20) default 'abc', - created_at datetime default sysdate, - index_value integer default 0 - ) - -The above example illustrates the two typical use cases for :paramref:`_schema.Column.server_default`, -that of the SQL function (SYSDATE in the above example) as well as a server-side constant -value (the integer "0" in the above example). It is advisable to use the -:func:`_expression.text` construct for any literal SQL values as opposed to passing the -raw value, as SQLAlchemy does not typically perform any quoting or escaping on -these values. - -Like client-generated expressions, :paramref:`_schema.Column.server_default` can accommodate -SQL expressions in general, however it is expected that these will usually be simple -functions and expressions, and not the more complex cases like an embedded SELECT. - - -.. _triggered_columns: - -Marking Implicitly Generated Values, timestamps, and Triggered Columns ----------------------------------------------------------------------- - -Columns which generate a new value on INSERT or UPDATE based on other -server-side database mechanisms, such as database-specific auto-generating -behaviors such as seen with TIMESTAMP columns on some platforms, as well as -custom triggers that invoke upon INSERT or UPDATE to generate a new value, -may be called out using :class:`.FetchedValue` as a marker:: - - from sqlalchemy.schema import FetchedValue - - t = Table( - "test", - metadata_obj, - Column("id", Integer, primary_key=True), - Column("abc", TIMESTAMP, server_default=FetchedValue()), - Column("def", String(20), server_onupdate=FetchedValue()), - ) - -The :class:`.FetchedValue` indicator does not affect the rendered DDL for the -CREATE TABLE. Instead, it marks the column as one that will have a new value -populated by the database during the process of an INSERT or UPDATE statement, -and for supporting databases may be used to indicate that the column should be -part of a RETURNING or OUTPUT clause for the statement. Tools such as the -SQLAlchemy ORM then make use of this marker in order to know how to get at the -value of the column after such an operation. In particular, the -:meth:`.ValuesBase.return_defaults` method can be used with an :class:`_expression.Insert` -or :class:`_expression.Update` construct to indicate that these values should be -returned. - -For details on using :class:`.FetchedValue` with the ORM, see -:ref:`orm_server_defaults`. - -.. warning:: The :paramref:`_schema.Column.server_onupdate` directive - **does not** currently produce MySQL's - "ON UPDATE CURRENT_TIMESTAMP()" clause. See - :ref:`mysql_timestamp_onupdate` for background on how to produce - this clause. - - -.. seealso:: - - :ref:`orm_server_defaults` - -.. _defaults_sequences: - -Defining Sequences ------------------- - -SQLAlchemy represents database sequences using the -:class:`~sqlalchemy.schema.Sequence` object, which is considered to be a -special case of "column default". It only has an effect on databases which have -explicit support for sequences, which among SQLAlchemy's included dialects -includes PostgreSQL, Oracle, MS SQL Server, and MariaDB. The -:class:`~sqlalchemy.schema.Sequence` object is otherwise ignored. - -.. tip:: - - In newer database engines, the :class:`.Identity` construct should likely - be preferred vs. :class:`.Sequence` for generation of integer primary key - values. See the section :ref:`identity_ddl` for background on this - construct. - -The :class:`~sqlalchemy.schema.Sequence` may be placed on any column as a -"default" generator to be used during INSERT operations, and can also be -configured to fire off during UPDATE operations if desired. It is most -commonly used in conjunction with a single integer primary key column:: - - table = Table( - "cartitems", - metadata_obj, - Column( - "cart_id", - Integer, - Sequence("cart_id_seq", start=1), - primary_key=True, - ), - Column("description", String(40)), - Column("createdate", DateTime()), - ) - -Where above, the table ``cartitems`` is associated with a sequence named -``cart_id_seq``. Emitting :meth:`.MetaData.create_all` for the above -table will include: - -.. sourcecode:: sql - - CREATE SEQUENCE cart_id_seq START WITH 1 - - CREATE TABLE cartitems ( - cart_id INTEGER NOT NULL, - description VARCHAR(40), - createdate TIMESTAMP WITHOUT TIME ZONE, - PRIMARY KEY (cart_id) - ) - -.. tip:: - - When using tables with explicit schema names (detailed at - :ref:`schema_table_schema_name`), the configured schema of the :class:`.Table` - is **not** automatically shared by an embedded :class:`.Sequence`, instead, - specify :paramref:`.Sequence.schema`:: - - Sequence("cart_id_seq", start=1, schema="some_schema") - - The :class:`.Sequence` may also be made to automatically make use of the - :paramref:`.MetaData.schema` setting on the :class:`.MetaData` in use; - see :ref:`sequence_metadata` for background. - -When :class:`_dml.Insert` DML constructs are invoked against the ``cartitems`` -table, without an explicit value passed for the ``cart_id`` column, the -``cart_id_seq`` sequence will be used to generate a value on participating -backends. Typically, the sequence function is embedded in the INSERT statement, -which is combined with RETURNING so that the newly generated value can be -returned to the Python process: - -.. sourcecode:: sql - - INSERT INTO cartitems (cart_id, description, createdate) - VALUES (next_val(cart_id_seq), 'some description', '2015-10-15 12:00:15') - RETURNING cart_id - -When using :meth:`.Connection.execute` to invoke an :class:`_dml.Insert` construct, -newly generated primary key identifiers, including but not limited to those -generated using :class:`.Sequence`, are available from the :class:`.CursorResult` -construct using the :attr:`.CursorResult.inserted_primary_key` attribute. - -When the :class:`~sqlalchemy.schema.Sequence` is associated with a -:class:`_schema.Column` as its **Python-side** default generator, the -:class:`.Sequence` will also be subject to "CREATE SEQUENCE" and "DROP -SEQUENCE" DDL when similar DDL is emitted for the owning :class:`_schema.Table`, -such as when using :meth:`.MetaData.create_all` to generate DDL for a series -of tables. - -The :class:`.Sequence` may also be associated with a -:class:`.MetaData` construct directly. This allows the :class:`.Sequence` -to be used in more than one :class:`.Table` at a time and also allows the -:paramref:`.MetaData.schema` parameter to be inherited. See the section -:ref:`sequence_metadata` for background. - -Associating a Sequence on a SERIAL column -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -PostgreSQL's SERIAL datatype is an auto-incrementing type that implies -the implicit creation of a PostgreSQL sequence when CREATE TABLE is emitted. -The :class:`.Sequence` construct, when indicated for a :class:`_schema.Column`, -may indicate that it should not be used in this specific case by specifying -a value of ``True`` for the :paramref:`.Sequence.optional` parameter. -This allows the given :class:`.Sequence` to be used for backends that have no -alternative primary key generation system but to ignore it for backends -such as PostgreSQL which will automatically generate a sequence for a particular -column:: - - table = Table( - "cartitems", - metadata_obj, - Column( - "cart_id", - Integer, - # use an explicit Sequence where available, but not on - # PostgreSQL where SERIAL will be used - Sequence("cart_id_seq", start=1, optional=True), - primary_key=True, - ), - Column("description", String(40)), - Column("createdate", DateTime()), - ) - -In the above example, ``CREATE TABLE`` for PostgreSQL will make use of the -``SERIAL`` datatype for the ``cart_id`` column, and the ``cart_id_seq`` -sequence will be ignored. However on Oracle, the ``cart_id_seq`` sequence -will be created explicitly. - -.. tip:: - - This particular interaction of SERIAL and SEQUENCE is fairly legacy, and - as in other cases, using :class:`.Identity` instead will simplify the - operation to simply use ``IDENTITY`` on all supported backends. - - -Executing a Sequence Standalone -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -A SEQUENCE is a first class schema object in SQL and can be used to generate -values independently in the database. If you have a :class:`.Sequence` -object, it can be invoked with its "next value" instruction by -passing it directly to a SQL execution method:: - - with my_engine.connect() as conn: - seq = Sequence("some_sequence", start=1) - nextid = conn.execute(seq) - -In order to embed the "next value" function of a :class:`.Sequence` -inside of a SQL statement like a SELECT or INSERT, use the :meth:`.Sequence.next_value` -method, which will render at statement compilation time a SQL function that is -appropriate for the target backend: - -.. sourcecode:: pycon+sql - - >>> my_seq = Sequence("some_sequence", start=1) - >>> stmt = select(my_seq.next_value()) - >>> print(stmt.compile(dialect=postgresql.dialect())) - {printsql}SELECT nextval('some_sequence') AS next_value_1 - -.. _sequence_metadata: - -Associating a Sequence with the MetaData -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -For a :class:`.Sequence` that is to be associated with arbitrary -:class:`.Table` objects, the :class:`.Sequence` may be associated with -a particular :class:`_schema.MetaData`, using the -:paramref:`.Sequence.metadata` parameter:: - - seq = Sequence("my_general_seq", metadata=metadata_obj, start=1) - -Such a sequence can then be associated with columns in the usual way:: - - table = Table( - "cartitems", - metadata_obj, - seq, - Column("description", String(40)), - Column("createdate", DateTime()), - ) - -In the above example, the :class:`.Sequence` object is treated as an -independent schema construct that can exist on its own or be shared among -tables. - -Explicitly associating the :class:`.Sequence` with :class:`_schema.MetaData` -allows for the following behaviors: - -* The :class:`.Sequence` will inherit the :paramref:`_schema.MetaData.schema` - parameter specified to the target :class:`_schema.MetaData`, which - affects the production of CREATE / DROP DDL as well as how the - :meth:`.Sequence.next_value` function is rendered in SQL statements. - -* The :meth:`_schema.MetaData.create_all` and :meth:`_schema.MetaData.drop_all` - methods will emit CREATE / DROP for this :class:`.Sequence`, - even if the :class:`.Sequence` is not associated with any - :class:`_schema.Table` / :class:`_schema.Column` that's a member of this - :class:`_schema.MetaData`. - -Associating a Sequence as the Server Side Default -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. note:: The following technique is known to work only with the PostgreSQL - database. It does not work with Oracle. - -The preceding sections illustrate how to associate a :class:`.Sequence` with a -:class:`_schema.Column` as the **Python side default generator**:: - - Column( - "cart_id", - Integer, - Sequence("cart_id_seq", metadata=metadata_obj, start=1), - primary_key=True, - ) - -In the above case, the :class:`.Sequence` will automatically be subject -to CREATE SEQUENCE / DROP SEQUENCE DDL when the related :class:`_schema.Table` -is subject to CREATE / DROP. However, the sequence will **not** be present -as the server-side default for the column when CREATE TABLE is emitted. - -If we want the sequence to be used as a server-side default, -meaning it takes place even if we emit INSERT commands to the table from -the SQL command line, we can use the :paramref:`_schema.Column.server_default` -parameter in conjunction with the value-generation function of the -sequence, available from the :meth:`.Sequence.next_value` method. Below -we illustrate the same :class:`.Sequence` being associated with the -:class:`_schema.Column` both as the Python-side default generator as well as -the server-side default generator:: - - cart_id_seq = Sequence("cart_id_seq", metadata=metadata_obj, start=1) - table = Table( - "cartitems", - metadata_obj, - Column( - "cart_id", - Integer, - cart_id_seq, - server_default=cart_id_seq.next_value(), - primary_key=True, - ), - Column("description", String(40)), - Column("createdate", DateTime()), - ) - -or with the ORM:: - - class CartItem(Base): - __tablename__ = "cartitems" - - cart_id_seq = Sequence("cart_id_seq", metadata=Base.metadata, start=1) - cart_id = Column( - Integer, cart_id_seq, server_default=cart_id_seq.next_value(), primary_key=True - ) - description = Column(String(40)) - createdate = Column(DateTime) - -When the "CREATE TABLE" statement is emitted, on PostgreSQL it would be -emitted as: - -.. sourcecode:: sql - - CREATE TABLE cartitems ( - cart_id INTEGER DEFAULT nextval('cart_id_seq') NOT NULL, - description VARCHAR(40), - createdate TIMESTAMP WITHOUT TIME ZONE, - PRIMARY KEY (cart_id) - ) - -Placement of the :class:`.Sequence` in both the Python-side and server-side -default generation contexts ensures that the "primary key fetch" logic -works in all cases. Typically, sequence-enabled databases also support -RETURNING for INSERT statements, which is used automatically by SQLAlchemy -when emitting this statement. However if RETURNING is not used for a particular -insert, then SQLAlchemy would prefer to "pre-execute" the sequence outside -of the INSERT statement itself, which only works if the sequence is -included as the Python-side default generator function. - -The example also associates the :class:`.Sequence` with the enclosing -:class:`_schema.MetaData` directly, which again ensures that the :class:`.Sequence` -is fully associated with the parameters of the :class:`_schema.MetaData` collection -including the default schema, if any. - -.. seealso:: - - :ref:`postgresql_sequences` - in the PostgreSQL dialect documentation - - :ref:`oracle_returning` - in the Oracle dialect documentation - -.. _computed_ddl: - -Computed Columns (GENERATED ALWAYS AS) --------------------------------------- - -.. versionadded:: 1.3.11 - -The :class:`.Computed` construct allows a :class:`_schema.Column` to be declared in -DDL as a "GENERATED ALWAYS AS" column, that is, one which has a value that is -computed by the database server. The construct accepts a SQL expression -typically declared textually using a string or the :func:`_expression.text` construct, in -a similar manner as that of :class:`.CheckConstraint`. The SQL expression is -then interpreted by the database server in order to determine the value for the -column within a row. - -Example:: - - from sqlalchemy import Table, Column, MetaData, Integer, Computed - - metadata_obj = MetaData() - - square = Table( - "square", - metadata_obj, - Column("id", Integer, primary_key=True), - Column("side", Integer), - Column("area", Integer, Computed("side * side")), - Column("perimeter", Integer, Computed("4 * side")), - ) - -The DDL for the ``square`` table when run on a PostgreSQL 12 backend will look -like: - -.. sourcecode:: sql - - CREATE TABLE square ( - id SERIAL NOT NULL, - side INTEGER, - area INTEGER GENERATED ALWAYS AS (side * side) STORED, - perimeter INTEGER GENERATED ALWAYS AS (4 * side) STORED, - PRIMARY KEY (id) - ) - -Whether the value is persisted upon INSERT and UPDATE, or if it is calculated -on fetch, is an implementation detail of the database; the former is known as -"stored" and the latter is known as "virtual". Some database implementations -support both, but some only support one or the other. The optional -:paramref:`.Computed.persisted` flag may be specified as ``True`` or ``False`` -to indicate if the "STORED" or "VIRTUAL" keyword should be rendered in DDL, -however this will raise an error if the keyword is not supported by the target -backend; leaving it unset will use a working default for the target backend. - -The :class:`.Computed` construct is a subclass of the :class:`.FetchedValue` -object, and will set itself up as both the "server default" and "server -onupdate" generator for the target :class:`_schema.Column`, meaning it will be treated -as a default generating column when INSERT and UPDATE statements are generated, -as well as that it will be fetched as a generating column when using the ORM. -This includes that it will be part of the RETURNING clause of the database -for databases which support RETURNING and the generated values are to be -eagerly fetched. - -.. note:: A :class:`_schema.Column` that is defined with the :class:`.Computed` - construct may not store any value outside of that which the server applies - to it; SQLAlchemy's behavior when a value is passed for such a column - to be written in INSERT or UPDATE is currently that the value will be - ignored. - -"GENERATED ALWAYS AS" is currently known to be supported by: - -* MySQL version 5.7 and onwards - -* MariaDB 10.x series and onwards - -* PostgreSQL as of version 12 - -* Oracle - with the caveat that RETURNING does not work correctly with UPDATE - (a warning will be emitted to this effect when the UPDATE..RETURNING that - includes a computed column is rendered) - -* Microsoft SQL Server - -* SQLite as of version 3.31 - -When :class:`.Computed` is used with an unsupported backend, if the target -dialect does not support it, a :class:`.CompileError` is raised when attempting -to render the construct. Otherwise, if the dialect supports it but the -particular database server version in use does not, then a subclass of -:class:`.DBAPIError`, usually :class:`.OperationalError`, is raised when the -DDL is emitted to the database. - -.. seealso:: - - :class:`.Computed` - -.. _identity_ddl: - -Identity Columns (GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY) ------------------------------------------------------------------ - -.. versionadded:: 1.4 - -The :class:`.Identity` construct allows a :class:`_schema.Column` to be declared -as an identity column and rendered in DDL as "GENERATED { ALWAYS | BY DEFAULT } -AS IDENTITY". An identity column has its value automatically generated by the -database server using an incrementing (or decrementing) sequence. The construct -shares most of its option to control the database behaviour with -:class:`.Sequence`. - -Example:: - - from sqlalchemy import Table, Column, MetaData, Integer, Identity, String - - metadata_obj = MetaData() - - data = Table( - "data", - metadata_obj, - Column("id", Integer, Identity(start=42, cycle=True), primary_key=True), - Column("data", String), - ) - -The DDL for the ``data`` table when run on a PostgreSQL 12 backend will look -like: - -.. sourcecode:: sql - - CREATE TABLE data ( - id INTEGER GENERATED BY DEFAULT AS IDENTITY (START WITH 42 CYCLE) NOT NULL, - data VARCHAR, - PRIMARY KEY (id) - ) - -The database will generate a value for the ``id`` column upon insert, -starting from ``42``, if the statement did not already contain a value for -the ``id`` column. -An identity column can also require that the database generates the value -of the column, ignoring the value passed with the statement or raising an -error, depending on the backend. To activate this mode, set the parameter -:paramref:`_schema.Identity.always` to ``True`` in the -:class:`.Identity` construct. Updating the previous -example to include this parameter will generate the following DDL: - -.. sourcecode:: sql - - CREATE TABLE data ( - id INTEGER GENERATED ALWAYS AS IDENTITY (START WITH 42 CYCLE) NOT NULL, - data VARCHAR, - PRIMARY KEY (id) - ) - -The :class:`.Identity` construct is a subclass of the :class:`.FetchedValue` -object, and will set itself up as the "server default" generator for the -target :class:`_schema.Column`, meaning it will be treated -as a default generating column when INSERT statements are generated, -as well as that it will be fetched as a generating column when using the ORM. -This includes that it will be part of the RETURNING clause of the database -for databases which support RETURNING and the generated values are to be -eagerly fetched. - -The :class:`.Identity` construct is currently known to be supported by: - -* PostgreSQL as of version 10. - -* Oracle as of version 12. It also supports passing ``always=None`` to - enable the default generated mode and the parameter ``on_null=True`` to - specify "ON NULL" in conjunction with a "BY DEFAULT" identity column. - -* Microsoft SQL Server. MSSQL uses a custom syntax that only supports the - ``start`` and ``increment`` parameters, and ignores all other. - -When :class:`.Identity` is used with an unsupported backend, it is ignored, -and the default SQLAlchemy logic for autoincrementing columns is used. - -An error is raised when a :class:`_schema.Column` specifies both an -:class:`.Identity` and also sets :paramref:`_schema.Column.autoincrement` -to ``False``. - -.. seealso:: - - :class:`.Identity` - - -Default Objects API -------------------- - -.. autoclass:: Computed - :members: - - -.. autoclass:: ColumnDefault - - -.. autoclass:: DefaultClause - - -.. autoclass:: DefaultGenerator - - -.. autoclass:: FetchedValue - - -.. autoclass:: Sequence - :members: - - -.. autoclass:: Identity - :members: diff --git a/doc/build/core/expression_api.rst b/doc/build/core/expression_api.rst index 8000735a11..8802e15473 100644 --- a/doc/build/core/expression_api.rst +++ b/doc/build/core/expression_api.rst @@ -17,8 +17,5 @@ in the :ref:`unified_tutorial`. operators selectable dml - functions compiler - serializer foundation - visitors diff --git a/doc/build/core/functions.rst b/doc/build/core/functions.rst deleted file mode 100644 index 6fcee6edaa..0000000000 --- a/doc/build/core/functions.rst +++ /dev/null @@ -1,143 +0,0 @@ -.. _functions_toplevel: -.. _generic_functions: - -========================= -SQL and Generic Functions -========================= - -.. currentmodule:: sqlalchemy.sql.functions - -SQL functions are invoked by using the :data:`_sql.func` namespace. -See the tutorial at :ref:`tutorial_functions` for background on how to -use the :data:`_sql.func` object to render SQL functions in statements. - -.. seealso:: - - :ref:`tutorial_functions` - in the :ref:`unified_tutorial` - -Function API ------------- - -The base API for SQL functions, which provides for the :data:`_sql.func` -namespace as well as classes that may be used for extensibility. - -.. autoclass:: AnsiFunction - :exclude-members: inherit_cache, __new__ - -.. autoclass:: Function - -.. autoclass:: FunctionElement - :members: - :exclude-members: inherit_cache, __new__ - -.. autoclass:: GenericFunction - :exclude-members: inherit_cache, __new__ - -.. autofunction:: register_function - - -Selected "Known" Functions --------------------------- - -These are :class:`.GenericFunction` implementations for a selected set of -common SQL functions that set up the expected return type for each function -automatically. The are invoked in the same way as any other member of the -:data:`_sql.func` namespace:: - - select(func.count("*")).select_from(some_table) - -Note that any name not known to :data:`_sql.func` generates the function name -as is - there is no restriction on what SQL functions can be called, known or -unknown to SQLAlchemy, built-in or user defined. The section here only -describes those functions where SQLAlchemy already knows what argument and -return types are in use. - -.. autoclass:: array_agg - :no-members: - -.. autoclass:: char_length - :no-members: - -.. autoclass:: coalesce - :no-members: - -.. autoclass:: concat - :no-members: - -.. autoclass:: count - :no-members: - -.. autoclass:: cube - :no-members: - -.. autoclass:: cume_dist - :no-members: - -.. autoclass:: current_date - :no-members: - -.. autoclass:: current_time - :no-members: - -.. autoclass:: current_timestamp - :no-members: - -.. autoclass:: current_user - :no-members: - -.. autoclass:: dense_rank - :no-members: - -.. autoclass:: grouping_sets - :no-members: - -.. autoclass:: localtime - :no-members: - -.. autoclass:: localtimestamp - :no-members: - -.. autoclass:: max - :no-members: - -.. autoclass:: min - :no-members: - -.. autoclass:: mode - :no-members: - -.. autoclass:: next_value - :no-members: - -.. autoclass:: now - :no-members: - -.. autoclass:: percent_rank - :no-members: - -.. autoclass:: percentile_cont - :no-members: - -.. autoclass:: percentile_disc - :no-members: - -.. autoclass:: random - :no-members: - -.. autoclass:: rank - :no-members: - -.. autoclass:: rollup - :no-members: - -.. autoclass:: session_user - :no-members: - -.. autoclass:: sum - :no-members: - -.. autoclass:: sysdate - :no-members: - -.. autoclass:: user - :no-members: diff --git a/doc/build/core/future.rst b/doc/build/core/future.rst deleted file mode 100644 index 9c171b9db5..0000000000 --- a/doc/build/core/future.rst +++ /dev/null @@ -1,17 +0,0 @@ -:orphan: - -SQLAlchemy 2.0 Future (Core) -============================ - -.. admonition:: We're 2.0! - - This page described the "future" mode provided in SQLAlchemy 1.4 - for the purposes of 1.4 -> 2.0 transition. For 2.0, the "future" - parameter on :func:`_sa.create_engine` and :class:`_orm.Session` - continues to remain available for backwards-compatibility support, however - if specified must be left at the value of ``True``. - - .. seealso:: - - :ref:`migration_20_toplevel` - Introduction to the 2.0 series of SQLAlchemy - diff --git a/doc/build/core/index.rst b/doc/build/core/index.rst index 764247ab56..4629bd82e8 100644 --- a/doc/build/core/index.rst +++ b/doc/build/core/index.rst @@ -12,8 +12,8 @@ Language provides a schema-centric usage paradigm. :maxdepth: 2 expression_api + engines_connections schema types - engines_connections api_basics diff --git a/doc/build/core/inspection.rst b/doc/build/core/inspection.rst deleted file mode 100644 index 7816cd3fd8..0000000000 --- a/doc/build/core/inspection.rst +++ /dev/null @@ -1,49 +0,0 @@ -.. _core_inspection_toplevel: -.. _inspection_toplevel: - -Runtime Inspection API -====================== - -.. automodule:: sqlalchemy.inspection - -.. autofunction:: sqlalchemy.inspect - - -Available Inspection Targets ----------------------------- - -Below is a listing of many of the most common inspection targets. - -* :class:`.Connectable` (i.e. :class:`_engine.Engine`, - :class:`_engine.Connection`) - returns an :class:`_reflection.Inspector` object. -* :class:`_expression.ClauseElement` - all SQL expression components, including - :class:`_schema.Table`, :class:`_schema.Column`, serve as their own inspection objects, - meaning any of these objects passed to :func:`_sa.inspect` return themselves. -* ``object`` - an object given will be checked by the ORM for a mapping - - if so, an :class:`.InstanceState` is returned representing the mapped - state of the object. The :class:`.InstanceState` also provides access - to per attribute state via the :class:`.AttributeState` interface as well - as the per-flush "history" of any attribute via the :class:`.History` - object. - - .. seealso:: - - :ref:`orm_mapper_inspection_instancestate` - -* ``type`` (i.e. a class) - a class given will be checked by the ORM for a - mapping - if so, a :class:`_orm.Mapper` for that class is returned. - - .. seealso:: - - :ref:`orm_mapper_inspection_mapper` - -* mapped attribute - passing a mapped attribute to :func:`_sa.inspect`, such - as ``inspect(MyClass.some_attribute)``, returns a :class:`.QueryableAttribute` - object, which is the :term:`descriptor` associated with a mapped class. - This descriptor refers to a :class:`.MapperProperty`, which is usually - an instance of :class:`.ColumnProperty` - or :class:`.RelationshipProperty`, via its :attr:`.QueryableAttribute.property` - attribute. -* :class:`.AliasedClass` - returns an :class:`.AliasedInsp` object. - - diff --git a/doc/build/core/reflection.rst b/doc/build/core/reflection.rst deleted file mode 100644 index 1ff517510e..0000000000 --- a/doc/build/core/reflection.rst +++ /dev/null @@ -1,556 +0,0 @@ -.. currentmodule:: sqlalchemy.schema - -.. _metadata_reflection_toplevel: -.. _metadata_reflection: - - -Reflecting Database Objects -=========================== - -A :class:`~sqlalchemy.schema.Table` object can be instructed to load -information about itself from the corresponding database schema object already -existing within the database. This process is called *reflection*. In the -most simple case you need only specify the table name, a :class:`~sqlalchemy.schema.MetaData` -object, and the ``autoload_with`` argument:: - - >>> messages = Table("messages", metadata_obj, autoload_with=engine) - >>> [c.name for c in messages.columns] - ['message_id', 'message_name', 'date'] - -The above operation will use the given engine to query the database for -information about the ``messages`` table, and will then generate -:class:`~sqlalchemy.schema.Column`, :class:`~sqlalchemy.schema.ForeignKey`, -and other objects corresponding to this information as though the -:class:`~sqlalchemy.schema.Table` object were hand-constructed in Python. - -When tables are reflected, if a given table references another one via foreign -key, a second :class:`~sqlalchemy.schema.Table` object is created within the -:class:`~sqlalchemy.schema.MetaData` object representing the connection. -Below, assume the table ``shopping_cart_items`` references a table named -``shopping_carts``. Reflecting the ``shopping_cart_items`` table has the -effect such that the ``shopping_carts`` table will also be loaded:: - - >>> shopping_cart_items = Table("shopping_cart_items", metadata_obj, autoload_with=engine) - >>> "shopping_carts" in metadata_obj.tables - True - -The :class:`~sqlalchemy.schema.MetaData` has an interesting "singleton-like" -behavior such that if you requested both tables individually, -:class:`~sqlalchemy.schema.MetaData` will ensure that exactly one -:class:`~sqlalchemy.schema.Table` object is created for each distinct table -name. The :class:`~sqlalchemy.schema.Table` constructor actually returns to -you the already-existing :class:`~sqlalchemy.schema.Table` object if one -already exists with the given name. Such as below, we can access the already -generated ``shopping_carts`` table just by naming it:: - - shopping_carts = Table("shopping_carts", metadata_obj) - -Of course, it's a good idea to use ``autoload_with=engine`` with the above table -regardless. This is so that the table's attributes will be loaded if they have -not been already. The autoload operation only occurs for the table if it -hasn't already been loaded; once loaded, new calls to -:class:`~sqlalchemy.schema.Table` with the same name will not re-issue any -reflection queries. - -.. _reflection_overriding_columns: - -Overriding Reflected Columns ----------------------------- - -Individual columns can be overridden with explicit values when reflecting -tables; this is handy for specifying custom datatypes, constraints such as -primary keys that may not be configured within the database, etc.:: - - >>> mytable = Table( - ... "mytable", - ... metadata_obj, - ... Column( - ... "id", Integer, primary_key=True - ... ), # override reflected 'id' to have primary key - ... Column("mydata", Unicode(50)), # override reflected 'mydata' to be Unicode - ... # additional Column objects which require no change are reflected normally - ... autoload_with=some_engine, - ... ) - -.. seealso:: - - :ref:`custom_and_decorated_types_reflection` - illustrates how the above - column override technique applies to the use of custom datatypes with - table reflection. - -Reflecting Views ----------------- - -The reflection system can also reflect views. Basic usage is the same as that -of a table:: - - my_view = Table("some_view", metadata, autoload_with=engine) - -Above, ``my_view`` is a :class:`~sqlalchemy.schema.Table` object with -:class:`~sqlalchemy.schema.Column` objects representing the names and types of -each column within the view "some_view". - -Usually, it's desired to have at least a primary key constraint when -reflecting a view, if not foreign keys as well. View reflection doesn't -extrapolate these constraints. - -Use the "override" technique for this, specifying explicitly those columns -which are part of the primary key or have foreign key constraints:: - - my_view = Table( - "some_view", - metadata, - Column("view_id", Integer, primary_key=True), - Column("related_thing", Integer, ForeignKey("othertable.thing_id")), - autoload_with=engine, - ) - -Reflecting All Tables at Once ------------------------------ - -The :class:`~sqlalchemy.schema.MetaData` object can also get a listing of -tables and reflect the full set. This is achieved by using the -:func:`~sqlalchemy.schema.MetaData.reflect` method. After calling it, all -located tables are present within the :class:`~sqlalchemy.schema.MetaData` -object's dictionary of tables:: - - metadata_obj = MetaData() - metadata_obj.reflect(bind=someengine) - users_table = metadata_obj.tables["users"] - addresses_table = metadata_obj.tables["addresses"] - -``metadata.reflect()`` also provides a handy way to clear or delete all the rows in a database:: - - metadata_obj = MetaData() - metadata_obj.reflect(bind=someengine) - for table in reversed(metadata_obj.sorted_tables): - someengine.execute(table.delete()) - -.. _metadata_reflection_schemas: - -Reflecting Tables from Other Schemas ------------------------------------- - -The section :ref:`schema_table_schema_name` introduces the concept of table -schemas, which are namespaces within a database that contain tables and other -objects, and which can be specified explicitly. The "schema" for a -:class:`_schema.Table` object, as well as for other objects like views, indexes and -sequences, can be set up using the :paramref:`_schema.Table.schema` parameter, -and also as the default schema for a :class:`_schema.MetaData` object using the -:paramref:`_schema.MetaData.schema` parameter. - -The use of this schema parameter directly affects where the table reflection -feature will look when it is asked to reflect objects. For example, given -a :class:`_schema.MetaData` object configured with a default schema name -"project" via its :paramref:`_schema.MetaData.schema` parameter:: - - >>> metadata_obj = MetaData(schema="project") - -The :meth:`.MetaData.reflect` will then utilize that configured ``.schema`` -for reflection:: - - >>> # uses `schema` configured in metadata_obj - >>> metadata_obj.reflect(someengine) - -The end result is that :class:`_schema.Table` objects from the "project" -schema will be reflected, and they will be populated as schema-qualified -with that name:: - - >>> metadata_obj.tables["project.messages"] - Table('messages', MetaData(), Column('message_id', INTEGER(), table=), schema='project') - -Similarly, an individual :class:`_schema.Table` object that includes the -:paramref:`_schema.Table.schema` parameter will also be reflected from that -database schema, overriding any default schema that may have been configured on the -owning :class:`_schema.MetaData` collection:: - - >>> messages = Table("messages", metadata_obj, schema="project", autoload_with=someengine) - >>> messages - Table('messages', MetaData(), Column('message_id', INTEGER(), table=), schema='project') - -Finally, the :meth:`_schema.MetaData.reflect` method itself also allows a -:paramref:`_schema.MetaData.reflect.schema` parameter to be passed, so we -could also load tables from the "project" schema for a default configured -:class:`_schema.MetaData` object:: - - >>> metadata_obj = MetaData() - >>> metadata_obj.reflect(someengine, schema="project") - -We can call :meth:`_schema.MetaData.reflect` any number of times with different -:paramref:`_schema.MetaData.schema` arguments (or none at all) to continue -populating the :class:`_schema.MetaData` object with more objects:: - - >>> # add tables from the "customer" schema - >>> metadata_obj.reflect(someengine, schema="customer") - >>> # add tables from the default schema - >>> metadata_obj.reflect(someengine) - -.. _reflection_schema_qualified_interaction: - -Interaction of Schema-qualified Reflection with the Default Schema -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. admonition:: Section Best Practices Summarized - - In this section, we discuss SQLAlchemy's reflection behavior regarding - tables that are visible in the "default schema" of a database session, - and how these interact with SQLAlchemy directives that include the schema - explicitly. As a best practice, ensure the "default" schema for a database - is just a single name, and not a list of names; for tables that are - part of this "default" schema and can be named without schema qualification - in DDL and SQL, leave corresponding :paramref:`_schema.Table.schema` and - similar schema parameters set to their default of ``None``. - -As described at :ref:`schema_metadata_schema_name`, databases that have -the concept of schemas usually also include the concept of a "default" schema. -The reason for this is naturally that when one refers to table objects without -a schema as is common, a schema-capable database will still consider that -table to be in a "schema" somewhere. Some databases such as PostgreSQL -take this concept further into the notion of a -`schema search path -`_ -where *multiple* schema names can be considered in a particular database -session to be "implicit"; referring to a table name that it's any of those -schemas will not require that the schema name be present (while at the same time -it's also perfectly fine if the schema name *is* present). - -Since most relational databases therefore have the concept of a particular -table object which can be referred towards both in a schema-qualified way, as -well as an "implicit" way where no schema is present, this presents a -complexity for SQLAlchemy's reflection -feature. Reflecting a table in -a schema-qualified manner will always populate its :attr:`_schema.Table.schema` -attribute and additionally affect how this :class:`_schema.Table` is organized -into the :attr:`_schema.MetaData.tables` collection, that is, in a schema -qualified manner. Conversely, reflecting the **same** table in a non-schema -qualified manner will organize it into the :attr:`_schema.MetaData.tables` -collection **without** being schema qualified. The end result is that there -would be two separate :class:`_schema.Table` objects in the single -:class:`_schema.MetaData` collection representing the same table in the -actual database. - -To illustrate the ramifications of this issue, consider tables from the -"project" schema in the previous example, and suppose also that the "project" -schema is the default schema of our database connection, or if using a database -such as PostgreSQL suppose the "project" schema is set up in the PostgreSQL -``search_path``. This would mean that the database accepts the following -two SQL statements as equivalent: - -.. sourcecode:: sql - - -- schema qualified - SELECT message_id FROM project.messages - - -- non-schema qualified - SELECT message_id FROM messages - -This is not a problem as the table can be found in both ways. However -in SQLAlchemy, it's the **identity** of the :class:`_schema.Table` object -that determines its semantic role within a SQL statement. Based on the current -decisions within SQLAlchemy, this means that if we reflect the same "messages" table in -both a schema-qualified as well as a non-schema qualified manner, we get -**two** :class:`_schema.Table` objects that will **not** be treated as -semantically equivalent:: - - >>> # reflect in non-schema qualified fashion - >>> messages_table_1 = Table("messages", metadata_obj, autoload_with=someengine) - >>> # reflect in schema qualified fashion - >>> messages_table_2 = Table( - ... "messages", metadata_obj, schema="project", autoload_with=someengine - ... ) - >>> # two different objects - >>> messages_table_1 is messages_table_2 - False - >>> # stored in two different ways - >>> metadata.tables["messages"] is messages_table_1 - True - >>> metadata.tables["project.messages"] is messages_table_2 - True - -The above issue becomes more complicated when the tables being reflected contain -foreign key references to other tables. Suppose "messages" has a "project_id" -column which refers to rows in another schema-local table "projects", meaning -there is a :class:`_schema.ForeignKeyConstraint` object that is part of the -definition of the "messages" table. - -We can find ourselves in a situation where one :class:`_schema.MetaData` -collection may contain as many as four :class:`_schema.Table` objects -representing these two database tables, where one or two of the additional -tables were generated by the reflection process; this is because when -the reflection process encounters a foreign key constraint on a table -being reflected, it branches out to reflect that referenced table as well. -The decision making it uses to assign the schema to this referenced -table is that SQLAlchemy will **omit a default schema** from the reflected -:class:`_schema.ForeignKeyConstraint` object if the owning -:class:`_schema.Table` also omits its schema name and also that these two objects -are in the same schema, but will **include** it if -it were not omitted. - -The common scenario is when the reflection of a table in a schema qualified -fashion then loads a related table that will also be performed in a schema -qualified fashion:: - - >>> # reflect "messages" in a schema qualified fashion - >>> messages_table_1 = Table( - ... "messages", metadata_obj, schema="project", autoload_with=someengine - ... ) - -The above ``messages_table_1`` will refer to ``projects`` also in a schema -qualified fashion. This "projects" table will be reflected automatically by -the fact that "messages" refers to it:: - - >>> messages_table_1.c.project_id - Column('project_id', INTEGER(), ForeignKey('project.projects.project_id'), table=) - -if some other part of the code reflects "projects" in a non-schema qualified -fashion, there are now two projects tables that are not the same: - - >>> # reflect "projects" in a non-schema qualified fashion - >>> projects_table_1 = Table("projects", metadata_obj, autoload_with=someengine) - - >>> # messages does not refer to projects_table_1 above - >>> messages_table_1.c.project_id.references(projects_table_1.c.project_id) - False - - >>> # it refers to this one - >>> projects_table_2 = metadata_obj.tables["project.projects"] - >>> messages_table_1.c.project_id.references(projects_table_2.c.project_id) - True - - >>> # they're different, as one non-schema qualified and the other one is - >>> projects_table_1 is projects_table_2 - False - -The above confusion can cause problems within applications that use table -reflection to load up application-level :class:`_schema.Table` objects, as -well as within migration scenarios, in particular such as when using Alembic -Migrations to detect new tables and foreign key constraints. - -The above behavior can be remedied by sticking to one simple practice: - -* Don't include the :paramref:`_schema.Table.schema` parameter for any - :class:`_schema.Table` that expects to be located in the **default** schema - of the database. - -For PostgreSQL and other databases that support a "search" path for schemas, -add the following additional practice: - -* Keep the "search path" narrowed down to **one schema only, which is the - default schema**. - - -.. seealso:: - - :ref:`postgresql_schema_reflection` - additional details of this behavior - as regards the PostgreSQL database. - - -.. _metadata_reflection_inspector: - -Fine Grained Reflection with Inspector --------------------------------------- - -A low level interface which provides a backend-agnostic system of loading -lists of schema, table, column, and constraint descriptions from a given -database is also available. This is known as the "Inspector":: - - from sqlalchemy import create_engine - from sqlalchemy import inspect - - engine = create_engine("...") - insp = inspect(engine) - print(insp.get_table_names()) - -.. autoclass:: sqlalchemy.engine.reflection.Inspector - :members: - :undoc-members: - -.. autoclass:: sqlalchemy.engine.interfaces.ReflectedColumn - :members: - :inherited-members: dict - -.. autoclass:: sqlalchemy.engine.interfaces.ReflectedComputed - :members: - :inherited-members: dict - -.. autoclass:: sqlalchemy.engine.interfaces.ReflectedCheckConstraint - :members: - :inherited-members: dict - -.. autoclass:: sqlalchemy.engine.interfaces.ReflectedForeignKeyConstraint - :members: - :inherited-members: dict - -.. autoclass:: sqlalchemy.engine.interfaces.ReflectedIdentity - :members: - :inherited-members: dict - -.. autoclass:: sqlalchemy.engine.interfaces.ReflectedIndex - :members: - :inherited-members: dict - -.. autoclass:: sqlalchemy.engine.interfaces.ReflectedPrimaryKeyConstraint - :members: - :inherited-members: dict - -.. autoclass:: sqlalchemy.engine.interfaces.ReflectedUniqueConstraint - :members: - :inherited-members: dict - -.. autoclass:: sqlalchemy.engine.interfaces.ReflectedTableComment - :members: - :inherited-members: dict - - -.. _metadata_reflection_dbagnostic_types: - -Reflecting with Database-Agnostic Types ---------------------------------------- - -When the columns of a table are reflected, using either the -:paramref:`_schema.Table.autoload_with` parameter of :class:`_schema.Table` or -the :meth:`_reflection.Inspector.get_columns` method of -:class:`_reflection.Inspector`, the datatypes will be as specific as possible -to the target database. This means that if an "integer" datatype is reflected -from a MySQL database, the type will be represented by the -:class:`sqlalchemy.dialects.mysql.INTEGER` class, which includes MySQL-specific -attributes such as "display_width". Or on PostgreSQL, a PostgreSQL-specific -datatype such as :class:`sqlalchemy.dialects.postgresql.INTERVAL` or -:class:`sqlalchemy.dialects.postgresql.ENUM` may be returned. - -There is a use case for reflection which is that a given :class:`_schema.Table` -is to be transferred to a different vendor database. To suit this use case, -there is a technique by which these vendor-specific datatypes can be converted -on the fly to be instance of SQLAlchemy backend-agnostic datatypes, for -the examples above types such as :class:`_types.Integer`, :class:`_types.Interval` -and :class:`_types.Enum`. This may be achieved by intercepting the -column reflection using the :meth:`_events.DDLEvents.column_reflect` event -in conjunction with the :meth:`_types.TypeEngine.as_generic` method. - -Given a table in MySQL (chosen because MySQL has a lot of vendor-specific -datatypes and options): - -.. sourcecode:: sql - - CREATE TABLE IF NOT EXISTS my_table ( - id INTEGER PRIMARY KEY AUTO_INCREMENT, - data1 VARCHAR(50) CHARACTER SET latin1, - data2 MEDIUMINT(4), - data3 TINYINT(2) - ) - -The above table includes MySQL-only integer types ``MEDIUMINT`` and -``TINYINT`` as well as a ``VARCHAR`` that includes the MySQL-only ``CHARACTER -SET`` option. If we reflect this table normally, it produces a -:class:`_schema.Table` object that will contain those MySQL-specific datatypes -and options: - -.. sourcecode:: pycon+sql - - >>> from sqlalchemy import MetaData, Table, create_engine - >>> mysql_engine = create_engine("mysql+mysqldb://scott:tiger@localhost/test") - >>> metadata_obj = MetaData() - >>> my_mysql_table = Table("my_table", metadata_obj, autoload_with=mysql_engine) - -The above example reflects the above table schema into a new :class:`_schema.Table` -object. We can then, for demonstration purposes, print out the MySQL-specific -"CREATE TABLE" statement using the :class:`_schema.CreateTable` construct: - -.. sourcecode:: pycon+sql - - >>> from sqlalchemy.schema import CreateTable - >>> print(CreateTable(my_mysql_table).compile(mysql_engine)) - {printsql}CREATE TABLE my_table ( - id INTEGER(11) NOT NULL AUTO_INCREMENT, - data1 VARCHAR(50) CHARACTER SET latin1, - data2 MEDIUMINT(4), - data3 TINYINT(2), - PRIMARY KEY (id) - )ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 - - -Above, the MySQL-specific datatypes and options were maintained. If we wanted -a :class:`_schema.Table` that we could instead transfer cleanly to another -database vendor, replacing the special datatypes -:class:`sqlalchemy.dialects.mysql.MEDIUMINT` and -:class:`sqlalchemy.dialects.mysql.TINYINT` with :class:`_types.Integer`, we can -choose instead to "genericize" the datatypes on this table, or otherwise change -them in any way we'd like, by establishing a handler using the -:meth:`_events.DDLEvents.column_reflect` event. The custom handler will make use -of the :meth:`_types.TypeEngine.as_generic` method to convert the above -MySQL-specific type objects into generic ones, by replacing the ``"type"`` -entry within the column dictionary entry that is passed to the event handler. -The format of this dictionary is described at :meth:`_reflection.Inspector.get_columns`: - -.. sourcecode:: pycon+sql - - >>> from sqlalchemy import event - >>> metadata_obj = MetaData() - - >>> @event.listens_for(metadata_obj, "column_reflect") - ... def genericize_datatypes(inspector, tablename, column_dict): - ... column_dict["type"] = column_dict["type"].as_generic() - - >>> my_generic_table = Table("my_table", metadata_obj, autoload_with=mysql_engine) - -We now get a new :class:`_schema.Table` that is generic and uses -:class:`_types.Integer` for those datatypes. We can now emit a -"CREATE TABLE" statement for example on a PostgreSQL database: - -.. sourcecode:: pycon+sql - - >>> pg_engine = create_engine("postgresql+psycopg2://scott:tiger@localhost/test", echo=True) - >>> my_generic_table.create(pg_engine) - {execsql}CREATE TABLE my_table ( - id SERIAL NOT NULL, - data1 VARCHAR(50), - data2 INTEGER, - data3 INTEGER, - PRIMARY KEY (id) - ) - -Noting above also that SQLAlchemy will usually make a decent guess for other -behaviors, such as that the MySQL ``AUTO_INCREMENT`` directive is represented -in PostgreSQL most closely using the ``SERIAL`` auto-incrementing datatype. - -.. versionadded:: 1.4 Added the :meth:`_types.TypeEngine.as_generic` method - and additionally improved the use of the :meth:`_events.DDLEvents.column_reflect` - event such that it may be applied to a :class:`_schema.MetaData` object - for convenience. - - -Limitations of Reflection -------------------------- - -It's important to note that the reflection process recreates :class:`_schema.Table` -metadata using only information which is represented in the relational database. -This process by definition cannot restore aspects of a schema that aren't -actually stored in the database. State which is not available from reflection -includes but is not limited to: - -* Client side defaults, either Python functions or SQL expressions defined using - the ``default`` keyword of :class:`_schema.Column` (note this is separate from ``server_default``, - which specifically is what's available via reflection). - -* Column information, e.g. data that might have been placed into the - :attr:`_schema.Column.info` dictionary - -* The value of the ``.quote`` setting for :class:`_schema.Column` or :class:`_schema.Table` - -* The association of a particular :class:`.Sequence` with a given :class:`_schema.Column` - -The relational database also in many cases reports on table metadata in a -different format than what was specified in SQLAlchemy. The :class:`_schema.Table` -objects returned from reflection cannot be always relied upon to produce the identical -DDL as the original Python-defined :class:`_schema.Table` objects. Areas where -this occurs includes server defaults, column-associated sequences and various -idiosyncrasies regarding constraints and datatypes. Server side defaults may -be returned with cast directives (typically PostgreSQL will include a ``::`` -cast) or different quoting patterns than originally specified. - -Another category of limitation includes schema structures for which reflection -is only partially or not yet defined. Recent improvements to reflection allow -things like views, indexes and foreign key options to be reflected. As of this -writing, structures like CHECK constraints, table comments, and triggers are -not reflected. - diff --git a/doc/build/core/schema.rst b/doc/build/core/schema.rst index 5a4f939bf7..670e472683 100644 --- a/doc/build/core/schema.rst +++ b/doc/build/core/schema.rst @@ -36,8 +36,6 @@ in creating real schema generation scripts. :maxdepth: 3 metadata - reflection - defaults constraints ddl diff --git a/doc/build/core/serializer.rst b/doc/build/core/serializer.rst deleted file mode 100644 index 5423306a92..0000000000 --- a/doc/build/core/serializer.rst +++ /dev/null @@ -1,6 +0,0 @@ -Expression Serializer Extension -=============================== - -.. automodule:: sqlalchemy.ext.serializer - :members: - :undoc-members: diff --git a/doc/build/core/visitors.rst b/doc/build/core/visitors.rst deleted file mode 100644 index 06d839d54c..0000000000 --- a/doc/build/core/visitors.rst +++ /dev/null @@ -1,27 +0,0 @@ -Visitor and Traversal Utilities -================================ - -The :mod:`sqlalchemy.sql.visitors` module consists of classes and functions -that serve the purpose of generically **traversing** a Core SQL expression -structure. This is not unlike the Python ``ast`` module in that is presents -a system by which a program can operate upon each component of a SQL -expression. Common purposes this serves are locating various kinds of -elements such as :class:`_schema.Table` or :class:`.BindParameter` objects, -as well as altering the state of the structure such as replacing certain FROM -clauses with others. - -.. note:: the :mod:`sqlalchemy.sql.visitors` module is an internal API and - is not fully public. It is subject to change and may additionally not - function as expected for use patterns that aren't considered within - SQLAlchemy's own internals. - -The :mod:`sqlalchemy.sql.visitors` module is part of the **internals** of -SQLAlchemy and it is not usually used by calling application code. It is -however used in certain edge cases such as when constructing caching routines -as well as when building out custom SQL expressions using the -:ref:`Custom SQL Constructs and Compilation Extension `. - -.. automodule:: sqlalchemy.sql.visitors - :members: - :private-members: - diff --git a/doc/build/dialects/index.rst b/doc/build/dialects/index.rst index 06b4132640..9a4badee23 100644 --- a/doc/build/dialects/index.rst +++ b/doc/build/dialects/index.rst @@ -19,18 +19,7 @@ Included Dialects :glob: postgresql - mysql - sqlite - oracle - mssql -Support Levels for Included Dialects -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The following table summarizes the support level for each included dialect. - -.. dialect-table:: **Supported database versions for included dialects** - :header-rows: 1 Support Definitions ^^^^^^^^^^^^^^^^^^^ diff --git a/doc/build/dialects/mssql.rst b/doc/build/dialects/mssql.rst deleted file mode 100644 index 3aff100873..0000000000 --- a/doc/build/dialects/mssql.rst +++ /dev/null @@ -1,163 +0,0 @@ -.. _mssql_toplevel: - -Microsoft SQL Server -==================== - -.. automodule:: sqlalchemy.dialects.mssql.base - -SQL Server SQL Constructs -------------------------- - -.. currentmodule:: sqlalchemy.dialects.mssql - -.. autofunction:: try_cast - -SQL Server Data Types ---------------------- - -As with all SQLAlchemy dialects, all UPPERCASE types that are known to be -valid with SQL server are importable from the top level dialect, whether -they originate from :mod:`sqlalchemy.types` or from the local dialect:: - - from sqlalchemy.dialects.mssql import ( - BIGINT, - BINARY, - BIT, - CHAR, - DATE, - DATETIME, - DATETIME2, - DATETIMEOFFSET, - DECIMAL, - FLOAT, - IMAGE, - INTEGER, - JSON, - MONEY, - NCHAR, - NTEXT, - NUMERIC, - NVARCHAR, - REAL, - SMALLDATETIME, - SMALLINT, - SMALLMONEY, - SQL_VARIANT, - TEXT, - TIME, - TIMESTAMP, - TINYINT, - UNIQUEIDENTIFIER, - VARBINARY, - VARCHAR, - ) - -Types which are specific to SQL Server, or have SQL Server-specific -construction arguments, are as follows: - -.. note: where :noindex: is used, indicates a type that is not redefined - in the dialect module, just imported from sqltypes. this avoids warnings - in the sphinx build - -.. currentmodule:: sqlalchemy.dialects.mssql - -.. autoclass:: BIT - :members: __init__ - - -.. autoclass:: CHAR - :members: __init__ - :noindex: - - -.. autoclass:: DATETIME2 - :members: __init__ - - -.. autoclass:: DATETIMEOFFSET - :members: __init__ - - -.. autoclass:: IMAGE - :members: __init__ - - -.. autoclass:: JSON - :members: __init__ - - -.. autoclass:: MONEY - :members: __init__ - - -.. autoclass:: NCHAR - :members: __init__ - :noindex: - - -.. autoclass:: NTEXT - :members: __init__ - - -.. autoclass:: NVARCHAR - :members: __init__ - :noindex: - -.. autoclass:: REAL - :members: __init__ - -.. autoclass:: ROWVERSION - :members: __init__ - -.. autoclass:: SMALLDATETIME - :members: __init__ - - -.. autoclass:: SMALLMONEY - :members: __init__ - - -.. autoclass:: SQL_VARIANT - :members: __init__ - - -.. autoclass:: TEXT - :members: __init__ - :noindex: - -.. autoclass:: TIME - :members: __init__ - - -.. autoclass:: TIMESTAMP - :members: __init__ - -.. autoclass:: TINYINT - :members: __init__ - - -.. autoclass:: UNIQUEIDENTIFIER - :members: __init__ - - -.. autoclass:: VARBINARY - :members: __init__ - :noindex: - -.. autoclass:: VARCHAR - :members: __init__ - :noindex: - - -.. autoclass:: XML - :members: __init__ - - -PyODBC ------- -.. automodule:: sqlalchemy.dialects.mssql.pyodbc - -pymssql -------- -.. automodule:: sqlalchemy.dialects.mssql.pymssql - diff --git a/doc/build/dialects/mysql.rst b/doc/build/dialects/mysql.rst deleted file mode 100644 index a46bf721e2..0000000000 --- a/doc/build/dialects/mysql.rst +++ /dev/null @@ -1,260 +0,0 @@ -.. _mysql_toplevel: - -MySQL and MariaDB -================= - -.. automodule:: sqlalchemy.dialects.mysql.base - -MySQL SQL Constructs --------------------- - -.. currentmodule:: sqlalchemy.dialects.mysql - -.. autoclass:: match - :members: - -MySQL Data Types ----------------- - -As with all SQLAlchemy dialects, all UPPERCASE types that are known to be -valid with MySQL are importable from the top level dialect:: - - from sqlalchemy.dialects.mysql import ( - BIGINT, - BINARY, - BIT, - BLOB, - BOOLEAN, - CHAR, - DATE, - DATETIME, - DECIMAL, - DECIMAL, - DOUBLE, - ENUM, - FLOAT, - INTEGER, - LONGBLOB, - LONGTEXT, - MEDIUMBLOB, - MEDIUMINT, - MEDIUMTEXT, - NCHAR, - NUMERIC, - NVARCHAR, - REAL, - SET, - SMALLINT, - TEXT, - TIME, - TIMESTAMP, - TINYBLOB, - TINYINT, - TINYTEXT, - VARBINARY, - VARCHAR, - YEAR, - ) - -Types which are specific to MySQL, or have MySQL-specific -construction arguments, are as follows: - -.. note: where :noindex: is used, indicates a type that is not redefined - in the dialect module, just imported from sqltypes. this avoids warnings - in the sphinx build - -.. currentmodule:: sqlalchemy.dialects.mysql - -.. autoclass:: BIGINT - :members: __init__ - - -.. autoclass:: BINARY - :noindex: - :members: __init__ - - -.. autoclass:: BIT - :members: __init__ - - -.. autoclass:: BLOB - :members: __init__ - :noindex: - - -.. autoclass:: BOOLEAN - :members: __init__ - :noindex: - - -.. autoclass:: CHAR - :members: __init__ - - -.. autoclass:: DATE - :members: __init__ - :noindex: - - -.. autoclass:: DATETIME - :members: __init__ - - -.. autoclass:: DECIMAL - :members: __init__ - - -.. autoclass:: DOUBLE - :members: __init__ - :noindex: - -.. autoclass:: ENUM - :members: __init__ - - -.. autoclass:: FLOAT - :members: __init__ - - -.. autoclass:: INTEGER - :members: __init__ - -.. autoclass:: JSON - :members: - -.. autoclass:: LONGBLOB - :members: __init__ - - -.. autoclass:: LONGTEXT - :members: __init__ - - -.. autoclass:: MEDIUMBLOB - :members: __init__ - - -.. autoclass:: MEDIUMINT - :members: __init__ - - -.. autoclass:: MEDIUMTEXT - :members: __init__ - - -.. autoclass:: NCHAR - :members: __init__ - - -.. autoclass:: NUMERIC - :members: __init__ - - -.. autoclass:: NVARCHAR - :members: __init__ - - -.. autoclass:: REAL - :members: __init__ - - -.. autoclass:: SET - :members: __init__ - - -.. autoclass:: SMALLINT - :members: __init__ - - -.. autoclass:: TEXT - :members: __init__ - :noindex: - - -.. autoclass:: TIME - :members: __init__ - - -.. autoclass:: TIMESTAMP - :members: __init__ - - -.. autoclass:: TINYBLOB - :members: __init__ - - -.. autoclass:: TINYINT - :members: __init__ - - -.. autoclass:: TINYTEXT - :members: __init__ - - -.. autoclass:: VARBINARY - :members: __init__ - :noindex: - - -.. autoclass:: VARCHAR - :members: __init__ - - -.. autoclass:: YEAR - :members: __init__ - -MySQL DML Constructs -------------------------- - -.. autofunction:: sqlalchemy.dialects.mysql.insert - -.. autoclass:: sqlalchemy.dialects.mysql.Insert - :members: - - - -mysqlclient (fork of MySQL-Python) ----------------------------------- - -.. automodule:: sqlalchemy.dialects.mysql.mysqldb - -PyMySQL -------- - -.. automodule:: sqlalchemy.dialects.mysql.pymysql - -MariaDB-Connector ------------------- - -.. automodule:: sqlalchemy.dialects.mysql.mariadbconnector - -MySQL-Connector ---------------- - -.. automodule:: sqlalchemy.dialects.mysql.mysqlconnector - -.. _asyncmy: - -asyncmy -------- - -.. automodule:: sqlalchemy.dialects.mysql.asyncmy - - -.. _aiomysql: - -aiomysql --------- - -.. automodule:: sqlalchemy.dialects.mysql.aiomysql - -cymysql -------- - -.. automodule:: sqlalchemy.dialects.mysql.cymysql - -pyodbc ------- - -.. automodule:: sqlalchemy.dialects.mysql.pyodbc diff --git a/doc/build/dialects/oracle.rst b/doc/build/dialects/oracle.rst deleted file mode 100644 index 02f5122141..0000000000 --- a/doc/build/dialects/oracle.rst +++ /dev/null @@ -1,84 +0,0 @@ -.. _oracle_toplevel: - -Oracle -====== - -.. automodule:: sqlalchemy.dialects.oracle.base - -Oracle Data Types ------------------ - -As with all SQLAlchemy dialects, all UPPERCASE types that are known to be -valid with Oracle are importable from the top level dialect, whether -they originate from :mod:`sqlalchemy.types` or from the local dialect:: - - from sqlalchemy.dialects.oracle import ( - BFILE, - BLOB, - CHAR, - CLOB, - DATE, - DOUBLE_PRECISION, - FLOAT, - INTERVAL, - LONG, - NCLOB, - NCHAR, - NUMBER, - NVARCHAR, - NVARCHAR2, - RAW, - TIMESTAMP, - VARCHAR, - VARCHAR2, - ) - -.. versionadded:: 1.2.19 Added :class:`_types.NCHAR` to the list of datatypes - exported by the Oracle dialect. - -Types which are specific to Oracle, or have Oracle-specific -construction arguments, are as follows: - -.. currentmodule:: sqlalchemy.dialects.oracle - -.. autoclass:: BFILE - :members: __init__ - -.. autoclass:: DATE - :members: __init__ - -.. autoclass:: FLOAT - :members: __init__ - -.. autoclass:: INTERVAL - :members: __init__ - -.. autoclass:: NCLOB - :members: __init__ - -.. autoclass:: NUMBER - :members: __init__ - -.. autoclass:: LONG - :members: __init__ - -.. autoclass:: RAW - :members: __init__ - -.. autoclass:: TIMESTAMP - :members: __init__ - -.. _cx_oracle: - -cx_Oracle ---------- - -.. automodule:: sqlalchemy.dialects.oracle.cx_oracle - -.. _oracledb: - -python-oracledb ---------------- - -.. automodule:: sqlalchemy.dialects.oracle.oracledb - diff --git a/doc/build/dialects/postgresql.rst b/doc/build/dialects/postgresql.rst index fce0e4610e..faaff5b5a6 100644 --- a/doc/build/dialects/postgresql.rst +++ b/doc/build/dialects/postgresql.rst @@ -3,7 +3,9 @@ PostgreSQL ========== -.. automodule:: sqlalchemy.dialects.postgresql.base +The PG module + +.. .. automodule:: sqlalchemy.dialects.postgresql.base ARRAY Types ----------- @@ -533,31 +535,3 @@ PostgreSQL DML Constructs .. _postgresql_psycopg2: -psycopg2 --------- - -.. automodule:: sqlalchemy.dialects.postgresql.psycopg2 - -.. _postgresql_psycopg: - -psycopg --------- - -.. automodule:: sqlalchemy.dialects.postgresql.psycopg - -pg8000 ------- - -.. automodule:: sqlalchemy.dialects.postgresql.pg8000 - -.. _dialect-postgresql-asyncpg: - -asyncpg -------- - -.. automodule:: sqlalchemy.dialects.postgresql.asyncpg - -psycopg2cffi ------------- - -.. automodule:: sqlalchemy.dialects.postgresql.psycopg2cffi diff --git a/doc/build/dialects/sqlite.rst b/doc/build/dialects/sqlite.rst deleted file mode 100644 index d25301fa53..0000000000 --- a/doc/build/dialects/sqlite.rst +++ /dev/null @@ -1,71 +0,0 @@ -.. _sqlite_toplevel: - -SQLite -====== - -.. automodule:: sqlalchemy.dialects.sqlite.base - -SQLite Data Types ------------------ - -As with all SQLAlchemy dialects, all UPPERCASE types that are known to be -valid with SQLite are importable from the top level dialect, whether -they originate from :mod:`sqlalchemy.types` or from the local dialect:: - - from sqlalchemy.dialects.sqlite import ( - BLOB, - BOOLEAN, - CHAR, - DATE, - DATETIME, - DECIMAL, - FLOAT, - INTEGER, - NUMERIC, - JSON, - SMALLINT, - TEXT, - TIME, - TIMESTAMP, - VARCHAR, - ) - -.. module:: sqlalchemy.dialects.sqlite - -.. autoclass:: DATETIME - -.. autoclass:: DATE - -.. autoclass:: JSON - -.. autoclass:: TIME - -SQLite DML Constructs -------------------------- - -.. autofunction:: sqlalchemy.dialects.sqlite.insert - -.. autoclass:: sqlalchemy.dialects.sqlite.Insert - :members: - -.. _pysqlite: - -Pysqlite --------- - -.. automodule:: sqlalchemy.dialects.sqlite.pysqlite - -.. _aiosqlite: - -Aiosqlite ---------- - -.. automodule:: sqlalchemy.dialects.sqlite.aiosqlite - - -.. _pysqlcipher: - -Pysqlcipher ------------ - -.. automodule:: sqlalchemy.dialects.sqlite.pysqlcipher diff --git a/doc/build/orm/backref.rst b/doc/build/orm/backref.rst deleted file mode 100644 index 01f4c90736..0000000000 --- a/doc/build/orm/backref.rst +++ /dev/null @@ -1,190 +0,0 @@ -.. _relationships_backref: - -Using the legacy 'backref' relationship parameter --------------------------------------------------- - -.. note:: The :paramref:`_orm.relationship.backref` keyword should be considered - legacy, and use of :paramref:`_orm.relationship.back_populates` with explicit - :func:`_orm.relationship` constructs should be preferred. Using - individual :func:`_orm.relationship` constructs provides advantages - including that both ORM mapped classes will include their attributes - up front as the class is constructed, rather than as a deferred step, - and configuration is more straightforward as all arguments are explicit. - New :pep:`484` features in SQLAlchemy 2.0 also take advantage of - attributes being explicitly present in source code rather than - using dynamic attribute generation. - -.. seealso:: - - For general information about bidirectional relationships, see the - following sections: - - :ref:`tutorial_orm_related_objects` - in the :ref:`unified_tutorial`, - presents an overview of bi-directional relationship configuration - and behaviors using :paramref:`_orm.relationship.back_populates` - - :ref:`back_populates_cascade` - notes on bi-directional :func:`_orm.relationship` - behavior regarding :class:`_orm.Session` cascade behaviors. - - :paramref:`_orm.relationship.back_populates` - - -The :paramref:`_orm.relationship.backref` keyword argument on the -:func:`_orm.relationship` construct allows the -automatic generation of a new :func:`_orm.relationship` that will be automatically -be added to the ORM mapping for the related class. It will then be -placed into a :paramref:`_orm.relationship.back_populates` configuration -against the current :func:`_orm.relationship` being configured, with both -:func:`_orm.relationship` constructs referring to each other. - -Starting with the following example:: - - from sqlalchemy import Column, ForeignKey, Integer, String - from sqlalchemy.orm import DeclarativeBase, relationship - - - class Base(DeclarativeBase): - pass - - - class User(Base): - __tablename__ = "user" - id = mapped_column(Integer, primary_key=True) - name = mapped_column(String) - - addresses = relationship("Address", backref="user") - - - class Address(Base): - __tablename__ = "address" - id = mapped_column(Integer, primary_key=True) - email = mapped_column(String) - user_id = mapped_column(Integer, ForeignKey("user.id")) - -The above configuration establishes a collection of ``Address`` objects on ``User`` called -``User.addresses``. It also establishes a ``.user`` attribute on ``Address`` which will -refer to the parent ``User`` object. Using :paramref:`_orm.relationship.back_populates` -it's equivalent to the following:: - - from sqlalchemy import Column, ForeignKey, Integer, String - from sqlalchemy.orm import DeclarativeBase, relationship - - - class Base(DeclarativeBase): - pass - - - class User(Base): - __tablename__ = "user" - id = mapped_column(Integer, primary_key=True) - name = mapped_column(String) - - addresses = relationship("Address", back_populates="user") - - - class Address(Base): - __tablename__ = "address" - id = mapped_column(Integer, primary_key=True) - email = mapped_column(String) - user_id = mapped_column(Integer, ForeignKey("user.id")) - - user = relationship("User", back_populates="addresses") - -The behavior of the ``User.addresses`` and ``Address.user`` relationships -is that they now behave in a **bi-directional** way, indicating that -changes on one side of the relationship impact the other. An example -and discussion of this behavior is in the :ref:`unified_tutorial` -at :ref:`tutorial_orm_related_objects`. - - -Backref Default Arguments -~~~~~~~~~~~~~~~~~~~~~~~~~ - -Since :paramref:`_orm.relationship.backref` generates a whole new -:func:`_orm.relationship`, the generation process by default -will attempt to include corresponding arguments in the new -:func:`_orm.relationship` that correspond to the original arguments. -As an example, below is a :func:`_orm.relationship` that includes a -:ref:`custom join condition ` -which also includes the :paramref:`_orm.relationship.backref` keyword:: - - from sqlalchemy import Column, ForeignKey, Integer, String - from sqlalchemy.orm import DeclarativeBase, relationship - - - class Base(DeclarativeBase): - pass - - - class User(Base): - __tablename__ = "user" - id = mapped_column(Integer, primary_key=True) - name = mapped_column(String) - - addresses = relationship( - "Address", - primaryjoin=( - "and_(User.id==Address.user_id, Address.email.startswith('tony'))" - ), - backref="user", - ) - - - class Address(Base): - __tablename__ = "address" - id = mapped_column(Integer, primary_key=True) - email = mapped_column(String) - user_id = mapped_column(Integer, ForeignKey("user.id")) - -When the "backref" is generated, the :paramref:`_orm.relationship.primaryjoin` -condition is copied to the new :func:`_orm.relationship` as well:: - - >>> print(User.addresses.property.primaryjoin) - "user".id = address.user_id AND address.email LIKE :email_1 || '%%' - >>> - >>> print(Address.user.property.primaryjoin) - "user".id = address.user_id AND address.email LIKE :email_1 || '%%' - >>> - -Other arguments that are transferrable include the -:paramref:`_orm.relationship.secondary` parameter that refers to a -many-to-many association table, as well as the "join" arguments -:paramref:`_orm.relationship.primaryjoin` and -:paramref:`_orm.relationship.secondaryjoin`; "backref" is smart enough to know -that these two arguments should also be "reversed" when generating -the opposite side. - -Specifying Backref Arguments -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Lots of other arguments for a "backref" are not implicit, and -include arguments like -:paramref:`_orm.relationship.lazy`, -:paramref:`_orm.relationship.remote_side`, -:paramref:`_orm.relationship.cascade` and -:paramref:`_orm.relationship.cascade_backrefs`. For this case we use -the :func:`.backref` function in place of a string; this will store -a specific set of arguments that will be transferred to the new -:func:`_orm.relationship` when generated:: - - # - from sqlalchemy.orm import backref - - - class User(Base): - __tablename__ = "user" - id = mapped_column(Integer, primary_key=True) - name = mapped_column(String) - - addresses = relationship( - "Address", - backref=backref("user", lazy="joined"), - ) - -Where above, we placed a ``lazy="joined"`` directive only on the ``Address.user`` -side, indicating that when a query against ``Address`` is made, a join to the ``User`` -entity should be made automatically which will populate the ``.user`` attribute of each -returned ``Address``. The :func:`.backref` function formatted the arguments we gave -it into a form that is interpreted by the receiving :func:`_orm.relationship` as additional -arguments to be applied to the new relationship it creates. - diff --git a/doc/build/orm/classical.rst b/doc/build/orm/classical.rst deleted file mode 100644 index a0bc70d890..0000000000 --- a/doc/build/orm/classical.rst +++ /dev/null @@ -1,5 +0,0 @@ -:orphan: - -Moved! :ref:`orm_imperative_mapping` - - diff --git a/doc/build/orm/collections.rst b/doc/build/orm/collections.rst deleted file mode 100644 index 43cbef23a9..0000000000 --- a/doc/build/orm/collections.rst +++ /dev/null @@ -1,12 +0,0 @@ -:orphan: - -======================================= -Collection Configuration and Techniques -======================================= - -This page has been broken into two separate pages: - -:doc:`large_collections` - -:doc:`collection_api` - diff --git a/doc/build/orm/composites.rst b/doc/build/orm/composites.rst deleted file mode 100644 index 4eaf7024a9..0000000000 --- a/doc/build/orm/composites.rst +++ /dev/null @@ -1,469 +0,0 @@ -.. currentmodule:: sqlalchemy.orm - -.. _mapper_composite: - -Composite Column Types -====================== - -Sets of columns can be associated with a single user-defined datatype, -which in modern use is normally a Python dataclass_. The ORM -provides a single attribute which represents the group of columns using the -class you provide. - -A simple example represents pairs of :class:`_types.Integer` columns as a -``Point`` object, with attributes ``.x`` and ``.y``. Using a -dataclass, these attributes are defined with the corresponding ``int`` -Python type:: - - import dataclasses - - - @dataclasses.dataclass - class Point: - x: int - y: int - -Non-dataclass forms are also accepted, but require additional methods -to be implemented. For an example using a non-dataclass class, see the section -:ref:`composite_legacy_no_dataclass`. - -.. versionadded:: 2.0 The :func:`_orm.composite` construct fully supports - Python dataclasses including the ability to derive mapped column datatypes - from the composite class. - -We will create a mapping to a table ``vertices``, which represents two points -as ``x1/y1`` and ``x2/y2``. The ``Point`` class is associated with -the mapped columns using the :func:`_orm.composite` construct. - -The example below illustrates the most modern form of :func:`_orm.composite` as -used with a fully -:ref:`Annotated Declarative Table ` -configuration. :func:`_orm.mapped_column` constructs representing each column -are passed directly to :func:`_orm.composite`, indicating zero or more aspects -of the columns to be generated, in this case the names; the -:func:`_orm.composite` construct derives the column types (in this case -``int``, corresponding to :class:`_types.Integer`) from the dataclass directly:: - - from sqlalchemy.orm import DeclarativeBase, Mapped - from sqlalchemy.orm import composite, mapped_column - - - class Base(DeclarativeBase): - pass - - - class Vertex(Base): - __tablename__ = "vertices" - - id: Mapped[int] = mapped_column(primary_key=True) - - start: Mapped[Point] = composite(mapped_column("x1"), mapped_column("y1")) - end: Mapped[Point] = composite(mapped_column("x2"), mapped_column("y2")) - - def __repr__(self): - return f"Vertex(start={self.start}, end={self.end})" - -The above mapping would correspond to a CREATE TABLE statement as: - -.. sourcecode:: pycon+sql - - >>> from sqlalchemy.schema import CreateTable - >>> print(CreateTable(Vertex.__table__)) - {printsql}CREATE TABLE vertices ( - id INTEGER NOT NULL, - x1 INTEGER NOT NULL, - y1 INTEGER NOT NULL, - x2 INTEGER NOT NULL, - y2 INTEGER NOT NULL, - PRIMARY KEY (id) - ) - - -Working with Mapped Composite Column Types -------------------------------------------- - -With a mapping as illustrated in the top section, we can work with the -``Vertex`` class, where the ``.start`` and ``.end`` attributes will -transparently refer to the columns referred towards by the ``Point`` class, as -well as with instances of the ``Vertex`` class, where the ``.start`` and -``.end`` attributes will refer to instances of the ``Point`` class. The ``x1``, -``y1``, ``x2``, and ``y2`` columns are handled transparently: - -* **Persisting Point objects** - - We can create a ``Vertex`` object, assign ``Point`` objects as members, - and they will be persisted as expected: - - .. sourcecode:: pycon+sql - - >>> v = Vertex(start=Point(3, 4), end=Point(5, 6)) - >>> session.add(v) - >>> session.commit() - {execsql}BEGIN (implicit) - INSERT INTO vertices (x1, y1, x2, y2) VALUES (?, ?, ?, ?) - [generated in ...] (3, 4, 5, 6) - COMMIT - -* **Selecting Point objects as columns** - - :func:`_orm.composite` will allow the ``Vertex.start`` and ``Vertex.end`` - attributes to behave like a single SQL expression to as much an extent - as possible when using the ORM :class:`_orm.Session` (including the legacy - :class:`_orm.Query` object) to select ``Point`` objects: - - .. sourcecode:: pycon+sql - - >>> stmt = select(Vertex.start, Vertex.end) - >>> session.execute(stmt).all() - {execsql}SELECT vertices.x1, vertices.y1, vertices.x2, vertices.y2 - FROM vertices - [...] () - {stop}[(Point(x=3, y=4), Point(x=5, y=6))] - -* **Comparing Point objects in SQL expressions** - - The ``Vertex.start`` and ``Vertex.end`` attributes may be used in - WHERE criteria and similar, using ad-hoc ``Point`` objects for comparisons: - - .. sourcecode:: pycon+sql - - >>> stmt = select(Vertex).where(Vertex.start == Point(3, 4)).where(Vertex.end < Point(7, 8)) - >>> session.scalars(stmt).all() - {execsql}SELECT vertices.id, vertices.x1, vertices.y1, vertices.x2, vertices.y2 - FROM vertices - WHERE vertices.x1 = ? AND vertices.y1 = ? AND vertices.x2 < ? AND vertices.y2 < ? - [...] (3, 4, 7, 8) - {stop}[Vertex(Point(x=3, y=4), Point(x=5, y=6))] - - .. versionadded:: 2.0 :func:`_orm.composite` constructs now support - "ordering" comparisons such as ``<``, ``>=``, and similar, in addition - to the already-present support for ``==``, ``!=``. - - .. tip:: The "ordering" comparison above using the "less than" operator (``<``) - as well as the "equality" comparison using ``==``, when used to generate - SQL expressions, are implemented by the :class:`_orm.Composite.Comparator` - class, and don't make use of the comparison methods on the composite class - itself, e.g. the ``__lt__()`` or ``__eq__()`` methods. From this it - follows that the ``Point`` dataclass above also need not implement the - dataclasses ``order=True`` parameter for the above SQL operations to work. - The section :ref:`composite_operations` contains background on how - to customize the comparison operations. - -* **Updating Point objects on Vertex Instances** - - By default, the ``Point`` object **must be replaced by a new object** for - changes to be detected: - - .. sourcecode:: pycon+sql - - >>> v1 = session.scalars(select(Vertex)).one() - {execsql}SELECT vertices.id, vertices.x1, vertices.y1, vertices.x2, vertices.y2 - FROM vertices - [...] () - {stop} - - >>> v1.end = Point(x=10, y=14) - >>> session.commit() - {execsql}UPDATE vertices SET x2=?, y2=? WHERE vertices.id = ? - [...] (10, 14, 1) - COMMIT - - In order to allow in place changes on the composite object, the - :ref:`mutable_toplevel` extension must be used. See the section - :ref:`mutable_composites` for examples. - - - -.. _orm_composite_other_forms: - -Other mapping forms for composites ----------------------------------- - -The :func:`_orm.composite` construct may be passed the relevant columns -using a :func:`_orm.mapped_column` construct, a :class:`_schema.Column`, -or the string name of an existing mapped column. The following examples -illustrate an equvalent mapping as that of the main section above. - -* Map columns directly, then pass to composite - - Here we pass the existing :func:`_orm.mapped_column` instances to the - :func:`_orm.composite` construct, as in the non-annotated example below - where we also pass the ``Point`` class as the first argument to - :func:`_orm.composite`:: - - from sqlalchemy import Integer - from sqlalchemy.orm import mapped_column, composite - - - class Vertex(Base): - __tablename__ = "vertices" - - id = mapped_column(Integer, primary_key=True) - x1 = mapped_column(Integer) - y1 = mapped_column(Integer) - x2 = mapped_column(Integer) - y2 = mapped_column(Integer) - - start = composite(Point, x1, y1) - end = composite(Point, x2, y2) - -* Map columns directly, pass attribute names to composite - - We can write the same example above using more annotated forms where we have - the option to pass attribute names to :func:`_orm.composite` instead of - full column constructs:: - - from sqlalchemy.orm import mapped_column, composite, Mapped - - - class Vertex(Base): - __tablename__ = "vertices" - - id: Mapped[int] = mapped_column(primary_key=True) - x1: Mapped[int] - y1: Mapped[int] - x2: Mapped[int] - y2: Mapped[int] - - start: Mapped[Point] = composite("x1", "y1") - end: Mapped[Point] = composite("x2", "y2") - -* Imperative mapping and imperative table - - When using :ref:`imperative table ` or - fully :ref:`imperative ` mappings, we have access - to :class:`_schema.Column` objects directly. These may be passed to - :func:`_orm.composite` as well, as in the imperative example below:: - - mapper_registry.map_imperatively( - Vertex, - vertices_table, - properties={ - "start": composite(Point, vertices_table.c.x1, vertices_table.c.y1), - "end": composite(Point, vertices_table.c.x2, vertices_table.c.y2), - }, - ) - -.. _composite_legacy_no_dataclass: - -Using Legacy Non-Dataclasses ----------------------------- - - -If not using a dataclass, the requirements for the custom datatype class are -that it have a constructor -which accepts positional arguments corresponding to its column format, and -also provides a method ``__composite_values__()`` which returns the state of -the object as a list or tuple, in order of its column-based attributes. It -also should supply adequate ``__eq__()`` and ``__ne__()`` methods which test -the equality of two instances. - -To illustrate the equivalent ``Point`` class from the main section -not using a dataclass:: - - class Point: - def __init__(self, x, y): - self.x = x - self.y = y - - def __composite_values__(self): - return self.x, self.y - - def __repr__(self): - return f"Point(x={self.x!r}, y={self.y!r})" - - def __eq__(self, other): - return isinstance(other, Point) and other.x == self.x and other.y == self.y - - def __ne__(self, other): - return not self.__eq__(other) - -Usage with :func:`_orm.composite` then proceeds where the columns to be -associated with the ``Point`` class must also be declared with explicit -types, using one of the forms at :ref:`orm_composite_other_forms`. - - -Tracking In-Place Mutations on Composites ------------------------------------------ - -In-place changes to an existing composite value are -not tracked automatically. Instead, the composite class needs to provide -events to its parent object explicitly. This task is largely automated -via the usage of the :class:`.MutableComposite` mixin, which uses events -to associate each user-defined composite object with all parent associations. -Please see the example in :ref:`mutable_composites`. - -.. _composite_operations: - -Redefining Comparison Operations for Composites ------------------------------------------------ - -The "equals" comparison operation by default produces an AND of all -corresponding columns equated to one another. This can be changed using -the ``comparator_factory`` argument to :func:`.composite`, where we -specify a custom :class:`.CompositeProperty.Comparator` class -to define existing or new operations. -Below we illustrate the "greater than" operator, implementing -the same expression that the base "greater than" does:: - - import dataclasses - - from sqlalchemy.orm import composite - from sqlalchemy.orm import CompositeProperty - from sqlalchemy.orm import DeclarativeBase - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - from sqlalchemy.sql import and_ - - - @dataclasses.dataclass - class Point: - x: int - y: int - - - class PointComparator(CompositeProperty.Comparator): - def __gt__(self, other): - """redefine the 'greater than' operation""" - - return and_( - *[ - a > b - for a, b in zip( - self.__clause_element__().clauses, - dataclasses.astuple(other), - ) - ] - ) - - - class Base(DeclarativeBase): - pass - - - class Vertex(Base): - __tablename__ = "vertices" - - id: Mapped[int] = mapped_column(primary_key=True) - - start: Mapped[Point] = composite( - mapped_column("x1"), mapped_column("y1"), comparator_factory=PointComparator - ) - end: Mapped[Point] = composite( - mapped_column("x2"), mapped_column("y2"), comparator_factory=PointComparator - ) - -Since ``Point`` is a dataclass, we may make use of -``dataclasses.astuple()`` to get a tuple form of ``Point`` instances. - -The custom comparator then returns the appropriate SQL expression: - -.. sourcecode:: pycon+sql - - >>> print(Vertex.start > Point(5, 6)) - {printsql}vertices.x1 > :x1_1 AND vertices.y1 > :y1_1 - - -Nesting Composites -------------------- - -Composite objects can be defined to work in simple nested schemes, by -redefining behaviors within the composite class to work as desired, then -mapping the composite class to the full length of individual columns normally. -This requires that additional methods to move between the "nested" and -"flat" forms are defined. - -Below we reorganize the ``Vertex`` class to itself be a composite object which -refers to ``Point`` objects. ``Vertex`` and ``Point`` can be dataclasses, -however we will add a custom construction method to ``Vertex`` that can be used -to create new ``Vertex`` objects given four column values, which will will -arbitrarily name ``_generate()`` and define as a classmethod so that we can -make new ``Vertex`` objects by passing values to the ``Vertex._generate()`` -method. - -We will also implement the ``__composite_values__()`` method, which is a fixed -name recognized by the :func:`_orm.composite` construct (introduced previously -at :ref:`composite_legacy_no_dataclass`) that indicates a standard way of -receiving the object as a flat tuple of column values, which in this case will -supersede the usual dataclass-oriented methodology. - -With our custom ``_generate()`` constructor and -``__composite_values__()`` serializer method, we can now move between -a flat tuple of columns and ``Vertex`` objects that contain ``Point`` -instances. The ``Vertex._generate`` method is passed as the -first argument to the :func:`_orm.composite` construct as the source of new -``Vertex`` instances, and the ``__composite_values__()`` method will be -used implicitly by :func:`_orm.composite`. - -For the purposes of the example, the ``Vertex`` composite is then mapped to a -class called ``HasVertex``, which is where the :class:`.Table` containing the -four source columns ultimately resides:: - - from __future__ import annotations - - import dataclasses - from typing import Any - from typing import Tuple - - from sqlalchemy.orm import composite - from sqlalchemy.orm import DeclarativeBase - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - - - @dataclasses.dataclass - class Point: - x: int - y: int - - - @dataclasses.dataclass - class Vertex: - start: Point - end: Point - - @classmethod - def _generate(cls, x1: int, y1: int, x2: int, y2: int) -> Vertex: - """generate a Vertex from a row""" - return Vertex(Point(x1, y1), Point(x2, y2)) - - def __composite_values__(self) -> Tuple[Any, ...]: - """generate a row from a Vertex""" - return dataclasses.astuple(self.start) + dataclasses.astuple(self.end) - - - class Base(DeclarativeBase): - pass - - - class HasVertex(Base): - __tablename__ = "has_vertex" - id: Mapped[int] = mapped_column(primary_key=True) - x1: Mapped[int] - y1: Mapped[int] - x2: Mapped[int] - y2: Mapped[int] - - vertex: Mapped[Vertex] = composite(Vertex._generate, "x1", "y1", "x2", "y2") - -The above mapping can then be used in terms of ``HasVertex``, ``Vertex``, and -``Point``:: - - hv = HasVertex(vertex=Vertex(Point(1, 2), Point(3, 4))) - - session.add(hv) - session.commit() - - stmt = select(HasVertex).where(HasVertex.vertex == Vertex(Point(1, 2), Point(3, 4))) - - hv = session.scalars(stmt).first() - print(hv.vertex.start) - print(hv.vertex.end) - -.. _dataclass: https://docs.python.org/3/library/dataclasses.html - -Composite API -------------- - -.. autofunction:: composite - diff --git a/doc/build/orm/contextual.rst b/doc/build/orm/contextual.rst deleted file mode 100644 index 3e03e93167..0000000000 --- a/doc/build/orm/contextual.rst +++ /dev/null @@ -1,283 +0,0 @@ -.. _unitofwork_contextual: - -Contextual/Thread-local Sessions -================================ - -Recall from the section :ref:`session_faq_whentocreate`, the concept of -"session scopes" was introduced, with an emphasis on web applications -and the practice of linking the scope of a :class:`.Session` with that -of a web request. Most modern web frameworks include integration tools -so that the scope of the :class:`.Session` can be managed automatically, -and these tools should be used as they are available. - -SQLAlchemy includes its own helper object, which helps with the establishment -of user-defined :class:`.Session` scopes. It is also used by third-party -integration systems to help construct their integration schemes. - -The object is the :class:`.scoped_session` object, and it represents a -**registry** of :class:`.Session` objects. If you're not familiar with the -registry pattern, a good introduction can be found in `Patterns of Enterprise -Architecture `_. - -.. warning:: - - The :class:`.scoped_session` registry by default uses a Python - ``threading.local()`` - in order to track :class:`_orm.Session` instances. **This is not - necessarily compatible with all application servers**, particularly those - which make use of greenlets or other alternative forms of concurrency - control, which may lead to race conditions (e.g. randomly occurring - failures) when used in moderate to high concurrency scenarios. - Please read :ref:`unitofwork_contextual_threadlocal` and - :ref:`session_lifespan` below to more fully understand the implications - of using ``threading.local()`` to track :class:`_orm.Session` objects - and consider more explicit means of scoping when using application servers - which are not based on traditional threads. - -.. note:: - - The :class:`.scoped_session` object is a very popular and useful object - used by many SQLAlchemy applications. However, it is important to note - that it presents **only one approach** to the issue of :class:`.Session` - management. If you're new to SQLAlchemy, and especially if the - term "thread-local variable" seems strange to you, we recommend that - if possible you familiarize first with an off-the-shelf integration - system such as `Flask-SQLAlchemy `_ - or `zope.sqlalchemy `_. - -A :class:`.scoped_session` is constructed by calling it, passing it a -**factory** which can create new :class:`.Session` objects. A factory -is just something that produces a new object when called, and in the -case of :class:`.Session`, the most common factory is the :class:`.sessionmaker`, -introduced earlier in this section. Below we illustrate this usage:: - - >>> from sqlalchemy.orm import scoped_session - >>> from sqlalchemy.orm import sessionmaker - - >>> session_factory = sessionmaker(bind=some_engine) - >>> Session = scoped_session(session_factory) - -The :class:`.scoped_session` object we've created will now call upon the -:class:`.sessionmaker` when we "call" the registry:: - - >>> some_session = Session() - -Above, ``some_session`` is an instance of :class:`.Session`, which we -can now use to talk to the database. This same :class:`.Session` is also -present within the :class:`.scoped_session` registry we've created. If -we call upon the registry a second time, we get back the **same** :class:`.Session`:: - - >>> some_other_session = Session() - >>> some_session is some_other_session - True - -This pattern allows disparate sections of the application to call upon a global -:class:`.scoped_session`, so that all those areas may share the same session -without the need to pass it explicitly. The :class:`.Session` we've established -in our registry will remain, until we explicitly tell our registry to dispose of it, -by calling :meth:`.scoped_session.remove`:: - - >>> Session.remove() - -The :meth:`.scoped_session.remove` method first calls :meth:`.Session.close` on -the current :class:`.Session`, which has the effect of releasing any connection/transactional -resources owned by the :class:`.Session` first, then discarding the :class:`.Session` -itself. "Releasing" here means that connections are returned to their connection pool and any transactional state is rolled back, ultimately using the ``rollback()`` method of the underlying DBAPI connection. - -At this point, the :class:`.scoped_session` object is "empty", and will create -a **new** :class:`.Session` when called again. As illustrated below, this -is not the same :class:`.Session` we had before:: - - >>> new_session = Session() - >>> new_session is some_session - False - -The above series of steps illustrates the idea of the "registry" pattern in a -nutshell. With that basic idea in hand, we can discuss some of the details -of how this pattern proceeds. - -Implicit Method Access ----------------------- - -The job of the :class:`.scoped_session` is simple; hold onto a :class:`.Session` -for all who ask for it. As a means of producing more transparent access to this -:class:`.Session`, the :class:`.scoped_session` also includes **proxy behavior**, -meaning that the registry itself can be treated just like a :class:`.Session` -directly; when methods are called on this object, they are **proxied** to the -underlying :class:`.Session` being maintained by the registry:: - - Session = scoped_session(some_factory) - - # equivalent to: - # - # session = Session() - # print(session.scalars(select(MyClass)).all()) - # - print(Session.scalars(select(MyClass)).all()) - -The above code accomplishes the same task as that of acquiring the current -:class:`.Session` by calling upon the registry, then using that :class:`.Session`. - -.. _unitofwork_contextual_threadlocal: - -Thread-Local Scope ------------------- - -Users who are familiar with multithreaded programming will note that representing -anything as a global variable is usually a bad idea, as it implies that the -global object will be accessed by many threads concurrently. The :class:`.Session` -object is entirely designed to be used in a **non-concurrent** fashion, which -in terms of multithreading means "only in one thread at a time". So our -above example of :class:`.scoped_session` usage, where the same :class:`.Session` -object is maintained across multiple calls, suggests that some process needs -to be in place such that multiple calls across many threads don't actually get -a handle to the same session. We call this notion **thread local storage**, -which means, a special object is used that will maintain a distinct object -per each application thread. Python provides this via the -`threading.local() `_ -construct. The :class:`.scoped_session` object by default uses this object -as storage, so that a single :class:`.Session` is maintained for all who call -upon the :class:`.scoped_session` registry, but only within the scope of a single -thread. Callers who call upon the registry in a different thread get a -:class:`.Session` instance that is local to that other thread. - -Using this technique, the :class:`.scoped_session` provides a quick and relatively -simple (if one is familiar with thread-local storage) way of providing -a single, global object in an application that is safe to be called upon -from multiple threads. - -The :meth:`.scoped_session.remove` method, as always, removes the current -:class:`.Session` associated with the thread, if any. However, one advantage of the -``threading.local()`` object is that if the application thread itself ends, the -"storage" for that thread is also garbage collected. So it is in fact "safe" to -use thread local scope with an application that spawns and tears down threads, -without the need to call :meth:`.scoped_session.remove`. However, the scope -of transactions themselves, i.e. ending them via :meth:`.Session.commit` or -:meth:`.Session.rollback`, will usually still be something that must be explicitly -arranged for at the appropriate time, unless the application actually ties the -lifespan of a thread to the lifespan of a transaction. - -.. _session_lifespan: - -Using Thread-Local Scope with Web Applications ----------------------------------------------- - -As discussed in the section :ref:`session_faq_whentocreate`, a web application -is architected around the concept of a **web request**, and integrating -such an application with the :class:`.Session` usually implies that the :class:`.Session` -will be associated with that request. As it turns out, most Python web frameworks, -with notable exceptions such as the asynchronous frameworks Twisted and -Tornado, use threads in a simple way, such that a particular web request is received, -processed, and completed within the scope of a single *worker thread*. When -the request ends, the worker thread is released to a pool of workers where it -is available to handle another request. - -This simple correspondence of web request and thread means that to associate a -:class:`.Session` with a thread implies it is also associated with the web request -running within that thread, and vice versa, provided that the :class:`.Session` is -created only after the web request begins and torn down just before the web request ends. -So it is a common practice to use :class:`.scoped_session` as a quick way -to integrate the :class:`.Session` with a web application. The sequence -diagram below illustrates this flow: - -.. sourcecode:: text - - Web Server Web Framework SQLAlchemy ORM Code - -------------- -------------- ------------------------------ - startup -> Web framework # Session registry is established - initializes Session = scoped_session(sessionmaker()) - - incoming - web request -> web request -> # The registry is *optionally* - starts # called upon explicitly to create - # a Session local to the thread and/or request - Session() - - # the Session registry can otherwise - # be used at any time, creating the - # request-local Session() if not present, - # or returning the existing one - Session.execute(select(MyClass)) # ... - - Session.add(some_object) # ... - - # if data was modified, commit the - # transaction - Session.commit() - - web request ends -> # the registry is instructed to - # remove the Session - Session.remove() - - sends output <- - outgoing web <- - response - -Using the above flow, the process of integrating the :class:`.Session` with the -web application has exactly two requirements: - -1. Create a single :class:`.scoped_session` registry when the web application - first starts, ensuring that this object is accessible by the rest of the - application. -2. Ensure that :meth:`.scoped_session.remove` is called when the web request ends, - usually by integrating with the web framework's event system to establish - an "on request end" event. - -As noted earlier, the above pattern is **just one potential way** to integrate a :class:`.Session` -with a web framework, one which in particular makes the significant assumption -that the **web framework associates web requests with application threads**. It is -however **strongly recommended that the integration tools provided with the web framework -itself be used, if available**, instead of :class:`.scoped_session`. - -In particular, while using a thread local can be convenient, it is preferable that the :class:`.Session` be -associated **directly with the request**, rather than with -the current thread. The next section on custom scopes details a more advanced configuration -which can combine the usage of :class:`.scoped_session` with direct request based scope, or -any kind of scope. - -Using Custom Created Scopes ---------------------------- - -The :class:`.scoped_session` object's default behavior of "thread local" scope is only -one of many options on how to "scope" a :class:`.Session`. A custom scope can be defined -based on any existing system of getting at "the current thing we are working with". - -Suppose a web framework defines a library function ``get_current_request()``. An application -built using this framework can call this function at any time, and the result will be -some kind of ``Request`` object that represents the current request being processed. -If the ``Request`` object is hashable, then this function can be easily integrated with -:class:`.scoped_session` to associate the :class:`.Session` with the request. Below we illustrate -this in conjunction with a hypothetical event marker provided by the web framework -``on_request_end``, which allows code to be invoked whenever a request ends:: - - from my_web_framework import get_current_request, on_request_end - from sqlalchemy.orm import scoped_session, sessionmaker - - Session = scoped_session(sessionmaker(bind=some_engine), scopefunc=get_current_request) - - - @on_request_end - def remove_session(req): - Session.remove() - -Above, we instantiate :class:`.scoped_session` in the usual way, except that we pass -our request-returning function as the "scopefunc". This instructs :class:`.scoped_session` -to use this function to generate a dictionary key whenever the registry is called upon -to return the current :class:`.Session`. In this case it is particularly important -that we ensure a reliable "remove" system is implemented, as this dictionary is not -otherwise self-managed. - - -Contextual Session API ----------------------- - -.. autoclass:: sqlalchemy.orm.scoped_session - :members: - :inherited-members: - -.. autoclass:: sqlalchemy.util.ScopedRegistry - :members: - -.. autoclass:: sqlalchemy.util.ThreadLocalRegistry - -.. autoclass:: sqlalchemy.orm.QueryPropertyDescriptor diff --git a/doc/build/orm/dataclasses.rst b/doc/build/orm/dataclasses.rst deleted file mode 100644 index bbd05fcd5e..0000000000 --- a/doc/build/orm/dataclasses.rst +++ /dev/null @@ -1,1085 +0,0 @@ -.. _orm_dataclasses_toplevel: - -====================================== -Integration with dataclasses and attrs -====================================== - -SQLAlchemy as of version 2.0 features "native dataclass" integration where -an :ref:`Annotated Declarative Table ` -mapping may be turned into a Python dataclass_ by adding a single mixin -or decorator to mapped classes. - -.. versionadded:: 2.0 Integrated dataclass creation with ORM Declarative classes - -There are also patterns available that allow existing dataclasses to be -mapped, as well as to map classes instrumented by the -attrs_ third party integration library. - -.. _orm_declarative_native_dataclasses: - -Declarative Dataclass Mapping -------------------------------- - -SQLAlchemy :ref:`Annotated Declarative Table ` -mappings may be augmented with an additional -mixin class or decorator directive, which will add an additional step to -the Declarative process after the mapping is complete that will convert -the mapped class **in-place** into a Python dataclass_, before completing -the mapping process which applies ORM-specific :term:`instrumentation` -to the class. The most prominent behavioral addition this provides is -generation of an ``__init__()`` method with fine-grained control over -positional and keyword arguments with or without defaults, as well as -generation of methods like ``__repr__()`` and ``__eq__()``. - -From a :pep:`484` typing perspective, the class is recognized -as having Dataclass-specific behaviors, most notably by taking advantage of :pep:`681` -"Dataclass Transforms", which allows typing tools to consider the class -as though it were explicitly decorated using the ``@dataclasses.dataclass`` -decorator. - -.. note:: Support for :pep:`681` in typing tools as of **April 4, 2023** is - limited and is currently known to be supported by Pyright_ as well - as Mypy_ as of **version 1.2**. Note that Mypy 1.1.1 introduced - :pep:`681` support but did not correctly accommodate Python descriptors - which will lead to errors when using SQLAlhcemy's ORM mapping scheme. - - .. seealso:: - - https://peps.python.org/pep-0681/#the-dataclass-transform-decorator - background - on how libraries like SQLAlchemy enable :pep:`681` support - - -Dataclass conversion may be added to any Declarative class either by adding the -:class:`_orm.MappedAsDataclass` mixin to a :class:`_orm.DeclarativeBase` class -hierarchy, or for decorator mapping by using the -:meth:`_orm.registry.mapped_as_dataclass` class decorator. - -The :class:`_orm.MappedAsDataclass` mixin may be applied either -to the Declarative ``Base`` class or any superclass, as in the example -below:: - - - from sqlalchemy.orm import DeclarativeBase - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - from sqlalchemy.orm import MappedAsDataclass - - - class Base(MappedAsDataclass, DeclarativeBase): - """subclasses will be converted to dataclasses""" - - - class User(Base): - __tablename__ = "user_account" - - id: Mapped[int] = mapped_column(init=False, primary_key=True) - name: Mapped[str] - -Or may be applied directly to classes that extend from the Declarative base:: - - from sqlalchemy.orm import DeclarativeBase - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - from sqlalchemy.orm import MappedAsDataclass - - - class Base(DeclarativeBase): - pass - - - class User(MappedAsDataclass, Base): - """User class will be converted to a dataclass""" - - __tablename__ = "user_account" - - id: Mapped[int] = mapped_column(init=False, primary_key=True) - name: Mapped[str] - -When using the decorator form, only the :meth:`_orm.registry.mapped_as_dataclass` -decorator is supported:: - - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - from sqlalchemy.orm import registry - - - reg = registry() - - - @reg.mapped_as_dataclass - class User: - __tablename__ = "user_account" - - id: Mapped[int] = mapped_column(init=False, primary_key=True) - name: Mapped[str] - -Class level feature configuration -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Support for dataclasses features is partial. Currently **supported** are -the ``init``, ``repr``, ``eq``, ``order`` and ``unsafe_hash`` features, -``match_args`` and ``kw_only`` are supported on Python 3.10+. -Currently **not supported** are the ``frozen`` and ``slots`` features. - -When using the mixin class form with :class:`_orm.MappedAsDataclass`, -class configuration arguments are passed as class-level parameters:: - - from sqlalchemy.orm import DeclarativeBase - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - from sqlalchemy.orm import MappedAsDataclass - - - class Base(DeclarativeBase): - pass - - - class User(MappedAsDataclass, Base, repr=False, unsafe_hash=True): - """User class will be converted to a dataclass""" - - __tablename__ = "user_account" - - id: Mapped[int] = mapped_column(init=False, primary_key=True) - name: Mapped[str] - -When using the decorator form with :meth:`_orm.registry.mapped_as_dataclass`, -class configuration arguments are passed to the decorator directly:: - - from sqlalchemy.orm import registry - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - - - reg = registry() - - - @reg.mapped_as_dataclass(unsafe_hash=True) - class User: - """User class will be converted to a dataclass""" - - __tablename__ = "user_account" - - id: Mapped[int] = mapped_column(init=False, primary_key=True) - name: Mapped[str] - -For background on dataclass class options, see the dataclasses_ documentation -at `@dataclasses.dataclass `_. - -Attribute Configuration -^^^^^^^^^^^^^^^^^^^^^^^ - -SQLAlchemy native dataclasses differ from normal dataclasses in that -attributes to be mapped are described using the :class:`_orm.Mapped` -generic annotation container in all cases. Mappings follow the same -forms as those documented at :ref:`orm_declarative_table`, and all -features of :func:`_orm.mapped_column` and :class:`_orm.Mapped` are supported. - -Additionally, ORM attribute configuration constructs including -:func:`_orm.mapped_column`, :func:`_orm.relationship` and :func:`_orm.composite` -support **per-attribute field options**, including ``init``, ``default``, -``default_factory`` and ``repr``. The names of these arguments is fixed -as specified in :pep:`681`. Functionality is equivalent to dataclasses: - -* ``init``, as in :paramref:`_orm.mapped_column.init`, - :paramref:`_orm.relationship.init`, if False indicates the field should - not be part of the ``__init__()`` method -* ``default``, as in :paramref:`_orm.mapped_column.default`, - :paramref:`_orm.relationship.default` - indicates a default value for the field as given as a keyword argument - in the ``__init__()`` method. -* ``default_factory``, as in :paramref:`_orm.mapped_column.default_factory`, - :paramref:`_orm.relationship.default_factory`, indicates a callable function - that will be invoked to generate a new default value for a parameter - if not passed explicitly to the ``__init__()`` method. -* ``repr`` True by default, indicates the field should be part of the generated - ``__repr__()`` method - - -Another key difference from dataclasses is that default values for attributes -**must** be configured using the ``default`` parameter of the ORM construct, -such as ``mapped_column(default=None)``. A syntax that resembles dataclass -syntax which accepts simple Python values as defaults without using -``@dataclases.field()`` is not supported. - -As an example using :func:`_orm.mapped_column`, the mapping below will -produce an ``__init__()`` method that accepts only the fields ``name`` and -``fullname``, where ``name`` is required and may be passed positionally, -and ``fullname`` is optional. The ``id`` field, which we expect to be -database-generated, is not part of the constructor at all:: - - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - from sqlalchemy.orm import registry - - reg = registry() - - - @reg.mapped_as_dataclass - class User: - __tablename__ = "user_account" - - id: Mapped[int] = mapped_column(init=False, primary_key=True) - name: Mapped[str] - fullname: Mapped[str] = mapped_column(default=None) - - - # 'fullname' is optional keyword argument - u1 = User("name") - -Column Defaults -~~~~~~~~~~~~~~~ - -In order to accommodate the name overlap of the ``default`` argument with -the existing :paramref:`_schema.Column.default` parameter of the :class:`_schema.Column` -construct, the :func:`_orm.mapped_column` construct disambiguates the two -names by adding a new parameter :paramref:`_orm.mapped_column.insert_default`, -which will be populated directly into the -:paramref:`_schema.Column.default` parameter of :class:`_schema.Column`, -independently of what may be set on -:paramref:`_orm.mapped_column.default`, which is always used for the -dataclasses configuration. For example, to configure a datetime column with -a :paramref:`_schema.Column.default` set to the ``func.utc_timestamp()`` SQL function, -but where the parameter is optional in the constructor:: - - from datetime import datetime - - from sqlalchemy import func - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - from sqlalchemy.orm import registry - - reg = registry() - - - @reg.mapped_as_dataclass - class User: - __tablename__ = "user_account" - - id: Mapped[int] = mapped_column(init=False, primary_key=True) - created_at: Mapped[datetime] = mapped_column( - insert_default=func.utc_timestamp(), default=None - ) - -With the above mapping, an ``INSERT`` for a new ``User`` object where no -parameter for ``created_at`` were passed proceeds as: - -.. sourcecode:: pycon+sql - - >>> with Session(e) as session: - ... session.add(User()) - ... session.commit() - {execsql}BEGIN (implicit) - INSERT INTO user_account (created_at) VALUES (utc_timestamp()) - [generated in 0.00010s] () - COMMIT - - - -Integration with Annotated -~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The approach introduced at :ref:`orm_declarative_mapped_column_pep593` illustrates -how to use :pep:`593` ``Annotated`` objects to package whole -:func:`_orm.mapped_column` constructs for re-use. This feature is supported -with the dataclasses feature. One aspect of the feature however requires -a workaround when working with typing tools, which is that the -:pep:`681`-specific arguments ``init``, ``default``, ``repr``, and ``default_factory`` -**must** be on the right hand side, packaged into an explicit :func:`_orm.mapped_column` -construct, in order for the typing tool to interpret the attribute correctly. -As an example, the approach below will work perfectly fine at runtime, -however typing tools will consider the ``User()`` construction to be -invalid, as they do not see the ``init=False`` parameter present:: - - from typing import Annotated - - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - from sqlalchemy.orm import registry - - # typing tools will ignore init=False here - intpk = Annotated[int, mapped_column(init=False, primary_key=True)] - - reg = registry() - - - @reg.mapped_as_dataclass - class User: - __tablename__ = "user_account" - id: Mapped[intpk] - - - # typing error: Argument missing for parameter "id" - u1 = User() - -Instead, :func:`_orm.mapped_column` must be present on the right side -as well with an explicit setting for :paramref:`_orm.mapped_column.init`; -the other arguments can remain within the ``Annotated`` construct:: - - from typing import Annotated - - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - from sqlalchemy.orm import registry - - intpk = Annotated[int, mapped_column(primary_key=True)] - - reg = registry() - - - @reg.mapped_as_dataclass - class User: - __tablename__ = "user_account" - - # init=False and other pep-681 arguments must be inline - id: Mapped[intpk] = mapped_column(init=False) - - - u1 = User() - -.. _orm_declarative_dc_mixins: - -Using mixins and abstract superclasses -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Any mixins or base classes that are used in a :class:`_orm.MappedAsDataclass` -mapped class which include :class:`_orm.Mapped` attributes must themselves be -part of a :class:`_orm.MappedAsDataclass` -hierarchy, such as in the example below using a mixin:: - - - class Mixin(MappedAsDataclass): - - create_user: Mapped[int] = mapped_column() - update_user: Mapped[Optional[int]] = mapped_column(default=None, init=False) - - - class Base(DeclarativeBase, MappedAsDataclass): - pass - - - class User(Base, Mixin): - __tablename__ = "sys_user" - - uid: Mapped[str] = mapped_column( - String(50), init=False, default_factory=uuid4, primary_key=True - ) - username: Mapped[str] = mapped_column() - email: Mapped[str] = mapped_column() - -Python type checkers which support :pep:`681` will otherwise not consider -attributes from non-dataclass mixins to be part of the dataclass. - -.. deprecated:: 2.0.8 Using mixins and abstract bases within - :class:`_orm.MappedAsDataclass` or - :meth:`_orm.registry.mapped_as_dataclass` hierarchies which are not - themselves dataclasses is deprecated, as these fields are not supported - by :pep:`681` as belonging to the dataclass. A warning is emitted for this - case which will later be an error. - - .. seealso:: - - :ref:`error_dcmx` - background on rationale - - - - -Relationship Configuration -^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The :class:`_orm.Mapped` annotation in combination with -:func:`_orm.relationship` is used in the same way as described at -:ref:`relationship_patterns`. When specifying a collection-based -:func:`_orm.relationship` as an optional keyword argument, the -:paramref:`_orm.relationship.default_factory` parameter must be passed and it -must refer to the collection class that's to be used. Many-to-one and -scalar object references may make use of -:paramref:`_orm.relationship.default` if the default value is to be ``None``:: - - from typing import List - - from sqlalchemy import ForeignKey - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - from sqlalchemy.orm import registry - from sqlalchemy.orm import relationship - - reg = registry() - - - @reg.mapped_as_dataclass - class Parent: - __tablename__ = "parent" - id: Mapped[int] = mapped_column(primary_key=True) - children: Mapped[List["Child"]] = relationship( - default_factory=list, back_populates="parent" - ) - - - @reg.mapped_as_dataclass - class Child: - __tablename__ = "child" - id: Mapped[int] = mapped_column(primary_key=True) - parent_id: Mapped[int] = mapped_column(ForeignKey("parent.id")) - parent: Mapped["Parent"] = relationship(default=None) - -The above mapping will generate an empty list for ``Parent.children`` when a -new ``Parent()`` object is constructed without passing ``children``, and -similarly a ``None`` value for ``Child.parent`` when a new ``Child()`` object -is constructed without passsing ``parent``. - -While the :paramref:`_orm.relationship.default_factory` can be automatically -derived from the given collection class of the :func:`_orm.relationship` -itself, this would break compatibility with dataclasses, as the presence -of :paramref:`_orm.relationship.default_factory` or -:paramref:`_orm.relationship.default` is what determines if the parameter is -to be required or optional when rendered into the ``__init__()`` method. - -.. _orm_declarative_native_dataclasses_non_mapped_fields: - -Using Non-Mapped Dataclass Fields -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -When using Declarative dataclasses, non-mapped fields may be used on the -class as well, which will be part of the dataclass construction process but -will not be mapped. Any field that does not use :class:`.Mapped` will -be ignored by the mapping process. In the example below, the fields -``ctrl_one`` and ``ctrl_two`` will be part of the instance-level state -of the object, but will not be persisted by the ORM:: - - - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - from sqlalchemy.orm import registry - - reg = registry() - - - @reg.mapped_as_dataclass - class Data: - __tablename__ = "data" - - id: Mapped[int] = mapped_column(init=False, primary_key=True) - status: Mapped[str] - - ctrl_one: Optional[str] = None - ctrl_two: Optional[str] = None - -Instance of ``Data`` above can be created as:: - - d1 = Data(status="s1", ctrl_one="ctrl1", ctrl_two="ctrl2") - -A more real world example might be to make use of the Dataclasses -``InitVar`` feature in conjunction with the ``__post_init__()`` feature to -receive init-only fields that can be used to compose persisted data. -In the example below, the ``User`` -class is declared using ``id``, ``name`` and ``password_hash`` as mapped features, -but makes use of init-only ``password`` and ``repeat_password`` fields to -represent the user creation process (note: to run this example, replace -the function ``your_crypt_function_here()`` with a third party crypt -function, such as `bcrypt `_ or -`argon2-cffi `_):: - - from dataclasses import InitVar - from typing import Optional - - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - from sqlalchemy.orm import registry - - reg = registry() - - - @reg.mapped_as_dataclass - class User: - __tablename__ = "user_account" - - id: Mapped[int] = mapped_column(init=False, primary_key=True) - name: Mapped[str] - - password: InitVar[str] - repeat_password: InitVar[str] - - password_hash: Mapped[str] = mapped_column(init=False, nullable=False) - - def __post_init__(self, password: str, repeat_password: str): - if password != repeat_password: - raise ValueError("passwords do not match") - - self.password_hash = your_crypt_function_here(password) - -The above object is created with parameters ``password`` and -``repeat_password``, which are consumed up front so that the ``password_hash`` -variable may be generated:: - - >>> u1 = User(name="some_user", password="xyz", repeat_password="xyz") - >>> u1.password_hash - '$6$9ppc... (example crypted string....)' - -.. versionchanged:: 2.0.0rc1 When using :meth:`_orm.registry.mapped_as_dataclass` - or :class:`.MappedAsDataclass`, fields that do not include the - :class:`.Mapped` annotation may be included, which will be treated as part - of the resulting dataclass but not be mapped, without the need to - also indicate the ``__allow_unmapped__`` class attribute. Previous 2.0 - beta releases would require this attribute to be explicitly present, - even though the purpose of this attribute was only to allow legacy - ORM typed mappings to continue to function. - -.. _dataclasses_pydantic: - -Integrating with Alternate Dataclass Providers such as Pydantic -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -SQLAlchemy's :class:`_orm.MappedAsDataclass` class -and :meth:`_orm.registry.mapped_as_dataclass` method call directly into -the Python standard library ``dataclasses.dataclass`` class decorator, after -the declarative mapping process has been applied to the class. This -function call may be swapped out for alternateive dataclasses providers, -such as that of Pydantic, using the ``dataclass_callable`` parameter -accepted by :class:`_orm.MappedAsDataclass` as a class keyword argument -as well as by :meth:`_orm.registry.mapped_as_dataclass`:: - - from sqlalchemy.orm import DeclarativeBase - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - from sqlalchemy.orm import MappedAsDataclass - from sqlalchemy.orm import registry - - - class Base( - MappedAsDataclass, - DeclarativeBase, - dataclass_callable=pydantic.dataclasses.dataclass, - ): - pass - - - class User(Base): - __tablename__ = "user" - - id: Mapped[int] = mapped_column(primary_key=True) - name: Mapped[str] - -The above ``User`` class will be applied as a dataclass, using Pydantic's -``pydantic.dataclasses.dataclasses`` callable. The process is available -both for mapped classes as well as mixins that extend from -:class:`_orm.MappedAsDataclass` or which have -:meth:`_orm.registry.mapped_as_dataclass` applied directly. - -.. versionadded:: 2.0.4 Added the ``dataclass_callable`` class and method - parameters for :class:`_orm.MappedAsDataclass` and - :meth:`_orm.registry.mapped_as_dataclass`, and adjusted some of the - dataclass internals to accommodate more strict dataclass functions such as - that of Pydantic. - - -.. _orm_declarative_dataclasses: - -Applying ORM Mappings to an existing dataclass (legacy dataclass use) ---------------------------------------------------------------------- - -.. legacy:: - - The approaches described here are superseded by - the :ref:`orm_declarative_native_dataclasses` feature new in the 2.0 - series of SQLAlchemy. This newer version of the feature builds upon - the dataclass support first added in version 1.4, which is described - in this section. - -To map an existing dataclass, SQLAlchemy's "inline" declarative directives -cannot be used directly; ORM directives are assigned using one of three -techniques: - -* Using "Declarative with Imperative Table", the table / column to be mapped - is defined using a :class:`_schema.Table` object assigned to the - ``__table__`` attribute of the class; relationships are defined within - ``__mapper_args__`` dictionary. The class is mapped using the - :meth:`_orm.registry.mapped` decorator. An example is below at - :ref:`orm_declarative_dataclasses_imperative_table`. - -* Using full "Declarative", the Declarative-interpreted directives such as - :class:`_schema.Column`, :func:`_orm.relationship` are added to the - ``.metadata`` dictionary of the ``dataclasses.field()`` construct, where - they are consumed by the declarative process. The class is again - mapped using the :meth:`_orm.registry.mapped` decorator. See the example - below at :ref:`orm_declarative_dataclasses_declarative_table`. - -* An "Imperative" mapping can be applied to an existing dataclass using - the :meth:`_orm.registry.map_imperatively` method to produce the mapping - in exactly the same way as described at :ref:`orm_imperative_mapping`. - This is illustrated below at :ref:`orm_imperative_dataclasses`. - -The general process by which SQLAlchemy applies mappings to a dataclass -is the same as that of an ordinary class, but also includes that -SQLAlchemy will detect class-level attributes that were part of the -dataclasses declaration process and replace them at runtime with -the usual SQLAlchemy ORM mapped attributes. The ``__init__`` method that -would have been generated by dataclasses is left intact, as is the same -for all the other methods that dataclasses generates such as -``__eq__()``, ``__repr__()``, etc. - -.. _orm_declarative_dataclasses_imperative_table: - -Mapping pre-existing dataclasses using Declarative With Imperative Table -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -An example of a mapping using ``@dataclass`` using -:ref:`orm_imperative_table_configuration` is below. A complete -:class:`_schema.Table` object is constructed explicitly and assigned to the -``__table__`` attribute. Instance fields are defined using normal dataclass -syntaxes. Additional :class:`.MapperProperty` -definitions such as :func:`.relationship`, are placed in the -:ref:`__mapper_args__ ` class-level -dictionary underneath the ``properties`` key, corresponding to the -:paramref:`_orm.Mapper.properties` parameter:: - - from __future__ import annotations - - from dataclasses import dataclass, field - from typing import List, Optional - - from sqlalchemy import Column, ForeignKey, Integer, String, Table - from sqlalchemy.orm import registry, relationship - - mapper_registry = registry() - - - @mapper_registry.mapped - @dataclass - class User: - __table__ = Table( - "user", - mapper_registry.metadata, - Column("id", Integer, primary_key=True), - Column("name", String(50)), - Column("fullname", String(50)), - Column("nickname", String(12)), - ) - id: int = field(init=False) - name: Optional[str] = None - fullname: Optional[str] = None - nickname: Optional[str] = None - addresses: List[Address] = field(default_factory=list) - - __mapper_args__ = { # type: ignore - "properties": { - "addresses": relationship("Address"), - } - } - - - @mapper_registry.mapped - @dataclass - class Address: - __table__ = Table( - "address", - mapper_registry.metadata, - Column("id", Integer, primary_key=True), - Column("user_id", Integer, ForeignKey("user.id")), - Column("email_address", String(50)), - ) - id: int = field(init=False) - user_id: int = field(init=False) - email_address: Optional[str] = None - -In the above example, the ``User.id``, ``Address.id``, and ``Address.user_id`` -attributes are defined as ``field(init=False)``. This means that parameters for -these won't be added to ``__init__()`` methods, but -:class:`.Session` will still be able to set them after getting their values -during flush from autoincrement or other default value generator. To -allow them to be specified in the constructor explicitly, they would instead -be given a default value of ``None``. - -For a :func:`_orm.relationship` to be declared separately, it needs to be -specified directly within the :paramref:`_orm.Mapper.properties` dictionary -which itself is specified within the ``__mapper_args__`` dictionary, so that it -is passed to the constructor for :class:`_orm.Mapper`. An alternative to this -approach is in the next example. - -.. _orm_declarative_dataclasses_declarative_table: - -Mapping pre-existing dataclasses using Declarative-style fields -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. legacy:: This approach to Declarative mapping with - dataclasses should be considered as legacy. It will remain supported - however is unlikely to offer any advantages against the new - approach detailed at :ref:`orm_declarative_native_dataclasses`. - - Note that **mapped_column() is not supported with this use**; - the :class:`_schema.Column` construct should continue to be used to declare - table metadata within the ``metadata`` field of ``dataclasses.field()``. - -The fully declarative approach requires that :class:`_schema.Column` objects -are declared as class attributes, which when using dataclasses would conflict -with the dataclass-level attributes. An approach to combine these together -is to make use of the ``metadata`` attribute on the ``dataclass.field`` -object, where SQLAlchemy-specific mapping information may be supplied. -Declarative supports extraction of these parameters when the class -specifies the attribute ``__sa_dataclass_metadata_key__``. This also -provides a more succinct method of indicating the :func:`_orm.relationship` -association:: - - - from __future__ import annotations - - from dataclasses import dataclass, field - from typing import List - - from sqlalchemy import Column, ForeignKey, Integer, String - from sqlalchemy.orm import registry, relationship - - mapper_registry = registry() - - - @mapper_registry.mapped - @dataclass - class User: - __tablename__ = "user" - - __sa_dataclass_metadata_key__ = "sa" - id: int = field(init=False, metadata={"sa": Column(Integer, primary_key=True)}) - name: str = field(default=None, metadata={"sa": Column(String(50))}) - fullname: str = field(default=None, metadata={"sa": Column(String(50))}) - nickname: str = field(default=None, metadata={"sa": Column(String(12))}) - addresses: List[Address] = field( - default_factory=list, metadata={"sa": relationship("Address")} - ) - - - @mapper_registry.mapped - @dataclass - class Address: - __tablename__ = "address" - __sa_dataclass_metadata_key__ = "sa" - id: int = field(init=False, metadata={"sa": Column(Integer, primary_key=True)}) - user_id: int = field(init=False, metadata={"sa": Column(ForeignKey("user.id"))}) - email_address: str = field(default=None, metadata={"sa": Column(String(50))}) - -.. _orm_declarative_dataclasses_mixin: - -Using Declarative Mixins with pre-existing dataclasses -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -In the section :ref:`orm_mixins_toplevel`, Declarative Mixin classes -are introduced. One requirement of declarative mixins is that certain -constructs that can't be easily duplicated must be given as callables, -using the :class:`_orm.declared_attr` decorator, such as in the -example at :ref:`orm_declarative_mixins_relationships`:: - - class RefTargetMixin: - @declared_attr - def target_id(cls): - return Column("target_id", ForeignKey("target.id")) - - @declared_attr - def target(cls): - return relationship("Target") - -This form is supported within the Dataclasses ``field()`` object by using -a lambda to indicate the SQLAlchemy construct inside the ``field()``. -Using :func:`_orm.declared_attr` to surround the lambda is optional. -If we wanted to produce our ``User`` class above where the ORM fields -came from a mixin that is itself a dataclass, the form would be:: - - @dataclass - class UserMixin: - __tablename__ = "user" - - __sa_dataclass_metadata_key__ = "sa" - - id: int = field(init=False, metadata={"sa": Column(Integer, primary_key=True)}) - - addresses: List[Address] = field( - default_factory=list, metadata={"sa": lambda: relationship("Address")} - ) - - - @dataclass - class AddressMixin: - __tablename__ = "address" - __sa_dataclass_metadata_key__ = "sa" - id: int = field(init=False, metadata={"sa": Column(Integer, primary_key=True)}) - user_id: int = field( - init=False, metadata={"sa": lambda: Column(ForeignKey("user.id"))} - ) - email_address: str = field(default=None, metadata={"sa": Column(String(50))}) - - - @mapper_registry.mapped - class User(UserMixin): - pass - - - @mapper_registry.mapped - class Address(AddressMixin): - pass - -.. versionadded:: 1.4.2 Added support for "declared attr" style mixin attributes, - namely :func:`_orm.relationship` constructs as well as :class:`_schema.Column` - objects with foreign key declarations, to be used within "Dataclasses - with Declarative Table" style mappings. - - - -.. _orm_imperative_dataclasses: - -Mapping pre-existing dataclasses using Imperative Mapping -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -As described previously, a class which is set up as a dataclass using the -``@dataclass`` decorator can then be further decorated using the -:meth:`_orm.registry.mapped` decorator in order to apply declarative-style -mapping to the class. As an alternative to using the -:meth:`_orm.registry.mapped` decorator, we may also pass the class through the -:meth:`_orm.registry.map_imperatively` method instead, so that we may pass all -:class:`_schema.Table` and :class:`_orm.Mapper` configuration imperatively to -the function rather than having them defined on the class itself as class -variables:: - - from __future__ import annotations - - from dataclasses import dataclass - from dataclasses import field - from typing import List - - from sqlalchemy import Column - from sqlalchemy import ForeignKey - from sqlalchemy import Integer - from sqlalchemy import MetaData - from sqlalchemy import String - from sqlalchemy import Table - from sqlalchemy.orm import registry - from sqlalchemy.orm import relationship - - mapper_registry = registry() - - - @dataclass - class User: - id: int = field(init=False) - name: str = None - fullname: str = None - nickname: str = None - addresses: List[Address] = field(default_factory=list) - - - @dataclass - class Address: - id: int = field(init=False) - user_id: int = field(init=False) - email_address: str = None - - - metadata_obj = MetaData() - - user = Table( - "user", - metadata_obj, - Column("id", Integer, primary_key=True), - Column("name", String(50)), - Column("fullname", String(50)), - Column("nickname", String(12)), - ) - - address = Table( - "address", - metadata_obj, - Column("id", Integer, primary_key=True), - Column("user_id", Integer, ForeignKey("user.id")), - Column("email_address", String(50)), - ) - - mapper_registry.map_imperatively( - User, - user, - properties={ - "addresses": relationship(Address, backref="user", order_by=address.c.id), - }, - ) - - mapper_registry.map_imperatively(Address, address) - -.. _orm_declarative_attrs_imperative_table: - -Applying ORM mappings to an existing attrs class -------------------------------------------------- - -The attrs_ library is a popular third party library that provides similar -features as dataclasses, with many additional features provided not -found in ordinary dataclasses. - -A class augmented with attrs_ uses the ``@define`` decorator. This decorator -initiates a process to scan the class for attributes that define the class' -behavior, which are then used to generate methods, documentation, and -annotations. - -The SQLAlchemy ORM supports mapping an attrs_ class using **Declarative with -Imperative Table** or **Imperative** mapping. The general form of these two -styles is fully equivalent to the -:ref:`orm_declarative_dataclasses_declarative_table` and -:ref:`orm_declarative_dataclasses_imperative_table` mapping forms used with -dataclasses, where the inline attribute directives used by dataclasses or attrs -are unchanged, and SQLAlchemy's table-oriented instrumentation is applied at -runtime. - -The ``@define`` decorator of attrs_ by default replaces the annotated class -with a new __slots__ based class, which is not supported. When using the old -style annotation ``@attr.s`` or using ``define(slots=False)``, the class -does not get replaced. Furthermore attrs removes its own class-bound attributes -after the decorator runs, so that SQLAlchemy's mapping process takes over these -attributes without any issue. Both decorators, ``@attr.s`` and ``@define(slots=False)`` -work with SQLAlchemy. - -Mapping attrs with Declarative "Imperative Table" -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -In the "Declarative with Imperative Table" style, a :class:`_schema.Table` -object is declared inline with the declarative class. The -``@define`` decorator is applied to the class first, then the -:meth:`_orm.registry.mapped` decorator second:: - - from __future__ import annotations - - from typing import List - from typing import Optional - - from attrs import define - from sqlalchemy import Column - from sqlalchemy import ForeignKey - from sqlalchemy import Integer - from sqlalchemy import MetaData - from sqlalchemy import String - from sqlalchemy import Table - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import registry - from sqlalchemy.orm import relationship - - mapper_registry = registry() - - - @mapper_registry.mapped - @define(slots=False) - class User: - __table__ = Table( - "user", - mapper_registry.metadata, - Column("id", Integer, primary_key=True), - Column("name", String(50)), - Column("FullName", String(50), key="fullname"), - Column("nickname", String(12)), - ) - id: Mapped[int] - name: Mapped[str] - fullname: Mapped[str] - nickname: Mapped[str] - addresses: Mapped[List[Address]] - - __mapper_args__ = { # type: ignore - "properties": { - "addresses": relationship("Address"), - } - } - - - @mapper_registry.mapped - @define(slots=False) - class Address: - __table__ = Table( - "address", - mapper_registry.metadata, - Column("id", Integer, primary_key=True), - Column("user_id", Integer, ForeignKey("user.id")), - Column("email_address", String(50)), - ) - id: Mapped[int] - user_id: Mapped[int] - email_address: Mapped[Optional[str]] - -.. note:: The ``attrs`` ``slots=True`` option, which enables ``__slots__`` on - a mapped class, cannot be used with SQLAlchemy mappings without fully - implementing alternative - :ref:`attribute instrumentation `, as mapped - classes normally rely upon direct access to ``__dict__`` for state storage. - Behavior is undefined when this option is present. - - - -Mapping attrs with Imperative Mapping -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Just as is the case with dataclasses, we can make use of -:meth:`_orm.registry.map_imperatively` to map an existing ``attrs`` class -as well:: - - from __future__ import annotations - - from typing import List - - from attrs import define - from sqlalchemy import Column - from sqlalchemy import ForeignKey - from sqlalchemy import Integer - from sqlalchemy import MetaData - from sqlalchemy import String - from sqlalchemy import Table - from sqlalchemy.orm import registry - from sqlalchemy.orm import relationship - - mapper_registry = registry() - - - @define(slots=False) - class User: - id: int - name: str - fullname: str - nickname: str - addresses: List[Address] - - - @define(slots=False) - class Address: - id: int - user_id: int - email_address: Optional[str] - - - metadata_obj = MetaData() - - user = Table( - "user", - metadata_obj, - Column("id", Integer, primary_key=True), - Column("name", String(50)), - Column("fullname", String(50)), - Column("nickname", String(12)), - ) - - address = Table( - "address", - metadata_obj, - Column("id", Integer, primary_key=True), - Column("user_id", Integer, ForeignKey("user.id")), - Column("email_address", String(50)), - ) - - mapper_registry.map_imperatively( - User, - user, - properties={ - "addresses": relationship(Address, backref="user", order_by=address.c.id), - }, - ) - - mapper_registry.map_imperatively(Address, address) - -The above form is equivalent to the previous example using -Declarative with Imperative Table. - - - -.. _dataclass: https://docs.python.org/3/library/dataclasses.html -.. _dataclasses: https://docs.python.org/3/library/dataclasses.html -.. _attrs: https://pypi.org/project/attrs/ -.. _mypy: https://mypy.readthedocs.io/en/stable/ -.. _pyright: https://github.com/microsoft/pyright diff --git a/doc/build/orm/examples.rst b/doc/build/orm/examples.rst deleted file mode 100644 index 9e38768b32..0000000000 --- a/doc/build/orm/examples.rst +++ /dev/null @@ -1,154 +0,0 @@ -.. _examples_toplevel: - -============ -ORM Examples -============ - -The SQLAlchemy distribution includes a variety of code examples illustrating -a select set of patterns, some typical and some not so typical. All are -runnable and can be found in the ``/examples`` directory of the -distribution. Descriptions and source code for all can be found here. - -Additional SQLAlchemy examples, some user contributed, are available on the -wiki at ``_. - - -Mapping Recipes -=============== - -.. _examples_adjacencylist: - -Adjacency List --------------- - -.. automodule:: examples.adjacency_list - -.. _examples_associations: - -Associations ------------- - -.. automodule:: examples.association - -.. _examples_asyncio: - -Asyncio Integration -------------------- - -.. automodule:: examples.asyncio - -Directed Graphs ---------------- - -.. automodule:: examples.graphs - -Dynamic Relations as Dictionaries ---------------------------------- - -.. automodule:: examples.dynamic_dict - -.. _examples_generic_associations: - -Generic Associations --------------------- - -.. automodule:: examples.generic_associations - - -Materialized Paths ------------------- - -.. automodule:: examples.materialized_paths - -Nested Sets ------------ - -.. automodule:: examples.nested_sets - -.. _examples_performance: - -Performance ------------ - -.. automodule:: examples.performance - - -.. _examples_spaceinvaders: - -Space Invaders --------------- - -.. automodule:: examples.space_invaders - - -.. _examples_versioning: - -Versioning Objects ------------------- - -.. _examples_versioned_history: - -Versioning with a History Table -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. automodule:: examples.versioned_history - -.. _examples_versioned_rows: - -Versioning using Temporal Rows -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. automodule:: examples.versioned_rows - -.. _examples_vertical_tables: - -Vertical Attribute Mapping --------------------------- - -.. automodule:: examples.vertical - - -.. _examples_inheritance: - -Inheritance Mapping Recipes -=========================== - -Basic Inheritance Mappings --------------------------- - -.. automodule:: examples.inheritance - -Special APIs -============ - -.. _examples_instrumentation: - -Attribute Instrumentation -------------------------- - -.. automodule:: examples.custom_attributes - -.. _examples_sharding: - -Horizontal Sharding -------------------- - -.. automodule:: examples.sharding - -Extending the ORM -================= - -.. _examples_session_orm_events: - -ORM Query Events ------------------ - -.. automodule:: examples.extending_query - -.. _examples_caching: - -Dogpile Caching ---------------- - -.. automodule:: examples.dogpile_caching - diff --git a/doc/build/orm/extensions/associationproxy.rst b/doc/build/orm/extensions/associationproxy.rst deleted file mode 100644 index 036969f37b..0000000000 --- a/doc/build/orm/extensions/associationproxy.rst +++ /dev/null @@ -1,716 +0,0 @@ -.. _associationproxy_toplevel: - -Association Proxy -================= - -.. module:: sqlalchemy.ext.associationproxy - -``associationproxy`` is used to create a read/write view of a -target attribute across a relationship. It essentially conceals -the usage of a "middle" attribute between two endpoints, and -can be used to cherry-pick fields from a collection of -related objects or to reduce the verbosity of using the association -object pattern. Applied creatively, the association proxy allows -the construction of sophisticated collections and dictionary -views of virtually any geometry, persisted to the database using -standard, transparently configured relational patterns. - -.. _associationproxy_scalar_collections: - -Simplifying Scalar Collections ------------------------------- - -Consider a many-to-many mapping between two classes, ``User`` and ``Keyword``. -Each ``User`` can have any number of ``Keyword`` objects, and vice-versa -(the many-to-many pattern is described at :ref:`relationships_many_to_many`). -The example below illustrates this pattern in the same way, with the -exception of an extra attribute added to the ``User`` class called -``User.keywords``:: - - from __future__ import annotations - - from typing import Final - - from sqlalchemy import Column - from sqlalchemy import ForeignKey - from sqlalchemy import Integer - from sqlalchemy import String - from sqlalchemy import Table - from sqlalchemy.orm import DeclarativeBase - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - from sqlalchemy.orm import relationship - from sqlalchemy.ext.associationproxy import association_proxy - from sqlalchemy.ext.associationproxy import AssociationProxy - - - class Base(DeclarativeBase): - pass - - - class User(Base): - __tablename__ = "user" - id: Mapped[int] = mapped_column(primary_key=True) - name: Mapped[str] = mapped_column(String(64)) - kw: Mapped[List[Keyword]] = relationship(secondary=lambda: user_keyword_table) - - def __init__(self, name: str): - self.name = name - - # proxy the 'keyword' attribute from the 'kw' relationship - keywords: AssociationProxy[List[str]] = association_proxy("kw", "keyword") - - - class Keyword(Base): - __tablename__ = "keyword" - id: Mapped[int] = mapped_column(primary_key=True) - keyword: Mapped[str] = mapped_column(String(64)) - - def __init__(self, keyword: str): - self.keyword = keyword - - - user_keyword_table: Final[Table] = Table( - "user_keyword", - Base.metadata, - Column("user_id", Integer, ForeignKey("user.id"), primary_key=True), - Column("keyword_id", Integer, ForeignKey("keyword.id"), primary_key=True), - ) - -In the above example, :func:`.association_proxy` is applied to the ``User`` -class to produce a "view" of the ``kw`` relationship, which exposes the string -value of ``.keyword`` associated with each ``Keyword`` object. It also -creates new ``Keyword`` objects transparently when strings are added to the -collection:: - - >>> user = User("jek") - >>> user.keywords.append("cheese-inspector") - >>> user.keywords.append("snack-ninja") - >>> print(user.keywords) - ['cheese-inspector', 'snack-ninja'] - -To understand the mechanics of this, first review the behavior of -``User`` and ``Keyword`` without using the ``.keywords`` association proxy. -Normally, reading and manipulating the collection of "keyword" strings associated -with ``User`` requires traversal from each collection element to the ``.keyword`` -attribute, which can be awkward. The example below illustrates the identical -series of operations applied without using the association proxy:: - - >>> # identical operations without using the association proxy - >>> user = User("jek") - >>> user.kw.append(Keyword("cheese-inspector")) - >>> user.kw.append(Keyword("snack-ninja")) - >>> print([keyword.keyword for keyword in user.kw]) - ['cheese-inspector', 'snack-ninja'] - -The :class:`.AssociationProxy` object produced by the :func:`.association_proxy` function -is an instance of a `Python descriptor `_, -and is not considered to be "mapped" by the :class:`.Mapper` in any way. Therefore, -it's always indicated inline within the class definition of the mapped class, -regardless of whether Declarative or Imperative mappings are used. - -The proxy functions by operating upon the underlying mapped attribute -or collection in response to operations, and changes made via the proxy are immediately -apparent in the mapped attribute, as well as vice versa. The underlying -attribute remains fully accessible. - -When first accessed, the association proxy performs introspection -operations on the target collection so that its behavior corresponds correctly. -Details such as if the locally proxied attribute is a collection (as is typical) -or a scalar reference, as well as if the collection acts like a set, list, -or dictionary is taken into account, so that the proxy should act just like -the underlying collection or attribute does. - -.. _associationproxy_creator: - -Creation of New Values ----------------------- - -When a list ``append()`` event (or set ``add()``, dictionary ``__setitem__()``, -or scalar assignment event) is intercepted by the association proxy, it -instantiates a new instance of the "intermediary" object using its constructor, -passing as a single argument the given value. In our example above, an -operation like:: - - user.keywords.append("cheese-inspector") - -Is translated by the association proxy into the operation:: - - user.kw.append(Keyword("cheese-inspector")) - -The example works here because we have designed the constructor for ``Keyword`` -to accept a single positional argument, ``keyword``. For those cases where a -single-argument constructor isn't feasible, the association proxy's creational -behavior can be customized using the :paramref:`.association_proxy.creator` -argument, which references a callable (i.e. Python function) that will produce -a new object instance given the singular argument. Below we illustrate this -using a lambda as is typical:: - - class User(Base): - ... - - # use Keyword(keyword=kw) on append() events - keywords: AssociationProxy[List[str]] = association_proxy( - "kw", "keyword", creator=lambda kw: Keyword(keyword=kw) - ) - -The ``creator`` function accepts a single argument in the case of a list- -or set- based collection, or a scalar attribute. In the case of a dictionary-based -collection, it accepts two arguments, "key" and "value". An example -of this is below in :ref:`proxying_dictionaries`. - -Simplifying Association Objects -------------------------------- - -The "association object" pattern is an extended form of a many-to-many -relationship, and is described at :ref:`association_pattern`. Association -proxies are useful for keeping "association objects" out of the way during -regular use. - -Suppose our ``user_keyword`` table above had additional columns -which we'd like to map explicitly, but in most cases we don't -require direct access to these attributes. Below, we illustrate -a new mapping which introduces the ``UserKeywordAssociation`` class, which -is mapped to the ``user_keyword`` table illustrated earlier. -This class adds an additional column ``special_key``, a value which -we occasionally want to access, but not in the usual case. We -create an association proxy on the ``User`` class called -``keywords``, which will bridge the gap from the ``user_keyword_associations`` -collection of ``User`` to the ``.keyword`` attribute present on each -``UserKeywordAssociation``:: - - from __future__ import annotations - - from typing import List - from typing import Optional - - from sqlalchemy import ForeignKey - from sqlalchemy import String - from sqlalchemy.ext.associationproxy import association_proxy - from sqlalchemy.ext.associationproxy import AssociationProxy - from sqlalchemy.orm import DeclarativeBase - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - from sqlalchemy.orm import relationship - - - class Base(DeclarativeBase): - pass - - - class User(Base): - __tablename__ = "user" - - id: Mapped[int] = mapped_column(primary_key=True) - name: Mapped[str] = mapped_column(String(64)) - - user_keyword_associations: Mapped[List[UserKeywordAssociation]] = relationship( - back_populates="user", - cascade="all, delete-orphan", - ) - - # association proxy of "user_keyword_associations" collection - # to "keyword" attribute - keywords: AssociationProxy[List[Keyword]] = association_proxy( - "user_keyword_associations", - "keyword", - creator=lambda keyword_obj: UserKeywordAssociation(keyword=keyword_obj), - ) - - def __init__(self, name: str): - self.name = name - - - class UserKeywordAssociation(Base): - __tablename__ = "user_keyword" - user_id: Mapped[int] = mapped_column(ForeignKey("user.id"), primary_key=True) - keyword_id: Mapped[int] = mapped_column(ForeignKey("keyword.id"), primary_key=True) - special_key: Mapped[Optional[str]] = mapped_column(String(50)) - - user: Mapped[User] = relationship(back_populates="user_keyword_associations") - - keyword: Mapped[Keyword] = relationship() - - - class Keyword(Base): - __tablename__ = "keyword" - id: Mapped[int] = mapped_column(primary_key=True) - keyword: Mapped[str] = mapped_column("keyword", String(64)) - - def __init__(self, keyword: str): - self.keyword = keyword - - def __repr__(self) -> str: - return f"Keyword({self.keyword!r})" - -With the above configuration, we can operate upon the ``.keywords`` collection -of each ``User`` object, each of which exposes a collection of ``Keyword`` -objects that are obtained from the underlying ``UserKeywordAssociation`` elements:: - - >>> user = User("log") - >>> for kw in (Keyword("new_from_blammo"), Keyword("its_big")): - ... user.keywords.append(kw) - >>> print(user.keywords) - [Keyword('new_from_blammo'), Keyword('its_big')] - -This example is in contrast to the example illustrated previously at -:ref:`associationproxy_scalar_collections`, where the association proxy exposed -a collection of strings, rather than a collection of composed objects. -In this case, each ``.keywords.append()`` operation is equivalent to:: - - >>> user.user_keyword_associations.append( - ... UserKeywordAssociation(keyword=Keyword("its_heavy")) - ... ) - -The ``UserKeywordAssociation`` object has two attributes that are both -populated within the scope of the ``append()`` operation of the association -proxy; ``.keyword``, which refers to the -``Keyword`` object, and ``.user``, which refers to the ``User`` object. -The ``.keyword`` attribute is populated first, as the association proxy -generates a new ``UserKeywordAssociation`` object in response to the ``.append()`` -operation, assigning the given ``Keyword`` instance to the ``.keyword`` -attribute. Then, as the ``UserKeywordAssociation`` object is appended to the -``User.user_keyword_associations`` collection, the ``UserKeywordAssociation.user`` attribute, -configured as ``back_populates`` for ``User.user_keyword_associations``, is initialized -upon the given ``UserKeywordAssociation`` instance to refer to the parent ``User`` -receiving the append operation. The ``special_key`` -argument above is left at its default value of ``None``. - -For those cases where we do want ``special_key`` to have a value, we -create the ``UserKeywordAssociation`` object explicitly. Below we assign all -three attributes, wherein the assignment of ``.user`` during -construction, has the effect of appending the new ``UserKeywordAssociation`` to -the ``User.user_keyword_associations`` collection (via the relationship):: - - >>> UserKeywordAssociation( - ... keyword=Keyword("its_wood"), user=user, special_key="my special key" - ... ) - -The association proxy returns to us a collection of ``Keyword`` objects represented -by all these operations:: - - >>> print(user.keywords) - [Keyword('new_from_blammo'), Keyword('its_big'), Keyword('its_heavy'), Keyword('its_wood')] - -.. _proxying_dictionaries: - -Proxying to Dictionary Based Collections ----------------------------------------- - -The association proxy can proxy to dictionary based collections as well. SQLAlchemy -mappings usually use the :func:`.attribute_keyed_dict` collection type to -create dictionary collections, as well as the extended techniques described in -:ref:`dictionary_collections`. - -The association proxy adjusts its behavior when it detects the usage of a -dictionary-based collection. When new values are added to the dictionary, the -association proxy instantiates the intermediary object by passing two -arguments to the creation function instead of one, the key and the value. As -always, this creation function defaults to the constructor of the intermediary -class, and can be customized using the ``creator`` argument. - -Below, we modify our ``UserKeywordAssociation`` example such that the ``User.user_keyword_associations`` -collection will now be mapped using a dictionary, where the ``UserKeywordAssociation.special_key`` -argument will be used as the key for the dictionary. We also apply a ``creator`` -argument to the ``User.keywords`` proxy so that these values are assigned appropriately -when new elements are added to the dictionary:: - - from __future__ import annotations - from typing import Dict - - from sqlalchemy import ForeignKey - from sqlalchemy import String - from sqlalchemy.ext.associationproxy import association_proxy - from sqlalchemy.ext.associationproxy import AssociationProxy - from sqlalchemy.orm import DeclarativeBase - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - from sqlalchemy.orm import relationship - from sqlalchemy.orm.collections import attribute_keyed_dict - - - class Base(DeclarativeBase): - pass - - - class User(Base): - __tablename__ = "user" - id: Mapped[int] = mapped_column(primary_key=True) - name: Mapped[str] = mapped_column(String(64)) - - # user/user_keyword_associations relationship, mapping - # user_keyword_associations with a dictionary against "special_key" as key. - user_keyword_associations: Mapped[Dict[str, UserKeywordAssociation]] = relationship( - back_populates="user", - collection_class=attribute_keyed_dict("special_key"), - cascade="all, delete-orphan", - ) - # proxy to 'user_keyword_associations', instantiating - # UserKeywordAssociation assigning the new key to 'special_key', - # values to 'keyword'. - keywords: AssociationProxy[Dict[str, Keyword]] = association_proxy( - "user_keyword_associations", - "keyword", - creator=lambda k, v: UserKeywordAssociation(special_key=k, keyword=v), - ) - - def __init__(self, name: str): - self.name = name - - - class UserKeywordAssociation(Base): - __tablename__ = "user_keyword" - user_id: Mapped[int] = mapped_column(ForeignKey("user.id"), primary_key=True) - keyword_id: Mapped[int] = mapped_column(ForeignKey("keyword.id"), primary_key=True) - special_key: Mapped[str] - - user: Mapped[User] = relationship( - back_populates="user_keyword_associations", - ) - keyword: Mapped[Keyword] = relationship() - - - class Keyword(Base): - __tablename__ = "keyword" - id: Mapped[int] = mapped_column(primary_key=True) - keyword: Mapped[str] = mapped_column(String(64)) - - def __init__(self, keyword: str): - self.keyword = keyword - - def __repr__(self) -> str: - return f"Keyword({self.keyword!r})" - -We illustrate the ``.keywords`` collection as a dictionary, mapping the -``UserKeywordAssociation.special_key`` value to ``Keyword`` objects:: - - >>> user = User("log") - - >>> user.keywords["sk1"] = Keyword("kw1") - >>> user.keywords["sk2"] = Keyword("kw2") - - >>> print(user.keywords) - {'sk1': Keyword('kw1'), 'sk2': Keyword('kw2')} - -.. _composite_association_proxy: - -Composite Association Proxies ------------------------------ - -Given our previous examples of proxying from relationship to scalar -attribute, proxying across an association object, and proxying dictionaries, -we can combine all three techniques together to give ``User`` -a ``keywords`` dictionary that deals strictly with the string value -of ``special_key`` mapped to the string ``keyword``. Both the ``UserKeywordAssociation`` -and ``Keyword`` classes are entirely concealed. This is achieved by building -an association proxy on ``User`` that refers to an association proxy -present on ``UserKeywordAssociation``:: - - from __future__ import annotations - - from sqlalchemy import ForeignKey - from sqlalchemy import String - from sqlalchemy.ext.associationproxy import association_proxy - from sqlalchemy.ext.associationproxy import AssociationProxy - from sqlalchemy.orm import DeclarativeBase - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - from sqlalchemy.orm import relationship - from sqlalchemy.orm.collections import attribute_keyed_dict - - - class Base(DeclarativeBase): - pass - - - class User(Base): - __tablename__ = "user" - id: Mapped[int] = mapped_column(primary_key=True) - name: Mapped[str] = mapped_column(String(64)) - - user_keyword_associations: Mapped[Dict[str, UserKeywordAssociation]] = relationship( - back_populates="user", - collection_class=attribute_keyed_dict("special_key"), - cascade="all, delete-orphan", - ) - # the same 'user_keyword_associations'->'keyword' proxy as in - # the basic dictionary example. - keywords: AssociationProxy[Dict[str, str]] = association_proxy( - "user_keyword_associations", - "keyword", - creator=lambda k, v: UserKeywordAssociation(special_key=k, keyword=v), - ) - - def __init__(self, name: str): - self.name = name - - - class UserKeywordAssociation(Base): - __tablename__ = "user_keyword" - user_id: Mapped[int] = mapped_column(ForeignKey("user.id"), primary_key=True) - keyword_id: Mapped[int] = mapped_column(ForeignKey("keyword.id"), primary_key=True) - special_key: Mapped[str] = mapped_column(String(64)) - user: Mapped[User] = relationship( - back_populates="user_keyword_associations", - ) - - # the relationship to Keyword is now called - # 'kw' - kw: Mapped[Keyword] = relationship() - - # 'keyword' is changed to be a proxy to the - # 'keyword' attribute of 'Keyword' - keyword: AssociationProxy[Dict[str, str]] = association_proxy("kw", "keyword") - - - class Keyword(Base): - __tablename__ = "keyword" - id: Mapped[int] = mapped_column(primary_key=True) - keyword: Mapped[str] = mapped_column(String(64)) - - def __init__(self, keyword: str): - self.keyword = keyword - -``User.keywords`` is now a dictionary of string to string, where -``UserKeywordAssociation`` and ``Keyword`` objects are created and removed for us -transparently using the association proxy. In the example below, we illustrate -usage of the assignment operator, also appropriately handled by the -association proxy, to apply a dictionary value to the collection at once:: - - >>> user = User("log") - >>> user.keywords = {"sk1": "kw1", "sk2": "kw2"} - >>> print(user.keywords) - {'sk1': 'kw1', 'sk2': 'kw2'} - - >>> user.keywords["sk3"] = "kw3" - >>> del user.keywords["sk2"] - >>> print(user.keywords) - {'sk1': 'kw1', 'sk3': 'kw3'} - - >>> # illustrate un-proxied usage - ... print(user.user_keyword_associations["sk3"].kw) - <__main__.Keyword object at 0x12ceb90> - -One caveat with our example above is that because ``Keyword`` objects are created -for each dictionary set operation, the example fails to maintain uniqueness for -the ``Keyword`` objects on their string name, which is a typical requirement for -a tagging scenario such as this one. For this use case the recipe -`UniqueObject `_, or -a comparable creational strategy, is -recommended, which will apply a "lookup first, then create" strategy to the constructor -of the ``Keyword`` class, so that an already existing ``Keyword`` is returned if the -given name is already present. - -Querying with Association Proxies ---------------------------------- - -The :class:`.AssociationProxy` features simple SQL construction capabilities -which work at the class level in a similar way as other ORM-mapped attributes, -and provide rudimentary filtering support primarily based on the -SQL ``EXISTS`` keyword. - - -.. note:: The primary purpose of the association proxy extension is to allow - for improved persistence and object-access patterns with mapped object - instances that are already loaded. The class-bound querying feature - is of limited use and will not replace the need to refer to the underlying - attributes when constructing SQL queries with JOINs, eager loading - options, etc. - -For this section, assume a class with both an association proxy -that refers to a column, as well as an association proxy that refers -to a related object, as in the example mapping below:: - - from __future__ import annotations - from sqlalchemy import Column, ForeignKey, Integer, String - from sqlalchemy.ext.associationproxy import association_proxy, AssociationProxy - from sqlalchemy.orm import DeclarativeBase, relationship - from sqlalchemy.orm.collections import attribute_keyed_dict - from sqlalchemy.orm.collections import Mapped - - - class Base(DeclarativeBase): - pass - - - class User(Base): - __tablename__ = "user" - id: Mapped[int] = mapped_column(primary_key=True) - name: Mapped[str] = mapped_column(String(64)) - - user_keyword_associations: Mapped[UserKeywordAssociation] = relationship( - cascade="all, delete-orphan", - ) - - # object-targeted association proxy - keywords: AssociationProxy[List[Keyword]] = association_proxy( - "user_keyword_associations", - "keyword", - ) - - # column-targeted association proxy - special_keys: AssociationProxy[List[str]] = association_proxy( - "user_keyword_associations", "special_key" - ) - - - class UserKeywordAssociation(Base): - __tablename__ = "user_keyword" - user_id: Mapped[int] = mapped_column(ForeignKey("user.id"), primary_key=True) - keyword_id: Mapped[int] = mapped_column(ForeignKey("keyword.id"), primary_key=True) - special_key: Mapped[str] = mapped_column(String(64)) - keyword: Mapped[Keyword] = relationship() - - - class Keyword(Base): - __tablename__ = "keyword" - id: Mapped[int] = mapped_column(primary_key=True) - keyword: Mapped[str] = mapped_column(String(64)) - -The SQL generated takes the form of a correlated subquery against -the EXISTS SQL operator so that it can be used in a WHERE clause without -the need for additional modifications to the enclosing query. If the -immediate target of an association proxy is a **mapped column expression**, -standard column operators can be used which will be embedded in the subquery. -For example a straight equality operator: - -.. sourcecode:: pycon+sql - - >>> print(session.scalars(select(User).where(User.special_keys == "jek"))) - {printsql}SELECT "user".id AS user_id, "user".name AS user_name - FROM "user" - WHERE EXISTS (SELECT 1 - FROM user_keyword - WHERE "user".id = user_keyword.user_id AND user_keyword.special_key = :special_key_1) - -a LIKE operator: - -.. sourcecode:: pycon+sql - - >>> print(session.scalars(select(User).where(User.special_keys.like("%jek")))) - {printsql}SELECT "user".id AS user_id, "user".name AS user_name - FROM "user" - WHERE EXISTS (SELECT 1 - FROM user_keyword - WHERE "user".id = user_keyword.user_id AND user_keyword.special_key LIKE :special_key_1) - -For association proxies where the immediate target is a **related object or collection, -or another association proxy or attribute on the related object**, relationship-oriented -operators can be used instead, such as :meth:`_orm.PropComparator.has` and -:meth:`_orm.PropComparator.any`. The ``User.keywords`` attribute is in fact -two association proxies linked together, so when using this proxy for generating -SQL phrases, we get two levels of EXISTS subqueries: - -.. sourcecode:: pycon+sql - - >>> print(session.scalars(select(User).where(User.keywords.any(Keyword.keyword == "jek")))) - {printsql}SELECT "user".id AS user_id, "user".name AS user_name - FROM "user" - WHERE EXISTS (SELECT 1 - FROM user_keyword - WHERE "user".id = user_keyword.user_id AND (EXISTS (SELECT 1 - FROM keyword - WHERE keyword.id = user_keyword.keyword_id AND keyword.keyword = :keyword_1))) - -This is not the most efficient form of SQL, so while association proxies can be -convenient for generating WHERE criteria quickly, SQL results should be -inspected and "unrolled" into explicit JOIN criteria for best use, especially -when chaining association proxies together. - - -.. versionchanged:: 1.3 Association proxy features distinct querying modes - based on the type of target. See :ref:`change_4351`. - - - -.. _cascade_scalar_deletes: - -Cascading Scalar Deletes ------------------------- - -.. versionadded:: 1.3 - -Given a mapping as:: - - from __future__ import annotations - from sqlalchemy import Column, ForeignKey, Integer, String - from sqlalchemy.ext.associationproxy import association_proxy, AssociationProxy - from sqlalchemy.orm import DeclarativeBase, relationship - from sqlalchemy.orm.collections import attribute_keyed_dict - from sqlalchemy.orm.collections import Mapped - - - class Base(DeclarativeBase): - pass - - - class A(Base): - __tablename__ = "test_a" - id: Mapped[int] = mapped_column(primary_key=True) - ab: Mapped[AB] = relationship(uselist=False) - b: AssociationProxy[B] = association_proxy( - "ab", "b", creator=lambda b: AB(b=b), cascade_scalar_deletes=True - ) - - - class B(Base): - __tablename__ = "test_b" - id: Mapped[int] = mapped_column(primary_key=True) - - - class AB(Base): - __tablename__ = "test_ab" - a_id: Mapped[int] = mapped_column(ForeignKey(A.id), primary_key=True) - b_id: Mapped[int] = mapped_column(ForeignKey(B.id), primary_key=True) - - b: Mapped[B] = relationship() - -An assignment to ``A.b`` will generate an ``AB`` object:: - - a.b = B() - -The ``A.b`` association is scalar, and includes use of the parameter -:paramref:`.AssociationProxy.cascade_scalar_deletes`. When this parameter -is enabled, setting ``A.b`` -to ``None`` will remove ``A.ab`` as well:: - - a.b = None - assert a.ab is None - -When :paramref:`.AssociationProxy.cascade_scalar_deletes` is not set, -the association object ``a.ab`` above would remain in place. - -Note that this is not the behavior for collection-based association proxies; -in that case, the intermediary association object is always removed when -members of the proxied collection are removed. Whether or not the row is -deleted depends on the relationship cascade setting. - -.. seealso:: - - :ref:`unitofwork_cascades` - -API Documentation ------------------ - -.. autofunction:: association_proxy - -.. autoclass:: AssociationProxy - :members: - :undoc-members: - :inherited-members: - -.. autoclass:: AssociationProxyInstance - :members: - :undoc-members: - :inherited-members: - -.. autoclass:: ObjectAssociationProxyInstance - :members: - :inherited-members: - -.. autoclass:: ColumnAssociationProxyInstance - :members: - :inherited-members: - -.. autoclass:: AssociationProxyExtensionType - :members: diff --git a/doc/build/orm/extensions/asyncio.rst b/doc/build/orm/extensions/asyncio.rst deleted file mode 100644 index c57f1199cd..0000000000 --- a/doc/build/orm/extensions/asyncio.rst +++ /dev/null @@ -1,1032 +0,0 @@ -.. _asyncio_toplevel: - -Asynchronous I/O (asyncio) -========================== - -Support for Python asyncio. Support for Core and ORM usage is -included, using asyncio-compatible dialects. - -.. versionadded:: 1.4 - -.. warning:: Please read :ref:`asyncio_install` for important platform - installation notes for many platforms, including **Apple M1 Architecture**. - -.. seealso:: - - :ref:`change_3414` - initial feature announcement - - :ref:`examples_asyncio` - example scripts illustrating working examples - of Core and ORM use within the asyncio extension. - -.. _asyncio_install: - -Asyncio Platform Installation Notes (Including Apple M1) ---------------------------------------------------------- - -The asyncio extension requires Python 3 only. It also depends -upon the `greenlet `_ library. This -dependency is installed by default on common machine platforms including: - -.. sourcecode:: text - - x86_64 aarch64 ppc64le amd64 win32 - -For the above platforms, ``greenlet`` is known to supply pre-built wheel files. -For other platforms, **greenlet does not install by default**; -the current file listing for greenlet can be seen at -`Greenlet - Download Files `_. -Note that **there are many architectures omitted, including Apple M1**. - -To install SQLAlchemy while ensuring the ``greenlet`` dependency is present -regardless of what platform is in use, the -``[asyncio]`` `setuptools extra `_ -may be installed -as follows, which will include also instruct ``pip`` to install ``greenlet``: - -.. sourcecode:: text - - pip install sqlalchemy[asyncio] - -Note that installation of ``greenlet`` on platforms that do not have a pre-built -wheel file means that ``greenlet`` will be built from source, which requires -that Python's development libraries also be present. - - -Synopsis - Core ---------------- - -For Core use, the :func:`_asyncio.create_async_engine` function creates an -instance of :class:`_asyncio.AsyncEngine` which then offers an async version of -the traditional :class:`_engine.Engine` API. The -:class:`_asyncio.AsyncEngine` delivers an :class:`_asyncio.AsyncConnection` via -its :meth:`_asyncio.AsyncEngine.connect` and :meth:`_asyncio.AsyncEngine.begin` -methods which both deliver asynchronous context managers. The -:class:`_asyncio.AsyncConnection` can then invoke statements using either the -:meth:`_asyncio.AsyncConnection.execute` method to deliver a buffered -:class:`_engine.Result`, or the :meth:`_asyncio.AsyncConnection.stream` method -to deliver a streaming server-side :class:`_asyncio.AsyncResult`:: - - import asyncio - - from sqlalchemy import Column - from sqlalchemy import MetaData - from sqlalchemy import select - from sqlalchemy import String - from sqlalchemy import Table - from sqlalchemy.ext.asyncio import create_async_engine - - meta = MetaData() - t1 = Table("t1", meta, Column("name", String(50), primary_key=True)) - - - async def async_main() -> None: - engine = create_async_engine( - "postgresql+asyncpg://scott:tiger@localhost/test", - echo=True, - ) - - async with engine.begin() as conn: - await conn.run_sync(meta.create_all) - - await conn.execute( - t1.insert(), [{"name": "some name 1"}, {"name": "some name 2"}] - ) - - async with engine.connect() as conn: - # select a Result, which will be delivered with buffered - # results - result = await conn.execute(select(t1).where(t1.c.name == "some name 1")) - - print(result.fetchall()) - - # for AsyncEngine created in function scope, close and - # clean-up pooled connections - await engine.dispose() - - - asyncio.run(async_main()) - -Above, the :meth:`_asyncio.AsyncConnection.run_sync` method may be used to -invoke special DDL functions such as :meth:`_schema.MetaData.create_all` that -don't include an awaitable hook. - -.. tip:: It's advisable to invoke the :meth:`_asyncio.AsyncEngine.dispose` method - using ``await`` when using the :class:`_asyncio.AsyncEngine` object in a - scope that will go out of context and be garbage collected, as illustrated in the - ``async_main`` function in the above example. This ensures that any - connections held open by the connection pool will be properly disposed - within an awaitable context. Unlike when using blocking IO, SQLAlchemy - cannot properly dispose of these connections within methods like ``__del__`` - or weakref finalizers as there is no opportunity to invoke ``await``. - Failing to explicitly dispose of the engine when it falls out of scope - may result in warnings emitted to standard out resembling the form - ``RuntimeError: Event loop is closed`` within garbage collection. - -The :class:`_asyncio.AsyncConnection` also features a "streaming" API via -the :meth:`_asyncio.AsyncConnection.stream` method that returns an -:class:`_asyncio.AsyncResult` object. This result object uses a server-side -cursor and provides an async/await API, such as an async iterator:: - - async with engine.connect() as conn: - async_result = await conn.stream(select(t1)) - - async for row in async_result: - print("row: %s" % (row,)) - -.. _asyncio_orm: - - -Synopsis - ORM ---------------- - -Using :term:`2.0 style` querying, the :class:`_asyncio.AsyncSession` class -provides full ORM functionality. Within the default mode of use, special care -must be taken to avoid :term:`lazy loading` or other expired-attribute access -involving ORM relationships and column attributes; the next -section :ref:`asyncio_orm_avoid_lazyloads` details this. The example below -illustrates a complete example including mapper and session configuration:: - - from __future__ import annotations - - import asyncio - import datetime - from typing import List - - from sqlalchemy import ForeignKey - from sqlalchemy import func - from sqlalchemy import select - from sqlalchemy.ext.asyncio import async_sessionmaker - from sqlalchemy.ext.asyncio import AsyncSession - from sqlalchemy.ext.asyncio import create_async_engine - from sqlalchemy.orm import DeclarativeBase - from sqlalchemy.orm import Mapped - from sqlalchemy.orm import mapped_column - from sqlalchemy.orm import relationship - from sqlalchemy.orm import selectinload - - - class Base(DeclarativeBase): - pass - - - class A(Base): - __tablename__ = "a" - - id: Mapped[int] = mapped_column(primary_key=True) - data: Mapped[str] - create_date: Mapped[datetime.datetime] = mapped_column(server_default=func.now()) - bs: Mapped[List[B]] = relationship(lazy="raise") - - - class B(Base): - __tablename__ = "b" - id: Mapped[int] = mapped_column(primary_key=True) - a_id: Mapped[int] = mapped_column(ForeignKey("a.id")) - data: Mapped[str] - - - async def insert_objects(async_session: async_sessionmaker[AsyncSession]) -> None: - - async with async_session() as session: - async with session.begin(): - session.add_all( - [ - A(bs=[B(), B()], data="a1"), - A(bs=[], data="a2"), - A(bs=[B(), B()], data="a3"), - ] - ) - - - async def select_and_update_objects( - async_session: async_sessionmaker[AsyncSession], - ) -> None: - - async with async_session() as session: - stmt = select(A).options(selectinload(A.bs)) - - result = await session.execute(stmt) - - for a1 in result.scalars(): - print(a1) - print(f"created at: {a1.create_date}") - for b1 in a1.bs: - print(b1) - - result = await session.execute(select(A).order_by(A.id).limit(1)) - - a1 = result.scalars().one() - - a1.data = "new data" - - await session.commit() - - # access attribute subsequent to commit; this is what - # expire_on_commit=False allows - print(a1.data) - - - async def async_main() -> None: - engine = create_async_engine( - "postgresql+asyncpg://scott:tiger@localhost/test", - echo=True, - ) - - # async_sessionmaker: a factory for new AsyncSession objects. - # expire_on_commit - don't expire objects after transaction commit - async_session = async_sessionmaker(engine, expire_on_commit=False) - - async with engine.begin() as conn: - await conn.run_sync(Base.metadata.create_all) - - await insert_objects(async_session) - await select_and_update_objects(async_session) - - # for AsyncEngine created in function scope, close and - # clean-up pooled connections - await engine.dispose() - - - asyncio.run(async_main()) - -In the example above, the :class:`_asyncio.AsyncSession` is instantiated using -the optional :class:`_asyncio.async_sessionmaker` helper, which provides -a factory for new :class:`_asyncio.AsyncSession` objects with a fixed set -of parameters, which here includes associating it with -an :class:`_asyncio.AsyncEngine` against particular database URL. It is then -passed to other methods where it may be used in a Python asynchronous context -manager (i.e. ``async with:`` statement) so that it is automatically closed at -the end of the block; this is equivalent to calling the -:meth:`_asyncio.AsyncSession.close` method. - - -.. _asyncio_orm_avoid_lazyloads: - -Preventing Implicit IO when Using AsyncSession -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Using traditional asyncio, the application needs to avoid any points at which -IO-on-attribute access may occur. Techniques that can be used to help -this are below, many of which are illustrated in the preceding example. - -* Collections can be replaced with **write only collections** that will never - emit IO implicitly, by using the :ref:`write_only_relationship` feature in - SQLAlchemy 2.0. Using this feature, collections are never read from, only - queried using explicit SQL calls. See the example - ``async_orm_writeonly.py`` in the :ref:`examples_asyncio` section for - an example of write-only collections used with asyncio. - - When using write only collections, the program's behavior is simple and easy - to predict regarding collections. However, the downside is that there is not - any built-in system for loading many of these collections all at once, which - instead would need to be performed manually. Therefore, many of the - bullets below address specific techniques when using traditional lazy-loaded - relationships with asyncio, which requires more care. - -* If using traditional ORM relationships which are subject to lazy loading, - relationships can be declared with ``lazy="raise"`` so that by - default they will not attempt to emit SQL. In order to load collections, - :term:`eager loading` must be used in all cases. - -* The most useful eager loading strategy is the - :func:`_orm.selectinload` eager loader, which is employed in the previous - example in order to eagerly - load the ``A.bs`` collection within the scope of the - ``await session.execute()`` call:: - - stmt = select(A).options(selectinload(A.bs)) - -* When constructing new objects, **collections are always assigned a default, - empty collection**, such as a list in the above example:: - - A(bs=[], data="a2") - - This allows the ``.bs`` collection on the above ``A`` object to be present and - readable when the ``A`` object is flushed; otherwise, when the ``A`` is - flushed, ``.bs`` would be unloaded and would raise an error on access. - -* The :class:`_asyncio.AsyncSession` is configured using - :paramref:`_orm.Session.expire_on_commit` set to False, so that we may access - attributes on an object subsequent to a call to - :meth:`_asyncio.AsyncSession.commit`, as in the line at the end where we - access an attribute:: - - # create AsyncSession with expire_on_commit=False - async_session = AsyncSession(engine, expire_on_commit=False) - - # sessionmaker version - async_session = async_sessionmaker(engine, expire_on_commit=False) - - async with async_session() as session: - result = await session.execute(select(A).order_by(A.id)) - - a1 = result.scalars().first() - - # commit would normally expire all attributes - await session.commit() - - # access attribute subsequent to commit; this is what - # expire_on_commit=False allows - print(a1.data) - -Other guidelines include: - -* Methods like :meth:`_asyncio.AsyncSession.expire` should be avoided in favor of - :meth:`_asyncio.AsyncSession.refresh`; **if** expiration is absolutely needed. - Expiration should generally **not** be needed as - :paramref:`_orm.Session.expire_on_commit` - should normally be set to ``False`` when using asyncio. - -* A lazy-loaded relationship **can be loaded explicitly under asyncio** using - :meth:`_asyncio.AsyncSession.refresh`, **if** the desired attribute name - is passed explicitly to - :paramref:`_orm.Session.refresh.attribute_names`, e.g.:: - - # assume a_obj is an A that has lazy loaded A.bs collection - a_obj = await async_session.get(A, [1]) - - # force the collection to load by naming it in attribute_names - await async_session.refresh(a_obj, ["bs"]) - - # collection is present - print(f"bs collection: {a_obj.bs}") - - It's of course preferable to use eager loading up front in order to have - collections already set up without the need to lazy-load. - - .. versionadded:: 2.0.4 Added support for - :meth:`_asyncio.AsyncSession.refresh` and the underlying - :meth:`_orm.Session.refresh` method to force lazy-loaded relationships - to load, if they are named explicitly in the - :paramref:`_orm.Session.refresh.attribute_names` parameter. - In previous versions, the relationship would be silently skipped even - if named in the parameter. - -* Avoid using the ``all`` cascade option documented at :ref:`unitofwork_cascades` - in favor of listing out the desired cascade features explicitly. The - ``all`` cascade option implies among others the :ref:`cascade_refresh_expire` - setting, which means that the :meth:`.AsyncSession.refresh` method will - expire the attributes on related objects, but not necessarily refresh those - related objects assuming eager loading is not configured within the - :func:`_orm.relationship`, leaving them in an expired state. - -* Appropriate loader options should be employed for :func:`_orm.deferred` - columns, if used at all, in addition to that of :func:`_orm.relationship` - constructs as noted above. See :ref:`orm_queryguide_column_deferral` for - background on deferred column loading. - -.. _dynamic_asyncio: - -* The "dynamic" relationship loader strategy described at - :ref:`dynamic_relationship` is not compatible by default with the asyncio approach. - It can be used directly only if invoked within the - :meth:`_asyncio.AsyncSession.run_sync` method described at - :ref:`session_run_sync`, or by using its ``.statement`` attribute - to obtain a normal select:: - - user = await session.get(User, 42) - addresses = (await session.scalars(user.addresses.statement)).all() - stmt = user.addresses.statement.where(Address.email_address.startswith("patrick")) - addresses_filter = (await session.scalars(stmt)).all() - - The :ref:`write only ` technique, introduced in - version 2.0 of SQLAlchemy, is fully compatible with asyncio and should be - preferred. - - .. seealso:: - - :ref:`migration_20_dynamic_loaders` - notes on migration to 2.0 style - -* If using asyncio with a database that does not support RETURNING, such as - MySQL 8, server default values such as generated timestamps will not be - available on newly flushed objects unless the - :paramref:`_orm.Mapper.eager_defaults` option is used. In SQLAlchemy 2.0, - this behavior is applied automatically to backends like PostgreSQL, SQLite - and MariaDB which use RETURNING to fetch new values when rows are - INSERTed. - -.. _session_run_sync: - -Running Synchronous Methods and Functions under asyncio -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. deepalchemy:: This approach is essentially exposing publicly the - mechanism by which SQLAlchemy is able to provide the asyncio interface - in the first place. While there is no technical issue with doing so, overall - the approach can probably be considered "controversial" as it works against - some of the central philosophies of the asyncio programming model, which - is essentially that any programming statement that can potentially result - in IO being invoked **must** have an ``await`` call, lest the program - does not make it explicitly clear every line at which IO may occur. - This approach does not change that general idea, except that it allows - a series of synchronous IO instructions to be exempted from this rule - within the scope of a function call, essentially bundled up into a single - awaitable. - -As an alternative means of integrating traditional SQLAlchemy "lazy loading" -within an asyncio event loop, an **optional** method known as -:meth:`_asyncio.AsyncSession.run_sync` is provided which will run any -Python function inside of a greenlet, where traditional synchronous -programming concepts will be translated to use ``await`` when they reach the -database driver. A hypothetical approach here is an asyncio-oriented -application can package up database-related methods into functions that are -invoked using :meth:`_asyncio.AsyncSession.run_sync`. - -Altering the above example, if we didn't use :func:`_orm.selectinload` -for the ``A.bs`` collection, we could accomplish our treatment of these -attribute accesses within a separate function:: - - import asyncio - - from sqlalchemy import select - from sqlalchemy.ext.asyncio import AsyncSession, create_async_engine - - - def fetch_and_update_objects(session): - """run traditional sync-style ORM code in a function that will be - invoked within an awaitable. - - """ - - # the session object here is a traditional ORM Session. - # all features are available here including legacy Query use. - - stmt = select(A) - - result = session.execute(stmt) - for a1 in result.scalars(): - print(a1) - - # lazy loads - for b1 in a1.bs: - print(b1) - - # legacy Query use - a1 = session.query(A).order_by(A.id).first() - - a1.data = "new data" - - - async def async_main(): - engine = create_async_engine( - "postgresql+asyncpg://scott:tiger@localhost/test", - echo=True, - ) - async with engine.begin() as conn: - await conn.run_sync(Base.metadata.drop_all) - await conn.run_sync(Base.metadata.create_all) - - async with AsyncSession(engine) as session: - async with session.begin(): - session.add_all( - [ - A(bs=[B(), B()], data="a1"), - A(bs=[B()], data="a2"), - A(bs=[B(), B()], data="a3"), - ] - ) - - await session.run_sync(fetch_and_update_objects) - - await session.commit() - - # for AsyncEngine created in function scope, close and - # clean-up pooled connections - await engine.dispose() - - - asyncio.run(async_main()) - -The above approach of running certain functions within a "sync" runner -has some parallels to an application that runs a SQLAlchemy application -on top of an event-based programming library such as ``gevent``. The -differences are as follows: - -1. unlike when using ``gevent``, we can continue to use the standard Python - asyncio event loop, or any custom event loop, without the need to integrate - into the ``gevent`` event loop. - -2. There is no "monkeypatching" whatsoever. The above example makes use of - a real asyncio driver and the underlying SQLAlchemy connection pool is also - using the Python built-in ``asyncio.Queue`` for pooling connections. - -3. The program can freely switch between async/await code and contained - functions that use sync code with virtually no performance penalty. There - is no "thread executor" or any additional waiters or synchronization in use. - -4. The underlying network drivers are also using pure Python asyncio - concepts, no third party networking libraries as ``gevent`` and ``eventlet`` - provides are in use. - -.. _asyncio_events: - -Using events with the asyncio extension ---------------------------------------- - -The SQLAlchemy :ref:`event system ` is not directly exposed -by the asyncio extension, meaning there is not yet an "async" version of a -SQLAlchemy event handler. - -However, as the asyncio extension surrounds the usual synchronous SQLAlchemy -API, regular "synchronous" style event handlers are freely available as they -would be if asyncio were not used. - -As detailed below, there are two current strategies to register events given -asyncio-facing APIs: - -* Events can be registered at the instance level (e.g. a specific - :class:`_asyncio.AsyncEngine` instance) by associating the event with the - ``sync`` attribute that refers to the proxied object. For example to register - the :meth:`_events.PoolEvents.connect` event against an - :class:`_asyncio.AsyncEngine` instance, use its - :attr:`_asyncio.AsyncEngine.sync_engine` attribute as target. Targets - include: - - :attr:`_asyncio.AsyncEngine.sync_engine` - - :attr:`_asyncio.AsyncConnection.sync_connection` - - :attr:`_asyncio.AsyncConnection.sync_engine` - - :attr:`_asyncio.AsyncSession.sync_session` - -* To register an event at the class level, targeting all instances of the same type (e.g. - all :class:`_asyncio.AsyncSession` instances), use the corresponding - sync-style class. For example to register the - :meth:`_ormevents.SessionEvents.before_commit` event against the - :class:`_asyncio.AsyncSession` class, use the :class:`_orm.Session` class as - the target. - -* To register at the :class:`_orm.sessionmaker` level, combine an explicit - :class:`_orm.sessionmaker` with an :class:`_asyncio.async_sessionmaker` - using :paramref:`_asyncio.async_sessionmaker.sync_session_class`, and - associate events with the :class:`_orm.sessionmaker`. - -When working within an event handler that is within an asyncio context, objects -like the :class:`_engine.Connection` continue to work in their usual -"synchronous" way without requiring ``await`` or ``async`` usage; when messages -are ultimately received by the asyncio database adapter, the calling style is -transparently adapted back into the asyncio calling style. For events that -are passed a DBAPI level connection, such as :meth:`_events.PoolEvents.connect`, -the object is a :term:`pep-249` compliant "connection" object which will adapt -sync-style calls into the asyncio driver. - -Examples of Event Listeners with Async Engines / Sessions / Sessionmakers -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Some examples of sync style event handlers associated with async-facing API -constructs are illustrated below: - -* **Core Events on AsyncEngine** - - In this example, we access the :attr:`_asyncio.AsyncEngine.sync_engine` - attribute of :class:`_asyncio.AsyncEngine` as the target for - :class:`.ConnectionEvents` and :class:`.PoolEvents`:: - - import asyncio - - from sqlalchemy import event - from sqlalchemy import text - from sqlalchemy.engine import Engine - from sqlalchemy.ext.asyncio import create_async_engine - - engine = create_async_engine("postgresql+asyncpg://scott:tiger@localhost:5432/test") - - - # connect event on instance of Engine - @event.listens_for(engine.sync_engine, "connect") - def my_on_connect(dbapi_con, connection_record): - print("New DBAPI connection:", dbapi_con) - cursor = dbapi_con.cursor() - - # sync style API use for adapted DBAPI connection / cursor - cursor.execute("select 'execute from event'") - print(cursor.fetchone()[0]) - - - # before_execute event on all Engine instances - @event.listens_for(Engine, "before_execute") - def my_before_execute( - conn, - clauseelement, - multiparams, - params, - execution_options, - ): - print("before execute!") - - - async def go(): - async with engine.connect() as conn: - await conn.execute(text("select 1")) - await engine.dispose() - - - asyncio.run(go()) - - Output: - - .. sourcecode:: text - - New DBAPI connection: > - execute from event - before execute! - - -* **ORM Events on AsyncSession** - - In this example, we access :attr:`_asyncio.AsyncSession.sync_session` as the - target for :class:`_orm.SessionEvents`:: - - import asyncio - - from sqlalchemy import event - from sqlalchemy import text - from sqlalchemy.ext.asyncio import AsyncSession - from sqlalchemy.ext.asyncio import create_async_engine - from sqlalchemy.orm import Session - - engine = create_async_engine("postgresql+asyncpg://scott:tiger@localhost:5432/test") - - session = AsyncSession(engine) - - - # before_commit event on instance of Session - @event.listens_for(session.sync_session, "before_commit") - def my_before_commit(session): - print("before commit!") - - # sync style API use on Session - connection = session.connection() - - # sync style API use on Connection - result = connection.execute(text("select 'execute from event'")) - print(result.first()) - - - # after_commit event on all Session instances - @event.listens_for(Session, "after_commit") - def my_after_commit(session): - print("after commit!") - - - async def go(): - await session.execute(text("select 1")) - await session.commit() - - await session.close() - await engine.dispose() - - - asyncio.run(go()) - - Output: - - .. sourcecode:: text - - before commit! - execute from event - after commit! - - -* **ORM Events on async_sessionmaker** - - For this use case, we make a :class:`_orm.sessionmaker` as the event target, - then assign it to the :class:`_asyncio.async_sessionmaker` using - the :paramref:`_asyncio.async_sessionmaker.sync_session_class` parameter:: - - import asyncio - - from sqlalchemy import event - from sqlalchemy.ext.asyncio import async_sessionmaker - from sqlalchemy.orm import sessionmaker - - sync_maker = sessionmaker() - maker = async_sessionmaker(sync_session_class=sync_maker) - - - @event.listens_for(sync_maker, "before_commit") - def before_commit(session): - print("before commit") - - - async def main(): - async_session = maker() - - await async_session.commit() - - - asyncio.run(main()) - - Output: - - .. sourcecode:: text - - before commit - - -.. topic:: asyncio and events, two opposites - - SQLAlchemy events by their nature take place within the **interior** of a - particular SQLAlchemy process; that is, an event always occurs *after* some - particular SQLAlchemy API has been invoked by end-user code, and *before* - some other internal aspect of that API occurs. - - Contrast this to the architecture of the asyncio extension, which takes - place on the **exterior** of SQLAlchemy's usual flow from end-user API to - DBAPI function. - - The flow of messaging may be visualized as follows: - - .. sourcecode:: text - - SQLAlchemy SQLAlchemy SQLAlchemy SQLAlchemy plain - asyncio asyncio ORM/Core asyncio asyncio - (public (internal) (internal) - facing) - -------------|------------|------------------------|-----------|------------ - asyncio API | | | | - call -> | | | | - | -> -> | | -> -> | - |~~~~~~~~~~~~| sync API call -> |~~~~~~~~~~~| - | asyncio | event hooks -> | sync | - | to | invoke action -> | to | - | sync | event hooks -> | asyncio | - | (greenlet) | dialect -> | (leave | - |~~~~~~~~~~~~| event hooks -> | greenlet) | - | -> -> | sync adapted |~~~~~~~~~~~| - | | DBAPI -> | -> -> | asyncio - | | | | driver -> database - - - Where above, an API call always starts as asyncio, flows through the - synchronous API, and ends as asyncio, before results are propagated through - this same chain in the opposite direction. In between, the message is - adapted first into sync-style API use, and then back out to async style. - Event hooks then by their nature occur in the middle of the "sync-style API - use". From this it follows that the API presented within event hooks - occurs inside the process by which asyncio API requests have been adapted - to sync, and outgoing messages to the database API will be converted - to asyncio transparently. - -.. _asyncio_events_run_async: - -Using awaitable-only driver methods in connection pool and other events -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -As discussed in the above section, event handlers such as those oriented -around the :class:`.PoolEvents` event handlers receive a sync-style "DBAPI" connection, -which is a wrapper object supplied by SQLAlchemy asyncio dialects to adapt -the underlying asyncio "driver" connection into one that can be used by -SQLAlchemy's internals. A special use case arises when the user-defined -implementation for such an event handler needs to make use of the -ultimate "driver" connection directly, using awaitable only methods on that -driver connection. One such example is the ``.set_type_codec()`` method -supplied by the asyncpg driver. - -To accommodate this use case, SQLAlchemy's :class:`.AdaptedConnection` -class provides a method :meth:`.AdaptedConnection.run_async` that allows -an awaitable function to be invoked within the "synchronous" context of -an event handler or other SQLAlchemy internal. This method is directly -analogous to the :meth:`_asyncio.AsyncConnection.run_sync` method that -allows a sync-style method to run under async. - -:meth:`.AdaptedConnection.run_async` should be passed a function that will -accept the innermost "driver" connection as a single argument, and return -an awaitable that will be invoked by the :meth:`.AdaptedConnection.run_async` -method. The given function itself does not need to be declared as ``async``; -it's perfectly fine for it to be a Python ``lambda:``, as the return awaitable -value will be invoked after being returned:: - - from sqlalchemy import event - from sqlalchemy.ext.asyncio import create_async_engine - - engine = create_async_engine(...) - - - @event.listens_for(engine.sync_engine, "connect") - def register_custom_types(dbapi_connection, *args): - dbapi_connection.run_async( - lambda connection: connection.set_type_codec( - "MyCustomType", - encoder, - decoder, # ... - ) - ) - -Above, the object passed to the ``register_custom_types`` event handler -is an instance of :class:`.AdaptedConnection`, which provides a DBAPI-like -interface to an underlying async-only driver-level connection object. -The :meth:`.AdaptedConnection.run_async` method then provides access to an -awaitable environment where the underlying driver level connection may be -acted upon. - -.. versionadded:: 1.4.30 - - -Using multiple asyncio event loops ----------------------------------- - -An application that makes use of multiple event loops, for example in the -uncommon case of combining asyncio with multithreading, should not share the -same :class:`_asyncio.AsyncEngine` with different event loops when using the -default pool implementation. - -If an :class:`_asyncio.AsyncEngine` is be passed from one event loop to another, -the method :meth:`_asyncio.AsyncEngine.dispose()` should be called before it's -re-used on a new event loop. Failing to do so may lead to a ``RuntimeError`` -along the lines of -``Task got Future attached to a different loop`` - -If the same engine must be shared between different loop, it should be configured -to disable pooling using :class:`~sqlalchemy.pool.NullPool`, preventing the Engine -from using any connection more than once:: - - from sqlalchemy.ext.asyncio import create_async_engine - from sqlalchemy.pool import NullPool - - engine = create_async_engine( - "postgresql+asyncpg://user:pass@host/dbname", - poolclass=NullPool, - ) - -.. _asyncio_scoped_session: - -Using asyncio scoped session ----------------------------- - -The "scoped session" pattern used in threaded SQLAlchemy with the -:class:`.scoped_session` object is also available in asyncio, using -an adapted version called :class:`_asyncio.async_scoped_session`. - -.. tip:: SQLAlchemy generally does not recommend the "scoped" pattern - for new development as it relies upon mutable global state that must also be - explicitly torn down when work within the thread or task is complete. - Particularly when using asyncio, it's likely a better idea to pass the - :class:`_asyncio.AsyncSession` directly to the awaitable functions that need - it. - -When using :class:`_asyncio.async_scoped_session`, as there's no "thread-local" -concept in the asyncio context, the "scopefunc" parameter must be provided to -the constructor. The example below illustrates using the -``asyncio.current_task()`` function for this purpose:: - - from asyncio import current_task - - from sqlalchemy.ext.asyncio import ( - async_scoped_session, - async_sessionmaker, - ) - - async_session_factory = async_sessionmaker( - some_async_engine, - expire_on_commit=False, - ) - AsyncScopedSession = async_scoped_session( - async_session_factory, - scopefunc=current_task, - ) - some_async_session = AsyncScopedSession() - -.. warning:: The "scopefunc" used by :class:`_asyncio.async_scoped_session` - is invoked **an arbitrary number of times** within a task, once for each - time the underlying :class:`_asyncio.AsyncSession` is accessed. The function - should therefore be **idempotent** and lightweight, and should not attempt - to create or mutate any state, such as establishing callbacks, etc. - -.. warning:: Using ``current_task()`` for the "key" in the scope requires that - the :meth:`_asyncio.async_scoped_session.remove` method is called from - within the outermost awaitable, to ensure the key is removed from the - registry when the task is complete, otherwise the task handle as well as - the :class:`_asyncio.AsyncSession` will remain in memory, essentially - creating a memory leak. See the following example which illustrates - the correct use of :meth:`_asyncio.async_scoped_session.remove`. - -:class:`_asyncio.async_scoped_session` includes **proxy -behavior** similar to that of :class:`.scoped_session`, which means it can be -treated as a :class:`_asyncio.AsyncSession` directly, keeping in mind that -the usual ``await`` keywords are necessary, including for the -:meth:`_asyncio.async_scoped_session.remove` method:: - - async def some_function(some_async_session, some_object): - # use the AsyncSession directly - some_async_session.add(some_object) - - # use the AsyncSession via the context-local proxy - await AsyncScopedSession.commit() - - # "remove" the current proxied AsyncSession for the local context - await AsyncScopedSession.remove() - -.. versionadded:: 1.4.19 - -.. currentmodule:: sqlalchemy.ext.asyncio - - -.. _asyncio_inspector: - -Using the Inspector to inspect schema objects ---------------------------------------------------- - -SQLAlchemy does not yet offer an asyncio version of the -:class:`_reflection.Inspector` (introduced at :ref:`metadata_reflection_inspector`), -however the existing interface may be used in an asyncio context by -leveraging the :meth:`_asyncio.AsyncConnection.run_sync` method of -:class:`_asyncio.AsyncConnection`:: - - import asyncio - - from sqlalchemy import inspect - from sqlalchemy.ext.asyncio import create_async_engine - - engine = create_async_engine("postgresql+asyncpg://scott:tiger@localhost/test") - - - def use_inspector(conn): - inspector = inspect(conn) - # use the inspector - print(inspector.get_view_names()) - # return any value to the caller - return inspector.get_table_names() - - - async def async_main(): - async with engine.connect() as conn: - tables = await conn.run_sync(use_inspector) - - - asyncio.run(async_main()) - -.. seealso:: - - :ref:`metadata_reflection` - - :ref:`inspection_toplevel` - -Engine API Documentation -------------------------- - -.. autofunction:: create_async_engine - -.. autofunction:: async_engine_from_config - -.. autofunction:: create_async_pool_from_url - -.. autoclass:: AsyncEngine - :members: - -.. autoclass:: AsyncConnection - :members: - -.. autoclass:: AsyncTransaction - :members: - -Result Set API Documentation ----------------------------------- - -The :class:`_asyncio.AsyncResult` object is an async-adapted version of the -:class:`_result.Result` object. It is only returned when using the -:meth:`_asyncio.AsyncConnection.stream` or :meth:`_asyncio.AsyncSession.stream` -methods, which return a result object that is on top of an active database -cursor. - -.. autoclass:: AsyncResult - :members: - :inherited-members: - -.. autoclass:: AsyncScalarResult - :members: - :inherited-members: - -.. autoclass:: AsyncMappingResult - :members: - :inherited-members: - -.. autoclass:: AsyncTupleResult - -ORM Session API Documentation ------------------------------ - -.. autofunction:: async_object_session - -.. autofunction:: async_session - -.. autoclass:: async_sessionmaker - :members: - :inherited-members: - -.. autoclass:: async_scoped_session - :members: - :inherited-members: - -.. autoclass:: AsyncSession - :members: - :exclude-members: sync_session_class - - .. autoattribute:: sync_session_class - -.. autoclass:: AsyncSessionTransaction - :members: - - - diff --git a/doc/build/orm/extensions/automap.rst b/doc/build/orm/extensions/automap.rst deleted file mode 100644 index d1d200609f..0000000000 --- a/doc/build/orm/extensions/automap.rst +++ /dev/null @@ -1,22 +0,0 @@ -.. _automap_toplevel: - -Automap -======= - -.. automodule:: sqlalchemy.ext.automap - -API Reference -------------- - -.. autofunction:: automap_base - -.. autoclass:: AutomapBase - :members: - -.. autofunction:: classname_for_table - -.. autofunction:: name_for_scalar_relationship - -.. autofunction:: name_for_collection_relationship - -.. autofunction:: generate_relationship diff --git a/doc/build/orm/extensions/baked.rst b/doc/build/orm/extensions/baked.rst deleted file mode 100644 index b495f42a42..0000000000 --- a/doc/build/orm/extensions/baked.rst +++ /dev/null @@ -1,483 +0,0 @@ -.. _baked_toplevel: - -Baked Queries -============= - -.. module:: sqlalchemy.ext.baked - -``baked`` provides an alternative creational pattern for -:class:`~.query.Query` objects, which allows for caching of the object's -construction and string-compilation steps. This means that for a -particular :class:`~.query.Query` building scenario that is used more than -once, all of the Python function invocation involved in building the query -from its initial construction up through generating a SQL string will only -occur **once**, rather than for each time that query is built up and executed. - -The rationale for this system is to greatly reduce Python interpreter -overhead for everything that occurs **before the SQL is emitted**. -The caching of the "baked" system does **not** in any way reduce SQL calls or -cache the **return results** from the database. A technique that demonstrates -the caching of the SQL calls and result sets themselves is available in -:ref:`examples_caching`. - -.. deprecated:: 1.4 SQLAlchemy 1.4 and 2.0 feature an all-new direct query - caching system that removes the need for the :class:`.BakedQuery` system. - Caching is now transparently active for all Core and ORM queries with no - action taken by the user, using the system described at :ref:`sql_caching`. - - -.. deepalchemy:: - - The :mod:`sqlalchemy.ext.baked` extension is **not for beginners**. Using - it correctly requires a good high level understanding of how SQLAlchemy, the - database driver, and the backend database interact with each other. This - extension presents a very specific kind of optimization that is not ordinarily - needed. As noted above, it **does not cache queries**, only the string - formulation of the SQL itself. - -Synopsis --------- - -Usage of the baked system starts by producing a so-called "bakery", which -represents storage for a particular series of query objects:: - - from sqlalchemy.ext import baked - - bakery = baked.bakery() - -The above "bakery" will store cached data in an LRU cache that defaults -to 200 elements, noting that an ORM query will typically contain one entry -for the ORM query as invoked, as well as one entry per database dialect for -the SQL string. - -The bakery allows us to build up a :class:`~.query.Query` object by specifying -its construction as a series of Python callables, which are typically lambdas. -For succinct usage, it overrides the ``+=`` operator so that a typical -query build-up looks like the following:: - - from sqlalchemy import bindparam - - - def search_for_user(session, username, email=None): - baked_query = bakery(lambda session: session.query(User)) - baked_query += lambda q: q.filter(User.name == bindparam("username")) - - baked_query += lambda q: q.order_by(User.id) - - if email: - baked_query += lambda q: q.filter(User.email == bindparam("email")) - - result = baked_query(session).params(username=username, email=email).all() - - return result - -Following are some observations about the above code: - -1. The ``baked_query`` object is an instance of :class:`.BakedQuery`. This - object is essentially the "builder" for a real orm :class:`~.query.Query` - object, but it is not itself the *actual* :class:`~.query.Query` - object. - -2. The actual :class:`~.query.Query` object is not built at all, until the - very end of the function when :meth:`_baked.Result.all` is called. - -3. The steps that are added to the ``baked_query`` object are all expressed - as Python functions, typically lambdas. The first lambda given - to the :func:`.bakery` function receives a :class:`.Session` as its - argument. The remaining lambdas each receive a :class:`~.query.Query` - as their argument. - -4. In the above code, even though our application may call upon - ``search_for_user()`` many times, and even though within each invocation - we build up an entirely new :class:`.BakedQuery` object, - *all of the lambdas are only called once*. Each lambda is **never** called - a second time for as long as this query is cached in the bakery. - -5. The caching is achieved by storing references to the **lambda objects - themselves** in order to formulate a cache key; that is, the fact that the - Python interpreter assigns an in-Python identity to these functions is - what determines how to identify the query on successive runs. For - those invocations of ``search_for_user()`` where the ``email`` parameter - is specified, the callable ``lambda q: q.filter(User.email == bindparam('email'))`` - will be part of the cache key that's retrieved; when ``email`` is - ``None``, this callable is not part of the cache key. - -6. Because the lambdas are all called only once, it is essential that no - variables which may change across calls are referenced **within** the - lambdas; instead, assuming these are values to be bound into the - SQL string, we use :func:`.bindparam` to construct named parameters, - where we apply their actual values later using :meth:`_baked.Result.params`. - - -Performance ------------ - -The baked query probably looks a little odd, a little bit awkward and -a little bit verbose. However, the savings in -Python performance for a query which is invoked lots of times in an -application are very dramatic. The example suite ``short_selects`` -demonstrated in :ref:`examples_performance` illustrates a comparison -of queries which each return only one row, such as the following regular -query:: - - session = Session(bind=engine) - for id_ in random.sample(ids, n): - session.query(Customer).filter(Customer.id == id_).one() - -compared to the equivalent "baked" query:: - - bakery = baked.bakery() - s = Session(bind=engine) - for id_ in random.sample(ids, n): - q = bakery(lambda s: s.query(Customer)) - q += lambda q: q.filter(Customer.id == bindparam("id")) - q(s).params(id=id_).one() - -The difference in Python function call count for an iteration of 10000 -calls to each block are: - -.. sourcecode:: text - - test_baked_query : test a baked query of the full entity. - (10000 iterations); total fn calls 1951294 - - test_orm_query : test a straight ORM query of the full entity. - (10000 iterations); total fn calls 7900535 - -In terms of number of seconds on a powerful laptop, this comes out as: - -.. sourcecode:: text - - test_baked_query : test a baked query of the full entity. - (10000 iterations); total time 2.174126 sec - - test_orm_query : test a straight ORM query of the full entity. - (10000 iterations); total time 7.958516 sec - -Note that this test very intentionally features queries that only return one row. -For queries that return many rows, the performance advantage of the baked query will have -less and less of an impact, proportional to the time spent fetching rows. -It is critical to keep in mind that the **baked query feature only applies to -building the query itself, not the fetching of results**. Using the -baked feature is by no means a guarantee to a much faster application; it is -only a potentially useful feature for those applications that have been measured -as being impacted by this particular form of overhead. - -.. topic:: Measure twice, cut once - - For background on how to profile a SQLAlchemy application, please see - the section :ref:`faq_performance`. It is essential that performance - measurement techniques are used when attempting to improve the performance - of an application. - -Rationale ---------- - -The "lambda" approach above is a superset of what would be a more -traditional "parameterized" approach. Suppose we wished to build -a simple system where we build a :class:`~.query.Query` just once, then -store it in a dictionary for re-use. This is possible right now by -just building up the query, and removing its :class:`.Session` by calling -``my_cached_query = query.with_session(None)``:: - - my_simple_cache = {} - - - def lookup(session, id_argument): - if "my_key" not in my_simple_cache: - query = session.query(Model).filter(Model.id == bindparam("id")) - my_simple_cache["my_key"] = query.with_session(None) - else: - query = my_simple_cache["my_key"].with_session(session) - - return query.params(id=id_argument).all() - -The above approach gets us a very minimal performance benefit. -By re-using a :class:`~.query.Query`, we save on the Python work within -the ``session.query(Model)`` constructor as well as calling upon -``filter(Model.id == bindparam('id'))``, which will skip for us the building -up of the Core expression as well as sending it to :meth:`_query.Query.filter`. -However, the approach still regenerates the full :class:`_expression.Select` -object every time when :meth:`_query.Query.all` is called and additionally this -brand new :class:`_expression.Select` is sent off to the string compilation step every -time, which for a simple case like the above is probably about 70% of the -overhead. - -To reduce the additional overhead, we need some more specialized logic, -some way to memoize the construction of the select object and the -construction of the SQL. There is an example of this on the wiki -in the section `BakedQuery `_, -a precursor to this feature, however in that system, we aren't caching -the *construction* of the query. In order to remove all the overhead, -we need to cache both the construction of the query as well as the SQL -compilation. Let's assume we adapted the recipe in this way -and made ourselves a method ``.bake()`` that pre-compiles the SQL for the -query, producing a new object that can be invoked with minimal overhead. -Our example becomes:: - - my_simple_cache = {} - - - def lookup(session, id_argument): - if "my_key" not in my_simple_cache: - query = session.query(Model).filter(Model.id == bindparam("id")) - my_simple_cache["my_key"] = query.with_session(None).bake() - else: - query = my_simple_cache["my_key"].with_session(session) - - return query.params(id=id_argument).all() - -Above, we've fixed the performance situation, but we still have this -string cache key to deal with. - -We can use the "bakery" approach to re-frame the above in a way that -looks less unusual than the "building up lambdas" approach, and more like -a simple improvement upon the simple "reuse a query" approach:: - - bakery = baked.bakery() - - - def lookup(session, id_argument): - def create_model_query(session): - return session.query(Model).filter(Model.id == bindparam("id")) - - parameterized_query = bakery.bake(create_model_query) - return parameterized_query(session).params(id=id_argument).all() - -Above, we use the "baked" system in a manner that is -very similar to the simplistic "cache a query" system. However, it -uses two fewer lines of code, does not need to manufacture a cache key of -"my_key", and also includes the same feature as our custom "bake" function -that caches 100% of the Python invocation work from the -constructor of the query, to the filter call, to the production -of the :class:`_expression.Select` object, to the string compilation step. - -From the above, if we ask ourselves, "what if lookup needs to make conditional decisions -as to the structure of the query?", this is where hopefully it becomes apparent -why "baked" is the way it is. Instead of a parameterized query building -off from exactly one function (which is how we thought baked might work -originally), we can build it from *any number* of functions. Consider -our naive example, if we needed to have an additional clause in our -query on a conditional basis:: - - my_simple_cache = {} - - - def lookup(session, id_argument, include_frobnizzle=False): - if include_frobnizzle: - cache_key = "my_key_with_frobnizzle" - else: - cache_key = "my_key_without_frobnizzle" - - if cache_key not in my_simple_cache: - query = session.query(Model).filter(Model.id == bindparam("id")) - if include_frobnizzle: - query = query.filter(Model.frobnizzle == True) - - my_simple_cache[cache_key] = query.with_session(None).bake() - else: - query = my_simple_cache[cache_key].with_session(session) - - return query.params(id=id_argument).all() - -Our "simple" parameterized system must now be tasked with generating -cache keys which take into account whether or not the "include_frobnizzle" -flag was passed, as the presence of this flag means that the generated -SQL would be entirely different. It should be apparent that as the -complexity of query building goes up, the task of caching these queries -becomes burdensome very quickly. We can convert the above example -into a direct use of "bakery" as follows:: - - - bakery = baked.bakery() - - - def lookup(session, id_argument, include_frobnizzle=False): - def create_model_query(session): - return session.query(Model).filter(Model.id == bindparam("id")) - - parameterized_query = bakery.bake(create_model_query) - - if include_frobnizzle: - - def include_frobnizzle_in_query(query): - return query.filter(Model.frobnizzle == True) - - parameterized_query = parameterized_query.with_criteria( - include_frobnizzle_in_query - ) - - return parameterized_query(session).params(id=id_argument).all() - -Above, we again cache not just the query object but all the work it needs -to do in order to generate SQL. We also no longer need to deal with -making sure we generate a cache key that accurately takes into account -all of the structural modifications we've made; this is now handled -automatically and without the chance of mistakes. - -This code sample is a few lines shorter than the naive example, removes -the need to deal with cache keys, and has the vast performance benefits -of the full so-called "baked" feature. But -still a little verbose! Hence we take methods like :meth:`.BakedQuery.add_criteria` -and :meth:`.BakedQuery.with_criteria` and shorten them into operators, and -encourage (though certainly not require!) using simple lambdas, only as a -means to reduce verbosity:: - - bakery = baked.bakery() - - - def lookup(session, id_argument, include_frobnizzle=False): - parameterized_query = bakery.bake( - lambda s: s.query(Model).filter(Model.id == bindparam("id")) - ) - - if include_frobnizzle: - parameterized_query += lambda q: q.filter(Model.frobnizzle == True) - - return parameterized_query(session).params(id=id_argument).all() - -Where above, the approach is simpler to implement and much more similar -in code flow to what a non-cached querying function would look like, -hence making code easier to port. - -The above description is essentially a summary of the design process used -to arrive at the current "baked" approach. Starting from the -"normal" approaches, the additional issues of cache key construction and -management, removal of all redundant Python execution, and queries built up -with conditionals needed to be addressed, leading to the final approach. - -Special Query Techniques ------------------------- - -This section will describe some techniques for specific query situations. - -.. _baked_in: - -Using IN expressions -^^^^^^^^^^^^^^^^^^^^ - -The :meth:`.ColumnOperators.in_` method in SQLAlchemy historically renders -a variable set of bound parameters based on the list of items that's passed -to the method. This doesn't work for baked queries as the length of that -list can change on different calls. To solve this problem, the -:paramref:`.bindparam.expanding` parameter supports a late-rendered IN -expression that is safe to be cached inside of baked query. The actual list -of elements is rendered at statement execution time, rather than at -statement compilation time:: - - bakery = baked.bakery() - - baked_query = bakery(lambda session: session.query(User)) - baked_query += lambda q: q.filter(User.name.in_(bindparam("username", expanding=True))) - - result = baked_query.with_session(session).params(username=["ed", "fred"]).all() - -.. seealso:: - - :paramref:`.bindparam.expanding` - - :meth:`.ColumnOperators.in_` - -Using Subqueries -^^^^^^^^^^^^^^^^ - -When using :class:`_query.Query` objects, it is often needed that one :class:`_query.Query` -object is used to generate a subquery within another. In the case where the -:class:`_query.Query` is currently in baked form, an interim method may be used to -retrieve the :class:`_query.Query` object, using the :meth:`.BakedQuery.to_query` -method. This method is passed the :class:`.Session` or :class:`_query.Query` that is -the argument to the lambda callable used to generate a particular step -of the baked query:: - - bakery = baked.bakery() - - # a baked query that will end up being used as a subquery - my_subq = bakery(lambda s: s.query(User.id)) - my_subq += lambda q: q.filter(User.id == Address.user_id) - - # select a correlated subquery in the top columns list, - # we have the "session" argument, pass that - my_q = bakery(lambda s: s.query(Address.id, my_subq.to_query(s).as_scalar())) - - # use a correlated subquery in some of the criteria, we have - # the "query" argument, pass that. - my_q += lambda q: q.filter(my_subq.to_query(q).exists()) - -.. versionadded:: 1.3 - -.. _baked_with_before_compile: - -Using the before_compile event -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -As of SQLAlchemy 1.3.11, the use of the :meth:`.QueryEvents.before_compile` -event against a particular :class:`_query.Query` will disallow the baked query -system from caching the query, if the event hook returns a new :class:`_query.Query` -object that is different from the one passed in. This is so that the -:meth:`.QueryEvents.before_compile` hook may be invoked against a particular -:class:`_query.Query` every time it is used, to accommodate for hooks that -alter the query differently each time. To allow a -:meth:`.QueryEvents.before_compile` to alter a :meth:`_query.Query` object, but -still to allow the result to be cached, the event can be registered -passing the ``bake_ok=True`` flag:: - - @event.listens_for(Query, "before_compile", retval=True, bake_ok=True) - def my_event(query): - for desc in query.column_descriptions: - if desc["type"] is User: - entity = desc["entity"] - query = query.filter(entity.deleted == False) - return query - -The above strategy is appropriate for an event that will modify a -given :class:`_query.Query` in exactly the same way every time, not dependent -on specific parameters or external state that changes. - -.. versionadded:: 1.3.11 - added the "bake_ok" flag to the - :meth:`.QueryEvents.before_compile` event and disallowed caching via - the "baked" extension from occurring for event handlers that - return a new :class:`_query.Query` object if this flag is not set. - - -Disabling Baked Queries Session-wide ------------------------------------- - -The flag :paramref:`.Session.enable_baked_queries` may be set to False, -causing all baked queries to not use the cache when used against that -:class:`.Session`:: - - session = Session(engine, enable_baked_queries=False) - -Like all session flags, it is also accepted by factory objects like -:class:`.sessionmaker` and methods like :meth:`.sessionmaker.configure`. - -The immediate rationale for this flag is so that an application -which is seeing issues potentially due to cache key conflicts from user-defined -baked queries or other baked query issues can turn the behavior off, in -order to identify or eliminate baked queries as the cause of an issue. - -.. versionadded:: 1.2 - -Lazy Loading Integration ------------------------- - -.. versionchanged:: 1.4 As of SQLAlchemy 1.4, the "baked query" system is no - longer part of the relationship loading system. - The :ref:`native caching ` system is used instead. - - -API Documentation ------------------ - -.. autofunction:: bakery - -.. autoclass:: BakedQuery - :members: - -.. autoclass:: Bakery - :members: - -.. autoclass:: Result - :members: - :noindex: - diff --git a/doc/build/orm/extensions/declarative/api.rst b/doc/build/orm/extensions/declarative/api.rst deleted file mode 100644 index 98924c2e27..0000000000 --- a/doc/build/orm/extensions/declarative/api.rst +++ /dev/null @@ -1,28 +0,0 @@ -:orphan: - -.. automodule:: sqlalchemy.ext.declarative - -=============== -Declarative API -=============== - -API Reference -============= - -.. versionchanged:: 1.4 The fundamental structures of the declarative - system are now part of SQLAlchemy ORM directly. For these components - see: - - * :func:`_orm.declarative_base` - - * :class:`_orm.declared_attr` - - * :func:`_orm.has_inherited_table` - - * :func:`_orm.synonym_for` - - * :meth:`_orm.as_declarative` - -See :ref:`declarative_toplevel` for the remaining Declarative extension -classes. - diff --git a/doc/build/orm/extensions/declarative/basic_use.rst b/doc/build/orm/extensions/declarative/basic_use.rst deleted file mode 100644 index 49903559d5..0000000000 --- a/doc/build/orm/extensions/declarative/basic_use.rst +++ /dev/null @@ -1,40 +0,0 @@ -:orphan: - -========= -Basic Use -========= - -This section has moved to :ref:`orm_declarative_mapping`. - -Defining Attributes -=================== - -This section is covered by :ref:`mapping_columns_toplevel` - - - -Accessing the MetaData -====================== - -This section has moved to :ref:`orm_declarative_metadata`. - - -Class Constructor -================= - -This section has moved to :ref:`orm_mapper_configuration_overview`. - -Mapper Configuration -==================== - -This section is moved to :ref:`orm_declarative_mapper_options`. - - -.. _declarative_sql_expressions: - -Defining SQL Expressions -======================== - -See :ref:`mapper_sql_expressions` for examples on declaratively -mapping attributes to SQL expressions. - diff --git a/doc/build/orm/extensions/declarative/index.rst b/doc/build/orm/extensions/declarative/index.rst deleted file mode 100644 index 6cf1a60a1c..0000000000 --- a/doc/build/orm/extensions/declarative/index.rst +++ /dev/null @@ -1,24 +0,0 @@ -.. _declarative_toplevel: - -.. currentmodule:: sqlalchemy.ext.declarative - -====================== -Declarative Extensions -====================== - -Extensions specific to the :ref:`Declarative ` -mapping API. - -.. versionchanged:: 1.4 The vast majority of the Declarative extension is now - integrated into the SQLAlchemy ORM and is importable from the - ``sqlalchemy.orm`` namespace. See the documentation at - :ref:`orm_declarative_mapping` for new documentation. - For an overview of the change, see :ref:`change_5508`. - -.. autoclass:: AbstractConcreteBase - -.. autoclass:: ConcreteBase - -.. autoclass:: DeferredReflection - :members: - diff --git a/doc/build/orm/extensions/declarative/inheritance.rst b/doc/build/orm/extensions/declarative/inheritance.rst deleted file mode 100644 index 849664a3c3..0000000000 --- a/doc/build/orm/extensions/declarative/inheritance.rst +++ /dev/null @@ -1,8 +0,0 @@ -:orphan: - -.. _declarative_inheritance: - -Declarative Inheritance -======================= - -See :ref:`inheritance_toplevel` for this section. diff --git a/doc/build/orm/extensions/declarative/mixins.rst b/doc/build/orm/extensions/declarative/mixins.rst deleted file mode 100644 index 7a18f07a7f..0000000000 --- a/doc/build/orm/extensions/declarative/mixins.rst +++ /dev/null @@ -1,8 +0,0 @@ -:orphan: - -.. _declarative_mixins: - -Mixin and Custom Base Classes -============================= - -See :ref:`orm_mixins_toplevel` for this section. diff --git a/doc/build/orm/extensions/declarative/relationships.rst b/doc/build/orm/extensions/declarative/relationships.rst deleted file mode 100644 index c0df8b49cf..0000000000 --- a/doc/build/orm/extensions/declarative/relationships.rst +++ /dev/null @@ -1,25 +0,0 @@ -:orphan: - -.. _declarative_configuring_relationships: - -========================= -Configuring Relationships -========================= - -This section is covered by :ref:`orm_declarative_properties`. - -.. _declarative_relationship_eval: - -Evaluation of relationship arguments -===================================== - -This section is moved to :ref:`orm_declarative_relationship_eval`. - - -.. _declarative_many_to_many: - -Configuring Many-to-Many Relationships -====================================== - -This section is moved to :ref:`orm_declarative_relationship_secondary_eval`. - diff --git a/doc/build/orm/extensions/declarative/table_config.rst b/doc/build/orm/extensions/declarative/table_config.rst deleted file mode 100644 index 05ad46d6cc..0000000000 --- a/doc/build/orm/extensions/declarative/table_config.rst +++ /dev/null @@ -1,24 +0,0 @@ -:orphan: - -.. _declarative_table_args: - -=================== -Table Configuration -=================== - -This section has moved; see :ref:`orm_declarative_table_configuration`. - - -.. _declarative_hybrid_table: - -Using a Hybrid Approach with __table__ -====================================== - -This section has moved; see :ref:`orm_imperative_table_configuration`. - - -Using Reflection with Declarative -================================= - -This section has moved to :ref:`orm_declarative_reflected`. - diff --git a/doc/build/orm/extensions/horizontal_shard.rst b/doc/build/orm/extensions/horizontal_shard.rst deleted file mode 100644 index b0467f1abe..0000000000 --- a/doc/build/orm/extensions/horizontal_shard.rst +++ /dev/null @@ -1,19 +0,0 @@ -.. _horizontal_sharding_toplevel: - -Horizontal Sharding -=================== - -.. automodule:: sqlalchemy.ext.horizontal_shard - -API Documentation ------------------ - -.. autoclass:: ShardedSession - :members: - -.. autoclass:: set_shard_id - :members: - -.. autoclass:: ShardedQuery - :members: - diff --git a/doc/build/orm/extensions/hybrid.rst b/doc/build/orm/extensions/hybrid.rst deleted file mode 100644 index 9773316d49..0000000000 --- a/doc/build/orm/extensions/hybrid.rst +++ /dev/null @@ -1,21 +0,0 @@ -.. _hybrids_toplevel: - -Hybrid Attributes -================= - -.. automodule:: sqlalchemy.ext.hybrid - -API Reference -------------- - -.. autoclass:: hybrid_method - :members: - -.. autoclass:: hybrid_property - :members: - -.. autoclass:: Comparator - - -.. autoclass:: HybridExtensionType - :members: diff --git a/doc/build/orm/extensions/index.rst b/doc/build/orm/extensions/index.rst deleted file mode 100644 index 0dda58affa..0000000000 --- a/doc/build/orm/extensions/index.rst +++ /dev/null @@ -1,30 +0,0 @@ -.. _plugins: -.. _sqlalchemy.ext: - -ORM Extensions -============== - -SQLAlchemy has a variety of ORM extensions available, which add additional -functionality to the core behavior. - -The extensions build almost entirely on public core and ORM APIs and users should -be encouraged to read their source code to further their understanding of their -behavior. In particular the "Horizontal Sharding", "Hybrid Attributes", and -"Mutation Tracking" extensions are very succinct. - -.. toctree:: - :maxdepth: 1 - - asyncio - associationproxy - automap - baked - declarative/index - mypy - mutable - orderinglist - horizontal_shard - hybrid - indexable - instrumentation - diff --git a/doc/build/orm/extensions/indexable.rst b/doc/build/orm/extensions/indexable.rst deleted file mode 100644 index 8639e5782d..0000000000 --- a/doc/build/orm/extensions/indexable.rst +++ /dev/null @@ -1,13 +0,0 @@ -.. _indexable_toplevel: - -Indexable -========= - -.. automodule:: sqlalchemy.ext.indexable - -API Reference -------------- - -.. autoclass:: sqlalchemy.ext.indexable.index_property - :members: - diff --git a/doc/build/orm/extensions/instrumentation.rst b/doc/build/orm/extensions/instrumentation.rst deleted file mode 100644 index 422721ffee..0000000000 --- a/doc/build/orm/extensions/instrumentation.rst +++ /dev/null @@ -1,25 +0,0 @@ -.. _instrumentation_toplevel: - -Alternate Class Instrumentation -=============================== - -.. automodule:: sqlalchemy.ext.instrumentation - -API Reference -------------- - -.. autodata:: INSTRUMENTATION_MANAGER - -.. autoclass:: sqlalchemy.orm.instrumentation.InstrumentationFactory - -.. autoclass:: InstrumentationManager - :members: - :undoc-members: - -.. autodata:: instrumentation_finders - -.. autoclass:: ExtendedInstrumentationRegistry - :members: - - - diff --git a/doc/build/orm/extensions/mutable.rst b/doc/build/orm/extensions/mutable.rst deleted file mode 100644 index 3e49b86cb1..0000000000 --- a/doc/build/orm/extensions/mutable.rst +++ /dev/null @@ -1,34 +0,0 @@ -.. _mutable_toplevel: - -Mutation Tracking -================= - -.. automodule:: sqlalchemy.ext.mutable - -API Reference -------------- - -.. autoclass:: MutableBase - :members: _parents, coerce - -.. autoclass:: Mutable - :members: - :inherited-members: - :private-members: - -.. autoclass:: MutableComposite - :members: - -.. autoclass:: MutableDict - :members: - :undoc-members: - -.. autoclass:: MutableList - :members: - :undoc-members: - -.. autoclass:: MutableSet - :members: - :undoc-members: - - diff --git a/doc/build/orm/extensions/mypy.rst b/doc/build/orm/extensions/mypy.rst deleted file mode 100644 index 6639924e94..0000000000 --- a/doc/build/orm/extensions/mypy.rst +++ /dev/null @@ -1,593 +0,0 @@ -.. _mypy_toplevel: - -Mypy / Pep-484 Support for ORM Mappings -======================================== - -Support for :pep:`484` typing annotations as well as the -MyPy_ type checking tool when using SQLAlchemy -:ref:`declarative ` mappings -that refer to the :class:`_schema.Column` object directly, rather than -the :func:`_orm.mapped_column` construct introduced in SQLAlchemy 2.0. - -.. topic:: SQLAlchemy Mypy Plugin Status Update - - **Updated December 2022** - - For SQLAlchemy 2.0, the Mypy plugin continues to work at the level at which - it reached in the SQLAlchemy 1.4 release. However, SQLAlchemy 2.0, - when released, will feature an - :ref:`all new typing system ` - for ORM Declarative models that removes the need for the Mypy plugin and - delivers much more consistent behavior with generally superior capabilities. - Note that this new capability is **not - part of SQLAlchemy 1.4, it is only in SQLAlchemy 2.0, which is out with beta - releases as of December 2022**. - - The SQLAlchemy Mypy plugin, while it has technically never left the "alpha" - stage, should **now be considered as deprecated in SQLAlchemy 2.0, even - though it is still necessary for full Mypy support when using - SQLAlchemy 1.4**. - - The Mypy plugin itself does not solve the issue of supplying correct typing - with other typing tools such as Pylance/Pyright, Pytype, Pycharm, etc, which - cannot make use of Mypy plugins. Additionally, Mypy plugins are extremely - difficult to develop, maintain and test, as a Mypy plugin must be deeply - integrated with Mypy's internal datastructures and processes, which itself - are not stable within the Mypy project itself. The SQLAlchemy Mypy plugin - has lots of limitations when used with code that deviates from very basic - patterns which are reported regularly. - - For these reasons, new non-regression issues reported against the Mypy - plugin are unlikely to be fixed. When SQLAlchemy 2.0 is released, it will - continue to include the plugin, which will have been updated to continue to - function as well as it does in SQLAlchemy 1.4, when running under SQLAlchemy - 2.0. **Existing code that passes Mypy checks using the plugin with - SQLAlchemy 1.4 installed will continue to pass all checks in SQLAlchemy 2.0 - without any changes required, provided the plugin is still used. The - upcoming API to be released with SQLAlchemy 2.0 is fully backwards - compatible with the SQLAlchemy 1.4 API and Mypy plugin behavior.** - - End-user code that passes all checks under SQLAlchemy 1.4 with the Mypy - plugin will be able to incrementally migrate to the new structures, once - that code is running exclusively on SQLAlchemy 2.0. See the section - :ref:`whatsnew_20_orm_declarative_typing` for background on how this - migration may proceed. - - Code that is running exclusively on SQLAlchemy version - 2.0 and has fully migrated to the new declarative constructs will enjoy full - compliance with pep-484 as well as working correctly within IDEs and other - typing tools, without the need for plugins. - - -Installation ------------- - -For **SQLAlchemy 2.0 only**: No stubs should be installed and packages -like sqlalchemy-stubs_ and sqlalchemy2-stubs_ should be fully uninstalled. - -The Mypy_ package itself is a dependency. - -Mypy may be installed using the "mypy" extras hook using pip: - -.. sourcecode:: text - - pip install sqlalchemy[mypy] - -The plugin itself is configured as described in -`Configuring mypy to use Plugins `_, -using the ``sqlalchemy.ext.mypy.plugin`` module name, such as within -``setup.cfg``:: - - [mypy] - plugins = sqlalchemy.ext.mypy.plugin - -.. _sqlalchemy-stubs: https://github.com/dropbox/sqlalchemy-stubs - -.. _sqlalchemy2-stubs: https://github.com/sqlalchemy/sqlalchemy2-stubs - -What the Plugin Does --------------------- - -The primary purpose of the Mypy plugin is to intercept and alter the static -definition of SQLAlchemy -:ref:`declarative mappings ` so that -they match up to how they are structured after they have been -:term:`instrumented` by their :class:`_orm.Mapper` objects. This allows both -the class structure itself as well as code that uses the class to make sense to -the Mypy tool, which otherwise would not be the case based on how declarative -mappings currently function. The plugin is not unlike similar plugins -that are required for libraries like -`dataclasses `_ which -alter classes dynamically at runtime. - -To cover the major areas where this occurs, consider the following ORM -mapping, using the typical example of the ``User`` class:: - - from sqlalchemy import Column, Integer, String, select - from sqlalchemy.orm import declarative_base - - # "Base" is a class that is created dynamically from the - # declarative_base() function - Base = declarative_base() - - - class User(Base): - __tablename__ = "user" - - id = Column(Integer, primary_key=True) - name = Column(String) - - - # "some_user" is an instance of the User class, which - # accepts "id" and "name" kwargs based on the mapping - some_user = User(id=5, name="user") - - # it has an attribute called .name that's a string - print(f"Username: {some_user.name}") - - # a select() construct makes use of SQL expressions derived from the - # User class itself - select_stmt = select(User).where(User.id.in_([3, 4, 5])).where(User.name.contains("s")) - -Above, the steps that the Mypy extension can take include: - -* Interpretation of the ``Base`` dynamic class generated by - :func:`_orm.declarative_base`, so that classes which inherit from it - are known to be mapped. It also can accommodate the class decorator - approach described at :ref:`orm_declarative_decorator`. - -* Type inference for ORM mapped attributes that are defined in declarative - "inline" style, in the above example the ``id`` and ``name`` attributes of - the ``User`` class. This includes that an instance of ``User`` will use - ``int`` for ``id`` and ``str`` for ``name``. It also includes that when the - ``User.id`` and ``User.name`` class-level attributes are accessed, as they - are above in the ``select()`` statement, they are compatible with SQL - expression behavior, which is derived from the - :class:`_orm.InstrumentedAttribute` attribute descriptor class. - -* Application of an ``__init__()`` method to mapped classes that do not - already include an explicit constructor, which accepts keyword arguments - of specific types for all mapped attributes detected. - -When the Mypy plugin processes the above file, the resulting static class -definition and Python code passed to the Mypy tool is equivalent to the -following:: - - from sqlalchemy import Column, Integer, String, select - from sqlalchemy.orm import Mapped - from sqlalchemy.orm.decl_api import DeclarativeMeta - - - class Base(metaclass=DeclarativeMeta): - __abstract__ = True - - - class User(Base): - __tablename__ = "user" - - id: Mapped[Optional[int]] = Mapped._special_method( - Column(Integer, primary_key=True) - ) - name: Mapped[Optional[str]] = Mapped._special_method(Column(String)) - - def __init__(self, id: Optional[int] = ..., name: Optional[str] = ...) -> None: - ... - - - some_user = User(id=5, name="user") - - print(f"Username: {some_user.name}") - - select_stmt = select(User).where(User.id.in_([3, 4, 5])).where(User.name.contains("s")) - -The key steps which have been taken above include: - -* The ``Base`` class is now defined in terms of the :class:`_orm.DeclarativeMeta` - class explicitly, rather than being a dynamic class. - -* The ``id`` and ``name`` attributes are defined in terms of the - :class:`_orm.Mapped` class, which represents a Python descriptor that - exhibits different behaviors at the class vs. instance levels. The - :class:`_orm.Mapped` class is now the base class for the :class:`_orm.InstrumentedAttribute` - class that is used for all ORM mapped attributes. - - :class:`_orm.Mapped` is defined as a generic class against arbitrary Python - types, meaning specific occurrences of :class:`_orm.Mapped` are associated - with a specific Python type, such as ``Mapped[Optional[int]]`` and - ``Mapped[Optional[str]]`` above. - -* The right-hand side of the declarative mapped attribute assignments are - **removed**, as this resembles the operation that the :class:`_orm.Mapper` - class would normally be doing, which is that it would be replacing these - attributes with specific instances of :class:`_orm.InstrumentedAttribute`. - The original expression is moved into a function call that will allow it to - still be type-checked without conflicting with the left-hand side of the - expression. For Mypy purposes, the left-hand typing annotation is sufficient - for the attribute's behavior to be understood. - -* A type stub for the ``User.__init__()`` method is added which includes the - correct keywords and datatypes. - -Usage ------- - -The following subsections will address individual uses cases that have -so far been considered for pep-484 compliance. - - -Introspection of Columns based on TypeEngine -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -For mapped columns that include an explicit datatype, when they are mapped -as inline attributes, the mapped type will be introspected automatically:: - - class MyClass(Base): - # ... - - id = Column(Integer, primary_key=True) - name = Column("employee_name", String(50), nullable=False) - other_name = Column(String(50)) - -Above, the ultimate class-level datatypes of ``id``, ``name`` and -``other_name`` will be introspected as ``Mapped[Optional[int]]``, -``Mapped[Optional[str]]`` and ``Mapped[Optional[str]]``. The types are by -default **always** considered to be ``Optional``, even for the primary key and -non-nullable column. The reason is because while the database columns "id" and -"name" can't be NULL, the Python attributes ``id`` and ``name`` most certainly -can be ``None`` without an explicit constructor:: - - >>> m1 = MyClass() - >>> m1.id - None - -The types of the above columns can be stated **explicitly**, providing the -two advantages of clearer self-documentation as well as being able to -control which types are optional:: - - class MyClass(Base): - # ... - - id: int = Column(Integer, primary_key=True) - name: str = Column("employee_name", String(50), nullable=False) - other_name: Optional[str] = Column(String(50)) - -The Mypy plugin will accept the above ``int``, ``str`` and ``Optional[str]`` -and convert them to include the ``Mapped[]`` type surrounding them. The -``Mapped[]`` construct may also be used explicitly:: - - from sqlalchemy.orm import Mapped - - - class MyClass(Base): - # ... - - id: Mapped[int] = Column(Integer, primary_key=True) - name: Mapped[str] = Column("employee_name", String(50), nullable=False) - other_name: Mapped[Optional[str]] = Column(String(50)) - -When the type is non-optional, it simply means that the attribute as accessed -from an instance of ``MyClass`` will be considered to be non-None:: - - mc = MyClass(...) - - # will pass mypy --strict - name: str = mc.name - -For optional attributes, Mypy considers that the type must include None -or otherwise be ``Optional``:: - - mc = MyClass(...) - - # will pass mypy --strict - other_name: Optional[str] = mc.name - -Whether or not the mapped attribute is typed as ``Optional``, the -generation of the ``__init__()`` method will **still consider all keywords -to be optional**. This is again matching what the SQLAlchemy ORM actually -does when it creates the constructor, and should not be confused with the -behavior of a validating system such as Python ``dataclasses`` which will -generate a constructor that matches the annotations in terms of optional -vs. required attributes. - - -Columns that Don't have an Explicit Type -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Columns that include a :class:`_schema.ForeignKey` modifier do not need -to specify a datatype in a SQLAlchemy declarative mapping. For -this type of attribute, the Mypy plugin will inform the user that it -needs an explicit type to be sent:: - - # .. other imports - from sqlalchemy.sql.schema import ForeignKey - - Base = declarative_base() - - - class User(Base): - __tablename__ = "user" - - id = Column(Integer, primary_key=True) - name = Column(String) - - - class Address(Base): - __tablename__ = "address" - - id = Column(Integer, primary_key=True) - user_id = Column(ForeignKey("user.id")) - -The plugin will deliver the message as follows: - -.. sourcecode:: text - - $ mypy test3.py --strict - test3.py:20: error: [SQLAlchemy Mypy plugin] Can't infer type from - ORM mapped expression assigned to attribute 'user_id'; please specify a - Python type or Mapped[] on the left hand side. - Found 1 error in 1 file (checked 1 source file) - -To resolve, apply an explicit type annotation to the ``Address.user_id`` -column:: - - class Address(Base): - __tablename__ = "address" - - id = Column(Integer, primary_key=True) - user_id: int = Column(ForeignKey("user.id")) - -Mapping Columns with Imperative Table -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -In :ref:`imperative table style `, the -:class:`_schema.Column` definitions are given inside of a :class:`_schema.Table` -construct which is separate from the mapped attributes themselves. The Mypy -plugin does not consider this :class:`_schema.Table`, but instead supports that -the attributes can be explicitly stated with a complete annotation that -**must** use the :class:`_orm.Mapped` class to identify them as mapped attributes:: - - class MyClass(Base): - __table__ = Table( - "mytable", - Base.metadata, - Column(Integer, primary_key=True), - Column("employee_name", String(50), nullable=False), - Column(String(50)), - ) - - id: Mapped[int] - name: Mapped[str] - other_name: Mapped[Optional[str]] - -The above :class:`_orm.Mapped` annotations are considered as mapped columns and -will be included in the default constructor, as well as provide the correct -typing profile for ``MyClass`` both at the class level and the instance level. - -Mapping Relationships -^^^^^^^^^^^^^^^^^^^^^^ - -The plugin has limited support for using type inference to detect the types -for relationships. For all those cases where it can't detect the type, -it will emit an informative error message, and in all cases the appropriate -type may be provided explicitly, either with the :class:`_orm.Mapped` -class or optionally omitting it for an inline declaration. The plugin -also needs to determine whether or not the relationship refers to a collection -or a scalar, and for that it relies upon the explicit value of -the :paramref:`_orm.relationship.uselist` and/or :paramref:`_orm.relationship.collection_class` -parameters. An explicit type is needed if neither of these parameters are -present, as well as if the target type of the :func:`_orm.relationship` -is a string or callable, and not a class:: - - class User(Base): - __tablename__ = "user" - - id = Column(Integer, primary_key=True) - name = Column(String) - - - class Address(Base): - __tablename__ = "address" - - id = Column(Integer, primary_key=True) - user_id: int = Column(ForeignKey("user.id")) - - user = relationship(User) - -The above mapping will produce the following error: - -.. sourcecode:: text - - test3.py:22: error: [SQLAlchemy Mypy plugin] Can't infer scalar or - collection for ORM mapped expression assigned to attribute 'user' - if both 'uselist' and 'collection_class' arguments are absent from the - relationship(); please specify a type annotation on the left hand side. - Found 1 error in 1 file (checked 1 source file) - -The error can be resolved either by using ``relationship(User, uselist=False)`` -or by providing the type, in this case the scalar ``User`` object:: - - class Address(Base): - __tablename__ = "address" - - id = Column(Integer, primary_key=True) - user_id: int = Column(ForeignKey("user.id")) - - user: User = relationship(User) - -For collections, a similar pattern applies, where in the absence of -``uselist=True`` or a :paramref:`_orm.relationship.collection_class`, -a collection annotation such as ``List`` may be used. It is also fully -appropriate to use the string name of the class in the annotation as supported -by pep-484, ensuring the class is imported with in -the `TYPE_CHECKING block `_ -as appropriate:: - - from typing import TYPE_CHECKING, List - - from .mymodel import Base - - if TYPE_CHECKING: - # if the target of the relationship is in another module - # that cannot normally be imported at runtime - from .myaddressmodel import Address - - - class User(Base): - __tablename__ = "user" - - id = Column(Integer, primary_key=True) - name = Column(String) - addresses: List["Address"] = relationship("Address") - -As is the case with columns, the :class:`_orm.Mapped` class may also be -applied explicitly:: - - class User(Base): - __tablename__ = "user" - - id = Column(Integer, primary_key=True) - name = Column(String) - - addresses: Mapped[List["Address"]] = relationship("Address", back_populates="user") - - - class Address(Base): - __tablename__ = "address" - - id = Column(Integer, primary_key=True) - user_id: int = Column(ForeignKey("user.id")) - - user: Mapped[User] = relationship(User, back_populates="addresses") - -.. _mypy_declarative_mixins: - -Using @declared_attr and Declarative Mixins -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The :class:`_orm.declared_attr` class allows Declarative mapped attributes to -be declared in class level functions, and is particularly useful when using -:ref:`declarative mixins `. For these functions, the return -type of the function should be annotated using either the ``Mapped[]`` -construct or by indicating the exact kind of object returned by the function. -Additionally, "mixin" classes that are not otherwise mapped (i.e. don't extend -from a :func:`_orm.declarative_base` class nor are they mapped with a method -such as :meth:`_orm.registry.mapped`) should be decorated with the -:func:`_orm.declarative_mixin` decorator, which provides a hint to the Mypy -plugin that a particular class intends to serve as a declarative mixin:: - - from sqlalchemy.orm import declarative_mixin, declared_attr - - - @declarative_mixin - class HasUpdatedAt: - @declared_attr - def updated_at(cls) -> Column[DateTime]: # uses Column - return Column(DateTime) - - - @declarative_mixin - class HasCompany: - @declared_attr - def company_id(cls) -> Mapped[int]: # uses Mapped - return Column(ForeignKey("company.id")) - - @declared_attr - def company(cls) -> Mapped["Company"]: - return relationship("Company") - - - class Employee(HasUpdatedAt, HasCompany, Base): - __tablename__ = "employee" - - id = Column(Integer, primary_key=True) - name = Column(String) - -Note the mismatch between the actual return type of a method like -``HasCompany.company`` vs. what is annotated. The Mypy plugin converts -all ``@declared_attr`` functions into simple annotated attributes to avoid -this complexity:: - - # what Mypy sees - class HasCompany: - company_id: Mapped[int] - company: Mapped["Company"] - -Combining with Dataclasses or Other Type-Sensitive Attribute Systems -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The examples of Python dataclasses integration at :ref:`orm_declarative_dataclasses` -presents a problem; Python dataclasses expect an explicit type that it will -use to build the class, and the value given in each assignment statement -is significant. That is, a class as follows has to be stated exactly -as it is in order to be accepted by dataclasses:: - - mapper_registry: registry = registry() - - - @mapper_registry.mapped - @dataclass - class User: - __table__ = Table( - "user", - mapper_registry.metadata, - Column("id", Integer, primary_key=True), - Column("name", String(50)), - Column("fullname", String(50)), - Column("nickname", String(12)), - ) - id: int = field(init=False) - name: Optional[str] = None - fullname: Optional[str] = None - nickname: Optional[str] = None - addresses: List[Address] = field(default_factory=list) - - __mapper_args__ = { # type: ignore - "properties": {"addresses": relationship("Address")} - } - -We can't apply our ``Mapped[]`` types to the attributes ``id``, ``name``, -etc. because they will be rejected by the ``@dataclass`` decorator. Additionally, -Mypy has another plugin for dataclasses explicitly which can also get in the -way of what we're doing. - -The above class will actually pass Mypy's type checking without issue; the -only thing we are missing is the ability for attributes on ``User`` to be -used in SQL expressions, such as:: - - stmt = select(User.name).where(User.id.in_([1, 2, 3])) - -To provide a workaround for this, the Mypy plugin has an additional feature -whereby we can specify an extra attribute ``_mypy_mapped_attrs``, that is -a list that encloses the class-level objects or their string names. -This attribute can be conditional within the ``TYPE_CHECKING`` variable:: - - @mapper_registry.mapped - @dataclass - class User: - __table__ = Table( - "user", - mapper_registry.metadata, - Column("id", Integer, primary_key=True), - Column("name", String(50)), - Column("fullname", String(50)), - Column("nickname", String(12)), - ) - id: int = field(init=False) - name: Optional[str] = None - fullname: Optional[str] - nickname: Optional[str] - addresses: List[Address] = field(default_factory=list) - - if TYPE_CHECKING: - _mypy_mapped_attrs = [id, name, "fullname", "nickname", addresses] - - __mapper_args__ = { # type: ignore - "properties": {"addresses": relationship("Address")} - } - -With the above recipe, the attributes listed in ``_mypy_mapped_attrs`` -will be applied with the :class:`_orm.Mapped` typing information so that the -``User`` class will behave as a SQLAlchemy mapped class when used in a -class-bound context. - -.. _Mypy: https://mypy.readthedocs.io/ diff --git a/doc/build/orm/extensions/orderinglist.rst b/doc/build/orm/extensions/orderinglist.rst deleted file mode 100644 index 19599a5782..0000000000 --- a/doc/build/orm/extensions/orderinglist.rst +++ /dev/null @@ -1,18 +0,0 @@ -Ordering List -============= - -.. automodule:: sqlalchemy.ext.orderinglist - -API Reference -------------- - -.. autofunction:: ordering_list - -.. autofunction:: count_from_0 - -.. autofunction:: count_from_1 - -.. autofunction:: count_from_n_factory - -.. autoclass:: OrderingList - :members: diff --git a/doc/build/orm/index.rst b/doc/build/orm/index.rst index c3fa9929b3..d14dba571f 100644 --- a/doc/build/orm/index.rst +++ b/doc/build/orm/index.rst @@ -17,6 +17,4 @@ tutorial. queryguide/index session extending - extensions/index - examples diff --git a/doc/build/orm/large_collections.rst b/doc/build/orm/large_collections.rst deleted file mode 100644 index ef1e651e1f..0000000000 --- a/doc/build/orm/large_collections.rst +++ /dev/null @@ -1,686 +0,0 @@ -.. highlight:: pycon+sql -.. doctest-enable - -.. currentmodule:: sqlalchemy.orm - -.. _largecollections: - -Working with Large Collections -============================== - -The default behavior of :func:`_orm.relationship` is to fully load -the contents of collections into memory, based on a configured -:ref:`loader strategy ` that controls -when and how these contents are loaded from the database. Related collections -may be loaded into memory not just when they are accessed, or eagerly loaded, -but in most cases will require population when the collection -itself is mutated, as well as in cases where the owning object is to be -deleted by the unit of work system. - -When a related collection is potentially very large, it may not be feasible -for such a collection to be populated into memory under any circumstances, -as the operation may be overly consuming of time, network and memory -resources. - -This section includes API features intended to allow :func:`_orm.relationship` -to be used with large collections while maintaining adequate performance. - - -.. _write_only_relationship: - -Write Only Relationships ------------------------- - -The **write only** loader strategy is the primary means of configuring a -:func:`_orm.relationship` that will remain writeable, but will not load -its contents into memory. A write-only ORM configuration in modern -type-annotated Declarative form is illustrated below:: - - >>> from decimal import Decimal - >>> from datetime import datetime - - >>> from sqlalchemy import ForeignKey - >>> from sqlalchemy import func - >>> from sqlalchemy.orm import DeclarativeBase - >>> from sqlalchemy.orm import Mapped - >>> from sqlalchemy.orm import mapped_column - >>> from sqlalchemy.orm import relationship - >>> from sqlalchemy.orm import Session - >>> from sqlalchemy.orm import WriteOnlyMapped - - >>> class Base(DeclarativeBase): - ... pass - - >>> class Account(Base): - ... __tablename__ = "account" - ... id: Mapped[int] = mapped_column(primary_key=True) - ... identifier: Mapped[str] - ... - ... account_transactions: WriteOnlyMapped["AccountTransaction"] = relationship( - ... cascade="all, delete-orphan", - ... passive_deletes=True, - ... order_by="AccountTransaction.timestamp", - ... ) - ... - ... def __repr__(self): - ... return f"Account(identifier={self.identifier!r})" - - >>> class AccountTransaction(Base): - ... __tablename__ = "account_transaction" - ... id: Mapped[int] = mapped_column(primary_key=True) - ... account_id: Mapped[int] = mapped_column( - ... ForeignKey("account.id", ondelete="cascade") - ... ) - ... description: Mapped[str] - ... amount: Mapped[Decimal] - ... timestamp: Mapped[datetime] = mapped_column(default=func.now()) - ... - ... def __repr__(self): - ... return ( - ... f"AccountTransaction(amount={self.amount:.2f}, " - ... f"timestamp={self.timestamp.isoformat()!r})" - ... ) - ... - ... __mapper_args__ = {"eager_defaults": True} - - -.. setup code not for display - - >>> from sqlalchemy import create_engine - >>> from sqlalchemy import event - >>> engine = create_engine("sqlite://", echo=True) - >>> @event.listens_for(engine, "connect") - ... def set_sqlite_pragma(dbapi_connection, connection_record): - ... cursor = dbapi_connection.cursor() - ... cursor.execute("PRAGMA foreign_keys=ON") - ... cursor.close() - - >>> Base.metadata.create_all(engine) - BEGIN... - - -Above, the ``account_transactions`` relationship is configured not using the -ordinary :class:`.Mapped` annotation, but instead -using the :class:`.WriteOnlyMapped` type annotation, which at runtime will -assign the :ref:`loader strategy ` of -``lazy="write_only"`` to the target :func:`_orm.relationship`. -The :class:`.WriteOnlyMapped` annotation is an -alternative form of the :class:`_orm.Mapped` annotation which indicate the use -of the :class:`_orm.WriteOnlyCollection` collection type on instances of the -object. - -The above :func:`_orm.relationship` configuration also includes several -elements that are specific to what action to take when ``Account`` objects -are deleted, as well as when ``AccountTransaction`` objects are removed from the -``account_transactions`` collection. These elements are: - -* ``passive_deletes=True`` - allows the :term:`unit of work` to forego having - to load the collection when ``Account`` is deleted; see - :ref:`passive_deletes`. -* ``ondelete="cascade"`` configured on the :class:`.ForeignKey` constraint. - This is also detailed at :ref:`passive_deletes`. -* ``cascade="all, delete-orphan"`` - instructs the :term:`unit of work` to - delete ``AccountTransaction`` objects when they are removed from the - collection. See :ref:`cascade_delete_orphan` in the :ref:`unitofwork_cascades` - document. - -.. versionadded:: 2.0 Added "Write only" relationship loaders. - - -Creating and Persisting New Write Only Collections -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The write-only collection allows for direct assignment of the collection -as a whole **only** for :term:`transient` or :term:`pending` objects. -With our above mapping, this indicates we can create a new ``Account`` -object with a sequence of ``AccountTransaction`` objects to be added -to a :class:`_orm.Session`. Any Python iterable may be used as the -source of objects to start, where below we use a Python ``list``:: - - >>> new_account = Account( - ... identifier="account_01", - ... account_transactions=[ - ... AccountTransaction(description="initial deposit", amount=Decimal("500.00")), - ... AccountTransaction(description="transfer", amount=Decimal("1000.00")), - ... AccountTransaction(description="withdrawal", amount=Decimal("-29.50")), - ... ], - ... ) - - >>> with Session(engine) as session: - ... session.add(new_account) - ... session.commit() - {execsql}BEGIN (implicit) - INSERT INTO account (identifier) VALUES (?) - [...] ('account_01',) - INSERT INTO account_transaction (account_id, description, amount, timestamp) - VALUES (?, ?, ?, CURRENT_TIMESTAMP) RETURNING id, timestamp - [... (insertmanyvalues) 1/3 (ordered; batch not supported)] (1, 'initial deposit', 500.0) - INSERT INTO account_transaction (account_id, description, amount, timestamp) - VALUES (?, ?, ?, CURRENT_TIMESTAMP) RETURNING id, timestamp - [insertmanyvalues 2/3 (ordered; batch not supported)] (1, 'transfer', 1000.0) - INSERT INTO account_transaction (account_id, description, amount, timestamp) - VALUES (?, ?, ?, CURRENT_TIMESTAMP) RETURNING id, timestamp - [insertmanyvalues 3/3 (ordered; batch not supported)] (1, 'withdrawal', -29.5) - COMMIT - - -Once an object is database-persisted (i.e. in the :term:`persistent` or -:term:`detached` state), the collection has the ability to be extended with new -items as well as the ability for individual items to be removed. However, the -collection may **no longer be re-assigned with a full replacement collection**, -as such an operation requires that the previous collection is fully -loaded into memory in order to reconcile the old entries with the new ones:: - - >>> new_account.account_transactions = [ - ... AccountTransaction(description="some transaction", amount=Decimal("10.00")) - ... ] - Traceback (most recent call last): - ... - sqlalchemy.exc.InvalidRequestError: Collection "Account.account_transactions" does not - support implicit iteration; collection replacement operations can't be used - -Adding New Items to an Existing Collection -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -For write-only collections of persistent objects, -modifications to the collection using :term:`unit of work` processes may proceed -only by using the :meth:`.WriteOnlyCollection.add`, -:meth:`.WriteOnlyCollection.add_all` and :meth:`.WriteOnlyCollection.remove` -methods:: - - >>> from sqlalchemy import select - >>> session = Session(engine, expire_on_commit=False) - >>> existing_account = session.scalar(select(Account).filter_by(identifier="account_01")) - {execsql}BEGIN (implicit) - SELECT account.id, account.identifier - FROM account - WHERE account.identifier = ? - [...] ('account_01',) - {stop} - >>> existing_account.account_transactions.add_all( - ... [ - ... AccountTransaction(description="paycheck", amount=Decimal("2000.00")), - ... AccountTransaction(description="rent", amount=Decimal("-800.00")), - ... ] - ... ) - >>> session.commit() - {execsql}INSERT INTO account_transaction (account_id, description, amount, timestamp) - VALUES (?, ?, ?, CURRENT_TIMESTAMP) RETURNING id, timestamp - [... (insertmanyvalues) 1/2 (ordered; batch not supported)] (1, 'paycheck', 2000.0) - INSERT INTO account_transaction (account_id, description, amount, timestamp) - VALUES (?, ?, ?, CURRENT_TIMESTAMP) RETURNING id, timestamp - [insertmanyvalues 2/2 (ordered; batch not supported)] (1, 'rent', -800.0) - COMMIT - - -The items added above are held in a pending queue within the -:class:`_orm.Session` until the next flush, at which point they are INSERTed -into the database, assuming the added objects were previously :term:`transient`. - -Querying Items -~~~~~~~~~~~~~~ - -The :class:`_orm.WriteOnlyCollection` does not at any point store a reference -to the current contents of the collection, nor does it have any behavior where -it would directly emit a SELECT to the database in order to load them; the -overriding assumption is that the collection may contain many thousands or -millions of rows, and should never be fully loaded into memory as a side effect -of any other operation. - -Instead, the :class:`_orm.WriteOnlyCollection` includes SQL-generating helpers -such as :meth:`_orm.WriteOnlyCollection.select`, which will generate -a :class:`.Select` construct pre-configured with the correct WHERE / FROM -criteria for the current parent row, which can then be further modified in -order to SELECT any range of rows desired, as well as invoked using features -like :ref:`server side cursors ` for processes that -wish to iterate through the full collection in a memory-efficient manner. - -The statement generated is illustrated below. Note it also includes ORDER BY -criteria, indicated in the example mapping by the -:paramref:`_orm.relationship.order_by` parameter of :func:`_orm.relationship`; -this criteria would be omitted if the parameter were not configured:: - - >>> print(existing_account.account_transactions.select()) - {printsql}SELECT account_transaction.id, account_transaction.account_id, account_transaction.description, - account_transaction.amount, account_transaction.timestamp - FROM account_transaction - WHERE :param_1 = account_transaction.account_id ORDER BY account_transaction.timestamp - -We may use this :class:`.Select` construct along with the :class:`_orm.Session` -in order to query for ``AccountTransaction`` objects, most easily using the -:meth:`_orm.Session.scalars` method that will return a :class:`.Result` that -yields ORM objects directly. It's typical, though not required, that the -:class:`.Select` would be modified further to limit the records returned; in -the example below, additional WHERE criteria to load only "debit" account -transactions is added, along with "LIMIT 10" to retrieve only the first ten -rows:: - - >>> account_transactions = session.scalars( - ... existing_account.account_transactions.select() - ... .where(AccountTransaction.amount < 0) - ... .limit(10) - ... ).all() - {execsql}BEGIN (implicit) - SELECT account_transaction.id, account_transaction.account_id, account_transaction.description, - account_transaction.amount, account_transaction.timestamp - FROM account_transaction - WHERE ? = account_transaction.account_id AND account_transaction.amount < ? - ORDER BY account_transaction.timestamp LIMIT ? OFFSET ? - [...] (1, 0, 10, 0) - {stop}>>> print(account_transactions) - [AccountTransaction(amount=-29.50, timestamp='...'), AccountTransaction(amount=-800.00, timestamp='...')] - - -Removing Items -~~~~~~~~~~~~~~ - -Individual items that are loaded in the :term:`persistent` -state against the current :class:`_orm.Session` may be marked for removal -from the collection using the :meth:`.WriteOnlyCollection.remove` method. -The flush process will implicitly consider the object to be already part -of the collection when the operation proceeds. The example below -illustrates removal of an individual ``AccountTransaction`` item, -which per :ref:`cascade ` settings results in a -DELETE of that row:: - - >>> existing_transaction = account_transactions[0] - >>> existing_account.account_transactions.remove(existing_transaction) - >>> session.commit() - {execsql}DELETE FROM account_transaction WHERE account_transaction.id = ? - [...] (3,) - COMMIT - -As with any ORM-mapped collection, object removal may proceed either to -de-associate the object from the collection while leaving the object present in -the database, or may issue a DELETE for its row, based on the -:ref:`cascade_delete_orphan` configuration of the :func:`_orm.relationship`. - -Collection removal without deletion involves setting foreign key columns to -NULL for a :ref:`one-to-many ` relationship, or -deleting the corresponding association row for a -:ref:`many-to-many ` relationship. - - - -Bulk INSERT of New Items -~~~~~~~~~~~~~~~~~~~~~~~~ - -The :class:`.WriteOnlyCollection` can generate DML constructs such as -:class:`_dml.Insert` objects, which may be used in an ORM context to -produce bulk insert behavior. See the section -:ref:`orm_queryguide_bulk_insert` for an overview of ORM bulk inserts. - -One to Many Collections -^^^^^^^^^^^^^^^^^^^^^^^ -For a **regular one to many collection only**, the :meth:`.WriteOnlyCollection.insert` -method will produce an :class:`_dml.Insert` construct which is pre-established with -VALUES criteria corresponding to the parent object. As this VALUES criteria -is entirely against the related table, the statement can be used to -INSERT new rows that will at the same time become new records in the -related collection:: - - >>> session.execute( - ... existing_account.account_transactions.insert(), - ... [ - ... {"description": "transaction 1", "amount": Decimal("47.50")}, - ... {"description": "transaction 2", "amount": Decimal("-501.25")}, - ... {"description": "transaction 3", "amount": Decimal("1800.00")}, - ... {"description": "transaction 4", "amount": Decimal("-300.00")}, - ... ], - ... ) - {execsql}BEGIN (implicit) - INSERT INTO account_transaction (account_id, description, amount, timestamp) VALUES (?, ?, ?, CURRENT_TIMESTAMP) - [...] [(1, 'transaction 1', 47.5), (1, 'transaction 2', -501.25), (1, 'transaction 3', 1800.0), (1, 'transaction 4', -300.0)] - <...> - {stop} - >>> session.commit() - COMMIT - -.. seealso:: - - :ref:`orm_queryguide_bulk_insert` - in the :ref:`queryguide_toplevel` - - :ref:`relationship_patterns_o2m` - at :ref:`relationship_patterns` - - -Many to Many Collections -^^^^^^^^^^^^^^^^^^^^^^^^ - -For a **many to many collection**, the relationship between two classes -involves a third table that is configured using the -:paramref:`_orm.relationship.secondary` parameter of :class:`_orm.relationship`. -To bulk insert rows into a collection of this type using -:class:`.WriteOnlyCollection`, the new records may be bulk-inserted separately -first, retrieved using RETURNING, and those records then passed to the -:meth:`.WriteOnlyCollection.add_all` method where the unit of work process -will proceed to persist them as part of the collection. - -Supposing a class ``BankAudit`` referred to many ``AccountTransaction`` -records using a many-to-many table:: - - >>> from sqlalchemy import Table, Column - >>> audit_to_transaction = Table( - ... "audit_transaction", - ... Base.metadata, - ... Column("audit_id", ForeignKey("audit.id", ondelete="CASCADE"), primary_key=True), - ... Column( - ... "transaction_id", - ... ForeignKey("account_transaction.id", ondelete="CASCADE"), - ... primary_key=True, - ... ), - ... ) - >>> class BankAudit(Base): - ... __tablename__ = "audit" - ... id: Mapped[int] = mapped_column(primary_key=True) - ... account_transactions: WriteOnlyMapped["AccountTransaction"] = relationship( - ... secondary=audit_to_transaction, passive_deletes=True - ... ) - -.. setup code not for display - - >>> Base.metadata.create_all(engine) - BEGIN... - -To illustrate the two operations, we add more ``AccountTransaction`` objects -using bulk insert, which we retrieve using RETURNING by adding -``returning(AccountTransaction)`` to the bulk INSERT statement (note that -we could just as easily use existing ``AccountTransaction`` objects as well):: - - >>> new_transactions = session.scalars( - ... existing_account.account_transactions.insert().returning(AccountTransaction), - ... [ - ... {"description": "odd trans 1", "amount": Decimal("50000.00")}, - ... {"description": "odd trans 2", "amount": Decimal("25000.00")}, - ... {"description": "odd trans 3", "amount": Decimal("45.00")}, - ... ], - ... ).all() - {execsql}BEGIN (implicit) - INSERT INTO account_transaction (account_id, description, amount, timestamp) VALUES - (?, ?, ?, CURRENT_TIMESTAMP), (?, ?, ?, CURRENT_TIMESTAMP), (?, ?, ?, CURRENT_TIMESTAMP) - RETURNING id, account_id, description, amount, timestamp - [...] (1, 'odd trans 1', 50000.0, 1, 'odd trans 2', 25000.0, 1, 'odd trans 3', 45.0) - {stop} - -With a list of ``AccountTransaction`` objects ready, the -:meth:`_orm.WriteOnlyCollection.add_all` method is used to associate many rows -at once with a new ``BankAudit`` object:: - - >>> bank_audit = BankAudit() - >>> session.add(bank_audit) - >>> bank_audit.account_transactions.add_all(new_transactions) - >>> session.commit() - {execsql}INSERT INTO audit DEFAULT VALUES - [...] () - INSERT INTO audit_transaction (audit_id, transaction_id) VALUES (?, ?) - [...] [(1, 10), (1, 11), (1, 12)] - COMMIT - -.. seealso:: - - :ref:`orm_queryguide_bulk_insert` - in the :ref:`queryguide_toplevel` - - :ref:`relationships_many_to_many` - at :ref:`relationship_patterns` - - -Bulk UPDATE and DELETE of Items -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -In a similar way in which :class:`.WriteOnlyCollection` can generate -:class:`.Select` constructs with WHERE criteria pre-established, it can -also generate :class:`.Update` and :class:`.Delete` constructs with that -same WHERE criteria, to allow criteria-oriented UPDATE and DELETE statements -against the elements in a large collection. - -One To Many Collections -^^^^^^^^^^^^^^^^^^^^^^^ - -As is the case with INSERT, this feature is most straightforward with **one -to many collections**. - -In the example below, the :meth:`.WriteOnlyCollection.update` method is used -to generate an UPDATE statement is emitted against the elements -in the collection, locating rows where the "amount" is equal to ``-800`` and -adding the amount of ``200`` to them:: - - >>> session.execute( - ... existing_account.account_transactions.update() - ... .values(amount=AccountTransaction.amount + 200) - ... .where(AccountTransaction.amount == -800), - ... ) - {execsql}BEGIN (implicit) - UPDATE account_transaction SET amount=(account_transaction.amount + ?) - WHERE ? = account_transaction.account_id AND account_transaction.amount = ? - [...] (200, 1, -800) - {stop}<...> - -In a similar way, :meth:`.WriteOnlyCollection.delete` will produce a -DELETE statement that is invoked in the same way:: - - >>> session.execute( - ... existing_account.account_transactions.delete().where( - ... AccountTransaction.amount.between(0, 30) - ... ), - ... ) - {execsql}DELETE FROM account_transaction WHERE ? = account_transaction.account_id - AND account_transaction.amount BETWEEN ? AND ? RETURNING id - [...] (1, 0, 30) - <...> - {stop} - -Many to Many Collections -^^^^^^^^^^^^^^^^^^^^^^^^ - -.. tip:: - - The techniques here involve multi-table UPDATE expressions, which are - slightly more advanced. - -For bulk UPDATE and DELETE of **many to many collections**, in order for -an UPDATE or DELETE statement to relate to the primary key of the -parent object, the association table must be explicitly part of the -UPDATE/DELETE statement, which requires -either that the backend includes supports for non-standard SQL syntaxes, -or extra explicit steps when constructing the UPDATE or DELETE statement. - -For backends that support multi-table versions of UPDATE, the -:meth:`.WriteOnlyCollection.update` method should work without extra steps -for a many-to-many collection, as in the example below where an UPDATE -is emitted against ``AccountTransaction`` objects in terms of the -many-to-many ``BankAudit.account_transactions`` collection:: - - >>> session.execute( - ... bank_audit.account_transactions.update().values( - ... description=AccountTransaction.description + " (audited)" - ... ) - ... ) - {execsql}UPDATE account_transaction SET description=(account_transaction.description || ?) - FROM audit_transaction WHERE ? = audit_transaction.audit_id - AND account_transaction.id = audit_transaction.transaction_id RETURNING id - [...] (' (audited)', 1) - {stop}<...> - -The above statement automatically makes use of "UPDATE..FROM" syntax, -supported by SQLite and others, to name the additional ``audit_transaction`` -table in the WHERE clause. - -To UPDATE or DELETE a many-to-many collection where multi-table syntax is -not available, the many-to-many criteria may be moved into SELECT that -for example may be combined with IN to match rows. -The :class:`.WriteOnlyCollection` still helps us here, as we use the -:meth:`.WriteOnlyCollection.select` method to generate this SELECT for -us, making use of the :meth:`_sql.Select.with_only_columns` method to -produce a :term:`scalar subquery`:: - - >>> from sqlalchemy import update - >>> subq = bank_audit.account_transactions.select().with_only_columns(AccountTransaction.id) - >>> session.execute( - ... update(AccountTransaction) - ... .values(description=AccountTransaction.description + " (audited)") - ... .where(AccountTransaction.id.in_(subq)) - ... ) - {execsql}UPDATE account_transaction SET description=(account_transaction.description || ?) - WHERE account_transaction.id IN (SELECT account_transaction.id - FROM audit_transaction - WHERE ? = audit_transaction.audit_id AND account_transaction.id = audit_transaction.transaction_id) - RETURNING id - [...] (' (audited)', 1) - <...> - -Write Only Collections - API Documentation -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - - -.. autoclass:: sqlalchemy.orm.WriteOnlyCollection - :members: - :inherited-members: - -.. autoclass:: sqlalchemy.orm.WriteOnlyMapped - :members: - -.. highlight:: python -.. doctest-disable - -.. _dynamic_relationship: - -Dynamic Relationship Loaders ----------------------------- - -.. legacy:: The "dynamic" lazy loader strategy is the legacy form of what is - now the "write_only" strategy described in the section - :ref:`write_only_relationship`. - - The "dynamic" strategy produces a legacy :class:`_orm.Query` object from the - related collection. However, a major drawback of "dynamic" relationships is - that there are several cases where the collection will fully iterate, some - of which are non-obvious, which can only be prevented with careful - programming and testing on a case-by-case basis. Therefore, for truly large - collection management, the :class:`_orm.WriteOnlyCollection` should be - preferred. - - The dynamic loader is also not compatible with the :ref:`asyncio_toplevel` - extension. It can be used with some limitations, as indicated in - :ref:`Asyncio dynamic guidelines `, but again the - :class:`_orm.WriteOnlyCollection`, which is fully compatible with asyncio, - should be preferred. - -The dynamic relationship strategy allows configuration of a -:func:`_orm.relationship` which when accessed on an instance will return a -legacy :class:`_orm.Query` object in place of the collection. The -:class:`_orm.Query` can then be modified further so that the database -collection may be iterated based on filtering criteria. The returned -:class:`_orm.Query` object is an instance of :class:`_orm.AppenderQuery`, which -combines the loading and iteration behavior of :class:`_orm.Query` along with -rudimentary collection mutation methods such as -:meth:`_orm.AppenderQuery.append` and :meth:`_orm.AppenderQuery.remove`. - -The "dynamic" loader strategy may be configured with -type-annotated Declarative form using the :class:`_orm.DynamicMapped` -annotation class:: - - from sqlalchemy.orm import DynamicMapped - - - class User(Base): - __tablename__ = "user" - - id: Mapped[int] = mapped_column(primary_key=True) - posts: DynamicMapped[Post] = relationship() - -Above, the ``User.posts`` collection on an individual ``User`` object -will return the :class:`_orm.AppenderQuery` object, which is a subclass -of :class:`_orm.Query` that also supports basic collection mutation -operations:: - - - jack = session.get(User, id) - - # filter Jack's blog posts - posts = jack.posts.filter(Post.headline == "this is a post") - - # apply array slices - posts = jack.posts[5:20] - -The dynamic relationship supports limited write operations, via the -:meth:`_orm.AppenderQuery.append` and :meth:`_orm.AppenderQuery.remove` methods:: - - oldpost = jack.posts.filter(Post.headline == "old post").one() - jack.posts.remove(oldpost) - - jack.posts.append(Post("new post")) - -Since the read side of the dynamic relationship always queries the -database, changes to the underlying collection will not be visible -until the data has been flushed. However, as long as "autoflush" is -enabled on the :class:`.Session` in use, this will occur -automatically each time the collection is about to emit a -query. - - -Dynamic Relationship Loaders - API -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. autoclass:: sqlalchemy.orm.AppenderQuery - :members: - :inherited-members: Query - -.. autoclass:: sqlalchemy.orm.DynamicMapped - :members: - -.. _collections_raiseload: - -Setting RaiseLoad ------------------ - -A "raise"-loaded relationship will raise an -:exc:`~sqlalchemy.exc.InvalidRequestError` where the attribute would normally -emit a lazy load:: - - class MyClass(Base): - __tablename__ = "some_table" - - # ... - - children: Mapped[List[MyRelatedClass]] = relationship(lazy="raise") - -Above, attribute access on the ``children`` collection will raise an exception -if it was not previously populated. This includes read access but for -collections will also affect write access, as collections can't be mutated -without first loading them. The rationale for this is to ensure that an -application is not emitting any unexpected lazy loads within a certain context. -Rather than having to read through SQL logs to determine that all necessary -attributes were eager loaded, the "raise" strategy will cause unloaded -attributes to raise immediately if accessed. The raise strategy is -also available on a query option basis using the :func:`_orm.raiseload` -loader option. - -.. seealso:: - - :ref:`prevent_lazy_with_raiseload` - -Using Passive Deletes ---------------------- - -An important aspect of collection management in SQLAlchemy is that when an -object that refers to a collection is deleted, SQLAlchemy needs to consider the -objects that are inside this collection. Those objects will need to be -de-associated from the parent, which for a one-to-many collection would mean -that foreign key columns are set to NULL, or based on -:ref:`cascade ` settings, may instead want to emit a -DELETE for these rows. - -The :term:`unit of work` process only considers objects on a row-by-row basis, -meaning a DELETE operation implies that all rows within a collection must be -fully loaded into memory inside the flush process. This is not feasible for -large collections, so we instead seek to rely upon the database's own -capability to update or delete the rows automatically using foreign key ON -DELETE rules, instructing the unit of work to forego actually needing to load -these rows in order to handle them. The unit of work can be instructed to work -in this manner by configuring :paramref:`_orm.relationship.passive_deletes` on -the :func:`_orm.relationship` construct; the foreign key constraints in use -must also be correctly configured. - -For further detail on a complete "passive delete" configuration, see the -section :ref:`passive_deletes`. - - - diff --git a/doc/build/orm/loading_columns.rst b/doc/build/orm/loading_columns.rst deleted file mode 100644 index c1fad1710d..0000000000 --- a/doc/build/orm/loading_columns.rst +++ /dev/null @@ -1,4 +0,0 @@ -:orphan: - -This document has moved to :doc:`queryguide/columns` - diff --git a/doc/build/orm/mapped_sql_expr.rst b/doc/build/orm/mapped_sql_expr.rst deleted file mode 100644 index 357949c8fe..0000000000 --- a/doc/build/orm/mapped_sql_expr.rst +++ /dev/null @@ -1,347 +0,0 @@ -.. currentmodule:: sqlalchemy.orm - -.. _mapper_sql_expressions: - -SQL Expressions as Mapped Attributes -==================================== - -Attributes on a mapped class can be linked to SQL expressions, which can -be used in queries. - -Using a Hybrid --------------- - -The easiest and most flexible way to link relatively simple SQL expressions to a class is to use a so-called -"hybrid attribute", -described in the section :ref:`hybrids_toplevel`. The hybrid provides -for an expression that works at both the Python level as well as at the -SQL expression level. For example, below we map a class ``User``, -containing attributes ``firstname`` and ``lastname``, and include a hybrid that -will provide for us the ``fullname``, which is the string concatenation of the two:: - - from sqlalchemy.ext.hybrid import hybrid_property - - - class User(Base): - __tablename__ = "user" - id = mapped_column(Integer, primary_key=True) - firstname = mapped_column(String(50)) - lastname = mapped_column(String(50)) - - @hybrid_property - def fullname(self): - return self.firstname + " " + self.lastname - -Above, the ``fullname`` attribute is interpreted at both the instance and -class level, so that it is available from an instance:: - - some_user = session.scalars(select(User).limit(1)).first() - print(some_user.fullname) - -as well as usable within queries:: - - some_user = session.scalars( - select(User).where(User.fullname == "John Smith").limit(1) - ).first() - -The string concatenation example is a simple one, where the Python expression -can be dual purposed at the instance and class level. Often, the SQL expression -must be distinguished from the Python expression, which can be achieved using -:meth:`.hybrid_property.expression`. Below we illustrate the case where a conditional -needs to be present inside the hybrid, using the ``if`` statement in Python and the -:func:`_expression.case` construct for SQL expressions:: - - from sqlalchemy.ext.hybrid import hybrid_property - from sqlalchemy.sql import case - - - class User(Base): - __tablename__ = "user" - id = mapped_column(Integer, primary_key=True) - firstname = mapped_column(String(50)) - lastname = mapped_column(String(50)) - - @hybrid_property - def fullname(self): - if self.firstname is not None: - return self.firstname + " " + self.lastname - else: - return self.lastname - - @fullname.expression - def fullname(cls): - return case( - (cls.firstname != None, cls.firstname + " " + cls.lastname), - else_=cls.lastname, - ) - -.. _mapper_column_property_sql_expressions: - -Using column_property ---------------------- - -The :func:`_orm.column_property` function can be used to map a SQL -expression in a manner similar to a regularly mapped :class:`_schema.Column`. -With this technique, the attribute is loaded -along with all other column-mapped attributes at load time. This is in some -cases an advantage over the usage of hybrids, as the value can be loaded -up front at the same time as the parent row of the object, particularly if -the expression is one which links to other tables (typically as a correlated -subquery) to access data that wouldn't normally be -available on an already loaded object. - -Disadvantages to using :func:`_orm.column_property` for SQL expressions include that -the expression must be compatible with the SELECT statement emitted for the class -as a whole, and there are also some configurational quirks which can occur -when using :func:`_orm.column_property` from declarative mixins. - -Our "fullname" example can be expressed using :func:`_orm.column_property` as -follows:: - - from sqlalchemy.orm import column_property - - - class User(Base): - __tablename__ = "user" - id = mapped_column(Integer, primary_key=True) - firstname = mapped_column(String(50)) - lastname = mapped_column(String(50)) - fullname = column_property(firstname + " " + lastname) - -Correlated subqueries may be used as well. Below we use the -:func:`_expression.select` construct to create a :class:`_sql.ScalarSelect`, -representing a column-oriented SELECT statement, that links together the count -of ``Address`` objects available for a particular ``User``:: - - from sqlalchemy.orm import column_property - from sqlalchemy import select, func - from sqlalchemy import Column, Integer, String, ForeignKey - - from sqlalchemy.orm import DeclarativeBase - - - class Base(DeclarativeBase): - pass - - - class Address(Base): - __tablename__ = "address" - id = mapped_column(Integer, primary_key=True) - user_id = mapped_column(Integer, ForeignKey("user.id")) - - - class User(Base): - __tablename__ = "user" - id = mapped_column(Integer, primary_key=True) - address_count = column_property( - select(func.count(Address.id)) - .where(Address.user_id == id) - .correlate_except(Address) - .scalar_subquery() - ) - -In the above example, we define a :func:`_expression.ScalarSelect` construct like the following:: - - stmt = ( - select(func.count(Address.id)) - .where(Address.user_id == id) - .correlate_except(Address) - .scalar_subquery() - ) - -Above, we first use :func:`_sql.select` to create a :class:`_sql.Select` -construct, which we then convert into a :term:`scalar subquery` using the -:meth:`_sql.Select.scalar_subquery` method, indicating our intent to use this -:class:`_sql.Select` statement in a column expression context. - -Within the :class:`_sql.Select` itself, we select the count of ``Address.id`` rows -where the ``Address.user_id`` column is equated to ``id``, which in the context -of the ``User`` class is the :class:`_schema.Column` named ``id`` (note that ``id`` is -also the name of a Python built in function, which is not what we want to use -here - if we were outside of the ``User`` class definition, we'd use ``User.id``). - -The :meth:`_sql.Select.correlate_except` method indicates that each element in the -FROM clause of this :func:`_expression.select` may be omitted from the FROM list (that is, correlated -to the enclosing SELECT statement against ``User``) except for the one corresponding -to ``Address``. This isn't strictly necessary, but prevents ``Address`` from -being inadvertently omitted from the FROM list in the case of a long string -of joins between ``User`` and ``Address`` tables where SELECT statements against -``Address`` are nested. - -For a :func:`.column_property` that refers to columns linked from a -many-to-many relationship, use :func:`.and_` to join the fields of the -association table to both tables in a relationship:: - - from sqlalchemy import and_ - - - class Author(Base): - # ... - - book_count = column_property( - select(func.count(books.c.id)) - .where( - and_( - book_authors.c.author_id == authors.c.id, - book_authors.c.book_id == books.c.id, - ) - ) - .scalar_subquery() - ) - -Adding column_property() to an existing Declarative mapped class -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -If import issues prevent the :func:`.column_property` from being defined -inline with the class, it can be assigned to the class after both -are configured. When using mappings that make use of a Declarative -base class (i.e. produced by the :class:`_orm.DeclarativeBase` superclass -or legacy functions such as :func:`_orm.declarative_base`), -this attribute assignment has the effect of calling :meth:`_orm.Mapper.add_property` -to add an additional property after the fact:: - - # only works if a declarative base class is in use - User.address_count = column_property( - select(func.count(Address.id)).where(Address.user_id == User.id).scalar_subquery() - ) - -When using mapping styles that don't use Declarative base classes -such as the :meth:`_orm.registry.mapped` decorator, the :meth:`_orm.Mapper.add_property` -method may be invoked explicitly on the underlying :class:`_orm.Mapper` object, -which can be obtained using :func:`_sa.inspect`:: - - from sqlalchemy.orm import registry - - reg = registry() - - - @reg.mapped - class User: - __tablename__ = "user" - - # ... additional mapping directives - - - # later ... - - # works for any kind of mapping - from sqlalchemy import inspect - - inspect(User).add_property( - column_property( - select(func.count(Address.id)) - .where(Address.user_id == User.id) - .scalar_subquery() - ) - ) - -.. seealso:: - - :ref:`orm_declarative_table_adding_columns` - - -.. _mapper_column_property_sql_expressions_composed: - -Composing from Column Properties at Mapping Time -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -It is possible to create mappings that combine multiple -:class:`.ColumnProperty` objects together. The :class:`.ColumnProperty` will -be interpreted as a SQL expression when used in a Core expression context, -provided that it is targeted by an existing expression object; this works by -the Core detecting that the object has a ``__clause_element__()`` method which -returns a SQL expression. However, if the :class:`.ColumnProperty` is used as -a lead object in an expression where there is no other Core SQL expression -object to target it, the :attr:`.ColumnProperty.expression` attribute will -return the underlying SQL expression so that it can be used to build SQL -expressions consistently. Below, the ``File`` class contains an attribute -``File.path`` that concatenates a string token to the ``File.filename`` -attribute, which is itself a :class:`.ColumnProperty`:: - - - class File(Base): - __tablename__ = "file" - - id = mapped_column(Integer, primary_key=True) - name = mapped_column(String(64)) - extension = mapped_column(String(8)) - filename = column_property(name + "." + extension) - path = column_property("C:/" + filename.expression) - -When the ``File`` class is used in expressions normally, the attributes -assigned to ``filename`` and ``path`` are usable directly. The use of the -:attr:`.ColumnProperty.expression` attribute is only necessary when using -the :class:`.ColumnProperty` directly within the mapping definition:: - - stmt = select(File.path).where(File.filename == "foo.txt") - -Using Column Deferral with ``column_property()`` -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The column deferral feature introduced in the :ref:`queryguide_toplevel` -at :ref:`orm_queryguide_column_deferral` may be applied at mapping time -to a SQL expression mapped by :func:`_orm.column_property` by using the -:func:`_orm.deferred` function in place of :func:`_orm.column_property`:: - - from sqlalchemy.orm import deferred - - - class User(Base): - __tablename__ = "user" - - id: Mapped[int] = mapped_column(primary_key=True) - firstname: Mapped[str] = mapped_column() - lastname: Mapped[str] = mapped_column() - fullname: Mapped[str] = deferred(firstname + " " + lastname) - -.. seealso:: - - :ref:`orm_queryguide_deferred_imperative` - - - -Using a plain descriptor ------------------------- - -In cases where a SQL query more elaborate than what :func:`_orm.column_property` -or :class:`.hybrid_property` can provide must be emitted, a regular Python -function accessed as an attribute can be used, assuming the expression -only needs to be available on an already-loaded instance. The function -is decorated with Python's own ``@property`` decorator to mark it as a read-only -attribute. Within the function, :func:`.object_session` -is used to locate the :class:`.Session` corresponding to the current object, -which is then used to emit a query:: - - from sqlalchemy.orm import object_session - from sqlalchemy import select, func - - - class User(Base): - __tablename__ = "user" - id = mapped_column(Integer, primary_key=True) - firstname = mapped_column(String(50)) - lastname = mapped_column(String(50)) - - @property - def address_count(self): - return object_session(self).scalar( - select(func.count(Address.id)).where(Address.user_id == self.id) - ) - -The plain descriptor approach is useful as a last resort, but is less performant -in the usual case than both the hybrid and column property approaches, in that -it needs to emit a SQL query upon each access. - -.. _mapper_querytime_expression: - -Query-time SQL expressions as mapped attributes ------------------------------------------------ - -In addition to being able to configure fixed SQL expressions on mapped classes, -the SQLAlchemy ORM also includes a feature wherein objects may be loaded -with the results of arbitrary SQL expressions which are set up at query time as part -of their state. This behavior is available by configuring an ORM mapped -attribute using :func:`_orm.query_expression` and then using the -:func:`_orm.with_expression` loader option at query time. See the section -:ref:`orm_queryguide_with_expression` for an example mapping and usage. - diff --git a/doc/build/orm/mapper_config.rst b/doc/build/orm/mapper_config.rst index 68218491d5..b9ae434b47 100644 --- a/doc/build/orm/mapper_config.rst +++ b/doc/build/orm/mapper_config.rst @@ -22,16 +22,8 @@ in SQLAlchemy, it's first introduced in the :ref:`unified_tutorial` at mapping_styles declarative_mapping - dataclasses - mapped_sql_expr mapped_attributes - composites inheritance nonstandard_mappings - versioning mapping_api -.. toctree:: - :hidden: - - scalar_mapping diff --git a/doc/build/orm/mapping_columns.rst b/doc/build/orm/mapping_columns.rst deleted file mode 100644 index 25c6604faf..0000000000 --- a/doc/build/orm/mapping_columns.rst +++ /dev/null @@ -1,9 +0,0 @@ -.. _mapping_columns_toplevel: - -Mapping Table Columns -===================== - -This section has been integrated into the -:ref:`orm_declarative_table_config_toplevel` Declarative section. - - diff --git a/doc/build/orm/relationships.rst b/doc/build/orm/relationships.rst index ab0402721d..bd75123a90 100644 --- a/doc/build/orm/relationships.rst +++ b/doc/build/orm/relationships.rst @@ -15,9 +15,7 @@ of its usage. For an introduction to relationships, start with basic_relationships self_referential join_conditions - large_collections collection_api relationship_persistence - backref relationship_api diff --git a/doc/build/orm/scalar_mapping.rst b/doc/build/orm/scalar_mapping.rst deleted file mode 100644 index f6863edada..0000000000 --- a/doc/build/orm/scalar_mapping.rst +++ /dev/null @@ -1,13 +0,0 @@ -=============================== -Mapping SQL Expressions -=============================== - -This page has been merged into the -:ref:`mapper_config_toplevel` index. - - -.. toctree:: - :hidden: - - mapping_columns - diff --git a/doc/build/orm/session.rst b/doc/build/orm/session.rst index 00d2e6f6a1..9f3fbd2062 100644 --- a/doc/build/orm/session.rst +++ b/doc/build/orm/session.rst @@ -16,11 +16,9 @@ persistence operations is the :maxdepth: 3 session_basics - session_state_management cascades session_transaction persistence_techniques - contextual session_events session_api diff --git a/doc/build/orm/session_state_management.rst b/doc/build/orm/session_state_management.rst deleted file mode 100644 index 3538bdc224..0000000000 --- a/doc/build/orm/session_state_management.rst +++ /dev/null @@ -1,657 +0,0 @@ -State Management -================ - -.. _session_object_states: - -Quickie Intro to Object States ------------------------------- - -It's helpful to know the states which an instance can have within a session: - -* **Transient** - an instance that's not in a session, and is not saved to the - database; i.e. it has no database identity. The only relationship such an - object has to the ORM is that its class has a :class:`_orm.Mapper` associated - with it. - -* **Pending** - when you :meth:`~.Session.add` a transient - instance, it becomes pending. It still wasn't actually flushed to the - database yet, but it will be when the next flush occurs. - -* **Persistent** - An instance which is present in the session and has a record - in the database. You get persistent instances by either flushing so that the - pending instances become persistent, or by querying the database for - existing instances (or moving persistent instances from other sessions into - your local session). - -* **Deleted** - An instance which has been deleted within a flush, but - the transaction has not yet completed. Objects in this state are essentially - in the opposite of "pending" state; when the session's transaction is committed, - the object will move to the detached state. Alternatively, when - the session's transaction is rolled back, a deleted object moves - *back* to the persistent state. - -* **Detached** - an instance which corresponds, or previously corresponded, - to a record in the database, but is not currently in any session. - The detached object will contain a database identity marker, however - because it is not associated with a session, it is unknown whether or not - this database identity actually exists in a target database. Detached - objects are safe to use normally, except that they have no ability to - load unloaded attributes or attributes that were previously marked - as "expired". - -For a deeper dive into all possible state transitions, see the -section :ref:`session_lifecycle_events` which describes each transition -as well as how to programmatically track each one. - -Getting the Current State of an Object -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The actual state of any mapped object can be viewed at any time using -the :func:`_sa.inspect` function on a mapped instance; this function will -return the corresponding :class:`.InstanceState` object which manages the -internal ORM state for the object. :class:`.InstanceState` provides, among -other accessors, boolean attributes indicating the persistence state -of the object, including: - -* :attr:`.InstanceState.transient` -* :attr:`.InstanceState.pending` -* :attr:`.InstanceState.persistent` -* :attr:`.InstanceState.deleted` -* :attr:`.InstanceState.detached` - -E.g.:: - - >>> from sqlalchemy import inspect - >>> insp = inspect(my_object) - >>> insp.persistent - True - -.. seealso:: - - :ref:`orm_mapper_inspection_instancestate` - further examples of - :class:`.InstanceState` - -.. _session_attributes: - -Session Attributes ------------------- - -The :class:`~sqlalchemy.orm.session.Session` itself acts somewhat like a -set-like collection. All items present may be accessed using the iterator -interface:: - - for obj in session: - print(obj) - -And presence may be tested for using regular "contains" semantics:: - - if obj in session: - print("Object is present") - -The session is also keeping track of all newly created (i.e. pending) objects, -all objects which have had changes since they were last loaded or saved (i.e. -"dirty"), and everything that's been marked as deleted:: - - # pending objects recently added to the Session - session.new - - # persistent objects which currently have changes detected - # (this collection is now created on the fly each time the property is called) - session.dirty - - # persistent objects that have been marked as deleted via session.delete(obj) - session.deleted - - # dictionary of all persistent objects, keyed on their - # identity key - session.identity_map - -(Documentation: :attr:`.Session.new`, :attr:`.Session.dirty`, -:attr:`.Session.deleted`, :attr:`.Session.identity_map`). - - -.. _session_referencing_behavior: - -Session Referencing Behavior ----------------------------- - -Objects within the session are *weakly referenced*. This -means that when they are dereferenced in the outside application, they fall -out of scope from within the :class:`~sqlalchemy.orm.session.Session` as well -and are subject to garbage collection by the Python interpreter. The -exceptions to this include objects which are pending, objects which are marked -as deleted, or persistent objects which have pending changes on them. After a -full flush, these collections are all empty, and all objects are again weakly -referenced. - -To cause objects in the :class:`.Session` to remain strongly -referenced, usually a simple approach is all that's needed. Examples -of externally managed strong-referencing behavior include loading -objects into a local dictionary keyed to their primary key, or into -lists or sets for the span of time that they need to remain -referenced. These collections can be associated with a -:class:`.Session`, if desired, by placing them into the -:attr:`.Session.info` dictionary. - -An event based approach is also feasible. A simple recipe that provides -"strong referencing" behavior for all objects as they remain within -the :term:`persistent` state is as follows:: - - from sqlalchemy import event - - - def strong_reference_session(session): - @event.listens_for(session, "pending_to_persistent") - @event.listens_for(session, "deleted_to_persistent") - @event.listens_for(session, "detached_to_persistent") - @event.listens_for(session, "loaded_as_persistent") - def strong_ref_object(sess, instance): - if "refs" not in sess.info: - sess.info["refs"] = refs = set() - else: - refs = sess.info["refs"] - - refs.add(instance) - - @event.listens_for(session, "persistent_to_detached") - @event.listens_for(session, "persistent_to_deleted") - @event.listens_for(session, "persistent_to_transient") - def deref_object(sess, instance): - sess.info["refs"].discard(instance) - -Above, we intercept the :meth:`.SessionEvents.pending_to_persistent`, -:meth:`.SessionEvents.detached_to_persistent`, -:meth:`.SessionEvents.deleted_to_persistent` and -:meth:`.SessionEvents.loaded_as_persistent` event hooks in order to intercept -objects as they enter the :term:`persistent` transition, and the -:meth:`.SessionEvents.persistent_to_detached` and -:meth:`.SessionEvents.persistent_to_deleted` hooks to intercept -objects as they leave the persistent state. - -The above function may be called for any :class:`.Session` in order to -provide strong-referencing behavior on a per-:class:`.Session` basis:: - - from sqlalchemy.orm import Session - - my_session = Session() - strong_reference_session(my_session) - -It may also be called for any :class:`.sessionmaker`:: - - from sqlalchemy.orm import sessionmaker - - maker = sessionmaker() - strong_reference_session(maker) - -.. _unitofwork_merging: - -Merging -------- - -:meth:`~.Session.merge` transfers state from an -outside object into a new or already existing instance within a session. It -also reconciles the incoming data against the state of the -database, producing a history stream which will be applied towards the next -flush, or alternatively can be made to produce a simple "transfer" of -state without producing change history or accessing the database. Usage is as follows:: - - merged_object = session.merge(existing_object) - -When given an instance, it follows these steps: - -* It examines the primary key of the instance. If it's present, it attempts - to locate that instance in the local identity map. If the ``load=True`` - flag is left at its default, it also checks the database for this primary - key if not located locally. -* If the given instance has no primary key, or if no instance can be found - with the primary key given, a new instance is created. -* The state of the given instance is then copied onto the located/newly created - instance. For attribute values which are present on the source instance, the - value is transferred to the target instance. For attribute values that aren't - present on the source instance, the corresponding attribute on the target - instance is :term:`expired` from memory, which discards any locally - present value from the target instance for that attribute, but no - direct modification is made to the database-persisted value for that - attribute. - - If the ``load=True`` flag is left at its default, - this copy process emits events and will load the target object's - unloaded collections for each attribute present on the source object, - so that the incoming state can be reconciled against what's - present in the database. If ``load`` - is passed as ``False``, the incoming data is "stamped" directly without - producing any history. -* The operation is cascaded to related objects and collections, as - indicated by the ``merge`` cascade (see :ref:`unitofwork_cascades`). -* The new instance is returned. - -With :meth:`~.Session.merge`, the given "source" -instance is not modified nor is it associated with the target :class:`.Session`, -and remains available to be merged with any number of other :class:`.Session` -objects. :meth:`~.Session.merge` is useful for -taking the state of any kind of object structure without regard for its -origins or current session associations and copying its state into a -new session. Here's some examples: - -* An application which reads an object structure from a file and wishes to - save it to the database might parse the file, build up the - structure, and then use - :meth:`~.Session.merge` to save it - to the database, ensuring that the data within the file is - used to formulate the primary key of each element of the - structure. Later, when the file has changed, the same - process can be re-run, producing a slightly different - object structure, which can then be ``merged`` in again, - and the :class:`~sqlalchemy.orm.session.Session` will - automatically update the database to reflect those - changes, loading each object from the database by primary key and - then updating its state with the new state given. - -* An application is storing objects in an in-memory cache, shared by - many :class:`.Session` objects simultaneously. :meth:`~.Session.merge` - is used each time an object is retrieved from the cache to create - a local copy of it in each :class:`.Session` which requests it. - The cached object remains detached; only its state is moved into - copies of itself that are local to individual :class:`~.Session` - objects. - - In the caching use case, it's common to use the ``load=False`` - flag to remove the overhead of reconciling the object's state - with the database. There's also a "bulk" version of - :meth:`~.Session.merge` called :meth:`_query.Query.merge_result` - that was designed to work with cache-extended :class:`_query.Query` - objects - see the section :ref:`examples_caching`. - -* An application wants to transfer the state of a series of objects - into a :class:`.Session` maintained by a worker thread or other - concurrent system. :meth:`~.Session.merge` makes a copy of each object - to be placed into this new :class:`.Session`. At the end of the operation, - the parent thread/process maintains the objects it started with, - and the thread/worker can proceed with local copies of those objects. - - In the "transfer between threads/processes" use case, the application - may want to use the ``load=False`` flag as well to avoid overhead and - redundant SQL queries as the data is transferred. - -Merge Tips -~~~~~~~~~~ - -:meth:`~.Session.merge` is an extremely useful method for many purposes. However, -it deals with the intricate border between objects that are transient/detached and -those that are persistent, as well as the automated transference of state. -The wide variety of scenarios that can present themselves here often require a -more careful approach to the state of objects. Common problems with merge usually involve -some unexpected state regarding the object being passed to :meth:`~.Session.merge`. - -Lets use the canonical example of the User and Address objects:: - - class User(Base): - __tablename__ = "user" - - id = mapped_column(Integer, primary_key=True) - name = mapped_column(String(50), nullable=False) - addresses = relationship("Address", backref="user") - - - class Address(Base): - __tablename__ = "address" - - id = mapped_column(Integer, primary_key=True) - email_address = mapped_column(String(50), nullable=False) - user_id = mapped_column(Integer, ForeignKey("user.id"), nullable=False) - -Assume a ``User`` object with one ``Address``, already persistent:: - - >>> u1 = User(name="ed", addresses=[Address(email_address="ed@ed.com")]) - >>> session.add(u1) - >>> session.commit() - -We now create ``a1``, an object outside the session, which we'd like -to merge on top of the existing ``Address``:: - - >>> existing_a1 = u1.addresses[0] - >>> a1 = Address(id=existing_a1.id) - -A surprise would occur if we said this:: - - >>> a1.user = u1 - >>> a1 = session.merge(a1) - >>> session.commit() - sqlalchemy.orm.exc.FlushError: New instance
- with identity key (, (1,)) conflicts with - persistent instance
- -Why is that ? We weren't careful with our cascades. The assignment -of ``a1.user`` to a persistent object cascaded to the backref of ``User.addresses`` -and made our ``a1`` object pending, as though we had added it. Now we have -*two* ``Address`` objects in the session:: - - >>> a1 = Address() - >>> a1.user = u1 - >>> a1 in session - True - >>> existing_a1 in session - True - >>> a1 is existing_a1 - False - -Above, our ``a1`` is already pending in the session. The -subsequent :meth:`~.Session.merge` operation essentially -does nothing. Cascade can be configured via the :paramref:`_orm.relationship.cascade` -option on :func:`_orm.relationship`, although in this case it -would mean removing the ``save-update`` cascade from the -``User.addresses`` relationship - and usually, that behavior -is extremely convenient. The solution here would usually be to not assign -``a1.user`` to an object already persistent in the target -session. - -The ``cascade_backrefs=False`` option of :func:`_orm.relationship` -will also prevent the ``Address`` from -being added to the session via the ``a1.user = u1`` assignment. - -Further detail on cascade operation is at :ref:`unitofwork_cascades`. - -Another example of unexpected state:: - - >>> a1 = Address(id=existing_a1.id, user_id=u1.id) - >>> a1.user = None - >>> a1 = session.merge(a1) - >>> session.commit() - sqlalchemy.exc.IntegrityError: (IntegrityError) address.user_id - may not be NULL - -Above, the assignment of ``user`` takes precedence over the foreign key -assignment of ``user_id``, with the end result that ``None`` is applied -to ``user_id``, causing a failure. - -Most :meth:`~.Session.merge` issues can be examined by first checking - -is the object prematurely in the session ? - -.. sourcecode:: pycon+sql - - >>> a1 = Address(id=existing_a1, user_id=user.id) - >>> assert a1 not in session - >>> a1 = session.merge(a1) - -Or is there state on the object that we don't want ? Examining ``__dict__`` -is a quick way to check:: - - >>> a1 = Address(id=existing_a1, user_id=user.id) - >>> a1.user - >>> a1.__dict__ - {'_sa_instance_state': , - 'user_id': 1, - 'id': 1, - 'user': None} - >>> # we don't want user=None merged, remove it - >>> del a1.user - >>> a1 = session.merge(a1) - >>> # success - >>> session.commit() - -Expunging ---------- - -Expunge removes an object from the Session, sending persistent instances to -the detached state, and pending instances to the transient state: - -.. sourcecode:: python+sql - - session.expunge(obj1) - -To remove all items, call :meth:`~.Session.expunge_all` -(this method was formerly known as ``clear()``). - -.. _session_expire: - -Refreshing / Expiring ---------------------- - -:term:`Expiring` means that the database-persisted data held inside a series -of object attributes is erased, in such a way that when those attributes -are next accessed, a SQL query is emitted which will refresh that data from -the database. - -When we talk about expiration of data we are usually talking about an object -that is in the :term:`persistent` state. For example, if we load an object -as follows:: - - user = session.scalars(select(User).filter_by(name="user1").limit(1)).first() - -The above ``User`` object is persistent, and has a series of attributes -present; if we were to look inside its ``__dict__``, we'd see that state -loaded:: - - >>> user.__dict__ - { - 'id': 1, 'name': u'user1', - '_sa_instance_state': <...>, - } - -where ``id`` and ``name`` refer to those columns in the database. -``_sa_instance_state`` is a non-database-persisted value used by SQLAlchemy -internally (it refers to the :class:`.InstanceState` for the instance. -While not directly relevant to this section, if we want to get at it, -we should use the :func:`_sa.inspect` function to access it). - -At this point, the state in our ``User`` object matches that of the loaded -database row. But upon expiring the object using a method such as -:meth:`.Session.expire`, we see that the state is removed:: - - >>> session.expire(user) - >>> user.__dict__ - {'_sa_instance_state': <...>} - -We see that while the internal "state" still hangs around, the values which -correspond to the ``id`` and ``name`` columns are gone. If we were to access -one of these columns and are watching SQL, we'd see this: - -.. sourcecode:: pycon+sql - - >>> print(user.name) - {execsql}SELECT user.id AS user_id, user.name AS user_name - FROM user - WHERE user.id = ? - (1,) - {stop}user1 - -Above, upon accessing the expired attribute ``user.name``, the ORM initiated -a :term:`lazy load` to retrieve the most recent state from the database, -by emitting a SELECT for the user row to which this user refers. Afterwards, -the ``__dict__`` is again populated:: - - >>> user.__dict__ - { - 'id': 1, 'name': u'user1', - '_sa_instance_state': <...>, - } - -.. note:: While we are peeking inside of ``__dict__`` in order to see a bit - of what SQLAlchemy does with object attributes, we **should not modify** - the contents of ``__dict__`` directly, at least as far as those attributes - which the SQLAlchemy ORM is maintaining (other attributes outside of SQLA's - realm are fine). This is because SQLAlchemy uses :term:`descriptors` in - order to track the changes we make to an object, and when we modify ``__dict__`` - directly, the ORM won't be able to track that we changed something. - -Another key behavior of both :meth:`~.Session.expire` and :meth:`~.Session.refresh` -is that all un-flushed changes on an object are discarded. That is, -if we were to modify an attribute on our ``User``:: - - >>> user.name = "user2" - -but then we call :meth:`~.Session.expire` without first calling :meth:`~.Session.flush`, -our pending value of ``'user2'`` is discarded:: - - >>> session.expire(user) - >>> user.name - 'user1' - -The :meth:`~.Session.expire` method can be used to mark as "expired" all ORM-mapped -attributes for an instance:: - - # expire all ORM-mapped attributes on obj1 - session.expire(obj1) - -it can also be passed a list of string attribute names, referring to specific -attributes to be marked as expired:: - - # expire only attributes obj1.attr1, obj1.attr2 - session.expire(obj1, ["attr1", "attr2"]) - -The :meth:`.Session.expire_all` method allows us to essentially call -:meth:`.Session.expire` on all objects contained within the :class:`.Session` -at once:: - - session.expire_all() - -The :meth:`~.Session.refresh` method has a similar interface, but instead -of expiring, it emits an immediate SELECT for the object's row immediately:: - - # reload all attributes on obj1 - session.refresh(obj1) - -:meth:`~.Session.refresh` also accepts a list of string attribute names, -but unlike :meth:`~.Session.expire`, expects at least one name to -be that of a column-mapped attribute:: - - # reload obj1.attr1, obj1.attr2 - session.refresh(obj1, ["attr1", "attr2"]) - -.. tip:: - - An alternative method of refreshing which is often more flexible is to - use the :ref:`orm_queryguide_populate_existing` feature of the ORM, - available for :term:`2.0 style` queries with :func:`_sql.select` as well - as from the :meth:`_orm.Query.populate_existing` method of :class:`_orm.Query` - within :term:`1.x style` queries. Using this execution option, - all of the ORM objects returned in the result set of the statement - will be refreshed with data from the database:: - - stmt = ( - select(User) - .execution_options(populate_existing=True) - .where((User.name.in_(["a", "b", "c"]))) - ) - for user in session.execute(stmt).scalars(): - print(user) # will be refreshed for those columns that came back from the query - - See :ref:`orm_queryguide_populate_existing` for further detail. - - -What Actually Loads -~~~~~~~~~~~~~~~~~~~ - -The SELECT statement that's emitted when an object marked with :meth:`~.Session.expire` -or loaded with :meth:`~.Session.refresh` varies based on several factors, including: - -* The load of expired attributes is triggered from **column-mapped attributes only**. - While any kind of attribute can be marked as expired, including a - :func:`_orm.relationship` - mapped attribute, accessing an expired :func:`_orm.relationship` - attribute will emit a load only for that attribute, using standard - relationship-oriented lazy loading. Column-oriented attributes, even if - expired, will not load as part of this operation, and instead will load when - any column-oriented attribute is accessed. - -* :func:`_orm.relationship`- mapped attributes will not load in response to - expired column-based attributes being accessed. - -* Regarding relationships, :meth:`~.Session.refresh` is more restrictive than - :meth:`.Session.expire` with regards to attributes that aren't column-mapped. - Calling :meth:`.Session.refresh` and passing a list of names that only includes - relationship-mapped attributes will actually raise an error. - In any case, non-eager-loading :func:`_orm.relationship` attributes will not be - included in any refresh operation. - -* :func:`_orm.relationship` attributes configured as "eager loading" via the - :paramref:`_orm.relationship.lazy` parameter will load in the case of - :meth:`~.Session.refresh`, if either no attribute names are specified, or - if their names are included in the list of attributes to be - refreshed. - -* Attributes that are configured as :func:`.deferred` will not normally load, - during either the expired-attribute load or during a refresh. - An unloaded attribute that's :func:`.deferred` instead loads on its own when directly - accessed, or if part of a "group" of deferred attributes where an unloaded - attribute in that group is accessed. - -* For expired attributes that are loaded on access, a joined-inheritance table - mapping will emit a SELECT that typically only includes those tables for which - unloaded attributes are present. The action here is sophisticated enough - to load only the parent or child table, for example, if the subset of columns - that were originally expired encompass only one or the other of those tables. - -* When :meth:`~.Session.refresh` is used on a joined-inheritance table mapping, - the SELECT emitted will resemble that of when :meth:`.Session.query` is - used on the target object's class. This is typically all those tables that - are set up as part of the mapping. - - -When to Expire or Refresh -~~~~~~~~~~~~~~~~~~~~~~~~~ - -The :class:`.Session` uses the expiration feature automatically whenever -the transaction referred to by the session ends. Meaning, whenever :meth:`.Session.commit` -or :meth:`.Session.rollback` is called, all objects within the :class:`.Session` -are expired, using a feature equivalent to that of the :meth:`.Session.expire_all` -method. The rationale is that the end of a transaction is a -demarcating point at which there is no more context available in order to know -what the current state of the database is, as any number of other transactions -may be affecting it. Only when a new transaction starts can we again have access -to the current state of the database, at which point any number of changes -may have occurred. - -.. sidebar:: Transaction Isolation - - Of course, most databases are capable of handling - multiple transactions at once, even involving the same rows of data. When - a relational database handles multiple transactions involving the same - tables or rows, this is when the :term:`isolation` aspect of the database comes - into play. The isolation behavior of different databases varies considerably - and even on a single database can be configured to behave in different ways - (via the so-called :term:`isolation level` setting). In that sense, the :class:`.Session` - can't fully predict when the same SELECT statement, emitted a second time, - will definitely return the data we already have, or will return new data. - So as a best guess, it assumes that within the scope of a transaction, unless - it is known that a SQL expression has been emitted to modify a particular row, - there's no need to refresh a row unless explicitly told to do so. - -The :meth:`.Session.expire` and :meth:`.Session.refresh` methods are used in -those cases when one wants to force an object to re-load its data from the -database, in those cases when it is known that the current state of data -is possibly stale. Reasons for this might include: - -* some SQL has been emitted within the transaction outside of the - scope of the ORM's object handling, such as if a :meth:`_schema.Table.update` construct - were emitted using the :meth:`.Session.execute` method; - -* if the application - is attempting to acquire data that is known to have been modified in a - concurrent transaction, and it is also known that the isolation rules in effect - allow this data to be visible. - -The second bullet has the important caveat that "it is also known that the isolation rules in effect -allow this data to be visible." This means that it cannot be assumed that an -UPDATE that happened on another database connection will yet be visible here -locally; in many cases, it will not. This is why if one wishes to use -:meth:`.Session.expire` or :meth:`.Session.refresh` in order to view data between ongoing -transactions, an understanding of the isolation behavior in effect is essential. - -.. seealso:: - - :meth:`.Session.expire` - - :meth:`.Session.expire_all` - - :meth:`.Session.refresh` - - :ref:`orm_queryguide_populate_existing` - allows any ORM query - to refresh objects as they would be loaded normally, refreshing - all matching objects in the identity map against the results of a - SELECT statement. - - :term:`isolation` - glossary explanation of isolation which includes links - to Wikipedia. - - `The SQLAlchemy Session In-Depth `_ - a video + slides with an in-depth discussion of the object - lifecycle including the role of data expiration. diff --git a/doc/build/orm/versioning.rst b/doc/build/orm/versioning.rst deleted file mode 100644 index 87865917cd..0000000000 --- a/doc/build/orm/versioning.rst +++ /dev/null @@ -1,254 +0,0 @@ -.. _mapper_version_counter: - -Configuring a Version Counter -============================= - -The :class:`_orm.Mapper` supports management of a :term:`version id column`, which -is a single table column that increments or otherwise updates its value -each time an ``UPDATE`` to the mapped table occurs. This value is checked each -time the ORM emits an ``UPDATE`` or ``DELETE`` against the row to ensure that -the value held in memory matches the database value. - -.. warning:: - - Because the versioning feature relies upon comparison of the **in memory** - record of an object, the feature only applies to the :meth:`.Session.flush` - process, where the ORM flushes individual in-memory rows to the database. - It does **not** take effect when performing - a multirow UPDATE or DELETE using :meth:`_query.Query.update` or :meth:`_query.Query.delete` - methods, as these methods only emit an UPDATE or DELETE statement but otherwise - do not have direct access to the contents of those rows being affected. - -The purpose of this feature is to detect when two concurrent transactions -are modifying the same row at roughly the same time, or alternatively to provide -a guard against the usage of a "stale" row in a system that might be re-using -data from a previous transaction without refreshing (e.g. if one sets ``expire_on_commit=False`` -with a :class:`.Session`, it is possible to re-use the data from a previous -transaction). - -.. topic:: Concurrent transaction updates - - When detecting concurrent updates within transactions, it is typically the - case that the database's transaction isolation level is below the level of - :term:`repeatable read`; otherwise, the transaction will not be exposed - to a new row value created by a concurrent update which conflicts with - the locally updated value. In this case, the SQLAlchemy versioning - feature will typically not be useful for in-transaction conflict detection, - though it still can be used for cross-transaction staleness detection. - - The database that enforces repeatable reads will typically either have locked the - target row against a concurrent update, or is employing some form - of multi version concurrency control such that it will emit an error - when the transaction is committed. SQLAlchemy's version_id_col is an alternative - which allows version tracking to occur for specific tables within a transaction - that otherwise might not have this isolation level set. - - .. seealso:: - - `Repeatable Read Isolation Level `_ - PostgreSQL's implementation of repeatable read, including a description of the error condition. - -Simple Version Counting ------------------------ - -The most straightforward way to track versions is to add an integer column -to the mapped table, then establish it as the ``version_id_col`` within the -mapper options:: - - class User(Base): - __tablename__ = "user" - - id = mapped_column(Integer, primary_key=True) - version_id = mapped_column(Integer, nullable=False) - name = mapped_column(String(50), nullable=False) - - __mapper_args__ = {"version_id_col": version_id} - -.. note:: It is **strongly recommended** that the ``version_id`` column - be made NOT NULL. The versioning feature **does not support** a NULL - value in the versioning column. - -Above, the ``User`` mapping tracks integer versions using the column -``version_id``. When an object of type ``User`` is first flushed, the -``version_id`` column will be given a value of "1". Then, an UPDATE -of the table later on will always be emitted in a manner similar to the -following: - -.. sourcecode:: sql - - UPDATE user SET version_id=:version_id, name=:name - WHERE user.id = :user_id AND user.version_id = :user_version_id - -- {"name": "new name", "version_id": 2, "user_id": 1, "user_version_id": 1} - -The above UPDATE statement is updating the row that not only matches -``user.id = 1``, it also is requiring that ``user.version_id = 1``, where "1" -is the last version identifier we've been known to use on this object. -If a transaction elsewhere has modified the row independently, this version id -will no longer match, and the UPDATE statement will report that no rows matched; -this is the condition that SQLAlchemy tests, that exactly one row matched our -UPDATE (or DELETE) statement. If zero rows match, that indicates our version -of the data is stale, and a :exc:`.StaleDataError` is raised. - -.. _custom_version_counter: - -Custom Version Counters / Types -------------------------------- - -Other kinds of values or counters can be used for versioning. Common types include -dates and GUIDs. When using an alternate type or counter scheme, SQLAlchemy -provides a hook for this scheme using the ``version_id_generator`` argument, -which accepts a version generation callable. This callable is passed the value of the current -known version, and is expected to return the subsequent version. - -For example, if we wanted to track the versioning of our ``User`` class -using a randomly generated GUID, we could do this (note that some backends -support a native GUID type, but we illustrate here using a simple string):: - - import uuid - - - class User(Base): - __tablename__ = "user" - - id = mapped_column(Integer, primary_key=True) - version_uuid = mapped_column(String(32), nullable=False) - name = mapped_column(String(50), nullable=False) - - __mapper_args__ = { - "version_id_col": version_uuid, - "version_id_generator": lambda version: uuid.uuid4().hex, - } - -The persistence engine will call upon ``uuid.uuid4()`` each time a -``User`` object is subject to an INSERT or an UPDATE. In this case, our -version generation function can disregard the incoming value of ``version``, -as the ``uuid4()`` function -generates identifiers without any prerequisite value. If we were using -a sequential versioning scheme such as numeric or a special character system, -we could make use of the given ``version`` in order to help determine the -subsequent value. - -.. seealso:: - - :ref:`custom_guid_type` - -.. _server_side_version_counter: - -Server Side Version Counters ----------------------------- - -The ``version_id_generator`` can also be configured to rely upon a value -that is generated by the database. In this case, the database would need -some means of generating new identifiers when a row is subject to an INSERT -as well as with an UPDATE. For the UPDATE case, typically an update trigger -is needed, unless the database in question supports some other native -version identifier. The PostgreSQL database in particular supports a system -column called `xmin `_ -which provides UPDATE versioning. We can make use -of the PostgreSQL ``xmin`` column to version our ``User`` -class as follows:: - - from sqlalchemy import FetchedValue - - - class User(Base): - __tablename__ = "user" - - id = mapped_column(Integer, primary_key=True) - name = mapped_column(String(50), nullable=False) - xmin = mapped_column("xmin", String, system=True, server_default=FetchedValue()) - - __mapper_args__ = {"version_id_col": xmin, "version_id_generator": False} - -With the above mapping, the ORM will rely upon the ``xmin`` column for -automatically providing the new value of the version id counter. - -.. topic:: creating tables that refer to system columns - - In the above scenario, as ``xmin`` is a system column provided by PostgreSQL, - we use the ``system=True`` argument to mark it as a system-provided - column, omitted from the ``CREATE TABLE`` statement. The datatype of this - column is an internal PostgreSQL type called ``xid`` which acts mostly - like a string, so we use the :class:`_types.String` datatype. - - -The ORM typically does not actively fetch the values of database-generated -values when it emits an INSERT or UPDATE, instead leaving these columns as -"expired" and to be fetched when they are next accessed, unless the ``eager_defaults`` -:class:`_orm.Mapper` flag is set. However, when a -server side version column is used, the ORM needs to actively fetch the newly -generated value. This is so that the version counter is set up *before* -any concurrent transaction may update it again. This fetching is also -best done simultaneously within the INSERT or UPDATE statement using :term:`RETURNING`, -otherwise if emitting a SELECT statement afterwards, there is still a potential -race condition where the version counter may change before it can be fetched. - -When the target database supports RETURNING, an INSERT statement for our ``User`` class will look -like this: - -.. sourcecode:: sql - - INSERT INTO "user" (name) VALUES (%(name)s) RETURNING "user".id, "user".xmin - -- {'name': 'ed'} - -Where above, the ORM can acquire any newly generated primary key values along -with server-generated version identifiers in one statement. When the backend -does not support RETURNING, an additional SELECT must be emitted for **every** -INSERT and UPDATE, which is much less efficient, and also introduces the possibility of -missed version counters: - -.. sourcecode:: sql - - INSERT INTO "user" (name) VALUES (%(name)s) - -- {'name': 'ed'} - - SELECT "user".version_id AS user_version_id FROM "user" where - "user".id = :param_1 - -- {"param_1": 1} - -It is *strongly recommended* that server side version counters only be used -when absolutely necessary and only on backends that support :term:`RETURNING`, -currently PostgreSQL, Oracle, MariaDB 10.5, SQLite 3.35, and SQL Server. - - -Programmatic or Conditional Version Counters --------------------------------------------- - -When ``version_id_generator`` is set to False, we can also programmatically -(and conditionally) set the version identifier on our object in the same way -we assign any other mapped attribute. Such as if we used our UUID example, but -set ``version_id_generator`` to ``False``, we can set the version identifier -at our choosing:: - - import uuid - - - class User(Base): - __tablename__ = "user" - - id = mapped_column(Integer, primary_key=True) - version_uuid = mapped_column(String(32), nullable=False) - name = mapped_column(String(50), nullable=False) - - __mapper_args__ = {"version_id_col": version_uuid, "version_id_generator": False} - - - u1 = User(name="u1", version_uuid=uuid.uuid4()) - - session.add(u1) - - session.commit() - - u1.name = "u2" - u1.version_uuid = uuid.uuid4() - - session.commit() - -We can update our ``User`` object without incrementing the version counter -as well; the value of the counter will remain unchanged, and the UPDATE -statement will still check against the previous value. This may be useful -for schemes where only certain classes of UPDATE are sensitive to concurrency -issues:: - - # will leave version_uuid unchanged - u1.name = "u3" - session.commit()