From: Vlastimil Zíma <vlastimil.zima@nic.cz>
Date: Thu, 26 May 2022 08:53:01 +0000 (+0200)
Subject: Add docs for data migrations
X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=refs%2Fpull%2F1040%2Fhead;p=thirdparty%2Fsqlalchemy%2Falembic.git

Add docs for data migrations
---

diff --git a/docs/build/cookbook.rst b/docs/build/cookbook.rst
index 23202ea9..ee4bd5b7 100644
--- a/docs/build/cookbook.rst
+++ b/docs/build/cookbook.rst
@@ -1573,4 +1573,41 @@ the same ``env.py`` file can be invoked using asyncio as::
             await conn.run_sync(run_upgrade, config.Config("alembic.ini"))
 
 
-    asyncio.run(run_async_upgrade())
\ No newline at end of file
+    asyncio.run(run_async_upgrade())
+
+
+Data migrations
+===============
+
+Alembic migrations are designed for schema migrations.
+The nature of data migrations are inherently different and it's not in fact advisable in the general case to write data migrations that integrate with Alembic's schema versioning model.
+For example downgrades are difficult to address since they might require deletion of data, which may even not be possible to detect.
+
+.. warning::
+
+  The solution needs to be designed specifically for each individual application and migration.
+  There are no general rules and the following text is only a recommendation based on experience.
+
+There are three basic approaches for the data migrations.
+
+Small data
+----------
+Small data migrations are easy to perform, especially in cases of initial data to a new table.
+These can be handled using :meth:`.Operations.bulk_insert`.
+
+Separate migration script
+-------------------------
+One possibility is a completely separate script aside of alembic migrations.
+The complete migration is then processed in following steps:
+
+1. Run the initial alembic migrations (new columns etc.)
+2. Run the separate data migration script
+3. Run the final alembic migrations (database constraints, delete columns etc.)
+
+The data migration script may also need a separate ORM model to handle intermediate state of the database.
+
+Online migration
+----------------
+The application maintains a version of schema with both versions.
+Writes are performed on both places, while the background script move all the remaining data across.
+This technique is very challenging and time demanding.