--- /dev/null
+
+This file contains notes regarding the implementation of the OTA extension.
+User documentation is in sqlite3ota.h.
+
+SQLite Hacks
+------------
+
+1) PRAGMA ota_mode:
+
+ This is a new flag pragma. If the flag is set:
+
+ * INSERT/DELETE/UPDATE commands are prevented from updating any but the main
+ b-tree for each table (the PK index for WITHOUT ROWID tables or the
+ rowid b-tree for others).
+
+ * The above statements do not check UNIQUE constraints - except those enforced
+ by the main b-tree.
+
+ * All non-temporary triggers are disabled.
+
+
+2) PRAGMA pager_ota_mode:
+
+ This pragma sets a flag on the pager associated with the main database only. In
+ a zipvfs system, this pragma is intercepted by zipvfs and the flag is set on
+ the lower level pager only.
+
+ The flag can only be set when there is no open transaction and the pager does
+ not already have an open WAL file. Attempting to do so is an error.
+
+ Once the flag has been set, it is not possible to open a regular WAL file.
+ If, when the next read-transaction is opened, a *-wal file is found or the
+ database header flags indicate that it is a wal-mode database,
+ SQLITE_CANTOPEN is returned.
+
+ Otherwise, if no WAL file or flags are found, the pager opens the *-oal file
+ and uses it as a write-ahead-log with the *-shm data stored in heap-memory.
+
+ The 8-bytes of "salt" at teh start of an *-oal file is a copy of the 8 bytes
+ starting at offset 24 of the database file header (the change counter and the
+ number of pages in the file). If the *-oal file already exists when it is
+ opened, SQLite checks that the salt still matches the database header fields.
+ If not, it concludes that the database file has been written by a rollback-mode
+ client since the *-oal wa created and an SQLITE_BUSY_SNAPSHOT error is
+ returned. No read-transaction can be opened in this case.
+
+ A pager with the pager_ota_mode flag set never runs a checkpoint.
+
+ Other clients see a rollback-mode database on which the pager_ota_mode client
+ is holding a SHARED lock. There are no locks to arbitrate between multiple
+ pager_ota_mode connections. If two or more such connections attempt to write
+ simultaneously, the results are undefined.
+
+
+3) sqlite3_index_writer()
+
+ This new API function is used to create VMs that can insert or delete entries
+ from individual index b-trees within the database. The VMs apply affinities
+ and check that UNIQUE constraints are not violated before updating index
+ b-trees.
+
+
+The OTA extension
+-----------------
+
+The OTA extension requires that the OTA update be packaged as an SQLite
+database. The tables it expects to find are described in sqlite3ota.h.
+Essentially, for each table xyz in the target database that the user wishes
+to write to, a corresponding data_xyz table is created in the OTA database
+and populated with one row for each row to update, insert or delete from
+the target table.
+
+The OTA extension opens the target and OTA update databases using a single
+database handle (the target database is "main", and the OTA update database is
+attached as "ota"). It executes both the "pager_ota_mode" and "ota_mode"
+pragmas described above. For each data_xyz table in then:
+
+ * CREATEs an ota_xyz table in the OTA update database.
+
+ * Loops through the data_xyz table, running the INSERT, UPDATE or DELETE
+ command on the corresponding target database table. Only the main b-tree
+ is updated by these statements. Modified pages are appended to the *-oal
+ file.
+
+ Temporary triggers installed on the target database catch the old.*
+ values associated with any UPDATEd or DELETEd rows and store them in
+ the ota_xyz table (in the OTA update database).
+
+ * For each index on the data_xyz table in the target database:
+
+ Loop through a union of the data_xyz and ota_xyz tables in the order
+ specified by the data_xyz index. In other words, if the index is on
+ columns (a, b), read rows from the OTA update database using:
+
+ SELECT * FROM data_xyz UNION ALL ota_xyz ORDER BY a, b;
+
+ For each row visited, use an sqlite3_index_writer() VM to update the index
+ in the target database.
+
+ * DROPs the ota_xyz table.
+
+At any point in the above, the process may be suspended by the user. In this
+case the "ota_state" table is created in the OTA database, containing a single
+row indicating the current table/index being processed and the number of updates
+already performed on it, and the transaction on the target database is committed
+to the *-oal file. The next OTA client will use the contents of the ota_state
+table to continue the update from where this one left off.
+
+Alternatively, if the OTA update is completely applied, the transaction is
+committed to the *-oal file and the database connection closed. sqlite3ota.c
+then uses a rename() call to move the *-oal file to the corresponding *-wal
+path. At that point it is finished - it does not take responsibility for
+checkpointing the *-wal file.
+
+
+Problems
+--------
+
+The rename() call might not be portable. And in theory it is unsafe if some
+other client starts writing the db file.
+
+When state is saved, the commit to the *-oal file and the commit to the OTA
+update database are not atomic. So if the power fails at the wrong moment they
+might get out of sync. As the main database will be committed before the OTA
+update database this will likely either just pass unnoticed, or result in
+SQLITE_CONSTRAINT errors (due to UNIQUE constraint violations).
+
+If some client does modify the target database mid OTA update, or some other
+error occurs, the OTA extension will keep throwing errors. It's not really
+clear how to get out of this state. The system could just by delete the OTA
+update database and *-oal file and have the device download the update again
+and start over.
+
+At present, for an UPDATE, both the new.* and old.* records are collected in
+the ota_xyz table. And for both UPDATEs and DELETEs all fields are collected.
+This means we're probably writing a lot more data to disk when saving the
+state of an ongoing update to the OTA update database than is strictly
+necessary.
+
+
+
+
-C Add\san\sAPI\sto\squery\san\sota\shandle\sfor\sthe\stotal\snumber\sof\skey/value\soperations\sperformed\sso\sfar.
-D 2014-09-18T14:48:38.579
+C Add\snew\sfile\sext/ota/README.txt,\scontaining\snotes\sregarding\sthe\simplementation\sof\sthe\sota\sextension.
+D 2014-09-18T15:22:48.861
F Makefile.arm-wince-mingw32ce-gcc d6df77f1f48d690bd73162294bbba7f59507c72f
F Makefile.in cf57f673d77606ab0f2d9627ca52a9ba1464146a
F Makefile.linux-gcc 91d710bdc4998cb015f39edf3cb314ec4f4d7e23
F ext/misc/vfslog.c fe40fab5c077a40477f7e5eba994309ecac6cc95
F ext/misc/vtshim.c babb0dc2bf116029e3e7c9a618b8a1377045303e
F ext/misc/wholenumber.c 784b12543d60702ebdd47da936e278aa03076212
+F ext/ota/README.txt cb11e39bfeba952ac8896dab860ada9d54731fb8
F ext/ota/ota.c c11a85af71dccc45976622fe7a51169a481caa91
F ext/ota/ota1.test 7cbf37a9f6cd29320f47b041cfeb0cc1d7eaa916
F ext/ota/ota2.test 716f9c66e8bf8b0ad2fe3a5d8323e6cf460a2e27
F tool/warnings-clang.sh f6aa929dc20ef1f856af04a730772f59283631d4
F tool/warnings.sh 0abfd78ceb09b7f7c27c688c8e3fe93268a13b32
F tool/win/sqlite.vsix deb315d026cc8400325c5863eef847784a219a2f
-P 67ea2979d5831b6d0d55173bd9413b21644cf6a1
-R 38d6ede7d1ef519b3ce4a32ffbb6512b
+P e3943fa7bbbfc5e16f73a494d8fa54d19e9cfcf9
+R fb02b3fe38b193b1aa6184b0012599dc
U dan
-Z 2cc714eb22b38452d154c6b43dc8544c
+Z 4cd279ef72288cc868c417c33c545f0f
projects / thirdparty / sqlite.git / commitdiff
