From: dan Date: Fri, 11 Jan 2019 14:58:03 +0000 (+0000) Subject: Add new file doc/wal2.md to this branch. X-Git-Url: http://git.ipfire.org/?a=commitdiff_plain;h=f552a2374008e72ec6ebd62bf060ce5655b67148;p=thirdparty%2Fsqlite.git Add new file doc/wal2.md to this branch. FossilOrigin-Name: b495dce153f7f886f3dba09593f29ce2375718bf6508f2cfffd1af8071a995ae --- diff --git a/doc/wal2.md b/doc/wal2.md new file mode 100644 index 0000000000..5afd8ae606 --- /dev/null +++ b/doc/wal2.md @@ -0,0 +1,98 @@ + +Wal2 Mode Notes +=============== + +## Activating/Deactivating Wal2 Mode + +"Wal2" mode is very similar to "wal" mode. To change a database to wal2 mode, +use the command: + +> + PRAGMA journal_mode = wal2; + +It is not possible to change a database directly from "wal" mode to "wal2" +mode. Instead, it must first be changed to rollback mode. So, to change a wal +mode database to wal2 mode, the following two commands may be used: + +> + PRAGMA journal_mode = delete; + PRAGMA journal_mode = wal2; + +A database in wal2 mode may only be accessed by versions of SQLite compiled +from this branch. Attempting to use any other version of SQLite results in an +SQLITE_NOTADB error. A wal2 mode database may be changed back to rollback mode +(making it accessible by all versions of SQLite) using: + +> + PRAGMA journal_mode = delete; + +## The Advantage of Wal2 Mode + +In legacy wal mode, when a writer writes data to the database, it doesn't +modify the database file directly. Instead, it appends new data to the +"<database>-wal" file. Readers read data from both the original database +file and the "<database>-wal" file. At some point, data is copied from the +"<database>-wal" file into the database file, after which the wal file can +be deleted or overwritten. Copying data from the wal file into the database +file is called a "checkpoint", and may be done explictly (either by "PRAGMA +wal_checkpoint" or sqlite3_wal_checkpoint_v2()), or +automatically (by configuring "PRAGMA wal_autocheckpoint" - this is the +default). + +Checkpointers do not block writers, and writers do not block checkpointers. +However, if a writer writes to the database while a checkpoint is ongoing, +then the new data is appended to the end of the wal file. This means that, +even following the checkpoint, the wal file cannot be overwritten or deleted, +and so all subsequent transactions must also be appended to the wal file. The +work of the checkpointer is not wasted - SQLite remembers which parts of the +wal file have already been copied into the db file so that the next checkpoint +does not have to do so again - but it does mean that the wal file may grow +indefinitely if the checkpointer never gets a chance to finish without a +writer appending to the wal file. There are also circumstances in which +long-running readers may prevent a checkpointer from checkpointing the entire +wal file - also causing the wal file to grow indefinitely in a busy system. + +Wal2 mode does not have this problem. In wal2 mode, wal files do not grow +indefinitely even if the checkpointer never has a chance to finish +uninterrupted. + +In wal2 mode, the system uses two wal files instead of one. The files are named +"<database>-wal" and "<database>-wal2", where "<database>" is of +course the name of the database file. When data is written to the database, the +writer begins by appending the new data to the first wal file. Once the first +wal file has grown large enough, writers switch to appending data to the second +wal file. At this point the first wal file can be checkpointed (after which it +can be overwritten). Then, once the second wal file has grown large enough and +the first wal file has been checkpointed, writers switch back to the first wal +file. And so on. + +## Application Programming + +From the point of view of the user, the main differences between wal and +wal2 mode are to do with checkpointing: + + * In wal mode, a checkpoint may be attempted at any time. In wal2 + mode, the checkpointer has to wait until writers have switched + to the "other" wal file before a checkpoint can take place. + + * In wal mode, the wal-hook (callback registered using + sqlite3_wal_hook()) is invoked after a transaction is committed + with the total number of pages in the wal file as an argument. In wal2 + mode, the argument is either the total number of uncheckpointed pages in + both wal files, or - if the "other" wal file is empty or already + checkpointed - 0. + +Clients are recommended to use the same strategies for checkpointing wal2 mode +databases as for wal databases - by registering a wal-hook using +sqlite3_wal_hook() and attempting a checkpoint when the parameter +exceeds a certain threshold. + +However, it should be noted that although the wal-hook is invoked after each +transaction is committed to disk and database locks released, it is still +invoked from within the sqlite3_step() call used to execute the "COMMIT" +command. In BEGIN CONCURRENT systems, where the "COMMIT" is often protected by +an application mutex, this may reduce concurrency. In such systems, instead of +executing a checkpoint from within the wal-hook, a thread might defer this +action until after the application mutex has been released. + + diff --git a/manifest b/manifest index 575be8a9db..6af6923c47 100644 --- a/manifest +++ b/manifest @@ -1,5 +1,5 @@ -C Reinstate\sassert()\sstatements\sin\sos_unix.c\sthat\swere\sremoved\sto\sallow\swal-mode\nSHARED\slocks\sto\sbe\staken\sover\smore\sthan\sone\slocking\sslot\s(this\sbranch\sno\nlonger\sdoes\sthat,\sso\sthe\sassert()\sstatements\scan\sgo\sback\sin). -D 2019-01-02T17:00:00.197 +C Add\snew\sfile\sdoc/wal2.md\sto\sthis\sbranch. +D 2019-01-11T14:58:03.764 F .fossil-settings/empty-dirs dbb81e8fc0401ac46a1491ab34a7f2c7c0452f2f06b54ebb845d024ca8283ef1 F .fossil-settings/ignore-glob 35175cdfcf539b2318cb04a9901442804be81cd677d8b889fcc9149c21f239ea F Makefile.in d8b254f8bb81bab43c340d70d17dc3babab40fcc8a348c8255881f780a45fee6 @@ -40,6 +40,7 @@ F doc/F2FS.txt c1d4a0ae9711cfe0e1d8b019d154f1c29e0d3abfe820787ba1e9ed7691160fcd F doc/lemon.html 24956ab2995e55fe171e55bdd04f22b553957dc8bb43501dbb9311e30187e0d3 F doc/pager-invariants.txt 27fed9a70ddad2088750c4a2b493b63853da2710 F doc/vfs-shm.txt e101f27ea02a8387ce46a05be2b1a902a021d37a +F doc/wal2.md a807405a05e19a4945c5905a9ffa0fe45b8560dd7572461192501f565c19cdb5 F ext/README.md fd5f78013b0a2bc6f0067afb19e6ad040e89a10179b4f6f03eee58fac5f169bd F ext/async/README.txt e12275968f6fde133a80e04387d0e839b0c51f91 F ext/async/sqlite3async.c 0f3070cc3f5ede78f2b9361fb3b629ce200d7d74 @@ -1806,7 +1807,7 @@ F vsixtest/vsixtest.tcl 6a9a6ab600c25a91a7acc6293828957a386a8a93 F vsixtest/vsixtest.vcxproj.data 2ed517e100c66dc455b492e1a33350c1b20fbcdc F vsixtest/vsixtest.vcxproj.filters 37e51ffedcdb064aad6ff33b6148725226cd608e F vsixtest/vsixtest_TemporaryKey.pfx e5b1b036facdb453873e7084e1cae9102ccc67a0 -P 87ef68f9174b5e21fa2fb4f2fa23e0ecd6a0ea854422c8aa74cd7306a1ed8427 -R 81c151bc78b917b5a505ba7d0041b59f +P 8445fb6d5dae98a0129514a10a15f08412bac5a1d8114b12e592c744ea2bc4c9 +R 4b991801ac5f384be1f414fb71a05937 U dan -Z 69b1e094c88bc1f9442e8f44aee16f43 +Z 736399187a1b654bff250254a2a37532 diff --git a/manifest.uuid b/manifest.uuid index 56ed701164..1695a54411 100644 --- a/manifest.uuid +++ b/manifest.uuid @@ -1 +1 @@ -8445fb6d5dae98a0129514a10a15f08412bac5a1d8114b12e592c744ea2bc4c9 \ No newline at end of file +b495dce153f7f886f3dba09593f29ce2375718bf6508f2cfffd1af8071a995ae \ No newline at end of file