From: Pascal Volk Date: Sun, 5 Oct 2014 20:18:20 +0000 (+0000) Subject: man: Reworked dsync.1 and renamed it to doveadm-sync.1. X-Git-Tag: 2.2.14~59 X-Git-Url: http://git.ipfire.org/?a=commitdiff_plain;h=b0bc86df07375ae7b5344cf925fbb6dc71fa3a85;p=thirdparty%2Fdovecot%2Fcore.git man: Reworked dsync.1 and renamed it to doveadm-sync.1. --- diff --git a/.hgignore b/.hgignore index 6342391ef6..52fbe3148e 100644 --- a/.hgignore +++ b/.hgignore @@ -105,5 +105,5 @@ src/plugins/quota/quota-status syntax: regexp src/.*/test-[^\.]*$ -doc/man/doveadm-(acl|altmove|auth|batch|deduplicate|director|dump|exec|expunge|fetch|flags|fts|import|instance|index|force-resync|help|kick|log|mailbox|mount|move|penalty|purge|pw|quota|search|user|who)\.1$ -doc/man/(doveadm|doveconf|dovecot-lda|dovecot|dsync)\.1$ +doc/man/doveadm-(acl|altmove|auth|batch|deduplicate|director|dump|exec|expunge|fetch|flags|fts|import|instance|index|force-resync|help|kick|log|mailbox|mount|move|penalty|purge|pw|quota|search|sync|user|who)\.1$ +doc/man/(doveadm|doveconf|dovecot-lda|dovecot)\.1$ diff --git a/doc/man/Makefile.am b/doc/man/Makefile.am index a03e91b617..7825644a81 100644 --- a/doc/man/Makefile.am +++ b/doc/man/Makefile.am @@ -4,10 +4,12 @@ SUFFIXES = .1.in .1 dist_man1_MANS = \ deliver.1 \ + doveadm-backup.1 \ doveadm-config.1 \ doveadm-copy.1 \ doveadm-reload.1 \ - doveadm-stop.1 + doveadm-stop.1 \ + dsync.1 dist_man7_MANS = \ doveadm-search-query.7 @@ -41,12 +43,12 @@ nodist_man1_MANS = \ doveadm-pw.1 \ doveadm-quota.1 \ doveadm-search.1 \ + doveadm-sync.1 \ doveadm-user.1 \ doveadm-who.1 \ doveconf.1 \ dovecot.1 \ - dovecot-lda.1 \ - dsync.1 + dovecot-lda.1 man_includefiles = \ $(srcdir)/global-options-formatter.inc \ @@ -85,12 +87,12 @@ EXTRA_DIST = \ doveadm-pw.1.in \ doveadm-quota.1.in \ doveadm-search.1.in \ + doveadm-sync.1.in \ doveadm-user.1.in \ doveadm-who.1.in \ doveconf.1.in \ dovecot.1.in \ dovecot-lda.1.in \ - dsync.1.in \ sed.sh \ $(man_includefiles) diff --git a/doc/man/doveadm-backup.1 b/doc/man/doveadm-backup.1 new file mode 100644 index 0000000000..106321cd44 --- /dev/null +++ b/doc/man/doveadm-backup.1 @@ -0,0 +1 @@ +.so man1/doveadm-sync.1 \ No newline at end of file diff --git a/doc/man/doveadm-sync.1.in b/doc/man/doveadm-sync.1.in new file mode 100644 index 0000000000..c0f7e9942e --- /dev/null +++ b/doc/man/doveadm-sync.1.in @@ -0,0 +1,361 @@ +.\" Copyright (c) 2014 Dovecot authors, see the included COPYING file +.TH DOVEADM\-SYNC 1 "2014-10-05" "Dovecot v2.2" "Dovecot" +.SH NAME +doveadm\-sync \- Dovecot\(aqs two\-way mailbox synchronization utility +.br +doveadm\-backup \- Dovecot\(aqs one\-way mailbox synchronization utility +.\"------------------------------------------------------------------------ +.SH SYNOPSIS +.BR "doveadm sync" " [" \-Dv ] +[\fB\-u\fP \fIuser\fP|\fB\-A\fP] +[\fB\-S\fP \fIsocket_path\fP] +.RB [ \-1fPRU ] +[\fB\-l\fP \fIsecs\fP] +[\fB\-r\fP \fIrawlog_path\fP] +[\fB\-m\fP \fImailbox\fP] +[\fB\-g\fP \fImailbox_guid\fP] +[\fB\-n\fP \fInamespace\fP|\fB\-N\fP] +[\fB\-x\fP \fIexclude\fP] +[\fB\-s\fP \fIstate\fP] +\fB\-d\fP|\fIdestination\fP +.\"------------------------------------- +.PP +.BR "doveadm backup" " [" \-Dv ] +[\fB\-u\fP \fIuser\fP|\fB\-A\fP] +[\fB\-S\fP \fIsocket_path\fP] +.RB [ \-fPRU ] +[\fB\-l\fP \fIsecs\fP] +[\fB\-r\fP \fIrawlog_path\fP] +[\fB\-m\fP \fImailbox\fP] +[\fB\-g\fP \fImailbox_guid\fP] +[\fB\-n\fP \fInamespace\fP|\fB\-N\fP] +[\fB\-x\fP \fIexclude\fP] +[\fB\-s\fP \fIstate\fP] +\fB\-d\fP|\fIdestination\fP +.\"------------------------------------------------------------------------ +.SH DESCRIPTION +dsync is Dovecot\(aqs mailbox synchronization utility. +It can be used for several different use cases: Two\-way synchronization of +mailboxes, creating backups of mails, and convert mailboxes from/to +different mailbox formats. +All of these can be used within the same server or between different +servers (via +.BR ssh (1) +or tcp connections). +Remote mailboxes can be accessed also via IMAP protocol, which allows using +dsync for mailbox migration purposes. +.PP +You can run dsync in one of three modes: +.RS \(bu 4 +.\"------------------------------------- +.IP \(bu +.B doveadm backup +performs one\-way synchronization. +If there are any changes in the destination they will be deleted, so the +destination will look exactly like the source. +.\"------------------------------------- +.IP \(bu +.B doveadm sync +performs two\-way synchronization. +It merges all changes without losing anything. +Both the mailboxes will end up looking identical after the synchronization +is finished. +.\"------------------------------------- +.IP \(bu +.B doveadm sync \-1 +performs one\-way synchronization, but it merges the changes in destination +without deleting anything. +This doesn\(aqt currently work perfectly, so its use should be limited. +Its main purpose is that during mailbox migration you can run +.B doveadm backup +multiple times, then switch mails to be delivered to the new mailbox and +run +.B doveadm sync \-1 +once more to transfer any last new mails from the old mailbox. +.\"------------------------------------- +.RE +.PP +There are also three different synchronization algorithms: +.RS \(bu 4 +.\"------------------------------------- +.IP \(bu +Full synchronization (\-f parameter) scans through all the messages in all +the mailboxes. +This guarantees that everything will be synchronized, but it\(aqs +unnecessarily slow for incremental synchronization. +.\"------------------------------------- +.IP \(bu +Fast synchronization (default) first attempts to find mailboxes that have +changed, and synchronize only those. +This is done by checking the mailboxes\(aq metadata (NEXTUID and +HIGHESTMODSEQ). +Usually this works fine, especially with one\-way synchronization, but if +both sides do exactly the same number of changes, the metadata may end up +containing the same values even if the changes were different. +.\"------------------------------------- +.IP \(bu +Stateful synchronization (\-s parameter) is the most efficient way to +synchronize mailboxes. +It relies on having the earlier dsync run\(aqs state saved somewhere and +being passed to the next dsync run. +Based on this state dsync can send only the changes that happened after the +previous dsync run. +As long as the state or the mailboxes aren\(aqt corrupted this algorithm +should work perfectly. +The replicator process uses this internally to perform most of the +synchronization. +.\"------------------------------------- +.RE +.PP +The syncing is done as perfectly as possible: an IMAP or a POP3 client +shouldn\(aqt be able to notice any differences between the two mailboxes. +Two\-way syncing means that it\(aqs safe to do any kind of modifications in +both sides, and dsync will merge the changes without losing any changes +done on either side. +This is possible because dsync can access Dovecot\(aqs index logs that keep +track of changes. +It\(aqs of course possible to have conflicts during merging, these are +resolved in a safe way. +See the +.I dsync design +document for more information. +.PP +dsync uses the same configuration files as the rest of Dovecot (via +.BR doveconf (1) +binary). +The entire configuration can be changed by giving \-c parameter to another +configuration file, or using \-o parameter to override specific settings. +When executing a remote dsync program it works the same way: +it uses its own local configuration. +.PP +dsync can be run completely standalone. +It doesn\(aqt require any Dovecot server processes to be running, except +when using \-u parameter to do a +.I userdb +lookup from auth process. +.PP +dsync can sync either one or multiple users using the \-u or \-A +parameters. +For continuous replication you can use the Dovecot replicator process, +which automatically runs dsync whenever messages have changed. +.\"------------------------------------------------------------------------ +@INCLUDE:global-options@ +.\" --- command specific options --- "/. +.PP +Command specific +.IR options : +.TP +.B \-1 +Do one\-way synchronization instead of two\-way synchronization. +.\"------------------------------------- +@INCLUDE:option-A@ +.\"------------------------------------- +.TP +.B \-N +Synchronize all the available namespaces. +By default only namespaces that don\(aqt have explicit location setting +are synchronized. +.\"------------------------------------- +.TP +.B \-P +Run a +.BR doveadm\-purge (1) +for the destination (remote) storage after synchronization. +.\"------------------------------------- +@INCLUDE:option-S-socket@ +.\"------------------------------------- +.TP +.B \-U +This is used internally by replicator to have dsync notify it when the +synchronization is finished. +.\"------------------------------------- +.TP +.B \-d +Use the default destination, which is looked up from the +.I mail_replica userdb +extra field. +.\"------------------------------------- +.TP +.BI \-g \ mailbox_guid +Same as \-m, but find the mailbox to be synchronized by its GUID instead +of by name. +.\"------------------------------------- +.TP +.BI \-l \ secs +Lock the dsync for this user. +Wait for maximum +.I secs +before giving up. +This parameter should be used to avoid broken synchronization if it\(aqs +possible that dsync is being run concurrently for the same user. +.\"------------------------------------- +.TP +.BI \-n \ namespace +Synchronize only the specified namespace. +This parameter can be used multiple times. +.\"------------------------------------- +.TP +.BI \-r \ rawlog_path +Running dsync remotely, write the remote input/output traffic to the +specified log file. +.\"------------------------------------- +.TP +.BI \-s \ previous_state +Use stateful synchronization. +If the previous state is unknown, use an empty string. +The new state is always printed to standard output. +.\"------------------------------------- +@INCLUDE:option-u-user@ +.\"------------------------------------- +.TP +.BI \-x \ mailbox_mask +Exclude the specified mailbox name/mask. +The mask may contain \(dq\fB?\fP\(dq and \(dq\fB*\fP\(dq wildcards. +This parameter can be used multiple times. +.\"------------------------------------------------------------------------ +.SH ARGUMENTS +.TP +.I destination +This argument specifies the synchronized destination. +It can be one of: +.RS +.TP +location +Same as +.I mail_location +setting, e.g. maildir:\(ti/Maildir +.TP +.BI remote: login@host +Uses +.I dsync_remote_cmd +setting to connect to the remote host (usually via ssh) +.TP +.I remoteprefix:login@host +This is the same as remote, except \(dquser@domain\(rsn\(dq is sent before +dsync protocol starts. +This allows implementing a trusted wrapper script that runs doveadm +dsync\-server by reading the username from the first line. +.TP +.BI tcp: host[:port] +Connects to remote doveadm server via TCP. +The default port is specified by +.IR doveadm_port " setting." +.TP +.BI tcps: host[:port] +This is the same as tcp, but with SSL. +.RE +.\"------------------------------------------------------------------------ +.SH "EXIT STATUS" +.B dsync +will exit with one of the following values: +.TP 4 +.B 0 +Synchronization was done perfectly. +.TP +.B 2 +Synchronization was done without errors, but some changes couldn\(aqt be done, +so the mailboxes aren\(aqt perfectly synchronized. Running dsync again +usually fixes this. Typically this occurs for message modification +sequences with newly created mailboxes. It can also occur if one of the +mailboxes change during the syncing. +.TP +.B 1, >2 +Synchronization failed. +.\"------------------------------------------------------------------------ +.SH EXAMPLE +.SS SYNCHRONIZATION +Synchronize mailboxes with a remote server. +Any errors are written to stderr. +.PP +.RS +.nf +.B doveadm sync \-u username@example.com remote:server\-replica.example.com +.fi +.RE +.PP +If you need more complex parameters to ssh, you can use e.g.: +.PP +.RS +.nf +.B doveadm sync \-u username@example.com ssh \-i id_dsa.dovecot \(rs +.B mailuser@example.com doveadm dsync\-server \-u username@example.com +.fi +.RE +.\"------------------------------------------------------------------------ +.SS CONVERTING +Assuming that the +.I mail_location +setting in +.I @pkgsysconfdir@/conf.d/10\-mail.conf +is set to: +.BR "mail_location = mdbox:\(ti/mdbox" , +a logged in system user may convert her/his mails from its Maildir in +her/his home directory to the mdbox mailbox format. +The user has to execute the command: +.PP +.RS +.nf +.B doveadm sync maildir:\(ti/Maildir +.fi +.RE +.PP +If you want to do this without any downtime, you can do the conversion one +user at a time. +Initially: +.RS 4 +.IP \(bu 4 +Configuration uses +.B mail_location = maildir:\(ti/Maildir +.IP \(bu +Set up the possibility of doing per\-user mail location using +.I userdb +extra fields. +.RE +.PP +Then for each user: +.RS 4 +.IP 1. 4 +Run +.I doveadm sync +once to do the initial conversion. +.IP 2. +Run +.I doveadm sync +again, because the initial conversion could have taken a while and new +changes could have occurred during it. +This second time only applies changes, so it should be fast. +.IP 3. +Update mail extra field in userdb to +.BR mdbox:\(ti/mdbox . +If you\(aqre using auth cache, you need to flush it, e.g. +.BR "doveadm auth cache flush" . +.IP 4. +Wait for a few seconds and then kill (doveadm kick) the user\(aqs all +existing imap and pop3 sessions (that are still using maildir). +.IP 5. +Run +.I doveadm sync +once more to apply final changes that were possibly done. +After this there should be no changes to Maildir, because the user\(aqs +mail location has been changed and all existing processes using it have +been killed. +.RE +.PP +Once all users have been converted, you can set the default +.I mail_location +to mdbox and remove the per\-user mail locations from +.IR userdb . +.\"------------------------------------------------------------------------ +@INCLUDE:reporting-bugs@ +.\"------------------------------------------------------------------------ +.SH SEE ALSO +.BR doveadm (1), +.BR doveadm\-auth (1), +.BR doveadm\-kick (1), +.BR doveadm\-purge (1), +.BR doveconf (1) +.\"------------------------------------- +.PP +Additional resources: +.IP "dsync design" +http://wiki2.dovecot.org/Design/Dsync \ No newline at end of file diff --git a/doc/man/dsync.1 b/doc/man/dsync.1 new file mode 100644 index 0000000000..106321cd44 --- /dev/null +++ b/doc/man/dsync.1 @@ -0,0 +1 @@ +.so man1/doveadm-sync.1 \ No newline at end of file diff --git a/doc/man/dsync.1.in b/doc/man/dsync.1.in deleted file mode 100644 index e1e4c23528..0000000000 --- a/doc/man/dsync.1.in +++ /dev/null @@ -1,302 +0,0 @@ -.\" Copyright (c) 2010 Dovecot authors, see the included COPYING file -.TH DSYNC 1 "2011-01-16" "Dovecot v2.2" "Dovecot" -.SH NAME -dsync \- Dovecot\(aqs mailbox synchronization utility -.\"------------------------------------------------------------------------ -.SH SYNOPSIS -.B dsync -.RI [ options ] -.BI mirror\ location2 -.\"------------------------------------- -.br -.B dsync -.RI [ options ] -.BI backup\ location2 -.\"------------------------------------------------------------------------ -.SH DESCRIPTION -.B dsync -is Dovecot\(aqs mailbox synchronization utility. -It can be used for several different use cases: -Two\-way synchronization of mailboxes in different servers (via -.BR ssh (1)), -creating backups of mails to a remote server, and convert mailboxes from/to -different mailbox formats. -.PP -The syncing is done as perfectly as possible: an IMAP or a POP3 client -shouldn\(aqt be able to notice any differences between the two mailboxes. -Two\-way syncing means that it\(aqs safe to do any kind of modifications in -both sides, and -.B dsync -will merge the changes without losing any changes done on either side. This -is possible because -.B dsync -can access Dovecot\(aqs index logs that keep track of changes. It\(aqs of -course possible to have conflicts during merging, these are resolved in a -safe way. See the -.B dsync -design document for more information. -.PP -.B dsync -uses the same configuration files as the rest of Dovecot (via doveconf -binary). The entire configuration can be changed by giving \-c parameter to -another configuration file, or using \-o parameter to override specific -settings. When executing a remote -.B dsync -program it works the same way: it uses its own local configuration. -.PP -.B dsync -can be run completely standalone. It doesn\(aqt require any Dovecot server -processes to be running, except when using \-u parameter to do a userdb -lookup from auth process. -.PP -.B dsync -can currently sync only one user at a time. If you want to -.B dsync -all users, you\(aqll need to get a list of all users and execute -.B dsync -separately for each one. - -Any errors are written to stderr. -.\"------------------------------------------------------------------------ -.SH OPTIONS -.B dsync -recognizes the following command line options: -.TP -.BI \-c \ config\-file -read configuration from the given -.IR config\-file . -By default -.I @pkgsysconfdir@/dovecot.conf -will be used. -.\"--------------------------------- -.BI \-C\ alt_char -Specifies an alternative mailbox name character. -If source and destination mailbox formats are different, it\(aqs possible -that on one side there exists a mailbox name that isn\(aqt valid for the -other side. -These invalid mailbox names are fixed by replacing such invalid -characters with the given -.IR alt_char . -The default is -.RB \(aq _ \(aq. -.\"--------------------------------- -.TP -.B \-D -Activates debug messages and makes -.B dsync -more verbose. -.\"--------------------------------- -.TP -.B \-f -Makes -.B dsync -run in \(dqfull sync\(dq mode rather than \(dqfast sync\(dq mode. -In fast sync mode -.B dsync -might skip syncing a mailbox, if both locations had modified it equally -many times (i.e. highest\-modseqs were equal), but with different changes. -.\"--------------------------------- -.TP -.BI \-m\ mailbox -Specifies the -.I mailbox -that should be synchronized or from which mails should be converted. -The default is to synchronize all respectively convert from all mailboxes. -.\"--------------------------------- -.TP -.BI \-o\ setting = value -Overrides the configuration -.I setting -from -.I @pkgsysconfdir@/dovecot.conf -and from the userdb with the given -.IR value . -In order to override multiple settings, the -.B \-o -option may be specified multiple times. -.\"--------------------------------- -.TP -.B \-R -Reverse backup direction, so mails in -.I location2 -are backed up to default mail location. -.\"--------------------------------- -.TP -.BI \-u\ user -Specifies that the userdb lookup for the given -.I user -should be done and used to set up the environment (uid, gid, home, etc.). -By default the system user\(aqs current environment will be used. -.\"--------------------------------- -.TP -.B \-v -Makes -.B dsync -more verbose. -.\"------------------------------------------------------------------------ -.SH ARGUMENTS -.TP -.I location2 -The first mail location is based on configuration -.RI ( mail_location " or " userdb " settings). -It\(aqs also possible to override it by giving -.BI \-o\ mail_location= mail_location -setting. -This parameter defines the other mail location that is used. -.sp -If the location is on local filesystem, you can use a regular -mail_location, such as maildir:/backup/user/Maildir -.sp -If the location is on a remote server, -.B dsync -can ssh to it by giving -.I host -or -.I user@host -as the parameter. -If user is specified, it\(aqs given as -.B \-u -parameter to -.BR dsync , -not to ssh. -The ssh username is always the default. -.sp -The final way to specify a location is to give a full command line or a -path to a script that executes the -.BR dsync . -For example: -.sp -.nf -ssh mailuser@host dsync \-u user -.fi -.\"------------------------------------------------------------------------ -.SH COMMANDS -.B dsync -provides the following commands: -.\"------------------------------------------------------------------------ -.SS mirror -Does a two\-way synchronization between two mail locations. -Changes in both locations are synchronized to the other one, without losing -any changes made by either of them. -Any potential UID conflicts are resolved by giving them new UIDs. -.\"------------------------------------------------------------------------ -.SS backup -Backup mails from default mail location to -.I location2 -(or vice versa, if -.B \-R -parameter is given). -No changes are ever done to the source location. -Any changes done in destination are discarded. -.\"------------------------------------------------------------------------ -.SH "EXIT STATUS" -.B dsync -will exit with one of the following values: -.TP 4 -.B 0 -Synchronization was done perfectly. -.TP -.B 2 -Synchronization was done without errors, but some changes couldn\(aqt be done, -so the mailboxes aren\(aqt perfectly synchronized. Running dsync again -usually fixes this. Typically this occurs for message modification -sequences with newly created mailboxes. It can also occur if one of the -mailboxes change during the syncing. -.TP -.B 1, >2 -Synchronization failed. -.\"------------------------------------------------------------------------ -.SH EXAMPLE -.SS MIRRORING -Mirror mailboxes to a remote server. -Any errors are written to stderr. -.PP -.RS -.nf -.B dsync -u username mirror username@example.com -.fi -.RE -.PP -If you need more complex parameters to ssh, you can use e.g.: -.PP -.RS -.nf -.B dsync \-u username mirror ssh \-i id_dsa.dovecot mailuser@example.com dsync \-u username -.fi -.RE -.\"------------------------------------------------------------------------ -.SS CONVERTING -Assuming that the -.I mail_location -setting in -.I @pkgsysconfdir@/conf.d/10\-mail.conf -is set to: -.BR "mail_location = mdbox:~/mdbox" , -a logged in system user may convert her/his mails from its Maildir in -her/his home directory to the mdbox mailbox format. -The user has to execute the command: -.PP -.RS -.nf -.B dsync mirror maildir:~/Maildir -.fi -.RE -.PP -If you want to do this without any downtime, you can do the conversion one -user at a time. -Initially: -.RS 4 -.IP \(bu 4 -Configuration uses -.B mail_location = maildir:~/Maildir -.IP \(bu -Set up the possibility of doing per\-user mail location using -.I userdb -extra fields. -.RE -.PP -Then for each user: -.RS 4 -.IP 1. 4 -Run -.I dsync mirror -once to do the initial conversion. -.IP 2. -Run -.I dsync mirror -again, because the initial conversion could have taken a while and new -changes could have occurred during it. -This second time only applies changes, so it should be fast. -.IP 3. -Update mail extra field in userdb to -.BR mdbox:~/mdbox . -If you\(aqre using auth cache, you need to flush it. -.IP 4. -Wait for a few seconds and then kill the user\(aqs all existing imap and -pop3 sessions (that are still using maildir). -.IP 5. -Run -.I dsync mirror -once more to apply final changes that were possibly done. -After this there should be no changes to Maildir, because the user\(aqs -mail location has been changed and all existing processes using it have -been killed. -.RE -.PP -Once all users have been converted, you can set the default -.I mail_location -to mdbox and remove the per\-user mail locations from -.IR userdb . -.\"------------------------------------------------------------------------ -@INCLUDE:reporting-bugs@ -.\"------------------------------------------------------------------------ -.SH SEE ALSO -.BR doveadm (1), -.BR doveadm\-kick (1), -.BR doveconf (1), -.BR dovecot (1) -.\"------------------------------------- -.PP -Additional resources: -.IP "dsync design" -http://wiki2.dovecot.org/Design/Dsync \ No newline at end of file