* text=auto eol=lf
+
+# The rsync-web/ subdirectory holds the project website source content
+# (mirrors what gets pushed to https://rsync.samba.org). Exclude it from
+# `git archive` output so the release source tarball produced by
+# packaging/release.py step_7_tarball does not bloat with HTML the
+# tarball doesn't need.
+/rsync-web/ export-ignore
--- /dev/null
+/.xvpics
+/doxygen/
+/tech_report/IMG_PARAMS.dir
+/tech_report/IMG_PARAMS.pag
+/upload
+/netware
+/pre-change
+/index.html-*
+/rsync-and-debian/rsync-and-debian.html
+/rsync-and-debian/rsync-and-debian.ps
+/badge.svg
--- /dev/null
+ GNU GENERAL PUBLIC LICENSE
+ Version 3, 29 June 2007
+
+ Copyright (C) 2007 Free Software Foundation, Inc. <<a href="https://fsf.org/">https://fsf.org/</a>>
+ Everyone is permitted to copy and distribute verbatim copies
+ of this license document, but changing it is not allowed.
+
+ Preamble
+
+ The GNU General Public License is a free, copyleft license for
+software and other kinds of works.
+
+ The licenses for most software and other practical works are designed
+to take away your freedom to share and change the works. By contrast,
+the GNU General Public License is intended to guarantee your freedom to
+share and change all versions of a program--to make sure it remains free
+software for all its users. We, the Free Software Foundation, use the
+GNU General Public License for most of our software; it applies also to
+any other work released this way by its authors. You can apply it to
+your programs, too.
+
+ When we speak of free software, we are referring to freedom, not
+price. Our General Public Licenses are designed to make sure that you
+have the freedom to distribute copies of free software (and charge for
+them if you wish), that you receive source code or can get it if you
+want it, that you can change the software or use pieces of it in new
+free programs, and that you know you can do these things.
+
+ To protect your rights, we need to prevent others from denying you
+these rights or asking you to surrender the rights. Therefore, you have
+certain responsibilities if you distribute copies of the software, or if
+you modify it: responsibilities to respect the freedom of others.
+
+ For example, if you distribute copies of such a program, whether
+gratis or for a fee, you must pass on to the recipients the same
+freedoms that you received. You must make sure that they, too, receive
+or can get the source code. And you must show them these terms so they
+know their rights.
+
+ Developers that use the GNU GPL protect your rights with two steps:
+(1) assert copyright on the software, and (2) offer you this License
+giving you legal permission to copy, distribute and/or modify it.
+
+ For the developers' and authors' protection, the GPL clearly explains
+that there is no warranty for this free software. For both users' and
+authors' sake, the GPL requires that modified versions be marked as
+changed, so that their problems will not be attributed erroneously to
+authors of previous versions.
+
+ Some devices are designed to deny users access to install or run
+modified versions of the software inside them, although the manufacturer
+can do so. This is fundamentally incompatible with the aim of
+protecting users' freedom to change the software. The systematic
+pattern of such abuse occurs in the area of products for individuals to
+use, which is precisely where it is most unacceptable. Therefore, we
+have designed this version of the GPL to prohibit the practice for those
+products. If such problems arise substantially in other domains, we
+stand ready to extend this provision to those domains in future versions
+of the GPL, as needed to protect the freedom of users.
+
+ Finally, every program is threatened constantly by software patents.
+States should not allow patents to restrict development and use of
+software on general-purpose computers, but in those that do, we wish to
+avoid the special danger that patents applied to a free program could
+make it effectively proprietary. To prevent this, the GPL assures that
+patents cannot be used to render the program non-free.
+
+ The precise terms and conditions for copying, distribution and
+modification follow.
+
+ TERMS AND CONDITIONS
+
+ 0. Definitions.
+
+ "This License" refers to version 3 of the GNU General Public License.
+
+ "Copyright" also means copyright-like laws that apply to other kinds of
+works, such as semiconductor masks.
+
+ "The Program" refers to any copyrightable work licensed under this
+License. Each licensee is addressed as "you". "Licensees" and
+"recipients" may be individuals or organizations.
+
+ To "modify" a work means to copy from or adapt all or part of the work
+in a fashion requiring copyright permission, other than the making of an
+exact copy. The resulting work is called a "modified version" of the
+earlier work or a work "based on" the earlier work.
+
+ A "covered work" means either the unmodified Program or a work based
+on the Program.
+
+ To "propagate" a work means to do anything with it that, without
+permission, would make you directly or secondarily liable for
+infringement under applicable copyright law, except executing it on a
+computer or modifying a private copy. Propagation includes copying,
+distribution (with or without modification), making available to the
+public, and in some countries other activities as well.
+
+ To "convey" a work means any kind of propagation that enables other
+parties to make or receive copies. Mere interaction with a user through
+a computer network, with no transfer of a copy, is not conveying.
+
+ An interactive user interface displays "Appropriate Legal Notices"
+to the extent that it includes a convenient and prominently visible
+feature that (1) displays an appropriate copyright notice, and (2)
+tells the user that there is no warranty for the work (except to the
+extent that warranties are provided), that licensees may convey the
+work under this License, and how to view a copy of this License. If
+the interface presents a list of user commands or options, such as a
+menu, a prominent item in the list meets this criterion.
+
+ 1. Source Code.
+
+ The "source code" for a work means the preferred form of the work
+for making modifications to it. "Object code" means any non-source
+form of a work.
+
+ A "Standard Interface" means an interface that either is an official
+standard defined by a recognized standards body, or, in the case of
+interfaces specified for a particular programming language, one that
+is widely used among developers working in that language.
+
+ The "System Libraries" of an executable work include anything, other
+than the work as a whole, that (a) is included in the normal form of
+packaging a Major Component, but which is not part of that Major
+Component, and (b) serves only to enable use of the work with that
+Major Component, or to implement a Standard Interface for which an
+implementation is available to the public in source code form. A
+"Major Component", in this context, means a major essential component
+(kernel, window system, and so on) of the specific operating system
+(if any) on which the executable work runs, or a compiler used to
+produce the work, or an object code interpreter used to run it.
+
+ The "Corresponding Source" for a work in object code form means all
+the source code needed to generate, install, and (for an executable
+work) run the object code and to modify the work, including scripts to
+control those activities. However, it does not include the work's
+System Libraries, or general-purpose tools or generally available free
+programs which are used unmodified in performing those activities but
+which are not part of the work. For example, Corresponding Source
+includes interface definition files associated with source files for
+the work, and the source code for shared libraries and dynamically
+linked subprograms that the work is specifically designed to require,
+such as by intimate data communication or control flow between those
+subprograms and other parts of the work.
+
+ The Corresponding Source need not include anything that users
+can regenerate automatically from other parts of the Corresponding
+Source.
+
+ The Corresponding Source for a work in source code form is that
+same work.
+
+ 2. Basic Permissions.
+
+ All rights granted under this License are granted for the term of
+copyright on the Program, and are irrevocable provided the stated
+conditions are met. This License explicitly affirms your unlimited
+permission to run the unmodified Program. The output from running a
+covered work is covered by this License only if the output, given its
+content, constitutes a covered work. This License acknowledges your
+rights of fair use or other equivalent, as provided by copyright law.
+
+ You may make, run and propagate covered works that you do not
+convey, without conditions so long as your license otherwise remains
+in force. You may convey covered works to others for the sole purpose
+of having them make modifications exclusively for you, or provide you
+with facilities for running those works, provided that you comply with
+the terms of this License in conveying all material for which you do
+not control copyright. Those thus making or running the covered works
+for you must do so exclusively on your behalf, under your direction
+and control, on terms that prohibit them from making any copies of
+your copyrighted material outside their relationship with you.
+
+ Conveying under any other circumstances is permitted solely under
+the conditions stated below. Sublicensing is not allowed; section 10
+makes it unnecessary.
+
+ 3. Protecting Users' Legal Rights From Anti-Circumvention Law.
+
+ No covered work shall be deemed part of an effective technological
+measure under any applicable law fulfilling obligations under article
+11 of the WIPO copyright treaty adopted on 20 December 1996, or
+similar laws prohibiting or restricting circumvention of such
+measures.
+
+ When you convey a covered work, you waive any legal power to forbid
+circumvention of technological measures to the extent such circumvention
+is effected by exercising rights under this License with respect to
+the covered work, and you disclaim any intention to limit operation or
+modification of the work as a means of enforcing, against the work's
+users, your or third parties' legal rights to forbid circumvention of
+technological measures.
+
+ 4. Conveying Verbatim Copies.
+
+ You may convey verbatim copies of the Program's source code as you
+receive it, in any medium, provided that you conspicuously and
+appropriately publish on each copy an appropriate copyright notice;
+keep intact all notices stating that this License and any
+non-permissive terms added in accord with section 7 apply to the code;
+keep intact all notices of the absence of any warranty; and give all
+recipients a copy of this License along with the Program.
+
+ You may charge any price or no price for each copy that you convey,
+and you may offer support or warranty protection for a fee.
+
+ 5. Conveying Modified Source Versions.
+
+ You may convey a work based on the Program, or the modifications to
+produce it from the Program, in the form of source code under the
+terms of section 4, provided that you also meet all of these conditions:
+
+ a) The work must carry prominent notices stating that you modified
+ it, and giving a relevant date.
+
+ b) The work must carry prominent notices stating that it is
+ released under this License and any conditions added under section
+ 7. This requirement modifies the requirement in section 4 to
+ "keep intact all notices".
+
+ c) You must license the entire work, as a whole, under this
+ License to anyone who comes into possession of a copy. This
+ License will therefore apply, along with any applicable section 7
+ additional terms, to the whole of the work, and all its parts,
+ regardless of how they are packaged. This License gives no
+ permission to license the work in any other way, but it does not
+ invalidate such permission if you have separately received it.
+
+ d) If the work has interactive user interfaces, each must display
+ Appropriate Legal Notices; however, if the Program has interactive
+ interfaces that do not display Appropriate Legal Notices, your
+ work need not make them do so.
+
+ A compilation of a covered work with other separate and independent
+works, which are not by their nature extensions of the covered work,
+and which are not combined with it such as to form a larger program,
+in or on a volume of a storage or distribution medium, is called an
+"aggregate" if the compilation and its resulting copyright are not
+used to limit the access or legal rights of the compilation's users
+beyond what the individual works permit. Inclusion of a covered work
+in an aggregate does not cause this License to apply to the other
+parts of the aggregate.
+
+ 6. Conveying Non-Source Forms.
+
+ You may convey a covered work in object code form under the terms
+of sections 4 and 5, provided that you also convey the
+machine-readable Corresponding Source under the terms of this License,
+in one of these ways:
+
+ a) Convey the object code in, or embodied in, a physical product
+ (including a physical distribution medium), accompanied by the
+ Corresponding Source fixed on a durable physical medium
+ customarily used for software interchange.
+
+ b) Convey the object code in, or embodied in, a physical product
+ (including a physical distribution medium), accompanied by a
+ written offer, valid for at least three years and valid for as
+ long as you offer spare parts or customer support for that product
+ model, to give anyone who possesses the object code either (1) a
+ copy of the Corresponding Source for all the software in the
+ product that is covered by this License, on a durable physical
+ medium customarily used for software interchange, for a price no
+ more than your reasonable cost of physically performing this
+ conveying of source, or (2) access to copy the
+ Corresponding Source from a network server at no charge.
+
+ c) Convey individual copies of the object code with a copy of the
+ written offer to provide the Corresponding Source. This
+ alternative is allowed only occasionally and noncommercially, and
+ only if you received the object code with such an offer, in accord
+ with subsection 6b.
+
+ d) Convey the object code by offering access from a designated
+ place (gratis or for a charge), and offer equivalent access to the
+ Corresponding Source in the same way through the same place at no
+ further charge. You need not require recipients to copy the
+ Corresponding Source along with the object code. If the place to
+ copy the object code is a network server, the Corresponding Source
+ may be on a different server (operated by you or a third party)
+ that supports equivalent copying facilities, provided you maintain
+ clear directions next to the object code saying where to find the
+ Corresponding Source. Regardless of what server hosts the
+ Corresponding Source, you remain obligated to ensure that it is
+ available for as long as needed to satisfy these requirements.
+
+ e) Convey the object code using peer-to-peer transmission, provided
+ you inform other peers where the object code and Corresponding
+ Source of the work are being offered to the general public at no
+ charge under subsection 6d.
+
+ A separable portion of the object code, whose source code is excluded
+from the Corresponding Source as a System Library, need not be
+included in conveying the object code work.
+
+ A "User Product" is either (1) a "consumer product", which means any
+tangible personal property which is normally used for personal, family,
+or household purposes, or (2) anything designed or sold for incorporation
+into a dwelling. In determining whether a product is a consumer product,
+doubtful cases shall be resolved in favor of coverage. For a particular
+product received by a particular user, "normally used" refers to a
+typical or common use of that class of product, regardless of the status
+of the particular user or of the way in which the particular user
+actually uses, or expects or is expected to use, the product. A product
+is a consumer product regardless of whether the product has substantial
+commercial, industrial or non-consumer uses, unless such uses represent
+the only significant mode of use of the product.
+
+ "Installation Information" for a User Product means any methods,
+procedures, authorization keys, or other information required to install
+and execute modified versions of a covered work in that User Product from
+a modified version of its Corresponding Source. The information must
+suffice to ensure that the continued functioning of the modified object
+code is in no case prevented or interfered with solely because
+modification has been made.
+
+ If you convey an object code work under this section in, or with, or
+specifically for use in, a User Product, and the conveying occurs as
+part of a transaction in which the right of possession and use of the
+User Product is transferred to the recipient in perpetuity or for a
+fixed term (regardless of how the transaction is characterized), the
+Corresponding Source conveyed under this section must be accompanied
+by the Installation Information. But this requirement does not apply
+if neither you nor any third party retains the ability to install
+modified object code on the User Product (for example, the work has
+been installed in ROM).
+
+ The requirement to provide Installation Information does not include a
+requirement to continue to provide support service, warranty, or updates
+for a work that has been modified or installed by the recipient, or for
+the User Product in which it has been modified or installed. Access to a
+network may be denied when the modification itself materially and
+adversely affects the operation of the network or violates the rules and
+protocols for communication across the network.
+
+ Corresponding Source conveyed, and Installation Information provided,
+in accord with this section must be in a format that is publicly
+documented (and with an implementation available to the public in
+source code form), and must require no special password or key for
+unpacking, reading or copying.
+
+ 7. Additional Terms.
+
+ "Additional permissions" are terms that supplement the terms of this
+License by making exceptions from one or more of its conditions.
+Additional permissions that are applicable to the entire Program shall
+be treated as though they were included in this License, to the extent
+that they are valid under applicable law. If additional permissions
+apply only to part of the Program, that part may be used separately
+under those permissions, but the entire Program remains governed by
+this License without regard to the additional permissions.
+
+ When you convey a copy of a covered work, you may at your option
+remove any additional permissions from that copy, or from any part of
+it. (Additional permissions may be written to require their own
+removal in certain cases when you modify the work.) You may place
+additional permissions on material, added by you to a covered work,
+for which you have or can give appropriate copyright permission.
+
+ Notwithstanding any other provision of this License, for material you
+add to a covered work, you may (if authorized by the copyright holders of
+that material) supplement the terms of this License with terms:
+
+ a) Disclaiming warranty or limiting liability differently from the
+ terms of sections 15 and 16 of this License; or
+
+ b) Requiring preservation of specified reasonable legal notices or
+ author attributions in that material or in the Appropriate Legal
+ Notices displayed by works containing it; or
+
+ c) Prohibiting misrepresentation of the origin of that material, or
+ requiring that modified versions of such material be marked in
+ reasonable ways as different from the original version; or
+
+ d) Limiting the use for publicity purposes of names of licensors or
+ authors of the material; or
+
+ e) Declining to grant rights under trademark law for use of some
+ trade names, trademarks, or service marks; or
+
+ f) Requiring indemnification of licensors and authors of that
+ material by anyone who conveys the material (or modified versions of
+ it) with contractual assumptions of liability to the recipient, for
+ any liability that these contractual assumptions directly impose on
+ those licensors and authors.
+
+ All other non-permissive additional terms are considered "further
+restrictions" within the meaning of section 10. If the Program as you
+received it, or any part of it, contains a notice stating that it is
+governed by this License along with a term that is a further
+restriction, you may remove that term. If a license document contains
+a further restriction but permits relicensing or conveying under this
+License, you may add to a covered work material governed by the terms
+of that license document, provided that the further restriction does
+not survive such relicensing or conveying.
+
+ If you add terms to a covered work in accord with this section, you
+must place, in the relevant source files, a statement of the
+additional terms that apply to those files, or a notice indicating
+where to find the applicable terms.
+
+ Additional terms, permissive or non-permissive, may be stated in the
+form of a separately written license, or stated as exceptions;
+the above requirements apply either way.
+
+ 8. Termination.
+
+ You may not propagate or modify a covered work except as expressly
+provided under this License. Any attempt otherwise to propagate or
+modify it is void, and will automatically terminate your rights under
+this License (including any patent licenses granted under the third
+paragraph of section 11).
+
+ However, if you cease all violation of this License, then your
+license from a particular copyright holder is reinstated (a)
+provisionally, unless and until the copyright holder explicitly and
+finally terminates your license, and (b) permanently, if the copyright
+holder fails to notify you of the violation by some reasonable means
+prior to 60 days after the cessation.
+
+ Moreover, your license from a particular copyright holder is
+reinstated permanently if the copyright holder notifies you of the
+violation by some reasonable means, this is the first time you have
+received notice of violation of this License (for any work) from that
+copyright holder, and you cure the violation prior to 30 days after
+your receipt of the notice.
+
+ Termination of your rights under this section does not terminate the
+licenses of parties who have received copies or rights from you under
+this License. If your rights have been terminated and not permanently
+reinstated, you do not qualify to receive new licenses for the same
+material under section 10.
+
+ 9. Acceptance Not Required for Having Copies.
+
+ You are not required to accept this License in order to receive or
+run a copy of the Program. Ancillary propagation of a covered work
+occurring solely as a consequence of using peer-to-peer transmission
+to receive a copy likewise does not require acceptance. However,
+nothing other than this License grants you permission to propagate or
+modify any covered work. These actions infringe copyright if you do
+not accept this License. Therefore, by modifying or propagating a
+covered work, you indicate your acceptance of this License to do so.
+
+ 10. Automatic Licensing of Downstream Recipients.
+
+ Each time you convey a covered work, the recipient automatically
+receives a license from the original licensors, to run, modify and
+propagate that work, subject to this License. You are not responsible
+for enforcing compliance by third parties with this License.
+
+ An "entity transaction" is a transaction transferring control of an
+organization, or substantially all assets of one, or subdividing an
+organization, or merging organizations. If propagation of a covered
+work results from an entity transaction, each party to that
+transaction who receives a copy of the work also receives whatever
+licenses to the work the party's predecessor in interest had or could
+give under the previous paragraph, plus a right to possession of the
+Corresponding Source of the work from the predecessor in interest, if
+the predecessor has it or can get it with reasonable efforts.
+
+ You may not impose any further restrictions on the exercise of the
+rights granted or affirmed under this License. For example, you may
+not impose a license fee, royalty, or other charge for exercise of
+rights granted under this License, and you may not initiate litigation
+(including a cross-claim or counterclaim in a lawsuit) alleging that
+any patent claim is infringed by making, using, selling, offering for
+sale, or importing the Program or any portion of it.
+
+ 11. Patents.
+
+ A "contributor" is a copyright holder who authorizes use under this
+License of the Program or a work on which the Program is based. The
+work thus licensed is called the contributor's "contributor version".
+
+ A contributor's "essential patent claims" are all patent claims
+owned or controlled by the contributor, whether already acquired or
+hereafter acquired, that would be infringed by some manner, permitted
+by this License, of making, using, or selling its contributor version,
+but do not include claims that would be infringed only as a
+consequence of further modification of the contributor version. For
+purposes of this definition, "control" includes the right to grant
+patent sublicenses in a manner consistent with the requirements of
+this License.
+
+ Each contributor grants you a non-exclusive, worldwide, royalty-free
+patent license under the contributor's essential patent claims, to
+make, use, sell, offer for sale, import and otherwise run, modify and
+propagate the contents of its contributor version.
+
+ In the following three paragraphs, a "patent license" is any express
+agreement or commitment, however denominated, not to enforce a patent
+(such as an express permission to practice a patent or covenant not to
+sue for patent infringement). To "grant" such a patent license to a
+party means to make such an agreement or commitment not to enforce a
+patent against the party.
+
+ If you convey a covered work, knowingly relying on a patent license,
+and the Corresponding Source of the work is not available for anyone
+to copy, free of charge and under the terms of this License, through a
+publicly available network server or other readily accessible means,
+then you must either (1) cause the Corresponding Source to be so
+available, or (2) arrange to deprive yourself of the benefit of the
+patent license for this particular work, or (3) arrange, in a manner
+consistent with the requirements of this License, to extend the patent
+license to downstream recipients. "Knowingly relying" means you have
+actual knowledge that, but for the patent license, your conveying the
+covered work in a country, or your recipient's use of the covered work
+in a country, would infringe one or more identifiable patents in that
+country that you have reason to believe are valid.
+
+ If, pursuant to or in connection with a single transaction or
+arrangement, you convey, or propagate by procuring conveyance of, a
+covered work, and grant a patent license to some of the parties
+receiving the covered work authorizing them to use, propagate, modify
+or convey a specific copy of the covered work, then the patent license
+you grant is automatically extended to all recipients of the covered
+work and works based on it.
+
+ A patent license is "discriminatory" if it does not include within
+the scope of its coverage, prohibits the exercise of, or is
+conditioned on the non-exercise of one or more of the rights that are
+specifically granted under this License. You may not convey a covered
+work if you are a party to an arrangement with a third party that is
+in the business of distributing software, under which you make payment
+to the third party based on the extent of your activity of conveying
+the work, and under which the third party grants, to any of the
+parties who would receive the covered work from you, a discriminatory
+patent license (a) in connection with copies of the covered work
+conveyed by you (or copies made from those copies), or (b) primarily
+for and in connection with specific products or compilations that
+contain the covered work, unless you entered into that arrangement,
+or that patent license was granted, prior to 28 March 2007.
+
+ Nothing in this License shall be construed as excluding or limiting
+any implied license or other defenses to infringement that may
+otherwise be available to you under applicable patent law.
+
+ 12. No Surrender of Others' Freedom.
+
+ If conditions are imposed on you (whether by court order, agreement or
+otherwise) that contradict the conditions of this License, they do not
+excuse you from the conditions of this License. If you cannot convey a
+covered work so as to satisfy simultaneously your obligations under this
+License and any other pertinent obligations, then as a consequence you may
+not convey it at all. For example, if you agree to terms that obligate you
+to collect a royalty for further conveying from those to whom you convey
+the Program, the only way you could satisfy both those terms and this
+License would be to refrain entirely from conveying the Program.
+
+ 13. Use with the GNU Affero General Public License.
+
+ Notwithstanding any other provision of this License, you have
+permission to link or combine any covered work with a work licensed
+under version 3 of the GNU Affero General Public License into a single
+combined work, and to convey the resulting work. The terms of this
+License will continue to apply to the part which is the covered work,
+but the special requirements of the GNU Affero General Public License,
+section 13, concerning interaction through a network will apply to the
+combination as such.
+
+ 14. Revised Versions of this License.
+
+ The Free Software Foundation may publish revised and/or new versions of
+the GNU General Public License from time to time. Such new versions will
+be similar in spirit to the present version, but may differ in detail to
+address new problems or concerns.
+
+ Each version is given a distinguishing version number. If the
+Program specifies that a certain numbered version of the GNU General
+Public License "or any later version" applies to it, you have the
+option of following the terms and conditions either of that numbered
+version or of any later version published by the Free Software
+Foundation. If the Program does not specify a version number of the
+GNU General Public License, you may choose any version ever published
+by the Free Software Foundation.
+
+ If the Program specifies that a proxy can decide which future
+versions of the GNU General Public License can be used, that proxy's
+public statement of acceptance of a version permanently authorizes you
+to choose that version for the Program.
+
+ Later license versions may give you additional or different
+permissions. However, no additional obligations are imposed on any
+author or copyright holder as a result of your choosing to follow a
+later version.
+
+ 15. Disclaimer of Warranty.
+
+ THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
+APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
+HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
+OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
+THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
+IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
+ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
+
+ 16. Limitation of Liability.
+
+ IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
+WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
+THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
+GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
+USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
+DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
+PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
+EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
+SUCH DAMAGES.
+
+ 17. Interpretation of Sections 15 and 16.
+
+ If the disclaimer of warranty and limitation of liability provided
+above cannot be given local legal effect according to their terms,
+reviewing courts shall apply local law that most closely approximates
+an absolute waiver of all civil liability in connection with the
+Program, unless a warranty or assumption of liability accompanies a
+copy of the Program in return for a fee.
+
+ END OF TERMS AND CONDITIONS
+
+ How to Apply These Terms to Your New Programs
+
+ If you develop a new program, and you want it to be of the greatest
+possible use to the public, the best way to achieve this is to make it
+free software which everyone can redistribute and change under these terms.
+
+ To do so, attach the following notices to the program. It is safest
+to attach them to the start of each source file to most effectively
+state the exclusion of warranty; and each file should have at least
+the "copyright" line and a pointer to where the full notice is found.
+
+ <one line to give the program's name and a brief idea of what it does.>
+ Copyright (C) <year> <name of author>
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <<a href="https://www.gnu.org/licenses/">https://www.gnu.org/licenses/</a>>.
+
+Also add information on how to contact you by electronic and paper mail.
+
+ If the program does terminal interaction, make it output a short
+notice like this when it starts in an interactive mode:
+
+ <program> Copyright (C) <year> <name of author>
+ This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
+ This is free software, and you are welcome to redistribute it
+ under certain conditions; type `show c' for details.
+
+The hypothetical commands `show w' and `show c' should show the appropriate
+parts of the General Public License. Of course, your program's commands
+might be different; for a GUI interface, you would use an "about box".
+
+ You should also get your employer (if you work as a programmer) or school,
+if any, to sign a "copyright disclaimer" for the program, if necessary.
+For more information on this, and how to apply and follow the GNU GPL, see
+<<a href="https://www.gnu.org/licenses/">https://www.gnu.org/licenses/</a>>.
+
+ The GNU General Public License does not permit incorporating your program
+into proprietary programs. If your program is a subroutine library, you
+may consider it more useful to permit linking proprietary applications with
+the library. If this is what you want to do, use the GNU Lesser General
+Public License instead of this License. But first, please read
+<<a href="https://www.gnu.org/philosophy/why-not-lgpl.html">https://www.gnu.org/philosophy/why-not-lgpl.html</a>>.
--- /dev/null
+ GNU GENERAL PUBLIC LICENSE
+ Version 2, June 1991
+
+ Copyright (C) 1989, 1991 Free Software Foundation, Inc.
+ 51 Franklin Street - Fifth Floor, Boston, MA 02110-1301, USA.
+ Everyone is permitted to copy and distribute verbatim copies
+ of this license document, but changing it is not allowed.
+
+ Preamble
+
+ The licenses for most software are designed to take away your
+freedom to share and change it. By contrast, the GNU General Public
+License is intended to guarantee your freedom to share and change free
+software--to make sure the software is free for all its users. This
+General Public License applies to most of the Free Software
+Foundation's software and to any other program whose authors commit to
+using it. (Some other Free Software Foundation software is covered by
+the GNU Library General Public License instead.) You can apply it to
+your programs, too.
+
+ When we speak of free software, we are referring to freedom, not
+price. Our General Public Licenses are designed to make sure that you
+have the freedom to distribute copies of free software (and charge for
+this service if you wish), that you receive source code or can get it
+if you want it, that you can change the software or use pieces of it
+in new free programs; and that you know you can do these things.
+
+ To protect your rights, we need to make restrictions that forbid
+anyone to deny you these rights or to ask you to surrender the rights.
+These restrictions translate to certain responsibilities for you if you
+distribute copies of the software, or if you modify it.
+
+ For example, if you distribute copies of such a program, whether
+gratis or for a fee, you must give the recipients all the rights that
+you have. You must make sure that they, too, receive or can get the
+source code. And you must show them these terms so they know their
+rights.
+
+ We protect your rights with two steps: (1) copyright the software, and
+(2) offer you this license which gives you legal permission to copy,
+distribute and/or modify the software.
+
+ Also, for each author's protection and ours, we want to make certain
+that everyone understands that there is no warranty for this free
+software. If the software is modified by someone else and passed on, we
+want its recipients to know that what they have is not the original, so
+that any problems introduced by others will not reflect on the original
+authors' reputations.
+
+ Finally, any free program is threatened constantly by software
+patents. We wish to avoid the danger that redistributors of a free
+program will individually obtain patent licenses, in effect making the
+program proprietary. To prevent this, we have made it clear that any
+patent must be licensed for everyone's free use or not licensed at all.
+
+ The precise terms and conditions for copying, distribution and
+modification follow.
+<hr>
+ GNU GENERAL PUBLIC LICENSE
+ TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
+
+ 0. This License applies to any program or other work which contains
+a notice placed by the copyright holder saying it may be distributed
+under the terms of this General Public License. The "Program", below,
+refers to any such program or work, and a "work based on the Program"
+means either the Program or any derivative work under copyright law:
+that is to say, a work containing the Program or a portion of it,
+either verbatim or with modifications and/or translated into another
+language. (Hereinafter, translation is included without limitation in
+the term "modification".) Each licensee is addressed as "you".
+
+Activities other than copying, distribution and modification are not
+covered by this License; they are outside its scope. The act of
+running the Program is not restricted, and the output from the Program
+is covered only if its contents constitute a work based on the
+Program (independent of having been made by running the Program).
+Whether that is true depends on what the Program does.
+
+ 1. You may copy and distribute verbatim copies of the Program's
+source code as you receive it, in any medium, provided that you
+conspicuously and appropriately publish on each copy an appropriate
+copyright notice and disclaimer of warranty; keep intact all the
+notices that refer to this License and to the absence of any warranty;
+and give any other recipients of the Program a copy of this License
+along with the Program.
+
+You may charge a fee for the physical act of transferring a copy, and
+you may at your option offer warranty protection in exchange for a fee.
+
+ 2. You may modify your copy or copies of the Program or any portion
+of it, thus forming a work based on the Program, and copy and
+distribute such modifications or work under the terms of Section 1
+above, provided that you also meet all of these conditions:
+
+ a) You must cause the modified files to carry prominent notices
+ stating that you changed the files and the date of any change.
+
+ b) You must cause any work that you distribute or publish, that in
+ whole or in part contains or is derived from the Program or any
+ part thereof, to be licensed as a whole at no charge to all third
+ parties under the terms of this License.
+
+ c) If the modified program normally reads commands interactively
+ when run, you must cause it, when started running for such
+ interactive use in the most ordinary way, to print or display an
+ announcement including an appropriate copyright notice and a
+ notice that there is no warranty (or else, saying that you provide
+ a warranty) and that users may redistribute the program under
+ these conditions, and telling the user how to view a copy of this
+ License. (Exception: if the Program itself is interactive but
+ does not normally print such an announcement, your work based on
+ the Program is not required to print an announcement.)
+<hr>
+These requirements apply to the modified work as a whole. If
+identifiable sections of that work are not derived from the Program,
+and can be reasonably considered independent and separate works in
+themselves, then this License, and its terms, do not apply to those
+sections when you distribute them as separate works. But when you
+distribute the same sections as part of a whole which is a work based
+on the Program, the distribution of the whole must be on the terms of
+this License, whose permissions for other licensees extend to the
+entire whole, and thus to each and every part regardless of who wrote it.
+
+Thus, it is not the intent of this section to claim rights or contest
+your rights to work written entirely by you; rather, the intent is to
+exercise the right to control the distribution of derivative or
+collective works based on the Program.
+
+In addition, mere aggregation of another work not based on the Program
+with the Program (or with a work based on the Program) on a volume of
+a storage or distribution medium does not bring the other work under
+the scope of this License.
+
+ 3. You may copy and distribute the Program (or a work based on it,
+under Section 2) in object code or executable form under the terms of
+Sections 1 and 2 above provided that you also do one of the following:
+
+ a) Accompany it with the complete corresponding machine-readable
+ source code, which must be distributed under the terms of Sections
+ 1 and 2 above on a medium customarily used for software interchange; or,
+
+ b) Accompany it with a written offer, valid for at least three
+ years, to give any third party, for a charge no more than your
+ cost of physically performing source distribution, a complete
+ machine-readable copy of the corresponding source code, to be
+ distributed under the terms of Sections 1 and 2 above on a medium
+ customarily used for software interchange; or,
+
+ c) Accompany it with the information you received as to the offer
+ to distribute corresponding source code. (This alternative is
+ allowed only for noncommercial distribution and only if you
+ received the program in object code or executable form with such
+ an offer, in accord with Subsection b above.)
+
+The source code for a work means the preferred form of the work for
+making modifications to it. For an executable work, complete source
+code means all the source code for all modules it contains, plus any
+associated interface definition files, plus the scripts used to
+control compilation and installation of the executable. However, as a
+special exception, the source code distributed need not include
+anything that is normally distributed (in either source or binary
+form) with the major components (compiler, kernel, and so on) of the
+operating system on which the executable runs, unless that component
+itself accompanies the executable.
+
+If distribution of executable or object code is made by offering
+access to copy from a designated place, then offering equivalent
+access to copy the source code from the same place counts as
+distribution of the source code, even though third parties are not
+compelled to copy the source along with the object code.
+<hr>
+ 4. You may not copy, modify, sublicense, or distribute the Program
+except as expressly provided under this License. Any attempt
+otherwise to copy, modify, sublicense or distribute the Program is
+void, and will automatically terminate your rights under this License.
+However, parties who have received copies, or rights, from you under
+this License will not have their licenses terminated so long as such
+parties remain in full compliance.
+
+ 5. You are not required to accept this License, since you have not
+signed it. However, nothing else grants you permission to modify or
+distribute the Program or its derivative works. These actions are
+prohibited by law if you do not accept this License. Therefore, by
+modifying or distributing the Program (or any work based on the
+Program), you indicate your acceptance of this License to do so, and
+all its terms and conditions for copying, distributing or modifying
+the Program or works based on it.
+
+ 6. Each time you redistribute the Program (or any work based on the
+Program), the recipient automatically receives a license from the
+original licensor to copy, distribute or modify the Program subject to
+these terms and conditions. You may not impose any further
+restrictions on the recipients' exercise of the rights granted herein.
+You are not responsible for enforcing compliance by third parties to
+this License.
+
+ 7. If, as a consequence of a court judgment or allegation of patent
+infringement or for any other reason (not limited to patent issues),
+conditions are imposed on you (whether by court order, agreement or
+otherwise) that contradict the conditions of this License, they do not
+excuse you from the conditions of this License. If you cannot
+distribute so as to satisfy simultaneously your obligations under this
+License and any other pertinent obligations, then as a consequence you
+may not distribute the Program at all. For example, if a patent
+license would not permit royalty-free redistribution of the Program by
+all those who receive copies directly or indirectly through you, then
+the only way you could satisfy both it and this License would be to
+refrain entirely from distribution of the Program.
+
+If any portion of this section is held invalid or unenforceable under
+any particular circumstance, the balance of the section is intended to
+apply and the section as a whole is intended to apply in other
+circumstances.
+
+It is not the purpose of this section to induce you to infringe any
+patents or other property right claims or to contest validity of any
+such claims; this section has the sole purpose of protecting the
+integrity of the free software distribution system, which is
+implemented by public license practices. Many people have made
+generous contributions to the wide range of software distributed
+through that system in reliance on consistent application of that
+system; it is up to the author/donor to decide if he or she is willing
+to distribute software through any other system and a licensee cannot
+impose that choice.
+
+This section is intended to make thoroughly clear what is believed to
+be a consequence of the rest of this License.
+<hr>
+ 8. If the distribution and/or use of the Program is restricted in
+certain countries either by patents or by copyrighted interfaces, the
+original copyright holder who places the Program under this License
+may add an explicit geographical distribution limitation excluding
+those countries, so that distribution is permitted only in or among
+countries not thus excluded. In such case, this License incorporates
+the limitation as if written in the body of this License.
+
+ 9. The Free Software Foundation may publish revised and/or new versions
+of the General Public License from time to time. Such new versions will
+be similar in spirit to the present version, but may differ in detail to
+address new problems or concerns.
+
+Each version is given a distinguishing version number. If the Program
+specifies a version number of this License which applies to it and "any
+later version", you have the option of following the terms and conditions
+either of that version or of any later version published by the Free
+Software Foundation. If the Program does not specify a version number of
+this License, you may choose any version ever published by the Free Software
+Foundation.
+
+ 10. If you wish to incorporate parts of the Program into other free
+programs whose distribution conditions are different, write to the author
+to ask for permission. For software which is copyrighted by the Free
+Software Foundation, write to the Free Software Foundation; we sometimes
+make exceptions for this. Our decision will be guided by the two goals
+of preserving the free status of all derivatives of our free software and
+of promoting the sharing and reuse of software generally.
+
+ NO WARRANTY
+
+ 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
+FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN
+OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
+PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
+OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS
+TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE
+PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
+REPAIR OR CORRECTION.
+
+ 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
+WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
+REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
+INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
+OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
+TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
+YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
+PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
+POSSIBILITY OF SUCH DAMAGES.
+
+ END OF TERMS AND CONDITIONS
+<hr>
+ How to Apply These Terms to Your New Programs
+
+ If you develop a new program, and you want it to be of the greatest
+possible use to the public, the best way to achieve this is to make it
+free software which everyone can redistribute and change under these terms.
+
+ To do so, attach the following notices to the program. It is safest
+to attach them to the start of each source file to most effectively
+convey the exclusion of warranty; and each file should have at least
+the "copyright" line and a pointer to where the full notice is found.
+
+ <one line to give the program's name and a brief idea of what it does.>
+ Copyright (C) <year> <name of author>
+
+ This program is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 2 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program; if not, write to the Free Software
+ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+
+
+Also add information on how to contact you by electronic and paper mail.
+
+If the program is interactive, make it output a short notice like this
+when it starts in an interactive mode:
+
+ Gnomovision version 69, Copyright (C) year name of author
+ Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
+ This is free software, and you are welcome to redistribute it
+ under certain conditions; type `show c' for details.
+
+The hypothetical commands `show w' and `show c' should show the appropriate
+parts of the General Public License. Of course, the commands you use may
+be called something other than `show w' and `show c'; they could even be
+mouse-clicks or menu items--whatever suits your program.
+
+You should also get your employer (if you work as a programmer) or your
+school, if any, to sign a "copyright disclaimer" for the program, if
+necessary. Here is a sample; alter the names:
+
+ Yoyodyne, Inc., hereby disclaims all copyright interest in the program
+ `Gnomovision' (which makes passes at compilers) written by James Hacker.
+
+ <signature of Ty Coon>, 1 April 1989
+ Ty Coon, President of Vice
+
+This General Public License does not permit incorporating your program into
+proprietary programs. If your program is a subroutine library, you may
+consider it more useful to permit linking proprietary applications with the
+library. If this is what you want to do, use the GNU Library General
+Public License instead of this License.
--- /dev/null
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
+<HTML>
+<HEAD>
+<TITLE>rsync FAQ</TITLE>
+</HEAD>
+<!--#include virtual="header.html" -->
+
+<H2 align="center">Frequently Asked Questions</H2>
+
+<table><tr valign=top><td><ul>
+<li><a href="#1">the transfer fails to finish</a><br>
+<li><a href="#2">rsync recopies the same files</a><br>
+<li><a href="#3">is your shell clean</a><br>
+<li><a href="#4">memory usage</a><br>
+<li><a href="#5">out of memory</a><br>
+<li><a href="#6">rsync through a firewall</a><br>
+<li><a href="#7">rsync and cron</a><br>
+</ul></td><td> </td><td><ul>
+<li><a href="#8">rsync: Command not found</a><br>
+<li><a href="#9">spaces in filenames</a><br>
+<li><a href="#10">ignore "vanished files" warning</a><br>
+<li><a href="#11">read-only file system</a><br>
+<li><a href="#12">multiplexing overflow 101:7104843</a><br>
+<li><a href="#13">inflate (token) returned -5</a><br>
+</ul></td></tr></table>
+
+<hr>
+<h3><a name=1>the transfer fails to finish</a></h3>
+
+<p>If you get an error like one of these:
+
+<pre>rsync: error writing 4 unbuffered bytes - exiting: Broken pipe
+rsync error: error in rsync protocol data stream (code 12) at io.c(463)
+</pre>
+
+<p>or
+
+<pre>rsync: connection unexpectedly closed (24 bytes read so far)
+rsync error: error in rsync protocol data stream (code 12) at io.c(342)
+</pre>
+
+<p>please read the <a href="issues.html">issues and debugging page</a>
+for details on how you can try to figure out what is going wrong.
+
+<hr>
+<h3><a name=2>rsync recopies the same files</a></h3>
+
+<p>Some people occasionally report that rsync copies too many files when
+they expect it to copy only a few. In most cases the explanation is
+that you forgot to include the --times (-t) option in the original copy,
+so rsync is forced to (efficiently) transfer every file that differs in
+its modified time to discover what data (if any) has changed.
+
+<p>Another common cause involves sending files to an Microsoft filesystem:
+if the file's modified time is an odd value but the receiving filesystem
+can only store even values, then rsync will re-transfer too many files.
+You can avoid this by specifying the --modify-window=1 option.
+
+<p>Yet another periodic case can happen when daylight-savings time
+changes if your OS+filesystem saves file times in local time instead of
+UTC. For a full explanation of this and some suggestions on how to
+avoid them problem, see <a href="daylight-savings.html">this document</a>.
+
+<p>Something else that can trip up rsync is a filesystem changeing the
+filename behind the scenes. This can happen when a filesystem changes
+an all-uppercase name into lowercase, or when it decomposes UTF-8 behind
+your back.
+
+<blockquote>
+
+<p>An example of the latter can occur with HFS+ on Mac OS X: if you
+copy a directory with a file that has a UTF-8 character sequence in it,
+say a 2-byte umlaut-u (\0303\0274), the file will get that character
+stored by the filesystem using 3 bytes (\0165\0314\0210), and rsync will
+not know that these differing filenames are the same file (it will, in
+fact, remove a prior copy of the file if --delete is enabled, and then
+recreate it).
+
+<p>You can avoid a charset problem by passing an appropriate --iconv
+option to rsync that tells it what character-set the source files are,
+and what character-set the destination files get stored in. For
+instance, the above Mac OS X problem would be dealt with by using
+--iconv=UTF-8,UTF8-MAC (UTF8-MAC is a pseudo-charset recognized by Mac
+OS X iconv in which all characters are decomposed).
+
+</blockquote>
+
+<p>If you think that rsync is copying too many files, look at the
+itemized output (-i) to see why rsync is doing the update (e.g. the 't'
+flag indicates that the time differs, or all pluses indicates that rsync
+thinks the file doesn't exist). You can also look at the stats produced
+with -v and see if rsync is really sending all the data. See also the
+--checksum (-c) option for one way to avoid the extra copying of files
+that don't have synchronized modified times (but keep in mind that the
+-c option eats lots of disk I/O, and can be rather slow).
+
+<hr>
+<h3><a name=3>is your shell clean</a></h3>
+
+<p>The "is your shell clean" message and the "protocol mismatch" message
+are usually caused by having some sort of program in your .cshrc, .profile,
+.bashrc or equivalent file that writes a message every time you connect
+using a remote-shell program (such as ssh or rsh). Data written in this
+way corrupts the rsync data stream. rsync detects this at startup and
+produces those error messages. However, if you are using rsync-daemon
+syntax (host::path or rsync://) without using a remote-shell program (no
+--rsh or -e option), there is not remote-shell program involved, and the
+problem is probably caused by an error on the daemon side (so check the
+daemon logs).
+
+<p>A good way to test if your remote-shell connection is clean is to try
+something like this (use ssh or rsh, as appropriate):
+
+<blockquote><pre>ssh remotesystem /bin/true > test.dat</pre></blockquote>
+
+<p>That should create a file called test.dat with nothing in it. If
+test.dat is not of zero length then your shell is not clean. Look at the
+contents of test.dat to see what was sent. Look at all the startup files on
+remotesystem to try and find the problem.
+
+<hr>
+<h3><a name=4>memory usage</a></h3>
+
+<p>Rsync versions before 3.0.0 always build the entire list of files to be
+transferred at the beginning and hold it in memory for the entire run. Rsync
+needs about 100 bytes to store all the relevant information for one file,
+so (for example) a run with 800,000 files would consume about 80M of
+memory. -H and --delete increase the memory usage further.
+
+<p>Version 3.0.0 slightly reduced the memory used per file by not storing fields
+not needed for a particular file. It also introduced an incremental recursion
+mode that builds the file list in chunks and holds each chunk in memory only as
+long as it is needed. This mode dramatically reduces memory usage, but it
+only works provided that both sides are 3.0.0 or newer and certain options that
+rsync currently can't handle in this mode are not being used.
+
+<hr>
+<h3><a name=5>out of memory</a></h3>
+
+<p>The usual reason for "out of memory" when running rsync is that you are
+transferring a _very_ large number of files. The size of the files doesn't
+matter, only the total number of files. If memory is a problem, first try to
+use the incremental recursion mode: upgrade both sides to rsync 3.0.0 or
+newer and avoid options that disable incremental recursion (e.g., use
+<tt>--delete-delay</tt> instead of <tt>--delete-after</tt>). If this is not
+possible, you can break the rsync run into smaller chunks operating on
+individual subdirectories using <tt>--relative</tt> and/or exclude rules.
+
+<hr>
+<h3><a name=6>rsync through a firewall</a></h3>
+
+<p>If you have a setup where there is no way to directly connect two
+systems for an rsync transfer, there are several ways to get a firewall
+system to act as an intermediary in the transfer. You'll find full details
+on the <a href="firewall.html">firewall methods</a> page.
+
+<hr>
+<h3><a name=7>rsync and cron</a></h3>
+
+<p>On some systems (notably SunOS4) cron supplies what looks like a socket
+to rsync, so rsync thinks that stdin is a socket. This means that if you
+start rsync with the --daemon switch from a cron job you end up rsync
+thinking it has been started from inetd. The fix is simple—just
+redirect stdin from /dev/null in your cron job.
+
+<hr>
+<h3><a name=8>rsync: Command not found</a></h3>
+
+<p>This error is produced when the remote shell is unable to locate the rsync
+binary in your path. There are 3 possible solutions:
+
+<ol>
+
+<li>install rsync in a "standard" location that is in your remote path.
+
+<li>modify your .cshrc, .bashrc etc on the remote system to include the path
+that rsync is in
+
+<li>use the --rsync-path option to explicitly specify the path on the
+remote system where rsync is installed
+
+</ol>
+
+<p>You may echo find the command:
+
+<blockquote><pre>ssh host 'echo $PATH'</pre></blockquote>
+
+<p>for determining what your remote path is.
+
+<hr>
+<h3><a name=9>spaces in filenames</a></h3>
+
+<p>Can rsync copy files with spaces in them?
+
+<p>Short answer: Yes, rsync can handle filenames with spaces.
+
+<p>Long answer:
+
+<p>Rsync handles spaces just like any other unix command line application.
+Within the code spaces are treated just like any other character so a
+filename with a space is no different from a filename with any other
+character in it.
+
+<p>The problem of spaces is in the argv processing done to interpret the
+command line. As with any other unix application you have to escape spaces
+in some way on the command line or they will be used to separate arguments.
+
+<p>It is slightly trickier in rsync (and other remote-copy programs like
+scp) because rsync sends a command line to the remote system to launch the
+peer copy of rsync (this assumes that we're not talking about daemon mode,
+which is not affected by this problem because no remote shell is involved
+in the reception of the filenames). The command line is interpreted by the
+remote shell and thus the spaces need to arrive on the remote system
+escaped so that the shell doesn't split such filenames into multiple
+arguments.
+
+<p>For example:
+
+<blockquote><pre>rsync -av host:'a long filename' /tmp/</pre></blockquote>
+
+<p>This is usually a request for rsync to copy 3 files from the remote
+system, "a", "long", and "filename" (the only exception to this is for a
+system running a shell that does not word-split arguments in its commands,
+and that is exceedingly rare). If you wanted to request a single file with
+spaces, you need to get some kind of space-quoting characters to the remote
+shell that is running the remote rsync command. The following commands
+should all work:
+
+<blockquote><pre>rsync -av host:'"a long filename"' /tmp/
+rsync -av host:'a\ long\ filename' /tmp/
+rsync -av host:a\\\ long\\\ filename /tmp/</pre></blockquote>
+
+<p>You might also like to use a '?' in place of a space as long as there
+are no other matching filenames than the one with spaces (since '?' matches
+any character):
+
+<blockquote><pre>rsync -av host:a?long?filename /tmp/</pre></blockquote>
+
+<p>As long as you know that the remote filenames on the command line
+are interpreted by the remote shell then it all works fine.
+
+<hr>
+<h3><a name=10>ignore "vanished files" warning</a></h3>
+
+<p>Some folks would like to ignore the "vanished files" warning, which
+manifests as an exit-code 24. The easiest way to do this is to create
+a shell script wrapper. For instance, name this something like
+"rsync-no24":
+
+<blockquote><pre>#!/bin/sh
+rsync "$@"
+e=$?
+if test $e = 24; then
+ exit 0
+fi
+exit $e</pre></blockquote>
+
+<hr>
+<h3><a name=11>read-only file system</a></h3>
+
+<p>If you get "Read-only file system" as an error when sending to a rsync
+daemon then you probably forgot to set "read only = no" for that module.
+
+<hr>
+<h3><a name=12>multiplexing overflow 101:7104843</a></h3>
+
+<p>This mysterious error, or the similar "invalid message 101:7104843", can
+happen if one of the rsync processes is killed for some reason and a message
+beginning with the four characters "Kill" gets inserted into the protocol
+stream as a result. To solve the problem, you'll need to figure out why rsync
+is being killed.
+
+<hr>
+<h3><a name=13>inflate (token) returned -5</a></h3>
+
+This error means that rsync failed to handle an expected error from the
+compression code for a file that happened to be transferred with a block size
+of 32816 bytes. You can avoid this issue for the affected file by transferring
+it with a manually-set block size (e.g. --block-size=33000), or by upgrading
+the receiving side to rsync 3.0.7.
+
+<hr>
+
+<!--#include virtual="footer.html" -->
--- /dev/null
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
+<HTML>
+<HEAD>
+<TITLE>rsync's license</TITLE>
+</HEAD>
+<!--#include virtual="header.html" -->
+
+Beginning with 3.0.0, rsync is available under the <b>GNU General Public
+License version 3</b>. <i>(Older releases were available under the
+<a href="GPL2.html">GPL version 2</a>.)</i>
+
+<pre><small>
+<!--#include virtual="COPYING.html" -->
+</small></pre>
+
+<!--#include virtual="footer.html" -->
--- /dev/null
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
+<HTML>
+<HEAD>
+<TITLE>rsync's license</TITLE>
+</HEAD>
+<!--#include virtual="header.html" -->
+
+Releases <b>prior</b> to 3.0.0 were released under the
+<b>GNU General Public License version 2</b>:
+
+<pre><small>
+<!--#include virtual="COPYING2.html" -->
+</small></pre>
+
+<!--#include virtual="footer.html" -->
--- /dev/null
+#!/bin/sh
+
+# This script does personal backups to a rsync backup server. You will end up
+# with a 7 day rotating incremental backup. The incrementals will go
+# into subdirectories named after the day of the week, and the current
+# full backup goes into a directory called "current"
+# tridge@linuxcare.com
+
+# directory to backup
+BDIR=/home/$USER
+
+# excludes file - this contains a wildcard pattern per line of files to exclude
+EXCLUDES=$HOME/cron/excludes
+
+# the name of the backup machine
+BSERVER=owl
+
+# your password on the backup server
+export RSYNC_PASSWORD=XXXXXX
+
+
+########################################################################
+
+BACKUPDIR=`date +%A`
+OPTS="--force --ignore-errors --delete-excluded --exclude-from=$EXCLUDES
+ --delete --backup --backup-dir=/$BACKUPDIR -a"
+
+export PATH=$PATH:/bin:/usr/bin:/usr/local/bin
+
+# the following line clears the last weeks incremental directory
+[ -d $HOME/emptydir ] || mkdir $HOME/emptydir
+rsync --delete -a $HOME/emptydir/ $BSERVER::$USER/$BACKUPDIR/
+rmdir $HOME/emptydir
+
+# now the actual transfer
+rsync $OPTS $BDIR $BSERVER::$USER/current
--- /dev/null
+#!/bin/sh
+rm -f badge.svg
+wget https://github.com/RsyncProject/rsync/workflows/build/badge.svg
+rsync -aiic --inplace --remove-source-files badge.svg $SAMBA_HOST:/home/httpd/html/rsync/
--- /dev/null
+#!/bin/sh
+if [ -d rsync-and-debian ]; then
+ rsync -aviOHFFc --del -f._filt . $SAMBA_HOST:/home/httpd/html/rsync/ "${@}"
+else
+ echo "Run this from the root of the html hierarchy."
+ exit 1
+fi
--- /dev/null
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
+<HTML>
+<HEAD>
+<TITLE>rsync bug-tracking</TITLE>
+</HEAD>
+<!--#include virtual="header.html" -->
+
+<H2 align="center">rsync bug-tracking</H2>
+
+<p> Please use this checklist combined with the help on the
+<a href="issues.html">issues and debugging</a> page before
+reporting a bug. Thanks!
+
+<ul>
+
+<li> If you're not using the latest released version, please upgrade before
+reporting a bug.
+
+<li> If you're using the latest released version, consult the
+<a href="https://github.com/RsyncProject/rsync/blob/master/NEWS.md">NEWS file from the git repository</a> to see if what
+you're seeing has already been handled in the version under development.
+
+<li> It is also helpful to search the bug reports at the
+<a href="https://github.com/RsyncProject/rsync/issues?q=is%3Aissue">GitHub issues tracker</a>
+to see if the problem is already known.
+
+<li> See also the <a href="issues.html">issues and debugging</a> page to
+help you figure out if what you're seeing is a known bug and perhaps to
+help diagnose what is going wrong.
+
+<li> Discuss the bug on the
+<a href="https://lists.samba.org/mailman/listinfo/rsync">rsync mailing list</a>
+(which is at <tt>rsync@lists.samba.org</tt>) to help you figure out if what
+you're seeing is really a bug or a mistake.
+
+<li>There are several patches for features that are under consideration that
+can be found in the <a href="https://github.com/RsyncProject/rsync-patches">rsync-patches repo</a>.
+
+<li> If you haven't already done so, please take a couple of minutes to read Simon Tatham's
+<a href="https://www.chiark.greenend.org.uk/~sgtatham/bugs.html">advice on how to report bugs</a>.
+
+</ul>
+
+<p> To report a bug or make suggestions, use one of these methods:
+
+<ul>
+
+<li> The mailing list (mentioned above) is a good resource for discussing
+bugs and suggesting new features. It accepts patches (typically as MIME
+attachments), but for fixes is often easier to attach a patch to an
+appropriate GitHub issue report or use a pull request. Note that there is
+no mandate to use pull requests for patches, as that can be a pretty high
+bar of git know-how that not everyone needs to be familiar with.
+
+<li> If you'd like to see a bug-report or feature-request get officially noted,
+<a href="https://github.com/RsyncProject/rsync/issues">create an issue on GitHub</a>
+(this does require that you have created a GitHub account). If you want to
+stay abreast of what's going on with the issues, make use of the GitHub
+subscriptions to pick and choose what kind of notifications you want to
+receive (e.g. just a single issue, all issues, all rsync activity, etc.).
+
+<li>For security issues please send email
+ to <a href="mailto:rsync.project@gmail.com">rsync.project@gmail.com</a>
+ with details of the issue
+</ul>
+
+<p> Thanks for helping out!
+
+<!--#include virtual="footer.html" -->
--- /dev/null
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
+<HTML>
+<HEAD>
+<TITLE>rsync bug-tracking</TITLE>
+</HEAD>
+
+<meta http-equiv="refresh" content="0;URL='https://rsync.samba.org/bug-tracking.html'" />
--- /dev/null
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
+<HTML>
+<HEAD>
+<TITLE>rsync bug-tracking</TITLE>
+</HEAD>
+
+<meta http-equiv="refresh" content="0;URL='https://rsync.samba.org/bug-tracking.html'" />
--- /dev/null
+#!/usr/bin/perl
+use strict;
+
+while (<>) {
+ s/&/&/g;
+ s/</</g;
+ s/>/>/g;
+ s/\f/<hr>/;
+ s#(<)(https?:.*?)(>)#$1<a href="$2">$2</a>$3#g;
+ print $_;
+}
--- /dev/null
+<HTML><HEAD>
+<TITLE>DST change and date comparisons</TITLE>
+</HEAD><BODY>
+
+<p><small><i>J.W. Schultz wrote the following text in a
+<a href="http://www.cygwin.com/ml/cygwin/2003-10/msg00995.html">post</a>
+to the cygwin mailing list. It has been slightly edited and beautified
+for inclusion here.</i></small>
+
+<h1>How the DST Change can adversely affect FAT filesystems</h1>
+
+<p>
+The vernal and autumnal transitions to and from daylight-savings
+time have important implications
+for those with Microsoft systems and use utilities that
+compare file timestamps on different filesystem types or
+with filesystems on other operating systems that can lead to
+a problem in how the file's date is handled.
+This problem lies in the way FAT filesystems stores
+timestamps and how Windows converts between local time and
+UTC.
+
+<h2>Background</h2>
+
+<p>
+In UNIX and UNIX-like systems (such as Linux) file timestamps
+are stored in UTC (universal time) and are only converted to
+local-time by user-space programs for display purposes. At
+the system call level all time values are in UTC and
+utilities that compare timestamps do so in UTC. Also, the
+standard UTC->local and local->UTC conversion functions are
+aware of DST and conversions reflect this so that if a
+timestamp was recorded during ST it will be converted using
+the ST offset even when the current system time is DST.
+
+<p>
+In Windows things are not so simple. Windows operates in
+local-time. Timestamps in the various FAT derived
+filesystems are stored in local-time. Timestamps in NTFS
+filesystems are stored in UTC. This inconsistency is
+further complicated by the fact that the conversion routines
+used are not DST aware. Instead of being DST aware the
+system has a fixed offset to convert between local-time and
+UTC regardless of the date in the timestamp. This fixed
+offset is calculated at boot time and only changed when
+systems transition to or from DST. As a result the apparent
+modification time of a file on NTFS as reported in a windows
+utility will change by one hour when reported in local-time
+and FAT based files when reported in UTC.
+
+<p>
+The difficulty that this produces is that any utilities that
+compare timestamps between FAT and NTFS filesystems or
+between Windows and other platforms will view files that
+have not changed as having a different modified time. Among other things
+this will affect rsync, rdiff, unison, wget, and make. However,
+for the purposes of this document, we will only discuss rsync.
+
+<p>
+With the reduced cost of hard disks many newer backup
+systems are using hard disk based storage and take advantage
+of timestamp comparison to detect file changes for the sake
+of efficiency. Rsync is probably premier in this role and
+is used by a fair number of free and even commercial backup
+systems as well as being the basis for many home-brew backup
+solutions.
+
+<p>
+With rsync and similar systems the effect of this is that
+every file will appear to have been changed. The result is
+any space savings associated with linking (--link-dest) or
+with decremental backup approaches (--compare-dest and
+--backup-dir) will be defeated. Perhaps worse, because
+every file will appear to have changed the time required to
+do a backup or a non-backup rsync will be much longer than
+normal. In some cases backups that normally complete in
+less than one hour can take several days.
+
+<p>
+So what can be done about it? Several things, there are
+ways to merely mitigate the problem, to correct it and finally
+to prevent the problem entirely.
+
+<h2>Mitigation</h2>
+
+<p>
+Rsync has a --modify-window option. Many of you already use
+--modify-window=1 to cope with the fact that windows often
+stores timestamps with a two second resolution. Using a
+--modify-window=3601 will cause rsync to ignore timestamp
+differences of up to one hour.
+
+<p>
+This if often not particularly dangerous because a file would have to
+be changed, synced and changed again without changing size
+within a single hour and have no subsequent changes for this copy
+to miss a file change. However, for some systems any such risk is
+unacceptable, so other solutions are needed.
+
+<h2>Correcting the Timestamps</h2>
+
+<p>
+There are two ways to correct.
+
+<p>For the first run after the time change, you can run rsync with the
+--checksum option in order to ensure that only files that have a changed
+checksum get transferred, and update the modified time on all unchanged
+files. This option has the drawback that it increases disk I/O by
+a large amount on both the sending and receiving side, slowing down the
+copy.
+
+<p>
+The other way to correct things is to change the timestamps
+on the files on the backup server before doing a copy after a
+time change has occurred.
+
+<p>
+Included here is an example perl script that will change the
+timestamps of files in a list on standard-input. Whether
+you use a positive or negative shift will depend on which
+end you decide to adjust.
+
+<p>
+This is an example of how to use the script:
+
+<blockquote><pre>
+touch -d '01:00 13-apr-03' /tmp/cmpfile
+find . -type f ! -newer /tmp/cmpfile | shifttime.pl 3600
+</pre></blockquote>
+
+<hr>
+<pre>
+#!/usr/bin/perl
+use strict;
+
+my $offset = shift() + 0;
+die "Usage: $0 OFFSET_SECONDS\n" unless $offset;
+
+while (<>) {
+ chomp;
+ my $mtime = (stat $_)[9];
+ next unless $mtime;
+ $mtime += $offset;
+ utime $mtime, $mtime, $_;
+}
+</pre>
+<hr>
+
+<h2>Prevention</h2>
+
+<p>
+To prevent the problem in the first place you need to
+prevent changing to DST. This can be done by either running
+the windows system in UTC, by disabling DST and changing
+the system time manually twice each year, or by avoiding the use
+of the FAT filesystem (perhaps by switching to a different OS).
+
+
+<h2>Notes and References</h2>
+
+<p>
+Here are some references that Wayne Piekarski collected
+while researching this problem. They contain a lot of
+information about the ways that Windows deals with
+timestamps on NTFS and FAT filesystems.
+
+<p>
+<a href="http://optics.ph.unimelb.edu.au/help/rsync/rsync_pc1.html#gotchas">http://optics.ph.unimelb.edu.au/help/rsync/rsync_pc1.html#gotchas</a></br>
+<a href="http://list-archive.xemacs.org/xemacs-nt/199911/msg00130.html">http://list-archive.xemacs.org/xemacs-nt/199911/msg00130.html</a></br>
+<a href="http://p2p.wrox.com/archive/c_plus_plus_programming/2001-06/53.asp">http://p2p.wrox.com/archive/c_plus_plus_programming/2001-06/53.asp</a></br>
+<a href="http://www.codeproject.com/datetime/dstbugs.asp">http://www.codeproject.com/datetime/dstbugs.asp</a></br>
+<a href="http://support.microsoft.com/default.aspx?scid=kb;[LN];158588">http://support.microsoft.com/default.aspx?scid=kb;[LN];158588</a>
+
+<p>
+I wish to thank Wayne Piekarski for having copiled the
+references and also supplying some additional insights.
+
+<p>
+Permission is granted without reservation reprint and
+distribute this in whole and in part to any interested
+parties.
+
+<p>—J.W. Schultz
--- /dev/null
+<!-- This file gets included into the resources and documentation pages -->
+
+<li> A nice tutorial on <a href="https://linuxize.com/post/how-to-setup-passwordless-ssh-login/">setting up ssh to avoid password prompts</a>.
+
+<li> Karsten Thygesen has written a doc on how to setup
+<A HREF="http://dslab.lzu.edu.cn:8080/members/wangbj/wangbaojun/howtos/rsync-mirror-HOWTO/">anonymous rsync servers</A>.
+
+<li> Anthony Wesley has written a doc on
+<A HREF="/win95.txt">how to build rsync for Windows95</A>.
+
+<li> Mike McHenry has written up some info on how to get <a href="nt.html">rsync working under NT</a>.
+
+<li> Michael Holve has written a very useful <a href="http://everythinglinux.org/rsync/">rsync tutorial</a>.
+
+<li> Daniel Teklu has a detailed HOWTO on <a href="http://www.netbits.us/docs/stunnel_rsync.html">rsync and stunnel</a>.
--- /dev/null
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
+<HTML>
+<HEAD>
+<TITLE>rsync documentation</TITLE>
+</HEAD>
+<!--#include virtual="header.html" -->
+
+<H2 align="center">documentation</H2>
+
+<ul>
+
+<li> An html version of the <a href="https://download.samba.org/pub/rsync/rsync.1"><b>rsync</b>(1) manpage</a>.
+
+<li> An html version of the
+<a href="https://download.samba.org/pub/rsync/rsync-ssl.1"><b>rsync-ssl</b>(1) manpage</a>.
+
+<li> An html version of the
+<a href="https://download.samba.org/pub/rsync/rsyncd.conf.5"><b>rsyncd.conf</b>(5) manpage</a>.
+
+<li> An html version of the
+<a href="https://download.samba.org/pub/rsync/rrsync.1"><b>rrsync</b>(1) manpage</a>
+
+<li> The <a href="FAQ.html">FAQ</a> (frequently asked questions list).
+
+<!--#include virtual="doc-resources.html" -->
+
+<!--
+<li>Cross-referenced
+<a href="doxygen/head/files.html">rsync source code</a> produced by
+<a href="http://doxygen.org/">Doxygen</a>.
+-->
+
+<li> A html version of the original
+<a href="tech_report/">rsync technical report</a>.
+
+<li> A copy of Andrew Tridgell's
+<a href="http://samba.org/~tridge/phd_thesis.pdf">PhD thesis</a>
+(which includes three chapters on rsync).
+
+<li> A nice page on <a href="how-rsync-works.html">how rsync works</a>.
+
+<li> A copy of John Langford's thesis on
+<a href="http://www-2.cs.cmu.edu/~jcl/research/mrsync/mrsync.ps">Multiround rsync</a>,
+which is not used in rsync, but is interesting none-the-less.
+
+<li>A <a href="http://www.devshed.com/c/a/Administration/File-Synchronization-With-Rsync/">DevShed tutorial on rsync</a>.
+
+<li> <a href="rsync-and-debian/">Notes on rsync and Debian mirrors</a>.
+
+</ul>
+
+<p> <i>(Some of the above items are also listed on the <a href="resources.html">rsync resources page</a>.</i>
+
+<!--#include virtual="footer.html" -->
--- /dev/null
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
+<html>
+<head>
+<TITLE>rsync download</TITLE>
+</head>
+<!--#include virtual="header.html" -->
+
+<h2 align="center">rsync download</h2>
+
+<div style="float: right">
+<a href="https://github.com/RsyncProject/rsync/actions">
+<img src="badge.svg">
+</a></div>
+
+<h2>Source-code releases</h2>
+
+<p>You can grab the latest source code and other related files in a variety of ways:
+
+<ul>
+
+<li><p>The latest version is linked on the <a href="https://rsync.samba.org/">main page</a>.
+
+<li><p>A directory listing of these latest files and various historical release and diff files
+are available via <a href="https://download.samba.org/pub/rsync/">this web page</a> and
+via <i>anonymous SSL rsync</i> using this command:
+<p><code><small>rsync-ssl rsync://download.samba.org/rsyncftp/</small></code></p>
+
+<li><p>You can also get .zip and .tar.gz versions of the various git repo's release
+tags via the <a href="https://github.com/RsyncProject/rsync/tags">rsync GitHub tags page</a>
+and the associated patches via the
+<a href="https://github.com/RsyncProject/rsync-patches/tags">rsync-patches GitHub tags page</a>.
+Keep in mind that these git-derived files do NOT come with the extra generated files that are included
+in the official release tar files.
+
+<li><p>You can browse the very latest source files, clone the source using git, or download a .zip file of the latest
+master branch from <a href="https://github.com/RsyncProject/rsync">rsync's GitHub page</a>.
+
+<li><p>The <a href="https://git.samba.org/?p=rsync.git">Samba git repo</a> is also available,
+though it might lag behind the GitHub repo every now and then.
+
+</ul>
+
+Once you have the source, read the <a
+href="https://download.samba.org/pub/rsync/INSTALL">INSTALL.md</a> file for
+details on some development libraries that you will need to build it.
+
+<h2>The GPG Signing Key</h2>
+
+The GPG signing key that is used to sign the release files is available from the public pgp key-server
+network. If you have automatic key-fetching enabled, just running a normal
+"gpg --verify" will grab my key automatically.
+Or, feel free to grab <a href="https://opencoder.net/WayneDavison.key">the gpp
+ key for Wayne Davison</a> manually.<p>
+
+From 3.4.0 and later releases will be signed by Andrew
+Tridgell. Please fetch the key for andrew@tridgell.net from https://keys.openpgp.org/
+
+<h2>Binaries</h2>
+
+<p>Precompiled binaries are available in most modern OS distributions, so
+you should first check if you can install an rsync package via your
+standard package-install tools for your OS.
+
+<h3>Ubuntu</h3>
+
+<p>The rsync project maintains a Launchpad PPA that tracks upstream stable
+releases for the currently supported Ubuntu series (jammy 22.04 LTS,
+noble 24.04 LTS, questing 25.10, resolute 26.04 LTS). This is the
+fastest way to get the latest upstream rsync on Ubuntu without waiting
+for the distro to update its packaged version:
+
+<p><code><small>sudo add-apt-repository ppa:rsyncproject/rsync<br>
+sudo apt update && sudo apt install rsync</small></code>
+
+<p>See the <a href="https://launchpad.net/~rsyncproject/+archive/ubuntu/rsync">PPA page on Launchpad</a>
+for build status across architectures.
+
+<p>The <a href="https://github.com/RsyncProject/rsync/actions">GitHub Actions
+page</a> has build events that each generate a few binary artifact zip files
+(just click through via the build's title to see them). The actions page is
+also linked via the various green build-status icons on the web pages here.
+These builds use the newest libraries, such as xxhash checksums and zstd
+compression, and are dynamically linked, so you may need to install some
+official library packages for your distribution. If you're curious how the
+build was done, you can look at the build rules in the "Workflow file" tab.
+See the <a href="https://download.samba.org/pub/rsync/INSTALL">INSTALL.md</a>
+file for some package name hints, though you can use the non-devel versions of
+the various lib packages and ignore the gcc/autoconf/awk packages.
+
+<p>There are also packages available from some <b>3rd-parties</b> (note that we
+cannot vouch for 3rd parties, so use a source that you trust):
+
+<ul>
+
+<li><p><a href="https://www.cygwin.com/">Cygwin</a> is a Posix runtime for MS
+Windows that includes rsync among their many packages.</p></li>
+
+<li><p><a href="https://www.itefix.net/cwrsync">cwRsync</a> is a native
+packaging of rsync for MS Windows (they appear to only provide paid releases,
+though).</p></li>
+
+</ul>
+
+<h2>Git source vs Release files</h2>
+
+The release tar files come with a few generated files that are not checked in to git.
+These mainly include the man pages and the configure related files. To make use of
+the git-derived files you will need autoconf, autoheader, and a version of python3
+that has the commonmark lib (OR cmarkgfm). If you have trouble with setting up the
+those required files, you can try running "./prepare-source fetchgen" to grab the
+very latest generated files that were created from the latest commit into the master
+branch.
+
+<p> <b>Note:</b> Since the source repository is a work in progress it may, at
+times, not compile or fail in various ways, though it is usually pretty good.
+
+<h2>Source repository patches</h2>
+
+<p>There are also various patch files in the "rsync-patches" repository that
+represent either some work-in-progress features or features that are considered
+to be a little too fringe-interest for the main release. See the github link
+above for how to look around at what is available, or snag a release tar file.
+The maintainer like to put the rsync-patches dir into his rsync checkout as a
+directory named "patches" and has some helper scripts for how to use local git
+branches to test and update the diffs.
+
+<!--#include virtual="footer.html" -->
--- /dev/null
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
+<HTML>
+<HEAD>
+<TITLE>rsync examples</TITLE>
+</HEAD>
+<!--#include virtual="header.html" -->
+
+<H2 align="center">rsync examples</H2>
+
+If you have an interesting example of how you use rsync then please
+submit it to the
+<A HREF="mailto:rsync-bugs@samba.org">rsync-bugs@samba.org</A>
+for inclusion on this page.
+
+<h2>backup to a central backup server with 7 day incremental</h2>
+
+<pre><small>
+<!--#include virtual="backup.txt" -->
+</small></pre>
+
+<H2>backup to a spare disk</H2>
+
+<pre><small>
+<!--#include virtual="horus.txt" -->
+</small></pre>
+
+<H2>mirroring vger CVS tree</H2>
+
+<pre><small>
+<!--#include virtual="vger.txt" -->
+</small></pre>
+
+<H2>automated backup at home</H2>
+
+<pre><small>
+<!--#include virtual="susan.txt" -->
+</small></pre>
+
+<H2>Fancy footwork with remote file lists</H2>
+
+<pre><small>
+One little known feature of rsync is the fact that when run over a
+remote shell (such as rsh or ssh) you can give any shell command as
+the remote file list. The shell command is expanded by your remote
+shell before rsync is called. For example, see if you can work out
+what this does:
+
+ rsync -avR remote:'`find /home -name "*.[ch]"`' /tmp/
+
+note that that is backquotes enclosed by quotes (some browsers don't
+show that correctly).
+</small></pre>
+
+<!--#include virtual="footer.html" -->
--- /dev/null
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
+<HTML>
+<HEAD>
+<TITLE>rsync features</TITLE>
+</HEAD>
+<!--#include virtual="header.html" -->
+
+<H2 align="center">rsync features</H2>
+
+rsync is a file transfer program for Unix systems. rsync uses the
+"rsync algorithm" which provides a very fast method for bringing
+remote files into sync. It does this by sending just the differences
+in the files across the link, without requiring that both sets of
+files are present at one of the ends of the link beforehand. <p>
+
+Some features of rsync include
+
+<ul>
+<li> can update whole directory trees and filesystems
+<li> optionally preserves symbolic links, hard links, file ownership,
+permissions, devices and times
+<li> requires no special privileges to install
+<li> internal pipelining reduces latency for multiple files
+<li> can use rsh, ssh or direct sockets as the transport
+<li> supports <A HREF="http://dslab.lzu.edu.cn:8080/members/wangbj/wangbaojun/howtos/rsync-mirror-HOWTO/rsync-mirroring02.html">anonymous rsync</A> which is ideal for mirroring
+</ul>
+
+<!--#include virtual="footer.html" -->
--- /dev/null
+P /badge.svg
+H /.git/
+H /.gitignore
+H /bin/
+H /upload/
+P /USE_THE_GIT_rsync-web_REPO_NOW
+H /filt
+H *.swp
+- /netware/
+- /pre-change
+- /rsync-and-debian/
+- /index.html-*
+- *~
--- /dev/null
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
+<HTML>
+<HEAD>
+<TITLE>rsync search</TITLE>
+</HEAD>
+<!--#include virtual="header.html" -->
+
+<H2 align="center">Search the rsync web pages</H2>
+
+<p>Searching samba.org will find results in the main web pages and in the
+archived mailing-list pages too.
+
+<!--#include virtual="footer.html" -->
--- /dev/null
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
+<HTML>
+<HEAD>
+<TITLE>rsync firewall tunneling</TITLE>
+</HEAD>
+<!--#include virtual="header.html" -->
+
+<H2 align="center">Using rsync through a firewall</H2>
+
+<p>If you have a setup where there is no way to directly connect two
+systems for an rsync transfer, there are several ways to get a firewall
+system to act as an intermediary in the transfer.
+
+<p>This first method should work for any remote-shell (e.g. ssh, rsh, etc).
+The other methods are all targeted at ssh (which has a lot of flexibility
+in making a tunneled connection possible).
+
+<h4>Method 1 -- should work with any remote-shell</h4>
+
+<p>Use your remote shell (e.g. ssh) to access the middle system and have it
+use a remote shell to hop over to the actual target system.
+
+<p>To effect this extra hop, you'll need to make sure that the remote-shell
+connection from the middle system to the target system does not involve any
+tty-based user interaction (such as prompting for a password) because there
+is no way for the middle system to access the local user's tty.
+
+<p>One way that should work for all remote-shell programs is to enable host-based
+authentication, which would allow all connections from the middle system to
+the target system to succeed (when the username remains the same).
+However, this may not be a desirable setup.
+
+<p>A better method that works with ssh (and is very safe) is to setup an ssh
+key (see the ssh-keygen manpage) and ensure that ssh-agent forwarding is turned
+on in your ssh client config (e.g. "ForwardAgent yes"). You would put
+the public version of your key onto the middle and target systems (in the
+~/.ssh/authorized_keys file), and the private key on your local system (which
+I recommend you encrypt). With this setup, a series of ssh connections that
+starts from the system where your private key is available will auto-authorize
+(after a pass-phrase prompt on the first system if your key is encrypted).
+See also ssh-agent for a way to keep a public key unlocked in memory for an
+extended time, and the keychain project for a way to manage a system-wide,
+per-user ssh-agent.
+
+<p>To test that you have things setup right, first test a series of
+remote-shell connections outside of rsync. A command such as the following
+should work without issuing multiple prompts (use the appropriate remote-shell
+and the substitute the real hostnames for "middle" and "target", of course):
+
+<blockquote><pre>ssh middle ssh target uptime</pre></blockquote>
+
+<p>If you get a password/passphrase prompt to get into the first ("middle")
+system that's fine, but the extra hop needs to occur without any extra user
+interaction.
+
+<p>Once that's done, you can do an rsync copy like this:
+
+<blockquote><pre>rsync -av -e "ssh middle ssh" target:/src/ /dest/</pre></blockquote>
+
+<h4>Method 2 -- requires ssh and nc (netcat)</h4>
+
+<p>Assuming you're using ssh as your remote shell, you can configure ssh to
+use a proxy command to get to the remote host you're interested in reaching.
+Doing this will allow the multi-hop connection to work with rsync, even if
+both hosts prompt for a password -- this is because both ssh connections
+originate from the localhost, and thus both instances of ssh have access to
+the local tty to use for an out-of-band password prompt.
+
+<p>Here is an example config for your ~/.ssh/config file (substitute "target",
+"target_user", and "middle" as appropriate):
+
+<blockquote><pre>Host target
+ ProxyCommand nohup ssh middle nc -w1 %h %p
+ User target_user
+</pre></blockquote>
+
+<p>This proxy setup uses "ssh" to login to the firewall system ("middle") and
+uses "nc" (netcat) to connect to the target host ("target") using the default
+port number. The use of "nohup" silences a warning at the end of the run, and
+the "-w1" option tells nc to shut down when the connection closes.
+
+<p>With this done, you can run a normal-looking rsync command to "target" that
+will run the proxy command to get through the firewall system:
+
+<blockquote><pre>rsync -av /src/ target:/dest/</pre></blockquote>
+
+<h4>Method 3 -- an alternate ssh method for those without nc (netcat)</h4>
+
+<p>Assuming you're using ssh as your remote shell, you can configure ssh to
+forward a local port through your middle system to the ssh port (22) on the
+target system. This method does not require the use of nc (it uses only
+ssh to effect the extra hop), but otherwise it is similar to, but slightly
+less convenient than, method 2.
+
+<p>The first thing we need is an ssh configuration that will allow us to
+connect to the forwarded port as if we were connecting to the target
+system, and we need ssh to know what we're doing so that it doesn't
+complain about the host keys being wrong. We can do this by adding this
+section to your ~/.ssh/config file (substitute "target" and "target_user"
+as appropriate, but leave "localhost" unchanged):
+
+<blockquote><pre>Host target
+ HostName localhost
+ Port 2222
+ HostKeyAlias target
+ User target_user
+</pre></blockquote>
+
+<p>Next, we need to enable the port forwarding:
+
+<blockquote><pre>ssh -fN -l middle_user -L 2222:target:22 middle</pre></blockquote>
+
+<p>What this does is cause a connection to port 2222 on the local system to
+get tunneled to the middle system and then turn into a connection to the
+target system's port 22. The -N option tells ssh not to start a shell on
+the remote system, which works with modern ssh versions (you can run a
+sleep command if -N doesn't work). The -f option tells ssh to put the
+command in the background after any password/passphrase prompts.
+
+<p>With this done, you could run a normal-looking rsync command to "target"
+that would use a connection to port 2222 on localhost automatically:
+
+<blockquote><pre>rsync -av target:/src/ /dest/</pre></blockquote>
+
+<p><b>Note:</b> starting an ssh tunnel allows anyone on the source system
+to connect to the localhost port 2222, not just you, but they'd still need
+to be able to login to the target system using their own credentials.
+
+<h4>Method 4 -- for using rsync in daemon-mode (requires nc)</h4>
+
+<p>Install and configure an rsync daemon on the target and use ssh and nc
+to send the socket data to the remote host.
+
+<blockquote><pre>RSYNC_CONNECT_PROG='ssh -l middle_user middle nc %H 873' \
+ rsync daemonuser@target::module/src/ /dest/</pre></blockquote>
+
+<p>(You can also export that variable into your environment if you want to
+perform a series of daemon rsync commands through the same middle host.)
+
+<p>This command takes advantage of the RSYNC_CONNECT_PROG environment
+variable, which tells rsync to pipe its socket data to an external program
+in place of making a direct socket connection. The command specifed above
+uses ssh to run the nc (netcat) command on the middle host, which forwards
+all socket data to port 873 on the target host (%H). The "%H" will be
+substituted with the target host from the rsync command as long as you're
+running rsync 3.0.0 or newer (you'll need to replace %H with the actual
+target hostname for earlier rsync versions, which makes the hostname
+specified in the rsync command superfluous).
+
+<h4>Method 5 -- for using rsync in daemon-mode (for those without nc)</h4>
+
+<p>Install and configure an rsync daemon on the target and use an ssh
+tunnel to reach the rsync sever. This is similar to method 3, but it
+tunnels the daemon port for those that prefer to use an rsync daemon.
+
+<p>(Note that this method also works to tunnel a socket connection
+directly to a destination system if you use "localhost" as the target
+hostname and your destination system's name as the middle hostname.)
+
+<p>Installing the rsync daemon is beyond the scope of this document, but
+see the rsyncd.conf manpage for more information. Keep in mind that you
+don't need to be root to run an rsync daemon as long as you don't use a
+protected port.
+
+<p>Once your rsync daemon is up and running, you build an ssh tunnel
+through your middle system like this:
+
+<blockquote><pre>ssh -fN -l middle_user -L 8873:target:873 middle</pre></blockquote>
+
+<p>What this does is cause a connection to port 8873 on the local system to
+turn into a connection from the middle system to the target system on port
+873. (Port 873 is the normal port for an rsync daemon.) The -N option
+tells ssh not to start a shell on the remote system, which works with
+modern ssh versions (you can run a sleep command if -N doesn't work). The
+-f option tells ssh to put the command in the background after any
+password/passphrase prompts.
+
+<p>Now when an rsync command is executed with a daemon-mode command-line
+syntax to the local system, the conversation is directed to the target
+system. For example:
+
+<blockquote><pre>rsync -av --port 8873 localhost::module/src/ dest/
+rsync -av rsync://localhost:8873/module/src/ dest/</pre></blockquote>
+
+<p><b>Note:</b> starting an ssh tunnel allows anyone on the source system
+to connect to the localhost port 8873, not just you, so you may want to
+enable username/password restrictions on your rsync daemon.
+
+<!--#include virtual="footer.html" -->
--- /dev/null
+</td>
+</tr>
+
+ <TR ALIGN="center">
+ <TD><BR><a name="search"></a><img src="bar1.jpg" WIDTH="493" HEIGHT="26" BORDER="0" alt="=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=">
+
+<!-- SiteSearch Google -->
+<form action="https://www.google.com/cse" id="cse-search-box"><div>
+<input type="hidden" name="cx" value="partner-pub-1444957896811922:vxjk7n-bst5" />
+<input type="hidden" name="ie" value="ISO-8859-1" />
+<input type="text" name="q" size="31" />
+<input type="submit" name="sa" value="Search" />
+</div></form>
+<script type="text/javascript" src="https://www.google.com/cse/brand?form=cse-search-box&lang=en"></script>
+<!-- SiteSearch Google -->
+
+ </TD>
+ </TR>
+</TABLE>
--- /dev/null
+<BODY BGCOLOR="#ffffff" TEXT="#000000"
+style="margin-top: 0">
+<TABLE BORDER=0 WIDTH="640" ALIGN="CENTER">
+ <tr VALIGN="middle">
+ <td ALIGN="left">
+ <ul>
+ <li><small><a href=".">home</a></small>
+ <li><small><a href="FAQ.html">FAQ</a></small>
+ <li><small><a href="resources.html">resources</a></small>
+ <li><small><a href="features.html">features</a></small>
+ <li><small><a href="examples.html">examples</a></small>
+ <li><small><a href="lists.html">mailing lists</a></small>
+ </ul>
+ </td>
+ <td align="center">
+ <a href="."><img src="newrsynclogo.jpg" border="0" alt="rsync"></a>
+ </td>
+ <td align="left">
+ <ul>
+ <li><small><a href="bug-tracking.html">bug-tracking</a></small>
+ <li><small><a href="issues.html">current issues and debugging</a></small>
+ <li><small><a href="download.html">download</a></small>
+ <li><small><a href="documentation.html">documentation</a></small>
+ <li><small><a href="find.html">search</a></small>
+ </ul>
+ </td>
+ </tr>
+
+ <TR ALIGN="center">
+ <TD COLSPAN="3">
+ <img src="bar1.jpg" WIDTH="493" HEIGHT="26"
+ BORDER="0"
+ alt="=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=">
+ </TD>
+ </TR>
+</TABLE>
+
+<TABLE BORDER=0 WIDTH="640" ALIGN="CENTER">
+ <tr VALIGN="middle">
+ <td ALIGN="left">
--- /dev/null
+I do local backups on several of my machines using rsync. I have an
+extra disk installed that can hold all the contents of the main
+disk. I then have a nightly cron job that backs up the main disk to
+the backup. This is the script I use on one of those machines.
+
+ #!/bin/sh
+
+ export PATH=/usr/local/bin:/usr/bin:/bin
+
+ LIST="rootfs usr data data2"
+
+ for d in $LIST; do
+ mount /backup/$d
+ rsync -ax --exclude fstab --delete /$d/ /backup/$d/
+ umount /backup/$d
+ done
+
+ DAY=`date "+%A"`
+
+ rsync -a --delete /usr/local/apache /data2/backups/$DAY
+ rsync -a --delete /data/solid /data2/backups/$DAY
+
+
+
+The first part does the backup on the spare disk. The second part
+backs up the critical parts to daily directories. I also backup the
+critical parts using a rsync over ssh to a remote machine.
+
--- /dev/null
+<html>
+<head>
+<title>How Rsync Works</title>
+</head>
+<body>
+<h1 align="center">How Rsync Works<br>A Practical Overview</h1>
+<h2 align="center">Foreword</h2>
+<p>
+The original
+<a href="http://rsync.samba.org/tech_report/">Rsync technical report</a>
+and
+Andrew Tridgell's
+<a href="http://samba.org/%7Etridge/phd_thesis.pdf">Phd thesis (pdf)</a>
+Are both excellent documents for understanding the
+theoretical mathematics and some of the mechanics of the rsync algorithm.
+Unfortunately they are more about the theory than the
+implementation of the rsync utility (hereafter referred to as
+Rsync).
+<p>
+In this document I hope to describe...
+<ul>
+ <li>A non-mathematical overview of the rsync algorithm.
+ <li>How that algorithm is implemented in the rsync utility.
+ <li>The protocol, in general terms, used by the rsync utility.
+ <li>The identifiable roles the rsync processes play.
+</ul>
+<p>
+This document be able to serve as a guide for programmers
+needing something of an entré into the source code but the
+primary purpose is to give the reader a foundation from
+which he may understand
+<ul>
+ <li>Why rsync behaves as it does.
+ <li>The limitations of rsync.
+ <li>Why a requested feature is unsuited to the code-base.
+</ul>
+<p>
+This document describes in general terms the construction
+and behaviour of Rsync. In some cases details and exceptions
+that would contribute to specific accuracy have
+been sacrificed for the sake meeting the broader goals.
+<h2 align="center">Processes and Roles</h2>
+<p>
+When we talk about Rsync we use specific terms to refer to
+various processes and their roles in the task performed by
+the utility. For effective communication it is important that we
+all be speaking the same language; likewise it is important
+that we mean the same things when we use certain terms in a
+given context. On the rsync mailing list there is often
+some confusion with regards to role and processes. For
+these reasons I will define a few terms
+used in the role and process contexts that will be used henceforth.
+
+<table cellspacing="20"><tr valign="top">
+<td>client
+</td><td>role
+</td><td>
+ The client initiates the synchronisation.
+</td></tr><tr valign="top">
+<td>server
+</td><td>role
+</td><td>
+ The remote rsync process or system to which the
+ client connects either within a local transfer, via
+ a remote shell or via a network socket.
+ <p>
+ This is a general term and should not be confused with the daemon.
+</td></tr><tr valign="top">
+<td>
+</td><td>
+</td><td bgcolor="#dddddd">
+ Once the connection between the client and server is established
+ the distinction between them is superseded by the
+ sender and receiver roles.
+</td></tr><tr valign="top">
+<td>daemon
+</td><td>Role and process
+</td><td>
+ An Rsync process that awaits connections from
+ clients. On a certain platform this would be called a
+ service.
+</td></tr><tr valign="top">
+<td>remote shell
+</td><td>role and set of processes
+</td><td>
+ One or more processes that provide connectivity
+ between an Rsync client and an Rsync server on a
+ remote system.
+</td></tr><tr valign="top">
+<td>sender
+</td><td>role and process
+</td><td>
+ The Rsync process that has access to the source
+ files being synchronised.
+</td></tr><tr valign="top">
+<td>receiver
+</td><td>role and process
+</td><td>
+ As a role the receiver is the destination system.
+ As a process the receiver is the process that
+ receives update data and writes it to disk.
+</td></tr><tr valign="top">
+<td>generator
+</td><td>process
+</td><td>
+ The generator process identifies changed files and
+ manages the file level logic.
+</td></tr></table>
+<p>
+<h2 align="center">Process Startup</h2>
+<p>
+When an Rsync client is started it will first establish a
+connection with a server process. This connection may be
+through pipes or over a network socket.
+<p>
+When Rsync communicates with a remote non-daemon server via
+a remote shell the startup method is to fork the remote
+shell which will start an Rsync server on the remote system.
+Both the Rsync client and server are communicating via pipes
+through the remote shell. As far as the rsync processes are
+concerned there is no network.
+In this mode the rsync options for the server process are
+passed on the command-line that is used to start the remote
+shell.
+<p>
+When Rsync is communicating with a daemon it is
+communicating directly with a network socket. This is the
+only sort of Rsync communication that could be called
+network aware.
+In this mode the rsync options must be sent over the socket, as
+described below.
+<p>
+At the very start of the communication between the client
+and the server, they each send the maximum protocol version
+they support to the other side.
+Each side then uses the minimum value as the the protocol
+level for the transfer.
+If this is a daemon-mode connection, rsync options are sent
+from the client to the server. Then, the exclude list is
+transmitted. From this point onward the
+client-server relationship is relevant only with regards
+to error and log message delivery.
+<p>
+Local Rsync jobs (when the source and destination are both on locally
+mounted filesystems) are done exactly like a push. The
+client, which becomes the sender, forks a server process to
+fulfill the receiver role. The client/sender and
+server/receiver communicate with each other over pipes.
+<h2 align="center">The File List</h2>
+The file list includes not only the pathnames but also
+ownership, mode, permissions, size and modtime.
+If the --checksum option has been specified it also includes
+the file checksums.
+<p>
+The first thing that happens once the startup has completed
+is that the sender will create the file list.
+While it is being built, each entry is transmitted to the
+receiving side in a network-optimised way.
+<p>
+When this is done, each side sorts the file list lexicographically by path
+relative to the base directory of the transfer.
+(The exact sorting algorithm varies depending on what protocol
+version is in effect for the transfer.)
+Once that has happened all references to files
+will be done by their index in the file list.
+<p>
+If necessary the sender follows the file list with id→name
+tables for users and groups which the receiver will use to
+do a id→name→id translation for every file in the file
+list.
+<p>
+After the file list has been received by the receiver, it
+will fork to become the generator and receiver pair
+completing the pipeline.
+<h2 align="center">The Pipeline</h2>
+Rsync is heavily pipelined. This means that it is a set of
+processes that communicate in a (largely) unidirectional
+way. Once the file list has been shared the pipeline
+behaves like this:
+<blockquote>
+ generator → sender → receiver
+</blockquote>
+<p>
+The output of the generator is input for the sender and the
+output of the sender is input for the receiver.
+Each process runs independently and is delayed only when the
+pipelines stall or when waiting for disk I/O or CPU resources.
+<h2 align="center">The Generator</h2>
+<p>
+The generator process compares the file list with its local
+directory tree. Prior to beginning its primary function, if
+--delete has been specified, it will first identify local
+files not on the sender and delete them on the receiver.
+<p>
+The generator will then start walking the file list. Each
+file will be checked to see if it can be skipped. In the
+most common mode of operation files are not skipped if the
+modification time or size differs. If --checksum was
+specified a file-level checksum will be created and
+compared. Directories, device nodes and symlinks are not
+skipped. Missing directories will be created.
+<p>
+If a file is not to be skipped, any existing version on the
+receiving side becomes the "basis file" for the transfer, and is
+used as a data source that will help to eliminate matching data
+from having to be sent by the sender. To effect this remote
+matching of data, block checksums are created for the basis file
+and sent to the sender immediately following the file's index
+number.
+An empty block checksum set is sent for new files and if
+--whole-file was specified.
+<p>
+The block size and, in later versions, the size of the
+block checksum are calculated on a per file basis according
+to the size of that file.
+<h2 align="center">The Sender</h2>
+The sender process reads the file index numbers and associated
+block checksum sets one at a time from the generator.
+<p>
+For each file id the generator sends it will store the
+block checksums and build a hash index of them for rapid lookup.
+<p>
+Then the local file is read and a checksum is
+generated for the block beginning with the first byte of the
+local file. This block checksum is looked for in the
+set that was sent by the generator, and if no match is found,
+the non-matching byte will be appended to the non-matching data
+and the block starting at the next byte will be compared.
+This is what
+is referred to as the “rolling checksum”
+<p>
+If a block checksum match is found it is considered a
+matching block and any accumulated non-matching data will be
+sent to the receiver followed by the offset and length in
+the receiver's file of the matching block and the block
+checksum generator will be advanced to the next byte after
+the matching block.
+<p>
+Matching blocks can be identified in this way even if
+the blocks are reordered or at different offsets.
+This process is the very heart of the rsync algorithm.
+<p>
+In this way, the sender will give the receiver instructions for
+how to reconstruct the source file into a new destination file.
+These instructions detail all the matching data that can be
+copied from the basis file (if one exists for the transfe),
+and includes any raw data that was not available locally.
+At the end of each file's processing a whole-file
+checksum is sent and the sender proceeds with the next
+file.
+<p>
+Generating the rolling checksums and searching for matches
+in the checksum set sent by the generator require a good
+deal of CPU power. Of all the rsync processes it is the
+sender that is the most CPU intensive.
+<h2 align="center">The Receiver</h2>
+<p>
+The receiver will read from the sender data for each file
+identified by the file index number. It will open the local
+file (called the basis) and will create a temporary file.
+<p>
+The receiver will expect to read non-matched data and/or to match
+records all in sequence for the final file contents. When
+non-matched data is read it will be written to the
+temp-file. When a block match record is received the
+receiver will seek to the block offset in the basis file
+and copy the block to the temp-file. In this way the
+temp-file is built from beginning to end.
+<p>
+The file's checksum is generated as the temp-file is built.
+At the end of the file, this checksum is compared with the
+file checksum from the sender. If the file checksums do not
+match the temp-file is deleted. If the file fails once it
+will be reprocessed in a second phase, and if it fails twice
+an error is reported.
+<p>
+After the temp-file has been completed, its ownership and
+permissions and modification time are set. It is then
+renamed to replace the basis file.
+<p>
+Copying data from the basis file to the temp-file make the
+receiver the most disk intensive of all the rsync processes.
+Small files may still be in disk cache mitigating this but
+for large files the cache may thrash as the generator has
+moved on to other files and there is further latency caused
+by the sender. As
+data is read possibly at random from one file and written to
+another, if the working set is larger than the disk cache,
+then what is called a seek storm can occur, further
+hurting performance.
+<h2 align="center">The Daemon</h2>
+The daemon process, like many daemons, forks for every
+connection. On startup, it parses the rsyncd.conf file
+to determine what modules exist and to set the global options.
+<p>
+When a connection is received for a defined module the
+daemon forks a new child process to handle the connection.
+That child process then reads the rsyncd.conf file to set
+the options for the requested module, which may chroot to the
+module path and may drop setuid and setgid for the
+process. After that it will behave just like any other
+rsync server process adopting either a sender or receiver
+role.
+<h2 align="center">The Rsync Protocol</h2>
+<p>
+A well-designed communications protocol has a number of
+characteristics.
+<ul>
+ <li>Everything is sent in well defined packets with
+ a header and an optional body or data payload.
+ <li>In each packet's header a type and or command
+ specified.
+ <li>Each packet has a definite length.
+</ul>
+<p>
+In addition to these characteristics, protocols have varying degrees of
+statefulness, inter-packet independence, human readability,
+and the ability to reestablish a disconnected session.
+<p>
+Rsync's protocol has none of these good characteristics. The data is
+transferred as an unbroken stream of bytes. With the
+exception of the unmatched file-data, there are no length
+specifiers nor counts. Instead the meaning of each byte is
+dependent on its context as defined by the protocol level.
+<p>
+As an example, when the sender is sending the file list it
+simply sends each file list entry and terminates the list
+with a null byte. Within the file list entries, a bitfield
+indicates which fields of the structure to expect and those
+that are variable length strings are simply null terminated.
+The generator sending file numbers and block checksum sets
+works the same way.
+<p>
+This method of communication works quite well on reliable
+connections and it certainly has less data overhead than the
+formal protocols. It unfortunately makes the protocol
+extremely difficult to document, debug or extend.
+Each version of the protocol will have subtle differences on
+the wire that can only be anticipated by knowing the exact
+protocol version.
+<h2 align="center">notes</h2>
+This document is a work in progress. The author expects
+that it has some glaring oversights and some portions that may be
+more confusing than enlightening for some readers. It is
+hoped that this could evolve into a useful reference.
+<p>
+Specific suggestions for improvement are welcome, as would be a
+complete rewrite.
+</body>
+</html>
--- /dev/null
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
+<HTML>
+<HEAD>
+<TITLE>rsync</TITLE>
+<style>
+.security { color: red; }
+h3 { margin-bottom: 0px; }
+.date { color: #D25A0B; }
+</style>
+</HEAD>
+<!--#include virtual="header.html" -->
+
+<H2 align="center">Welcome to the rsync web pages</H2>
+
+rsync is an <A HREF="https://www.opensource.org/">open source</A>
+utility that provides fast incremental file transfer. rsync is freely
+available under the <A HREF="GPL.html">GNU General Public
+License</A> and is currently being maintained by
+<a href="email:<tridge@samba.org>">Andrew Tridgell</a>.
+
+<p>A full changelog of all the releases, including upcoming releases, is in the
+<a href="https://download.samba.org/pub/rsync/NEWS">NEWS file</a>.
+
+<div style="float: right">
+<a href="https://github.com/RsyncProject/rsync/actions">
+<img src="badge.svg">
+</a></div>
+
+<h3>Rsync version 3.4.3 released</h3>
+<i class=date>May 20th, 2026</i>
+
+<p>Rsync version 3.4.3 has been released. This is a major security release.
+
+<p>See the <a href="https://download.samba.org/pub/rsync/NEWS#3.4.3">3.4.3 NEWS</a> for a detailed changelog.
+The latest manpages are also available for:<ul>
+<li><a href="https://download.samba.org/pub/rsync/rsync.1"><b>rsync</b>(1)</a>
+<li><a href="https://download.samba.org/pub/rsync/rsync-ssl.1"><b>rsync-ssl</b>(1)</a>
+<li><a href="https://download.samba.org/pub/rsync/rsyncd.conf.5"><b>rsyncd.conf</b>(5)</a>
+<li><a href="https://download.samba.org/pub/rsync/rrsync.1"><b>rrsync</b>(1)</a>
+</ul>
+
+<p>The source tar is available here:
+<b><a href="https://download.samba.org/pub/rsync/src/rsync-3.4.3.tar.gz">rsync-3.4.3.tar.gz</a>
+(<a href="https://download.samba.org/pub/rsync/src/rsync-3.4.3.tar.gz.asc">signature</a>)</b>,
+and the diffs from version 3.4.2 are available here:
+<b><a href="https://download.samba.org/pub/rsync/src-diffs/rsync-3.4.2-3.4.3.diffs.gz">rsync-3.4.2-3.4.3.diffs.gz</a>
+(<a href="https://download.samba.org/pub/rsync/src-diffs/rsync-3.4.2-3.4.3.diffs.gz.asc">signature</a>)</b>.
+
+<h3>Rsync version 3.4.2 released</h3>
+<i class=date>April 28th, 2026</i>
+
+<p>Rsync version 3.4.2 has been released.
+
+<p>See the <a href="https://download.samba.org/pub/rsync/NEWS#3.4.2">3.4.2 NEWS</a> for a detailed changelog.
+The latest manpages are also available for:<ul>
+<li><a href="https://download.samba.org/pub/rsync/rsync.1"><b>rsync</b>(1)</a>
+<li><a href="https://download.samba.org/pub/rsync/rsync-ssl.1"><b>rsync-ssl</b>(1)</a>
+<li><a href="https://download.samba.org/pub/rsync/rsyncd.conf.5"><b>rsyncd.conf</b>(5)</a>
+<li><a href="https://download.samba.org/pub/rsync/rrsync.1"><b>rrsync</b>(1)</a>
+</ul>
+
+<p>The source tar is available here:
+<b><a href="https://download.samba.org/pub/rsync/src/rsync-3.4.2.tar.gz">rsync-3.4.2.tar.gz</a>
+(<a href="https://download.samba.org/pub/rsync/src/rsync-3.4.2.tar.gz.asc">signature</a>)</b>,
+and the diffs from version 3.4.1 are available here:
+<b><a href="https://download.samba.org/pub/rsync/src-diffs/rsync-3.4.1-3.4.2.diffs.gz">rsync-3.4.1-3.4.2.diffs.gz</a>
+(<a href="https://download.samba.org/pub/rsync/src-diffs/rsync-3.4.1-3.4.2.diffs.gz.asc">signature</a>)</b>.
+
+<h3>Rsync version 3.4.1 released</h3>
+<i class=date>January 15th, 2025</i>
+
+<p>Rsync version 3.4.1 has been released.
+This is a fix for some regressions in 3.4.0
+
+<p>See the <a href="https://download.samba.org/pub/rsync/NEWS#3.4.1">3.4.1 NEWS</a> for a detailed changelog.
+The latest manpages are also available for:<ul>
+<li><a href="https://download.samba.org/pub/rsync/rsync.1"><b>rsync</b>(1)</a>
+<li><a href="https://download.samba.org/pub/rsync/rsync-ssl.1"><b>rsync-ssl</b>(1)</a>
+<li><a href="https://download.samba.org/pub/rsync/rsyncd.conf.5"><b>rsyncd.conf</b>(5)</a>
+<li><a href="https://download.samba.org/pub/rsync/rrsync.1"><b>rrsync</b>(1)</a>
+</ul>
+
+<p>The source tar is available here:
+<b><a href="https://download.samba.org/pub/rsync/src/rsync-3.4.1.tar.gz">rsync-3.4.1.tar.gz</a>
+(<a href="https://download.samba.org/pub/rsync/src/rsync-3.4.1.tar.gz.asc">signature</a>)</b>,
+with a tar file of the "rsync-patches" repository released in a separate file:
+<b><a href="https://download.samba.org/pub/rsync/src/rsync-patches-3.4.1.tar.gz">rsync-patches-3.4.1.tar.gz</a>
+(<a href="https://download.samba.org/pub/rsync/src/rsync-patches-3.4.1.tar.gz.asc">signature</a>)</b>,
+and the diffs from version 3.4.0 are available here:
+<b><a href="https://download.samba.org/pub/rsync/src-diffs/rsync-3.4.0-3.4.1.diffs.gz">rsync-3.4.0-3.4.1.diffs.gz</a>
+(<a href="https://download.samba.org/pub/rsync/src-diffs/rsync-3.4.0-3.4.1.diffs.gz.asc">signature</a>)</b>.
+
+
+<h3>Rsync version 3.4.0 released</h3>
+<i class=date>January 14th, 2025</i>
+
+<p>Rsync version 3.4.0 has been released.
+This is a security release, fixing several important security vulnerabilities.
+
+<p>See the <a href="https://download.samba.org/pub/rsync/NEWS#3.4.0">3.4.0 NEWS</a> for a detailed changelog.
+
+<p>The source tar is available here:
+<b><a href="https://download.samba.org/pub/rsync/src/rsync-3.4.0.tar.gz">rsync-3.4.0.tar.gz</a>
+(<a href="https://download.samba.org/pub/rsync/src/rsync-3.4.0.tar.gz.asc">signature</a>)</b>,
+with a tar file of the "rsync-patches" repository released in a separate file:
+<b><a href="https://download.samba.org/pub/rsync/src/rsync-patches-3.4.0.tar.gz">rsync-patches-3.4.0.tar.gz</a>
+(<a href="https://download.samba.org/pub/rsync/src/rsync-patches-3.4.0.tar.gz.asc">signature</a>)</b>,
+and the diffs from version 3.3.0 are available here:
+<b><a href="https://download.samba.org/pub/rsync/src-diffs/rsync-3.3.0-3.4.0.diffs.gz">rsync-3.3.0-3.4.0.diffs.gz</a>
+(<a href="https://download.samba.org/pub/rsync/src-diffs/rsync-3.3.0-3.4.0.diffs.gz.asc">signature</a>)</b>.
+
+<h3>Rsync version 3.3.0 released</h3>
+<i class=date>April 6th, 2024</i>
+
+<p>Rsync version 3.3.0 has been released.
+This is a bug-fix release.
+
+<p>See the <a href="https://download.samba.org/pub/rsync/NEWS#3.3.0">3.3.0 NEWS</a> for a detailed changelog.
+
+<p>The source tar is available here:
+<b><a href="https://download.samba.org/pub/rsync/src/rsync-3.3.0.tar.gz">rsync-3.3.0.tar.gz</a>
+(<a href="https://download.samba.org/pub/rsync/src/rsync-3.3.0.tar.gz.asc">signature</a>)</b>,
+with a tar file of the "rsync-patches" repository released in a separate file:
+<b><a href="https://download.samba.org/pub/rsync/src/rsync-patches-3.3.0.tar.gz">rsync-patches-3.3.0.tar.gz</a>
+(<a href="https://download.samba.org/pub/rsync/src/rsync-patches-3.3.0.tar.gz.asc">signature</a>)</b>,
+and the diffs from version 3.2.2 are available here:
+<b><a href="https://download.samba.org/pub/rsync/src-diffs/rsync-3.2.7-3.3.0.diffs.gz">rsync-3.2.7-3.3.0.diffs.gz</a>
+(<a href="https://download.samba.org/pub/rsync/src-diffs/rsync-3.2.7-3.3.0.diffs.gz.asc">signature</a>)</b>.
+
+<p><hr>
+<h3>Rsync version 3.2.7 released</h3>
+<i class=date>October 20th, 2022</i>
+
+<p>Rsync version 3.2.7 has been released.
+This has some new features & fixes, including various bug fixes for arg validation & filter-rule validation.
+
+<p>See the <a href="https://download.samba.org/pub/rsync/NEWS#3.2.7">3.2.7 NEWS</a> for a detailed changelog.
+
+<p>The source tar is available here:
+<b><a href="https://download.samba.org/pub/rsync/src/rsync-3.2.7.tar.gz">rsync-3.2.7.tar.gz</a>
+(<a href="https://download.samba.org/pub/rsync/src/rsync-3.2.7.tar.gz.asc">signature</a>)</b>,
+with a tar file of the "rsync-patches" repository released in a separate file:
+<b><a href="https://download.samba.org/pub/rsync/src/rsync-patches-3.2.7.tar.gz">rsync-patches-3.2.7.tar.gz</a>
+(<a href="https://download.samba.org/pub/rsync/src/rsync-patches-3.2.7.tar.gz.asc">signature</a>)</b>,
+and the diffs from version 3.2.2 are available here:
+<b><a href="https://download.samba.org/pub/rsync/src-diffs/rsync-3.2.6-3.2.7.diffs.gz">rsync-3.2.6-3.2.7.diffs.gz</a>
+(<a href="https://download.samba.org/pub/rsync/src-diffs/rsync-3.2.6-3.2.7.diffs.gz.asc">signature</a>)</b>.
+
+<p><hr>
+<h3>Rsync version 3.2.6 released</h3>
+<i class=date>September 9th, 2022</i>
+
+<p>Rsync version 3.2.6 has been released.
+This is a bug-fix release.
+
+<p>See the <a href="https://download.samba.org/pub/rsync/NEWS#3.2.6">3.2.6 NEWS</a> for a detailed changelog.
+
+<p>The source tar is available here:
+<b><a href="https://download.samba.org/pub/rsync/src/rsync-3.2.6.tar.gz">rsync-3.2.6.tar.gz</a>
+(<a href="https://download.samba.org/pub/rsync/src/rsync-3.2.6.tar.gz.asc">signature</a>)</b>,
+with a tar file of the "rsync-patches" repository released in a separate file:
+<b><a href="https://download.samba.org/pub/rsync/src/rsync-patches-3.2.6.tar.gz">rsync-patches-3.2.6.tar.gz</a>
+(<a href="https://download.samba.org/pub/rsync/src/rsync-patches-3.2.6.tar.gz.asc">signature</a>)</b>,
+and the diffs from version 3.2.2 are available here:
+<b><a href="https://download.samba.org/pub/rsync/src-diffs/rsync-3.2.5-3.2.6.diffs.gz">rsync-3.2.5-3.2.6.diffs.gz</a>
+(<a href="https://download.samba.org/pub/rsync/src-diffs/rsync-3.2.5-3.2.6.diffs.gz.asc">signature</a>)</b>.
+
+<p><hr>
+<h3>Rsync version 3.2.5 released</h3>
+<i class=date>August 14th, 2022</i>
+
+<p>Rsync version 3.2.5 has been released.
+This is a bug-fix and <a href="security.html" class=security>security</a> release.
+
+<p>See the <a href="https://download.samba.org/pub/rsync/NEWS#3.2.5">3.2.5 NEWS</a> for a detailed changelog.
+
+<p><hr>
+<h3>Rsync version 3.2.4 released</h3>
+<i class=date>April 15th, 2022</i>
+
+<p>Rsync version 3.2.4 has been released.
+Another typical release with both bug fixes and some enhancements. It also contains a
+<a href="security.html#s3_2_4" class=security>security fix</a>
+for the bundled zlib 1.2.8, which may or may not be used in your particular build configuration.
+
+<p>See the <a href="https://download.samba.org/pub/rsync/NEWS#3.2.4">3.2.4 NEWS</a>
+for a detailed changelog.
+
+<p><hr>
+<h3>Rsync version 3.2.3 released</h3>
+<i class=date>August 6th, 2020</i>
+
+<p>Rsync version 3.2.3 has been released.
+It contains a smattering of bug fixes and various enhancements.
+
+<p>See the <a href="https://download.samba.org/pub/rsync/NEWS#3.2.3">3.2.3 NEWS</a>
+for a detailed changelog.
+
+<p><hr>
+<h3>Rsync version 3.2.2 released</h3>
+<i class=date>July 4th, 2020</i>
+
+<p>Rsync version 3.2.2 has been released.
+This is a few more portability fixes, some improvements to the newest features, and some
+other simple changes. Hopefully this will be the last of these recent touch-up releases.
+
+<p>See the <a href="https://download.samba.org/pub/rsync/NEWS#3.2.2">3.2.2 NEWS</a>
+for a detailed changelog.
+
+<p><hr>
+<h3>Rsync version 3.2.1 released</h3>
+<i class=date>June 22th, 2020</i>
+
+<p>Rsync version 3.2.1 has been released.
+This is mainly a few fixes for some release issues and portability problems.
+There's also a couple new features, just for good measure.
+
+<p>See the <a href="https://download.samba.org/pub/rsync/NEWS#3.2.1">3.2.1 NEWS</a>
+for a detailed changelog.
+
+<p><hr>
+<h3>Rsync version 3.2.0 released</h3>
+<i class=date>June 19th, 2020</i>
+
+<p>Rsync version 3.2.0 has been released.
+This release has a good number of a few new features and various bug fixes.
+
+<p>See the <a href="https://download.samba.org/pub/rsync/NEWS#3.2.0">3.2.0 NEWS</a>
+for a detailed changelog.
+
+<p><hr>
+<h3>Rsync version 3.1.3 released</h3>
+<i class=date>January 28th, 2018</i>
+
+<p>Rsync version 3.1.3 has been released.
+This release has a
+<a href="security.html#s3_1_3" class=security>couple security fixes</a>,
+a few new features, and various bug fixes.
+
+<p>See the <a href="https://download.samba.org/pub/rsync/NEWS#3.1.3">3.1.3 NEWS</a>
+for a detailed changelog.
+
+<p><hr>
+<h3>Rsync version 3.1.2 released</h3>
+<i class=date>December 21st, 2015</i>
+
+<p>Rsync version 3.1.2 has been released. This is a bug-fix release.
+It includes a <a href="security.html#s3_1_2" class=security>security fix</a>
+for a transfer from a sender that you don't fully trust.
+
+<p>See the <a href="https://download.samba.org/pub/rsync/NEWS#3.1.2">3.1.2 NEWS</a>
+for a detailed changelog.
+
+<p><hr>
+<h3>Rsync version 3.1.1 released</h3>
+<i class=date>June 22nd, 2014</i>
+
+<p>Rsync version 3.1.1 has been released. This is a bug-fix release.
+
+<p>See the <a href="https://download.samba.org/pub/rsync/NEWS#3.1.1">3.1.1 NEWS</a>
+for a detailed changelog.
+
+<p><hr>
+<h3>Rsync version 3.1.0 released</h3>
+<i class=date>September 28th, 2013</i>
+
+<p>Rsync version 3.1.0 has been released. This is a
+feature release that improves performance, provides several new options, and
+fixes a few bugs along the way.
+
+<p>See the <a href="https://download.samba.org/pub/rsync/NEWS#3.1.0">3.1.0 NEWS</a>
+for a detailed changelog.
+
+<p><hr>
+<h3>Rsync version 3.0.9 released</h3>
+<i class=date>September 23th, 2011</i>
+
+<p>Rsync version 3.0.9 has been released. This is a bug-fix release.
+
+<p>See the <a href="https://download.samba.org/pub/rsync/NEWS#3.0.9">3.0.9 NEWS</a>
+for a detailed changelog.
+
+<p><hr>
+<h3>Rsync version 3.0.8 released</h3>
+<i class=date>March 26th, 2011</i>
+
+<p>Rsync version 3.0.8 has been released. This is a bug-fix release.
+
+<p>See the <a href="https://download.samba.org/pub/rsync/NEWS#3.0.8">3.0.8 NEWS</a>
+for a detailed changelog.
+
+<p><hr>
+<h3>Rsync version 3.0.7 released</h3>
+<i class=date>December 31th, 2009</i>
+
+<p>Rsync version 3.0.7 has been released. This is a bug-fix release.
+
+<p>See the <a href="https://download.samba.org/pub/rsync/NEWS#3.0.7">3.0.7 NEWS</a>
+for a detailed changelog.
+
+<p><hr>
+<h3>Rsync version 3.0.6 released</h3>
+<i class=date>May 8th, 2009</i>
+
+<p>Rsync version 3.0.6 has been released. This is a bug-fix release.
+
+<p>See the <a href="https://download.samba.org/pub/rsync/NEWS#3.0.6">3.0.6 NEWS</a>
+for a detailed changelog.
+
+<p><hr>
+<h3>Rsync version 3.0.5 released</h3>
+<i class=date>December 28th, 2008</i>
+
+<p>Rsync version 3.0.5 has been released. This is another bug-fix release.
+
+<p>See the <a href="https://download.samba.org/pub/rsync/NEWS#3.0.5">3.0.5 NEWS</a>
+for a detailed changelog.
+
+<p><hr>
+<h3>Rsync version 3.0.4 released</h3>
+<i class=date>September 6th, 2008</i>
+
+<p>Rsync version 3.0.4 has been released. This is a bug-fix release with the
+only enhancement being the adding of a way to interact with an
+overly-restrictive server that refuses rsync's behind-the-scenes use of the -e
+option.
+
+<p>See the <a href="https://download.samba.org/pub/rsync/NEWS#3.0.4">3.0.4 NEWS</a>
+for a detailed changelog.
+
+<p><hr>
+<h3>Rsync version 3.0.3 released</h3>
+<i class=date>June 29th, 2008</i>
+
+<p>Rsync version 3.0.3 has been released. This is a bug-fix release that has
+no new features (though it does have one new script in the support directory).
+
+<p>See the <a href="https://download.samba.org/pub/rsync/NEWS#3.0.3">3.0.3 NEWS</a>
+for a detailed changelog.
+
+<p><hr>
+<h3>Rsync version 3.0.2 released</h3>
+<i class=date>April 8th, 2008</i>
+
+<p>Rsync version 3.0.2 has been released. This is a
+<a href="security.html#s3_0_2" class=security>security release</a>
+that fixes a potential buffer-overflow issue.
+
+<p>See the <a href="https://download.samba.org/pub/rsync/NEWS#3.0.2">3.0.2 NEWS</a>
+for a detailed changelog.
+
+<p><hr>
+<h3>Rsync version 3.0.1 released</h3>
+<i class=date>April 3rd, 2008</i>
+
+<p>Rsync version 3.0.1 has been released. This is a bug-fix release, which also
+includes fixes/improvements for several issues in the daemon-exclude code.
+
+<p>See the <a href="https://download.samba.org/pub/rsync/NEWS#3.0.1">3.0.1 NEWS</a>
+for a detailed changelog.
+
+<p><hr>
+<h3>Rsync version 3.0.0 released</h3>
+<i class=date>March 1st, 2008</i>
+
+<p>Rsync version 3.0.0 is finally here! This is a feature release that
+also includes quite a few bug fixes.
+
+<p>The 3.0.0 version number is such a large bump up from 2.6.9 due to the
+addition of an
+incremental recursion scan (which helps a lot with large transfers) and the
+official arrival of several other new features, including ACL support, extended
+attribute support, filename character-set conversion, etc.
+
+<p>See the <a href="https://download.samba.org/pub/rsync/NEWS#3.0.0">3.0.0 NEWS</a>
+for a detailed changelog.
+
+<p><hr>
+<h3>Rsync version 2.6.9 released</h3>
+<i class=date>November 6th, 2006</i>
+
+<p>Rsync version 2.6.9 has been released. This is primarily a bug-fix
+release with a few minor new features.
+
+<p>See the <a href="https://download.samba.org/pub/rsync/NEWS#2.6.9">2.6.9 NEWS</a>
+for a detailed changelog.
+
+<p><hr>
+<h3>Rsync version 2.6.8 released</h3>
+<i class=date>April 22th, 2006</i>
+
+<p>Rsync version 2.6.8 has been released. This is a bug-fix release that
+primarily addresses an exclude problem that affected the --relative option,
+but also includes a <a href="security.html#s2_6_8" class=security>security fix</a> for
+the xattrs.diff patch (which is not an
+official part of rsync, but some packagers include it in their release).
+
+<p>See the <a href="https://download.samba.org/pub/rsync/NEWS#2.6.8">2.6.8 NEWS</a>
+for a detailed changelog.
+
+<p><hr>
+<h3>Rsync 2.6.7 released</h3>
+<i class=date>March 11th, 2006</i>
+
+<p>Rsync version 2.6.7 has been released. This release has both several new
+features and the usual accompaniment of bug fixes.
+
+<p>See the <a href="https://download.samba.org/pub/rsync/NEWS#2.6.7">2.6.7 NEWS</a>
+for a detailed changelog.
+
+<p><hr>
+<h3>Rsync 2.6.6 released</h3>
+<i class=date>July 28th, 2005</i>
+
+<p>Rsync version 2.6.6 has been released. This release is a bug-fix release
+which contains a <a href="security.html#s2_6_6" class=security>security fix</a>
+to handle a null-pointer bug that turned up in rsync's version of zlib
+1.1.4 (this is not the recent zlib 1.2.2 security fix, which did not
+affect rsync) and to squash a few other minor bugs. To deal with the
+zlib issue, rsync has been upgraded to include zlib 1.2.3.
+
+<p>See the <a href="https://download.samba.org/pub/rsync/NEWS#2.6.6">2.6.6 NEWS</a>
+for a detailed changelog.
+
+<p><hr>
+<h3>Rsync 2.6.5 released</h3>
+<i class=date>June 1st, 2005</i>
+
+<p>Rsync version 2.6.5 has been released. This release is primarily a bug-fix
+release to squash some annoying problems that made it into the (feature-filled)
+release of 2.6.4, plus a few minor enhancements.
+
+<p>See the <a href="https://download.samba.org/pub/rsync/NEWS#2.6.5">2.6.5 NEWS</a>
+for a detailed changelog.
+
+<p><hr>
+<h3>Rsync 2.6.4 released</h3>
+<i class=date>March 30th, 2005</i>
+
+<p>Rsync version 2.6.4 has been released. This release combines quite a
+few new features, some improved delete efficiency, and the usual array of
+bug fixes.
+
+<p>See the <a href="https://download.samba.org/pub/rsync/NEWS#2.6.4">2.6.4 NEWS</a>
+for a detailed changelog.
+
+<p><hr>
+<h3>Rsync 2.6.3 released</h3>
+<i class=date>September 30th, 2004</i>
+
+<p>Rsync version 2.6.3 has been released. It contains several new features
+and quite a few bug fixes, including a <a href="security.html#s2_6_3" class=security>security
+fix</a> for a patch-sanitizing bug in the daemon code.
+
+<p>See the <a href="https://download.samba.org/pub/rsync/NEWS#2.6.3">2.6.3 NEWS</a>
+for a detailed changelog.
+
+<p><hr>
+<h3>Rsync 2.6.2 released</h3>
+<i class=date>April 30th, 2004</i>
+
+<p>Rsync version 2.6.2 has been released. It is a bug-fix release that mainly
+fixes <b>a bug with the --relative option (-R) in 2.6.1</b>
+that could cause files to be transferred incorrectly. This only affected a
+source right at the root of the filesystem, such as "/" or "/*" (if you
+first "cd /" and then copy from ".", it would not tickle the bug).
+
+<p>See the <a href="https://download.samba.org/pub/rsync/NEWS#2.6.2">2.6.2 NEWS</a>
+for a detailed changelog.
+
+<p><hr>
+<h3>Rsync 2.6.1 released</h3>
+<i class=date>April 26th, 2004</i>
+
+<p>Rsync version 2.6.1 has been released. It is primarily a performance
+release that requires less memory to run, makes fewer write calls to the socket
+(lowering the system CPU time), does less string copying (lowering the user CPU
+time), and also reduces the amount of data that is transmitted over the wire.
+There have also been quite a few bug fixes, including a
+<a href="security.html#s2_6_1" class=security>security fix</a> for a daemon problem when chroot
+is not enabled. See the
+<a href="https://download.samba.org/pub/rsync/NEWS#2.6.1">2.6.1 NEWS</a>
+for a detailed changelog.
+
+<p><hr>
+<h3>One Cygwin hang-problem resolved</h3>
+
+<p>The problem with rsync hanging at the end of the transfer on
+<a href="https://www.cygwin.com/">Cygwin</a> had been previously traced to a
+signal-handling bug in their compatibility DLL. This bug appears to now be
+fixed in DLL version 1.5.7-1, and Cygwin users are reporting that upgrading the
+DLL removes the hang-at-end-of-transfer problem for their existing rsync executable.
+(Note that this doesn't solve a hang that some folks see in the middle of a
+transfer -- using daemon mode instead of ssh can work around that one.)
+
+<p><hr>
+<h3>Rsync 2.6.0 released</h3>
+<i class=date>January 1st, 2004</i>
+
+<P> Two important things to note in the new release:
+
+<ol>
+
+<li>The default remote shell is now "ssh" unless you tell configure you want to
+make something else the default.
+
+<li>Some bug fixes in the include/exclude code, while making things work
+properly, have resulted in some user-visible changes for certain wildcard
+strings. Read the BUG FIXES section in the
+<a href="https://download.samba.org/pub/rsync/NEWS#2.6.0">2.6.0 NEWS</a>
+to see if any of these changes apply to you.
+(Most people should be unaffected.)
+
+</ol>
+
+<p>One other item of note is that the oft-requested option "--files-from" is now
+available. This option lets you specify a list of files to transfer, and can
+be much more efficient than a recursive descent using include/exclude
+statements (if you know in advance what files you want to transfer). The list
+of files can come from either side of the connection, so it is possible for a
+server to provide the file-list that lets someone grab a server-specified set of
+files, for example. See the <a href="https://download.samba.org/pub/rsync/rsync.1">rsync man page</a>
+for more details.
+
+<p>See the <a href="https://download.samba.org/pub/rsync/NEWS#2.6.0">2.6.0 NEWS</a>
+for a detailed changelog.
+
+<!--#include virtual="footer.html" -->
--- /dev/null
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
+<HTML>
+<HEAD>
+<TITLE>rsync current issues</TITLE>
+<style>
+code {
+ font-family: monospace;
+ font-weight: bold;
+ white-space: pre;
+}
+pre code {
+ display: block;
+ font-weight: normal;
+}
+blockquote pre code {
+ background: #f1f1f1;
+}
+</style>
+</HEAD>
+<!--#include virtual="header.html" -->
+
+<H2 align="center">current issues and debugging</H2>
+
+<ol>
+
+<li><p><b>Q:</b>
+
+Rsync appears hung -- what should I do?
+
+<p><b>A:</b>
+
+When experiencing a hang or freeze <b>please</b> gather the following
+information before killing the rsync process:
+
+<ul>
+
+<li> The state of the send/receive queues shown with netstat on the two ends.
+
+<li> The system call that each of the 3 processes is stuck in (use truss on
+solaris, strace on Linux, etc.).
+
+</ul>
+
+<p>Try telling rsync on both sides of the connection to send messages to
+stderr, which might make the failure message visible. i.e., use:
+
+<blockquote<pre><code>
+--msgs2stderr -M--msgs2stderr
+</code></pre></blockquote>
+
+<p>That alone might get rsync to stop hanging. Also, if you're using more than
+one <code>--verbose</code> (<code>-v</code>) option then I have 2 simple words
+for you: stop it. If you need more info on what rsync is changing, using the
+<code>--itemize-changes</code> option (<code>-i</code>) and repeat it if you
+need to see unchanged files. This is a much better way to go that doesn't fill
+up the communication pipeline with a large quanity of debug messages.
+
+<p>See the "rsync-debug" script below for an example of how to grab strace
+information from the remote rsync process(es). If you need help, send email to
+the mailing list.
+
+<li><p><b>Q:</b>
+
+Why does my chrooted rsync daemon crash when doing an LDAP lookup for a user or
+group?
+
+<p><b>A:</b>
+
+There is a bug in some LDAP libraries (e.g. Fedora Core 3) where it crashes
+when someone looks up a name from inside a chrooted process (one that does not
+contain copies of the libraries to perform the lookup). This is a bug that the
+LDAP libraries will need to fix, and is out of rsync's hands. You can work
+around the problem by using the <code>--numeric-ids</code> option, turning
+chroot off, or getting rid of LDAP lookups.
+
+<li><p><b>Q:</b>
+
+Why does my transfer die with something like the following error?
+
+<blockquote><pre><code>
+rsync: error writing 4 unbuffered bytes - exiting: Broken pipe
+rsync error: error in rsync protocol data stream (code 12) at io.c(463)
+</code></pre></blockquote>
+
+<p>or
+
+<blockquote><pre><code>
+rsync: connection unexpectedly closed (24 bytes read so far)
+rsync error: error in rsync protocol data stream (code 12) at io.c(342)
+</code></pre></blockquote>
+
+<p><b>A:</b>
+
+This error tells you that the local rsync was trying to talk to the remote
+rsync, but the connection to that rsync is now gone. The thing you must
+figure out is <b>why</b>, and that can involve some investigative work.
+
+<p>It is a good idea use the <code>--msgs2stderr</code> options mentioned at
+the top of this page to get rsync to output any errors it encounters to stderr
+instead of trying to write them down the failing pipeline.
+
+<p>If the connection is via ssh (or other remote-shell command) then you should
+run some tests to make sure that you can actually run the remote rsync and that
+your shell isn't injecting extraneous output into the rsync stream. For instance,
+try running these two commands using whatever HOST (and user) options you need:
+
+<blockquote><pre><code>
+echo hi | ssh HOST cat
+ssh HOST rsync --version
+</code></pre></blockquote>
+
+<p>The first command should output just the string "hi" and nothing else. The
+second command should successfully start the remote rsync and report its version.
+
+<p>If the remote rsync is a daemon, your first step should be to look at the
+daemon's log file to see if it logged an error explaining why it aborted the
+transfer. Also double-check to ensure that the log file is setup right, as a
+wrong "log file" setting in your rsyncd.conf file can also cause this problem.
+You could also halt the daemon and run it interactively using the
+<code>--no-detach</code> and <code>--msgs2stderr</code> options and look for
+errors while someone tries the rsync copy in another window.
+
+<p>As for the cause of the remote rsync going away, there are several
+common issues that people run into:
+
+<ul>
+
+<li>The destination disk is full (remember that you need at least the
+size of the largest file that needs to be updated available in free
+disk space for the transfer to succeed).
+
+<li>An idle connection caused a router or remote-shell server to close
+the connection.
+
+<li>A network error caused the connection to be dropped.
+
+<li>The remote rsync executable wasn't found.
+
+<li>Your remote-shell setup isn't working right or isn't "clean"
+(i.e. it is sending spurious text to rsync).
+
+</ul>
+
+<p>If you think the problem might be an idle connection getting closed, you
+might be able to work around the problem by using a <code>--timeout</code>
+option (newer rsyncs send keep-alive messages during lulls). You can also
+configure ssh to send keep-alive messages when using Protocol 2 (look for
+KeepAlive, ServerAliveInterval, ClientAliveInterval, ServerAliveCountMax, and
+ClientAliveCountMax). You can also avoid some lulls by switching from
+<code>--delete</code> (aka <code>--delete-before</code>) to <code>--del</code>
+(aka <code>--delete-during</code>).
+
+<p>If you can't figure out why the failure happened, there are steps
+you can take to debug the situation. One way is to create a shell
+script on the remote system such as
+<a href="rsync-debug">this one named "rsync-debug"</a>.
+You would use the script like this:
+
+<blockquote><pre><code>
+rsync -av --rsync-path=/some/path/rsync-debug HOST:SOURCE DEST
+rsync -av --rsync-path=/some/path/rsync-debug SOURCE HOST:DEST
+</code></pre></blockquote>
+
+<p>This script enables core dumps and also logs all the OS system calls
+that lead up to the failure to a file in the /tmp dir. You can use the
+resulting files to help figure out why the remote rsync failed.
+
+<p>If you are rsyncing directly to an rsync daemon (without using a
+remote-shell transport), the above script won't have
+any effect. Instead, halt the current daemon and run a debug version
+with core-dumps enabled and (if desired) using a
+system-call tracing utility such as <i>strace</i>, <i>truss</i>, or
+<i>tusc</i>. For strace, you would do it like this (the -f option
+tells strace to follow the child processes too):
+
+<blockquote><pre><code>
+ulimit -c unlimited
+strace -f -t -s 1024 -o /tmp/rsync-$$.out rsync --daemon --no-detach
+</code></pre></blockquote>
+
+<p>Then, use a separate window to actually run the failing transfer, after
+which you can kill the debug rsync daemon (pressing Ctrl-C should do it).
+
+<p>If you are using rsync under inetd, I'd suggest temporarily disabling
+that and using the above daemon approach to debug what is going on.
+
+<li><p><b>Q:</b>
+
+Why does my connection to an rsync daemon (using the "::" syntax)
+fail immediately with an error like the following?
+
+<blockquote><pre><code>
+rsync: connection unexpectedly closed (24 bytes read so far)
+rsync error: error in rsync protocol data stream (code 12) at io.c(342)
+</code></pre></blockquote>
+
+<p><b>A:</b>
+
+Older rsync daemons (before 2.6.3) were unable to return errors that were
+generated during the option-parsing phase of the transfer. Look in the
+logfile on the server to see if an error was reported, such as a "refused"
+option, an option that the server rsync doesn't support (e.g. perhaps
+links are not supported by the server), or some other failure (such as
+trying to send data to a read-only module). Upgrading the version of rsync
+that is running as a daemon to at least 2.6.3 will allow these errors to
+get returned to all rsync clients, old or new alike.
+
+</ol>
+
+<!--#include virtual="footer.html" -->
--- /dev/null
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
+<HTML>
+<HEAD>
+<TITLE>rsync mailing lists</TITLE>
+</HEAD>
+<!--#include virtual="header.html" -->
+
+<H2 align="center">rsync mailing lists</H2>
+
+ <p>
+ There are three mailing lists for rsync. All are optionally
+ available in digest mode and also through web archives.
+ </p>
+
+ <p>
+ Since these lists can generate a lot of traffic we suggest that
+ you should not subscribe from a web-mail account such as Yahoo!
+ or Hotmail, because your mailbox is likely to overflow.
+ Instead, please read directly from the archives.
+ </p>
+
+ <ul>
+ <li>
+ <b>rsync</b> is the main mailing lists for developers and
+ users. It sees between zero and twenty messages per day.
+ This is a good place to send questions about rsync, but please
+ <a
+ href="https://www.catb.org/~esr/faqs/smart-questions.html">read
+ this before posting</a>.
+
+ [<a href="https://www.mail-archive.com/rsync@lists.samba.org/">archive 1</a>,
+ <a href="https://lists.samba.org/archive/rsync/">archive 2</a>,
+ <a href="https://lists.samba.org/mailman/listinfo/rsync">subscriptions</a>]
+ </li>
+
+
+ <li>
+ <b>rsync-announce</b> carries only messages from the
+ maintainer annoucing new releases, which happen at most a few
+ times per month.
+
+ [<a href="https://www.mail-archive.com/rsync-announce@lists.samba.org/">archive 1</a>,
+ <a href="https://lists.samba.org/archive/rsync-announce/">archive 2</a>,
+ <a href="https://lists.samba.org/mailman/listinfo/rsync-announce">subscriptions</a>]
+ </li>
+
+
+ <li>
+ <b>rsync-cvs</b> carries messages are automatically generated
+ whenever a developer changes the source code, which can happen
+ many times per day. If you subscribe, you should probably
+ choose digest mode.
+
+ [<a href="https://www.mail-archive.com/rsync-cvs@lists.samba.org/maillist.html">archive 1</a>,
+ <a href="https://lists.samba.org/archive/rsync-cvs/">archive 2</a>,
+ <a href="https://lists.samba.org/mailman/listinfo/rsync-cvs">subscriptions</a>]
+ </li>
+ </ul>
+
+ <p>
+ Please report problems with the lists to the <tt>postmaster</tt>
+ at <tt>samba.org</tt>, but note that you can control your own
+ subscription using the web interface.
+ </p>
+
+<!--#include virtual="footer.html" -->
--- /dev/null
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
+<HTML>
+<HEAD>
+<TITLE>rsync on NT</TITLE>
+</HEAD>
+<!--#include virtual="header.html" -->
+
+<H2 align="center">rsync on NT</H2>
+
+<pre>
+From: "Mike McHenry" <mmchen@minn.net>
+Subject: Rsync 2.3.1 WinNT binaries and instructions available
+Date: Fri, 15 Oct 1999 02:53:30 +1000
+
+Hello all,
+
+I have created Windows NT binaries for rsync 2.3.1 and have decided to make
+them available for others to download. These binaries have been tested on
+Windows NT Server 4.0 SP5 and WILL run in daemon mode if you follow my
+instructions below. I make no guarantees about these binaries, they have
+however been working for me for weeks on several NT machines.
+
+Binaries at ftp://ftp.minn.net/usr/mmchen/
+
+Instructions for running in daemon mode:
+
+1. You will need two files, rsync.exe and cygwin1.dll. Place rsync.exe
+anywhere you like (I chose c:\program files\rsync\rsync.exe) and put
+cygwin1.dll in c:\winnt\system32
+
+2. You will need a program from the NT Server resource kit called
+srvany.exe. This program allows you to run any executable as a service. If
+you simply install the entire service pack it will be located in c:\ntreskit
+
+3. Create a service for rsync by typing the following:
+ instsrv Rsync "C:\ntreskit\srvany.exe"
+
+4. You should now have a new service called Rsync and you can verify by
+looking in Start->Control Panel->Services DON'T START IT YET!
+
+5. If you want to run rsync in daemon mode you will need a configuration
+file. Here is the one I use, call it rsyncd.conf and place it in the same
+directory as rsync (C:\Program files\rsync\rsyncd.conf)
+ use chroot = false
+ strict modes = false
+ hosts allow = *
+
+ [backup]
+ path = /
+ read only = yes
+ list = no
+
+This example configuration will make one big anonymous anonymous rsync area
+available, I use this to backup my NT machines from a central Unix machine.
+This configuration might not be ideal for you, change to suit your tastes.
+The first two lines are important for rsync to work on Windows NT however.
+
+6. You are going to need to hack some keys in the registry to make it work.
+Don't do this unless you are comfortable with the changes! Run regedit32 and
+add the following keys and values (quotation marks ARE IMPORTANT):
+ HKEY_LOCAL_MACHINE->SYSTEM->CurrentControlSet->Services->Rsync
+ Edit->Add Key-> Key Name: Paramaters
+ Edit->Add Value-> Value Name: AppDirectory Value: "C:\programfiles\rsync"
+ Edit->Add Value-> Value Name: Application Value: "C:\programfiles\rsync\rsync.exe"
+ Edit->Add Value-> Value Name: AppParamters Value: --config="C:\programfiles\rsync\rsyncd.conf" --daemon
+
+7. That's it, you should be able to start and stop the rsync service at will
+using the Services Control Panel. When running with the above configuration
+you should be able to test by attempting to telnet to port 873 from a remote
+machine.
+ telnet rsync.server.com 873 (replacing rsync.server.com with your own
+server's address)
+You should get a connection to the rsync daemon running on your NT box.
+
+8. If you have problems you are on your own, sorry, I have enough to do :) I
+would suggest triple-checking your spelling on EVERYTHING (filenames,
+configs, reg keys). If you have any comments or suggestions I would be happy
+to hear them at mmchen@minn.net.
+
+Mike McHenry
+ Systems Administrator
+ MinnNet Communications, Inc.
+</pre>
+
+<!--#include virtual="footer.html" -->
--- /dev/null
+From: "Mike McHenry" <mmchen@minn.net>
+Subject: Rsync 2.3.1 WinNT binaries and instructions available
+Date: Fri, 15 Oct 1999 02:53:30 +1000
+
+
+Hello all,
+
+I have created Windows NT binaries for rsync 2.3.1 and have decided to make
+them available for others to download. These binaries have been tested on
+Windows NT Server 4.0 SP5 and WILL run in daemon mode if you follow my
+instructions below. I make no guarantees about these binaries, they have
+however been working for me for weeks on several NT machines.
+
+Binaries at ftp://ftp.minn.net/usr/mmchen/
+
+Instructions for running in daemon mode:
+
+1. You will need two files, rsync.exe and cygwin1.dll. Place rsync.exe
+anywhere you like (I chose c:\program files\rsync\rsync.exe) and put
+cygwin1.dll in c:\winnt\system32
+
+2. You will need a program from the NT Server resource kit called
+srvany.exe. This program allows you to run any executable as a service. If
+you simply install the entire service pack it will be located in c:\ntreskit
+
+3. Create a service for rsync by typing the following:
+ instsrv Rsync "C:\ntreskit\srvany.exe"
+
+4. You should now have a new service called Rsync and you can verify by
+looking in Start->Control Panel->Services DON'T START IT YET!
+
+5. If you want to run rsync in daemon mode you will need a configuration
+file. Here is the one I use, call it rsyncd.conf and place it in the same
+directory as rsync (C:\Program files\rsync\rsyncd.conf)
+ use chroot = false
+ strict modes = false
+ hosts allow = *
+
+ [backup]
+ path = /
+ read only = yes
+ list = no
+
+This example configuration will make one big anonymous anonymous rsync area
+available, I use this to backup my NT machines from a central Unix machine.
+This configuration might not be ideal for you, change to suit your tastes.
+The first two lines are important for rsync to work on Windows NT however.
+
+6. You are going to need to hack some keys in the registry to make it work.
+Don't do this unless you are comfortable with the changes! Run regedit32 and
+add the following keys and values (quotation marks ARE IMPORTANT):
+ HKEY_LOCAL_MACHINE->SYSTEM->CurrentControlSet->Services->Rsync
+ Edit->Add Key-> Key Name: Paramaters
+ Edit->Add Value-> Value Name: AppDirectory Value: "C:\programfiles\rsync"
+ Edit->Add Value-> Value Name: Application Value: "C:\programfiles\rsync\rsync.exe"
+ Edit->Add Value-> Value Name: AppParamters Value: --config="C:\programfiles\rsync\rsyncd.conf" --daemon
+
+7. That's it, you should be able to start and stop the rsync service at will
+using the Services Control Panel. When running with the above configuration
+you should be able to test by attempting to telnet to port 873 from a remote
+machine.
+ telnet rsync.server.com 873 (replacing rsync.server.com with your own
+server's address)
+You should get a connection to the rsync daemon running on your NT box.
+
+8. If you have problems you are on your own, sorry, I have enough to do :) I
+would suggest triple-checking your spelling on EVERYTHING (filenames,
+configs, reg keys). If you have any comments or suggestions I would be happy
+to hear them at mmchen@minn.net.
+
+Mike McHenry
+ Systems Administrator
+ MinnNet Communications, Inc.
+
--- /dev/null
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
+<HTML>
+<HEAD>
+<TITLE>rsync resources</TITLE>
+</HEAD>
+<!--#include virtual="header.html" -->
+
+<H2 align="center">rsync resources</H2>
+
+Please <a href="lists.html">let us know</a> if you have any rsync-related
+documents to add to this list:
+
+<ul>
+
+<li>Be sure to search for the latest rsync info to get up-to-the-minute
+results. You can use the search box at the top of the page for either
+web searching or project searching (they are done via Google).
+
+<li>
+2002-05-15: rsync is not official GNU software, but we try to
+work more or less in accordance with their <a
+href="http://www.gnu.org/prep/maintain_toc.html">Guidelines for
+Maintaining GNU Software</a>.
+
+<li> 2002-04-10: A new tutorial on using rsync to create a system of <a
+href="http://www.mikerubel.org/computers/rsync_snapshots/">rotating
+backups</a>, by Mike Rubel.
+
+<li>If you still don't know what rsync is, then take a look at the
+<A HREF="https://download.samba.org/pub/rsync/README">README</A>.
+
+<li>There is now a python script that implements
+<a href="https://download.samba.org/pub/unpacked/rsync/support/atomic-rsync">an
+atomic update</a> of the received files at the end of the transfer (when pulling).
+
+<li> Brian Elliott Finley has put together a great Linux install system based
+on rsync. You you read about it at <a href="http://thefinleys.com/SystemImager/">http://thefinleys.com/SystemImager/</a>
+
+<li><a href="http://www.dirvish.com/">Dirvish</a> is a fast, disk based,
+rotating network backup system that was originally written by JW Schultz.
+
+<li><a href="http://backuppc.sourceforge.net/">BackupPC</a>: a backup
+system using rsync. Hard-links all identical files (even between multiple
+runs and multiple backup sources), compresses the files, provides an easy
+interface to find and restore files, etc.
+
+<li><a href="https://github.com/CharlesMAtkinson/bung">Bung</a>: (BackUp Next
+Generation) backs up files, MariaDB, OpenLDAP, postgres, etc. via an extensible
+templates system with git support. The rsync-based "rolling full" backup is
+easy to browse and restore from using everyday tools.
+
+<li><a href="http://hacks.dlux.hu/drsync/">drsync</a>: a wrapper for rsync
+that remembers file sets between invocations so that a 2-way synchronization
+of two systems is possible.
+
+<li><a href="http://rsyncbackup.erlang.no/">rsyncbackup</a>: a helper
+script that uses config files to setup multiple backup scenarios and
+invokes rsync (or rsyncX on macOS).
+
+<li>Users who use the new character-set conversion option of rsync (--iconv)
+may want to check into the <a href="http://www.j3e.de/linux/convmv/man/">convmv</a>
+package that lets you convert the names of already-transferred files into a
+new characterset (for when you want to change or normalize the characterset
+of a hierarchy of files).
+
+<li>For those wanting to use launchd to run an rsync daemon (e.g. Mac
+OS X Tiger users), Glen Scott provides the necessary
+<a href="http://www.designsolution.co.uk/resources/rsync/">rsync.plist</a>
+file.
+
+<li>For the developer wanting to work on a branched rsync version based on
+one of the diffs in the patches dir, you may want to check into Matt's
+<a href="http://www.kepreon.com/~matt/myrsync/index.html#patchsync">patchsync</a>
+script.
+
+<!--#include virtual="doc-resources.html" -->
+
+<li>There are a few choices for making rsync work with OS X's resource forks.
+One is the official apple patch found on their opendarwin site, such as
+<a href="http://darwinsource.opendarwin.org/10.4/rsync-20">this one</a>
+(I've heard patch inefficiently transfers the entire resource fork information
+for every file on every transfer.) Another choice is to use a third-party
+adapted rsync, such as
+<a href="http://archive.macosxlabs.org/rsyncx/rsyncx.html">rsyncx</a> or a
+<a href="http://www.quesera.com/reynhout/misc/rsync+hfsmode">rsync+hfsmode
+patch</a> by D Andrew Reynhout. For the future, I would like to see an rsync
+that supports ACLs and Posix xattrs adapted to interact with resource forks in
+a seamless way (if that's possible).
+
+<li>Piero Orsoni wrote a GTK-based GUI for rsync called
+<a href="http://www.opbyte.it/grsync/">grsync</a>.
+
+<li>Those interested in using an rsync daemon over SSL may be interested in
+<a href="http://dozzie.jarowit.net/trac/wiki/RsyncSSL">this wiki page</a>
+that outlines a way to use a modern, simplified stunnel setup.
+
+<li>Thomas Roessler has written an rsync wrapper for
+<a href="ftp://riemann.iam.uni-bonn.de/pub/users/roessler/cvslock/">efficient,
+safe CVS mirroring</a>.
+
+<li>Rsync is distributed with the
+<a href="https://download.samba.org/pub/unpacked/rsync/support/rrsync">rrsync python script</a>
+that lets you restrict the rsync commands that can be run via ssh. (This is
+a enhanced and reworked version of Joe Smith's original perl version.)
+
+<li><a href="mailto:LEakin@Nostrum.COM">Lee Eakin</a> has written a <a href="rsync_wrapper.pl">perl wrapper for rsync</a>.
+
+<li>A wire-compatible <a href="http://search.cpan.org/~cbarratt/">rsync implementation in perl</a>.
+
+<li>A <a href="http://www.srehttp.org/apps/rxrsync/">REXX implementation of rsync</a>.
+
+<li>An initial version of a <a href="http://www.kolosy.com/wordpress/?p=8">rewrite of rsync for .Net</a>.
+
+<li>You might want to check out an encryption program that is being developed
+to produce more rsync-friendly output:
+<a href="http://rsyncrypto.lingnu.com/">rsyncrypto</a>.
+
+<li>If you need a 2-way synchronization because both ends of the transfer may
+be changing files, you may want to either look into a tool designed to do this
+(e.g. <a href="http://freshmeat.net/projects/unison/">unison</a>), or you may
+wish to use an external wrapper for rsync that keeps extra data about what was
+in the last transfer so that it can figure out if a file is new or deleted
+(e.g. <a href="http://freshmeat.net/projects/drsync/">drsync</a>).
+
+</ul>
+
+<!--#include virtual="footer.html" -->
--- /dev/null
+rsync-and-debian.html
+rsync-and-debian.ps
--- /dev/null
+<!doctype linuxdoc system>
+<linuxdoc>
+ <article>
+ <titlepag>
+ <title>About integration of rsync and Debian</title>
+ <author>
+ <name>Martin Pool <tt>mbp@samba.org</tt></name>
+ </author>
+ <date>$Date: 2002/05/14 03:10:09 $</date>
+ </titlepag>
+
+ <toc>
+
+ <sect>
+ <heading>Introduction</heading>
+
+ <p>
+ It seems like there is an rsync thread on <tt/debian-devel/
+ every month or two. Rather than going round in circles yet
+ again, I though I would summarise the issues in one place.
+ </p>
+
+ <p>
+ By way of background, I have been the maintainer of rsync for
+ about a year, and I wrote the <tt/librsync/ and <tt/rproxy/
+ packages. Incidentally, I run Debian on most of the machines
+ I use, so I do care about the two projects working together
+ well.
+ </p>
+
+ <p>
+ I have tried to respond to all the issues raised on the list
+ recently, but possibly have missed some. After putting many
+ hours into rsync I suppose I am quite fond of it and may be a
+ little biased, but I think I also know its strengths and
+ problems more than most.
+
+ <p>
+ If there are issues ommitted by this document, or if you think
+ the answers are incomplete, unbalanced, or incorrect, then
+ please mail <tt/mbp@samba.org/, <tt/rsync@lists.samba.org/
+ and/or <tt/debian-devel@lists.debian.org/.
+
+ </sect>
+
+
+ <sect>
+ <heading>Background</heading>
+
+
+ <sect1>
+ <heading>The rsync algorithm</heading>
+
+ <p>
+ rsync is really two independent but related ideas: the
+ <em/rsync algorithm/ for delta compression, and its
+ implementation in a mirroring program.
+ </p>
+
+ <p>
+ The rsync algorithm is described in detail in the ANU
+ Technical Report included with the distribution and on the
+ <htmlurl url="http://rsync.samba.org/" name="rsync.samba.org"> web site.
+ </p>
+
+ <p>
+ Briefly, the rsync algorithm provides an efficient means to
+ transfer a file <em/A/ from a source machine to a
+ destination machine, when a similar file <em/A'/ is already
+ present on the destination. The destination machine sends a
+ checksum of each consecutive block (of say 1kB) from <em/A'/
+ to the source machine. The source machine searches through
+ <em/A/ for matching blocks. Whatever does not match must be
+ different, and a description of these differences is sent to
+ the destination.
+ </p>
+
+ <p>
+ The algorithm could be embodied in other forms and uses than
+ the rsync program. Indeed, the algorithm has been
+ reimplemented in programs other than rsync itself: in other
+ languages (Rexx, CAML, Java), in a C library (<tt/librsync/)
+ and a derivative in xdelta. xdelta is a specialization of
+ the algorithm for the case where the two files are local and
+ can be directly compared to compute a minimal binary delta.
+ </p>
+
+ <p>
+ rsync deltas are much more efficient than diffs, for several
+ reasons: most importantly, useful diff formats include
+ context lines, which are wasted information. rsync gets the
+ same benefit of making sure that the change is applied to a
+ compatible version of the file, but in a much more efficient
+ way. diffs suffer from being human-readable and therefore
+ verbose, and cannot handle binary files. Both can be
+ transparently compressed. diffs are an invaluable tool for
+ software development, but not ideal for distribution of
+ deltas.
+ </p>
+ </sect1>
+
+
+
+ <sect1>
+ <heading/The rsync program/
+
+ <p>
+ The rsync mirroring program is similar in functionality to
+ other tools such as <tt/wget/, <tt/ftpmirror/, <tt/cvsup/
+ and <tt/rdist/. It has a number of functions that are
+ useful in association with file mirroring, such as bandwidth
+ limiting, access control, recursive mirroring, and selection
+ of files to copy in various ways.
+ </p>
+
+ <p>
+ In general, rsync is substantially faster than other
+ tools, because it uses delta compression, and because the
+ protocol is designed to be efficient both in traffic and
+ round trips. rsync can use <tt/zlib/ to compress traffic
+ (and introduce double-<tt/free()/ security holes :-), at the
+ option of the client. Compression may also be disabled by
+ the server administrator.
+ </p>
+
+ <p>
+ rsync has the important property that it is generally
+ idempotent: repeated runs of rsync, even if interrupted,
+ make the contents of the destination machine converge
+ towards those of the source machine.
+
+ <p>
+ Unlike wget and ftpmirror, rsync must be installed on both
+ the client and the server. It uses its own protocol, rather
+ than HTTP or FTP. This is probably the biggest practical
+ drawback at the moment: one cannot use rsync from arbitrary
+ web sites, but only from systems that have specifically
+ chosen to support it. On the other hand, many important
+ sites for free software do now support rsync, and the
+ developers see them as an important constituency to support.
+
+ <p>
+ rsync can tunnel through a proxy server that supports
+ <tt/HTTP CONNECT/, a SOCKS server (using <tt/socksify/), an
+ ssh tunnel, or various other methods.
+
+ <p>
+ rsync can run as a daemon, similar to an <tt/ftpd/, in which
+ it is controlled by the <tt>/etc/rsyncd.conf</tt> file. The
+ archetypal use is to offer public, anonymous, read-only
+ access to a software archive but as with <tt/ftpd/ other
+ configurations are possible and common.
+ </p>
+
+ <p>
+ rsync can also run across a tunnel program such as <tt/ssh/,
+ in which case it is very similar to <tt/scp/. This mode is
+ commonly used for making network backups, or uploading
+ information to a web server.
+ </p>
+
+ <p>
+ rsync may also be used locally, which is a degenerate case
+ with client and server connected across a unix-domain or
+ localhost socket.
+ </>
+
+ <p>
+ rsync runs on many varieties of Unix under both gcc and
+ native compilers, and also under Cygwin on Microsoft
+ platforms. There is somebody working on a VMS port.
+ </p>
+ </sect1>
+
+
+
+
+ <sect1>
+ <heading/librsync/
+
+ <p>
+ <tt/librsync/ is a library that reimplements the rsync
+ algorithm in a very flexible way, with the goal of allowing
+ any reasonable mode of operation, and integration with any
+ existing program. rsync does not currently link to
+ librsync, but it might do so in the long term.
+
+ <p>
+ <tt/librsync/ currently uses an encoding
+ format different to that of rsync 2.5, mostly because I did
+ not want to be constrained by historical code.
+
+ <p>
+ <tt/librsync/ is at version 0.9.5 and roughly as stable as
+ the version number suggests: it is used by Intermezzo and
+ some other projects, but is not yet really mature. The
+ <tt/librsync/ distribution comes with a tool <tt/rdiff/ that
+ exposes the functionality as a Unix tool to shell scripts,
+ etc. <tt/librsync/ is LGPL'd.
+
+ <p>
+ For example, you can imagine the server calculating and
+ caching the checksums, or the client calculating the delta
+ once and then sending it over UDP multicast to several
+ destinations, or uuencoded in email. Neither of these would
+ be straightforward to implement in the rsync codebase, but
+ all can be done with rdiff.
+
+ </sect1>
+
+
+ <sect1>
+ <heading/rproxy/
+
+ <p>
+ <tt/librsync/ is used in the <url
+ url="http://rproxy.samba.org/" name="rproxy"> program, which
+ is a prototype of transparent integration of delta
+ compression into HTTP.
+ </p>
+
+ <p>
+ rproxy implements delta compression of arbitrary HTTP
+ content, whether dynamic or static. For example, rproxy
+ gives very good compression on repeated visits to portal
+ sites such as Slashdot, since it can transmit only the
+ sections of the page modified since last time it was
+ viewed. Regular HTTP caches, by contrast, must either hit
+ on the whole page, or reload the whole page, and therefore
+ do poorly on template-based dynamic content.
+
+ <p>
+ rproxy adds compatible extensions to HTTP headers, and can
+ pass through HTTP caches. It requires upstream and
+ downstream support, which at the moment means two
+ installations of rproxy, but in the future could conceivably
+ be integrated into Mozilla, Squid, Apache and similar
+ programs.
+
+ <p>
+ rproxy is packaged in Debian and is moderately useful for
+ people with slow links.
+
+ <p>
+ rproxy is not HTML-specific, but HTML is by far the most
+ common case of dynamic HTTP content that can be
+ delta-compressed. However, HTML documents tend to be fairly
+ small, and as connectivity improves they're becoming of
+ decreasing interest as a compression problem.
+
+ <p>
+ (My personal interest in this project declined significantly
+ when I went from a 56k6 modem to ADSL at home. I realize
+ many people in the world still have slow connections.)
+
+ <p>
+ There are some Internet Drafts from Jeffrey Mogul and others
+ that add similar delta compression based on the server
+ storing all possible deltas. Last time I looked, there did
+ not seem much interest in wide adoption of these proposals.
+
+ <p>
+ Documenting a protocol extension to the standard expected by
+ an RFC can be a lot of work. A beginning on this work has
+ been made for rproxy, but more experiments with the
+ implementation are needed before proposal as a standard.
+
+ <p>
+ rproxy is not being actively developed at the moment.
+ Obviously I cannot answer why every programmer in the world
+ does not work on this. Personally I think that developing
+ rsync itself, and then librsync/rdiff, is likely to be more
+ useful; I suspect other people interested in working in this
+ area might have similar thoughts.
+
+ <p>
+ I don't think there are any problems in the code or project
+ that would actively prevent anybody from working on it. I'd
+ be happy to hand over maintenance to somebody else. Ben
+ Elliston has expressed interest in looking after it in the
+ last couple of months, and possibly it will be more active
+ in the future.
+ </sect1>
+
+
+
+ <sect1>
+ <heading>Introduction to Debian</heading>
+
+ <p>
+ Debian is a free operating system developed by the
+ cooperation of people all over the world. The most
+ important relation of rsync to Debian is copying of software
+ packages from developers to distribution servers to end
+ users.
+ </p>
+
+ <p>
+ Many Debian users get their software by internet download
+ from a public server, rather than pre-installed on a machine
+ or on CD-ROM.
+ </p>
+
+ <p>
+ Debian software archives include both software source,
+ binary packages for various architectures, and index
+ metadata, primarily the <tt/Packages/ files.
+ </p>
+
+ <p>
+ Because of Debian's community development process, new
+ packages are released very often. On a typical day in
+ Debian's <tt/unstable/ release, there might be fifty new
+ uploads.
+ </p>
+
+ <p>
+ Debian is quite different from other software distributions
+ in shipping so many packages so frequently. The BSDs (as I
+ understand it) base most of their development out of a CVS
+ or CVSup tree, which inherently distributes coarse-grained
+ deltas. Proprietary systems make binary releases, but much
+ less frequently. Possibly the development branches of other
+ distributions, such as Redhat "Rawhide" and Mandrake
+ "Cooker" are similar.
+ </p>
+ </sect1>
+
+
+
+
+ <sect1>
+ <heading>apt-proxy</heading>
+ <p><label id="apt-proxy">
+ <url url="http://apt-proxy.sourceforge.net/" name="apt-proxy"> is a caching proxy for Debian archives. It appears as an HTTP
+ server to apt clients, and uses rsync, http, or ftp to connect
+ to an upstream server.
+
+ <p>
+ Because rsync is less efficient than HTTP for transferring
+ compressed files, <tt/apt-proxy/ can selectively use rsync for
+ uncompressed Packages and files and HTTP or FTP for <tt/.deb/
+ files.
+
+ <p>
+ <tt/apt-proxy/, unlike Squid, has domain knowledge about
+ Debian archives and can therefore perform functions such as
+ purging old packages from the cache.
+
+ <p>
+ The original apt-proxy was written by Paul 'Rusty' Russel.
+ The current maintainer is Chris Halls.
+ </p>
+ </sect1>
+ </sect>
+
+
+
+ <sect>
+ <heading>Open Issues</heading>
+
+
+ <sect1>
+ <heading/Compressed files cannot be differenced/
+
+ <p>
+ <tt/gzip/, like most compression algorithms has the property
+ that a change in the source file at one point will cause
+ cascading changes in the output file from that point
+ onwards, and therefore make delta compression more or less
+ useless.
+ </p>
+
+ <p>
+ Although delta compression is not possible, rsync is still a
+ very useful tool for mirroring compressed files. The
+ efficient network protocol mean that it will generally be
+ slightly faster and use less traffic than HTTP or FTP.
+
+ <p>
+ There is a patch called <tt/--rsyncable/ for gzip that fixes
+ this behaviour: gzip files are basically broken up into
+ blocks so that changes (including insertion or deletion) in
+ the input file affect only the corresponding blocks in the
+ output file. (The blocks are not of fixed size, but rather
+ delimited by marker patterns at which a checksum hits a
+ particular value, so they move as data is inserted or
+ removed.)
+
+ <p>
+ I believe that this will merge into the upstream <tt/zlib/
+ soon and be on by default, at which point <tt/.deb/ files
+ will delta-compress well. This patch does seem to be
+ languishing, though, and it would be good to either get it
+ into the upstream, or into Debian's own version. Needless
+ to say it must be extremely thoroughly tested.
+
+ <p>
+ The scheme for containing changes is not specific to rsync,
+ and might also be useful with <tt/xdelta/, <tt/rdiff/, or
+ some other binary delta scheme invented in the future. It
+ also does not require a copy of the old file when
+ compressing the new one.
+
+ <p>
+ This scheme relies on the output file being determined only
+ by the input file. As far as I know compression schemes
+ like gzip, lzw, and bzip2 are deterministic in this way.
+ (GnuPG, for example, is not, since it uses a random session
+ key.)
+
+ <p>
+ Incidentally this would allow more efficient deltas between
+ Debian ISO images.
+ </p>
+
+ <p>
+ Alternatively, you could distribute an uncompressed package
+ tree that would rsync efficiently, but since the gzip patch
+ should merge soon this seems unnecessary.
+
+ <p>
+ There is also the interesting possibility of using
+ <tt/dpkg-repack/ to generate something similar to the
+ previous <tt/.deb/ and then use it as the basis of an rsync
+ download.
+
+ <p>
+ It could be possible to make an equivalent patch to
+ <tt/bzip2/, but possibly the large block size would cause
+ trouble.
+
+ <p>
+ A patch to gzip which implements this behaviour is available
+ from rsync CVS.
+ </sect1>
+
+
+
+ <sect1>
+ <heading/rsync is too hard on servers/
+
+ <p>
+ If it is, then I think we should fix the problems, rather
+ than invent a new system from scratch. I think the
+ scalability problems are accidents of the current codebase,
+ rather than anything inherent in the design.
+ </p>
+ </sect1>
+
+
+
+ <sect1>
+ <heading/We should throw out the current code and start from
+ scratch/
+
+ <p>
+ Some projects (Apache 2.0, Mozilla, ...) choose to do this;
+ some concentrate on piecemeal improvement (Linux).
+
+ <p>
+ As Joel Spolsky recently observed, throwing the code out to
+ start from scratch always seems like a nice idea, but rarely
+ works out. Essentially you are opting for the devil you
+ don't know, but he definitely exists.
+
+ <p>
+ Having dealt with some fairly crufty old code I sometimes
+ feel like doing this in rsync, but I can also see the
+ arguments against it. Starting from a new codebase would
+ bring in plenty of new bugs (including security bugs),
+ orphan existing users, and halt progress until it caught up
+ to the same level. It would require carefully documenting
+ all the existing internal protocols and behaviours, which is
+ a nice idea but not clearly a good use of the developers'
+ time.
+
+ <p>
+ If you throw out the code, you have the questions of whether
+ or not to keep the current network protocol, and whether or
+ not to keep the current command-line option.
+
+ <p>
+ Not being able to talk to old remote versions would be a
+ real pain, and would certainly slow adoption. (Think of how
+ unpopular SSH2 was when it wanted to be incompatible with
+ SSH1, but use the same TCP port.) On the other hand,
+ keeping the same network protocol limits your ability to
+ redesign things.
+
+
+ <p>
+ There is some cruft in the command line syntax, but throwing
+ it out would also break everybody's scripts, mental maps,
+ and documentation.
+
+
+ <p>
+ Possibly rsync 3.0 will be largely rewritten. <tt/librsync/
+ is a start in that direction. If you want to do this,
+ <tt/librsync/ might help you, but please think about it
+ before you begin hacking.
+
+ </sect1>
+
+
+
+
+ <sect1>
+ <heading/rsync development roadmap/
+
+ <p>
+ Debian's goals for rsync have to be balanced against the
+ requirements of other users (including the mirror sites that
+ distribute Debian) and the limited development resources.
+
+ <p>
+ As of April 2002, the 2.5.5 release is current and 2.5.6 is
+ pending. These versions seem to work quite well and we
+ encourage people to upgrade from stable 2.4.6 and previous
+ versions. It is very important to the developers to
+ establish a stable base release before starting out on new
+ development.
+ </p>
+
+ <p>
+ A number of enhancements are planned for the 2.6 series.
+ Some of them are discussed below, and more details can be
+ found in the <tt/TODO/ file in the rsync distribution.
+
+ <p>
+ If you want rsync to progress faster, the best thing you can
+ do is find reproducible failure cases and report them well,
+ or to help us write a regression test suite.
+
+ <p>
+ Nobody is very actively working on rproxy or librsync as far
+ as I know. Personally at the moment I feel supporting rsync
+ itself is more useful.
+ </sect1>
+
+
+
+
+ <sect1>
+ <heading/Goswin Brederlow's proposal to use the reverse rsync
+ algorithm over HTTP Range requests/
+
+ <p>
+ Goswin Brederlow has a nice proposal that allows rsync
+ compression using a regular HTTP server. It depends only
+ upon rsyncable archives, and development of new client and
+ support programs.
+
+ <p>
+ The idea, as I understand it, is that the checksums for each
+ package should be pre-calculated and stored on the HTTP
+ server along with the files they describe. (The checksum
+ files should be on the order of a thousand times smaller
+ than the archive file.)
+
+ <p>
+ The client program will download the checksum file over HTTP
+ if it exists. The client can then search for common blocks
+ in the local file, and therefore determine what new regions
+ it must fetch from the server. The client then sends an
+ HTTP request using the optional <tt/Range/ header for the
+ relevant sections of the new file, and patches them onto the
+ old file. The checksum file ought to also include a
+ whole-file message digest which can be used to ensure that
+ reassembly was successful.
+
+ <p>
+ This scheme has the great advantage that the server is
+ entirely passive, and only needs to support standard HTTP.
+ The checksum files can be generated once for each package
+ either during upload, or by the administrator of a
+ particular server.
+
+ <p>
+ (This is not the same as rproxy, which uses a different
+ encoding and requires upstream support, but can
+ transparently compress any request without requiring
+ signature files on the server.)
+
+ <p>
+ This scheme could be fairly easily built on top of rdiff or
+ librsync.
+
+ <p>
+ I think this sounds like a very promising scheme. I think
+ rsync might still be better for people who want to copy
+ large trees, but for users apt-getting single packages this
+ would be a simple way to get delta-compression in, pending
+ only --rsyncable files.
+
+
+ <p>
+ I'm not sure if all HTTP servers currently handle <tt/Range/
+ commands efficiently. This proposal would probably stress
+ them more than is common at the moment.
+
+ <p>
+ Because it requires a special client, and special checksum
+ files stored on the server it has a wider impact than just
+ using rsync. It ought to be done in a non-Debian-specific
+ way. Rather than adding knowledge about the deltas into
+ apt-get itself, we ought to make a separate tool which
+ downloads a delta over HTTP.
+
+ </sect1>
+
+
+
+
+ <sect1>
+ <heading>rsync only compares files with the exact same
+ name</heading>
+
+ <p>
+ Even for uncompressed files, rsync will not currently try to
+ use <tt/linux-2.4.17.tar/ to do delta-compression against
+ <tt/linux-2.4.18.tar/, because it cannot guess from the
+ names that the two files are related.
+
+ <p>
+ Paul Russell has contributed a patch which adds a heuristic
+ to detect this. It will probably be merged in 2.6.
+ </p>
+ </sect1>
+
+
+ <sect1>
+ <heading/rsync uses too much memory/
+
+ <p>
+ rsync traverses the entire directory tree before it begins
+ copying files. On machines with little virtual memory and a
+ lot of files to copy this can be a problem.
+
+ <p>
+ Some problems in this area have been fixed in the 2.5
+ series. Other solutions involving internal restructuring of
+ the <tt/flist/ and <tt/hlink/ code will be attempted in 2.6,
+ and they should yield a substantial improvement.
+
+ <p>
+ We are not likely to change the general approach of
+ traversing the tree up-front in the near future, because it
+ is tightly tied to the network protocol. It might be
+ attempted in the 3.0 timeframe, but it is not entirely clear
+ that it is really a problem.
+ </sect1>
+
+
+
+ <sect1>
+ <heading>rsync hangs?</heading>
+
+ <p>
+ There were a number of deadlock bugs in the past. We
+ believe they are all fixed in 2.5.5, but would welcome good
+ bug reports to the contrary. Some of them were in fact
+ Linux and Solaris kernel bugs, but they've been fixed by the
+ relevant parties quite some time ago.
+ </p>
+ </sect1>
+
+
+
+ <sect1>
+ <heading>Reduced network usage is not justified by increased CPU
+ usage
+ </heading>
+
+ <p>
+ The balance point will vary for each administrator. It is
+ hard to answer this universally. The balance is very
+ different in countries other than the US.
+ </p>
+
+ <p>
+ I suspect CPU cycles are falling in price faster than
+ network bandwidth, and for many people unused CPU cycles are
+ wasted but network packets cost money.
+ </p>
+ </sect1>
+
+
+
+ <sect1>
+ <heading/rsync can't saturate a fast link/
+
+ <p>
+ rsync with <tt>--whole-file</tt> (to turn off the
+ delta algorithm and be more comparable to ftp) floods a
+ 100Mbps link. If you have something faster than that to
+ your upstream Debian mirror, you're a lucky person.
+ </p>
+
+ <p>
+ It is not inherently impossible to make rsync use
+ <tt>sendfile()</tt> and <tt>mmap()</tt>; it's just that most
+ people don't need them. (Apache did not use them by default
+ either last time I looked, but that doesn't mean it's not
+ fast enough for most people.)
+ </p>
+ </sect1>
+
+
+
+ <sect1>
+ <heading>rsync is not actively developed</heading>
+
+ <p>
+ This was a pretty fair accusation a year ago, but I don't
+ think it is true anymore. We have made five releases this
+ year, and the <tt/diff -u/ since 2.4.6, the last version
+ from the previous maintainer, is 38305 lines.
+
+ <p>
+ Mail is answered promptly. The web site has been revised
+ and cleaned. We're working on a regression test suite, code
+ cleanup and documentation, and other quality issues.
+
+ <p>
+ I'm the most active developer at the moment, but there are a
+ number of other people with experience in the code who
+ regularly commit changes or send patches.
+
+ <p>
+ The stability expectations of our users require a somewhat
+ disciplined approach to accepting patches, but I don't think
+ that's a bad thing. The goal is to iterate through
+ <em/freeze/ and <em/flow/ phases similar to the kernel.
+
+ <p>
+ The Debian rsync package maintainer, Phil Hands, has been
+ somewhat inactive, but apparently he's back now. Colin
+ Walters has been helping out.
+
+ <p>
+ If you have patches that were dropped, please send them
+ through again.
+ </sect1>
+
+
+
+ <sect1>
+ <heading>Servers should cache deltas</heading>
+
+ <p>
+ This also smells like premature optimization. It might be
+ useful, but it certainly seems less important than some
+ other tasks.
+ </sect1>
+
+
+
+ <sect1>
+ <heading>Possible patent on rsync?</heading>
+
+ <p>
+ It has been suggested that some uses of rsync may conflict
+ with a US patent. I am not sure of the current situation so
+ I won't comment. I don't know of any suggestion that the
+ rsync program as it currently exists infringes any patent.
+ </p>
+
+ <p>
+ I do know that rsync has been in use for many years by large
+ and small organization with no complaints of patent
+ infringement.
+ </p>
+
+ <p>
+ I would point out that linked lists, mark-and-copy garbage
+ collection, and the Tab key are all patented too. Somebody
+ who always carefully checked first for software patents
+ would never write anything at all.
+ </p>
+
+ </sect1>
+
+
+ <sect1>
+ <heading>Debian should store more metadata outside of
+ the package file</heading>
+
+ <p>
+ I can see some reasons why this might be a good idea, but
+ it doesn't seem particularly relevant to rsync either way.
+ </p>
+ </sect1>
+
+
+
+ <sect1>
+ <heading>Debian should distribute diffs or xdeltas
+ between different versions of their packages</heading>
+
+ <p>
+ A problem with this is that Debian releases packages very
+ often, and so the number of delta files is large. This will
+ be a problem for mirror sites who cannot carry all of the
+ delta files for history, although of course the client could
+ fall back to directly fetching the new package if the
+ relevant deltas are not present.
+ </p>
+
+ <p>
+ Fetching a series of deltas which update the same area is
+ less efficient than calculating the deltas directly.
+ </p>
+
+ <p>
+ Although this idea does not seem very practical for people
+ using <em/unstable/, it could work well for <em/stable/
+ and other situations where there are relatively few
+ releases, or for people without network access. One could,
+ for example, distribute a CD-ROM of xdeltas. On the other
+ hand, since CD-ROMs are relatively large compared to the
+ distribution it might be simpler to just distribute the new
+ packages directly as is currently done.
+ </sect1>
+
+ <sect1>
+ <heading>rsync should be used by <tt/apt-get update/</heading>
+
+ <p>
+ Even if there are few gains from compressing <tt/.deb/s,
+ rsync might help in providing a more efficient transfer of
+ the <tt/Packages/ file, which has small but frequent
+ changes. This approach is used by <tt/apt-proxy/.
+ </sect1>
+ </sect>
+
+
+ <appendix>
+ <sect>
+ <heading>Revision history</heading>
+
+
+<p><tscreen><verb>
+$Log: rsync-and-debian.sgml,v $
+Revision 1.10 2002/05/14 03:10:09 mbp
+Add note about just transferring Package files.
+
+Revision 1.9 2002/05/14 02:35:43 mbp
+Update notes on apt-proxy based on mail from Chris Halls.
+
+Revision 1.8 2002/04/12 02:04:15 mbp
+More information about Goswin's idea, as I understand it.
+
+Revision 1.7 2002/04/12 01:39:52 mbp
+Lots more details about rproxy. Thanks to Brian May for prompting me
+to do this.
+
+Revision 1.6 2002/04/12 00:43:06 mbp
+Fix revision history SGML stuff.
+
+Revision 1.5 2002/04/12 00:24:50 mbp
+Fix mailing list name.
+Add revision history.
+
+ </verb></tscreen></p>
+ </sect>
+ </article>
+</linuxdoc>
\ No newline at end of file
--- /dev/null
+#!/bin/sh
+
+ulimit -c unlimited
+
+# Some systems have "truss" or "tusc" instead of "strace".
+# The -f option tells strace to follow children too.
+# The -t option asks for timestamps.
+# The -s 1024 option increases the string decoding limit per function call.
+# The -o option tells strace where to send its output.
+strace -f -t -s 1024 -o /tmp/rsync-$$.out rsync "${@}"
--- /dev/null
+# __
+# /\ \ From the mind of
+# / \ \
+# / /\ \ \_____ Lee Eakin <LEakin@Nostrum.COM>
+# / \ \ \______\ or <Lee@Eakin.ORG>
+# / /\ \ \/____ /
+# \ \ \ \____\/ / Wrapper module for the rsync program
+# \ \ \/____ / rsync can be found at http://rsync.samba.org/rsync/
+# \ \____\/ /
+# \/______/
+
+package Rsync;
+require 5.004;
+
+use FileHandle;
+use IPC::Open3 qw(open3);
+use Carp;
+
+use strict;
+
+=head1 NAME
+
+Rsync - perl module interface to B<rsync> http://rsync.samba.org/rsync/
+
+=head1 SYNOPSIS
+
+use Rsync;
+
+$obj = Rsync->new(qw(-az C<-e> /usr/local/bin/ssh
+ --rsync-path /usr/local/bin/rsync));
+
+$obj->exec(qw(localdir rhost:remdir))
+ or warn "rsync failed\n";
+
+=head1 DESCRIPTION
+
+Perl Convenience wrapper for B<rsync> program. Written for B<rsync> 2.3.1 but
+should perform properly with most versions.
+
+=cut
+
+# options from the rsync man pae
+###### Boolean flags ######
+# -h, --help show this help screen
+# -v, --verbose increase verbosity
+# -q, --quiet decrease verbosity
+# -c, --checksum always checksum
+# -a, --archive archive mode
+# -r, --recursive recurse into directories
+# -R, --relative use relative path names
+# -b, --backup make backups (default ~ suffix)
+# -u, --update update only (don't overwrite newer files)
+# -l, --links preserve soft links
+# -L, --copy-links treat soft links like regular files
+# --copy-unsafe-links copy links outside the source tree
+# --safe-links ignore links outside the destination tree
+# -H, --hard-links preserve hard links
+# -p, --perms preserve permissions
+# -o, --owner preserve owner (root only)
+# -g, --group preserve group
+# -D, --devices preserve devices (root only)
+# -t, --times preserve times
+# -S, --sparse handle sparse files efficiently
+# -n, --dry-run show what would have been transferred
+# -W, --whole-file copy whole files, no incremental checks
+# -x, --one-file-system don't cross filesystem boundaries
+# -C, --cvs-exclude auto ignore files in the same way CVS does
+# RCS SCCS CVS CVS.adm RCSLOG cvslog.* tags TAGS
+# .make.state .nse_depinfo *~ #* .#* ,* *.old *.bak
+# *.BAK *.orig *.rej .del-* *.a *.o *.obj *.so *.Z
+# *.elc *.ln core
+# --delete delete files that don't exist on the sending side
+# --delete-excluded also delete excluded files on the receiving side
+# --partial keep partially transferred files
+# --force force deletion of directories even if not empty
+# --numeric-ids don't map uid/gid values by user/group name
+# -I, --ignore-times don't exclude files that match length and time
+# --size-only only use file size when determining if a file
+# should be transferred
+# -z, --compress compress file data
+# --version print version number
+# --daemon run as a rsync daemon
+# --stats give some file transfer stats
+# --progress show progress during transfer
+###### scalar values ######
+# --csum-length=LENGTH <=16 bit md4 checksum size
+# -B, --block-size=SIZE checksum blocking size (default 700)
+# --timeout=TIME set IO timeout in seconds
+# --port=PORT specify alternate rsyncd port number
+# -e, --rsh=COMMAND specify rsh replacement
+# -T, --temp-dir=DIR create temporary files in directory DIR
+# --compare-dest=DIR also compare destination files relative to DIR
+# --exclude-from=FILE exclude patterns listed in FILE
+# --include-from=FILE don't exclude patterns listed in FILE
+# --config=FILE specify alternate rsyncd.conf file
+# --password-file=FILE get password from FILE
+# --log-format=FORMAT log file transfers using specified format
+# --suffix=SUFFIX override backup suffix
+# --rsync-path=PATH specify path to rsync on the remote machine
+###### array values ######
+# --exclude=PATTERN exclude files matching PATTERN
+# --include=PATTERN don't exclude files matching PATTERN
+#
+
+=over 4
+
+=item Rsync::new
+
+$obj = new Rsync;
+
+ or
+
+$obj = Rsync->new;
+
+ or
+
+$obj = Rsync->new(@options);
+
+=back
+
+Create an Rsync object. Any options passed at creation are stored in
+the object as defaults for all future exec call on that object. Options
+are the same as those in L<rsync> with the addition of
+--path-to-rsync which can be used to override the hardcoded default of
+/usr/local/bin/rsync, and --debug which causes the module functions to
+print some debugging information to STDERR.
+
+=cut
+
+sub new {
+ my $class=shift;
+
+ # seed the options hash, booleans, scalars, excludes, data,
+ # status, stderr/stdout storage for last exec
+ my $self={
+ # the full path name to the rsync binary
+ 'path-to-rsync' => '/usr/local/bin/rsync',
+ # these are the boolean flags to rsync, all default off, including them
+ # in the args list turns them on
+ 'flag' => {qw(
+ --archive 0 --backup 0 --checksum 0
+ --compress 0 --copy-links 0 --copy-unsafe-links 0
+ --cvs-exclude 0 --daemon 0 --delete 0
+ --delete-excluded 0 --devices 0 --dry-run 0
+ --force 0 --group 0 --hard-links 0
+ --help 0 --ignore-times 0 --links 0
+ --numeric-ids 0 --one-file-system 0 --owner 0
+ --partial 0 --perms 0 --progress 0
+ --quiet 0 --recursive 0 --relative 0
+ --safe-links 0 --size-only 0 --sparse 0
+ --stats 0 --times 0 --update 0
+ --verbose 0 --version 0 --whole-file 0
+ )},
+ # these have simple scalar args we cannot easily check
+ 'scalar' => {qw(
+ --block-size 0 --compare-dest 0 --config 0
+ --csum-length 0 --exclude-from 0 --include-from 0
+ --log-format 0 --password-file 0 --port 0
+ --rsh 0 --rsync-path 0 --suffix 0
+ --temp-dir 0 --timeout 0
+ )},
+ # these can be specified multiple times and are additive, the doc also
+ # specifies that it is an ordered list so we must preserve that order
+ 'exclude' => [],
+ # source/destination path names and hostnames
+ 'data' => [],
+ # return status from last exec
+ 'status' => 0,
+ 'realstatus' => 0,
+ # whether or not to print debug statements
+ 'debug' => 0,
+ # stderr from last exec in array format (messages from remote rsync proc)
+ 'err' => [],
+ # stdout from last exec in array format (messages from local rsync proc)
+ 'out' => [],
+ };
+ if (@_) {
+ &defopts($self,@_) or return undef;
+ }
+ bless $self, $class;
+}
+
+=over 4
+
+=item Rsync::defopts
+
+defopts $obj @options;
+
+ or
+
+$obj->defopts(@options);
+
+=back
+
+Set default options for future exec calls for the object. See L<rsync>
+for a complete list of valid options. This is really the internal
+function that B<new> calls but you can use it too. Presently there is no way
+to turn off the boolean options short of creating another object, but if it is
+needed and the B<rsync> guys don't use it, I may add hooks to let + and ++ or a
+leading no- toggle it back off similar to B<Getopt::Long> (the GNU way).
+
+=cut
+
+sub defopts {
+ my $self=shift;
+ my @opts=@_;
+
+ # need a conversion table in case someone uses the short options
+ my %short=(qw(
+ -B --block-size -C --cvs-exclude -D --devices -H --hard-links
+ -I --ignore-times -L --copy-links -R --relative -T --temp-dir
+ -W --whole-file -a --archive -b --backup -c --checksum
+ -e --rsh -g --group -h --help -l --links
+ -n --dry-run -o --owner -p --perms -q --quiet
+ -r --recursive -s --sparse -t --times -u --update
+ -v --verbose -x --one-file-system -z --compress
+ ));
+ while (my $opt=shift @opts) {
+ my $arg;
+ print(STDERR "setting debug flag\n"),$self->{debug}=1,next
+ if $opt eq '--debug';
+ print STDERR "processing option: $opt\n" if $self->{debug};
+ if ($opt=~/^-/) {
+ # handle short opts first
+ if ($opt=~/^-(\w+)$/) {
+ foreach (split '',$1) {
+ print STDERR "short option: -$_\n" if $self->{debug};
+ if (exists $short{'-'.$_}) {
+ $opt=$short{'-'.$_};
+ # convert it to the long form
+ $opt=$short{$opt} if exists $short{$opt};
+ # handle the 3 short opts that require args
+ $self->{scalar}{$opt}=shift(@opts),next if (/^[BeT]$/);
+ # handle the rest
+ $self->{flag}{$opt}=1,next if exists $self->{flag}{$opt};
+ }
+ carp "$opt - unknown option\n";
+ return undef;
+ }
+ }
+ # handle long opts with = args
+ if ($opt=~/^(--\w+[\w-]*)=(.*)$/) {
+ print STDERR "splitting longopt: $opt ($1 $2)\n" if $self->{debug};
+ ($opt,$arg)=($1,$2);
+ }
+ # handle boolean flags
+ $self->{flag}{$opt}=1,next if exists $self->{flag}{$opt};
+ # handle simple scalars
+ $self->{scalar}{$opt}=($arg || shift @opts),next
+ if exists $self->{scalar}{$opt};
+ # handle excludes
+ if ($opt eq '--exclude') {
+ $arg||=shift @opts;
+ # if they sent a reset, we will too
+ $self->{exclude}=[],next if $arg eq '!';
+ # otherwise add it to the list
+ push @{$self->{exclude}},$arg;
+ next;
+ }
+ # to preserve order we store both includes and excludes in the same
+ # array. We use the leading '+ ' (plus space) trick from the man
+ # page to accomplish this.
+ if ($opt eq '--include') {
+ $arg||=shift @opts;
+ # first check to see if this is really an exclude
+ push(@{$self->{exclude}},$arg),next if $arg=~s/^- //;
+ # next see if they sent a reset, if they did, we will too
+ $self->{exclude}=[],next if $arg eq '!';
+ # if it really is an include, fix it first, since we use exclude
+ $arg='+ '.$arg unless $arg=~/^\+ /;
+ push @{$self->{exclude}},$arg;
+ next;
+ }
+ # handle our special case to override hard-coded path to rsync
+ $self->{'path-to-rsync'}=($arg || shift @opts),next
+ if $opt eq '--path-to-rsync';
+ # if we get this far nothing matched so it must be an error
+ carp "$opt - unknown option\n";
+ return undef;
+ } else { # must be data (source/destination info)
+ print STDERR "adding to data array: $opt\n" if $self->{debug};
+ push(@{$self->{data}},$opt);
+ }
+ }
+ 1;
+}
+
+=over 4
+
+=item Rsunc::exec
+
+exec $obj @options or warn "rsync failed\n";
+
+ or
+
+$obj->exec(@options) or warn "rsync failed\n";
+
+=back
+
+This is the function that does the real work. Any options passed to this
+routine are appended to any pre-set options and are not saved. They effect
+the current execution of B<rsync> only. It returns 1 if the return status was
+zero (or true), if the B<rsync> return status was non-zero it returns undef and
+stores the return status. You can examine the return status from rsync and
+any output to stdout and stderr with the functions listed below.
+
+=cut
+
+sub exec {
+ my $self=shift;
+ my @cmd=($self->{'path-to-rsync'});
+
+ foreach (sort keys %{$self->{flag}}) {
+ push @cmd,$_ if $self->{flag}{$_};
+ }
+ foreach (sort keys %{$self->{scalar}}) {
+ push @cmd,$_.'='.$self->{scalar}{$_} if $self->{scalar}{$_};
+ }
+ foreach (@{$self->{exclude}}) {
+ push @cmd,'--exclude='.$_;
+ }
+ foreach (@{$self->{data}}) {
+ push @cmd,$_;
+ }
+ push @cmd,@_ if @_;
+ print STDERR "exec: @cmd\n" if $self->{debug};
+ my $in=FileHandle->new; my $out=FileHandle->new; my $err=FileHandle->new;
+ my $pid=eval{ open3 $in,$out,$err,@cmd };
+ if ($@) {
+ $self->{realstatus}=0;
+ $self->{status}=255;
+ $self->{err}=[$@,"Execution of rsync failed.\n"];
+ return undef;
+ }
+ $in->close; # we don't use it and neither should rsync (at least not yet)
+ $self->{err}=[ $err->getlines ];
+ $self->{out}=[ $out->getlines ];
+ $err->close;
+ $out->close;
+ waitpid $pid,0;
+ $self->{realstatus}=$?;
+ $self->{status}=$?>>8;
+ return undef if $self->{status};
+ return 1;
+}
+
+=over 4
+
+=item Rsync::status
+
+$rval = status $obj;
+
+ or
+
+$rval = $obj->status;
+
+=back
+
+Returns the status from last B<exec> call right shifted 8 bits.
+
+=cut
+
+sub status {
+ my $self=shift;
+ return $self->{status};
+}
+
+=over 4
+
+=item Rsync::realstatus
+
+$rval = realstatus $obj;
+
+ or
+
+$rval = $obj->realstatus;
+
+=back
+
+Returns the real status from last B<exec> call (not right shifted).
+
+=cut
+
+sub realstatus {
+ my $self=shift;
+ return $self->{realstatus};
+}
+
+=over 4
+
+=item Rsync::err
+
+$aref = err $obj;
+
+ or
+
+$aref = $obj->err;
+
+=back
+
+Returns an array or a reference to the array containing all output to stderr
+from the last B<exec> call. B<rsync> sends all messages from the remote
+B<rsync> process to stderr. This functions purpose is to make it easier for
+you to parse that output for appropriate information.
+
+=cut
+
+sub err {
+ my $self=shift;
+ return(wantarray ? @{$self->{err}} : $self->{err});
+}
+
+=over 4
+
+=item Rsync::out
+
+$aref = out $obj;
+
+ or
+
+$aref = $obj->out;
+
+=back
+
+Similar to the B<err> function, this returns an array or a reference to the
+array containing all output to stdout from the last B<exec> call. B<rsync>
+sends all messages from the local B<rsync> process to stdout.
+
+=cut
+
+sub out {
+ my $self=shift;
+ return(wantarray ? @{$self->{out}} : $self->{out});
+}
+
+=head1 Author
+
+Lee Eakin E<lt>leakin@nostrum.comE<gt>
+
+=head1 Credits
+
+Gerard Hickey C<PGP::Pipe>
+
+Russ Allbery C<PGP::Sign>
+
+Graham Barr C<Net::*>
+
+Andrew Tridgell and Paul Mackerras C<rsync(1)>
+
+John Steele E<lt>steele@nostrum.comE<gt>
+
+Philip Kizer E<lt>pckizer@nostrum.comE<gt>
+
+Larry Wall C<perl(1)>
+
+I borrowed many clues on wrapping an external program from the PGP modules,
+and I would not have had such a useful tool to wrap except for the great work
+of the B<rsync> authors. Thanks also to Graham Barr, the author of the libnet
+modules and many others, for looking over this code. Of course I must mention
+the other half of my brain, John Steele, and his good friend Philip Kizer for
+finding B<rsync> and bringing it to my attention. And I would not have been
+able to enjoy writing useful tools if not for the creator of the B<perl>
+language.
+
+=head1 Copyrights
+
+ Copyleft (l) 1999, by Lee Eakin
+
+=cut
+
+1;
--- /dev/null
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
+<HTML>
+<HEAD>
+<TITLE>rsync</TITLE>
+<style>
+.security { color: red; }
+h3 { margin-bottom: 0px; }
+.date { color: #D25A0B; }
+</style>
+</HEAD>
+<!--#include virtual="header.html" -->
+
+<H2 align="center">Rsync Security Advisories</H2>
+
+<p>You should install a security fix for rsync when the rync you are running is:<ol>
+<li>older than 3.2.5 and pulling from an untrusted server
+<li>older than 3.2.5 and using the bundled zlib
+<li>older than 3.1.3 with --xattrs enabled
+<li>older than 3.1.3 with a writable rsync daemon
+<li>older than 2.6.6
+</ol>
+
+<p><a name="s3_2_5"></a><hr>
+<h3>Improved file-list validation in 3.2.5</h3>
+<i class=date>August 14th, 2022</i>
+
+<p>If you are running an rsync older than 3.2.5 and pulling files from an
+untrusted server, upgrade to 3.2.5 to get some added file-list validation rules
+that should prevent the sender from sneaking in extra top-level arguments
+and/or including files/dirs that should have been filtered out by the client's
+filter rules. Fixes CVE-2022-29154.
+
+<p><a name="s3_2_5-2"></a><hr>
+<h3>Zlib memory corruption bug in rsync 2.6.6 - 3.2.4</h3>
+<i class=date>August 14th, 2022</i>
+
+<p>If your rsync is configured to use the bundled zlib, you should upgrade to
+3.2.5 which contains the official zlib fix for a buffer overrun bug that was
+detailed in CVE-2022-37434. While you're at it, be sure to update your system's
+zlib.
+
+<p><a name="s3_2_4"></a><hr>
+<h3>Zlib memory corruption bug in rsync 2.6.6 - 3.2.3</h3>
+<i class=date>April 15th, 2022</i>
+
+<p>If your rsync is configured to use the bundled zlib, you should upgrade to
+3.2.4 which contains the official zlib fix for a memory corruption bug that was
+detailed in CVE-2018-25032. While you're at it, be sure to update your system's
+zlib.
+
+<p><a name="s3_1_3"></a><hr>
+<h3>File-list validation in 3.1.3</h3>
+<i class=date>January 28st, 2018</i>
+
+<p>If you are using a version of rsync older than 3.1.3 as a client and
+receiving xattrs from an rsync server that you might not fully trust, a
+malicious (modified) server could send a non-null-terminated xattr name to
+overflow the xattr read buffer.
+
+<p>If you are running a writable rsync daemon older than 3.1.3, you should add
+a rule "refuse options = protect-args" if you don't fully trust the users who
+are sending you files.
+
+<p><a name="s3_1_2"></a><hr>
+<h3>File-list validation in 3.1.2</h3>
+<i class=date>December 21st, 2015</i>
+
+<p>If you're using a version of rsync older than 3.1.2 as a client and
+receiving files from an rsync server that you might not fully trust, this
+version adds extra checking to the file list to prevent the sender from
+tweaking the paths and/or the transfer requests in a way that could cause
+a file to be received outside the transfer destination.
+
+<p><a name="s3_0_2"></a><hr>
+<h3>Xattr security fix in 3.0.2</h3>
+<i class=date>April 8th, 2008</i>
+
+<p>If you're using a version of rsync from 2.6.9 to 3.0.1 that has extended
+attribute (xattr) support enabled, you should upgrade to 3.0.2 to avoid a
+potential buffer overflow problem. All 3.x versions have the potential to
+support xattrs (depending on OS availability and the configure options used),
+but version 2.6.9 had to be patched for this support. You can run the command
+"rsync --version" and look for the string "xattrs" (as long as it is not
+"no xattrs") to see if your rsync is affected.
+
+<p>For those running affected versions, there is
+<a href="https://download.samba.org/pub/rsync/security/rsync-3.0.1-xattr-alloc.diff">a
+patch with the fix available</a>.
+
+<p>Those running a writable rsync daemon can opt to refuse the "xattrs" option
+as a way to avoid the problem without an upgrade:
+
+<blockquote><pre>refuse options = xattrs</pre></blockquote>
+
+<p>(If you already refuse options, be sure to append "xattrs" to your existing
+config parameter rather than adding another refuse directive.)
+
+<p><a name="s3_0_0"></a><hr>
+<h3>Daemon security fixes in 3.0.0 (with patches for 2.6.9)</h3>
+<i class=date>First published on November 28th, 2007;<br>
+Updated on December 16th, 2007;<br>
+Item 3 added on February 15th, 2008</i>
+
+<p>Three security advisories affect people who run a <b>writable</b> rsync
+daemon: The first affects only those with "use chroot = no" (which is not a
+very safe combination in general), the second affects a daemon that has
+daemon-excluded files that are being hidden in a module's hierarchy, and
+the third affects only those with "use chroot = yes".
+Included are simple config-change suggestions that should help you to
+avoid the security issues and patches that make things safer.
+These advisories affect all rsync versions.
+
+<h4>1. Daemon advisory for "use chroot = no"</h4>
+
+<p>If you are running a writable rsync daemon with "use chroot = no", there
+is at least one way for someone to trick rsync into creating a symlink
+that points outside of the module's hierarchy.
+
+<p>This means that if you are allowing access from users who you don't
+trust, that you should either figure out a way to turn on "use chroot",
+or configure the daemon to refuse the "links" option (see "refuse
+options" in the rsyncd.conf manpage) which will disable the ability of
+the rsync module to receive symlinks. After doing so, you should also
+check that any existing symlinks in the daemon hierarchy are safe.
+
+<p>Starting with the 3.0.0-pre6 release, there is a new daemon parameter
+available: "munge symlinks". This allows an rsync daemon to accept
+symlinks and return them intact (with even a leading slash still there,
+which is new for a non-chroot daemon), but will not allow the symlinks
+to be used while they are in the daemon's hierarchy.
+
+<p>For those running
+2.6.9, there is
+<a href="https://download.samba.org/pub/rsync/security/rsync-2.6.9-munge-symlinks.diff">a
+patch for 2.6.9 to implement this parameter</a>.
+
+<p>Any admin applying that patch should read the "munge symlinks" section
+of the modified rsyncd.conf manpage for more information. You can also
+read about this parameter in the
+<a href="https://download.samba.org/pub/rsync/rsyncd.conf.html">rsyncd.conf
+manpage from a 3.x version</a>.
+
+<h4>2. Daemon advisory for daemon excludes</h4>
+
+<p>If you are running a writable rsync daemon that is using one of the
+"exclude", "exclude from", or "filter" parameters in the rsyncd.conf file
+to hide data from your users, you should be aware that there are tricks
+that a user can play with symlinks and/or certain options that can allow
+a user that knows the name of a hidden file to access it or overwrite it
+(if file permissions allow that).
+
+<p>You can avoid the symlink problem using the suggestions in the advisory
+above.
+
+<p>When a daemon has "use chroot = no" set , there was some buggy
+exclude-checking for these options: <code>--compare-dest</code>,
+<code>--link-dest</code>, <code>--copy-dest</code>, <code>--partial-dir</code>,
+<code>--backup-dir</code>, <code>--temp-dir</code>, and
+<code>--files-from</code>. These are fixed in the 3.0.0pre7 release. For
+those running 2.6.9, there is <a
+href="https://download.samba.org/pub/rsync/security/rsync-2.6.9-daemon-exclude.diff">a patch for
+2.6.9</a> to fix these checks.
+
+<p>Some of the above options can still cause problems if an excluded
+sub-directory or filename is inside the option's directory hierarchy and the
+names of a transferred file clashes with it. The affected options are the
+various <code>--*-dest</code> options (of which only <code>--link-dest</code>
+is particularly worrisome),
+<code>--backup-dir</code>, and <code>--partial-dir</code>.
+
+<p>You can avoid these sub-path problems by putting the following "refuse
+options" setting into your rsyncd.conf file:
+
+<blockquote><pre>refuse options = link-dest backup-dir partial-dir</pre></blockquote>
+
+<p>Those who aren't using an rsync with the latest exclude fixes may want to add
+some of the other affected options as well.
+
+<h4>3. Daemon advisory for "use chroot = yes"</h4>
+
+<p>If you are running a writable rsync daemon with "use chroot = yes", you
+should take care that users cannot upload their own library files and attempt
+to load them.
+
+<p>Beginning with rsync 3.0.0pre10, you can specify an inside-chroot path that
+makes the top of the transfer a subdirectory inside the chroot area, and that
+automatically makes library loading occur outside the transfer area (assuming
+you didn't pick an unwise subdirectory name for the transfer area and you
+don't have symlinks that point outside the transfer area).
+
+<p>If that solution is not good for you, the easiest way to protect your daemon
+is to create some appropriate directories in the top of your module's path
+hierarchy, such as "/etc", "/lib", and "/usr" (and any other top-level dirs
+that might be in the load path), chown those directories to some other user
+than the one that the module runs as (so that rsync will not be able to write
+files there, assuming that it is not run as root), and then hide the dirs using
+an exclude directive (either add a new one or extend your existing one):
+
+<blockquote><pre>exclude = /etc /lib /usr</pre></blockquote>
+
+<p>Doing all that will assure you that no user will be able to use rsync to
+upload a library that can be potentially loaded while rsync is attempting to
+perform an action, such as translating a username. You can feel free to put
+trusted libraries that you want rsync to access in the protected hierarchies,
+as desired.
+
+<p>Also available in rsync 3.0.0pre10 is a new daemon parameter that allows you
+to avoid the accessing of user/group-name translation libraries by a chrooted
+rsync: the "numeric ids" daemon parameter lets you turn on a forced
+numeric-only mode. The default for a chroot module is to enable this
+parameter, while the default for a non-chroot module is to disable it.
+
+<p>For those running 2.6.9, there is <a
+href="https://download.samba.org/pub/rsync/security/rsync-2.6.9-daemon-ids.diff">a patch for
+2.6.9</a> to add the "numeric ids" daemon config parameter. (The patch will
+only apply cleanly if you've already applied the munge-symlinks diff mentioned
+above.)
+
+<p><a name="s2_6_8"></a><hr>
+<h3>Xattr security fix in 2.6.8</h3>
+<i class=date>April 22th, 2006</i>
+
+<p>If you're using a version of rsync prior to 2.6.8 that was patched to
+include extended attribute (xattr) support, you should upgrade to 2.6.8 or
+later to avoid a potential buffer overflow problem.
+
+<p><a name="s2_6_6"></a><hr>
+<h3>Zlib security fix in 2.6.6</h3>
+<i class=date>July 28th, 2005</i>
+
+<p>If you're using a version of rsync prior to 2.6.6, there is a potential
+null-pointer security bug in the zlib code. You can avoid its affect in an
+rsync daemon situation by configuring rsync to refuse the "compress" option.
+
+<p><a name="s2_6_3"></a><hr>
+<h3>Daemon path-sanitizing fix in 2.6.3</h3>
+<i class=date>August 12th, 2004</i>
+
+<p>There is a path-sanitizing bug that affects daemon-mode in
+rsync versions prior to 2.6.3, but only if "use chroot" is disabled. It
+does <b>not</b> affect the normal send/receive filenames that specify what
+files should be transferred (this is because these names happen to get
+sanitized twice, and thus the second call removes any lingering leading
+slash(es) that the first call left behind). It does affect certain
+option paths that cause auxiliary files to be read or written.
+
+<p>One potential fix that doesn't require recompiling rsync is to set
+"use chroot = true" for all the modules in the rsyncd.conf file.
+
+<p><a name="s2_6_1"></a><hr>
+<h3>Daemon security fix in 2.6.1</h3>
+<i class=date>April 26th, 2004</i>
+
+<p>There is a security problem in all versions prior to 2.6.1 that affects only
+people running a read/write daemon with "use chroot" disabled. If the user privs
+of a module in the daemon config is anything above "nobody", you are at risk
+of someone crafting an attack that could write a file outside of the module's
+"path" setting (where all its files should be stored). Please either enable
+chroot or upgrade to 2.6.1. People not running a daemon, running a read-only
+daemon, or running a chrooted daemon are totally unaffected.
+
+<p><a name="s2_5_7"></a><hr>
+<h3>Memory overflow fix in 2.5.7</h3>
+<i class=date>December 4th, 2003</i>
+
+<p>Rsync versions prior to 2.5.7 contain a heap overflow vulnerability that
+could be used to remotely run arbitrary code, but this only affects the use of
+rsync as an "rsync daemon" (where rsync handles incoming socket connections,
+typically on port 873).
+
+<!--#include virtual="footer.html" -->
+<br>
+<br>
+<br>
+<br>
+<br>
+<br>
+<br>
+<br>
+<br>
+<br>
+<br>
+<br>
+<br>
--- /dev/null
+I use rsync to backup my wifes home directory across a modem link each
+night. The cron job looks like this
+
+ #!/bin/sh
+ cd ~susan
+ {
+ echo
+ date
+ dest=~/backup/`date +%A`
+ mkdir $dest.new
+ find . -xdev -type f \( -mtime 0 -or -mtime 1 \) -exec cp -aPv "{}"
+ $dest.new \;
+ cnt=`find $dest.new -type f | wc -l`
+ if [ $cnt -gt 0 ]; then
+ rm -rf $dest
+ mv $dest.new $dest
+ fi
+ rm -rf $dest.new
+ rsync -Cavze ssh . samba:backup
+ } >> ~/backup/backup.log 2>&1
+
+
+note that most of this script isn't anything to do with rsync, it just
+creates a daily backup of Susans work in a ~susan/backup/ directory so
+she can retrieve any version from the last week. The last line does
+the rsync of her directory across the modem link to the host
+samba. Note that I am using the -C option which allows me to add
+entries to .cvsignore for stuff that doesn't need to be backed up.
+
+
+
--- /dev/null
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
+<!--Converted with LaTeX2HTML 98.1p1 release (March 2nd, 1998)
+originally by Nikos Drakos (nikos@cbl.leeds.ac.uk), CBLU, University of Leeds
+* revised and updated by: Marcus Hennecke, Ross Moore, Herb Swan
+* with significant contributions from:
+ Jens Lippmann, Marek Rouchal, Martin Wilck and others -->
+<HTML>
+<HEAD>
+<TITLE>Footnotes</TITLE>
+<META NAME="description" CONTENT="Footnotes">
+<META NAME="keywords" CONTENT="tech_report">
+<META NAME="resource-type" CONTENT="document">
+<META NAME="distribution" CONTENT="global">
+<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
+<LINK REL="STYLESHEET" HREF="tech_report.css">
+<LINK REL="previous" HREF="node7.html">
+<LINK REL="up" HREF="tech_report.html">
+</HEAD>
+<BODY >
+
+<DL>
+<DT><A NAME="foot10">... bytes</A><A NAME="foot10"
+ HREF="node2.html#tex2html1"><SUP>1</SUP></A>
+<DD>We have found that
+ values of S between 500 and 1000 are quite good for most purposes
+<PRE>.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+</PRE>
+<DT><A NAME="foot24">... size.</A><A NAME="foot24"
+ HREF="node6.html#tex2html2"><SUP>2</SUP></A>
+<DD>All the tests in this section were
+ carried out using rsync version 0.5
+<PRE>.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+</PRE>
+<DT><A NAME="foot40">... files.</A><A NAME="foot40"
+ HREF="node6.html#tex2html3"><SUP>3</SUP></A>
+<DD>The wall
+ clock time was approximately 2 minutes per run on a 50 MHz SPARC 10
+ running SunOS, using rsh over loopback for communication. GNU diff
+ took about 4 minutes.
+<PRE>.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+.
+</PRE>
+</DL><ADDRESS>
+<I>Andrew Tridgell</I>
+<BR><I>1998-11-09</I>
+</ADDRESS>
+</BODY>
+</HTML>
--- /dev/null
+This is TeX, Version 3.14159 (C version 6.1) (format=latex 97.10.16) 9 NOV 1998 20:10
+**./images.tex
+(images.tex
+LaTeX2e <1996/12/01> patch level 1
+Babel <v3.6h> and hyphenation patterns for american, german, loaded.
+
+(/usr/lib/texmf/texmf/tex/latex/base/article.cls
+Document Class: article 1996/10/31 v1.3u Standard LaTeX document class
+(/usr/lib/texmf/texmf/tex/latex/base/size10.clo
+File: size10.clo 1996/10/31 v1.3u Standard LaTeX file (size option)
+)
+\c@part=\count79
+\c@section=\count80
+\c@subsection=\count81
+\c@subsubsection=\count82
+\c@paragraph=\count83
+\c@subparagraph=\count84
+\c@figure=\count85
+\c@table=\count86
+\abovecaptionskip=\skip41
+\belowcaptionskip=\skip42
+\bibindent=\dimen102
+) (/usr/lib/texmf/texmf/tex/latex/graphics/color.sty
+Package: color 1997/01/07 v1.0d Standard LaTeX Color (DPC)
+(/usr/lib/texmf/texmf/tex/latex/config/color.cfg)
+Package color Info: Driver file: dvips.def on input line 128.
+(/usr/lib/texmf/texmf/tex/latex/graphics/dvips.def
+File: dvips.def 1996/12/12 v3.0d Driver-dependant file (DPC,SPQR)
+) (/usr/lib/texmf/texmf/tex/latex/graphics/dvipsnam.def
+File: dvipsnam.def 1996/12/12 v3.0d Driver-dependant file (DPC,SPQR)
+))
+\sizebox=\box26
+\lthtmlwrite=\write3
+No file images.aux.
+LaTeX Font Info: Checking defaults for OML/cmm/m/it on input line 74.
+LaTeX Font Info: ... okay on input line 74.
+LaTeX Font Info: Checking defaults for T1/cmr/m/n on input line 74.
+LaTeX Font Info: ... okay on input line 74.
+LaTeX Font Info: Checking defaults for OT1/cmr/m/n on input line 74.
+LaTeX Font Info: ... okay on input line 74.
+LaTeX Font Info: Checking defaults for OMS/cmsy/m/n on input line 74.
+LaTeX Font Info: ... okay on input line 74.
+LaTeX Font Info: Checking defaults for OMX/cmex/m/n on input line 74.
+LaTeX Font Info: ... okay on input line 74.
+LaTeX Font Info: Checking defaults for U/cmr/m/n on input line 74.
+LaTeX Font Info: ... okay on input line 74.
+latex2htmlLength hsize=349.0pt
+latex2htmlLength vsize=682.0pt
+latex2htmlLength hoffset=0.0pt
+latex2htmlLength voffset=0.0pt
+latex2htmlLength topmargin=0.0pt
+latex2htmlLength topskip=0.00003pt
+latex2htmlLength headheight=0.0pt
+latex2htmlLength headsep=0.0pt
+latex2htmlLength parskip=0.0pt plus 1.0pt
+latex2htmlLength oddsidemargin=53.0pt
+latex2htmlLength evensidemargin=53.0pt
+LaTeX Font Info: External font `cmex10' loaded for size
+(Font) <7> on input line 94.
+LaTeX Font Info: External font `cmex10' loaded for size
+(Font) <5> on input line 94.
+l2hSize :tex2html_wrap_inline88:6.45831pt::0.0pt::6.43404pt.
+[1
+
+
+]
+l2hSize :tex2html_wrap_inline90:7.44444pt::7.44444pt::6.18402pt.
+[2
+
+
+]
+l2hSize :displaymath158:32.38896pt::0.0pt::349.0pt.
+[3
+
+
+]
+l2hSize :displaymath160:32.38896pt::0.0pt::349.0pt.
+[4
+
+
+]
+l2hSize :tex2html_wrap_inline166:7.33331pt::7.33331pt::39.55894pt.
+[5
+
+
+]
+l2hSize :displaymath170:14.5pt::0.0pt::349.0pt.
+[6
+
+
+]
+l2hSize :displaymath172:14.5pt::0.0pt::349.0pt.
+[7
+
+
+] (images.aux) )
+Here is how much of TeX's memory you used:
+ 518 strings out of 10906
+ 6196 string characters out of 72186
+ 46914 words of memory out of 262141
+ 3454 multiletter control sequences out of 9500
+ 3640 words of font info for 14 fonts, out of 150000 for 255
+ 14 hyphenation exceptions out of 607
+ 23i,5n,19p,182b,215s stack positions out of 300i,40n,60p,3000b,4000s
+
+Output written on images.dvi (7 pages, 2296 bytes).
--- /dev/null
+# LaTeX2HTML 98.1p1 release (March 2nd, 1998)
+# Associate images original text with physical files.
+
+
+$key = q/{inline}alpha{inline}MSF=1.6;AAT;/;
+$cached_env_img{$key} = q|<IMG
+ WIDTH="15" HEIGHT="13" ALIGN="BOTTOM" BORDER="0"
+ SRC="img1.gif"
+ ALT="$\alpha$">|;
+
+$key = q/{displaymath}a(k+1,l+1)=(a(k,l)-X_k+X_l+1)bmodM{displaymath}MSF=1.6;AAT;/;
+$cached_env_img{$key} = q|<IMG
+ WIDTH="322" HEIGHT="28"
+ SRC="img6.gif"
+ ALT="\begin{displaymath}a(k+1,l+1) = (a(k,l) - X_k + X_{l+1}) \bmod M \end{displaymath}">|;
+
+$key = q/{displaymath}a(k,l)=(sum_i=k^lX_i)bmodM{displaymath}MSF=1.6;AAT;/;
+$cached_env_img{$key} = q|<IMG
+ WIDTH="176" HEIGHT="56"
+ SRC="img3.gif"
+ ALT="\begin{displaymath}a(k,l) = (\sum_{i=k}^l X_i) \bmod M \end{displaymath}">|;
+
+$key = q/{displaymath}b(k,l)=(sum_i=k^l(l-i+1)X_i)bmodM{displaymath}MSF=1.6;AAT;/;
+$cached_env_img{$key} = q|<IMG
+ WIDTH="240" HEIGHT="56"
+ SRC="img4.gif"
+ ALT="\begin{displaymath}b(k,l) = (\sum_{i=k}^l (l-i+1)X_i) \bmod M \end{displaymath}">|;
+
+$key = q/{inline}X_kldotsX_l{inline}MSF=1.6;AAT;/;
+$cached_env_img{$key} = q|<IMG
+ WIDTH="67" HEIGHT="29" ALIGN="MIDDLE" BORDER="0"
+ SRC="img5.gif"
+ ALT="$X_k \ldots X_l$">|;
+
+$key = q/{displaymath}b(k+1,l+1)=(b(k,l)-(l-k+1)X_k+a(k+1,l+1))bmodM{displaymath}MSF=1.6;AAT;/;
+$cached_env_img{$key} = q|<IMG
+ WIDTH="454" HEIGHT="28"
+ SRC="img7.gif"
+ ALT="\begin{displaymath}b(k+1,l+1) = (b(k,l) - (l-k+1) X_k + a(k+1,l+1)) \bmod M \end{displaymath}">|;
+
+$key = q/{inline}beta{inline}MSF=1.6;AAT;/;
+$cached_env_img{$key} = q|<IMG
+ WIDTH="14" HEIGHT="29" ALIGN="MIDDLE" BORDER="0"
+ SRC="img2.gif"
+ ALT="$\beta$">|;
+
+1;
+
--- /dev/null
+\batchmode
+\documentclass[a4paper]{article}
+\makeatletter
+\usepackage[dvips]{color}
+\pagecolor[gray]{.7}
+
+
+\makeatletter
+\count@=\the\catcode`\_ \catcode`\_=8
+\newenvironment{tex2html_wrap}{}{} \catcode`\_=\count@
+\makeatother
+\let\mathon=$
+\let\mathoff=$
+\ifx\AtBeginDocument\undefined \newcommand{\AtBeginDocument}[1]{}\fi
+\newbox\sizebox
+\setlength{\hoffset}{0pt}\setlength{\voffset}{0pt}
+\addtolength{\textheight}{\footskip}\setlength{\footskip}{0pt}
+\addtolength{\textheight}{\topmargin}\setlength{\topmargin}{0pt}
+\addtolength{\textheight}{\headheight}\setlength{\headheight}{0pt}
+\addtolength{\textheight}{\headsep}\setlength{\headsep}{0pt}
+\setlength{\textwidth}{349pt}
+\newwrite\lthtmlwrite
+\makeatletter
+\let\realnormalsize=\normalsize
+\global\topskip=2sp
+\def\preveqno{}\let\real@float=\@float \let\realend@float=\end@float
+\def\@float{\let\@savefreelist\@freelist\real@float}
+\def\end@float{\realend@float\global\let\@freelist\@savefreelist}
+\let\real@dbflt=\@dbflt \let\end@dblfloat=\end@float
+\let\@largefloatcheck=\relax
+\def\@dbflt{\let\@savefreelist\@freelist\real@dbflt}
+\def\adjustnormalsize{\def\normalsize{\mathsurround=0pt \realnormalsize
+ \parindent=0pt\abovedisplayskip=0pt\belowdisplayskip=0pt}\normalsize}%
+\def\lthtmltypeout#1{{\let\protect\string\immediate\write\lthtmlwrite{#1}}}%
+\newcommand\lthtmlhboxmathA{\adjustnormalsize\setbox\sizebox=\hbox\bgroup}%
+\newcommand\lthtmlvboxmathA{\adjustnormalsize\setbox\sizebox=\vbox\bgroup%
+ \let\ifinner=\iffalse }%
+\newcommand\lthtmlboxmathZ{\@next\next\@currlist{}{\def\next{\voidb@x}}%
+ \expandafter\box\next\egroup}%
+\newcommand\lthtmlmathtype[1]{\def\lthtmlmathenv{#1}}%
+\newcommand\lthtmllogmath{\lthtmltypeout{l2hSize %
+:\lthtmlmathenv:\the\ht\sizebox::\the\dp\sizebox::\the\wd\sizebox.\preveqno}}%
+\newcommand\lthtmlfigureA[1]{\let\@savefreelist\@freelist
+ \lthtmlmathtype{#1}\lthtmlvboxmathA}%
+\newcommand\lthtmlfigureZ{\lthtmlboxmathZ\lthtmllogmath\copy\sizebox
+ \global\let\@freelist\@savefreelist}%
+\newcommand\lthtmldisplayA[1]{\lthtmlmathtype{#1}\lthtmlvboxmathA}%
+\newcommand\lthtmldisplayB[1]{\edef\preveqno{(\theequation)}%
+ \lthtmldisplayA{#1}\let\@eqnnum\relax}%
+\newcommand\lthtmldisplayZ{\lthtmlboxmathZ\lthtmllogmath\lthtmlsetmath}%
+\newcommand\lthtmlinlinemathA[1]{\lthtmlmathtype{#1}\lthtmlhboxmathA \vrule height1.5ex width0pt }%
+\newcommand\lthtmlinlineA[1]{\lthtmlmathtype{#1}\lthtmlhboxmathA}%
+\newcommand\lthtmlinlineZ{\egroup\expandafter\ifdim\dp\sizebox>0pt %
+ \expandafter\centerinlinemath\fi\lthtmllogmath\lthtmlsetinline}
+\newcommand\lthtmlinlinemathZ{\egroup\expandafter\ifdim\dp\sizebox>0pt %
+ \expandafter\centerinlinemath\fi\lthtmllogmath\lthtmlsetmath}
+\def\lthtmlsetinline{\hbox{\vrule width.1em\vtop{\vbox{%
+ \kern.1em\copy\sizebox}\ifdim\dp\sizebox>0pt\kern.1em\else\kern.3pt\fi
+ \ifdim\hsize>\wd\sizebox \hrule depth1pt\fi}}}
+\def\lthtmlsetmath{\hbox{\vrule width.1em\vtop{\vbox{%
+ \kern.1em\kern0.8 pt\hbox{\hglue.17em\copy\sizebox\hglue0.8 pt}}\kern.3pt%
+ \ifdim\dp\sizebox>0pt\kern.1em\fi \kern0.8 pt%
+ \ifdim\hsize>\wd\sizebox \hrule depth1pt\fi}}}
+\def\centerinlinemath{%\dimen1=\ht\sizebox
+ \dimen1=\ifdim\ht\sizebox<\dp\sizebox \dp\sizebox\else\ht\sizebox\fi
+ \advance\dimen1by.5pt \vrule width0pt height\dimen1 depth\dimen1
+ \dp\sizebox=\dimen1\ht\sizebox=\dimen1\relax}
+
+\def\lthtmlcheckvsize{\ifdim\ht\sizebox<\vsize\expandafter\vfill
+ \else\expandafter\vss\fi}%
+\makeatletter \tracingstats = 1
+
+
+\begin{document}
+\pagestyle{empty}\thispagestyle{empty}%
+\lthtmltypeout{latex2htmlLength hsize=\the\hsize}%
+\lthtmltypeout{latex2htmlLength vsize=\the\vsize}%
+\lthtmltypeout{latex2htmlLength hoffset=\the\hoffset}%
+\lthtmltypeout{latex2htmlLength voffset=\the\voffset}%
+\lthtmltypeout{latex2htmlLength topmargin=\the\topmargin}%
+\lthtmltypeout{latex2htmlLength topskip=\the\topskip}%
+\lthtmltypeout{latex2htmlLength headheight=\the\headheight}%
+\lthtmltypeout{latex2htmlLength headsep=\the\headsep}%
+\lthtmltypeout{latex2htmlLength parskip=\the\parskip}%
+\lthtmltypeout{latex2htmlLength oddsidemargin=\the\oddsidemargin}%
+\makeatletter
+\if@twoside\lthtmltypeout{latex2htmlLength evensidemargin=\the\evensidemargin}%
+\else\lthtmltypeout{latex2htmlLength evensidemargin=\the\oddsidemargin}\fi%
+\makeatother
+\stepcounter{section}
+\stepcounter{section}
+{\newpage\clearpage
+\lthtmlinlinemathA{tex2html_wrap_inline88}%
+$\alpha$%
+\lthtmlinlinemathZ
+\hfill\lthtmlcheckvsize\clearpage}
+
+{\newpage\clearpage
+\lthtmlinlinemathA{tex2html_wrap_inline90}%
+$\beta$%
+\lthtmlinlinemathZ
+\hfill\lthtmlcheckvsize\clearpage}
+
+\stepcounter{section}
+{\newpage\clearpage
+\lthtmldisplayA{displaymath158}%
+\begin{displaymath}a(k,l) = (\sum_{i=k}^l X_i) \bmod M \end{displaymath}%
+\lthtmldisplayZ
+\hfill\lthtmlcheckvsize\clearpage}
+
+{\newpage\clearpage
+\lthtmldisplayA{displaymath160}%
+\begin{displaymath}b(k,l) = (\sum_{i=k}^l (l-i+1)X_i) \bmod M \end{displaymath}%
+\lthtmldisplayZ
+\hfill\lthtmlcheckvsize\clearpage}
+
+{\newpage\clearpage
+\lthtmlinlinemathA{tex2html_wrap_inline166}%
+$X_k \ldots X_l$%
+\lthtmlinlinemathZ
+\hfill\lthtmlcheckvsize\clearpage}
+
+{\newpage\clearpage
+\lthtmldisplayA{displaymath170}%
+\begin{displaymath}a(k+1,l+1) = (a(k,l) - X_k + X_{l+1}) \bmod M \end{displaymath}%
+\lthtmldisplayZ
+\hfill\lthtmlcheckvsize\clearpage}
+
+{\newpage\clearpage
+\lthtmldisplayA{displaymath172}%
+\begin{displaymath}b(k+1,l+1) = (b(k,l) - (l-k+1) X_k + a(k+1,l+1)) \bmod M \end{displaymath}%
+\lthtmldisplayZ
+\hfill\lthtmlcheckvsize\clearpage}
+
+\stepcounter{section}
+\stepcounter{section}
+\stepcounter{section}
+\stepcounter{section}
+
+\end{document}
--- /dev/null
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
+<!--Converted with LaTeX2HTML 98.1p1 release (March 2nd, 1998)
+originally by Nikos Drakos (nikos@cbl.leeds.ac.uk), CBLU, University of Leeds
+* revised and updated by: Marcus Hennecke, Ross Moore, Herb Swan
+* with significant contributions from:
+ Jens Lippmann, Marek Rouchal, Martin Wilck and others -->
+<HTML>
+<HEAD>
+<TITLE>The rsync algorithm</TITLE>
+<META NAME="description" CONTENT="The rsync algorithm">
+<META NAME="keywords" CONTENT="tech_report">
+<META NAME="resource-type" CONTENT="document">
+<META NAME="distribution" CONTENT="global">
+<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
+<LINK REL="STYLESHEET" HREF="tech_report.css">
+<LINK REL="next" HREF="node1.html">
+</HEAD>
+<BODY >
+<!--Navigation Panel-->
+<A NAME="tex2html4"
+ HREF="node1.html">
+<IMG WIDTH="37" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="next"
+ SRC="next.gif"></A>
+<IMG WIDTH="26" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="up"
+ SRC="up.gif">
+<IMG WIDTH="63" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="previous"
+ SRC="previous.gif">
+<BR>
+<B> Next:</B> <A NAME="tex2html5"
+ HREF="node1.html">The problem</A>
+<BR>
+<BR>
+<!--End of Navigation Panel-->
+
+<P>
+
+<P>
+
+<P>
+<H1 ALIGN="CENTER">The rsync algorithm</H1>
+<P ALIGN="CENTER"><STRONG>Andrew Tridgell Paul Mackerras
+<BR>
+Department of Computer Science <BR>
+Australian National University <BR>
+Canberra, ACT 0200, Australia</STRONG></P>
+<P ALIGN="LEFT"></P>
+
+<P>
+
+<H3>Abstract:</H3>
+<DIV>
+This report presents an algorithm for updating a file on one machine
+ to be identical to a file on another machine. We assume that the
+ two machines are connected by a low-bandwidth high-latency
+ bi-directional communications link. The algorithm identifies parts
+ of the source file which are identical to some part of the
+ destination file, and only sends those parts which cannot be matched
+ in this way. Effectively, the algorithm computes a set of
+ differences without having both files on the same machine. The
+ algorithm works best when the files are similar, but will also
+ function correctly and reasonably efficiently when the files are
+ quite different.
+</DIV>
+<P>
+<P>
+<BR><HR>
+<!--Table of Child-Links-->
+<A NAME="CHILD_LINKS"> </A>
+<UL>
+<LI><A NAME="tex2html6"
+ HREF="node1.html">The problem</A>
+<LI><A NAME="tex2html7"
+ HREF="node2.html">The rsync algorithm</A>
+<LI><A NAME="tex2html8"
+ HREF="node3.html">Rolling checksum</A>
+<LI><A NAME="tex2html9"
+ HREF="node4.html">Checksum searching</A>
+<LI><A NAME="tex2html10"
+ HREF="node5.html">Pipelining</A>
+<LI><A NAME="tex2html11"
+ HREF="node6.html">Results</A>
+<LI><A NAME="tex2html12"
+ HREF="node7.html">Availability</A>
+<LI><A NAME="tex2html13"
+ HREF="node8.html">About this document ... </A>
+</UL>
+<!--End of Table of Child-Links-->
+<BR><HR>
+<ADDRESS>
+<I>Andrew Tridgell</I>
+<BR><I>1998-11-09</I>
+</ADDRESS>
+</BODY>
+</HTML>
--- /dev/null
+# LaTeX2HTML 98.1p1 release (March 2nd, 1998)
+# Associate labels original text with physical files.
+
+
+1;
+
--- /dev/null
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
+<!--Converted with LaTeX2HTML 98.1p1 release (March 2nd, 1998)
+originally by Nikos Drakos (nikos@cbl.leeds.ac.uk), CBLU, University of Leeds
+* revised and updated by: Marcus Hennecke, Ross Moore, Herb Swan
+* with significant contributions from:
+ Jens Lippmann, Marek Rouchal, Martin Wilck and others -->
+<HTML>
+<HEAD>
+<TITLE>The problem</TITLE>
+<META NAME="description" CONTENT="The problem">
+<META NAME="keywords" CONTENT="tech_report">
+<META NAME="resource-type" CONTENT="document">
+<META NAME="distribution" CONTENT="global">
+<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
+<LINK REL="STYLESHEET" HREF="tech_report.css">
+<LINK REL="next" HREF="node2.html">
+<LINK REL="previous" HREF="tech_report.html">
+<LINK REL="up" HREF="tech_report.html">
+<LINK REL="next" HREF="node2.html">
+</HEAD>
+<BODY >
+<!--Navigation Panel-->
+<A NAME="tex2html22"
+ HREF="node2.html">
+<IMG WIDTH="37" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="next"
+ SRC="next.gif"></A>
+<A NAME="tex2html20"
+ HREF="tech_report.html">
+<IMG WIDTH="26" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="up"
+ SRC="up.gif"></A>
+<A NAME="tex2html14"
+ HREF="tech_report.html">
+<IMG WIDTH="63" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="previous"
+ SRC="previous.gif"></A>
+<BR>
+<B> Next:</B> <A NAME="tex2html23"
+ HREF="node2.html">The rsync algorithm</A>
+<B> Up:</B> <A NAME="tex2html21"
+ HREF="tech_report.html">The rsync algorithm</A>
+<B> Previous:</B> <A NAME="tex2html15"
+ HREF="tech_report.html">The rsync algorithm</A>
+<BR>
+<BR>
+<!--End of Navigation Panel-->
+
+<H1><A NAME="SECTION00010000000000000000">
+The problem</A>
+</H1>
+
+<P>
+Imagine you have two files, <I>A</I> and <I>B</I>, and you wish to update <I>B</I> to be
+the same as <I>A</I>. The obvious method is to copy <I>A</I> onto <I>B</I>.
+
+<P>
+Now imagine that the two files are on machines connected by a slow
+communications link, for example a dial up IP link. If <I>A</I> is large,
+copying <I>A</I> onto <I>B</I> will be slow. To make it faster you could
+compress <I>A</I> before sending it, but that will usually only gain a
+factor of 2 to 4.
+
+<P>
+Now assume that <I>A</I> and <I>B</I> are quite similar, perhaps both derived
+from the same original file. To really speed things up you would need
+to take advantage of this similarity. A common method is to send just
+the differences between <I>A</I> and <I>B</I> down the link and then use this
+list of differences to reconstruct the file.
+
+<P>
+The problem is that the normal methods for creating a set of
+differences between two files rely on being able to read both files.
+Thus they require that both files are available beforehand at one end
+of the link. If they are not both available on the same machine,
+these algorithms cannot be used (once you had copied the file over,
+you wouldn't need the differences). This is the problem that rsync
+addresses.
+
+<P>
+The rsync algorithm efficiently computes which parts of a source file
+match some part of an existing destination file. These parts need not
+be sent across the link; all that is needed is a reference to the part
+of the destination file. Only parts of the source file which are not
+matched in this way need to be sent verbatim. The receiver can then
+construct a copy of the source file using the references to parts of
+the existing destination file and the verbatim material.
+
+<P>
+Trivially, the data sent to the receiver can be compressed using any
+of a range of common compression algorithms, for further speed
+improvements.
+
+<P>
+<HR>
+<!--Navigation Panel-->
+<A NAME="tex2html22"
+ HREF="node2.html">
+<IMG WIDTH="37" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="next"
+ SRC="next.gif"></A>
+<A NAME="tex2html20"
+ HREF="tech_report.html">
+<IMG WIDTH="26" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="up"
+ SRC="up.gif"></A>
+<A NAME="tex2html14"
+ HREF="tech_report.html">
+<IMG WIDTH="63" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="previous"
+ SRC="previous.gif"></A>
+<BR>
+<B> Next:</B> <A NAME="tex2html23"
+ HREF="node2.html">The rsync algorithm</A>
+<B> Up:</B> <A NAME="tex2html21"
+ HREF="tech_report.html">The rsync algorithm</A>
+<B> Previous:</B> <A NAME="tex2html15"
+ HREF="tech_report.html">The rsync algorithm</A>
+<!--End of Navigation Panel-->
+<ADDRESS>
+<I>Andrew Tridgell</I>
+<BR><I>1998-11-09</I>
+</ADDRESS>
+</BODY>
+</HTML>
--- /dev/null
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
+<!--Converted with LaTeX2HTML 98.1p1 release (March 2nd, 1998)
+originally by Nikos Drakos (nikos@cbl.leeds.ac.uk), CBLU, University of Leeds
+* revised and updated by: Marcus Hennecke, Ross Moore, Herb Swan
+* with significant contributions from:
+ Jens Lippmann, Marek Rouchal, Martin Wilck and others -->
+<HTML>
+<HEAD>
+<TITLE>The rsync algorithm</TITLE>
+<META NAME="description" CONTENT="The rsync algorithm">
+<META NAME="keywords" CONTENT="tech_report">
+<META NAME="resource-type" CONTENT="document">
+<META NAME="distribution" CONTENT="global">
+<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
+<LINK REL="STYLESHEET" HREF="tech_report.css">
+<LINK REL="next" HREF="node3.html">
+<LINK REL="previous" HREF="node1.html">
+<LINK REL="up" HREF="tech_report.html">
+<LINK REL="next" HREF="node3.html">
+</HEAD>
+<BODY >
+<!--Navigation Panel-->
+<A NAME="tex2html32"
+ HREF="node3.html">
+<IMG WIDTH="37" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="next"
+ SRC="next.gif"></A>
+<A NAME="tex2html30"
+ HREF="tech_report.html">
+<IMG WIDTH="26" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="up"
+ SRC="up.gif"></A>
+<A NAME="tex2html24"
+ HREF="node1.html">
+<IMG WIDTH="63" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="previous"
+ SRC="previous.gif"></A>
+<BR>
+<B> Next:</B> <A NAME="tex2html33"
+ HREF="node3.html">Rolling checksum</A>
+<B> Up:</B> <A NAME="tex2html31"
+ HREF="tech_report.html">The rsync algorithm</A>
+<B> Previous:</B> <A NAME="tex2html25"
+ HREF="node1.html">The problem</A>
+<BR>
+<BR>
+<!--End of Navigation Panel-->
+
+<H1><A NAME="SECTION00020000000000000000">
+The rsync algorithm</A>
+</H1>
+
+<P>
+Suppose we have two general purpose computers <IMG
+ WIDTH="15" HEIGHT="13" ALIGN="BOTTOM" BORDER="0"
+ SRC="img1.gif"
+ ALT="$\alpha$">
+and <IMG
+ WIDTH="14" HEIGHT="29" ALIGN="MIDDLE" BORDER="0"
+ SRC="img2.gif"
+ ALT="$\beta$">.
+Computer <IMG
+ WIDTH="15" HEIGHT="13" ALIGN="BOTTOM" BORDER="0"
+ SRC="img1.gif"
+ ALT="$\alpha$">
+has access to a file <I>A</I> and <IMG
+ WIDTH="14" HEIGHT="29" ALIGN="MIDDLE" BORDER="0"
+ SRC="img2.gif"
+ ALT="$\beta$">
+has access to
+file <I>B</I>, where <I>A</I> and <I>B</I> are ``similar''. There is a slow
+communications link between <IMG
+ WIDTH="15" HEIGHT="13" ALIGN="BOTTOM" BORDER="0"
+ SRC="img1.gif"
+ ALT="$\alpha$">
+and <IMG
+ WIDTH="14" HEIGHT="29" ALIGN="MIDDLE" BORDER="0"
+ SRC="img2.gif"
+ ALT="$\beta$">.
+
+<P>
+The rsync algorithm consists of the following steps:
+
+<P>
+<DL COMPACT>
+<DT>1.
+<DD><IMG
+ WIDTH="14" HEIGHT="29" ALIGN="MIDDLE" BORDER="0"
+ SRC="img2.gif"
+ ALT="$\beta$">
+splits the file <I>B</I> into a series of non-overlapping
+fixed-sized blocks of size S bytes<A NAME="tex2html1"
+ HREF="footnode.html#foot10"><SUP>1</SUP></A>.
+ The last block may be shorter than <I>S</I> bytes.
+
+<P>
+<DT>2.
+<DD>For each of these blocks <IMG
+ WIDTH="14" HEIGHT="29" ALIGN="MIDDLE" BORDER="0"
+ SRC="img2.gif"
+ ALT="$\beta$">
+calculates two checksums:
+ a weak ``rolling'' 32-bit checksum (described below) and a strong
+ 128-bit MD4 checksum.
+
+<P>
+<DT>3.
+<DD><IMG
+ WIDTH="14" HEIGHT="29" ALIGN="MIDDLE" BORDER="0"
+ SRC="img2.gif"
+ ALT="$\beta$">
+sends these checksums to <IMG
+ WIDTH="15" HEIGHT="13" ALIGN="BOTTOM" BORDER="0"
+ SRC="img1.gif"
+ ALT="$\alpha$">.
+
+<DT>4.
+<DD><IMG
+ WIDTH="15" HEIGHT="13" ALIGN="BOTTOM" BORDER="0"
+ SRC="img1.gif"
+ ALT="$\alpha$">
+searches through <I>A</I> to find all blocks of length <I>S</I> bytes (at any offset, not just multiples of <I>S</I>) that have the same
+ weak and strong checksum as one of the blocks of <I>B</I>. This can be
+ done in a single pass very quickly using a special property of the
+ rolling checksum described below.
+
+<DT>5.
+<DD><IMG
+ WIDTH="15" HEIGHT="13" ALIGN="BOTTOM" BORDER="0"
+ SRC="img1.gif"
+ ALT="$\alpha$">
+sends <IMG
+ WIDTH="14" HEIGHT="29" ALIGN="MIDDLE" BORDER="0"
+ SRC="img2.gif"
+ ALT="$\beta$">
+a sequence of instructions for
+ constructing a copy of <I>A</I>. Each instruction is either a reference
+ to a block of <I>B</I>, or literal data. Literal data is sent only for
+ those sections of <I>A</I> which did not match any of the blocks of <I>B</I>.
+</DL>
+<P>
+The end result is that <IMG
+ WIDTH="14" HEIGHT="29" ALIGN="MIDDLE" BORDER="0"
+ SRC="img2.gif"
+ ALT="$\beta$">
+gets a copy of <I>A</I>, but only the pieces
+of <I>A</I> that are not found in <I>B</I> (plus a small amount of data for
+checksums and block indexes) are sent over the link. The algorithm
+also only requires one round trip, which minimises the impact of the
+link latency.
+
+<P>
+The most important details of the algorithm are the rolling checksum
+and the associated multi-alternate search mechanism which allows the
+all-offsets checksum search to proceed very quickly. These will be
+discussed in greater detail below.
+
+<P>
+<HR>
+<!--Navigation Panel-->
+<A NAME="tex2html32"
+ HREF="node3.html">
+<IMG WIDTH="37" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="next"
+ SRC="next.gif"></A>
+<A NAME="tex2html30"
+ HREF="tech_report.html">
+<IMG WIDTH="26" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="up"
+ SRC="up.gif"></A>
+<A NAME="tex2html24"
+ HREF="node1.html">
+<IMG WIDTH="63" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="previous"
+ SRC="previous.gif"></A>
+<BR>
+<B> Next:</B> <A NAME="tex2html33"
+ HREF="node3.html">Rolling checksum</A>
+<B> Up:</B> <A NAME="tex2html31"
+ HREF="tech_report.html">The rsync algorithm</A>
+<B> Previous:</B> <A NAME="tex2html25"
+ HREF="node1.html">The problem</A>
+<!--End of Navigation Panel-->
+<ADDRESS>
+<I>Andrew Tridgell</I>
+<BR><I>1998-11-09</I>
+</ADDRESS>
+</BODY>
+</HTML>
--- /dev/null
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
+<!--Converted with LaTeX2HTML 98.1p1 release (March 2nd, 1998)
+originally by Nikos Drakos (nikos@cbl.leeds.ac.uk), CBLU, University of Leeds
+* revised and updated by: Marcus Hennecke, Ross Moore, Herb Swan
+* with significant contributions from:
+ Jens Lippmann, Marek Rouchal, Martin Wilck and others -->
+<HTML>
+<HEAD>
+<TITLE>Rolling checksum</TITLE>
+<META NAME="description" CONTENT="Rolling checksum">
+<META NAME="keywords" CONTENT="tech_report">
+<META NAME="resource-type" CONTENT="document">
+<META NAME="distribution" CONTENT="global">
+<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
+<LINK REL="STYLESHEET" HREF="tech_report.css">
+<LINK REL="next" HREF="node4.html">
+<LINK REL="previous" HREF="node2.html">
+<LINK REL="up" HREF="tech_report.html">
+<LINK REL="next" HREF="node4.html">
+</HEAD>
+<BODY >
+<!--Navigation Panel-->
+<A NAME="tex2html42"
+ HREF="node4.html">
+<IMG WIDTH="37" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="next"
+ SRC="next.gif"></A>
+<A NAME="tex2html40"
+ HREF="tech_report.html">
+<IMG WIDTH="26" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="up"
+ SRC="up.gif"></A>
+<A NAME="tex2html34"
+ HREF="node2.html">
+<IMG WIDTH="63" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="previous"
+ SRC="previous.gif"></A>
+<BR>
+<B> Next:</B> <A NAME="tex2html43"
+ HREF="node4.html">Checksum searching</A>
+<B> Up:</B> <A NAME="tex2html41"
+ HREF="tech_report.html">The rsync algorithm</A>
+<B> Previous:</B> <A NAME="tex2html35"
+ HREF="node2.html">The rsync algorithm</A>
+<BR>
+<BR>
+<!--End of Navigation Panel-->
+
+<H1><A NAME="SECTION00030000000000000000">
+Rolling checksum</A>
+</H1>
+
+<P>
+The weak rolling checksum used in the rsync algorithm needs to have
+the property that it is very cheap to calculate the checksum of a
+buffer
+<!-- MATH: $X_2 .. X_{n+1}$ -->
+<I>X</I><SUB>2</SUB> .. <I>X</I><SUB><I>n</I>+1</SUB> given the checksum of buffer
+<!-- MATH: $X_1 .. X_n$ -->
+<I>X</I><SUB>1</SUB> .. <I>X</I><SUB><I>n</I></SUB> and
+the values of the bytes <I>X</I><SUB>1</SUB> and <I>X</I><SUB><I>n</I>+1</SUB>.
+
+<P>
+The weak checksum algorithm we used in our implementation was inspired
+by Mark Adler's adler-32 checksum. Our checksum is defined by
+<BR><P></P>
+<DIV ALIGN="CENTER">
+<!-- MATH: \begin{displaymath}
+a(k,l) = (\sum_{i=k}^l X_i) \bmod M
+\end{displaymath} -->
+
+
+<IMG
+ WIDTH="176" HEIGHT="56"
+ SRC="img3.gif"
+ ALT="\begin{displaymath}a(k,l) = (\sum_{i=k}^l X_i) \bmod M \end{displaymath}">
+</DIV>
+<BR CLEAR="ALL">
+<P></P>
+<BR><P></P>
+<DIV ALIGN="CENTER">
+<!-- MATH: \begin{displaymath}
+b(k,l) = (\sum_{i=k}^l (l-i+1)X_i) \bmod M
+\end{displaymath} -->
+
+
+<IMG
+ WIDTH="240" HEIGHT="56"
+ SRC="img4.gif"
+ ALT="\begin{displaymath}b(k,l) = (\sum_{i=k}^l (l-i+1)X_i) \bmod M \end{displaymath}">
+</DIV>
+<BR CLEAR="ALL">
+<P></P>
+<BR><P></P>
+<DIV ALIGN="CENTER">
+<!-- MATH: \begin{displaymath}
+s(k,l) = a(k,l) + 2^{16} b(k,l)
+\end{displaymath} -->
+
+
+<I>s</I>(<I>k</I>,<I>l</I>) = <I>a</I>(<I>k</I>,<I>l</I>) + 2<SUP>16</SUP> <I>b</I>(<I>k</I>,<I>l</I>)
+</DIV>
+<BR CLEAR="ALL">
+<P></P>
+<P>
+where <I>s</I>(<I>k</I>,<I>l</I>) is the rolling checksum of the bytes
+<!-- MATH: $X_k \ldots X_l$ -->
+<IMG
+ WIDTH="67" HEIGHT="29" ALIGN="MIDDLE" BORDER="0"
+ SRC="img5.gif"
+ ALT="$X_k \ldots X_l$">.
+For simplicity and speed, we use
+<!-- MATH: $M = 2^{16}$ -->
+<I>M</I> = 2<SUP>16</SUP>.
+
+<P>
+The important property of this checksum is that successive values can
+be computed very efficiently using the recurrence relations
+
+<P>
+<BR><P></P>
+<DIV ALIGN="CENTER">
+<!-- MATH: \begin{displaymath}
+a(k+1,l+1) = (a(k,l) - X_k + X_{l+1}) \bmod M
+\end{displaymath} -->
+
+
+<IMG
+ WIDTH="322" HEIGHT="28"
+ SRC="img6.gif"
+ ALT="\begin{displaymath}a(k+1,l+1) = (a(k,l) - X_k + X_{l+1}) \bmod M \end{displaymath}">
+</DIV>
+<BR CLEAR="ALL">
+<P></P>
+<BR><P></P>
+<DIV ALIGN="CENTER">
+<!-- MATH: \begin{displaymath}
+b(k+1,l+1) = (b(k,l) - (l-k+1) X_k + a(k+1,l+1)) \bmod M
+\end{displaymath} -->
+
+
+<IMG
+ WIDTH="454" HEIGHT="28"
+ SRC="img7.gif"
+ ALT="\begin{displaymath}b(k+1,l+1) = (b(k,l) - (l-k+1) X_k + a(k+1,l+1)) \bmod M \end{displaymath}">
+</DIV>
+<BR CLEAR="ALL">
+<P></P>
+<P>
+Thus the checksum can be calculated for blocks of length S at all
+possible offsets within a file in a ``rolling'' fashion, with very
+little computation at each point.
+
+<P>
+Despite its simplicity, this checksum was found to be quite adequate as
+a first level check for a match of two file blocks. We have found in
+practice that the probability of this checksum matching when the
+blocks are not equal is quite low. This is important because the much
+more expensive strong checksum must be calculated for each block where
+the weak checksum matches.
+
+<P>
+<HR>
+<!--Navigation Panel-->
+<A NAME="tex2html42"
+ HREF="node4.html">
+<IMG WIDTH="37" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="next"
+ SRC="next.gif"></A>
+<A NAME="tex2html40"
+ HREF="tech_report.html">
+<IMG WIDTH="26" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="up"
+ SRC="up.gif"></A>
+<A NAME="tex2html34"
+ HREF="node2.html">
+<IMG WIDTH="63" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="previous"
+ SRC="previous.gif"></A>
+<BR>
+<B> Next:</B> <A NAME="tex2html43"
+ HREF="node4.html">Checksum searching</A>
+<B> Up:</B> <A NAME="tex2html41"
+ HREF="tech_report.html">The rsync algorithm</A>
+<B> Previous:</B> <A NAME="tex2html35"
+ HREF="node2.html">The rsync algorithm</A>
+<!--End of Navigation Panel-->
+<ADDRESS>
+<I>Andrew Tridgell</I>
+<BR><I>1998-11-09</I>
+</ADDRESS>
+</BODY>
+</HTML>
--- /dev/null
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
+<!--Converted with LaTeX2HTML 98.1p1 release (March 2nd, 1998)
+originally by Nikos Drakos (nikos@cbl.leeds.ac.uk), CBLU, University of Leeds
+* revised and updated by: Marcus Hennecke, Ross Moore, Herb Swan
+* with significant contributions from:
+ Jens Lippmann, Marek Rouchal, Martin Wilck and others -->
+<HTML>
+<HEAD>
+<TITLE>Checksum searching</TITLE>
+<META NAME="description" CONTENT="Checksum searching">
+<META NAME="keywords" CONTENT="tech_report">
+<META NAME="resource-type" CONTENT="document">
+<META NAME="distribution" CONTENT="global">
+<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
+<LINK REL="STYLESHEET" HREF="tech_report.css">
+<LINK REL="next" HREF="node5.html">
+<LINK REL="previous" HREF="node3.html">
+<LINK REL="up" HREF="tech_report.html">
+<LINK REL="next" HREF="node5.html">
+</HEAD>
+<BODY >
+<!--Navigation Panel-->
+<A NAME="tex2html52"
+ HREF="node5.html">
+<IMG WIDTH="37" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="next"
+ SRC="next.gif"></A>
+<A NAME="tex2html50"
+ HREF="tech_report.html">
+<IMG WIDTH="26" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="up"
+ SRC="up.gif"></A>
+<A NAME="tex2html44"
+ HREF="node3.html">
+<IMG WIDTH="63" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="previous"
+ SRC="previous.gif"></A>
+<BR>
+<B> Next:</B> <A NAME="tex2html53"
+ HREF="node5.html">Pipelining</A>
+<B> Up:</B> <A NAME="tex2html51"
+ HREF="tech_report.html">The rsync algorithm</A>
+<B> Previous:</B> <A NAME="tex2html45"
+ HREF="node3.html">Rolling checksum</A>
+<BR>
+<BR>
+<!--End of Navigation Panel-->
+
+<H1><A NAME="SECTION00040000000000000000">
+Checksum searching</A>
+</H1>
+
+<P>
+Once <IMG
+ WIDTH="15" HEIGHT="13" ALIGN="BOTTOM" BORDER="0"
+ SRC="img1.gif"
+ ALT="$\alpha$">
+has received the list of checksums of the blocks of <I>B</I>,
+it must search <I>A</I> for any blocks at any offset that match the
+checksum of some block of <I>B</I>. The basic strategy is to compute the
+32-bit rolling checksum for a block of length <I>S</I> starting at each
+byte of <I>A</I> in turn, and for each checksum, search the list for a
+match. To do this our implementation uses a
+simple 3 level searching scheme.
+
+<P>
+The first level uses a 16-bit hash of the 32-bit rolling checksum and
+a 2<SUP>16</SUP> entry hash table. The list of checksum values (i.e., the
+checksums from the blocks of <I>B</I>) is sorted according to the 16-bit
+hash of the 32-bit rolling checksum. Each entry in the hash table
+points to the first element of the list for that hash value, or
+contains a null value if no element of the list has that hash value.
+
+<P>
+At each offset in the file the 32-bit rolling checksum and its 16-bit
+hash are calculated. If the hash table entry for that hash value is
+not a null value, the second level check is invoked.
+
+<P>
+The second level check involves scanning the sorted checksum list
+starting with the entry pointed to by the hash table entry, looking
+for an entry whose 32-bit rolling checksum matches the current value.
+The scan terminates when it reaches an entry whose 16-bit hash
+differs. If this search finds a match, the third level check is
+invoked.
+
+<P>
+The third level check involves calculating the strong checksum for the
+current offset in the file and comparing it with the strong checksum
+value in the current list entry. If the two strong checksums match,
+we assume that we have found a block of <I>A</I> which matches a block of
+<I>B</I>. In fact the blocks could be different, but the probability of
+this is microscopic, and in practice this is a reasonable assumption.
+
+<P>
+When a match is found, <IMG
+ WIDTH="15" HEIGHT="13" ALIGN="BOTTOM" BORDER="0"
+ SRC="img1.gif"
+ ALT="$\alpha$">
+sends <IMG
+ WIDTH="14" HEIGHT="29" ALIGN="MIDDLE" BORDER="0"
+ SRC="img2.gif"
+ ALT="$\beta$">
+the data in <I>A</I> between
+the current file offset and the end of the previous match, followed by
+the index of the block in <I>B</I> that matched. This data is sent
+immediately a match is found, which allows us to overlap the
+communication with further computation.
+
+<P>
+If no match is found at a given offset in the file, the rolling
+checksum is updated to the next offset and the search proceeds. If a
+match is found, the search is restarted at the end of the matched
+block. This strategy saves a considerable amount of computation for
+the common case where the two files are nearly identical. In
+addition, it would be a simple matter to encode the block indexes as
+runs, for the common case where a portion of <I>A</I> matches a series of
+blocks of <I>B</I> in order.
+
+<P>
+<HR>
+<!--Navigation Panel-->
+<A NAME="tex2html52"
+ HREF="node5.html">
+<IMG WIDTH="37" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="next"
+ SRC="next.gif"></A>
+<A NAME="tex2html50"
+ HREF="tech_report.html">
+<IMG WIDTH="26" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="up"
+ SRC="up.gif"></A>
+<A NAME="tex2html44"
+ HREF="node3.html">
+<IMG WIDTH="63" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="previous"
+ SRC="previous.gif"></A>
+<BR>
+<B> Next:</B> <A NAME="tex2html53"
+ HREF="node5.html">Pipelining</A>
+<B> Up:</B> <A NAME="tex2html51"
+ HREF="tech_report.html">The rsync algorithm</A>
+<B> Previous:</B> <A NAME="tex2html45"
+ HREF="node3.html">Rolling checksum</A>
+<!--End of Navigation Panel-->
+<ADDRESS>
+<I>Andrew Tridgell</I>
+<BR><I>1998-11-09</I>
+</ADDRESS>
+</BODY>
+</HTML>
--- /dev/null
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
+<!--Converted with LaTeX2HTML 98.1p1 release (March 2nd, 1998)
+originally by Nikos Drakos (nikos@cbl.leeds.ac.uk), CBLU, University of Leeds
+* revised and updated by: Marcus Hennecke, Ross Moore, Herb Swan
+* with significant contributions from:
+ Jens Lippmann, Marek Rouchal, Martin Wilck and others -->
+<HTML>
+<HEAD>
+<TITLE>Pipelining</TITLE>
+<META NAME="description" CONTENT="Pipelining">
+<META NAME="keywords" CONTENT="tech_report">
+<META NAME="resource-type" CONTENT="document">
+<META NAME="distribution" CONTENT="global">
+<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
+<LINK REL="STYLESHEET" HREF="tech_report.css">
+<LINK REL="next" HREF="node6.html">
+<LINK REL="previous" HREF="node4.html">
+<LINK REL="up" HREF="tech_report.html">
+<LINK REL="next" HREF="node6.html">
+</HEAD>
+<BODY >
+<!--Navigation Panel-->
+<A NAME="tex2html62"
+ HREF="node6.html">
+<IMG WIDTH="37" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="next"
+ SRC="next.gif"></A>
+<A NAME="tex2html60"
+ HREF="tech_report.html">
+<IMG WIDTH="26" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="up"
+ SRC="up.gif"></A>
+<A NAME="tex2html54"
+ HREF="node4.html">
+<IMG WIDTH="63" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="previous"
+ SRC="previous.gif"></A>
+<BR>
+<B> Next:</B> <A NAME="tex2html63"
+ HREF="node6.html">Results</A>
+<B> Up:</B> <A NAME="tex2html61"
+ HREF="tech_report.html">The rsync algorithm</A>
+<B> Previous:</B> <A NAME="tex2html55"
+ HREF="node4.html">Checksum searching</A>
+<BR>
+<BR>
+<!--End of Navigation Panel-->
+
+<H1><A NAME="SECTION00050000000000000000">
+Pipelining</A>
+</H1>
+
+<P>
+The above sections describe the process for constructing a copy of one
+file on a remote system. If we have a several files to copy, we can
+gain a considerable latency advantage by pipelining the process.
+
+<P>
+This involves <IMG
+ WIDTH="14" HEIGHT="29" ALIGN="MIDDLE" BORDER="0"
+ SRC="img2.gif"
+ ALT="$\beta$">
+initiating two independent processes. One of the
+processes generates and sends the checksums to <IMG
+ WIDTH="15" HEIGHT="13" ALIGN="BOTTOM" BORDER="0"
+ SRC="img1.gif"
+ ALT="$\alpha$">
+while the
+other receives the difference information from <IMG
+ WIDTH="15" HEIGHT="13" ALIGN="BOTTOM" BORDER="0"
+ SRC="img1.gif"
+ ALT="$\alpha$">
+and
+reconstructs the files.
+
+<P>
+If the communications link is buffered then these two processes can
+proceed independently and the link should be kept fully utilised in
+both directions for most of the time.
+
+<P>
+<BR><HR>
+<ADDRESS>
+<I>Andrew Tridgell</I>
+<BR><I>1998-11-09</I>
+</ADDRESS>
+</BODY>
+</HTML>
--- /dev/null
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
+<!--Converted with LaTeX2HTML 98.1p1 release (March 2nd, 1998)
+originally by Nikos Drakos (nikos@cbl.leeds.ac.uk), CBLU, University of Leeds
+* revised and updated by: Marcus Hennecke, Ross Moore, Herb Swan
+* with significant contributions from:
+ Jens Lippmann, Marek Rouchal, Martin Wilck and others -->
+<HTML>
+<HEAD>
+<TITLE>Results</TITLE>
+<META NAME="description" CONTENT="Results">
+<META NAME="keywords" CONTENT="tech_report">
+<META NAME="resource-type" CONTENT="document">
+<META NAME="distribution" CONTENT="global">
+<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
+<LINK REL="STYLESHEET" HREF="tech_report.css">
+<LINK REL="next" HREF="node7.html">
+<LINK REL="previous" HREF="node5.html">
+<LINK REL="up" HREF="tech_report.html">
+<LINK REL="next" HREF="node7.html">
+</HEAD>
+<BODY >
+<!--Navigation Panel-->
+<A NAME="tex2html72"
+ HREF="node7.html">
+<IMG WIDTH="37" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="next"
+ SRC="next.gif"></A>
+<A NAME="tex2html70"
+ HREF="tech_report.html">
+<IMG WIDTH="26" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="up"
+ SRC="up.gif"></A>
+<A NAME="tex2html64"
+ HREF="node5.html">
+<IMG WIDTH="63" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="previous"
+ SRC="previous.gif"></A>
+<BR>
+<B> Next:</B> <A NAME="tex2html73"
+ HREF="node7.html">Availability</A>
+<B> Up:</B> <A NAME="tex2html71"
+ HREF="tech_report.html">The rsync algorithm</A>
+<B> Previous:</B> <A NAME="tex2html65"
+ HREF="node5.html">Pipelining</A>
+<BR>
+<BR>
+<!--End of Navigation Panel-->
+
+<H1><A NAME="SECTION00060000000000000000">
+Results</A>
+</H1>
+
+<P>
+To test the algorithm, tar files were created of the Linux kernel
+sources for two versions of the kernel. The two kernel versions were
+1.99.10 and 2.0.0. These tar files are approximately 24MB in size and
+are separated by 5 released patch levels.
+
+<P>
+Out of the 2441 files in the 1.99.10 release, 291 files had changed in
+the 2.0.0 release, 19 files had been removed and 25 files had been
+added.
+
+<P>
+A ``diff'' of the two tar files using the standard GNU diff utility
+produced over 32 thousand lines of output totalling 2.1 MB.
+
+<P>
+The following table shows the results for rsync between the two files
+with a varying block size.<A NAME="tex2html2"
+ HREF="footnode.html#foot24"><SUP>2</SUP></A>
+
+<P>
+<BR>
+<BR>
+<TABLE CELLPADDING=3 BORDER="1">
+<TR><TD ALIGN="LEFT"><B> block</B></TD>
+<TD ALIGN="LEFT"><B> matches</B></TD>
+<TD ALIGN="LEFT"><B> tag</B></TD>
+<TD ALIGN="LEFT"><B> false</B></TD>
+<TD ALIGN="LEFT"><B> data</B></TD>
+<TD ALIGN="LEFT"><B> written</B></TD>
+<TD ALIGN="LEFT"><B> read</B></TD>
+</TR>
+<TR><TD ALIGN="LEFT"><B> size</B></TD>
+<TD ALIGN="LEFT"> </TD>
+<TD ALIGN="LEFT"><B> hits</B></TD>
+<TD ALIGN="LEFT"><B> alarms</B></TD>
+<TD ALIGN="LEFT"> </TD>
+<TD ALIGN="LEFT"> </TD>
+<TD ALIGN="LEFT"> </TD>
+</TR>
+<TR><TD ALIGN="LEFT"><P>
+300</TD>
+<TD ALIGN="LEFT">64247</TD>
+<TD ALIGN="LEFT">3817434</TD>
+<TD ALIGN="LEFT">948</TD>
+<TD ALIGN="LEFT">5312200</TD>
+<TD ALIGN="LEFT">5629158</TD>
+<TD ALIGN="LEFT">1632284</TD>
+</TR>
+<TR><TD ALIGN="LEFT">500</TD>
+<TD ALIGN="LEFT">46989</TD>
+<TD ALIGN="LEFT">620013</TD>
+<TD ALIGN="LEFT">64</TD>
+<TD ALIGN="LEFT">1091900</TD>
+<TD ALIGN="LEFT">1283906</TD>
+<TD ALIGN="LEFT">979384</TD>
+</TR>
+<TR><TD ALIGN="LEFT">700</TD>
+<TD ALIGN="LEFT">33255</TD>
+<TD ALIGN="LEFT">571970</TD>
+<TD ALIGN="LEFT">22</TD>
+<TD ALIGN="LEFT">1307800</TD>
+<TD ALIGN="LEFT">1444346</TD>
+<TD ALIGN="LEFT">699564</TD>
+</TR>
+<TR><TD ALIGN="LEFT">900</TD>
+<TD ALIGN="LEFT">25686</TD>
+<TD ALIGN="LEFT">525058</TD>
+<TD ALIGN="LEFT">24</TD>
+<TD ALIGN="LEFT">1469500</TD>
+<TD ALIGN="LEFT">1575438</TD>
+<TD ALIGN="LEFT">544124</TD>
+</TR>
+<TR><TD ALIGN="LEFT">1100</TD>
+<TD ALIGN="LEFT">20848</TD>
+<TD ALIGN="LEFT">496844</TD>
+<TD ALIGN="LEFT">21</TD>
+<TD ALIGN="LEFT">1654500</TD>
+<TD ALIGN="LEFT">1740838</TD>
+<TD ALIGN="LEFT">445204</TD>
+</TR>
+</TABLE>
+<BR>
+<BR>
+
+<P>
+In each case, the CPU time taken was less than the
+time it takes to run ``diff'' on the two files.<A NAME="tex2html3"
+ HREF="footnode.html#foot40"><SUP>3</SUP></A>
+
+<P>
+The columns in the table are as follows:
+
+<P>
+<DL>
+<DT><STRONG>block size</STRONG>
+<DD>The size in bytes of the checksummed blocks.
+<DT><STRONG>matches</STRONG>
+<DD>The number of times a block of <I>B</I> was found in <I>A</I>.
+<DT><STRONG>tag hits</STRONG>
+<DD>The number of times the 16 bit hash of the rolling
+checksum matched a hash of one of the checksums from <I>B</I>.
+<DT><STRONG>false alarms</STRONG>
+<DD>The number of times the 32 bit rolling checksum
+ matched but the strong checksum didn't.
+<DT><STRONG>data</STRONG>
+<DD>The amount of file data transferred verbatim, in bytes.
+<DT><STRONG>written</STRONG>
+<DD>The total number of bytes written by <IMG
+ WIDTH="15" HEIGHT="13" ALIGN="BOTTOM" BORDER="0"
+ SRC="img1.gif"
+ ALT="$\alpha$">
+including protocol overheads. This is almost all file data.
+<DT><STRONG>read</STRONG>
+<DD>The total number of bytes read by <IMG
+ WIDTH="15" HEIGHT="13" ALIGN="BOTTOM" BORDER="0"
+ SRC="img1.gif"
+ ALT="$\alpha$">
+including
+ protocol overheads. This is almost all checksum information.
+</DL>
+<P>
+The results demonstrate that for block sizes above 300 bytes, only a
+small fraction (around 5%) of the file was transferred. The amount
+transferred was also considerably less than the size of the diff file
+that would have been transferred if the diff/patch method of updating
+a remote file was used.
+
+<P>
+The checksums themselves took up a considerable amount of space,
+although much less than the size of the data transferred in each
+case. Each pair of checksums consumes 20 bytes: 4 bytes for the
+rolling checksum plus 16 bytes for the 128-bit MD4 checksum.
+
+<P>
+The number of false alarms was less than 1/1000 of the number of
+true matches, indicating that the 32 bit rolling checksum is quite
+good at screening out false matches.
+
+<P>
+The number of tag hits indicates that the second level of the
+checksum search algorithm was invoked about once every 50
+characters. This is quite high because the total number of blocks in
+the file is a large fraction of the size of the tag hash table. For
+smaller files we would expect the tag hit rate to be much closer to
+the number of matches. For extremely large files, we should probably
+increase the size of the hash table.
+
+<P>
+The next table shows similar results for a much smaller set of files.
+In this case the files were not packed into a tar file first. Rather,
+rsync was invoked with an option to recursively descend the directory
+tree. The files used were from two source releases of another software
+package called Samba. The total source code size is 1.7 MB and the
+diff between the two releases is 4155 lines long totalling 120 kB.
+
+<P>
+<BR>
+<BR>
+<TABLE CELLPADDING=3 BORDER="1">
+<TR><TD ALIGN="LEFT"><B> block</B></TD>
+<TD ALIGN="LEFT"><B> matches</B></TD>
+<TD ALIGN="LEFT"><B> tag</B></TD>
+<TD ALIGN="LEFT"><B> false</B></TD>
+<TD ALIGN="LEFT"><B> data</B></TD>
+<TD ALIGN="LEFT"><B> written</B></TD>
+<TD ALIGN="LEFT"><B> read</B></TD>
+</TR>
+<TR><TD ALIGN="LEFT"><B> size</B></TD>
+<TD ALIGN="LEFT"> </TD>
+<TD ALIGN="LEFT"><B> hits</B></TD>
+<TD ALIGN="LEFT"><B> alarms</B></TD>
+<TD ALIGN="LEFT"> </TD>
+<TD ALIGN="LEFT"> </TD>
+<TD ALIGN="LEFT"> </TD>
+</TR>
+<TR><TD ALIGN="LEFT"><P>
+300</TD>
+<TD ALIGN="LEFT">3727</TD>
+<TD ALIGN="LEFT">3899</TD>
+<TD ALIGN="LEFT">0</TD>
+<TD ALIGN="LEFT">129775</TD>
+<TD ALIGN="LEFT">153999</TD>
+<TD ALIGN="LEFT">83948</TD>
+</TR>
+<TR><TD ALIGN="LEFT">500</TD>
+<TD ALIGN="LEFT">2158</TD>
+<TD ALIGN="LEFT">2325</TD>
+<TD ALIGN="LEFT">0</TD>
+<TD ALIGN="LEFT">171574</TD>
+<TD ALIGN="LEFT">189330</TD>
+<TD ALIGN="LEFT">50908</TD>
+</TR>
+<TR><TD ALIGN="LEFT">700</TD>
+<TD ALIGN="LEFT">1517</TD>
+<TD ALIGN="LEFT">1649</TD>
+<TD ALIGN="LEFT">0</TD>
+<TD ALIGN="LEFT">195024</TD>
+<TD ALIGN="LEFT">210144</TD>
+<TD ALIGN="LEFT">36828</TD>
+</TR>
+<TR><TD ALIGN="LEFT">900</TD>
+<TD ALIGN="LEFT">1156</TD>
+<TD ALIGN="LEFT">1281</TD>
+<TD ALIGN="LEFT">0</TD>
+<TD ALIGN="LEFT">222847</TD>
+<TD ALIGN="LEFT">236471</TD>
+<TD ALIGN="LEFT">29048</TD>
+</TR>
+<TR><TD ALIGN="LEFT">1100</TD>
+<TD ALIGN="LEFT">921</TD>
+<TD ALIGN="LEFT">1049</TD>
+<TD ALIGN="LEFT">0</TD>
+<TD ALIGN="LEFT">250073</TD>
+<TD ALIGN="LEFT">262725</TD>
+<TD ALIGN="LEFT">23988</TD>
+</TR>
+</TABLE>
+<BR>
+<BR>
+
+<P>
+<HR>
+<!--Navigation Panel-->
+<A NAME="tex2html72"
+ HREF="node7.html">
+<IMG WIDTH="37" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="next"
+ SRC="next.gif"></A>
+<A NAME="tex2html70"
+ HREF="tech_report.html">
+<IMG WIDTH="26" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="up"
+ SRC="up.gif"></A>
+<A NAME="tex2html64"
+ HREF="node5.html">
+<IMG WIDTH="63" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="previous"
+ SRC="previous.gif"></A>
+<BR>
+<B> Next:</B> <A NAME="tex2html73"
+ HREF="node7.html">Availability</A>
+<B> Up:</B> <A NAME="tex2html71"
+ HREF="tech_report.html">The rsync algorithm</A>
+<B> Previous:</B> <A NAME="tex2html65"
+ HREF="node5.html">Pipelining</A>
+<!--End of Navigation Panel-->
+<ADDRESS>
+<I>Andrew Tridgell</I>
+<BR><I>1998-11-09</I>
+</ADDRESS>
+</BODY>
+</HTML>
--- /dev/null
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
+<!--Converted with LaTeX2HTML 98.1p1 release (March 2nd, 1998)
+originally by Nikos Drakos (nikos@cbl.leeds.ac.uk), CBLU, University of Leeds
+* revised and updated by: Marcus Hennecke, Ross Moore, Herb Swan
+* with significant contributions from:
+ Jens Lippmann, Marek Rouchal, Martin Wilck and others -->
+<HTML>
+<HEAD>
+<TITLE>Availability</TITLE>
+<META NAME="description" CONTENT="Availability">
+<META NAME="keywords" CONTENT="tech_report">
+<META NAME="resource-type" CONTENT="document">
+<META NAME="distribution" CONTENT="global">
+<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
+<LINK REL="STYLESHEET" HREF="tech_report.css">
+<LINK REL="next" HREF="node8.html">
+<LINK REL="previous" HREF="node6.html">
+<LINK REL="up" HREF="tech_report.html">
+<LINK REL="next" HREF="node8.html">
+</HEAD>
+<BODY >
+<!--Navigation Panel-->
+<A NAME="tex2html82"
+ HREF="node8.html">
+<IMG WIDTH="37" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="next"
+ SRC="next.gif"></A>
+<A NAME="tex2html80"
+ HREF="tech_report.html">
+<IMG WIDTH="26" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="up"
+ SRC="up.gif"></A>
+<A NAME="tex2html74"
+ HREF="node6.html">
+<IMG WIDTH="63" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="previous"
+ SRC="previous.gif"></A>
+<BR>
+<B> Next:</B> <A NAME="tex2html83"
+ HREF="node8.html">About this document ...</A>
+<B> Up:</B> <A NAME="tex2html81"
+ HREF="tech_report.html">The rsync algorithm</A>
+<B> Previous:</B> <A NAME="tex2html75"
+ HREF="node6.html">Results</A>
+<BR>
+<BR>
+<!--End of Navigation Panel-->
+
+<H1><A NAME="SECTION00070000000000000000">
+Availability</A>
+</H1>
+
+<P>
+An implementation of rsync which provides a convenient interface
+similar to the common UNIX command rcp has been written and is
+available for download from ftp://rsync.samba.org/pub/rsync.
+
+<P>
+<BR><HR>
+<ADDRESS>
+<I>Andrew Tridgell</I>
+<BR><I>1998-11-09</I>
+</ADDRESS>
+</BODY>
+</HTML>
--- /dev/null
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
+<!--Converted with LaTeX2HTML 98.1p1 release (March 2nd, 1998)
+originally by Nikos Drakos (nikos@cbl.leeds.ac.uk), CBLU, University of Leeds
+* revised and updated by: Marcus Hennecke, Ross Moore, Herb Swan
+* with significant contributions from:
+ Jens Lippmann, Marek Rouchal, Martin Wilck and others -->
+<HTML>
+<HEAD>
+<TITLE>About this document ... </TITLE>
+<META NAME="description" CONTENT="About this document ... ">
+<META NAME="keywords" CONTENT="tech_report">
+<META NAME="resource-type" CONTENT="document">
+<META NAME="distribution" CONTENT="global">
+<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
+<LINK REL="STYLESHEET" HREF="tech_report.css">
+<LINK REL="previous" HREF="node7.html">
+<LINK REL="up" HREF="tech_report.html">
+</HEAD>
+<BODY >
+<!--Navigation Panel-->
+<IMG WIDTH="37" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="next"
+ SRC="next.gif">
+<A NAME="tex2html88"
+ HREF="tech_report.html">
+<IMG WIDTH="26" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="up"
+ SRC="up.gif"></A>
+<A NAME="tex2html84"
+ HREF="node7.html">
+<IMG WIDTH="63" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="previous"
+ SRC="previous.gif"></A>
+<BR>
+<B> Up:</B> <A NAME="tex2html89"
+ HREF="tech_report.html">The rsync algorithm</A>
+<B> Previous:</B> <A NAME="tex2html85"
+ HREF="node7.html">Availability</A>
+<BR>
+<BR>
+<!--End of Navigation Panel-->
+
+<H1><A NAME="SECTION00080000000000000000">
+ About this document ... </A>
+</H1>
+ <STRONG>The rsync algorithm</STRONG><P>
+This document was generated using the
+<A HREF="http://www-dsed.llnl.gov/files/programs/unix/latex2html/manual/"><STRONG>LaTeX</STRONG>2<tt>HTML</tt></A> translator Version 98.1p1 release (March 2nd, 1998)
+<P>
+Copyright © 1993, 1994, 1995, 1996, 1997,
+<A HREF="http://cbl.leeds.ac.uk/nikos/personal.html">Nikos Drakos</A>,
+Computer Based Learning Unit, University of Leeds.
+<P>
+The command line arguments were: <BR>
+ <STRONG>latex2html</STRONG> <tt>tech_report.tex</tt>.
+<P>
+The translation was initiated by Andrew Tridgell on 1998-11-09
+<BR><HR>
+<ADDRESS>
+<I>Andrew Tridgell</I>
+<BR><I>1998-11-09</I>
+</ADDRESS>
+</BODY>
+</HTML>
--- /dev/null
+
+/* Century Schoolbook font is very similar to Computer Modern Math: cmmi */
+.MATH { font-family: "Century Schoolbook", serif; }
+.MATH I { font-family: "Century Schoolbook", serif; font-weight: bold }
+.BOLDMATH { font-family: "Century Schoolbook", serif; font-weight: bold }
+
+/* implement both fixed-size and relative sizes */
+SMALL.XTINY { font-size : xx-small }
+SMALL.TINY { font-size : x-small }
+SMALL.SCRIPTSIZE { font-size : smaller }
+SMALL.FOOTNOTESIZE { font-size : small }
+SMALL.SMALL { }
+BIG.LARGE { }
+BIG.XLARGE { font-size : large }
+BIG.XXLARGE { font-size : x-large }
+BIG.HUGE { font-size : larger }
+BIG.XHUGE { font-size : xx-large }
+
+/* document-specific styles come next */
--- /dev/null
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
+<!--Converted with LaTeX2HTML 98.1p1 release (March 2nd, 1998)
+originally by Nikos Drakos (nikos@cbl.leeds.ac.uk), CBLU, University of Leeds
+* revised and updated by: Marcus Hennecke, Ross Moore, Herb Swan
+* with significant contributions from:
+ Jens Lippmann, Marek Rouchal, Martin Wilck and others -->
+<HTML>
+<HEAD>
+<TITLE>The rsync algorithm</TITLE>
+<META NAME="description" CONTENT="The rsync algorithm">
+<META NAME="keywords" CONTENT="tech_report">
+<META NAME="resource-type" CONTENT="document">
+<META NAME="distribution" CONTENT="global">
+<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
+<LINK REL="STYLESHEET" HREF="tech_report.css">
+<LINK REL="next" HREF="node1.html">
+</HEAD>
+<BODY >
+<!--Navigation Panel-->
+<A NAME="tex2html4"
+ HREF="node1.html">
+<IMG WIDTH="37" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="next"
+ SRC="next.gif"></A>
+<IMG WIDTH="26" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="up"
+ SRC="up.gif">
+<IMG WIDTH="63" HEIGHT="24" ALIGN="BOTTOM" BORDER="0" ALT="previous"
+ SRC="previous.gif">
+<BR>
+<B> Next:</B> <A NAME="tex2html5"
+ HREF="node1.html">The problem</A>
+<BR>
+<BR>
+<!--End of Navigation Panel-->
+
+<P>
+
+<P>
+
+<P>
+<H1 ALIGN="CENTER">The rsync algorithm</H1>
+<P ALIGN="CENTER"><STRONG>Andrew Tridgell Paul Mackerras
+<BR>
+Department of Computer Science <BR>
+Australian National University <BR>
+Canberra, ACT 0200, Australia</STRONG></P>
+<P ALIGN="LEFT"></P>
+
+<P>
+
+<H3>Abstract:</H3>
+<DIV>
+This report presents an algorithm for updating a file on one machine
+ to be identical to a file on another machine. We assume that the
+ two machines are connected by a low-bandwidth high-latency
+ bi-directional communications link. The algorithm identifies parts
+ of the source file which are identical to some part of the
+ destination file, and only sends those parts which cannot be matched
+ in this way. Effectively, the algorithm computes a set of
+ differences without having both files on the same machine. The
+ algorithm works best when the files are similar, but will also
+ function correctly and reasonably efficiently when the files are
+ quite different.
+</DIV>
+<P>
+<P>
+<BR><HR>
+<!--Table of Child-Links-->
+<A NAME="CHILD_LINKS"> </A>
+<UL>
+<LI><A NAME="tex2html6"
+ HREF="node1.html">The problem</A>
+<LI><A NAME="tex2html7"
+ HREF="node2.html">The rsync algorithm</A>
+<LI><A NAME="tex2html8"
+ HREF="node3.html">Rolling checksum</A>
+<LI><A NAME="tex2html9"
+ HREF="node4.html">Checksum searching</A>
+<LI><A NAME="tex2html10"
+ HREF="node5.html">Pipelining</A>
+<LI><A NAME="tex2html11"
+ HREF="node6.html">Results</A>
+<LI><A NAME="tex2html12"
+ HREF="node7.html">Availability</A>
+<LI><A NAME="tex2html13"
+ HREF="node8.html">About this document ... </A>
+</UL>
+<!--End of Table of Child-Links-->
+<BR><HR>
+<ADDRESS>
+<I>Andrew Tridgell</I>
+<BR><I>1998-11-09</I>
+</ADDRESS>
+</BODY>
+</HTML>
--- /dev/null
+The vger.rutgers.edu cvs tree is mirrored onto cvs.samba.org via
+anonymous rsync using the following script.
+
+ #!/bin/bash
+
+ cd /var/www/cvs/vger/
+ PATH=/usr/local/bin:/usr/freeware/bin:/usr/bin:/bin
+
+ RUN=`lps x | grep rsync | grep -v grep | wc -l`
+ if [ "$RUN" -gt 0 ]; then
+ echo already running
+ exit 1
+ fi
+
+ rsync -az vger.rutgers.edu::cvs/CVSROOT/ChangeLog $HOME/ChangeLog
+
+ sum1=`sum $HOME/ChangeLog`
+ sum2=`sum /var/www/cvs/vger/CVSROOT/ChangeLog`
+
+ if [ "$sum1" = "$sum2" ]; then
+ echo nothing to do
+ exit 0
+ fi
+
+ rsync -az --delete --force vger.rutgers.edu::cvs/ /var/www/cvs/vger/
+ exit 0
+
+Note in particular the initial rsync of the ChangeLog to determine if
+anything has changed. This could be omitted but it would mean that the
+rsyncd on vger would have to build a complete listing of the cvs area
+at each run. As most of the time nothing will have changed I wanted to
+save the time on vger by only doing a full rsync if the ChangeLog has
+changed. This helped quite a lot because vger is low on memory and
+generally quite heavily loaded, so doing a listing on such a large
+tree every hour would have been excessive.
--- /dev/null
+Compiling rsync for MS Windows 95
+=================================
+
+Caveat:
+There are no doubt other ways to do this, but I have not tried them.
+
+This will produce an rsync binary which runs under win95/NT which can _send_
+data to a UNIX rsync host but not receive. This is suitable for my requirements
+as I am backing up a bunch of win95 machines to a linux server, but it may
+not be what _you_ require.
+
+NOTE: I am now told (by patl@curl.com) that receiving as well as
+sending works with the current cygwin32 toolkit.
+
+
+Step 1. Get the Cygnus development kit
+--------------------------------------
+
+If you do not already know about this wonderful project undertaken by cygnus,
+grab the latest copy of the gnu-win32 runtime system from
+
+ http://www.cygnus.com/misc/gnu-win32/
+
+You will want the CDK-B19.EXE self-installing binary (be warned - it's 12M
+in size) which contains gcc+binutils+DLLs required by the system. Read the
+instructions about this and install it. You will then have something which
+looks _very_ close to a UNIX box as far as any UNIX software ic concerned.
+
+Also grab the updated version (B19.1) of the DLL as this fixes some problems
+(it's pretty small at ~190k).
+
+Take a bit of time to play with this stuff - have a look at the registry
+entries that define mount points etc so you know where/how your windows
+directories are mapped.
+
+Step 2. Compile rsync
+---------------------
+
+Once the magic BASH$ prompt appears in your cygnus dos box, you can download
+and compile rsync right out of the box. You might want to do this:
+
+ tar xvzf rsync-2.0.17.tar.gz (or whatever version...)
+ cd rsync-2.0.17
+ chmod +x configure (exec is not set automatically)
+ ./configure
+ make
+
+and you will have a working RSYNC.EXE
+
+Before you can do a "make install" you must edit the Makefile so that
+it knows about "rsync.exe" rather than "rsync".
+
+Step 3 (optional) - compile and install SSH
+-------------------------------------------
+
+If you want to use ssh as the transport then you better grab a copy and compile
+it - again, no surprises. It compiles ok under gnuwin32, and you can generate
+keys etc in the normal way.
+
+Problems
+--------
+
+Well, it _almost_ works seamlessly. Here are a couple of problems I have
+encountered so far - I do not know why these symptoms occur, and they are
+probably nothing to do with rsync, but if you follow this course then you
+will probably run into these things also...
+
+1. rsync will not _receive_ data onto a windows box. It seems to do everything
+ ok, but then hangs at the last step and no files actually appear. When
+ transferring via my modem, I can see data being received, but it never
+ appears in my directory.
+
+NOTE: This now appears to be fixed.
+
+2. rsync, ssh and other gnuwin32-compiled programs can be called directly from
+ a DOS prompt, but argument passing seems to be broken. This means that
+ starting SSH directly is difficult.
+
+ After a bit of experimenting I found a workaround, which is not pretty
+ but worked for me :-)
+
+ Lets say you want to call ssh from a normal DOS prompt. Set up a batch
+ file called "SSH.BAT" which sets up the cygnus environment and calls bash
+ with the argument "ssh.sh %1 %2 %3 %4 %5 %6 %7 %8 %9". You can copy the
+ cygnus.bat file and use it as a template if you like. here is my ssh.bat:
+
+
+ @ECHO OFF
+ SET TERM=ansi
+ SET MAKE_MODE=unix
+ SET CYGFS=e:/apps/cygnus
+ SET GCC_EXEC_PREFIX=/lib\gcc-lib\
+ SET TCL_LIBRARY=%CYGROOT%\share\tcl8.0\
+ SET HOME=/home/awesley
+ SET GDBTK_LIBRARY=%CYGFS%/share/gdbtcl
+ bash /scripts/ssh.sh %1 %2 %3 %4 %5 %6 %7 %8 %9
+
+
+ Create a shell script called ssh.sh containing:
+
+ /usr/local/bin/ssh $*
+
+ which is then run by bash and calls the ssh binary (wherever you have
+ installed it).
+
+ I suspect that there is a better way, but I have not yet had time to
+ look for it.
+
+
+Cheers, Anthony Wesley
--- /dev/null
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
+<HTML>
+<HEAD>
+<TITLE>rsync y2k issues</TITLE>
+</HEAD>
+<!--#include virtual="header.html" -->
+
+<H2 align="center">Year 2000 Issues</H2>
+
+We are starting to get a lot of Y2K compliance questions. The answer
+is that rsync has no Y2K problems. rsync uses 32 bit "seconds since
+1970" date formats and thus has no problems with the turn of the
+century.<p>
+
+You must remember though that rsync is part of a larger system. It
+relies on a transport mechanism (rsh, ssh etc) and many operating
+systems services (such as inetd, syslog etc). If any of these other
+parts fail then rsync will of course be affected.<p>
+
+Also remember that rsync comes with absolutely no warranty. It is up
+to you to test whether rsync is suitable in your computing environment,
+both before and after the turn of the century.
+
+<!--#include virtual="footer.html" -->
projects / thirdparty / rsync.git / commitdiff
