Clock Cluster Algorithm

From: Harlan Stenn Date: Mon, 4 Oct 2010 05:14:16 +0000 (-0400) Subject: Documentation updates from Dave Mills X-Git-Tag: NTP_4_2_7P60~3 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=d7cac4770e996ef2f90a88ed6bb8bc7f10ed9cd3;p=thirdparty%2Fntp.git Documentation updates from Dave Mills bk: 4ca962a8TRPZx9oNGBmup3OMw0Xy8w --- diff --git a/ChangeLog b/ChangeLog index c84ce8c42..c94857292 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,4 @@ +* Documentation updates from Dave Mills. (4.2.7p59) 2010/10/02 Released by Harlan Stenn * Documentation updates from Dave Mills. * Variable name cleanup from Dave Mills. diff --git a/html/authentic.html b/html/authentic.html index 6be2194f9..6264eb1e0 100644 --- a/html/authentic.html +++ b/html/authentic.html @@ -19,7 +19,7 @@ color: #FF0000; gif

from Alice's Adventures in Wonderland, Lewis Carroll

Our resident cryptographer; now you see him, now you don't.

Last update: - 11-Sep-2010 19:15 + 02-Oct-2010 23:55 UTC

Commands and Options

Unless noted otherwise, further information about these commands is on the Authentication Support page.

diff --git a/html/cluster.html b/html/cluster.html new file mode 100644 index 000000000..d96f44a67 --- /dev/null +++ b/html/cluster.html @@ -0,0 +1,31 @@ + + + + + +Clock Cluster Algorithm + + + + +

Clock Cluster Algorithm

Last update: + 04-Oct-2010 2:42 + UTC

The clock cluster algorithm processes the truechimers produced by the clock select algorithm to produce the survivors used by the mitigation algorithms to discipline the system clock. It operates in a series of rounds, where at each round the truechimer furthest from the offset centroid is pruned from the population. The rounds are continued until a specified termination condition results. This page discusses the algorithm in detail.

First, the truechimer candidates are saved on a list of n entries sorted by root distance. For the ith entry on the list, a statistic called the select jitter is calculated as follows. Let

d_i(j) = q(j) - q(i),

where q(i) is the peer offset of the ith entry and q(j) is the peer offset of the jth entry, both produced by the clock filter algorithm. Then, the select jitter j_S(i) of the ith entry is the root distance of the ith entry times the RMS sum of d_i(j) as j ranges from 1 to n. For the purpose of notation in the example to follow, let j_R(i) be the peer jitter computed by the clock filter algorithm for the ith entry. In general, the expected error statistic for the ith entry is the RMS sum of these two components, but that is not needed by the clock cluster algorithm.

The object at each round is to prune the entry with the largest select jitter until the termination condition is met. Note that the select jitter must be recomputed at each round, but the peer jitter does not change. At each round the remaining entries on the list represent the survivors of that round. The list is always pruned to the maxclock threshold with default 10, but can be set by the maxclock option of the tos command. This threshold is useful to limit the number of survivors when automatic server discovery schemes are in use.

The termination condition has two parts. First, if the number of candidates is not greater than the sane threshold set by the minsane option of the tos command, or not greater than the minclock threshold set by the minclock option of the tos command, the pruning process terminates. The minsane default is 1 and the minclock default is 3. These thresholds can be changed to fit special conditions, as described on the Mitigation Rules and the prefer Keyword page.

Figure 1. Cluster Algorithm

The second termination condition is more intricate. Figure 1 shows a round where a candidates of (a) is pruned to yield the candidates of (b). Let j_max be the maximum select jitter and j_min be the minimum peer jitter over all entries on the list. In (a), candidate 1 has the highest select jitter, so j_max = j_S(1). Candidate 4 has the lowest peer jitter, so j_min = j_R(4). Since j_max > j_min, the algorithm prunes candidate 1 and continues. In (b), j_max = j_S(3) and j_min= j_R(4). Since j_max ≤ j_min, pruning additional candidates will not reduce select jitter. So, the algorithm terminates with candidates 2, 3 and 4 as survivors.

+ + + diff --git a/html/comdex.html b/html/comdex.html index e5da439bd..bab74a519 100644 --- a/html/comdex.html +++ b/html/comdex.html @@ -11,7 +11,7 @@ gif

from Alice's Adventures in Wonderland, Lewis Carrol

The Mad Hatter says "Bring it on".

Last update: - 07-Sep-2010 2:11 + 02-Oct-2010 23:55 UTC

More Help

+
+

More Help

Initial Startup

diff --git a/html/ntp_conf.html b/html/ntp_conf.html index 7caf89437..9dc5718be 100644 --- a/html/ntp_conf.html +++ b/html/ntp_conf.html @@ -11,10 +11,9 @@ gif

from Pogo, Walt Kelly

Racoon is shooting configuration bugs.

Last update: - 12-Sep-2010 3:45 + 03-Oct-2010 1:48 UTC

Synopsis
Detailed Description
Guidelines for Adding Configuration Commands

Synopsis

The NTP configuration process is driven by a phrase-structure grammar which is used to specify the format of the configuration commands and the actions needed to build an abstract syntax tree (AST). The grammar is fed to a parser generator (Bison) which produces a parser for the configuration file.

The generated parser is used to parse an NTP configuration file and check it for syntax and semantic errors. The result of the parse is an AST, which contains a representation of the various commands and options. This AST is then traversed to set up the NTP daemon to the correct configuration.

diff --git a/html/ntpdsim_new.html b/html/ntpdsim_new.html index 66ba03756..fe907c574 100644 --- a/html/ntpdsim_new.html +++ b/html/ntpdsim_new.html @@ -11,7 +11,7 @@ gif

from The Wizard of Oz, L. Frank Baum

All in a row.

Last update: - 04-Sep-2010 14:31 + 03-Oct-2010 1:48 UTC

Description

The ntpdsim program is used to simulate and study the behavior of an NTP daemon that derives its time from a number of different simulated time sources (servers). Each simulated server can be configured to have a different time offset, frequency offset, propagation delay, processing delay, network jitter and oscillator wander.

The ntpdsim program runs all the same selection, mitigation, and discipline diff --git a/html/select.html b/html/select.html new file mode 100644 index 000000000..196b47a3f --- /dev/null +++ b/html/select.html @@ -0,0 +1,32 @@ + + + + + +Clock Select Algorithm + + + +

Clock Select Algorithm

Last update: + 30-Sep-2010 19:59 + UTC

The clock select algorithm determines from a set of candidates, which are correct (truechimers) and which are not (falsetickers) according to a set of formal correctness assertions. The principles are based on the observation that the maximum error in determining the offset of a candidate cannot exceed one-half the roundtrip delay to the primary reference clock at the time of measurement. This must be increased by the maximum error that can accumulate since then. In NTP the total, called the synchronization distance, is one-half the roundtrip root delay plus the root dispersion plus minor error contributions not considered here.

Given the measured offset q₀ and synchronization distance l, this defines a correctness interval [q₀ - l, q₀ + l] of points where the true value of q lies somewhere on the interval. The given problem is to determine from a set of correctness intervals, which represent truechimers and which represent falsetickers. The principles must be given a precise definition. The intersection interval is the smallest interval containing points from the largest number of correctness intervals. An algorithm that finds the intersection interval was devised by Keith Marzullo in his doctoral dissertation. It was first implemented in the DTSS (Digital Time Synchronization Service) in the VMS operating system for the VAX.

While the NTP algorithm is based on DTSS, it remains to establish which point represents the best estimate of the offset for each candidate. The best point is at the midpoint q₀ of the correctness interval; however, the midpoint might not be within the intersection interval. A candidate with a correctness interval that contains points in the intersection interval is a truechimer and the best offset estimate is the midpoint of its correctness interval. A candidate with a correctness interval that contains no points in the intersection interval is a falseticker.

Figure 1. Intersection Interval

Figure 1 shows correctness intervals for each of four candidates A, B, C and D. We need to find the maximum number of candidates that contain points in common. The result is the interval labeled DTSS. In the figure there are three truechimers A, B and C, and one falseticker D. In theory, any point in the intersection interval can represent the true time. The clock is considered correct if the number of truechimers found in this way are greater than half the total number of candidates.

The question remains, which is the best point to represent the true time of each interval? Fortunately, we already have the maximum likelihood estimate at the midpoint of each correctness interval. But, while the midpoint of candidate C is outside the intersection interval, its correctness interval contains points in common with the intersection interval, so the candidate is a truechimer.

Figure 2. Clock Select Algorithm

The algorithm operates as shown in Figure 2.. Let m be the number of candidates and f the number of falsetickers, initially zero. Move a pointer from the leftmost endpoint towards the rightmost endpoint in Figure 1 and count the number of candidates, stopping when that number reaches m - f; this is the left endpoint of the intersection interval. Then, do the same, but moving from the rightmost endpoint towards the leftmost endpoint; this is the right endpoint of the intersection interval. If the left endpoint is greater than the right endpoint; i.e., the interval appears backwards. increase f by 1. If f is less than n / 2, try again; otherwise, the algorithm fails and no truechimers could be found..

The clock select algorithm then scans the associations. If the right endpoint of the correctness interval for a candidate is greater than the left endpoint of the intersection interval, or if the left endpoint of the correctness interval is less than the right endpoint of the intersection interval, the candidate is a truechimer; otherwise, it is a falseticker.

In practice, with fast LANs and modern computers, the correctness interval can be quite small, especially when the candidates are multiple reference clocks. In such cases the intersection interval may be empty, due to insignificant differences in the reference clock offsets. To avoid this, the synchronization distance must be at least the value of mindist, with default 1 ms. This value can be changed using the mindist option of the tos command.

+ + + diff --git a/html/warp.html b/html/warp.html index e61939daa..1f3274eba 100644 --- a/html/warp.html +++ b/html/warp.html @@ -9,48 +9,42 @@

How NTP Works

Last update: - 30-Sep-2010 21:39 + 03-Oct-2010 1:48 UTC

Introduction and Overview

NTP time synchronization services are widely available in the public Internet. The public NTP subnet in late 2010 includes several thousand servers in most countries and on every continent of the globe, including Antarctica, and sometimes in space and on the sea floor. These servers support a total population estimated at over 25 million computers in the global Internet.

The NTP subnet operates with a hierarchy of levels, where each level is assigned a number called the stratum. Stratum 1 (primary) servers at the lowest level are directly synchronized to national time services via satellite, radio and telephone mdem. Stratum 2 (secondary) servers at the next higher level are synchronize to stratum 1 servers and so on. Normally, NTP clients and servers with a relatively small number of clients do not synchronize to public primary servers. There are several hundred public secondary servers operating at higher strata and are the preferred choice.

The NTP subnet operates with a hierarchy of levels, where each level is assigned a number called the stratum. Stratum 1 (primary) servers at the lowest level are directly synchronized to national time services via satellite, radio and telephone modem. Stratum 2 (secondary) servers at the next higher level are synchronize to stratum 1 servers and so on. Normally, NTP clients and servers with a relatively small number of clients do not synchronize to public primary servers. There are several hundred public secondary servers operating at higher strata and are the preferred choice.

This page presents an overview of the NTP daemon included in this distribution. We refer to this as the reference implementation only because it was used to test and validate the NTPv4 specification RFC-5905. It is best read in conjunction with the briefings on the Network Time Synchronization Research Project page.

gif

Figure 1. NTP Daemon Processes and Algorithms

The overall organization of the NTP daemon is shown in Figure 1. It is useful in this context to consider the daemon as both a client of upstream servers and as a server for downstream clients. It includes a pair of peer/poll processes for each reference clock or remote server used as a synchronization source. The poll process sends NTP packets at intervals ranging from 8 s to 36 hr. The peer process receives NTP packets and runs the on-wire protocol that collects four timestamps: the origin timestamp T₁ upon departure of the client request, the receive timestamp T₂ upon arrival at the server, the transmit timestamp T₃ upon departure of the server reply, and the destination timestamp T₄ upon arrival at the client. These timestamps are used to calculate the clock offset and roundtrip delay:

The overall organization of the NTP daemon is shown in Figure 1. It is useful in this context to consider the daemon as both a client of upstream servers and as a server for downstream clients. It includes a pair of peer/poll processes for each reference clock or remote server used as a synchronization source. The poll process sends NTP packets at intervals ranging from 8 s to 36 hr. The peer process receives NTP packets and runs the on-wire protocol that collects four raw timestamps: the origin timestamp T₁ upon departure of the client request, the receive timestamp T₂ upon arrival at the server, the transmit timestamp T₃ upon departure of the server reply, and the destination timestamp T₄ upon arrival at the client. These timestamps, which are recorded as the rawstats option of the filegen command, are used to calculate the clock offset and roundtrip delay:

offset = [(T₂ - T₁) + (T₃ - T₄)] / 2
delay = (T₄ - T₁) - (T₃ - T₂).

The algorithm described on the Clock Filter Algorithm page uses a window of offset and delay samples to select the best ones. Those sources that have passed a number of sanity checks are declared selectable. From the selectable population the statistics are used by the algorithm described on the Clock Select Algorithm page to determine a number of truechimers according to correctness principles. From the truechimer population the algorithm described on the Clock Cluster Algorihtm page determines a number of survivors on the basis of statistical clustering principles. The algorithms described on the Mitigation Rules and the prefer Keyword page combine the survivor offsets, designate one of them as the system peer and produces the final offset used by the algorithm described on the Clock Discipline Algorithm page to adjust the system clock time and frequency. For additional details about these algorithms, see the Architecture Briefing on the Network Time Synchronization Research Project page.

The algorithm described on the Clock Filter Algorithm page uses a window of offset and delay samples to select the best ones. Those sources that have passed a number of sanity checks are declared selectable. From the selectable population the statistics are used by the algorithm described on the Clock Select Algorithm page to determine a number of truechimers according to correctness principles. From the truechimer population the algorithm described on the Clock Cluster Algorithm page determines a number of survivors on the basis of statistical clustering principles. The algorithms described on the Mitigation Rules and the prefer Keyword page combine the survivor offsets, designate one of them as the system peer and produces the final offset used by the algorithm described on the Clock Discipline Algorithm page to adjust the system clock time and frequency. For additional details about these algorithms, see the Architecture Briefing on the Network Time Synchronization Research Project page.

Statistics Budget

Each source is characterized by the offset and delay measured by the on-wire protocol and the dispersion and jitter calculated by the clock filter algorithm. This algorithm selects the offset sample with the lowest delay, which generally represents the most accurate data, so it and the associated offset sample become the peer variables of the same name. The peer dispersion is determined as a weighted average of the dispersion samples in the shift register. It continues to grow at the same rate as the sample dispersion. Finally, the peer jitter is determined as the root-mean-square (RMS) average of the offset samples in the shift register relative to the selected offset sample.

Each source is characterized by the offset and delay measured by the on-wire protocol and the dispersion and jitter calculated by the clock filter algorithm. This algorithm selects the offset sample with the lowest delay, which generally represents the most accurate data, so it and the associated offset sample become the peer offset and peer delay. The peer dispersion is determined as a weighted average of the dispersion samples in the shift register. It continues to grow at the same rate as the sample dispersion. Finally, the peer jitter is determined as the root-mean-square (RMS) average of the offset samples in the shift register relative to the selected offset sample. The peer offset, peer delay, peer dispersion and peer jitter are recorded as the peerstats option of the filegen command.

The clock filter algorithm continues to process packets in this way until the source is no longer reachable. Reachability is determined by an eight-bit shift register, which is shifted left by one bit as each poll packet is sent, with 0 replacing the vacated rightmost bit. Each time an update is received, the rightmost bit is set to 1. The source is considered reachable if any bit is set to 1 in the register; otherwise, it is considered unreachable.

A server is considered selectable only if it is reachable and a timing loop would not be created. A timing loop occurs when the server is apparently synchronized to the client or when the server is synchronized to the same server as the client. When a source is unreachable, a dummy sample with "infinite" dispersion is inserted in the shift register at each poll, thus displacing old samples.

The composition of the survivor population and the system peer selection is redetermined as each update from each source is received. The system variables are copied from the system peer variables of the same name and the system stratum set one greater than the system peer stratum. Like peer dispersion, the system dispersion increases at the same rate so, even if all sources have become unreachable, the daemon appears to upstratum clients at ever increasing dispersion.

Quality of Service

Of interest in this discussion is how the protocol determines the quality of service from a particular reference clock or remote server. It is determined from two statistics, expected error and maximum error. Expected error is determined from various jitter components; it represents the nominal error in determining the mean clock offset. However, it is not relevant to the discussion to follow. Maximum error is determined from delay and dispersion contributions and represents the worst-case error due to all causes. In order to simplify this presentation, certain minor contribution s to the maximum error statistic are ignored. Elsewhere in this documentation the maximum error is called synchronization distance.

The maximum error is computed as one-half the root delay to the primary source of time; i.e., the primary reference clock, plus the root dispersion. The root variables are included in the NTP packet header received from each server. When calculating maximum error, the root delay is the sum of the root delay in the packet and the peer delay, while the root dispersion is the sum of the root dispersion in the packet and the peer dispersion.

A source is considered selectable only if its maximum error is less than the select threshold, by default 1.5 s, but can be changed according to client preference. A common consequences is when an upstream server loses all sources and its maximum error apparent to clients begins to increase. The clients are not aware of this condition and continues to accept synchronization as long as the maximum error is less than the select threshold.

Of interest in this discussion is how the protocol determines the quality of service from a particular reference clock or remote server. It is determined from two statistics, expected error and maximum error. Expected error is determined from various jitter components; it represents the nominal error in determining the mean clock offset. The mitigation algorithms deliver two important statistics, system offset and system jitter. These statistics are determined by the mitigation algorithms's from the survivor statistics produced by the clock cluster algorithm. System offset is best interpreted as the maximum likelihood estimate of the system clock offset, while system jitter is best interpreted as the expected error of this estimate. These statistics are reported as the loopstats option of the filegen command.

Maximum error is determined from delay and dispersion contributions and represents the worst-case error due to all causes. In order to simplify this presentation, certain minor contribution s to the maximum error statistic are ignored. Elsewhere in this documentation the maximum error is called synchronization distance. If the precision time kernel support is available, both the estimated error and maximum error are reported to user programs via the ntp_gettimeofday() kernel system call. See the Kernel Model for Precision Timekeeping page for further information.

A source is considered selectable only if its maximum error is less than the select threshold, by default 1.5 s, but can be changed according to client preference using the maxdist option of the tos command. A common consequences is when an upstream server loses all sources and its maximum error apparent to dependent clients begins to increase. The clients are not aware of this condition and continues to accept synchronization as long as the maximum error is less than the select threshold.

Although it might seem counter-intuitive, a cardinal rule in the selection process is, once a sample has been selected by the clock filter algorithm, older samples are no longer selectable. This applies also to the select algorithm. Once the peer variables for a source have been selected, older variables of the same or other sources are no longer selectable. This means that not every sample can be used to update the peer variables and up to seven samples can be ignored between selected samples. This fact has been carefully considered in the discipline algorithm design with due consideration of the feedback loop delay and minimum sampling rate. In engineering terms, even if only one sample in eight survives, the resulting sample rate is twice the Nyquist rate at any time constant and poll interval.

Mitigation Algorithms

Some daemon configurations include a combination of reference clocks and remote servers in order to provide redundancy and backup. For example, a modem reference clock may furnish backup for a GPS reference clock, but used only if the GPS clock fails. In addition, the local clock might be used if all sources fail, or orphan mode might be used instead. The mitigation algorithms provide an orderly selection in such cases. Another function of these algorithms is when multiple sources of the same type are available, but for one reason or another, one or more of them are preferred over the others. Finally, some reference clocks provide a pulse-per-second (PPS) signal to augment the serial timecode. The mitigation algorithms have to figure out when the PPS signal is valid and which reference clock is to number the seconds. These intricate algorithms are described on the Mitigation Algorithms and the prefer Keyword page.

Clock Discipline Algorithm

At the heart of the NTP specification and reference implementation is the clock discipline algorithm, which is best described as an adaptive parameter, hybrid phase/frequency-lock feedback loop. It is an intricately crafted algorithm that automatically adapts for optimum performance while minimizing network overhead. Further details are on the Clock Discipline page.

Clock State Machine

In the NTPv4 specification and reference implementation a state machine is used to manage the system clock under exceptional conditions, as when the daemon is first started or when encountering severe network congestion. When the frequency file is present at startup is that the residual offset error is less than 0.5 ms within 300 s. When the frequency file is not present, this result is achieved within 600 s. Further details are on the Clock State Machine page.

Related Links

Related Links

Commands and Options

Clock Cluster Algorithm

Related Links

More Help

More Help

Initial Startup

Table of Contents

Synopsis

Related Links

Description

Clock Select Algorithm

How NTP Works

Related Links

Table of Contents

Introduction and Overview

Statistics Budget

Quality of Service

Mitigation Algorithms

Clock Discipline Algorithm

Clock State Machine