The clock filter algorithm processes the offset and delay samples produced by the on-wire protocol for each peer process separately. It uses a sliding window of eight samples and picks out the sample with the least expected error. This page describes the algorithm design principles along with an example of typical performance..

Figure 1. Wedge Scattergram

Figure 1 shows a wedge scattergram plotting sample points of offset versus delay collected over a 24-hr period. As the delay increases, the offset variation increases, so the best samples are those at the lowest delay. There are two limb lines at slope ±0.5, representing the limits of sample variation. This turns out to be useful in the huff-n'-puff filter, but will not be pursued here. However, it is apparent that, if a way could be found to find the sample of least delay, it would have the least offset variation and would be the best candidate to synchronize the system clock.

In the clock filter algorithm the offset and delay samples from the on-wire protocol are inserted as the youngest stage of an eight-stage shift register, thus discarding the oldest stage. Each time an NTP packet is received from a source, a dispersion sample is initialized as the sum of the precisions of the server and client. Precision is defined by the latency to read the system clock and various from 1000 ns to 100 ns in modern machines. The dispersion ample is inserted in the shift register along with the offset and delay samples. Subsequently, the sample dispersion in each stage is increased at a fixed rate of 15 ms/s, representing the worst case error due to skew between the server and client clock frequencies's.

In each peer process the clock filter algorithm selects the stage with the smallest delay, which generally represents the most accurate data, and it and the associated offset sample become the peer variables of the same name. The peer dispersion is determined as a weighted average of the dispersion samples in the shift register. It continues to grow at the same rate as the sample dispersion. Finally, the peer jitter is determined as the root-mean-square (RMS) average of all the offset samples in the shift register relative to the selected offset sample.

Figure 1 shows a wedge scattergram plotting sample points of offset versus delay collected over a 24-hr period. As the delay increases, the offset variation increases, so the best samples are those at the lowest delay. There are two limb lines at slope ±0.5, representing the limits of sample variation. This turns out to be useful, as described on the Huff-n'-Puff Filter page. However, it is apparent that, if a way could be found to find the sample of least delay, it would have the least offset variation and would be the best candidate to synchronize the system clock.

In the clock filter algorithm the offset and delay samples from the on-wire protocol are inserted as the youngest stage of an eight-stage shift register, thus discarding the oldest stage. Each time an NTP packet is received from a source, a dispersion sample is initialized as the sum of the precisions of the server and client. Precision is defined by the latency to read the system clock and various from 1000 ns to 100 ns in modern machines. The dispersion ample is inserted in the shift register along with the offset and delay samples. Subsequently, the dispersion sample in each stage is increased at a fixed rate of 15 ms/s, representing the worst case error due to skew between the server and client clock frequencies's.

In each peer process the clock filter algorithm selects the stage with the smallest delay, which generally represents the most accurate data, and it and the associated offset sample become the peer variables of the same name. The peer jitter statistic is computed as the root-mean-square (RMS) differences between the offset samples and the offset of the selected stage.

The peer dispersion statistic is determined as a weighted sum of the dispersion samples in the shift register. Initially, the dispersion of all shift register stages is set to a large number "infinity" equal to 16 s. The weight factor for each stage, starting from the youngest numbered i = 1, is 2^-i, which means the peer dispersion is approximately 16. As samples enter the register, the peer dispersion drops from 16 to 8, 4, 2... and so forth. In practice, the dispersion falls below the select threshold of 1.5 s in about four updates. This gives some time for meaningful comparison between sources, if more than one are available. The dispersion continues to grow at the same rate as the sample dispersion. As explained elsewhere, when a source becomes unreachable, the poll process inserts a dummy infinity sample in the shift register for each poll sent. After eight polls, the register returns to its original state.

Figure 2. Raw (left) and Filtered (right) Offsets

Figure 2 shows the performance of the algorithm using offsets for a typical Internet path over a 24-hr period. The graph on the left shows the raw offsets produced by the on-wired protocol, while the figure on the right shows the filtered offsets produced by the algorithm. If we consider the series formed as the absolute value of the offset samples, the mean error is defined as the mean of this series. Thus, the mean error of the raw samples is 0.724 ms, while the mean error of the filtered series is 0.192 ms. Radio engineers would interpret this as a processing gain of 11.5 dB.

The reader may notice the somewhat boxy characteristic of the filtered offsets. This is because only new samples are selected. Once a sample is selected, the same or older samples are never selected again. The reason for this is to preserve causality; that is, time always moves forward, never stands still or moves backward. The result can be the loss of up to seven samples in the shift register, or more to the point, the output sample rate can never be less than one in eight input samples. The clock discipline algorithm is specifically designed to operate at this rate.

The reader may notice the somewhat boxy characteristic of the filtered offsets. Once a sample is selected, it remains selected until a newer sample with lower delay is available. This commonly occurs when an older selected sample is discarded from the shift register. The reason for this is to preserve causality; that is, time always moves forward, never backward. The result can be the loss of up to seven samples in the shift register, or more to the point, the output sample rate can never be less than one in eight input samples. The clock discipline algorithm is specifically designed to operate at this rate.

diff --git a/html/huffpuff.html b/html/huffpuff.html index ae1958e06..935869ca2 100644 --- a/html/huffpuff.html +++ b/html/huffpuff.html @@ -9,11 +9,16 @@

The Huff-n'-Puff Filter

Last update: - 04-Sep-2010 23:17 + 27-Sep-2010 17:57 UTC

In scenarios where a considerable amount of data are to be downloaded or uploaded over telephone modems, timekeeping quality can be seriously degraded. This occurs because the differential delays on the two directions of transmission can be quite large. In many cases the apparent time errors are so large as to exceed the step threshold and a step correction can occur during and after the data transfer.

The huff-n'-puff filter is designed to correct the apparent time offset in these cases. It depends on knowledge of the propagation delay when no other traffic is present, such as during other than work hours. The filter remembers the minimum delay over the most recent interval measured usually in hours. Under conditions of severe delay, the filter corrects the apparent offset using the sign of the offset and the difference between the apparent delay and minimum delay. The name of the filter reflects the negative (huff) and positive (puff) correction, which depends on the sign of the offset. The filter is activated by the tinker huffpuff command, as described in the Miscellaneous Options page.

In scenarios where a considerable amount of data are downloaded or uploaded using DSL or telephone modem lines, timekeeping quality can be seriously degraded. This occurs because the differential delays on the two directions of transmission can be quite large. In many cases the apparent time errors are so large as to exceed the step threshold and a step correction can occur during and after the data transfer.

The huff-n'-puff filter is designed to correct the apparent time offset in these cases. It depends on knowledge of the propagation delay when no other traffic is present, such as during other than work hours. The filter remembers the minimum delay over the most recent interval measured usually in hours. Under conditions of large delay, the filter corrects the apparent offset using the sign of the offset and the difference between the apparent delay and minimum delay. The name of the filter reflects the negative (huff) and positive (puff) correction, which depends on the sign of the offset. The filter is activated by the tinker huffpuff command, as described in the Miscellaneous Options page.

Figure 1. Huff-n'-Puff Wedge Scattergram

Figure 1 shows how the huff-n'-puff filter works. Recall from the Clock Filter Algorithm page that the wedge scattergram plots sample points of offset versus delay and that the limb lines are at slope ±0.5. Note in the figure that the samples are clustered close to the upper limb line. Given the apparent minimum delay close to 0.1 s and a point (x, y) on the upper limb line, the delay is x - 0.1 s, so the offset y is corrected by subtracting (x - 0.1) / 2. If the point was on the lower limb line, the correction would be added instead.

diff --git a/html/scripts/special.txt b/html/scripts/special.txt index f1714116c..ee7f8a8ad 100644 --- a/html/scripts/special.txt +++ b/html/scripts/special.txt @@ -6,6 +6,9 @@ document.write("

Special Topics

NTP Interleaved Modes
Huff-n'-Puff Filter
Clock Filter Algorithm
Clock Select Algorithm
Clock Cluster Algorithm
Mitigation Rules and the prefer Keyword
Clock Discipline Algorithm
Clock State Machine
Leap Second Processing

How NTP Works

Last update: - 26-Sep-2010 3:55 + 30-Sep-2010 21:39 UTC

offset = [(T₂ - T₁) + (T₃ - T₄)] / 2
delay = (T₄ - T₁) - (T₃ - T₂).

Those sources that have passed a number of sanity checks are declared selectable. From the selectable population the statistics are used by the select algorithm to determine a number of truechimers according to correctness principles. From the truechimer population a number of survivors are determined on the basis of statistical clustering principles. One of the survivors is declared the system peer and the system statistics inherited from it. The combine algorithm computes a weighted average of the survivor offsets and jitters to produce the final offset used by the clock discipline algorithm to adjust the system clock time and frequency.

When started, the program requires several measurements for these a algorithms to work reliably before setting the clock. As the default poll interval is 64 s, it can take several minutes to set the clock. The time can be reduced using the iburst option on the Server Options page. For additional details about the clock filter, select, cluster and combine algorithms see the Architecture Briefing on the NTP Project Page.

The algorithm described on the Clock Filter Algorithm page uses a window of offset and delay samples to select the best ones. Those sources that have passed a number of sanity checks are declared selectable. From the selectable population the statistics are used by the algorithm described on the Clock Select Algorithm page to determine a number of truechimers according to correctness principles. From the truechimer population the algorithm described on the Clock Cluster Algorihtm page determines a number of survivors on the basis of statistical clustering principles. The algorithms described on the Mitigation Rules and the prefer Keyword page combine the survivor offsets, designate one of them as the system peer and produces the final offset used by the algorithm described on the Clock Discipline Algorithm page to adjust the system clock time and frequency. For additional details about these algorithms, see the Architecture Briefing on the Network Time Synchronization Research Project page.

Statistics Budget

Each source is characterized by the offset and delay measured by the on-wire protocol and the dispersion and jitter calculated by the clock filter algorithm of the peer process. This algorithm selects the offset sample with the lowest delay, which generally represents the most accurate data, and it and the associated offset sample become the peer variables of the same name. The peer dispersion is determined as a weighted average of the dispersion samples in the shift register. It continues to grow at the same rate as the sample dispersion. Finally, the peer jitter is determined as the root-mean-square (RMS) average of all the offset samples in the shift register relative to the selected offset sample. Details of the clock filter algorithm are on the Clock Filter Algorithm page.

The clock filter algorithm continues to process packets in this way until the source is no longer reachable. Reachability is determined by an 8-bit shift register, which is shifted left by one bit as each poll packet is sent, with zero replacing the vacated rightmost bit. Each time an update is received, the rightmost bit is set. The source is considered reachable if any bit is set in the register; otherwise, it is considered unreachable.

A server is considered selectable only if it is reachable and a timing loop would not be created. A timing loop occurs when the server is apparently synchronized to the client or when the server is synchronized to the same server as the client. When a source is unreachable, a dummy sample with "infinite" dispersion is inserted in the shift register, thus displacing old samples.

The composition of the survivor population and the system peer selection is redetermined as each update from each source is received. The system variables are copied from the peer variables of the same name and the system stratum set one greater than the system peer stratum. Like peer dispersion, the system dispersion increases at the same rate so, even if all sources have become unreachable, the daemon appears to upstratum clients at ever increasing dispersion.

Each source is characterized by the offset and delay measured by the on-wire protocol and the dispersion and jitter calculated by the clock filter algorithm. This algorithm selects the offset sample with the lowest delay, which generally represents the most accurate data, so it and the associated offset sample become the peer variables of the same name. The peer dispersion is determined as a weighted average of the dispersion samples in the shift register. It continues to grow at the same rate as the sample dispersion. Finally, the peer jitter is determined as the root-mean-square (RMS) average of the offset samples in the shift register relative to the selected offset sample.

The clock filter algorithm continues to process packets in this way until the source is no longer reachable. Reachability is determined by an eight-bit shift register, which is shifted left by one bit as each poll packet is sent, with 0 replacing the vacated rightmost bit. Each time an update is received, the rightmost bit is set to 1. The source is considered reachable if any bit is set to 1 in the register; otherwise, it is considered unreachable.

The composition of the survivor population and the system peer selection is redetermined as each update from each source is received. The system variables are copied from the system peer variables of the same name and the system stratum set one greater than the system peer stratum. Like peer dispersion, the system dispersion increases at the same rate so, even if all sources have become unreachable, the daemon appears to upstratum clients at ever increasing dispersion.

Quality of Service

Of interest in this discussion is how the protocol determines the quality of service from a particular reference clock or remote server. It is determined from two statistics, expected error and maximum error. Expected error is determined from various jitter components; it represents the nominal error in determining the mean clock offset. However, it is not relevant to the discussion to follow. Maximum error is determined from delay and dispersion contributions and represents the worst-case error due to all causes. In order to simplicity this presentation, certain minor contribution s to the maximum error statistic are ignored. Elsewhere in the documentation the maximum error is called synchronization distance.

Of interest in this discussion is how the protocol determines the quality of service from a particular reference clock or remote server. It is determined from two statistics, expected error and maximum error. Expected error is determined from various jitter components; it represents the nominal error in determining the mean clock offset. However, it is not relevant to the discussion to follow. Maximum error is determined from delay and dispersion contributions and represents the worst-case error due to all causes. In order to simplify this presentation, certain minor contribution s to the maximum error statistic are ignored. Elsewhere in this documentation the maximum error is called synchronization distance.

The maximum error is computed as one-half the root delay to the primary source of time; i.e., the primary reference clock, plus the root dispersion. The root variables are included in the NTP packet header received from each server. When calculating maximum error, the root delay is the sum of the root delay in the packet and the peer delay, while the root dispersion is the sum of the root dispersion in the packet and the peer dispersion.

A source is considered selectable only if its maximum error is less than the select threshold, by default 1.5 s, but can be changed due to client preference. A common consequences is when an upstream server loses all sources and its maximum error apparent to clients begins to increase. The clients are not aware of this condition and continues to accept synchronization as long as the maximum error is less than the select threshold.

Although it might seem counter-intuitive, a cardinal rule in the selection process is, once a sample has been selected by the clock filter algorithm, that sample and any older samples are no longer selectable. This applies also to the select algorithm. Once the peer variables for a source have been selected, older variables of the same or other sources are no longer selectable. This means that not every sample can be used to update the peer variables and up to seven samples can be ignored between selected samples. This fact has been carefully considered in the discipline algorithm design with due consideration of the feedback loop delay and minimum sampling rate. In engineering terms, even if only one sample in eight survives, the resulting sample rate is twice the Nyquist rate at any time constant and poll interval.

A source is considered selectable only if its maximum error is less than the select threshold, by default 1.5 s, but can be changed according to client preference. A common consequences is when an upstream server loses all sources and its maximum error apparent to clients begins to increase. The clients are not aware of this condition and continues to accept synchronization as long as the maximum error is less than the select threshold.

Although it might seem counter-intuitive, a cardinal rule in the selection process is, once a sample has been selected by the clock filter algorithm, older samples are no longer selectable. This applies also to the select algorithm. Once the peer variables for a source have been selected, older variables of the same or other sources are no longer selectable. This means that not every sample can be used to update the peer variables and up to seven samples can be ignored between selected samples. This fact has been carefully considered in the discipline algorithm design with due consideration of the feedback loop delay and minimum sampling rate. In engineering terms, even if only one sample in eight survives, the resulting sample rate is twice the Nyquist rate at any time constant and poll interval.

Mitigation Algorithms

Some daemon configurations include a combination of reference clocks and remote servers in order to provide redundancy and backup. For example, a modem reference clock may furnish backup for a GPS reference clock, but used only if the GPS clock fails. In addition, the local clock might be used if all sources fail, or orphan mode might be used instead. The mitigation algorithms provide an orderly selection in such cases. Another function of these algorithms is when multiple sources of the same type are available, but for one reason or another, one or more of them are preferred over the others. Finally, some reference clocks provide a pulse-per-second (PPS) signal to augment the serial timecode. The mitigation algorithms have to figure out when the PPS signal is valid and which reference clock is to number the seconds. These intricate algorithms are described on the Mitigation Algorithms and the prefer Keyword page.

Clock Filter Algorithm

The Huff-n'-Puff Filter

How NTP Works

Table of Contents

Statistics Budget

Quality of Service

Mitigation Algorithms

Clock Discipline Algorithm