<transport>_destination_concurrency_positive_feedback,
<transport>_destination_concurrency_failed_cohort_limit.
- Files: global/mail_params.h, qmgr/qmgr.c, qmgr/qmgr_transport.c,
- qmgr/qmge_queue.c, qmgr/qmgr_feedback.c, postconf/auto.awk.
+ Files: global/mail_params.h, *qmgr/qmgr.c, *qmgr/qmgr_transport.c,
+ *qmgr/qmgr_queue.c, *qmgr/qmgr_feedback.c, postconf/auto.awk.
+
+20071202
+
+ Feature: output rate control. For example, specify
+ "smtp_delivery_rate_delay = 5m" to insert a five-minute
+ delay between deliveries. This was an opportunity to define
+ the mutually exclusive states that a queue can have, and
+ to detect invalid transitions. This will make adding new
+ features code easier. Files: *qmgr/qmgr_transport.c,
+ *qmgr/qmgr_queue.c, *qmgr/qmgr_entry.c.
+
+ Bugfix: don't update the back-to-back delivery time stamp
+ while deferring mail. File: *qmgr/qmgr_entry.c.
L\bLi\bim\bmi\bit\bta\bat\bti\bio\bon\bns\bs o\bof\bf l\ble\bes\bss\bs-\b-t\bth\bha\ban\bn-\b-1\b1 p\bpe\ber\br d\bde\bel\bli\biv\bve\ber\bry\by f\bfe\bee\bed\bdb\bba\bac\bck\bk
-The delivery concurrency scheduler with less-than-1 concurrency feedback per
-delivery solves a problem with servers that have active concurrency limiters.
-This works only because feedback is handled in a peculiar manner: positive
-feedback will increment the concurrency by 1 at the e\ben\bnd\bd of a sequence of events
-of length 1/feedback, while negative feedback will decrement concurrency by 1
-at the b\bbe\beg\bgi\bin\bnn\bni\bin\bng\bg of such a sequence. This is how Postfix adjusts quickly for
-overshoot without causing lots of mail to be deferred. Without this difference
-in feedback treatment, less-than-1 feedback per delivery would defer 50% of the
+The scheduler with less-than-1 concurrency feedback per delivery solves a
+problem with servers that have active concurrency limiters. This works only
+because feedback is handled in a peculiar manner: positive feedback will
+increment the concurrency by 1 at the e\ben\bnd\bd of a sequence of events of length 1/
+feedback, while negative feedback will decrement concurrency by 1 at the
+b\bbe\beg\bgi\bin\bnn\bni\bin\bng\bg of such a sequence. This is how Postfix adjusts quickly for overshoot
+without causing lots of mail to be deferred. Without this difference in
+feedback treatment, less-than-1 feedback per delivery would defer 50% of the
mail, and would be no better in this respect than the old +/-1 feedback per
delivery.
behind the load balancer are perfectly capable of handling more traffic.
This noise problem gets worse as the amount of positive feedback per delivery
-gets smaller. A compromise is to avoid concurrency-dependent positive feedback,
-and to use fixed less-than-1 feedback values instead. For example, to tolerate
-1 of 4 bad servers in the above load balancer scenario, use positive feedback
-of 1/4 per "good" delivery (no connect or handshake error), and use an equal or
+gets smaller. A compromise is to use fixed less-than-1 positive feedback values
+instead of concurrency-dependent positive feedback. For example, to tolerate 1
+of 4 bad servers in the above load balancer scenario, use positive feedback of
+1/4 per "good" delivery (no connect or handshake error), and use an equal or
smaller amount of negative feedback per "bad" delivery. The downside of using
concurrency-independent feedback is that some of the old +/-1 feedback problems
will return at large concurrencies. Sites that deliver at non-trivial per-
If you upgrade from Postfix 2.3 or earlier, read RELEASE_NOTES-2.4
before proceeding.
+Major changes with Postfix snapshot 20071202
+============================================
+
+Output rate control in the queue manager. For example, specify
+"smtp_delivery_rate_delay = 5m", to pause five minutes between
+message deliveries. More information in the postconf(5) manual
+under "default_delivery_rate_delay".
+
Major changes with Postfix snapshot 20071130
============================================
<h3> <a name="concurrency_limitations"> Limitations of less-than-1 per delivery feedback </a> </h3>
-<p> The delivery concurrency scheduler with less-than-1 concurrency
+<p> The scheduler with less-than-1 concurrency
feedback per delivery solves a problem with servers that have active
concurrency limiters. This works only because feedback is handled
in a peculiar manner: positive feedback will increment the concurrency
balancer are perfectly capable of handling more traffic. </p>
<p> This noise problem gets worse as the amount of positive feedback
-per delivery gets smaller. A compromise is to avoid concurrency-dependent
-positive feedback, and to use fixed less-than-1 feedback values
-instead. For example, to tolerate 1 of 4 bad servers in the above
+per delivery gets smaller. A compromise is to use fixed less-than-1
+positive feedback values instead of concurrency-dependent positive
+feedback. For example, to tolerate 1 of 4 bad servers in the above
load balancer scenario, use positive feedback of 1/4 per "good"
delivery (no connect or handshake error), and use an equal or smaller
amount of negative feedback per "bad" delivery. The downside of
Idem, for delivery via the named message <i>transport</i>.
<b><a href="postconf.5.html#default_destination_concurrency_negative_feedback">default_destination_concurrency_negative_feedback</a> (1)</b>
- The per-destination amount of negative delivery
- concurrency feedback, after a delivery completes
- with a connection or handshake failure.
+ The per-destination amount of delivery concurrency
+ negative feedback, after a delivery completes with
+ a connection or handshake failure.
<b><a href="postconf.5.html#transport_destination_concurrency_positive_feedback"><i>transport</i>_destination_concurrency_negative_feedback</a></b>
<b>($<a href="postconf.5.html#default_destination_concurrency_negative_feedback">default_destination_concurrency_negative_feedback</a>)</b>
Idem, for delivery via the named message <i>transport</i>.
<b><a href="postconf.5.html#default_destination_concurrency_positive_feedback">default_destination_concurrency_positive_feedback</a> (1)</b>
- The per-destination amount of positive delivery
- concurrency feedback, after a delivery completes
- without connection or handshake failure.
+ The per-destination amount of delivery concurrency
+ positive feedback, after a delivery completes with-
+ out connection or handshake failure.
<b><a href="postconf.5.html#transport_destination_concurrency_positive_feedback"><i>transport</i>_destination_concurrency_positive_feedback</a></b>
<b>($<a href="postconf.5.html#default_destination_concurrency_positive_feedback">default_destination_concurrency_positive_feedback</a>)</b>
The maximal time a bounce message is queued before
it is considered undeliverable.
+ Available in Postfix version 2.5 and later:
+
+ <b><a href="postconf.5.html#default_delivery_rate_delay">default_delivery_rate_delay</a> (0s)</b>
+ The default amount of delay that is inserted
+ between individual deliveries to the same destina-
+ tion; with per-destination recipient limit > 1, a
+ destination is a domain, otherwise it is a recipi-
+ ent.
+
+ <b><a href="postconf.5.html#transport_delivery_rate_delay"><i>transport</i>_delivery_rate_delay</a> $<a href="postconf.5.html#default_delivery_rate_delay">default_delivery_rate_delay</a></b>
+ Idem, for delivery via the named message <i>transport</i>.
+
<b>MISCELLANEOUS CONTROLS</b>
<b><a href="postconf.5.html#config_directory">config_directory</a> (see 'postconf -d' output)</b>
- The default location of the Postfix <a href="postconf.5.html">main.cf</a> and
+ The default location of the Postfix <a href="postconf.5.html">main.cf</a> and
<a href="master.5.html">master.cf</a> configuration files.
<b><a href="postconf.5.html#daemon_timeout">daemon_timeout</a> (18000s)</b>
- How much time a Postfix daemon process may take to
- handle a request before it is terminated by a
+ How much time a Postfix daemon process may take to
+ handle a request before it is terminated by a
built-in watchdog timer.
<b><a href="postconf.5.html#defer_transports">defer_transports</a> (empty)</b>
"<b>sendmail -q</b>" or equivalent.
<b><a href="postconf.5.html#delay_logging_resolution_limit">delay_logging_resolution_limit</a> (2)</b>
- The maximal number of digits after the decimal
+ The maximal number of digits after the decimal
point when logging sub-second delay values.
<b><a href="postconf.5.html#helpful_warnings">helpful_warnings</a> (yes)</b>
- Log warnings about problematic configuration set-
+ Log warnings about problematic configuration set-
tings, and provide helpful suggestions.
<b><a href="postconf.5.html#ipc_timeout">ipc_timeout</a> (3600s)</b>
over an internal communication channel.
<b><a href="postconf.5.html#process_id">process_id</a> (read-only)</b>
- The process ID of a Postfix command or daemon
+ The process ID of a Postfix command or daemon
process.
<b><a href="postconf.5.html#process_name">process_name</a> (read-only)</b>
- The process name of a Postfix command or daemon
+ The process name of a Postfix command or daemon
process.
<b><a href="postconf.5.html#queue_directory">queue_directory</a> (see 'postconf -d' output)</b>
- The location of the Postfix top-level queue direc-
+ The location of the Postfix top-level queue direc-
tory.
<b><a href="postconf.5.html#syslog_facility">syslog_facility</a> (mail)</b>
The syslog facility of Postfix logging.
<b><a href="postconf.5.html#syslog_name">syslog_name</a> (postfix)</b>
- The mail system name that is prepended to the
- process name in syslog records, so that "smtpd"
+ The mail system name that is prepended to the
+ process name in syslog records, so that "smtpd"
becomes, for example, "postfix/smtpd".
<b>FILES</b>
<a href="QSHAPE_README.html">QSHAPE_README</a>, Postfix queue analysis
<b>LICENSE</b>
- The Secure Mailer license must be distributed with this
+ The Secure Mailer license must be distributed with this
software.
<b>AUTHOR(S)</b>
</pre>
+</DD>
+
+<DT><b><a name="default_delivery_rate_delay">default_delivery_rate_delay</a>
+(default: 0s)</b></DT><DD>
+
+<p> The default amount of delay that is inserted between individual
+deliveries to the same destination; with per-destination recipient
+limit > 1, a destination is a domain, otherwise it is a recipient.
+</p>
+
+<p> To enable the delay, specify a non-zero time value (an integral
+value plus an optional one-letter suffix that specifies the time
+unit). </p>
+
+<p> Time units: s (seconds), m (minutes), h (hours), d (days), w
+(weeks). The default time unit is s (seconds). </p>
+
+<p> NOTE: the delay is enforced by the queue manager. The delay
+timer state does not survive "postfix reload" or "postfix stop".
+</p>
+
+<p> Use <a href="postconf.5.html#transport_delivery_rate_delay"><i>transport</i>_delivery_rate_delay</a> to specify a
+transport-specific override, where <i>transport</i> is the <a href="master.5.html">master.cf</a>
+name of the message delivery transport.
+</p>
+
+<p> This feature is available in Postfix 2.5 and later. </p>
+
+
</DD>
<DT><b><a name="default_delivery_slot_cost">default_delivery_slot_cost</a>
are not extended by more than 20-25 percent even in the worst case.
</p>
+<p> Use <a href="postconf.5.html#transport_delivery_slot_cost"><i>transport</i>_delivery_slot_cost</a> to specify a
+transport-specific override, where <i>transport</i> is the <a href="master.5.html">master.cf</a>
+name of the message delivery transport.
+</p>
+
<p>
Examples:
</p>
another preemption can take place later.
</p>
+<p> Use <a href="postconf.5.html#transport_delivery_slot_discount"><a href="postconf.5.html#transport_delivery_slot_discount"><i>transport</i>_delivery_slot_discount</a></a> to specify a
+transport-specific override, where <i>transport</i> is the <a href="master.5.html">master.cf</a>
+name of the message delivery transport.
+</p>
+
</DD>
another preemption can take place later.
</p>
+<p> Use <a href="postconf.5.html#transport_delivery_slot_loan"><i>transport</i>_delivery_slot_loan</a> to specify a
+transport-specific override, where <i>transport</i> is the <a href="master.5.html">master.cf</a>
+name of the message delivery transport.
+</p>
+
</DD>
The default maximal number of parallel deliveries to the same
destination. This is the default limit for delivery via the <a href="lmtp.8.html">lmtp(8)</a>,
<a href="pipe.8.html">pipe(8)</a>, <a href="smtp.8.html">smtp(8)</a> and <a href="virtual.8.html">virtual(8)</a> delivery agents.
+With per-destination recipient limit > 1, a destination is a domain,
+otherwise it is a recipient.
+</p>
+
+<p> Use <a href="postconf.5.html#transport_destination_concurrency_limit"><i>transport</i>_destination_concurrency_limit</a> to specify a
+transport-specific override, where <i>transport</i> is the <a href="master.5.html">master.cf</a>
+name of the message delivery transport.
</p>
the corresponding per-destination concurrency limit from concurrency
per domain into concurrency per recipient. </p>
+<p> Use <a href="postconf.5.html#transport_destination_recipient_limit"><i>transport</i>_destination_recipient_limit</a> to specify a
+transport-specific override, where <i>transport</i> is the <a href="master.5.html">master.cf</a>
+name of the message delivery transport.
+</p>
+
</DD>
degradation.
</p>
+<p> Use <a href="postconf.5.html#transport_extra_recipient_limit"><i>transport</i>_extra_recipient_limit</a> to specify a
+transport-specific override, where <i>transport</i> is the <a href="master.5.html">master.cf</a>
+name of the message delivery transport.
+</p>
+
</DD>
(subject to slot cost parameter as well) are never preempted.
</p>
+<p> Use <a href="postconf.5.html#transport_minimum_delivery_slots"><i>transport</i>_minimum_delivery_slots</a> to specify a
+transport-specific override, where <i>transport</i> is the <a href="master.5.html">master.cf</a>
+name of the message delivery transport.
+</p>
+
</DD>
and <a href="postconf.5.html#qmgr_message_recipient_minimum">qmgr_message_recipient_minimum</a>.
</p>
+<p> Use <a href="postconf.5.html#transport_recipient_limit"><i>transport</i>_recipient_limit</a> to specify a
+transport-specific override, where <i>transport</i> is the <a href="master.5.html">master.cf</a>
+name of the message delivery transport.
+</p>
+
</DD>
$<a href="postconf.5.html#default_recipient_refill_limit">default_recipient_refill_limit</a> is too high for too slow deliveries.
</p>
+<p> Use <a href="postconf.5.html#transport_recipient_refill_delay"><i>transport</i>_recipient_refill_delay</a> to specify a
+transport-specific override, where <i>transport</i> is the <a href="master.5.html">master.cf</a>
+name of the message delivery transport.
+</p>
+
<p> This feature is available in Postfix 2.4 and later. </p>
lower than this when this limit is too high for too slow deliveries.
</p>
+<p> Use <a href="postconf.5.html#transport_recipient_refill_limit"><i>transport</i>_recipient_refill_limit</a> to specify a
+transport-specific override, where <i>transport</i> is the <a href="master.5.html">master.cf</a>
+name of the message delivery transport.
+</p>
+
<p> This feature is available in Postfix 2.4 and later. </p>
The initial per-destination concurrency level for parallel delivery
to the same destination. This limit applies to delivery via <a href="smtp.8.html">smtp(8)</a>,
and via the <a href="pipe.8.html">pipe(8)</a> and <a href="virtual.8.html">virtual(8)</a> delivery agents.
+With per-destination recipient limit > 1, a destination is a domain,
+otherwise it is a recipient.
</p>
<p> Use <a href="postconf.5.html#transport_initial_destination_concurrency"><i>transport</i>_initial_destination_concurrency</a> to specify
</p>
+</DD>
+
+<DT><b><a name="transport_delivery_rate_delay">transport_delivery_rate_delay</a>
+(default: $<a href="postconf.5.html#default_delivery_rate_delay">default_delivery_rate_delay</a>)</b></DT><DD>
+
+<p> A transport-specific override for the <a href="postconf.5.html#default_recipient_refill_delay">default_recipient_refill_delay</a>
+parameter value, where <i>transport</i> is the <a href="master.5.html">master.cf</a> name of
+the message delivery transport. </p>
+
+<p> This feature is available in Postfix 2.5 and later. </p>
+
+
</DD>
<DT><b><a name="transport_delivery_slot_cost">transport_delivery_slot_cost</a>
Idem, for delivery via the named message <i>transport</i>.
<b><a href="postconf.5.html#default_destination_concurrency_negative_feedback">default_destination_concurrency_negative_feedback</a> (1)</b>
- The per-destination amount of negative delivery
- concurrency feedback, after a delivery completes
- with a connection or handshake failure.
+ The per-destination amount of delivery concurrency
+ negative feedback, after a delivery completes with
+ a connection or handshake failure.
<b><a href="postconf.5.html#transport_destination_concurrency_positive_feedback"><i>transport</i>_destination_concurrency_negative_feedback</a></b>
<b>($<a href="postconf.5.html#default_destination_concurrency_negative_feedback">default_destination_concurrency_negative_feedback</a>)</b>
Idem, for delivery via the named message <i>transport</i>.
<b><a href="postconf.5.html#default_destination_concurrency_positive_feedback">default_destination_concurrency_positive_feedback</a> (1)</b>
- The per-destination amount of positive delivery
- concurrency feedback, after a delivery completes
- without connection or handshake failure.
+ The per-destination amount of delivery concurrency
+ positive feedback, after a delivery completes with-
+ out connection or handshake failure.
<b><a href="postconf.5.html#transport_destination_concurrency_positive_feedback"><i>transport</i>_destination_concurrency_positive_feedback</a></b>
<b>($<a href="postconf.5.html#default_destination_concurrency_positive_feedback">default_destination_concurrency_positive_feedback</a>)</b>
The default value for transport-specific _deliv-
ery_slot_discount settings.
- <b><a href="postconf.5.html#transport_delivery_slot_discount"><
i>transport</i>_delivery_slot_discount</a> ($<a href="postconf.5.html#default_delivery_slot_discount">default_deliv</a>-</b>
+ <b><a href="postconf.5.html#transport_delivery_slot_discount"><
a href="postconf.5.html#transport_delivery_slot_discount"><i>transport</i>_delivery_slot_discount</a></a> ($<a href="postconf.5.html#default_delivery_slot_discount">default_deliv</a>-</b>
<b><a href="postconf.5.html#default_delivery_slot_discount">ery_slot_discount</a>)</b>
Idem, for delivery via the named message <i>transport</i>.
The maximal time a bounce message is queued before
it is considered undeliverable.
+ Available in Postfix version 2.5 and later:
+
+ <b><a href="postconf.5.html#default_delivery_rate_delay">default_delivery_rate_delay</a> (0s)</b>
+ The default amount of delay that is inserted
+ between individual deliveries to the same destina-
+ tion; with per-destination recipient limit > 1, a
+ destination is a domain, otherwise it is a recipi-
+ ent.
+
+ <b><a href="postconf.5.html#transport_delivery_rate_delay"><i>transport</i>_delivery_rate_delay</a> $<a href="postconf.5.html#default_delivery_rate_delay">default_delivery_rate_delay</a></b>
+ Idem, for delivery via the named message <i>transport</i>.
+
<b>MISCELLANEOUS CONTROLS</b>
<b><a href="postconf.5.html#config_directory">config_directory</a> (see 'postconf -d' output)</b>
- The default location of the Postfix <a href="postconf.5.html">main.cf</a> and
+ The default location of the Postfix <a href="postconf.5.html">main.cf</a> and
<a href="master.5.html">master.cf</a> configuration files.
<b><a href="postconf.5.html#daemon_timeout">daemon_timeout</a> (18000s)</b>
- How much time a Postfix daemon process may take to
- handle a request before it is terminated by a
+ How much time a Postfix daemon process may take to
+ handle a request before it is terminated by a
built-in watchdog timer.
<b><a href="postconf.5.html#defer_transports">defer_transports</a> (empty)</b>
"<b>sendmail -q</b>" or equivalent.
<b><a href="postconf.5.html#delay_logging_resolution_limit">delay_logging_resolution_limit</a> (2)</b>
- The maximal number of digits after the decimal
+ The maximal number of digits after the decimal
point when logging sub-second delay values.
<b><a href="postconf.5.html#helpful_warnings">helpful_warnings</a> (yes)</b>
- Log warnings about problematic configuration set-
+ Log warnings about problematic configuration set-
tings, and provide helpful suggestions.
<b><a href="postconf.5.html#ipc_timeout">ipc_timeout</a> (3600s)</b>
over an internal communication channel.
<b><a href="postconf.5.html#process_id">process_id</a> (read-only)</b>
- The process ID of a Postfix command or daemon
+ The process ID of a Postfix command or daemon
process.
<b><a href="postconf.5.html#process_name">process_name</a> (read-only)</b>
- The process name of a Postfix command or daemon
+ The process name of a Postfix command or daemon
process.
<b><a href="postconf.5.html#queue_directory">queue_directory</a> (see 'postconf -d' output)</b>
- The location of the Postfix top-level queue direc-
+ The location of the Postfix top-level queue direc-
tory.
<b><a href="postconf.5.html#syslog_facility">syslog_facility</a> (mail)</b>
The syslog facility of Postfix logging.
<b><a href="postconf.5.html#syslog_name">syslog_name</a> (postfix)</b>
- The mail system name that is prepended to the
- process name in syslog records, so that "smtpd"
+ The mail system name that is prepended to the
+ process name in syslog records, so that "smtpd"
becomes, for example, "postfix/smtpd".
<b>FILES</b>
<a href="QSHAPE_README.html">QSHAPE_README</a>, Postfix queue analysis
<b>LICENSE</b>
- The Secure Mailer license must be distributed with this
+ The Secure Mailer license must be distributed with this
software.
<b>AUTHOR(S)</b>
.fi
.ad
.ft R
+.SH default_delivery_rate_delay (default: 0s)
+The default amount of delay that is inserted between individual
+deliveries to the same destination; with per-destination recipient
+limit > 1, a destination is a domain, otherwise it is a recipient.
+.PP
+To enable the delay, specify a non-zero time value (an integral
+value plus an optional one-letter suffix that specifies the time
+unit).
+.PP
+Time units: s (seconds), m (minutes), h (hours), d (days), w
+(weeks). The default time unit is s (seconds).
+.PP
+NOTE: the delay is enforced by the queue manager. The delay
+timer state does not survive "postfix reload" or "postfix stop".
+.PP
+Use \fItransport\fR_delivery_rate_delay to specify a
+transport-specific override, where \fItransport\fR is the master.cf
+name of the message delivery transport.
+.PP
+This feature is available in Postfix 2.5 and later.
.SH default_delivery_slot_cost (default: 5)
How often the Postfix queue manager's scheduler is allowed to
preempt delivery of one message with another.
message response times while making sure the mailing-list deliveries
are not extended by more than 20-25 percent even in the worst case.
.PP
+Use \fItransport\fR_delivery_slot_cost to specify a
+transport-specific override, where \fItransport\fR is the master.cf
+name of the message delivery transport.
+.PP
Examples:
.PP
.nf
plus transport_delivery_slot_loan still remains to be accumulated.
Note that the full amount will still have to be accumulated before
another preemption can take place later.
+.PP
+Use \fItransport\fR_delivery_slot_discount to specify a
+transport-specific override, where \fItransport\fR is the master.cf
+name of the message delivery transport.
.SH default_delivery_slot_loan (default: 3)
The default value for transport-specific _delivery_slot_loan
settings.
plus transport_delivery_slot_loan still remains to be accumulated.
Note that the full amount will still have to be accumulated before
another preemption can take place later.
+.PP
+Use \fItransport\fR_delivery_slot_loan to specify a
+transport-specific override, where \fItransport\fR is the master.cf
+name of the message delivery transport.
.SH default_destination_concurrency_failed_cohort_limit (default: 1)
How many pseudo-cohorts must suffer connection or handshake
failure before a specific destination is considered unavailable
The default maximal number of parallel deliveries to the same
destination. This is the default limit for delivery via the \fBlmtp\fR(8),
\fBpipe\fR(8), \fBsmtp\fR(8) and \fBvirtual\fR(8) delivery agents.
+With per-destination recipient limit > 1, a destination is a domain,
+otherwise it is a recipient.
+.PP
+Use \fItransport\fR_destination_concurrency_limit to specify a
+transport-specific override, where \fItransport\fR is the master.cf
+name of the message delivery transport.
.SH default_destination_concurrency_negative_feedback (default: 1)
The per-destination amount of delivery concurrency negative
feedback, after a delivery completes with a connection or handshake
Setting this parameter to a value of 1 changes the meaning of
the corresponding per-destination concurrency limit from concurrency
per domain into concurrency per recipient.
+.PP
+Use \fItransport\fR_destination_recipient_limit to specify a
+transport-specific override, where \fItransport\fR is the master.cf
+name of the message delivery transport.
.SH default_extra_recipient_limit (default: 1000)
The default value for the extra per-transport limit imposed on the
number of in-memory recipients. This extra recipient space is
preempts one message with another and suddenly needs some extra
recipients slots for the chosen message in order to avoid performance
degradation.
+.PP
+Use \fItransport\fR_extra_recipient_limit to specify a
+transport-specific override, where \fItransport\fR is the master.cf
+name of the message delivery transport.
.SH default_minimum_delivery_slots (default: 3)
How many recipients a message must have in order to invoke the
Postfix queue manager's scheduling algorithm at all. Messages
which would never accumulate at least this many delivery slots
(subject to slot cost parameter as well) are never preempted.
+.PP
+Use \fItransport\fR_minimum_delivery_slots to specify a
+transport-specific override, where \fItransport\fR is the master.cf
+name of the message delivery transport.
.SH default_privs (default: nobody)
The default rights used by the \fBlocal\fR(8) delivery agent for delivery
to external file or command. These rights are used when delivery
qmgr_message_recipient_limit after the message has been assigned
to the respective transports. See also default_extra_recipient_limit
and qmgr_message_recipient_minimum.
+.PP
+Use \fItransport\fR_recipient_limit to specify a
+transport-specific override, where \fItransport\fR is the master.cf
+name of the message delivery transport.
.SH default_recipient_refill_delay (default: 5s)
The default per-transport maximum delay between recipients refills.
When not all message recipients fit into the memory at once, keep loading
make sure the recipients are refilled in timely manner even when
$default_recipient_refill_limit is too high for too slow deliveries.
.PP
+Use \fItransport\fR_recipient_refill_delay to specify a
+transport-specific override, where \fItransport\fR is the master.cf
+name of the message delivery transport.
+.PP
This feature is available in Postfix 2.4 and later.
.SH default_recipient_refill_limit (default: 100)
The default per-transport limit on the number of recipients refilled at
$default_recipient_refill_delay, which may result in recipient batches
lower than this when this limit is too high for too slow deliveries.
.PP
+Use \fItransport\fR_recipient_refill_limit to specify a
+transport-specific override, where \fItransport\fR is the master.cf
+name of the message delivery transport.
+.PP
This feature is available in Postfix 2.4 and later.
.SH default_transport (default: smtp)
The default mail delivery transport and next-hop destination for
The initial per-destination concurrency level for parallel delivery
to the same destination. This limit applies to delivery via \fBsmtp\fR(8),
and via the \fBpipe\fR(8) and \fBvirtual\fR(8) delivery agents.
+With per-destination recipient limit > 1, a destination is a domain,
+otherwise it is a recipient.
.PP
Use \fItransport\fR_initial_destination_concurrency to specify
a transport-specific override, where \fItransport\fR is the master.cf
delivery is requested with "\fBsendmail -v\fR".
.PP
This feature is available in Postfix 2.1 and later.
+.SH transport_delivery_rate_delay (default: $default_delivery_rate_delay)
+A transport-specific override for the default_recipient_refill_delay
+parameter value, where \fItransport\fR is the master.cf name of
+the message delivery transport.
+.PP
+This feature is available in Postfix 2.5 and later.
.SH transport_delivery_slot_cost (default: $default_delivery_slot_cost)
A transport-specific override for the default_delivery_slot_cost
parameter value, where \fItransport\fR is the master.cf name of
.IP "\fItransport\fB_destination_concurrency_failed_cohort_limit ($default_destination_concurrency_failed_cohort_limit)\fR"
Idem, for delivery via the named message \fItransport\fR.
.IP "\fBdefault_destination_concurrency_negative_feedback (1)\fR"
-The per-destination amount of negative delivery concurrency
+The per-destination amount of delivery concurrency negative
feedback, after a delivery completes with a connection or handshake
failure.
.IP "\fItransport\fB_destination_concurrency_negative_feedback ($default_destination_concurrency_negative_feedback)\fR"
Idem, for delivery via the named message \fItransport\fR.
.IP "\fBdefault_destination_concurrency_positive_feedback (1)\fR"
-The per-destination amount of positive delivery concurrency
+The per-destination amount of delivery concurrency positive
feedback, after a delivery completes without connection or handshake
failure.
.IP "\fItransport\fB_destination_concurrency_positive_feedback ($default_destination_concurrency_positive_feedback)\fR"
.IP "\fBbounce_queue_lifetime (5d)\fR"
The maximal time a bounce message is queued before it is considered
undeliverable.
+.PP
+Available in Postfix version 2.5 and later:
+.IP "\fBdefault_delivery_rate_delay (0s)\fR"
+The default amount of delay that is inserted between individual
+deliveries to the same destination; with per-destination recipient
+limit > 1, a destination is a domain, otherwise it is a recipient.
+.IP "\fItransport\fB_delivery_rate_delay $default_delivery_rate_delay
+Idem, for delivery via the named message \fItransport\fR.
.SH MISCELLANEOUS CONTROLS
.ad
.fi
.IP "\fItransport\fB_destination_concurrency_failed_cohort_limit ($default_destination_concurrency_failed_cohort_limit)\fR"
Idem, for delivery via the named message \fItransport\fR.
.IP "\fBdefault_destination_concurrency_negative_feedback (1)\fR"
-The per-destination amount of negative delivery concurrency
+The per-destination amount of delivery concurrency negative
feedback, after a delivery completes with a connection or handshake
failure.
.IP "\fItransport\fB_destination_concurrency_negative_feedback ($default_destination_concurrency_negative_feedback)\fR"
Idem, for delivery via the named message \fItransport\fR.
.IP "\fBdefault_destination_concurrency_positive_feedback (1)\fR"
-The per-destination amount of positive delivery concurrency
+The per-destination amount of delivery concurrency positive
feedback, after a delivery completes without connection or handshake
failure.
.IP "\fItransport\fB_destination_concurrency_positive_feedback ($default_destination_concurrency_positive_feedback)\fR"
.IP "\fBbounce_queue_lifetime (5d)\fR"
The maximal time a bounce message is queued before it is considered
undeliverable.
+.PP
+Available in Postfix version 2.5 and later:
+.IP "\fBdefault_delivery_rate_delay (0s)\fR"
+The default amount of delay that is inserted between individual
+deliveries to the same destination; with per-destination recipient
+limit > 1, a destination is a domain, otherwise it is a recipient.
+.IP "\fItransport\fB_delivery_rate_delay $default_delivery_rate_delay
+Idem, for delivery via the named message \fItransport\fR.
.SH "MISCELLANEOUS CONTROLS"
.na
.nf
s;\bdefault_destination_concur[-</Bb>]*\n* *[<Bb>]*rency_positive_feedback\b;<a href="postconf.5.html#default_destination_concurrency_positive_feedback">$&</a>;g;
s;\bdefault_destination_con[-</Bb>]*\n* *[<Bb>]*currency_failed_cohort_limit\b;<a href="postconf.5.html#default_destination_concurrency_failed_cohort_limit">$&</a>;g;
s;\bdestination_concurrency_feedback_debug\b;<a href="postconf.5.html#destination_concurrency_feedback_debug">$&</a>;g;
+ s;\bdefault_delivery_rate_delay\b;<a href="postconf.5.html#default_delivery_rate_delay">$&</a>;g;
s;\bqmqpd_error_delay\b;<a href="postconf.5.html#qmqpd_error_delay">$&</a>;g;
s;\bqmqpd_timeout\b;<a href="postconf.5.html#qmqpd_timeout">$&</a>;g;
s;(<i>transport</i>)(<b>)?(_recipient_refill_delay)\b;$2<a href="postconf.5.html#transport_recipient_refill_delay">$1$3</a>;g;
s;(<i>transport</i>)(<b>)?(_recipient_refill_limit)\b;$2<a href="postconf.5.html#transport_recipient_refill_limit">$1$3</a>;g;
s;(<i>transport</i>)(<b>)?(_time_limit)\b;$2<a href="postconf.5.html#transport_time_limit">$1$3</a>;g;
- s;(<i>transport</i>)(<b>)?(delivery_slot_discount)\b;$2<a href="postconf.5.html#transportdelivery_slot_discount">$1$3</a>;g;
+ s;(<i>transport</i>)(<b>)?(_delivery_slot_discount)\b;$2<a href="postconf.5.html#transport_delivery_slot_discount">$1$3</a>;g;
+ s;(<i>transport</i>)(<b>)?(_delivery_rate_delay)\b;$2<a href="postconf.5.html#transport_delivery_rate_delay">$1$3</a>;g;
# Undo hyperlinks of manual pages with the same name as parameters.
<h3> <a name="concurrency_limitations"> Limitations of less-than-1 per delivery feedback </a> </h3>
-<p> The delivery concurrency scheduler with less-than-1 concurrency
+<p> The scheduler with less-than-1 concurrency
feedback per delivery solves a problem with servers that have active
concurrency limiters. This works only because feedback is handled
in a peculiar manner: positive feedback will increment the concurrency
balancer are perfectly capable of handling more traffic. </p>
<p> This noise problem gets worse as the amount of positive feedback
-per delivery gets smaller. A compromise is to avoid concurrency-dependent
-positive feedback, and to use fixed less-than-1 feedback values
-instead. For example, to tolerate 1 of 4 bad servers in the above
+per delivery gets smaller. A compromise is to use fixed less-than-1
+positive feedback values instead of concurrency-dependent positive
+feedback. For example, to tolerate 1 of 4 bad servers in the above
load balancer scenario, use positive feedback of 1/4 per "good"
delivery (no connect or handshake error), and use an equal or smaller
amount of negative feedback per "bad" delivery. The downside of
are not extended by more than 20-25 percent even in the worst case.
</p>
+<p> Use <i>transport</i>_delivery_slot_cost to specify a
+transport-specific override, where <i>transport</i> is the master.cf
+name of the message delivery transport.
+</p>
+
<p>
Examples:
</p>
The default maximal number of parallel deliveries to the same
destination. This is the default limit for delivery via the lmtp(8),
pipe(8), smtp(8) and virtual(8) delivery agents.
+With per-destination recipient limit > 1, a destination is a domain,
+otherwise it is a recipient.
+</p>
+
+<p> Use <i>transport</i>_destination_concurrency_limit to specify a
+transport-specific override, where <i>transport</i> is the master.cf
+name of the message delivery transport.
</p>
%PARAM default_destination_recipient_limit 50
the corresponding per-destination concurrency limit from concurrency
per domain into concurrency per recipient. </p>
+<p> Use <i>transport</i>_destination_recipient_limit to specify a
+transport-specific override, where <i>transport</i> is the master.cf
+name of the message delivery transport.
+</p>
+
%PARAM default_extra_recipient_limit 1000
<p>
degradation.
</p>
+<p> Use <i>transport</i>_extra_recipient_limit to specify a
+transport-specific override, where <i>transport</i> is the master.cf
+name of the message delivery transport.
+</p>
+
%PARAM default_minimum_delivery_slots 3
<p>
(subject to slot cost parameter as well) are never preempted.
</p>
+<p> Use <i>transport</i>_minimum_delivery_slots to specify a
+transport-specific override, where <i>transport</i> is the master.cf
+name of the message delivery transport.
+</p>
+
%PARAM default_privs nobody
<p>
and qmgr_message_recipient_minimum.
</p>
+<p> Use <i>transport</i>_recipient_limit to specify a
+transport-specific override, where <i>transport</i> is the master.cf
+name of the message delivery transport.
+</p>
+
%PARAM default_recipient_refill_limit 100
<p>
lower than this when this limit is too high for too slow deliveries.
</p>
+<p> Use <i>transport</i>_recipient_refill_limit to specify a
+transport-specific override, where <i>transport</i> is the master.cf
+name of the message delivery transport.
+</p>
+
<p> This feature is available in Postfix 2.4 and later. </p>
%PARAM default_recipient_refill_delay 5s
$default_recipient_refill_limit is too high for too slow deliveries.
</p>
+<p> Use <i>transport</i>_recipient_refill_delay to specify a
+transport-specific override, where <i>transport</i> is the master.cf
+name of the message delivery transport.
+</p>
+
<p> This feature is available in Postfix 2.4 and later. </p>
%PARAM default_transport smtp
The initial per-destination concurrency level for parallel delivery
to the same destination. This limit applies to delivery via smtp(8),
and via the pipe(8) and virtual(8) delivery agents.
+With per-destination recipient limit > 1, a destination is a domain,
+otherwise it is a recipient.
</p>
<p> Use <i>transport</i>_initial_destination_concurrency to specify
another preemption can take place later.
</p>
+<p> Use <i>transport</i>_delivery_slot_discount to specify a
+transport-specific override, where <i>transport</i> is the master.cf
+name of the message delivery transport.
+</p>
+
%PARAM default_delivery_slot_loan 3
<p>
another preemption can take place later.
</p>
+<p> Use <i>transport</i>_delivery_slot_loan to specify a
+transport-specific override, where <i>transport</i> is the master.cf
+name of the message delivery transport.
+</p>
+
%CLASS verp VERP Support
<p>
the message delivery transport. </p>
<p> This feature is available in Postfix 2.4 and later. </p>
+
+%PARAM default_delivery_rate_delay 0s
+
+<p> The default amount of delay that is inserted between individual
+deliveries to the same destination; with per-destination recipient
+limit > 1, a destination is a domain, otherwise it is a recipient.
+</p>
+
+<p> To enable the delay, specify a non-zero time value (an integral
+value plus an optional one-letter suffix that specifies the time
+unit). </p>
+
+<p> Time units: s (seconds), m (minutes), h (hours), d (days), w
+(weeks). The default time unit is s (seconds). </p>
+
+<p> NOTE: the delay is enforced by the queue manager. The delay
+timer state does not survive "postfix reload" or "postfix stop".
+</p>
+
+<p> Use <i>transport</i>_delivery_rate_delay to specify a
+transport-specific override, where <i>transport</i> is the master.cf
+name of the message delivery transport.
+</p>
+
+<p> This feature is available in Postfix 2.5 and later. </p>
+
+%PARAM transport_delivery_rate_delay $default_delivery_rate_delay
+
+<p> A transport-specific override for the default_recipient_refill_delay
+parameter value, where <i>transport</i> is the master.cf name of
+the message delivery transport. </p>
+
+<p> This feature is available in Postfix 2.5 and later. </p>
#define DEF_CONC_FDBACK_DEBUG 0
extern bool var_conc_feedback_debug;
+#define VAR_DEST_RATE_DELAY "default_delivery_rate_delay"
+#define _DEST_RATE_DELAY "_delivery_rate_delay"
+#define DEF_DEST_RATE_DELAY "0s"
+extern int var_dest_rate_delay;
+
/* LICENSE
/* .ad
/* .fi
/* .IP "\fItransport\fB_destination_concurrency_failed_cohort_limit ($default_destination_concurrency_failed_cohort_limit)\fR"
/* Idem, for delivery via the named message \fItransport\fR.
/* .IP "\fBdefault_destination_concurrency_negative_feedback (1)\fR"
-/* The per-destination amount of negative delivery concurrency
+/* The per-destination amount of delivery concurrency negative
/* feedback, after a delivery completes with a connection or handshake
/* failure.
/* .IP "\fItransport\fB_destination_concurrency_negative_feedback ($default_destination_concurrency_negative_feedback)\fR"
/* Idem, for delivery via the named message \fItransport\fR.
/* .IP "\fBdefault_destination_concurrency_positive_feedback (1)\fR"
-/* The per-destination amount of positive delivery concurrency
+/* The per-destination amount of delivery concurrency positive
/* feedback, after a delivery completes without connection or handshake
/* failure.
/* .IP "\fItransport\fB_destination_concurrency_positive_feedback ($default_destination_concurrency_positive_feedback)\fR"
/* .IP "\fBbounce_queue_lifetime (5d)\fR"
/* The maximal time a bounce message is queued before it is considered
/* undeliverable.
+/* .PP
+/* Available in Postfix version 2.5 and later:
+/* .IP "\fBdefault_delivery_rate_delay (0s)\fR"
+/* The default amount of delay that is inserted between individual
+/* deliveries to the same destination; with per-destination recipient
+/* limit > 1, a destination is a domain, otherwise it is a recipient.
+/* .IP "\fItransport\fB_delivery_rate_delay $default_delivery_rate_delay
+/* Idem, for delivery via the named message \fItransport\fR.
/* .SH MISCELLANEOUS CONTROLS
/* .ad
/* .fi
char *var_conc_neg_feedback;
int var_conc_cohort_limit;
int var_conc_feedback_debug;
+int var_dest_rate_delay;
static QMGR_SCAN *qmgr_scans[2];
VAR_DSN_QUEUE_TIME, DEF_DSN_QUEUE_TIME, &var_dsn_queue_time, 0, 8640000,
VAR_XPORT_RETRY_TIME, DEF_XPORT_RETRY_TIME, &var_transport_retry_time, 1, 0,
VAR_QMGR_CLOG_WARN_TIME, DEF_QMGR_CLOG_WARN_TIME, &var_qmgr_clog_warn_time, 0, 0,
+ VAR_DEST_RATE_DELAY, DEF_DEST_RATE_DELAY, &var_dest_rate_delay, 0, 0,
0,
};
static CONFIG_INT_TABLE int_table[] = {
QMGR_FEEDBACK pos_feedback; /* positive feedback control */
QMGR_FEEDBACK neg_feedback; /* negative feedback control */
int fail_cohort_limit; /* flow shutdown control */
+ int rate_delay; /* suspend per delivery */
};
#define QMGR_TRANSPORT_STAT_DEAD (1<<1)
extern void qmgr_queue_throttle(QMGR_QUEUE *, DSN *);
extern void qmgr_queue_unthrottle(QMGR_QUEUE *);
extern QMGR_QUEUE *qmgr_queue_find(QMGR_TRANSPORT *, const char *);
+extern void qmgr_queue_suspend(QMGR_QUEUE *, int);
-#define QMGR_QUEUE_THROTTLED(q) ((q)->window <= 0)
+ /*
+ * Exclusive queue states. Originally there were only two: "throttled" and
+ * "not throttled". It was natural to encode these in the queue window size.
+ * After 10 years it's not practical to rip out all the working code and
+ * change representations, so we just clean up the names a little.
+ *
+ * Note: only the "ready" state can reach every state (including itself);
+ * non-ready states can reach only the "ready" state. Other transitions are
+ * forbidden, because they would result in dangling event handlers.
+ */
+#define QMGR_QUEUE_STAT_THROTTLED 0 /* back-off timer */
+#define QMGR_QUEUE_STAT_SUSPENDED -1 /* voluntary delay timer */
+#define QMGR_QUEUE_STAT_SAVED -2 /* delayed cleanup timer */
+#define QMGR_QUEUE_STAT_BAD -3 /* can't happen */
+
+#define QMGR_QUEUE_READY(q) ((q)->window > 0)
+#define QMGR_QUEUE_THROTTLED(q) ((q)->window == QMGR_QUEUE_STAT_THROTTLED)
+#define QMGR_QUEUE_SUSPENDED(q) ((q)->window == QMGR_QUEUE_STAT_SUSPENDED)
+#define QMGR_QUEUE_SAVED(q) ((q)->window == QMGR_QUEUE_STAT_SAVED)
+#define QMGR_QUEUE_BAD(q) ((q)->window <= QMGR_QUEUE_STAT_BAD)
+
+#define QMGR_QUEUE_STATUS(q) ( \
+ QMGR_QUEUE_READY(q) ? "ready" : \
+ QMGR_QUEUE_THROTTLED(q) ? "throttled" : \
+ QMGR_QUEUE_SUSPENDED(q) ? "suspended" : \
+ QMGR_QUEUE_SAVED(q) ? "saved" : \
+ "invalid queue status" \
+ )
/*
* Structure of one next-hop queue entry. In order to save some copying
if (VSTRING_LEN(dsb->reason) == 0)
vstring_strcpy(dsb->reason, "unknown error");
vstring_prepend(dsb->reason, SUSPENDED, sizeof(SUSPENDED) - 1);
- if (queue->window > 0) {
+ if (QMGR_QUEUE_READY(queue)) {
qmgr_queue_throttle(queue, DSN_FROM_DSN_BUF(dsb));
- if (queue->window == 0)
+ if (QMGR_QUEUE_THROTTLED(queue))
qmgr_defer_todo(queue, &dsb->dsn);
}
}
void qmgr_enable_queue(QMGR_QUEUE *queue)
{
- if (queue->window == 0) {
+ if (QMGR_QUEUE_THROTTLED(queue)) {
if (msg_verbose)
msg_info("enable site %s/%s", queue->transport->name, queue->name);
qmgr_queue_unthrottle(queue);
}
- if (queue->todo.next == 0 && queue->busy.next == 0)
+ if (QMGR_QUEUE_READY(queue) && queue->todo.next == 0 && queue->busy.next == 0)
qmgr_queue_done(queue);
}
void qmgr_entry_done(QMGR_ENTRY *entry, int which)
{
+ const char *myname = "qmgr_entry_done";
QMGR_QUEUE *queue = entry->queue;
QMGR_MESSAGE *message = entry->message;
+ QMGR_TRANSPORT *transport = queue->transport;
/*
* Take this entry off the in-core queue.
*/
if (entry->stream != 0)
- msg_panic("qmgr_entry_done: file is open");
+ msg_panic("%s: file is open", myname);
if (which == QMGR_QUEUE_BUSY) {
QMGR_LIST_UNLINK(queue->busy, QMGR_ENTRY *, entry);
queue->busy_refcount--;
QMGR_LIST_UNLINK(queue->todo, QMGR_ENTRY *, entry);
queue->todo_refcount--;
} else {
- msg_panic("qmgr_entry_done: bad queue spec: %d", which);
+ msg_panic("%s: bad queue spec: %d", myname, which);
}
/*
/*
* Maintain back-to-back delivery status.
*/
- queue->last_done = event_time();
+ if (which == QMGR_QUEUE_BUSY)
+ queue->last_done = event_time();
+
+ /*
+ * Suspend a rate-limited queue, so that mail trickles out.
+ */
+ if (which == QMGR_QUEUE_BUSY && transport->rate_delay > 0) {
+ if (queue->window > 1)
+ msg_panic("%s: queue %s/%s: window %d > 1 on rate-limited service",
+ myname, transport->name, queue->name, queue->window);
+ if (QMGR_QUEUE_THROTTLED(queue)) /* XXX */
+ qmgr_queue_unthrottle(queue);
+ if (QMGR_QUEUE_READY(queue))
+ qmgr_queue_suspend(queue, transport->rate_delay);
+ }
/*
* When the in-core queue for this site is empty and when this site is
* See also: qmgr_entry_move_todo().
*/
if (queue->todo.next == 0 && queue->busy.next == 0) {
- if (queue->window == 0 && qmgr_queue_count > 2 * var_qmgr_rcpt_limit)
+ if (QMGR_QUEUE_THROTTLED(queue) && qmgr_queue_count > 2 * var_qmgr_rcpt_limit)
qmgr_queue_unthrottle(queue);
- if (queue->window > 0)
+ if (QMGR_QUEUE_READY(queue))
qmgr_queue_done(queue);
}
/*
* Sanity check.
*/
- if (queue->window == 0)
+ if (QMGR_QUEUE_THROTTLED(queue))
msg_panic("qmgr_entry_create: dead queue: %s", queue->name);
/*
/*
/* void qmgr_queue_unthrottle(queue)
/* QMGR_QUEUE *queue;
+/*
+/* void qmgr_queue_suspend(queue, delay)
+/* QMGR_QUEUE *queue;
+/* int delay;
/* DESCRIPTION
/* These routines add/delete/manipulate per-destination queues.
/* Each queue corresponds to a specific transport and destination.
/* provided that it does not exceed the destination concurrency
/* limit specified for the transport. This routine implements
/* "slow open" mode, and eliminates the "thundering herd" problem.
+/*
+/* qmgr_queue_suspend() suspends delivery for this destination
+/* briefly.
/* DIAGNOSTICS
-/* None
+/* Panic: consistency check failure.
/* LICENSE
/* .ad
/* .fi
myname, queue->name, queue->transport->dest_concurrency_limit, \
queue->window, queue->success, queue->failure, queue->fail_cohorts);
+/* qmgr_queue_resume - resume delivery to destination */
+
+static void qmgr_queue_resume(int event, char *context)
+{
+ QMGR_QUEUE *queue = (QMGR_QUEUE *) context;
+ const char *myname = "qmgr_queue_resume";
+
+ /*
+ * Sanity checks.
+ */
+ if (!QMGR_QUEUE_SUSPENDED(queue))
+ msg_panic("%s: bad queue status: %s", myname, QMGR_QUEUE_STATUS(queue));
+
+ /*
+ * We can't simply force delivery on this queue: the transport's pending
+ * count may already be maxed out, and there may be other constraints
+ * that definitely should be none of our business. The best we can do is
+ * to play by the same rules as everyone else: trigger *some* delivery
+ * via qmgr_active_drain() and let round-robin selection work for us.
+ */
+ queue->window = 1;
+ if (queue->todo_refcount > 0)
+ qmgr_active_drain();
+}
+
+/* qmgr_queue_suspend - briefly suspend a destination */
+
+void qmgr_queue_suspend(QMGR_QUEUE *queue, int delay)
+{
+ const char *myname = "qmgr_queue_suspend";
+
+ /*
+ * Sanity checks.
+ */
+ if (!QMGR_QUEUE_READY(queue))
+ msg_panic("%s: bad queue status: %s", myname, QMGR_QUEUE_STATUS(queue));
+ if (queue->busy_refcount > 0)
+ msg_panic("%s: queue is busy", myname);
+
+ /*
+ * Set the queue status to "suspended". No-one is supposed to remove a
+ * queue in suspended state.
+ */
+ queue->window = QMGR_QUEUE_STAT_SUSPENDED;
+ event_request_timer(qmgr_queue_resume, (char *) queue, delay);
+}
+
/* qmgr_queue_unthrottle_wrapper - in case (char *) != (struct *) */
static void qmgr_queue_unthrottle_wrapper(int unused_event, char *context)
* this in-core queue when it is empty and when this site is not dead.
*/
qmgr_queue_unthrottle(queue);
- if (queue->window > 0 && queue->todo.next == 0 && queue->busy.next == 0)
+ if (QMGR_QUEUE_READY(queue) && queue->todo.next == 0 && queue->busy.next == 0)
qmgr_queue_done(queue);
}
if (msg_verbose)
msg_info("%s: queue %s", myname, queue->name);
+ /*
+ * Sanity checks.
+ */
+ if (!QMGR_QUEUE_THROTTLED(queue) && !QMGR_QUEUE_READY(queue))
+ msg_panic("%s: bad queue status: %s", myname, QMGR_QUEUE_STATUS(queue));
+
/*
* Don't restart the negative feedback hysteresis cycle with every
* positive feedback. Restart it only when we make a positive concurrency
/*
* Special case when this site was dead.
*/
- if (queue->window == 0) {
+ if (QMGR_QUEUE_THROTTLED(queue)) {
event_cancel_timer(qmgr_queue_unthrottle_wrapper, (char *) queue);
if (queue->dsn == 0)
msg_panic("%s: queue %s: window 0 status 0", myname, queue->name);
/*
* Sanity checks.
*/
+ if (!QMGR_QUEUE_READY(queue))
+ msg_panic("%s: bad queue status: %s", myname, QMGR_QUEUE_STATUS(queue));
if (queue->dsn)
msg_panic("%s: queue %s: spurious reason %s",
myname, queue->name, queue->dsn->reason);
* This queue is declared dead after a configurable number of
* pseudo-cohort failures.
*/
- if (queue->window > 0) {
+ if (QMGR_QUEUE_READY(queue)) {
queue->fail_cohorts += 1.0 / queue->window;
if (transport->fail_cohort_limit > 0
&& queue->fail_cohorts >= transport->fail_cohort_limit)
* Even after reaching 1, we maintain the negative hysteresis cycle so that
* negative feedback can cancel out positive feedback.
*/
- if (queue->window > 0) {
+ if (QMGR_QUEUE_READY(queue)) {
feedback = QMGR_FEEDBACK_VAL(transport->neg_feedback, queue->window);
QMGR_LOG_FEEDBACK(feedback);
queue->failure -= feedback;
/*
* Special case for a site that just was declared dead.
*/
- if (queue->window == 0) {
+ if (QMGR_QUEUE_THROTTLED(queue)) {
queue->dsn = DSN_COPY(dsn);
event_request_timer(qmgr_queue_unthrottle_wrapper,
(char *) queue, var_min_backoff_time);
queue->busy_refcount + queue->todo_refcount);
if (queue->todo.next || queue->busy.next)
msg_panic("%s: queue not empty: %s", myname, queue->name);
- if (queue->window <= 0)
- msg_panic("%s: window %d", myname, queue->window);
+ if (!QMGR_QUEUE_READY(queue))
+ msg_panic("%s: bad queue status: %s", myname, QMGR_QUEUE_STATUS(queue));
if (queue->dsn)
msg_panic("%s: queue %s: spurious reason %s",
myname, queue->name, queue->dsn->reason);
transport->init_dest_concurrency =
get_mail_conf_int2(name, _INIT_DEST_CON,
var_init_dest_concurrency, 1, 0);
+ transport->rate_delay = get_mail_conf_time2(name, _DEST_RATE_DELAY,
+ var_dest_rate_delay,
+ 's', 0, 0);
+ if (transport->rate_delay > 0)
+ transport->dest_concurrency_limit = 1;
if (transport->dest_concurrency_limit != 0
&& transport->dest_concurrency_limit < transport->init_dest_concurrency)
transport->init_dest_concurrency = transport->dest_concurrency_limit;
vars["destination_concurrency_positive_feedback"] = "default_destination_concurrency_positive_feedback"
vars["destination_recipient_limit"] = "default_destination_recipient_limit"
vars["initial_destination_concurrency"] = "initial_destination_concurrency"
+ vars["delivery_rate_delay"] = "default_delivery_rate_delay"
# auto_table.h
/* .IP "\fItransport\fB_destination_concurrency_failed_cohort_limit ($default_destination_concurrency_failed_cohort_limit)\fR"
/* Idem, for delivery via the named message \fItransport\fR.
/* .IP "\fBdefault_destination_concurrency_negative_feedback (1)\fR"
-/* The per-destination amount of negative delivery concurrency
+/* The per-destination amount of delivery concurrency negative
/* feedback, after a delivery completes with a connection or handshake
/* failure.
/* .IP "\fItransport\fB_destination_concurrency_negative_feedback ($default_destination_concurrency_negative_feedback)\fR"
/* Idem, for delivery via the named message \fItransport\fR.
/* .IP "\fBdefault_destination_concurrency_positive_feedback (1)\fR"
-/* The per-destination amount of positive delivery concurrency
+/* The per-destination amount of delivery concurrency positive
/* feedback, after a delivery completes without connection or handshake
/* failure.
/* .IP "\fItransport\fB_destination_concurrency_positive_feedback ($default_destination_concurrency_positive_feedback)\fR"
/* .IP "\fBbounce_queue_lifetime (5d)\fR"
/* The maximal time a bounce message is queued before it is considered
/* undeliverable.
+/* .PP
+/* Available in Postfix version 2.5 and later:
+/* .IP "\fBdefault_delivery_rate_delay (0s)\fR"
+/* The default amount of delay that is inserted between individual
+/* deliveries to the same destination; with per-destination recipient
+/* limit > 1, a destination is a domain, otherwise it is a recipient.
+/* .IP "\fItransport\fB_delivery_rate_delay $default_delivery_rate_delay
+/* Idem, for delivery via the named message \fItransport\fR.
/* MISCELLANEOUS CONTROLS
/* .ad
/* .fi
char *var_conc_neg_feedback;
int var_conc_cohort_limit;
int var_conc_feedback_debug;
+int var_dest_rate_delay;
static QMGR_SCAN *qmgr_scans[2];
VAR_XPORT_RETRY_TIME, DEF_XPORT_RETRY_TIME, &var_transport_retry_time, 1, 0,
VAR_QMGR_CLOG_WARN_TIME, DEF_QMGR_CLOG_WARN_TIME, &var_qmgr_clog_warn_time, 0, 0,
VAR_XPORT_REFILL_DELAY, DEF_XPORT_REFILL_DELAY, &var_xport_refill_delay, 1, 0,
+ VAR_DEST_RATE_DELAY, DEF_DEST_RATE_DELAY, &var_dest_rate_delay, 0, 0,
0,
};
static CONFIG_INT_TABLE int_table[] = {
QMGR_FEEDBACK pos_feedback; /* positive feedback control */
QMGR_FEEDBACK neg_feedback; /* negative feedback control */
int fail_cohort_limit; /* flow shutdown control */
+ int rate_delay; /* suspend per delivery */
};
#define QMGR_TRANSPORT_STAT_DEAD (1<<1)
extern void qmgr_queue_throttle(QMGR_QUEUE *, DSN *);
extern void qmgr_queue_unthrottle(QMGR_QUEUE *);
extern QMGR_QUEUE *qmgr_queue_find(QMGR_TRANSPORT *, const char *);
+extern void qmgr_queue_suspend(QMGR_QUEUE *, int);
-#define QMGR_QUEUE_THROTTLED(q) ((q)->window <= 0)
+ /*
+ * Exclusive queue states. Originally there were only two: "throttled" and
+ * "not throttled". It was natural to encode these in the queue window size.
+ * After 10 years it's not practical to rip out all the working code and
+ * change representations, so we just clean up the names a little.
+ *
+ * Note: only the "ready" state can reach every state (including itself);
+ * non-ready states can reach only the "ready" state. Other transitions are
+ * forbidden, because they would result in dangling event handlers.
+ */
+#define QMGR_QUEUE_STAT_THROTTLED 0 /* back-off timer */
+#define QMGR_QUEUE_STAT_SUSPENDED -1 /* voluntary delay timer */
+#define QMGR_QUEUE_STAT_SAVED -2 /* delayed cleanup timer */
+#define QMGR_QUEUE_STAT_BAD -3 /* can't happen */
+
+#define QMGR_QUEUE_READY(q) ((q)->window > 0)
+#define QMGR_QUEUE_THROTTLED(q) ((q)->window == QMGR_QUEUE_STAT_THROTTLED)
+#define QMGR_QUEUE_SUSPENDED(q) ((q)->window == QMGR_QUEUE_STAT_SUSPENDED)
+#define QMGR_QUEUE_SAVED(q) ((q)->window == QMGR_QUEUE_STAT_SAVED)
+#define QMGR_QUEUE_BAD(q) ((q)->window <= QMGR_QUEUE_STAT_BAD)
+
+#define QMGR_QUEUE_STATUS(q) ( \
+ QMGR_QUEUE_READY(q) ? "ready" : \
+ QMGR_QUEUE_THROTTLED(q) ? "throttled" : \
+ QMGR_QUEUE_SUSPENDED(q) ? "suspended" : \
+ QMGR_QUEUE_SAVED(q) ? "saved" : \
+ "invalid queue status" \
+ )
/*
* Structure of one next-hop queue entry. In order to save some copying
if (VSTRING_LEN(dsb->reason) == 0)
vstring_strcpy(dsb->reason, "unknown error");
vstring_prepend(dsb->reason, SUSPENDED, sizeof(SUSPENDED) - 1);
- if (queue->window > 0) {
+ if (QMGR_QUEUE_READY(queue)) {
qmgr_queue_throttle(queue, DSN_FROM_DSN_BUF(dsb));
- if (queue->window == 0)
+ if (QMGR_QUEUE_THROTTLED(queue))
qmgr_defer_todo(queue, &dsb->dsn);
}
}
void qmgr_enable_queue(QMGR_QUEUE *queue)
{
- if (queue->window == 0) {
+ if (QMGR_QUEUE_THROTTLED(queue)) {
if (msg_verbose)
msg_info("enable site %s/%s", queue->transport->name, queue->name);
qmgr_queue_unthrottle(queue);
}
- if (queue->todo.next == 0 && queue->busy.next == 0)
+ if (QMGR_QUEUE_READY(queue) && queue->todo.next == 0 && queue->busy.next == 0)
qmgr_queue_done(queue);
}
QMGR_QUEUE *queue = entry->queue;
/*
- * Move the entry back to the todo lists. In case of the peer list,
- * put it back to the beginning, so the select()/unselect() does
- * not reorder entries. We use this in qmgr_message_assign()
- * to put recipients into existing entries when possible.
+ * Move the entry back to the todo lists. In case of the peer list, put
+ * it back to the beginning, so the select()/unselect() does not reorder
+ * entries. We use this in qmgr_message_assign() to put recipients into
+ * existing entries when possible.
*/
QMGR_LIST_UNLINK(queue->busy, QMGR_ENTRY *, entry, queue_peers);
queue->busy_refcount--;
void qmgr_entry_done(QMGR_ENTRY *entry, int which)
{
+ const char *myname = "qmgr_entry_done";
QMGR_QUEUE *queue = entry->queue;
QMGR_MESSAGE *message = entry->message;
QMGR_PEER *peer = entry->peer;
* Take this entry off the in-core queue.
*/
if (entry->stream != 0)
- msg_panic("qmgr_entry_done: file is open");
+ msg_panic("%s: file is open", myname);
if (which == QMGR_QUEUE_BUSY) {
QMGR_LIST_UNLINK(queue->busy, QMGR_ENTRY *, entry, queue_peers);
queue->busy_refcount--;
QMGR_LIST_UNLINK(queue->todo, QMGR_ENTRY *, entry, queue_peers);
queue->todo_refcount--;
} else {
- msg_panic("qmgr_entry_done: bad queue spec: %d", which);
+ msg_panic("%s: bad queue spec: %d", myname, which);
}
/*
transport->job_current = transport->job_list.next;
transport->candidate_cache_current = 0;
}
- if (queue->window > queue->busy_refcount || queue->window == 0)
+ if (queue->window > queue->busy_refcount || QMGR_QUEUE_THROTTLED(queue))
queue->blocker_tag = 0;
}
/*
* Maintain back-to-back delivery status.
*/
- queue->last_done = event_time();
+ if (which == QMGR_QUEUE_BUSY)
+ queue->last_done = event_time();
+
+ /*
+ * Suspend a rate-limited queue, so that mail trickles out.
+ */
+ if (which == QMGR_QUEUE_BUSY && transport->rate_delay > 0) {
+ if (queue->window > 1)
+ msg_panic("%s: queue %s/%s: window %d > 1 on rate-limited service",
+ myname, transport->name, queue->name, queue->window);
+ if (QMGR_QUEUE_THROTTLED(queue)) /* XXX */
+ qmgr_queue_unthrottle(queue);
+ if (QMGR_QUEUE_READY(queue))
+ qmgr_queue_suspend(queue, transport->rate_delay);
+ }
/*
* When the in-core queue for this site is empty and when this site is
- * not dead, discard the in-core queue. When this site is dead, but the
- * number of in-core queues exceeds some threshold, get rid of this
- * in-core queue anyway, in order to avoid running out of memory.
+ * not dead or suspended, discard the in-core queue. When this site is
+ * dead, but the number of in-core queues exceeds some threshold, get rid
+ * of this in-core queue anyway, in order to avoid running out of memory.
*/
if (queue->todo.next == 0 && queue->busy.next == 0) {
- if (queue->window == 0 && qmgr_queue_count > 2 * var_qmgr_rcpt_limit)
+ if (QMGR_QUEUE_THROTTLED(queue) && qmgr_queue_count > 2 * var_qmgr_rcpt_limit)
qmgr_queue_unthrottle(queue);
- if (queue->window > 0)
+ if (QMGR_QUEUE_READY(queue))
qmgr_queue_done(queue);
}
/*
* Sanity check.
*/
- if (queue->window == 0)
+ if (QMGR_QUEUE_THROTTLED(queue))
msg_panic("qmgr_entry_create: dead queue: %s", queue->name);
/*
/*
/* void qmgr_queue_unthrottle(queue)
/* QMGR_QUEUE *queue;
+/*
+/* void qmgr_queue_suspend(queue, delay)
+/* QMGR_QUEUE *queue;
+/* int delay;
/* DESCRIPTION
/* These routines add/delete/manipulate per-destination queues.
/* Each queue corresponds to a specific transport and destination.
/* provided that it does not exceed the destination concurrency
/* limit specified for the transport. This routine implements
/* "slow open" mode, and eliminates the "thundering herd" problem.
+/*
+/* qmgr_queue_suspend() suspends delivery for this destination
+/* briefly.
/* DIAGNOSTICS
/* Panic: consistency check failure.
/* LICENSE
myname, queue->name, queue->transport->dest_concurrency_limit, \
queue->window, queue->success, queue->failure, queue->fail_cohorts);
+/* qmgr_queue_resume - resume delivery to destination */
+
+static void qmgr_queue_resume(int event, char *context)
+{
+ QMGR_QUEUE *queue = (QMGR_QUEUE *) context;
+ const char *myname = "qmgr_queue_resume";
+
+ /*
+ * Sanity checks.
+ */
+ if (!QMGR_QUEUE_SUSPENDED(queue))
+ msg_panic("%s: bad queue status: %s", myname, QMGR_QUEUE_STATUS(queue));
+
+ /*
+ * We can't simply force delivery on this queue: the transport's pending
+ * count may already be maxed out, and there may be other constraints
+ * that definitely should be none of our business. The best we can do is
+ * to play by the same rules as everyone else: trigger *some* delivery
+ * via qmgr_active_drain() and let round-robin selection work for us.
+ */
+ queue->window = 1;
+ if (queue->todo_refcount > 0)
+ qmgr_active_drain();
+}
+
+/* qmgr_queue_suspend - briefly suspend a destination */
+
+void qmgr_queue_suspend(QMGR_QUEUE *queue, int delay)
+{
+ const char *myname = "qmgr_queue_suspend";
+
+ /*
+ * Sanity checks.
+ */
+ if (!QMGR_QUEUE_READY(queue))
+ msg_panic("%s: bad queue status: %s", myname, QMGR_QUEUE_STATUS(queue));
+ if (queue->busy_refcount > 0)
+ msg_panic("%s: queue is busy", myname);
+
+ /*
+ * Set the queue status to "suspended". No-one is supposed to remove a
+ * queue in suspended state.
+ */
+ queue->window = QMGR_QUEUE_STAT_SUSPENDED;
+ event_request_timer(qmgr_queue_resume, (char *) queue, delay);
+}
+
/* qmgr_queue_unthrottle_wrapper - in case (char *) != (struct *) */
static void qmgr_queue_unthrottle_wrapper(int unused_event, char *context)
* this in-core queue when it is empty and when this site is not dead.
*/
qmgr_queue_unthrottle(queue);
- if (queue->window > 0 && queue->todo.next == 0 && queue->busy.next == 0)
+ if (QMGR_QUEUE_READY(queue) && queue->todo.next == 0 && queue->busy.next == 0)
qmgr_queue_done(queue);
}
if (msg_verbose)
msg_info("%s: queue %s", myname, queue->name);
+ /*
+ * Sanity checks.
+ */
+ if (!QMGR_QUEUE_READY(queue) && !QMGR_QUEUE_THROTTLED(queue))
+ msg_panic("%s: bad queue status: %s", myname, QMGR_QUEUE_STATUS(queue));
+
/*
* Don't restart the negative feedback hysteresis cycle with every
* positive feedback. Restart it only when we make a positive concurrency
/*
* Special case when this site was dead.
*/
- if (queue->window == 0) {
+ if (QMGR_QUEUE_THROTTLED(queue)) {
event_cancel_timer(qmgr_queue_unthrottle_wrapper, (char *) queue);
if (queue->dsn == 0)
msg_panic("%s: queue %s: window 0 status 0", myname, queue->name);
/*
* Sanity checks.
*/
+ if (!QMGR_QUEUE_READY(queue))
+ msg_panic("%s: bad queue status: %s", myname, QMGR_QUEUE_STATUS(queue));
if (queue->dsn)
msg_panic("%s: queue %s: spurious reason %s",
myname, queue->name, queue->dsn->reason);
* This queue is declared dead after a configurable number of
* pseudo-cohort failures.
*/
- if (queue->window > 0) {
+ if (QMGR_QUEUE_READY(queue)) {
queue->fail_cohorts += 1.0 / queue->window;
if (transport->fail_cohort_limit > 0
&& queue->fail_cohorts >= transport->fail_cohort_limit)
* Even after reaching 1, we maintain the negative hysteresis cycle so that
* negative feedback can cancel out positive feedback.
*/
- if (queue->window > 0) {
+ if (QMGR_QUEUE_READY(queue)) {
feedback = QMGR_FEEDBACK_VAL(transport->neg_feedback, queue->window);
QMGR_LOG_FEEDBACK(feedback);
queue->failure -= feedback;
/*
* Special case for a site that just was declared dead.
*/
- if (queue->window == 0) {
+ if (QMGR_QUEUE_THROTTLED(queue)) {
queue->dsn = DSN_COPY(dsn);
event_request_timer(qmgr_queue_unthrottle_wrapper,
(char *) queue, var_min_backoff_time);
queue->busy_refcount + queue->todo_refcount);
if (queue->todo.next || queue->busy.next)
msg_panic("%s: queue not empty: %s", myname, queue->name);
- if (queue->window <= 0)
- msg_panic("%s: window %d", myname, queue->window);
+ if (!QMGR_QUEUE_READY(queue))
+ msg_panic("%s: bad queue status: %s", myname, QMGR_QUEUE_STATUS(queue));
if (queue->dsn)
msg_panic("%s: queue %s: spurious reason %s",
myname, queue->name, queue->dsn->reason);
transport->init_dest_concurrency =
get_mail_conf_int2(name, _INIT_DEST_CON,
var_init_dest_concurrency, 1, 0);
+ transport->rate_delay = get_mail_conf_time2(name, _DEST_RATE_DELAY,
+ var_dest_rate_delay,
+ 's', 0, 0);
+ if (transport->rate_delay > 0)
+ transport->dest_concurrency_limit = 1;
if (transport->dest_concurrency_limit != 0
&& transport->dest_concurrency_limit < transport->init_dest_concurrency)
transport->init_dest_concurrency = transport->dest_concurrency_limit;
projects / thirdparty / postfix.git / commitdiff
