From: Wietse Venema
Date: Tue, 20 Mar 2007 05:00:00 +0000 (-0500)
Subject: postfix-2.4-20070320
X-Git-Tag: v2.4.0-RC1~2
X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=235f9bbe629f9a095c29dc6d230ddac3db7f5494;p=thirdparty%2Fpostfix.git
postfix-2.4-20070320
---
diff --git a/postfix/HISTORY b/postfix/HISTORY
index 34c956f4e..54ec6b0ff 100644
--- a/postfix/HISTORY
+++ b/postfix/HISTORY
@@ -9851,7 +9851,7 @@ Apologies for any names omitted.
Bugfix: the test for "no debugger_command" was wrong.
Leandro Santi. File: global/debugger_command.c.
-20040117
+20041117
Robustness: the master-child protocol now includes a process
generation number besides the child process ID. The process
@@ -13305,8 +13305,83 @@ Apologies for any names omitted.
mailbox dotlock files on all systems, and creates dotlock
files before opening mailbox files. Files: util/sys_defs.h.
+20070301
+
+ Workaround: updated workaround for broken Solaris accept().
+ File: util/inet_listen.c.
+
+ Workaround: on some FreeBSD versions, accept(2) can fail
+ with a bogus EINVAL error. We now allow accept(2) to fail
+ for a limited number of times before terminating the process.
+ Files: master/single_server.c, master/multi_server.c.
+
+20070306
+
+ Bugfix (introduced with Postfix 2.3 Milter support): postdrop
+ reported "illegal seek" instead of "file too large". File:
+ postdrop/postdrop.c.
+
+20070310
+
+ Cleanup: specify "undisclosed_recipients_header =" to disable
+ Postfix's "To: undisclosed-recipients:;" header for mail
+ that lists no recipient. The To: header is not required as
+ of RFC 2822. The undisclosed_recipients_header parameter
+ value can now be an empty string, a value that was not
+ allowed with earlier Postfix versions. With Postfix 2.5 it
+ will be empty by default. Files: cleanup/cleanup.c,
+ cleanup/cleanup_message.c.
+
+20070312
+
+ Backwards compatibility: don't pad short message header
+ records when Milter support is turned off. This maintains
+ compatibility with Postfix versions that pre-date Milter
+ support. File: cleanup/cleanup_out.c.
+
+20070314
+
+ Bitrot: move the "don't run this daemon by hand" message
+ before other tests. Files: master/*server.c.
+
+20070315
+
+ Bitrot: New OpenLDAP APIs deprecate simplified interfaces,
+ that are the only ones available in Sun's LDAP SDK. Define
+ suitable macros that work with new OpenLDAP and Sun's code.
+ Victor Duchovni, Morgan Stanley. File: src/global/dict_ldap.c
+
+ Cleanup: new "leaf" and "terminal" result attributes support
+ fine-tuning of LDAP group expansion, and provide a solution
+ for the problem case where DN recursion returns both the
+ group address and the addresses of the member objects.
+ Victor Duchovni, Morgan Stanley. Files: src/global/dict_ldap.c,
+ proto/LDAP_README.html, proto/ldap_table
+
+20070317
+
+ Idioten Sicherheit: stamp every executable file and every
+ core dump file with "mail_version=xxxxx". Adding version
+ stamps and checks to every IPC message is too much change
+ after code freeze, and requires too much time for testing.
+ File: src/global/mail_version.h and every main program file.
+
+20070320
+
+ Bugfix (introduced between 20070120 and 20070121): the
+ cleanup server stored no "delayed mail warning" queue file
+ records with "sendmail -t", and no header_checks filter/redirect
+ records or content encoding records with other mail. File:
+ global/rec_type.h.
+
Wish list:
+ Bind all deliveries to the same local delivery process,
+ making Postfix perform as poorly as monolithic mailers,
+ but giving a possibility to eliminate duplicate deliveries.
+
+ Maybe declare loop when resolve_local(mxhost) is true?
+
Update message content length when adding/removing headers.
Need scache size limit.
@@ -13319,6 +13394,7 @@ Wish list:
am using now.
Update MILTER_README with Martinec info.
+ http://www.ijs.si/software/amavisd/amavisd-new-docs.html#dkim
Make postcat header/body aware so people can grep headers.
diff --git a/postfix/README_FILES/ADDRESS_VERIFICATION_README b/postfix/README_FILES/ADDRESS_VERIFICATION_README
index 5db4991d1..70948cde9 100644
--- a/postfix/README_FILES/ADDRESS_VERIFICATION_README
+++ b/postfix/README_FILES/ADDRESS_VERIFICATION_README
@@ -5,9 +5,9 @@ PPoossttffiixx AAddddrreessss VVeerriiffiiccaattiioonn
WWAARRNNIINNGG
The sender/recipient address verification feature described in this document is
-suitable only for low-traffic sites. It performs poorly under high load and may
-cause your site to be blacklisted by some providers. See the "Limitations"
-section below for details.
+suitable only for low-traffic sites. It performs poorly under high load;
+excessive sender address verification activity may even cause your site to be
+blacklisted by some providers. See the "Limitations" section below for details.
WWhhaatt PPoossttffiixx aaddddrreessss vveerriiffiiccaattiioonn ccaann ddoo ffoorr yyoouu
diff --git a/postfix/README_FILES/DATABASE_README b/postfix/README_FILES/DATABASE_README
index ef86f651c..c1202d22a 100644
--- a/postfix/README_FILES/DATABASE_README
+++ b/postfix/README_FILES/DATABASE_README
@@ -244,7 +244,7 @@ To find out what database types your Postfix system supports, use the "ppooss
in tcp_table(5). The lookup table name is "tcp:host:port" where "host"
specifies a symbolic hostname or a numeric IP address, and "port"
specifies a symbolic service name or a numeric port number. This
- protocol is not available up to and including Postfix version 2.2.
+ protocol is not available up to and including Postfix version 2.4.
uunniixx (read-only)
A limited way to query the UNIX authentication database. The following
tables are implemented:
diff --git a/postfix/README_FILES/LDAP_README b/postfix/README_FILES/LDAP_README
index 12f31d110..dead25113 100644
--- a/postfix/README_FILES/LDAP_README
+++ b/postfix/README_FILES/LDAP_README
@@ -18,6 +18,7 @@ Topics covered in this document:
* Configuring LDAP lookups
* Example: aliases
* Example: virtual domains/addresses
+ * Example: expanding LDAP groups
* Other uses of LDAP lookups
* Notes and things to think about
* Feedback
@@ -96,12 +97,12 @@ in main.cf, you have:
and in ldap:/etc/postfix/ldap-aliases.cf you have:
- server_host = ldap.my.com
- search_base = dc=my, dc=com
+ server_host = ldap.example.com
+ search_base = dc=example, dc=com
Upon receiving mail for a local address "ldapuser" that isn't found in the /
etc/aliases database, Postfix will search the LDAP server listening at port 389
-on ldap.my.com. It will bind anonymously, search for any directory entries
+on ldap.example.com. It will bind anonymously, search for any directory entries
whose mailacceptinggeneralid attribute is "ldapuser", read the "maildrop"
attributes of those found, and build a list of their maildrops, which will be
treated as RFC822 addresses to which the message will be delivered.
@@ -116,7 +117,7 @@ want to make sure all of your virtual recipient's mailacceptinggeneralid
attributes are fully qualified with their virtual domains. Finally, if you want
to designate a directory entry as the default user for a virtual domain, just
give it an additional mailacceptinggeneralid (or the equivalent in your
-directory) of "@virtual.dom". That's right, no user part. If you don't want a
+directory) of "@fake.dom". That's right, no user part. If you don't want a
catchall user, omit this step and mail to unknown users in the domain will
simply bounce.
@@ -143,12 +144,183 @@ this:
Normal users might simply have one mailacceptinggeneralid and maildrop, e.g.
"normaluser@fake.dom" and "normaluser@real.dom".
+EExxaammppllee:: eexxppaannddiinngg LLDDAAPP ggrroouuppss
+
+LDAP is frequently used to store group member information, and Postfix supports
+expanding a group's email address to the list of email addresses of the group
+members. There are a number of ways of handling LDAP groups, which will be
+illustrated via the mock LDAP entries and implied schema below. This shows two
+group entries "agroup" and "bgroup" and four user entries "auser", "buser",
+"cuser" and "duser". The group "agroup" has the users "auser" (1) and "buser"
+(2) as members via DN references in the multi-valued attribute "memberdn", and
+direct email addresses of two external users "auser@example.org" (3) and
+"buser@example.org" (4) stored in the multi-valued attribute "memberaddr". The
+same is true of "bgroup" and "cuser"/"duser" (6)/(7)/(8)/(9), but "bgroup" also
+has a "maildrop" attribute of "bgroup@mlm.example.com" (5):
+
+ dn: cn=agroup, dc=example, dc=com
+ objectclass: top
+ objectclass: ldapgroup
+ cn: agroup
+ mail: agroup@example.com
+ 1 -> memberdn: uid=auser, dc=example, dc=com
+ 2 -> memberdn: uid=buser, dc=example, dc=com
+ 3 -> memberaddr: auser@example.org
+ 4 -> memberaddr: buser@example.org
+
+ dn: cn=bgroup, dc=example, dc=com
+ objectclass: top
+ objectclass: ldapgroup
+ cn: bgroup
+ mail: bgroup@example.com
+ 5 -> maildrop: bgroup@mlm.example.com
+ 6 -> memberdn: uid=cuser, dc=example, dc=com
+ 7 -> memberdn: uid=duser, dc=example, dc=com
+ 8 -> memberaddr: cuser@example.org
+ 9 -> memberaddr: duser@example.org
+
+ dn: uid=auser, dc=example, dc=com
+ objectclass: top
+ objectclass: ldapuser
+ uid: auser
+ 10 -> mail: auser@example.com
+ 11 -> maildrop: auser@mailhub.example.com
+
+ dn: uid=buser, dc=example, dc=com
+ objectclass: top
+ objectclass: ldapuser
+ uid: buser
+ 12 -> mail: buser@example.com
+ 13 -> maildrop: buser@mailhub.example.com
+
+ dn: uid=cuser, dc=example, dc=com
+ objectclass: top
+ objectclass: ldapuser
+ uid: cuser
+ 14 -> mail: cuser@example.com
+
+ dn: uid=duser, dc=example, dc=com
+ objectclass: top
+ objectclass: ldapuser
+ uid: duser
+ 15 -> mail: duser@example.com
+
+Our first use case ignores the "memberdn" attributes, and assumes that groups
+hold only direct "memberaddr" strings as in (3), (4), (8) and (9). The goal is
+to map the group address to the list of constituent "memberaddr" values. This
+is simple, ignoring the various connection related settings (hosts, ports, bind
+settings, timeouts, ...) we have:
+
+ simple.cf:
+ ...
+ search_base = dc=example, dc=com
+ query_filter = mail=%s
+ result_attribute = memberaddr
+ $ postmap -q agroup@example.com ldap:simple.cf
+ auser@example.org,buser@example.org
+
+We search "dc=example, dc=com". The "mail" attribute is used in the
+query_filter to locate the right group, the "result_attribute" setting
+described in ldap_table(5) is used to specify that "memberaddr" values from the
+matching group are to be returned as a comma separated list. Always check
+tables using postmap(1) with the "-q" option, before deploying them into
+production use in main.cf.
+
+Our second use case also expands "memberdn" attributes (1), (2), (6) and (7),
+follows the DN references and returns the "maildrop" of the referenced user
+entries. Here we use the "special_result_attribute" setting from ldap_table(5)
+to designate the "memberdn" attribute as holding DNs of the desired member
+entries. The "result_attribute" setting selects which attributes are returned
+from the selected DNs. It is important to choose a result attribute that is not
+also present in the group object, because result attributes are collected from
+both the group and the member DNs. In this case we choose "maildrop" and assume
+for the moment that groups never have a "maildrop" (the "bgroup" "maildrop"
+attribute is for a different use case). The returned data for "auser" and
+"buser" is from items (11) and (13) in the mock data.
+
+ special.cf:
+ ...
+ search_base = dc=example, dc=com
+ query_filter = mail=%s
+ result_attribute = memberaddr, maildrop
+ special_result_attribute = memberdn
+ $ postmap -q agroup@example.com ldap:special.cf
+
+ auser@mailhub.example.com,buser@mailhub.example.com,auser@example.org,buser@example.org
+
+Note: if the desired member object result attribute is always also present in
+the group, you get suprising results, the expansion also returns the address of
+the group. This is a known limitation of Postfix releases prior to 2.4, and is
+addressed in the new with Postfix 2.4 "leaf_result_attribute" feature described
+in ldap_table(5).
+
+Our third use case has some groups that are expanded immediately, and other
+groups that are forwarded to a dedicated mailing list manager host for delayed
+expansion. This uses two LDAP tables, one for users and forwarded groups and a
+second for groups that can be expanded immediately. It is assumed that groups
+that require forwarding are never nested members of groups that are directly
+expanded.
+
+ no_expand.cf:
+ ...
+ search_base = dc=example, dc=com
+ query_filter = mail=%s
+ result_attribute = maildrop
+ expand.cf
+ ...
+ search_base = dc=example, dc=com
+ query_filter = mail=%s
+ result_attribute = memberaddr, maildrop
+ special_result_attribute = memberdn
+ $ postmap -q auser@example.com ldap:no_expand.cf ldap:expand.cf
+ auser@mailhub.example.com
+ $ postmap -q agroup@example.com ldap:no_expand.cf ldap:expand.cf
+
+ auser@mailhub.example.com,buser@mailhub.example.com,auser@example.org,buser@example.org
+ $ postmap -q bgroup@example.com ldap:no_expand.cf ldap:expand.cf
+ bgroup@mlm.example.com
+
+Non-group objects and groups with delayed expansion (those that have a maildrop
+attribute) are rewritten to a single maildrop value. Groups that don't have a
+maildrop are expanded as the second use case. This admits a more elegant
+solution with Postfix 2.4 and later.
+
+Our final use case is the same as the third, but this time uses new features in
+Postfix 2.4. We now are able to use just one LDAP table and no longer need to
+assume that forwarded groups are never nested inside expanded groups.
+
+ fancy.cf:
+ ...
+ search_base = dc=example, dc=com
+ query_filter = mail=%s
+ result_attribute = memberaddr
+ special_result_attribute = memberdn
+ terminal_result_attribute = maildrop
+ leaf_result_attribute = mail
+ $ postmap -q auser@example.com ldap:fancy.cf
+ auser@mailhub.example.com
+ $ postmap -q cuser@example.com ldap:fancy.cf
+ cuser@example.com
+ $ postmap -q agroup@example.com ldap:fancy.cf
+
+ auser@mailhub.example.com,buser@mailhub.example.com,auser@example.org,buser@example.org
+ $ postmap -q bgroup@example.com ldap:fancy.cf
+ bgroup@mlm.example.com
+
+Above, delayed expansion is enabled via "terminal_result_attribute", which, if
+present, is used as the sole result and all other expansion is suppressed.
+Otherwise, the "leaf_result_attribute" is only returned for leaf objects that
+don't have a "special_result_attribute" (non-groups), while the
+"result_attribute" (direct member address of groups) is returned at every level
+of recursive expansion, not just the leaf nodes. This fancy example illustrates
+all the features of Postfix 2.4 group expansion.
+
OOtthheerr uusseess ooff LLDDAAPP llooookkuuppss
Other common uses for LDAP lookups include rewriting senders and recipients
with Postfix's canonical lookups, for example in order to make mail leaving
-your site appear to be coming from "First.Last@site.dom" instead of
-"userid@site.dom".
+your site appear to be coming from "First.Last@example.com" instead of
+"userid@example.com".
NNootteess aanndd tthhiinnggss ttoo tthhiinnkk aabboouutt
@@ -166,9 +338,9 @@ NNootteess aanndd tthhiinnggss ttoo tthhiinnkk aabboouu
define an entry intended for use as a mailing list that looks like this
(Warning! Schema made up just for this example):
- dn: cn=Accounting Staff List, dc=my, dc=com
+ dn: cn=Accounting Staff List, dc=example, dc=com
cn: Accounting Staff List
- o: my.com
+ o: example.com
objectclass: maillist
mailacceptinggeneralid: accountingstaff
mailacceptinggeneralid: accounting-staff
diff --git a/postfix/README_FILES/QSHAPE_README b/postfix/README_FILES/QSHAPE_README
index fc04162d5..05c546a87 100644
--- a/postfix/README_FILES/QSHAPE_README
+++ b/postfix/README_FILES/QSHAPE_README
@@ -4,15 +4,11 @@ PPoossttffiixx BBoottttlleenneecckk AAnnaallyyssiiss
PPuurrppoossee ooff tthhiiss ddooccuummeenntt
-This document describes the qshape(1) program which helps the administrator
-understand the Postfix queue message distribution sorted by time and by sender
-or recipient domain. qshape(1) is bundled with the Postfix 2.1 source under the
-"auxiliary" directory.
-
-In order to understand the output of qshape(1), it useful to understand the
-various Postfix queues. To this end the role of each Postfix queue directory is
-described briefly in the "Background info: Postfix queue directories" section
-near the end of this document.
+This document is an introduction to Postfix queue congestion analysis. It
+explains how the qshape(1) program can help to track down the reason for queue
+congestion. qshape(1) is bundled with Postfix 2.1 and later source code, under
+the "auxiliary" directory. This document describes qshape(1) as bundled with
+Postfix 2.4.
This document covers the following topics:
@@ -22,7 +18,7 @@ This document covers the following topics:
* Example 2: Deferred queue full of dictionary attack bounces
* Example 3: Congestion in the active queue
* Example 4: High volume destination backlog
- * Background info: Postfix queue directories
+ * Postfix queue directories
o The "maildrop" queue
o The "hold" queue
@@ -69,14 +65,20 @@ sender domain distribution for captured spam in the "hold" queue:
10 and 20 minutes old, 1 between 320 and 640 minutes old and 12 older than
1280 minutes (1440 minutes in a day).
+When the output is a terminal intermediate results showing the top 20 domains
+(-n option) are displayed after every 1000 messages (-N option) and the final
+output also shows only the top 20 domains. This makes qshape useful even when
+the deferred queue is very large and it may otherwise take prohibitively long
+to read the entire deferred queue.
+
By default, qshape shows statistics for the union of both the incoming and
active queues which are the most relevant queues to look at when analyzing
performance.
One can request an alternate list of queues:
- $ qshape deferred | less
- $ qshape incoming active deferred | less
+ $ qshape deferred
+ $ qshape incoming active deferred
this will show the age distribution of the deferred queue or the union of the
incoming active and deferred queues.
@@ -95,18 +97,19 @@ stopped.
The problem destinations or sender domains appear near the top left corner of
the output table. Remember that the active queue can accommodate up to 20000
-($qmgr_message_active_limit) messages. To check wether this limit has been
+($qmgr_message_active_limit) messages. To check whether this limit has been
reached, use:
- $ qshape -s active | head (show sender statistics)
+ $ qshape -s active (show sender statistics)
If the total sender count is below 20000 the active queue is not yet saturated,
any high volume sender domains show near the top of the output.
-The active queue is also limited to at most 20000 recipient addresses
-($qmgr_message_recipient_limit). To check for exhaustion of this limit use:
+With oqmgr(8) the active queue is also limited to at most 20000 recipient
+addresses ($qmgr_message_recipient_limit). To check for exhaustion of this
+limit use:
- $ qshape active | head (show recipient statistics)
+ $ qshape active (show recipient statistics)
Having found the high volume domains, it is often useful to search the logs for
recent messages pertaining to the domains in question.
@@ -268,7 +271,10 @@ for the queue manager to mark the destination as "dead" despite the transient
nature of the errors. The destination will be retried again after the
expiration of a $minimal_backoff_time timer. If the error bursts are frequent
enough it may be that only a small quantity of email is delivered before the
-destination is again marked "dead".
+destination is again marked "dead". In some cases enabling static (not on
+demand) connection caching by listing the appropriate nexthop domain in a table
+included in "smtp_connection_cache_destinations" may help to reduce the error
+rate, because most messages will re-use existing connections.
The MTA that has been observed most frequently to exhibit such bursts of errors
is Microsoft Exchange, which refuses connections under load. Some proxy virus
@@ -276,14 +282,13 @@ scanners in front of the Exchange server propagate the refused connection to
the client as a "421" error.
Note that it is now possible to configure Postfix to exhibit similarly erratic
-behavior by misconfiguring the anvil(8) server (not included in Postfix 2.1.).
-Do not use anvil(8) for steady-state rate limiting, its purpose is DoS
-prevention and the rate limits set should be very generous!
+behavior by misconfiguring the anvil(8) service. Do not use anvil(8) for
+steady-state rate limiting, its purpose is (unintentional) DoS prevention and
+the rate limits set should be very generous!
-In the long run it is hoped that the Postfix dead host detection and
-concurrency control mechanism will be tuned to be more "noise" tolerant. If one
-finds oneself needing to deliver a high volume of mail to a destination that
-exhibits frequent brief bursts of errors, there is a subtle workaround.
+If one finds oneself needing to deliver a high volume of mail to a destination
+that exhibits frequent brief bursts of errors and connection caching does not
+solve the problem, there is a subtle workaround.
* In master.cf set up a dedicated clone of the "smtp" transport for the
destination in question.
@@ -292,15 +297,15 @@ exhibits frequent brief bursts of errors, there is a subtle workaround.
number in the 10-20 range is typical).
* IMPORTANT!!! In main.cf configure a very large initial and destination
- concurrency limit for this transport (say 200).
+ concurrency limit for this transport (say 2000).
/etc/postfix/main.cf:
- initial_destination_concurrency = 200
- transportname_destination_concurrency_limit = 200
+ initial_destination_concurrency = 2000
+ transportname_destination_concurrency_limit = 2000
Where transportname is the name of the master.cf entry in question.
-The effect of this surprising configuration is that up to 200 consecutive
+The effect of this surprising configuration is that up to 2000 consecutive
errors are tolerated without marking the destination dead, while the total
concurrency remains reasonable (10-20 processes). This trick is only for a very
specialized situation: high volume delivery into a channel with multi-error
@@ -334,7 +339,7 @@ connection caching is introduced.
Hopefully a more elegant solution to these problems will be found in the
future.
-BBaacckkggrroouunndd iinnffoo:: PPoossttffiixx qquueeuuee ddiirreeccttoorriieess
+PPoossttffiixx qquueeuuee ddiirreeccttoorriieess
The following sections describe Postfix queues: their purpose, what normal
behavior looks like, and how to diagnose abnormal behavior.
@@ -355,8 +360,9 @@ the "maildrop" queue and to notify the pickup(8) service of its arrival.
All mail that enters the main Postfix queue does so via the cleanup(8) service.
The cleanup service is responsible for envelope and header rewriting, header
-and body regular expression checks, automatic bcc recipient processing and
-guaranteed insertion of the message into the Postfix "incoming" queue.
+and body regular expression checks, automatic bcc recipient processing, milter
+content processing, and reliable insertion of the message into the Postfix
+"incoming" queue.
In the absence of excessive CPU consumption in cleanup(8) header or body
regular expression checks or other software consuming all available CPU
@@ -372,16 +378,16 @@ not negligible) of the cleanup service.
Congestion in this queue is indicative of an excessive local message submission
rate or perhaps excessive CPU consumption in the cleanup(8) service due to
-excessive body_checks.
+excessive body_checks, or (Postfix >= 2.3) high latency milters.
Note, that once the active queue is full, the cleanup service will attempt to
slow down message injection by pausing $in_flow_delay for each message. In this
case "maildrop" queue congestion may be a consequence of congestion downstream,
rather than a problem in its own right.
-Note also, that one should not attempt to deliver large volumes of mail via the
-pickup(8) service. High volume sites must avoid using content filters that
-reinject scanned mail via Postfix sendmail(1) and postdrop(1).
+Note, you should not attempt to deliver large volumes of mail via the pickup(8)
+service. High volume sites should avoid using "simple" content filters that re-
+inject scanned mail via Postfix sendmail(1) and postdrop(1).
A high arrival rate of locally submitted mail may be an indication of an
uncaught forwarding loop, or a run-away notification program. Try to keep the
@@ -401,13 +407,12 @@ processing and placed indefinitely in the "hold" queue. Messages placed in the
delivery attempts are made for messages in the "hold" queue. The postsuper(1)
command can be used to manually release messages into the "deferred" queue.
-Messages can potentially stay in the "hold" queue for a time exceeding the
-normal maximal queue lifetime (after which undelivered messages are bounced
-back to the sender). If such "old" messages need to be released from the "hold"
-queue, they should typically be moved into the "maildrop" queue, so that the
-message gets a new timestamp and is given more than one opportunity to be
-delivered. Messages that are "young" can be moved directly into the "deferred"
-queue.
+Messages can potentially stay in the "hold" queue longer than
+$maximal_queue_lifetime. If such "old" messages need to be released from the
+"hold" queue, they should typically be moved into the "maildrop" queue using
+"postsuper -r", so that the message gets a new timestamp and is given more than
+one opportunity to be delivered. Messages that are "young" can be moved
+directly into the "deferred" queue using "postsuper -H".
The "hold" queue plays little role in Postfix performance, and monitoring of
the "hold" queue is typically more closely motivated by tracking spam and
@@ -435,10 +440,14 @@ queue as soon as they become available.
The incoming queue grows when the message input rate spikes above the rate at
which the queue manager can import messages into the active queue. The main
-factor slowing down the queue manager is transport queries to the trivial-
-rewrite service. If the queue manager is routinely not keeping up, consider not
-using "slow" lookup services (MySQL, LDAP, ...) for transport lookups or
-speeding up the hosts that provide the lookup service.
+factors slowing down the queue manager are disk I/O and lookup queries to the
+trivial-rewrite service. If the queue manager is routinely not keeping up,
+consider not using "slow" lookup services (MySQL, LDAP, ...) for transport
+lookups or speeding up the hosts that provide the lookup service. If the
+problem is I/O starvation, consider striping the queue over more disks, faster
+controllers with a battery write cache, or other hardware improvements. At the
+very least, make sure that the queue directory is mounted with the "noatime"
+option if applicable to the underlying filesystem.
The in_flow_delay parameter is used to clamp the input rate when the queue
manager starts to fall behind. The cleanup(8) service will pause for
@@ -484,22 +493,37 @@ of recipients that share the same transport/nexthop combination; the group size
is capped by the transport's recipient concurrency limit.
Multiple recipient groups (from one or more messages) are queued for delivery
-via the common transport/nexthop combination. The destination concurrency limit
-for the transports caps the number of simultaneous delivery attempts for each
-nexthop. Transports with a recipient concurrency limit of 1 are special: these
-are grouped by the actual recipient address rather than the nexthop, thereby
-enabling per-recipient concurrency limits rather than per-domain concurrency
-limits. Per-recipient limits are appropriate when performing final delivery to
+grouped by transport/nexthop combination. The ddeessttiinnaattiioonn concurrency limit for
+the transports caps the number of simultaneous delivery attempts for each
+nexthop. Transports with a rreecciippiieenntt concurrency limit of 1 are special: these
+are grouped by the actual recipient address rather than the nexthop, yielding
+per-recipient concurrency limits rather than per-domain concurrency limits.
+Per-recipient limits are appropriate when performing final delivery to
mailboxes rather than when relaying to a remote server.
Congestion occurs in the active queue when one or more destinations drain
-slower than the corresponding message input rate. If a destination is down for
-some time, the queue manager will mark it dead, and immediately defer all mail
-for the destination without trying to assign it to a delivery agent. In this
-case the messages will quickly leave the active queue and end up in the
-deferred queue. If the destination is instead simply slow, or there is a
-problem causing an excessive arrival rate the active queue will grow and will
-become dominated by mail to the congested destination.
+slower than the corresponding message input rate.
+
+Input into the active queue comes both from new mail in the "incoming" queue,
+and retries of mail in the "deferred" queue. Should the "deferred" queue get
+really large, retries of old mail can dominate the arrival rate of new mail.
+Systems with more CPU, faster disks and more network bandwidth can deal with
+larger deferred queues, but as a rule of thumb the deferred queue scales to
+somewhere between 100,000 and 1,000,000 messages with good performance unlikely
+above that "limit". Systems with queues this large should typically stop
+accepting new mail, or put the backlog "on hold" until the underlying issue is
+fixed (provided that there is enough capacity to handle just the new mail).
+
+When a destination is down for some time, the queue manager will mark it dead,
+and immediately defer all mail for the destination without trying to assign it
+to a delivery agent. In this case the messages will quickly leave the active
+queue and end up in the deferred queue (with Postfix < 2.4, this is done
+directly by the queue manager, with Postfix >= 2.4 this is done via the "retry"
+delivery agent).
+
+When the destination is instead simply slow, or there is a problem causing an
+excessive arrival rate the active queue will grow and will become dominated by
+mail to the congested destination.
The only way to reduce congestion is to either reduce the input rate or
increase the throughput. Increasing the throughput requires either increasing
@@ -523,23 +547,52 @@ ESTABLISHED or SYN_SENT) reaches the process limit, mail is draining slowly and
the system and network are not loaded, raise the "smtp" and/or "relay" process
limits!
-Especially for the "relay" transport, consider lower SMTP connection timeouts
-(1-5 seconds) and higher than default destination concurrency limits. Compute
-the expected latency when 1 out of N of the MX hosts for a high volume site is
-down and not responding, and make sure that the configured concurrency divided
-by this latency exceeds the required steady-state message rate. If the
-destination is managed by you, consider load balancers in front of groups of MX
-hosts. Load balancers have higher uptime and will be able to hide individual MX
-host failures.
-
-If necessary, dedicate and tune custom transports for high volume destinations.
+When a high volume destination is served by multiple MX hosts with typically
+low delivery latency, performance can suffer dramatically when one of the MX
+hosts is unresponsive and SMTP connections to that host timeout. For example,
+if there are 2 equal weight MX hosts, the SMTP connection timeout is 30 seconds
+and one of the MX hosts is down, the average SMTP connection will take
+approximately 15 seconds to complete. With a default per-destination
+concurrency limit of 20 connections, throughput falls to just over 1 message
+per second.
+
+The best way to avoid bottlenecks when one or more MX hosts is non-responsive
+is to use connection caching. Connection caching was introduced with Postfix
+2.2 and is by default enabled on demand for destinations with a backlog of mail
+in the active queue. When connection caching is in effect for a particular
+destination, established connections are re-used to send additional messages,
+this reduces the number of connections made per message delivery and maintains
+good throughput even in the face of partial unavailability of the destination's
+MX hosts.
+
+If connection caching is not available (Postfix < 2.2) or does not provide a
+sufficient latency reduction, especially for the "relay" transport used to
+forward mail to "your own" domains, consider setting lower than default SMTP
+connection timeouts (1-5 seconds) and higher than default destination
+concurrency limits. This will further reduce latency and provide more
+concurrency to maintain throughput should latency rise.
+
+Setting high concurrency limits to domains that are not your own may be viewed
+as hostile by the receiving system, and steps may be taken to prevent you from
+monopolizing the destination system's resources. The defensive measures may
+substantially reduce your throughput or block access entirely. Do not set
+aggressive concurrency limits to remote domains without coordinating with the
+administrators of the target domain.
+
+If necessary, dedicate and tune custom transports for selected high volume
+destinations. The "relay" transport is provided for forwarding mail to domains
+for which your server is a primary or backup MX host. These can make up a
+substantial fraction of your email traffic. Use the "relay" and not the "smtp"
+transport to send email to these domains. Using the "relay" transport allocates
+a separate delivery agent pool to these destinations and allows separate tuning
+of timeouts and concurrency limits.
Another common cause of congestion is unwarranted flushing of the entire
deferred queue. The deferred queue holds messages that are likely to fail to be
-delivered and are also likely to be slow to fail delivery (timeouts). This
-means that the most common reaction to a large deferred queue (flush it!) is
-more than likely counter- productive, and is likely to make the problem worse.
-Do not flush the deferred queue unless you expect that most of its content has
+delivered and are also likely to be slow to fail delivery (time out). As a
+result the most common reaction to a large deferred queue (flush it!) is more
+than likely counter-productive, and typically makes the congestion worse. Do
+not flush the deferred queue unless you expect that most of its content has
recently become deliverable (e.g. relayhost back up after an outage)!
Note that whenever the queue manager is restarted, there may already be
@@ -549,7 +602,8 @@ the active queue messages back into the incoming queue, and then uses its
normal incoming queue scan to refill the active queue. The process of moving
all the messages back and forth, redoing transport table (trivial-rewrite(8)
resolve service) lookups, and re-importing the messages back into memory is
-expensive. At all costs, avoid frequent restarts of the queue manager.
+expensive. At all costs, avoid frequent restarts of the queue manager (e.g. via
+frequent execution of "postfix reload").
TThhee ""ddeeffeerrrreedd"" qquueeuuee
@@ -561,15 +615,15 @@ The queue manager scans the deferred queue periodically. The scan interval is
controlled by the queue_run_delay parameter. While a deferred queue scan is in
progress, if an incoming queue scan is also in progress (ideally these are
brief since the incoming queue should be short), the queue manager alternates
-between bringing a new "incoming" message and a new "deferred" message into the
+between looking for messages in the "incoming" queue and in the "deferred"
queue. This "round-robin" strategy prevents starvation of either the incoming
or the deferred queues.
Each deferred queue scan only brings a fraction of the deferred queue back into
the active queue for a retry. This is because each message in the deferred
queue is assigned a "cool-off" time when it is deferred. This is done by time-
-warping the modification times of the queue file into the future. The queue
-file is not eligible for a retry if its modification time is not yet reached.
+warping the modification time of the queue file into the future. The queue file
+is not eligible for a retry if its modification time is not yet reached.
The "cool-off" time is at least $minimal_backoff_time and at most
$maximal_backoff_time. The next retry time is set by doubling the message's age
@@ -578,24 +632,29 @@ that young messages are initially retried more often than old messages.
If a high volume site routinely has large deferred queues, it may be useful to
adjust the queue_run_delay, minimal_backoff_time and maximal_backoff_time to
-provide short enough delays on first failure, with perhaps longer delays after
-multiple failures, to reduce the retransmission rate of old messages and
-thereby reduce the quantity of previously deferred mail in the active queue.
+provide short enough delays on first failure (Postfix >= 2.4 has a sensibly low
+minimal backoff time by default), with perhaps longer delays after multiple
+failures, to reduce the retransmission rate of old messages and thereby reduce
+the quantity of previously deferred mail in the active queue. If you want a
+really low minimal_backoff_time, you may also want to lower queue_run_delay,
+but understand that more frequent scans will increase the demand for disk I/O.
One common cause of large deferred queues is failure to validate recipients at
the SMTP input stage. Since spammers routinely launch dictionary attacks from
unrepliable sender addresses, the bounces for invalid recipient addresses clog
the deferred queue (and at high volumes proportionally clog the active queue).
Recipient validation is strongly recommended through use of the
-local_recipient_maps and relay_recipient_maps parameters.
+local_recipient_maps and relay_recipient_maps parameters. Even when bounces
+drain quickly they inundate innocent victims of forgery with unwanted email. To
+avoid this, do not accept mail for invalid recipients.
When a host with lots of deferred mail is down for some time, it is possible
for the entire deferred queue to reach its retry time simultaneously. This can
lead to a very full active queue once the host comes back up. The phenomenon
can repeat approximately every maximal_backoff_time seconds if the messages are
-again deferred after a brief burst of congestion. Ideally, in the future
-Postfix will add a random offset to the retry time (or use a combination of
-strategies) to reduce the chances of repeated complete deferred queue flushes.
+again deferred after a brief burst of congestion. Perhaps, a future Postfix
+release will add a random offset to the retry time (or use a combination of
+strategies) to reduce the odds of repeated complete deferred queue flushes.
CCrreeddiittss
diff --git a/postfix/README_FILES/SASL_README b/postfix/README_FILES/SASL_README
index 5cf940db4..ee39ad617 100644
--- a/postfix/README_FILES/SASL_README
+++ b/postfix/README_FILES/SASL_README
@@ -86,18 +86,18 @@ Notes:
BBuuiillddiinngg tthhee CCyyrruuss SSAASSLL lliibbrraarryy
-Postfix appears to work with cyrus-sasl-1.5.5 or cyrus-sasl-2.1.1, which are
+Postfix appears to work with cyrus-sasl-1.5.x or cyrus-sasl-2.1.x, which are
available from:
ftp://ftp.andrew.cmu.edu/pub/cyrus-mail/
IMPORTANT: if you install the Cyrus SASL libraries as per the default, you will
-have to symlink /usr/lib/sasl -> /usr/local/lib/sasl for version 1.5.5 or /usr/
-lib/sasl2 -> /usr/local/lib/sasl2 for version 2.1.1.
+have to symlink /usr/lib/sasl -> /usr/local/lib/sasl for version 1.5.x or /usr/
+lib/sasl2 -> /usr/local/lib/sasl2 for version 2.1.x.
-Reportedly, Microsoft Internet Explorer version 5 requires the non-standard
-SASL LOGIN authentication method. To enable this authentication method, specify
-``./configure --enable-login''.
+Reportedly, Microsoft Outlook (Express) requires the non-standard LOGIN
+authentication method. To enable this authentication method, specify ``./
+configure --enable-login''.
BBuuiillddiinngg PPoossttffiixx wwiitthh CCyyrruuss SSAASSLL ssuuppppoorrtt
@@ -106,13 +106,13 @@ include, and that the Cyrus SASL libraries are in /usr/local/lib.
On some systems this generates the necessary Makefile definitions:
-(for Cyrus SASL version 1.5.5):
+(for Cyrus SASL version 1.5.x):
% make tidy # if you have left-over files from a previous build
% make makefiles CCARGS="-DUSE_SASL_AUTH -DUSE_CYRUS_SASL \
-I/usr/local/include" AUXLIBS="-L/usr/local/lib -lsasl"
-(for Cyrus SASL version 2.1.1):
+(for Cyrus SASL version 2.1.x):
% make tidy # if you have left-over files from a previous build
% make makefiles CCARGS="-DUSE_SASL_AUTH -DUSE_CYRUS_SASL \
@@ -121,14 +121,14 @@ On some systems this generates the necessary Makefile definitions:
On Solaris 2.x you need to specify run-time link information, otherwise ld.so
will not find the SASL shared library:
-(for Cyrus SASL version 1.5.5):
+(for Cyrus SASL version 1.5.x):
% make tidy # if you have left-over files from a previous build
% make makefiles CCARGS="-DUSE_SASL_AUTH -DUSE_CYRUS_SASL \
-I/usr/local/include" AUXLIBS="-L/usr/local/lib \
-R/usr/local/lib -lsasl"
-(for Cyrus SASL version 2.1.1):
+(for Cyrus SASL version 2.1.x):
% make tidy # if you have left-over files from a previous build
% make makefiles CCARGS="-DUSE_SASL_AUTH -DUSE_CYRUS_SASL \
@@ -158,8 +158,8 @@ Note: the SASL login names will be shared with the entire world.
Older Microsoft SMTP client software implements a non-standard version of the
AUTH protocol syntax, and expects that the SMTP server replies to EHLO with
-"250 AUTH=stuff" instead of "250 AUTH stuff". To accommodate such clients (in
-addition to conformant clients) use the following:
+"250 AUTH=mechanism-list" instead of "250 AUTH mechanism-list". To accommodate
+such clients (in addition to conformant clients) use the following:
/etc/postfix/main.cf:
broken_sasl_auth_clients = yes
@@ -201,71 +201,83 @@ authentication server.
CCyyrruuss SSAASSLL ccoonnffiigguurraattiioonn ffoorr tthhee PPoossttffiixx SSMMTTPP sseerrvveerr
-In /usr/local/lib/sasl/smtpd.conf (Cyrus SASL version 1.5.5) or /usr/local/lib/
-sasl2/smtpd.conf (Cyrus SASL version 2.1.1) you need to specify how the server
-should validate client passwords.
+You need to configure how the Cyrus SASL library should authenticate a client's
+username and password. These settings must be stored in a separate
+configuration file.
+
+The name of the configuration file (default: smtpd.conf) will be constructed
+from a value sent by Postfix to the Cyrus SASL library, which adds the suffix
+.conf. The value is configured using one of the following variables:
+
+ /etc/postfix/main.cf:
+ # Postfix 2.3 and later
+ smtpd_sasl_path = smtpd
+ # Postfix < 2.3
+ smtpd_sasl_application_name = smtpd
+
+Cyrus SASL searches for the configuration file in /usr/local/lib/sasl/ (Cyrus
+SASL version 1.5.5) or /usr/local/lib/sasl2/ (Cyrus SASL version 2.1.x).
Note: some Postfix distributions are modified and look for the smtpd.conf file
-in /etc/postfix.
+in /etc/postfix/sasl.
Note: some Cyrus SASL distributions look for the smtpd.conf file in /etc/sasl2.
- * To authenticate against the UNIX password database, try:
+ * To authenticate against the UNIX password database, use:
- (Cyrus SASL version 1.5.5)
+ (Cyrus SASL version 1.5.x)
/usr/local/lib/sasl/smtpd.conf:
pwcheck_method: pwcheck
- (Cyrus SASL version 2.1.1)
-
- /usr/local/lib/sasl2/smtpd.conf:
- pwcheck_method: pwcheck
-
- The name of the file in /usr/local/lib/sasl (Cyrus SASL version 1.5.5) or /
- usr/local/lib/sasl2 (Cyrus SASL version 2.1.1) used by the SASL library for
- configuration can be set with:
-
- /etc/postfix/main.cf:
- smtpd_sasl_application_name = smtpd (Postfix < 2.3)
- smtpd_sasl_path = smtpd (Postfix 2.3 and later)
-
- The pwcheck daemon is contained in the cyrus-sasl source tarball.
-
- IMPORTANT: postfix processes need to have group read+execute permission for
- the /var/pwcheck directory, otherwise authentication attempts will fail.
+ IMPORTANT: pwcheck establishes a UNIX domain socket in /var/pwcheck and
+ waits for authentication requests. Postfix processes must have
+ read+execute permission to this directory or authentication attempts
+ will fail.
- * Alternately, in Cyrus SASL 1.5.26 and later (including 2.1.1), try:
+ The pwcheck daemon is contained in the cyrus-sasl source tarball.
(Cyrus SASL version 1.5.26)
/usr/local/lib/sasl/smtpd.conf:
pwcheck_method: saslauthd
- (Cyrus SASL version 2.1.1)
+ (Cyrus SASL version 2.1.x)
/usr/local/lib/sasl2/smtpd.conf:
pwcheck_method: saslauthd
+ mech_list: PLAIN LOGIN
The saslauthd daemon is also contained in the cyrus-sasl source tarball. It
is more flexible than the pwcheck daemon, in that it can authenticate
against PAM and various other sources. To use PAM, start saslauthd with "-
a pam".
+ IMPORTANT: saslauthd usually establishes a UNIX domain socket in /var/run/
+ saslauthd and waits for authentication requests. Postfix processes must
+ have read+execute permission to this directory or authentication attempts
+ will fail.
+
+ Note: The directory where saslauthd puts the socket is configurable. See
+ the command-line option "-m /path/to/socket" in the saslauthd --help
+ listing.
+
* To authenticate against Cyrus SASL's own password database:
- (Cyrus SASL version 1.5.5)
+ (Cyrus SASL version 1.5.x)
/usr/local/lib/sasl/smtpd.conf:
- pwcheck_method: sasldb
+ pwcheck_method: sasldb
- (Cyrus SASL version 2.1.1)
+ (Cyrus SASL version 2.1.x)
/usr/local/lib/sasl2/smtpd.conf:
- pwcheck_method: auxprop
+ pwcheck_method: auxprop
+ auxprop_plugin: sasldb
+ mech_list: PLAIN LOGIN CRAM-MD5 DIGEST-MD5
This will use the Cyrus SASL password file (default: /etc/sasldb in version
- 1.5.5, or /etc/sasldb2 in version 2.1.1), which is maintained with the
+ 1.5.x, or /etc/sasldb2 in version 2.1.x), which is maintained with the
saslpasswd or saslpasswd2 command (part of the Cyrus SASL software). On
some poorly-supported systems the saslpasswd command needs to be run
multiple times before it stops complaining. The Postfix SMTP server needs
@@ -279,17 +291,17 @@ Note: some Cyrus SASL distributions look for the smtpd.conf file in /etc/sasl2.
EXAMPLE:
- (Cyrus SASL version 1.5.5)
+ (Cyrus SASL version 1.5.x)
% saslpasswd -c -u `postconf -h myhostname` exampleuser
- (Cyrus SASL version 2.1.1)
+ (Cyrus SASL version 2.1.x)
% saslpasswd2 -c -u `postconf -h myhostname` exampleuser
You can find out SASL's idea about the realms of the users in sasldb with
- sasldblistusers (Cyrus SASL version 1.5.5) or sasldblistusers2 (Cyrus SASL
- version 2.1.1).
+ sasldblistusers (Cyrus SASL version 1.5.x) or sasldblistusers2 (Cyrus SASL
+ version 2.1.x).
On the Postfix side, you can have only one realm per smtpd instance, and
only the users belonging to that realm would be able to authenticate. The
@@ -298,21 +310,19 @@ Note: some Cyrus SASL distributions look for the smtpd.conf file in /etc/sasl2.
/etc/postfix/main.cf:
smtpd_sasl_local_domain = $myhostname
-IMPORTANT: all users must be able to authenticate using ALL authentication
-mechanisms advertised by Postfix, otherwise the negotiation might end up with
-an unsupported mechanism, and authentication would fail. For example if you
-configure SASL to use saslauthd for authentication against PAM (pluggable
-authentication modules), only the PLAIN and LOGIN mechanisms are supported and
-stand a chance to succeed, yet the SASL library would also advertise other
-mechanisms, such as DIGEST-MD5. This happens because those mechanisms are made
-available by other plugins, and the SASL library have no way to know that your
-only valid authentication source is PAM. Thus you might need to limit the list
-of mechanisms advertised by Postfix.
+IMPORTANT: The Cyrus SASL password verification services pwcheck and saslauthd
+can only support the plaintext mechanisms PLAIN or LOGIN. However, the Cyrus
+SASL library doesn't know this, and will happily advertise other authentication
+mechanisms that the SASL library implements, such as DIGEST-MD5. As a result,
+if an SMTP client chooses any mechanism other than PLAIN or LOGIN while pwcheck
+or saslauthd are used, authentication will fail. Thus you may need to limit the
+list of mechanisms advertised by Postfix.
* With older Cyrus SASL versions you remove the corresponding library files
from the SASL plug-in directory (and again whenever the system is updated).
- * With Cyrus SASL version 2.1.1 or later:
+ * With Cyrus SASL version 2.1.x or later the mech_list variable can specify a
+ list of authentication mechanisms that Cyrus SASL may offer:
/usr/local/lib/sasl2/smtpd.conf:
mech_list: plain login
@@ -320,14 +330,14 @@ of mechanisms advertised by Postfix.
For the same reasons you might want to limit the list of plugins used for
authentication.
- * With Cyrus SASL version 1.5.5 your only choice is to delete the
+ * With Cyrus SASL version 1.5.x your only choice is to delete the
corresponding library files from the SASL plug-in directory.
- * With SASL version 2.1.1:
+ * With SASL version 2.1.x:
/usr/local/lib/sasl2/smtpd.conf:
- pwcheck_method: auxprop
- auxprop_plugin: sql
+ pwcheck_method: auxprop
+ auxprop_plugin: sql
To run software chrooted with SASL support is an interesting exercise. It
probably is not worth the trouble.
@@ -371,15 +381,17 @@ base64-encoded form.
TTrroouubbllee sshhoooottiinngg tthhee SSAASSLL iinntteerrnnaallss
In the Cyrus SASL sources you'll find a subdirectory named "sample". Run make
-there, "su" to the user postfix (or whatever your mail_owner directive is set
-to):
+there, then create a symbolic link from sample.conf to smtpd.conf in your Cyrus
+SASL library directory /usr/local/lib/sasl2. "su" to the user postfix (or
+whatever your mail_owner directive is set to):
% su postfix
-then run the resulting sample server and client in separate terminals. Strace /
-ktrace / truss the server to see what makes it unhappy, and fix the problem.
-Repeat the previous step until you can successfully authenticate with the
-sample client. Only then get back to Postfix.
+then run the resulting sample server and client in separate terminals. The
+sample applications send log messages to the syslog facility auth. Check the
+log to fix the problem or run strace / ktrace / truss on the server to see what
+makes it unhappy. Repeat the previous step until you can successfully
+authenticate with the sample client. Only then get back to Postfix.
EEnnaabblliinngg SSAASSLL aauutthheennttiiccaattiioonn iinn tthhee PPoossttffiixx SSMMTTPP cclliieenntt
@@ -401,6 +413,11 @@ that is specified with the relayhost parameter or with a transport(5) table.
[mail.myisp.net] username:password
[mail.myisp.net]:submission username:password
+The Postfix SASL client password file is opened before the SMTP server enters
+the optional chroot jail, so you can keep the file in /etc/postfix and set
+permissions read / write only for root to keep the username:password
+combinations away from other system users.
+
Postfix version 2.3 supports-per-sender SASL password information. To search
the Postfix SASL password by sender before it searches by destination, specify:
@@ -424,9 +441,6 @@ plaintext authentication specify, for example:
/etc/postfix/main.cf:
smtp_sasl_security_options = noanonymous
-The Postfix SASL client password file is opened before the SMTP server enters
-the optional chroot jail, so you can keep the file in /etc/postfix.
-
Note: Some SMTP servers support authentication mechanisms that, although
available on the client system, may not in practice work or possess the
appropriate credentials to authenticate to the server. It is possible via the
@@ -437,7 +451,7 @@ mechanisms that the smtp(8) client will take into consideration:
smtp_sasl_mechanism_filter = !gssapi, !external, static:all
In the above example, Postfix will decline to use mechanisms that require
-special infrastructure such as Kerberos.
+special infrastructure such as Kerberos or TLS.
The Postfix SMTP client is backwards compatible with SMTP servers that use the
non-standard "AUTH=method..." syntax in response to the EHLO command; there is
@@ -458,4 +472,6 @@ CCrreeddiittss
smtpd_sasl_path.
* The Dovecot SMTP server-only plug-in was originally implemented by Timo
Sirainen of Procontrol, Finland.
+ * Patrick Ben Koetter revised this document for Postfix 2.4 and made much
+ needed updates.
diff --git a/postfix/README_FILES/SMTPD_POLICY_README b/postfix/README_FILES/SMTPD_POLICY_README
index 6ce119b84..7735aaeaa 100644
--- a/postfix/README_FILES/SMTPD_POLICY_README
+++ b/postfix/README_FILES/SMTPD_POLICY_README
@@ -10,9 +10,11 @@ policy decisions to an external server that runs outside Postfix.
With this policy delegation mechanism, a simple greylist policy can be
implemented with only a dozen lines of Perl, as is shown at the end of this
-document. Another example of policy delegation is the SPF policy server by Meng
-Wong at http://spf.pobox.com/. Examples of both policies can be found in the
-Postfix source code, in the directory examples/smtpd-policy.
+document. A complete example can be found in the Postfix source code, in the
+directory examples/smtpd-policy.
+
+Another example of policy delegation is the SPF policy server at http://
+www.openspf.org/Software.
Policy delegation is now the preferred method for adding policies to Postfix.
It's much easier to develop a new feature in few lines of Perl, than trying to
diff --git a/postfix/README_FILES/TLS_LEGACY_README b/postfix/README_FILES/TLS_LEGACY_README
index c534beb48..78d128521 100644
--- a/postfix/README_FILES/TLS_LEGACY_README
+++ b/postfix/README_FILES/TLS_LEGACY_README
@@ -558,7 +558,7 @@ Their DSA counterparts:
/etc/postfix/main.cf:
smtp_tls_dcert_file = /etc/postfix/client-dsa.pem
- smtp_tls_dkey_file = $smtpd_tls_cert_file
+ smtp_tls_dkey_file = $smtp_tls_dcert_file
To verify a remote SMTP server certificate, the Postfix SMTP client needs to
trust the certificates of the issuing certification authorities. These
@@ -578,7 +578,7 @@ files in the directory when the information is needed. Thus, the
$smtp_tls_CApath directory needs to be accessible inside the optional chroot
jail.
-The choice between $smtp_tls_CAfile and $smtpd_tls_CApath is a space/time
+The choice between $smtp_tls_CAfile and $smtp_tls_CApath is a space/time
tradeoff. If there are many trusted CAs, the cost of preloading them all into
memory may not pay off in reduced access time when the certificate is needed.
diff --git a/postfix/README_FILES/TLS_README b/postfix/README_FILES/TLS_README
index dc12a8ff7..d8fbc02be 100644
--- a/postfix/README_FILES/TLS_README
+++ b/postfix/README_FILES/TLS_README
@@ -673,7 +673,7 @@ Their DSA counterparts:
/etc/postfix/main.cf:
smtp_tls_dcert_file = /etc/postfix/client-dsa.pem
- smtp_tls_dkey_file = $smtpd_tls_cert_file
+ smtp_tls_dkey_file = $smtp_tls_dcert_file
To verify a remote SMTP server certificate, the Postfix SMTP client needs to
trust the certificates of the issuing certification authorities. These
@@ -693,7 +693,7 @@ files in the directory when the information is needed. Thus, the
$smtp_tls_CApath directory needs to be accessible inside the optional chroot
jail.
-The choice between $smtp_tls_CAfile and $smtpd_tls_CApath is a space/time
+The choice between $smtp_tls_CAfile and $smtp_tls_CApath is a space/time
tradeoff. If there are many trusted CAs, the cost of preloading them all into
memory may not pay off in reduced access time when the certificate is needed.
diff --git a/postfix/README_FILES/TUNING_README b/postfix/README_FILES/TUNING_README
index c544ffc2a..000f98d21 100644
--- a/postfix/README_FILES/TUNING_README
+++ b/postfix/README_FILES/TUNING_README
@@ -141,7 +141,7 @@ Postfix version 2.0 and earlier:
MMeeaassuurreess aaggaaiinnsstt cclliieennttss tthhaatt mmaakkee ttoooo mmaannyy ccoonnnneeccttiioonnss
-Note: this feature is not included with Postfix version 2.1.
+Note: the anvil(8) service was introduced with Postfix version 2.2.
The Postfix smtpd(8) server can limit the number of simultaneous connections
from the same SMTP client, as well as the number of connections that a client
diff --git a/postfix/README_FILES/VIRTUAL_README b/postfix/README_FILES/VIRTUAL_README
index 9ee405dbd..5b47482f7 100644
--- a/postfix/README_FILES/VIRTUAL_README
+++ b/postfix/README_FILES/VIRTUAL_README
@@ -316,12 +316,14 @@ Notes:
NEVER list a virtual MAILBOX domain name as a virtual ALIAS domain!
* Lines 4, 7-13: The virtual_mailbox_maps parameter specifies the lookup
- table with all valid recipient addresses. The lookup result is ignored by
- Postfix. In the above example, info@example.com and sales@example.com are
- listed as valid addresses, and mail for anything else is rejected with
- "User unknown". If you intend to use LDAP, MySQL or PgSQL instead of local
- files, be sure to review the "local files versus databases" section at the
- top of this document!
+ table with all valid recipient addresses. The lookup result value is
+ ignored by Postfix. In the above example, info@example.com and
+ sales@example.com are listed as valid addresses; other mail for example.com
+ is rejected with "User unknown" by the Postfix SMTP server. It's left up to
+ the non-Postfix delivery agent to reject non-existent recipients from local
+ submission or from local alias expansion. If you intend to use LDAP, MySQL
+ or PgSQL instead of local files, be sure to review the "local files versus
+ databases" section at the top of this document!
* Line 12: The commented out entry (text after #) shows how one would inform
Postfix of the existence of a catch-all address. Again, the lookup result
diff --git a/postfix/auxiliary/qshape/qshape.pl b/postfix/auxiliary/qshape/qshape.pl
index 67d378984..243df445a 100644
--- a/postfix/auxiliary/qshape/qshape.pl
+++ b/postfix/auxiliary/qshape/qshape.pl
@@ -13,6 +13,7 @@
# \fBqshape\fR [\fB-s\fR] [\fB-p\fR] [\fB-m \fImin_subdomains\fR]
# [\fB-b \fIbucket_count\fR] [\fB-t \fIbucket_time\fR]
# [\fB-l\fR] [\fB-w \fIterminal_width\fR]
+# [\fB-N \fIbatch_msg_count\fR] [\fB-n \fIbatch_top_domains\fR]
# [\fB-c \fIconfig_directory\fR] [\fIqueue_name\fR ...]
# DESCRIPTION
# The \fBqshape\fR program helps the administrator understand the
@@ -63,6 +64,15 @@
# parent domain rows are shown as '.+' followed by the last 16 bytes
# of the domain name. If this is still too narrow to show the domain
# name and all the counters, the terminal_width limit is violated.
+# .IP "\fB-N \fIbatch_msg_count\fR"
+# When the output device is a terminal, intermediate results are
+# shown each "batch_msg_count" messages. This produces usable results
+# in a reasonable time even when the deferred queue is large. The
+# default is to show intermediate results every 1000 messages.
+# .IP "\fB-n \fIbatch_top_domains\fR"
+# When reporting intermediate or final results to a termainal, report
+# only the top "batch_top_domains" domains. The default limit is 20
+# domains.
# .IP "\fB-c \fIconfig_directory\fR"
# The \fBmain.cf\fR configuration file is in the named directory
# instead of the default configuration directory.
@@ -104,6 +114,9 @@ use IO::File;
use File::Find;
use Getopt::Std;
+my $cls; # Clear screen escape sequence
+my $batch_msg_count; # Interim result frequency
+my $batch_top_domains; # Interim result count
my %opts; # Command line switches
my %q; # domain counts for queues and buckets
my %sub; # subdomain counts for parent domains
@@ -120,6 +133,7 @@ do {
warn "$0: $_[0]" unless exists($opts{"h"});
die "Usage: $0 [ -s ] [ -p ] [ -m ] [ -l ]\n".
"\t[ -b ] [ -t ] [ -w ]\n".
+ "\t[ -N ] [ -n ]\n".
"\t[ -c ] [ ... ]\n".
"The 's' option shows sender domain counts.\n".
"The 'p' option shows address counts by for parent domains.\n".
@@ -142,7 +156,7 @@ do {
"not supported. If necessary, use explicit absolute paths for all queues.\n";
};
- getopts("lhc:psw:b:t:m:", \%opts);
+ getopts("lhc:psw:b:t:m:n:N:", \%opts);
warn "Help message" if (exists $opts{"h"});
@qlist = @ARGV if (@ARGV > 0);
@@ -169,6 +183,16 @@ $bnum = $opts{"b"} if (exists $opts{"b"} && $opts{"b"} > 0);
$tick = $opts{"t"} if (exists $opts{"t"} && $opts{"t"} > 0);
$minsub = $opts{"m"} if (exists $opts{"m"} && $opts{"m"} > 0);
+if ( -t STDOUT ) {
+ $batch_msg_count = 1000 unless defined($batch_msg_count = $opts{"N"});
+ $batch_top_domains = 20 unless defined ($batch_top_domains = $opts{"n"});
+ $cls = `clear`;
+} else {
+ $batch_msg_count = 0;
+ $batch_top_domains = 0;
+ $cls = "";
+}
+
sub rec_get {
my ($h) = @_;
my $r = getc($h) || return;
@@ -263,6 +287,7 @@ sub bucket {
# Collate by age of message in the selected queues.
#
+my $msgs;
sub wanted {
if (my ($t, $s, @r) = qenv($_)) {
my $b = bucket($t, $now);
@@ -281,26 +306,15 @@ sub wanted {
$new = ! $old;
} while ($opts{"p"} && $a =~ s/^(?:\.)?[^.]+\.(.*\.)/.$1/);
}
+ if ($batch_msg_count > 0 && ++$msgs % $batch_msg_count == 0) {
+ results();
+ }
}
}
-find(\&wanted, @qlist);
my @heads;
-my $fmt = "";
-my $dw = $width;
-
-for (my $i = 0, my $t = 0; $i <= $bnum; ) {
- $q{"TOTAL"}->[$i] ||= 0;
- my $l = length($q{"TOTAL"}->[$i]);
- my $h = ($i == 0) ? "T" : $t;
- $l = length($h) if (length($h) >= $l);
- $l = ($l > 2) ? $l + 1 : 3;
- push(@heads, $h);
- $fmt .= sprintf "%%%ds", $l;
- $dw -= $l;
- if (++$i < $bnum) { $t += ($t && !$opts{"l"}) ? $t : $tick; } else { $t = "$t+"; }
-}
-$dw = $dwidth if ($dw < $dwidth);
+my $fmt;
+my $dw;
sub pdomain {
my ($d, @count) = @_;
@@ -318,18 +332,45 @@ sub pdomain {
printf "$fmt\n", @count;
}
-# Print headings
-#
-pdomain("", @heads);
+sub results {
+ @heads = ();
+ $dw = $width;
+ $fmt = "";
+ for (my $i = 0, my $t = 0; $i <= $bnum; ) {
+ $q{"TOTAL"}->[$i] ||= 0;
+ my $l = length($q{"TOTAL"}->[$i]);
+ my $h = ($i == 0) ? "T" : $t;
+ $l = length($h) if (length($h) >= $l);
+ $l = ($l > 2) ? $l + 1 : 3;
+ push(@heads, $h);
+ $fmt .= sprintf "%%%ds", $l;
+ $dw -= $l;
+ if (++$i < $bnum) { $t += ($t && !$opts{"l"}) ? $t : $tick; } else { $t = "$t+"; }
+ }
+ $dw = $dwidth if ($dw < $dwidth);
-# Show per-domain totals
-#
-foreach my $d (sort { $q{$b}->[0] <=> $q{$a}->[0] ||
- length($a) <=> length($b) } keys %q) {
+ print $cls if ($batch_msg_count > 0);
- # Skip parent domains with < $minsub subdomains.
+ # Print headings
#
- next if ($d =~ /^\./ && $sub{$d} < $minsub);
+ pdomain("", @heads);
- pdomain($d, @{$q{$d}});
+ my $n = 0;
+
+ # Show per-domain totals
+ #
+ foreach my $d (sort { $q{$b}->[0] <=> $q{$a}->[0] ||
+ length($a) <=> length($b) } keys %q) {
+
+ # Skip parent domains with < $minsub subdomains.
+ #
+ next if ($d =~ /^\./ && $sub{$d} < $minsub);
+
+ last if ($batch_top_domains > 0 && ++$n > $batch_top_domains);
+
+ pdomain($d, @{$q{$d}});
+ }
}
+
+find(\&wanted, @qlist);
+results();
diff --git a/postfix/conf/access b/postfix/conf/access
index 633f4d03b..ad5c548c4 100644
--- a/postfix/conf/access
+++ b/postfix/conf/access
@@ -1,7 +1,7 @@
# ACCESS(5) ACCESS(5)
#
# NAME
-# access - Postfix access table format
+# access - Postfix SMTP server access table
#
# SYNOPSIS
# postmap /etc/postfix/access
@@ -11,20 +11,18 @@
# postmap -q - /etc/postfix/access The sender/recipient address verification feature described in this
document is suitable only for low-traffic sites. It performs poorly
-under high load and may cause your site to be blacklisted by some
+under high load; excessive sender address verification activity may
+even cause your site to be blacklisted by some
providers. See the "Limitations" section
below for details.
diff --git a/postfix/html/DATABASE_README.html b/postfix/html/DATABASE_README.html
index 84838b569..315fe1c20 100644
--- a/postfix/html/DATABASE_README.html
+++ b/postfix/html/DATABASE_README.html
@@ -54,7 +54,7 @@ documentation:
-/etc/postfix/main.cf:
+/etc/postfix/main.cf:
alias_maps = hash:/etc/postfix/aliases (local aliasing)
header_checks = regexp:/etc/postfix/header_checks (content filtering)
transport_maps = hash:/etc/postfix/transport (routing table)
@@ -364,7 +364,7 @@ described in tcp_table(5). The lookup table name
where "host" specifies a symbolic hostname or a numeric IP address,
and "port" specifies a symbolic service name or a numeric port
number. This protocol is not available up to and including Postfix
-version 2.2.
+version 2.4.
unix (read-only)
diff --git a/postfix/html/LDAP_README.html b/postfix/html/LDAP_README.html
index e828b2848..6fe7ef71b 100644
--- a/postfix/html/LDAP_README.html
+++ b/postfix/html/LDAP_README.html
@@ -41,6 +41,8 @@ it to each.
Example: virtual domains/addresses
+Example: expanding LDAP groups
+
Other uses of LDAP lookups
Notes and things to think about
@@ -124,7 +126,7 @@ option (e.g. '-R') so the executables can find it at runtime.
In order to use LDAP lookups, define an LDAP source
-as a table lookup in main.cf, for example:
+as a table lookup in main.cf, for example:
@@ -140,7 +142,7 @@ page.
Here's a basic example for using LDAP to look up local(8)
-aliases. Assume that in main.cf, you have:
+aliases. Assume that in main.cf, you have:
@@ -152,14 +154,14 @@ aliases. Assume that in main.cf, you have:
-server_host = ldap.my.com
-search_base = dc=my, dc=com
+server_host = ldap.example.com
+search_base = dc=example, dc=com
Upon receiving mail for a local address "ldapuser" that isn't
found in the /etc/aliases database, Postfix will search the LDAP
-server listening at port 389 on ldap.my.com. It will bind anonymously,
+server listening at port 389 on ldap.example.com. It will bind anonymously,
search for any directory entries whose mailacceptinggeneralid
attribute is "ldapuser", read the "maildrop" attributes of those
found, and build a list of their maildrops, which will be treated
@@ -176,7 +178,7 @@ of your virtual recipient's mailacceptinggeneralid attributes are
fully qualified with their virtual domains. Finally, if you want
to designate a directory entry as the default user for a virtual
domain, just give it an additional mailacceptinggeneralid (or the
-equivalent in your directory) of "@virtual.dom". That's right, no
+equivalent in your directory) of "@fake.dom". That's right, no
user part. If you don't want a catchall user, omit this step and
mail to unknown users in the domain will simply bounce.
@@ -212,12 +214,221 @@ go to this entry ...
maildrop, e.g. "normaluser@fake.dom" and "normaluser@real.dom".
+
+
+ LDAP is frequently used to store group member information, and Postfix
+supports expanding a group's email address to the list of email addresses
+of the group members. There are a number of ways of handling LDAP groups,
+which will be illustrated via the mock LDAP entries and implied schema
+below. This shows two group entries "agroup" and "bgroup" and four
+user entries "auser", "buser", "cuser" and "duser". The group "agroup"
+has the users "auser" (1) and "buser" (2) as members via DN references
+in the multi-valued attribute "memberdn", and direct email addresses of
+two external users "auser@example.org" (3) and "buser@example.org" (4)
+stored in the multi-valued attribute "memberaddr". The same is true of
+"bgroup" and "cuser"/"duser" (6)/(7)/(8)/(9), but "bgroup" also has a
+"maildrop" attribute of "bgroup@mlm.example.com" (5):
+
+
+
+ dn: cn=agroup, dc=example, dc=com
+ objectclass: top
+ objectclass: ldapgroup
+ cn: agroup
+ mail: agroup@example.com
+1 -> memberdn: uid=auser, dc=example, dc=com
+2 -> memberdn: uid=buser, dc=example, dc=com
+3 -> memberaddr: auser@example.org
+4 -> memberaddr: buser@example.org
+
+
+
+
+ dn: cn=bgroup, dc=example, dc=com
+ objectclass: top
+ objectclass: ldapgroup
+ cn: bgroup
+ mail: bgroup@example.com
+5 -> maildrop: bgroup@mlm.example.com
+6 -> memberdn: uid=cuser, dc=example, dc=com
+7 -> memberdn: uid=duser, dc=example, dc=com
+8 -> memberaddr: cuser@example.org
+9 -> memberaddr: duser@example.org
+
+
+
+
+ dn: uid=auser, dc=example, dc=com
+ objectclass: top
+ objectclass: ldapuser
+ uid: auser
+10 -> mail: auser@example.com
+11 -> maildrop: auser@mailhub.example.com
+
+
+
+
+ dn: uid=buser, dc=example, dc=com
+ objectclass: top
+ objectclass: ldapuser
+ uid: buser
+12 -> mail: buser@example.com
+13 -> maildrop: buser@mailhub.example.com
+
+
+
+
+ dn: uid=cuser, dc=example, dc=com
+ objectclass: top
+ objectclass: ldapuser
+ uid: cuser
+14 -> mail: cuser@example.com
+
+
+
+
+ dn: uid=duser, dc=example, dc=com
+ objectclass: top
+ objectclass: ldapuser
+ uid: duser
+15 -> mail: duser@example.com
+
+
+
+
+
+ Our first use case ignores the "memberdn" attributes, and assumes
+that groups hold only direct "memberaddr" strings as in (3), (4), (8) and
+(9). The goal is to map the group address to the list of constituent
+"memberaddr" values. This is simple, ignoring the various connection
+related settings (hosts, ports, bind settings, timeouts, ...) we have:
+
+
+
+
+ simple.cf:
+ ...
+ search_base = dc=example, dc=com
+ query_filter = mail=%s
+ result_attribute = memberaddr
+ $ postmap -q agroup@example.com ldap:simple.cf
+ auser@example.org,buser@example.org
+
+
+
+ We search "dc=example, dc=com". The "mail" attribute is used in the
+query_filter to locate the right group, the "result_attribute" setting
+described in ldap_table(5) is used to specify that "memberaddr" values
+from the matching group are to be returned as a comma separated list.
+Always check tables using postmap(1) with the "-q" option, before
+deploying them into production use in main.cf.
+
+ Our second use case also expands "memberdn" attributes (1), (2),
+(6) and (7), follows the DN references and returns the "maildrop" of the
+referenced user entries. Here we use the "special_result_attribute"
+setting from ldap_table(5) to designate the "memberdn" attribute
+as holding DNs of the desired member entries. The "result_attribute"
+setting selects which attributes are returned from the selected DNs. It
+is important to choose a result attribute that is not also present in
+the group object, because result attributes are collected from both
+the group and the member DNs. In this case we choose "maildrop" and
+assume for the moment that groups never have a "maildrop" (the "bgroup"
+"maildrop" attribute is for a different use case). The returned data for
+"auser" and "buser" is from items (11) and (13) in the mock data.
+
+
+
+ special.cf:
+ ...
+ search_base = dc=example, dc=com
+ query_filter = mail=%s
+ result_attribute = memberaddr, maildrop
+ special_result_attribute = memberdn
+ $ postmap -q agroup@example.com ldap:special.cf
+ auser@mailhub.example.com,buser@mailhub.example.com,auser@example.org,buser@example.org
+
+
+
+ Note: if the desired member object result attribute is always also
+present in the group, you get suprising results, the expansion also
+returns the address of the group. This is a known limitation of Postfix
+releases prior to 2.4, and is addressed in the new with Postfix 2.4
+"leaf_result_attribute" feature described in ldap_table(5).
+
+ Our third use case has some groups that are expanded immediately,
+and other groups that are forwarded to a dedicated mailing list manager
+host for delayed expansion. This uses two LDAP tables, one for users
+and forwarded groups and a second for groups that can be expanded
+immediately. It is assumed that groups that require forwarding are
+never nested members of groups that are directly expanded.
+
+
+
+ no_expand.cf:
+ ...
+ search_base = dc=example, dc=com
+ query_filter = mail=%s
+ result_attribute = maildrop
+ expand.cf
+ ...
+ search_base = dc=example, dc=com
+ query_filter = mail=%s
+ result_attribute = memberaddr, maildrop
+ special_result_attribute = memberdn
+ $ postmap -q auser@example.com ldap:no_expand.cf ldap:expand.cf
+ auser@mailhub.example.com
+ $ postmap -q agroup@example.com ldap:no_expand.cf ldap:expand.cf
+ auser@mailhub.example.com,buser@mailhub.example.com,auser@example.org,buser@example.org
+ $ postmap -q bgroup@example.com ldap:no_expand.cf ldap:expand.cf
+ bgroup@mlm.example.com
+
+
+
+ Non-group objects and groups with delayed expansion (those that have a
+maildrop attribute) are rewritten to a single maildrop value. Groups that
+don't have a maildrop are expanded as the second use case. This admits
+a more elegant solution with Postfix 2.4 and later.
+
+ Our final use case is the same as the third, but this time uses new
+features in Postfix 2.4. We now are able to use just one LDAP table and
+no longer need to assume that forwarded groups are never nested inside
+expanded groups.
+
+
+
+ fancy.cf:
+ ...
+ search_base = dc=example, dc=com
+ query_filter = mail=%s
+ result_attribute = memberaddr
+ special_result_attribute = memberdn
+ terminal_result_attribute = maildrop
+ leaf_result_attribute = mail
+ $ postmap -q auser@example.com ldap:fancy.cf
+ auser@mailhub.example.com
+ $ postmap -q cuser@example.com ldap:fancy.cf
+ cuser@example.com
+ $ postmap -q agroup@example.com ldap:fancy.cf
+ auser@mailhub.example.com,buser@mailhub.example.com,auser@example.org,buser@example.org
+ $ postmap -q bgroup@example.com ldap:fancy.cf
+ bgroup@mlm.example.com
+
+
+
+ Above, delayed expansion is enabled via "terminal_result_attribute",
+which, if present, is used as the sole result and all other expansion is
+suppressed. Otherwise, the "leaf_result_attribute" is only returned for
+leaf objects that don't have a "special_result_attribute" (non-groups),
+while the "result_attribute" (direct member address of groups) is returned
+at every level of recursive expansion, not just the leaf nodes. This fancy
+example illustrates all the features of Postfix 2.4 group expansion.
+
Other common uses for LDAP lookups include rewriting senders and
recipients with Postfix's canonical lookups, for example in order
to make mail leaving your site appear to be coming from
-"First.Last@site.dom" instead of "userid@site.dom".
+"First.Last@example.com" instead of "userid@example.com".
@@ -240,9 +451,9 @@ to make mail leaving your site appear to be coming from
-dn: cn=Accounting Staff List, dc=my, dc=com
+dn: cn=Accounting Staff List, dc=example, dc=com
cn: Accounting Staff List
-o: my.com
+o: example.com
objectclass: maillist
mailacceptinggeneralid: accountingstaff
mailacceptinggeneralid: accounting-staff
@@ -341,7 +552,7 @@ contents, please include the applicable bits of some directory entries.
external files (ldap:/path/ldap.cf) needed to securely store
passwords for plain auth.
-Liviu Daia revised the configuration interface and added the main.cf
+Liviu Daia revised the configuration interface and added the main.cf
configuration feature.
Liviu Daia with further refinements from Jose Luis Tallon and
diff --git a/postfix/html/QSHAPE_README.html b/postfix/html/QSHAPE_README.html
index c15d95efd..905bc5ebb 100644
--- a/postfix/html/QSHAPE_README.html
+++ b/postfix/html/QSHAPE_README.html
@@ -19,17 +19,11 @@
Purpose of this document
- This document describes the qshape(1) program which helps the
-administrator understand the Postfix queue message distribution
-sorted by time and by sender or recipient domain. qshape(1) is
-bundled with the Postfix 2.1 source under the "auxiliary" directory.
-
-
- In order to understand the output of qshape(1), it useful to
-understand the various Postfix queues. To this end the role of each
-Postfix queue directory is described briefly in the "Background
-info: Postfix queue directories" section near the end of this
-document.
+ This document is an introduction to Postfix queue congestion analysis.
+It explains how the qshape(1) program can help to track down the
+reason for queue congestion. qshape(1) is bundled with Postfix
+2.1 and later source code, under the "auxiliary" directory. This
+document describes qshape(1) as bundled with Postfix 2.4.
This document covers the following topics:
@@ -49,7 +43,7 @@ queue
Example 4: High volume destination backlog
-Background info: Postfix queue directories
+Postfix queue directories
-
When mail is draining slowly or the queue is unexpectedly large,
run qshape(1) as the super-user (root) to help zero in on the problem.
The qshape(1) program displays a tabular view of the Postfix queue
@@ -124,6 +117,12 @@ minutes old and 12 older than 1280 minutes (1440 minutes in a day).
+
When the output is a terminal intermediate results showing the top 20
+domains (-n option) are displayed after every 1000 messages (-N option)
+and the final output also shows only the top 20 domains. This makes
+qshape useful even when the deferred queue is very large and it may
+otherwise take prohibitively long to read the entire deferred queue.
+
By default, qshape shows statistics for the union of both the
incoming and active queues which are the most relevant queues to
look at when analyzing performance.
@@ -132,8 +131,8 @@ look at when analyzing performance.
-$ qshape deferred | less
-$ qshape incoming active deferred | less
+$ qshape deferred
+$ qshape incoming active deferred
@@ -157,11 +156,11 @@ a burst of mail started, and when it stopped.
The problem destinations or sender domains appear near the top
left corner of the output table. Remember that the active queue
can accommodate up to 20000 ($qmgr_message_active_limit) messages.
-To check wether this limit has been reached, use:
+To check whether this limit has been reached, use:
-$ qshape -s active | head (show sender statistics)
+$ qshape -s active (show sender statistics)
@@ -169,13 +168,13 @@ $ qshape -s active | head (show sender statistics)
not yet saturated, any high volume sender domains show near the
top of the output.
- The active queue is also limited to at most 20000 recipient
-addresses ($qmgr_message_recipient_limit). To check for exhaustion
-of this limit use:
+ With oqmgr(8) the active queue is also limited to at most 20000
+recipient addresses ($qmgr_message_recipient_limit). To check for
+exhaustion of this limit use:
-$ qshape active | head (show recipient statistics)
+$ qshape active (show recipient statistics)
@@ -381,14 +380,17 @@ queue congestion is a greater cause for alarm; one might need to
take measures to ensure that the mail is deferred instead or even
add an access(5) rule asking the sender to try again later.
- If a high volume destination exhibits frequent bursts of
-consecutive connections refused by all MX hosts or "421 Server busy
-errors", it is possible for the queue manager to mark the destination
-as "dead" despite the transient nature of the errors. The destination
-will be retried again after the expiration of a $minimal_backoff_time
-timer. If the error bursts are frequent enough it may be that only
-a small quantity of email is delivered before the destination is
-again marked "dead".
+ If a high volume destination exhibits frequent bursts of consecutive
+connections refused by all MX hosts or "421 Server busy errors", it
+is possible for the queue manager to mark the destination as "dead"
+despite the transient nature of the errors. The destination will be
+retried again after the expiration of a $minimal_backoff_time timer.
+If the error bursts are frequent enough it may be that only a small
+quantity of email is delivered before the destination is again marked
+"dead". In some cases enabling static (not on demand) connection
+caching by listing the appropriate nexthop domain in a table included in
+"smtp_connection_cache_destinations" may help to reduce the error rate,
+because most messages will re-use existing connections.
The MTA that has been observed most frequently to exhibit such
bursts of errors is Microsoft Exchange, which refuses connections
@@ -396,47 +398,44 @@ under load. Some proxy virus scanners in front of the Exchange
server propagate the refused connection to the client as a "421"
error.
- Note that it is now possible to configure Postfix to exhibit
-similarly erratic behavior by misconfiguring the anvil(8) server
-(not included in Postfix 2.1.). Do not use anvil(8) for steady-state
-rate limiting, its purpose is DoS prevention and the rate limits
-set should be very generous!
+ Note that it is now possible to configure Postfix to exhibit similarly
+erratic behavior by misconfiguring the anvil(8) service. Do not use
+anvil(8) for steady-state rate limiting, its purpose is (unintentional)
+DoS prevention and the rate limits set should be very generous!
- In the long run it is hoped that the Postfix dead host detection
-and concurrency control mechanism will be tuned to be more "noise"
-tolerant. If one finds oneself needing to deliver a high volume
-of mail to a destination that exhibits frequent brief bursts of
-errors, there is a subtle workaround.
+ If one finds oneself needing to deliver a high volume of mail to a
+destination that exhibits frequent brief bursts of errors and connection
+caching does not solve the problem, there is a subtle workaround.
--
In master.cf set up a dedicated clone of the "smtp"
+
-
In master.cf set up a dedicated clone of the "smtp"
transport for the destination in question.
- -
In master.cf configure a reasonable process limit for the
+
-
In master.cf configure a reasonable process limit for the
transport (a number in the 10-20 range is typical).
- -
IMPORTANT!!! In main.cf configure a very large initial
-and destination concurrency limit for this transport (say 200).
+ -
IMPORTANT!!! In main.cf configure a very large initial
+and destination concurrency limit for this transport (say 2000).
-/etc/postfix/main.cf:
- initial_destination_concurrency = 200
- transportname_destination_concurrency_limit = 200
+/etc/postfix/main.cf:
+ initial_destination_concurrency = 2000
+ transportname_destination_concurrency_limit = 2000
- Where transportname is the name of the master.cf entry
+
Where transportname is the name of the master.cf entry
in question.
- The effect of this surprising configuration is that up to 200
+
The effect of this surprising configuration is that up to 2000
consecutive errors are tolerated without marking the destination
dead, while the total concurrency remains reasonable (10-20
processes). This trick is only for a very specialized situation:
high volume delivery into a channel with multi-error bursts
that is capable of high throughput, but is repeatedly throttled by
-the bursts of errors.
+the bursts of errors.
When a destination is unable to handle the load even after the
Postfix process limit is reduced to 1, a desperate measure is to
@@ -447,7 +446,7 @@ insert brief delays between delivery attempts.
In the transport map entry for the problem destination,
specify a dead host as the primary nexthop.
- In the master.cf entry for the transport specify the
+
In the master.cf entry for the transport specify the
problem destination as the fallback_relay and specify a small
smtp_connect_timeout value.
@@ -455,7 +454,7 @@ problem destination as the fallback_rel
/etc/postfix/transport:
problem.example.com slow:[dead.host]
-/etc/postfix/master.cf:
+/etc/postfix/master.cf:
# service type private unpriv chroot wakeup maxproc command
slow unix - - n - 1 smtp
-o fallback_relay=problem.example.com
@@ -472,7 +471,7 @@ updated when SMTP connection caching is introduced.
Hopefully a more elegant solution to these problems will be
found in the future.
-
+
The following sections describe Postfix queues: their purpose,
what normal behavior looks like, and how to diagnose abnormal
@@ -497,8 +496,8 @@ to notify the pickup(8) service of its arrival.
All mail that enters the main Postfix queue does so via the
cleanup(8) service. The cleanup service is responsible for envelope
and header rewriting, header and body regular expression checks,
-automatic bcc recipient processing and guaranteed insertion of the
-message into the Postfix "incoming" queue.
+automatic bcc recipient processing, milter content processing, and
+reliable insertion of the message into the Postfix "incoming" queue.
In the absence of excessive CPU consumption in cleanup(8) header
or body regular expression checks or other software consuming all
@@ -514,9 +513,10 @@ one message at a time at a rate that does not exceed the reciprocal
disk I/O latency (+ CPU if not negligible) of the cleanup service.
- Congestion in this queue is indicative of an excessive local
-message submission rate or perhaps excessive CPU consumption in
-the cleanup(8) service due to excessive body_checks.
+ Congestion in this queue is indicative of an excessive local message
+submission rate or perhaps excessive CPU consumption in the cleanup(8)
+service due to excessive body_checks, or (Postfix ≥ 2.3) high latency
+milters.
Note, that once the active queue is full, the cleanup service
will attempt to slow down message injection by pausing $in_flow_delay
@@ -524,10 +524,10 @@ for each message. In this case "mail
a consequence of congestion downstream, rather than a problem in
its own right.
- Note also, that one should not attempt to deliver large volumes
-of mail via the pickup(8) service. High volume sites must avoid
-using content filters that reinject scanned mail via Postfix
-sendmail(1) and postdrop(1).
+ Note, you should not attempt to deliver large volumes of mail via
+the pickup(8) service. High volume sites should avoid using "simple"
+content filters that re-inject scanned mail via Postfix sendmail(1)
+and postdrop(1).
A high arrival rate of locally submitted mail may be an indication
of an uncaught forwarding loop, or a run-away notification program.
@@ -545,20 +545,19 @@ size of the "maildrop" queue.
The administrator can define "smtpd" access(5) policies, or
cleanup(8) header/body checks that cause messages to be automatically
diverted from normal processing and placed indefinitely in the
-"hold" queue. Messages placed in the "hold" queue stay there until
+"hold" queue. Messages placed in the "hold" queue stay there until
the administrator intervenes. No periodic delivery attempts are
made for messages in the "hold" queue. The postsuper(1) command
can be used to manually release messages into the "deferred" queue.
- Messages can potentially stay in the "hold" queue for a time
-exceeding the normal maximal queue lifetime (after which undelivered
-messages are bounced back to the sender). If such "old" messages
-need to be released from the "hold" queue, they should typically
-be moved into the "maildrop" queue, so that the message gets a new
-timestamp and is given more than one opportunity to be delivered.
-Messages that are "young" can be moved directly into the "deferred"
-queue.
+ Messages can potentially stay in the "hold" queue longer than
+$maximal_queue_lifetime. If such "old" messages need to be released from
+the "hold" queue, they should typically be moved into the "maildrop"
+queue using "postsuper -r", so that the message gets a new timestamp and
+is given more than one opportunity to be delivered. Messages that are
+"young" can be moved directly into the "deferred" queue using
+"postsuper -H".
The "hold" queue plays little role in Postfix performance, and
monitoring of the "hold" queue is typically more closely motivated
@@ -589,11 +588,15 @@ messages into the active queue as
The incoming queue grows when the message input rate spikes
above the rate at which the queue manager can import messages into
-the active queue. The main factor slowing down the queue manager
-is transport queries to the trivial-rewrite service. If the queue
+the active queue. The main factors slowing down the queue manager
+are disk I/O and lookup queries to the trivial-rewrite service. If the queue
manager is routinely not keeping up, consider not using "slow"
lookup services (MySQL, LDAP, ...) for transport lookups or speeding
-up the hosts that provide the lookup service.
+up the hosts that provide the lookup service. If the problem is I/O
+starvation, consider striping the queue over more disks, faster controllers
+with a battery write cache, or other hardware improvements. At the very
+least, make sure that the queue directory is mounted with the "noatime"
+option if applicable to the underlying filesystem.
The in_flow_delay parameter is used to clamp the input rate
when the queue manager starts to fall behind. The cleanup(8) service
@@ -645,26 +648,40 @@ combination; the group size is capped by the transport's recipient
concurrency limit.
Multiple recipient groups (from one or more messages) are queued
-for delivery via the common transport/nexthop combination. The
-destination concurrency limit for the transports caps the number
+for delivery grouped by transport/nexthop combination. The
+destination concurrency limit for the transports caps the number
of simultaneous delivery attempts for each nexthop. Transports with
-a recipient concurrency limit of 1 are special: these are grouped
-by the actual recipient address rather than the nexthop, thereby
-enabling per-recipient concurrency limits rather than per-domain
+a recipient concurrency limit of 1 are special: these are grouped
+by the actual recipient address rather than the nexthop, yielding
+per-recipient concurrency limits rather than per-domain
concurrency limits. Per-recipient limits are appropriate when
performing final delivery to mailboxes rather than when relaying
to a remote server.
Congestion occurs in the active queue when one or more destinations
-drain slower than the corresponding message input rate. If a
-destination is down for some time, the queue manager will mark it
-dead, and immediately defer all mail for the destination without
+drain slower than the corresponding message input rate.
+
+ Input into the active queue comes both from new mail in the "incoming"
+queue, and retries of mail in the "deferred" queue. Should the "deferred"
+queue get really large, retries of old mail can dominate the arrival
+rate of new mail. Systems with more CPU, faster disks and more network
+bandwidth can deal with larger deferred queues, but as a rule of thumb
+the deferred queue scales to somewhere between 100,000 and 1,000,000
+messages with good performance unlikely above that "limit". Systems with
+queues this large should typically stop accepting new mail, or put the
+backlog "on hold" until the underlying issue is fixed (provided that
+there is enough capacity to handle just the new mail).
+
+ When a destination is down for some time, the queue manager will
+mark it dead, and immediately defer all mail for the destination without
trying to assign it to a delivery agent. In this case the messages
-will quickly leave the active queue and end up in the deferred
-queue. If the destination is instead simply slow, or there is a
-problem causing an excessive arrival rate the active queue will
-grow and will become dominated by mail to the congested destination.
-
+will quickly leave the active queue and end up in the deferred queue
+(with Postfix < 2.4, this is done directly by the queue manager,
+with Postfix ≥ 2.4 this is done via the "retry" delivery agent).
+
+ When the destination is instead simply slow, or there is a problem
+causing an excessive arrival rate the active queue will grow and will
+become dominated by mail to the congested destination.
The only way to reduce congestion is to either reduce the input
rate or increase the throughput. Increasing the throughput requires
@@ -691,28 +708,56 @@ a high average latency. If the number of outbound SMTP connections
is draining slowly and the system and network are not loaded, raise
the "smtp" and/or "relay" process limits!
- Especially for the "relay" transport, consider lower SMTP
-connection timeouts (1-5 seconds) and higher than default destination
-concurrency limits. Compute the expected latency when 1 out of N
-of the MX hosts for a high volume site is down and not responding,
-and make sure that the configured concurrency divided by this
-latency exceeds the required steady-state message rate. If the
-destination is managed by you, consider load balancers in front of
-groups of MX hosts. Load balancers have higher uptime and will be
-able to hide individual MX host failures.
-
- If necessary, dedicate and tune custom transports for high
-volume destinations.
-
- Another common cause of congestion is unwarranted flushing of
-the entire deferred queue. The deferred queue holds messages that
-are likely to fail to be delivered and are also likely to be slow
-to fail delivery (timeouts). This means that the most common reaction
-to a large deferred queue (flush it!) is more than likely counter-
-productive, and is likely to make the problem worse. Do not flush
-the deferred queue unless you expect that most of its content has
-recently become deliverable (e.g. relayhost back up after an outage)!
-
+ When a high volume destination is served by multiple MX hosts with
+typically low delivery latency, performance can suffer dramatically when
+one of the MX hosts is unresponsive and SMTP connections to that host
+timeout. For example, if there are 2 equal weight MX hosts, the SMTP
+connection timeout is 30 seconds and one of the MX hosts is down, the
+average SMTP connection will take approximately 15 seconds to complete.
+With a default per-destination concurrency limit of 20 connections,
+throughput falls to just over 1 message per second.
+
+ The best way to avoid bottlenecks when one or more MX hosts is
+non-responsive is to use connection caching. Connection caching was
+introduced with Postfix 2.2 and is by default enabled on demand for
+destinations with a backlog of mail in the active queue. When connection
+caching is in effect for a particular destination, established connections
+are re-used to send additional messages, this reduces the number of
+connections made per message delivery and maintains good throughput even
+in the face of partial unavailability of the destination's MX hosts.
+
+ If connection caching is not available (Postfix < 2.2) or does
+not provide a sufficient latency reduction, especially for the "relay"
+transport used to forward mail to "your own" domains, consider setting
+lower than default SMTP connection timeouts (1-5 seconds) and higher
+than default destination concurrency limits. This will further reduce
+latency and provide more concurrency to maintain throughput should
+latency rise.
+
+ Setting high concurrency limits to domains that are not your own may
+be viewed as hostile by the receiving system, and steps may be taken
+to prevent you from monopolizing the destination system's resources.
+The defensive measures may substantially reduce your throughput or block
+access entirely. Do not set aggressive concurrency limits to remote
+domains without coordinating with the administrators of the target
+domain.
+
+ If necessary, dedicate and tune custom transports for selected high
+volume destinations. The "relay" transport is provided for forwarding mail
+to domains for which your server is a primary or backup MX host. These can
+make up a substantial fraction of your email traffic. Use the "relay" and
+not the "smtp" transport to send email to these domains. Using the "relay"
+transport allocates a separate delivery agent pool to these destinations
+and allows separate tuning of timeouts and concurrency limits.
+
+ Another common cause of congestion is unwarranted flushing of the
+entire deferred queue. The deferred queue holds messages that are likely
+to fail to be delivered and are also likely to be slow to fail delivery
+(time out). As a result the most common reaction to a large deferred queue
+(flush it!) is more than likely counter-productive, and typically makes
+the congestion worse. Do not flush the deferred queue unless you expect
+that most of its content has recently become deliverable (e.g. relayhost
+back up after an outage)!
Note that whenever the queue manager is restarted, there may
already be messages in the active queue directory, but the "real"
@@ -723,7 +768,7 @@ queue scan to refill the active queue<
the messages back and forth, redoing transport table (trivial-rewrite(8)
resolve service) lookups, and re-importing the messages back into
memory is expensive. At all costs, avoid frequent restarts of the
-queue manager.
+queue manager (e.g. via frequent execution of "postfix reload").
@@ -732,20 +777,19 @@ and for some recipients delivery failed for a transient reason (it
might succeed later), the message is placed in the deferred queue.
- The queue manager scans the deferred queue periodically. The
-scan interval is controlled by the queue_run_delay parameter.
-While a deferred queue scan is in progress, if an incoming queue
-scan is also in progress (ideally these are brief since the incoming
-queue should be short), the queue manager alternates between bringing
-a new "incoming" message and a new "deferred" message into the
-queue. This "round-robin" strategy prevents starvation of either
-the incoming or the deferred queues.
+ The queue manager scans the deferred queue periodically. The scan
+interval is controlled by the queue_run_delay parameter. While a deferred
+queue scan is in progress, if an incoming queue scan is also in progress
+(ideally these are brief since the incoming queue should be short), the
+queue manager alternates between looking for messages in the "incoming"
+queue and in the "deferred" queue. This "round-robin" strategy prevents
+starvation of either the incoming or the deferred queues.
Each deferred queue scan only brings a fraction of the deferred
queue back into the active queue for a retry. This is because each
message in the deferred queue is assigned a "cool-off" time when
it is deferred. This is done by time-warping the modification
-times of the queue file into the future. The queue file is not
+time of the queue file into the future. The queue file is not
eligible for a retry if its modification time is not yet reached.
@@ -756,28 +800,34 @@ within the limits. This means that young messages are initially
retried more often than old messages.
If a high volume site routinely has large deferred queues, it
-may be useful to adjust the queue_run_delay, minimal_backoff_time
-and maximal_backoff_time to provide short enough delays on first
-failure, with perhaps longer delays after multiple failures, to
-reduce the retransmission rate of old messages and thereby reduce
-the quantity of previously deferred mail in the active queue.
+may be useful to adjust the queue_run_delay, minimal_backoff_time and
+maximal_backoff_time to provide short enough delays on first failure
+(Postfix ≥ 2.4 has a sensibly low minimal backoff time by default),
+with perhaps longer delays after multiple failures, to reduce the
+retransmission rate of old messages and thereby reduce the quantity
+of previously deferred mail in the active queue. If you want a really
+low minimal_backoff_time, you may also want to lower queue_run_delay,
+but understand that more frequent scans will increase the demand for
+disk I/O.
One common cause of large deferred queues is failure to validate
recipients at the SMTP input stage. Since spammers routinely launch
dictionary attacks from unrepliable sender addresses, the bounces
-for invalid recipient addresses clog the deferred queue (and at
-high volumes proportionally clog the active queue). Recipient
-validation is strongly recommended through use of the local_recipient_maps
-and relay_recipient_maps parameters.
+for invalid recipient addresses clog the deferred queue (and at high
+volumes proportionally clog the active queue). Recipient validation
+is strongly recommended through use of the local_recipient_maps and
+relay_recipient_maps parameters. Even when bounces drain quickly they
+inundate innocent victims of forgery with unwanted email. To avoid
+this, do not accept mail for invalid recipients.
When a host with lots of deferred mail is down for some time,
it is possible for the entire deferred queue to reach its retry
time simultaneously. This can lead to a very full active queue once
the host comes back up. The phenomenon can repeat approximately
every maximal_backoff_time seconds if the messages are again deferred
-after a brief burst of congestion. Ideally, in the future Postfix
+after a brief burst of congestion. Perhaps, a future Postfix release
will add a random offset to the retry time (or use a combination
-of strategies) to reduce the chances of repeated complete deferred
+of strategies) to reduce the odds of repeated complete deferred
queue flushes.
diff --git a/postfix/html/SASL_README.html b/postfix/html/SASL_README.html
index 20b1a562f..3824e7de2 100644
--- a/postfix/html/SASL_README.html
+++ b/postfix/html/SASL_README.html
@@ -149,7 +149,7 @@ their CCARGS and AUXLIBS into the above command line.
- Postfix appears to work with cyrus-sasl-1.5.5 or cyrus-sasl-2.1.1,
+
Postfix appears to work with cyrus-sasl-1.5.x or cyrus-sasl-2.1.x,
which are available from:
@@ -160,11 +160,11 @@ which are available from:
IMPORTANT: if you install the Cyrus SASL libraries as per the
default, you will have to symlink /usr/lib/sasl -> /usr/local/lib/sasl
-for version 1.5.5 or /usr/lib/sasl2 -> /usr/local/lib/sasl2 for
-version 2.1.1.
+for version 1.5.x or /usr/lib/sasl2 -> /usr/local/lib/sasl2 for
+version 2.1.x.
- Reportedly, Microsoft Internet Explorer version 5 requires the
-non-standard SASL LOGIN authentication method. To enable this
+
Reportedly, Microsoft Outlook (Express) requires the
+non-standard LOGIN authentication method. To enable this
authentication method, specify ``./configure --enable-login''.
@@ -178,7 +178,7 @@ and that the Cyrus SASL libraries are in /usr/local/lib.
-- (for Cyrus SASL version 1.5.5):
+
- (for Cyrus SASL version 1.5.x):
-
% make tidy # if you have left-over files from a previous build
@@ -186,7 +186,7 @@ and that the Cyrus SASL libraries are in /usr/local/lib.
-I/usr/local/include" AUXLIBS="-L/usr/local/lib -lsasl"
- - (for Cyrus SASL version 2.1.1):
+
- (for Cyrus SASL version 2.1.x):
-
% make tidy # if you have left-over files from a previous build
@@ -201,7 +201,7 @@ otherwise ld.so will not find the SASL shared library:
-- (for Cyrus SASL version 1.5.5):
+
- (for Cyrus SASL version 1.5.x):
-
% make tidy # if you have left-over files from a previous build
@@ -210,7 +210,7 @@ otherwise ld.so will not find the SASL shared library:
-R/usr/local/lib -lsasl"
- - (for Cyrus SASL version 2.1.1):
+
- (for Cyrus SASL version 2.1.x):
-
% make tidy # if you have left-over files from a previous build
@@ -258,8 +258,9 @@ SMTP server
Older Microsoft SMTP client software implements a non-standard
version of the AUTH protocol syntax, and expects that the SMTP
-server replies to EHLO with "250 AUTH=stuff" instead of "250 AUTH
-stuff". To accommodate such clients (in addition to conformant
+server replies to EHLO with "250 AUTH=mechanism-list" instead of
+"250 AUTH mechanism-list". To accommodate such clients (in addition
+to conformant
clients) use the following:
@@ -318,22 +319,41 @@ the Dovecot authentication server.
- In /usr/local/lib/sasl/smtpd.conf (Cyrus SASL version 1.5.5) or
-/usr/local/lib/sasl2/smtpd.conf (Cyrus SASL version 2.1.1) you need to
-specify how the server should validate client passwords.
+ You need to configure how the Cyrus SASL library should
+authenticate a client's username and password. These settings must
+be stored in a separate configuration file.
+
+ The name of the configuration file (default: smtpd.conf) will
+be constructed from a value sent by Postfix to the Cyrus SASL
+library, which adds the suffix .conf. The value is configured using
+one of the following variables:
+
+
+
+/etc/postfix/main.cf:
+ # Postfix 2.3 and later
+ smtpd_sasl_path = smtpd
+ # Postfix < 2.3
+ smtpd_sasl_application_name = smtpd
+
+
+
+ Cyrus SASL searches for the configuration file in /usr/local/lib/sasl/
+(Cyrus SASL version 1.5.5) or /usr/local/lib/sasl2/ (Cyrus SASL
+version 2.1.x).
Note: some Postfix distributions are modified and look for
-the smtpd.conf file in /etc/postfix.
+the smtpd.conf file in /etc/postfix/sasl.
Note: some Cyrus SASL distributions look for the smtpd.conf
file in /etc/sasl2.
--
To authenticate against the UNIX password database, try:
+ -
To authenticate against the UNIX password database, use:
-- (Cyrus SASL version 1.5.5)
+
- (Cyrus SASL version 1.5.x)
-
/usr/local/lib/sasl/smtpd.conf:
@@ -341,39 +361,13 @@ file in /etc/sasl2.
- - (Cyrus SASL version 2.1.1)
-
-
-
-/usr/local/lib/sasl2/smtpd.conf:
- pwcheck_method: pwcheck
-
-
-
-
- The name of the file in /usr/local/lib/sasl (Cyrus SASL version
-1.5.5) or /usr/local/lib/sasl2 (Cyrus SASL version 2.1.1) used by
-the SASL
-library for configuration can be set with:
-
-
-
-/etc/postfix/main.cf:
- smtpd_sasl_application_name = smtpd (Postfix < 2.3)
- smtpd_sasl_path = smtpd (Postfix 2.3 and later)
-
-
+ IMPORTANT: pwcheck establishes a UNIX domain socket in /var/pwcheck
+and waits for authentication requests. Postfix processes must have
+read+execute permission to this directory or authentication attempts
+will fail.
The pwcheck daemon is contained in the cyrus-sasl source tarball.
- IMPORTANT: postfix processes need to have group read+execute
-permission for the /var/pwcheck directory, otherwise authentication
-attempts will fail.
-
- -
Alternately, in Cyrus SASL 1.5.26 and later (including
-2.1.1), try:
-
-
-
- (Cyrus SASL version 1.5.26)
-
@@ -381,11 +375,12 @@ attempts will fail.
pwcheck_method: saslauthd
- - (Cyrus SASL version 2.1.1)
+
- (Cyrus SASL version 2.1.x)
-
/usr/local/lib/sasl2/smtpd.conf:
pwcheck_method: saslauthd
+ mech_list: PLAIN LOGIN
@@ -395,27 +390,38 @@ tarball. It is more flexible than the pwcheck daemon, in that it
can authenticate against PAM and various other sources. To use PAM,
start saslauthd with "-a pam".
+ IMPORTANT: saslauthd usually establishes a UNIX domain socket
+in /var/run/saslauthd and waits for authentication requests. Postfix
+processes must have read+execute permission to this directory or
+authentication attempts will fail.
+
+ Note: The directory where saslauthd puts the socket is configurable.
+See the command-line option "-m /path/to/socket" in the saslauthd
+--help listing.
+
-
To authenticate against Cyrus SASL's own password database:
-- (Cyrus SASL version 1.5.5)
+
- (Cyrus SASL version 1.5.x)
-
/usr/local/lib/sasl/smtpd.conf:
- pwcheck_method: sasldb
+ pwcheck_method: sasldb
- - (Cyrus SASL version 2.1.1)
+
- (Cyrus SASL version 2.1.x)
-
/usr/local/lib/sasl2/smtpd.conf:
- pwcheck_method: auxprop
+ pwcheck_method: auxprop
+ auxprop_plugin: sasldb
+ mech_list: PLAIN LOGIN CRAM-MD5 DIGEST-MD5
This will use the Cyrus SASL password file (default: /etc/sasldb in
-version 1.5.5, or /etc/sasldb2 in version 2.1.1), which is maintained
+version 1.5.x, or /etc/sasldb2 in version 2.1.x), which is maintained
with the saslpasswd or saslpasswd2 command (part of the Cyrus SASL
software). On some poorly-supported systems the saslpasswd command needs
to be run multiple times before it stops complaining. The Postfix SMTP
@@ -430,13 +436,13 @@ domain (realm) to a fully qualified domain name.
EXAMPLE:
-- (Cyrus SASL version 1.5.5)
+
- (Cyrus SASL version 1.5.x)
-
% saslpasswd -c -u `postconf -h myhostname` exampleuser
- - (Cyrus SASL version 2.1.1)
+
- (Cyrus SASL version 2.1.x)
-
% saslpasswd2 -c -u `postconf -h myhostname` exampleuser
@@ -445,8 +451,8 @@ domain (realm) to a fully qualified domain name.
You can find out SASL's idea about the realms of the users
-in sasldb with sasldblistusers (Cyrus SASL version 1.5.5) or
-sasldblistusers2 (Cyrus SASL version 2.1.1).
+in sasldb with sasldblistusers (Cyrus SASL version 1.5.x) or
+sasldblistusers2 (Cyrus SASL version 2.1.x).
On the Postfix side, you can have only one realm per smtpd
instance, and only the users belonging to that realm would be able to
@@ -462,18 +468,14 @@ realm used by smtpd:
- IMPORTANT: all users must be able to authenticate using ALL
-authentication mechanisms advertised by Postfix, otherwise the
-negotiation might end up with an unsupported mechanism, and
-authentication would fail. For example if you configure SASL to
-use saslauthd for authentication against PAM (pluggable
-authentication modules), only the PLAIN and LOGIN mechanisms are
-supported and stand a chance to succeed, yet the SASL library would also
-advertise other mechanisms, such as DIGEST-MD5. This happens because
-those mechanisms are made available by other plugins, and the SASL
-library have no way to know that your only valid authentication source
-is PAM. Thus you might need to limit the list of mechanisms advertised
-by Postfix.
+ IMPORTANT: The Cyrus SASL password verification services pwcheck
+and saslauthd can only support the plaintext mechanisms PLAIN or
+LOGIN. However, the Cyrus SASL library doesn't know this, and will
+happily advertise other authentication mechanisms that the SASL
+library implements, such as DIGEST-MD5. As a result, if an SMTP
+client chooses any mechanism other than PLAIN or LOGIN while pwcheck
+or saslauthd are used, authentication will fail. Thus you may need
+to limit the list of mechanisms advertised by Postfix.
+ The Postfix SASL client password file is opened before the SMTP
+server enters the optional chroot jail, so you can keep the file
+in /etc/postfix and set permissions read / write only for root to
+keep the username:password combinations away from other system
+users.
+
Postfix version 2.3 supports-per-sender SASL password
information. To search the Postfix SASL password by sender
before it searches by destination, specify:
@@ -645,10 +658,6 @@ for example:
- The Postfix SASL client password file is opened before the SMTP server
-enters the optional chroot jail, so you can keep the file in
-/etc/postfix.
-
Note: Some SMTP servers support authentication mechanisms that,
although available on the client system, may not in practice work or
possess the appropriate credentials to authenticate to the server. It
@@ -664,7 +673,7 @@ into consideration:
In the above example, Postfix will decline to use mechanisms
-that require special infrastructure such as Kerberos.
+that require special infrastructure such as Kerberos or TLS.
The Postfix SMTP client is backwards compatible with SMTP
servers that use the non-standard "AUTH=method..." syntax in response
@@ -694,6 +703,9 @@ smtpd_sasl_application_name into smtpd
The Dovecot SMTP server-only plug-in was originally implemented by
Timo Sirainen of Procontrol, Finland.
+ Patrick Ben Koetter revised this document for Postfix 2.4 and
+made much needed updates.
+