From: Wietse Venema Overview
-This document describes features that require Postfix version 2.0
-or later. The examples use Perl Compatible Regular Expressions
-(Postfix pcre: tables), but also provide a translation to POSIX
-regular expressions (Postfix regexp: tables). PCRE is preferred
-primarily because the implementation is often faster.
This document describes features that require Postfix version +2.0 or later.
Topics covered in this document:
@@ -56,6 +53,11 @@ scanners +The examples use Perl Compatible Regular Expressions (Postfix +pcre: tables), but also provide a translation to POSIX regular +expressions (Postfix regexp: tables). PCRE is preferred primarily +because the implementation is often faster.
+When a spammer or worm sends mail with forged sender addresses, @@ -73,7 +75,7 @@ to=<yyyyyy@your.domain.here> proto=ESMTP helo=<zzzzzz> -
What you see are lots of "user unknown" errors with "from=<>". +
What you see are lots of "user unknown" errors with "from=<>". These are error reports from MAILER-DAEMONs elsewhere on the Internet.
diff --git a/postfix/html/CDB_README.html b/postfix/html/CDB_README.html index a7e51350a..ba1c5e8d6 100644 --- a/postfix/html/CDB_README.html +++ b/postfix/html/CDB_README.html @@ -27,7 +27,7 @@ updates: no single-record inserts or deletes are supported. CDB databases can be modified only by rebuilding them completely from scratch, hence the "constant" qualifier in the name. -Postfix CDB databases are specified as "cdb:name", where +
Postfix CDB databases are specified as "cdb:name", where name specifies the CDB file name without the ".cdb" suffix (another suffix, ".tmp", is used temporarily while a CDB file is under construction). CDB databases are maintained with the postmap(1) diff --git a/postfix/html/DATABASE_README.html b/postfix/html/DATABASE_README.html index d28def720..642ce78c4 100644 --- a/postfix/html/DATABASE_README.html +++ b/postfix/html/DATABASE_README.html @@ -267,7 +267,7 @@ without the ".db" suffix.
Beware: 30s is not a lot for applications that do a lot of DNS +
Beware: 30s may be too short for applications doing lots of DNS lookups. However, if you increase the above timeouts too much, remote SMTP clients may hang up and mail may be delivered multiple times. This is an inherent problem with before-queue filtering.
@@ -624,7 +624,7 @@ TOContent filters may break domain key etc. signatures. If you +
Content filters may break DKIM etc. signatures. If you
use an SMTP-based content filter, then you should add a line to
master.cf with "-o disable_mime_output_conversion=yes" (note: no
spaces around the "="), as described in the
- And they may insert a message header with "unknown-msgid" like
-this: And they may insert an ugly message header with "unknown-msgid"
+like this: This document describes the symptoms of Postfix SMTP server
+overload, and how to avoid the condition under normal conditions.
+When the condition is caused by botnets or other malware, the
+document suggests configuration settings that help to minimize the
+impact on legitimate mail. Finally, the document introduces Postfix
+stress-adaptive behavior, and how it can be used to automatically
+switch configuration settings under overload. Topics covered in this document: Under normal conditions, Postfix responds immediately when a
+remote SMTP client connects. The time needed to deliver mail to
+Postfix may depend on how busy the CPU or disk are, but that should
+be noticeable only with very large messages. Performance degrades
+more dramatically when the number of remote SMTP clients exceeds
+the number of Postfix SMTP server processes. When a client connects
+while all server processes are busy, the client must wait until a
+server process becomes available. Overload may be caused by a legitimate mail (example: a DNS
+registrar opens a new zone for registrations), by mistake (mail
+explosion caused by a forwarding loop) or by illegitimate mail (worm
+outbreak, botnet, or other malware activity). Symptoms of Postfix
+SMTP mail server overload are: Postfix logs a warning that all server ports are busy: Remote SMTP clients experience a long delay before Postfix
+sends the "220 hostname.example.com ESMTP Postfix" greeting. If
+this affects end-user mail clients, enable the "submission" service
+entry in master.cf (present since Postfix 2.1), and tell users to
+connect to this instead of the public SMTP service. The Postfix SMTP server logs an increased number of "lost
+connection after CONNECT" events. This happens because remote SMTP
+clients disconnect before Postfix answers the connection. NOTE: The last two symptoms also happen without overload. Broken DNS also causes lengthy delays before "220
+hostname.example.com
+..." while the Postfix SMTP server tries to look up the client's
+hostname. A portscan for open SMTP ports also results in "lost
+connection ..." logfile messages. Legitimate mail that doesn't get through during an episode of
+overload is not necessarily lost. It should still arrive once the
+situation returns to normal, as long as the overload condition is
+temporary. To service more SMTP clients simultaneously, you need to increase
+the number of SMTP server processes. This will improve the
+responsiveness for remote SMTP clients, as long as the server machine
+has enough hardware and software resources to run the additional
+processes, and as long as the file system can keep up with the
+additional load. You increase the number of SMTP server processes either
+by increasing the default_process_limit in main.cf (line 3 below),
+or by increasing the SMTP server's "maxproc" field in master.cf
+(line 10 below). Either way, you need to issue a "postfix reload"
+command to make the change effective. Process limits above 1000 require Postfix version 2.4 or
+later, and an operating system that supports kernel-based event
+filters (BSD kqueue(2), Linux epoll(4), or Solaris /dev/poll).
+ You can reduce the Postfix memory footprint by using cdb:
+lookup tables instead of Berkeley DB. NOTE: older versions of the SMTPD_POLICY_README document
+contain a mistake: they configure a fixed number of policy daemon
+processes. When you raise the SMTP server's "maxproc" field in
+master.cf, SMTP server processes will report problems when connecting
+to policy server processes, because there aren't enough of them.
+Examples of errors are "connection refused" or "operation timed
+out". To fix, edit master.cf and specify a zero "maxproc" field
+in all policy server entries; see line 6 in the example below.
+Issue a "postfix reload" command to make the change effective. When increasing the number of SMTP server processes is not
+practical, you can improve Postfix server responsiveness by eliminating
+unnecessary work. When Postfix spends less time per SMTP session, the
+same number of SMTP server processes can service more clients in
+the same amount of time. Eliminate non-functional RBL lookups (blocklists that are
+no longer in operation). These lookups can degrade performance.
+Postfix logs a warning when an RBL server does not respond. Eliminate redundant RBL lookups (people often use multiple
+Spamhaus RBLs that include each other). To find out whether RBLs
+include other RBLs, look up the websites that document the RBL's
+policies. Eliminate header_checks and body_checks, and keep just a few
+emergency patterns to block the latest worm explosion or backscatter
+mail. See BACKSCATTER_README for examples of the latter.
+
+ Group your header_checks and body_checks patterns to avoid
+unnecessary pattern matching operations.
+
+ Under conditions of overload you can improve Postfix SMTP server
+responsiveness by hanging up on suspicious clients, so that other
+clients get a chance to talk to Postfix. Use "421" reply codes for botnet-related RBLs or for
+selected non-RBL restrictions. This causes Postfix 2.3 and later
+to disconnect immediately without waiting for the remote SMTP
+client to send a QUIT command. You can set individual reject codes for RBLs, and for individual
+responses from a specific RBL. We'll use zen.spamhaus.org as an
+example; by the time you read this document, details may have
+changed. Right now, their documents say that a response of 127.0.0.10
+or 127.0.0.11 indicates a dynamic client IP address, which means
+that the machine is probably running a bot of some kind. To give
+a 421 response instead of the default 554 response, use something
+like: Although the above shows three RBL lookups (lines 4-6), Postfix
+will still only do a single DNS query, so the performance difference
+is negligible. The down-side of sending 421 instead of the default 554 is that
+it works only for zombies and other malware. If the client is running
+a real MTA, then it may connect again several times until the mail
+expires in its queue. When this is a problem, stick with the default
+554 reply, and use "smtpd_hard_error_limit = 1" as described below.
+ With Postfix 2.5, or with earlier releases that contain the
+stress-adaptive behavior patch, you can turn on the above under
+overload by replacing line 8 with: More information about automatic stress-adaptive behavior is
+at the end of this document. The following measures will still allow most legitimate
+clients to connect and send mail, but may affect some legitimate
+clients. Reduce smtpd_timeout (default: 300s). Experience on the
+postfix-users list from a variety of sysadmins shows that reducing
+the "normal" smtpd_timeout to 60s is unlikely to affect legitimate
+clients. However, it is unlikely to become the Postfix default
+because it's not RFC compliant. Setting smtpd_timeout to 10s (line
+2 below) or even 5s under stress will still allow most
+legitimate clients to connect and send mail, but may delay mail
+from some clients. No mail should be lost, as long as this measure
+is used only temporarily. Reduce smtpd_hard_error_limit (default: 20). Setting this
+to 1 under stress (line 3 below) helps by disconnecting clients
+after a single error, giving other clients a chance to connect.
+However, this may cause significant delays with legitimate mail,
+such as a mailing list that contains a few no-longer-active user
+names that didn't bother to unsubscribe. No mail should be lost,
+as long as this measure is used only temporarily. Disable remote SMTP client hostname lookups, so that all
+SMTP client hostnames become "unknown" (line 5 below). This feature
+was introduced with Postfix 2.3. Unfortunately, this measure is
+more problematic than the other ones proposed sofar. First, this
+will result in loss of mail when you use hostname-based access rules
+that reject mail from "unknown" SMTP clients (examples:
+reject_unknown_client_hostname, reject_unknown_reverse_client_hostname).
+Second, this may result in loss of mail when you subject "unknown"
+SMTP clients to additional restrictions such as reject_unverified_sender.
+ Except with the last measure, no mail should be lost, as long
+as these measures are used only temporarily. The next section of
+this document introduces a way to automate this process. Postfix version 2.5 introduces automatic stress-adaptive behavior.
+This is also available as an add-on patch for Postfix versions 2.4
+and 2.3 from the mirrors listed at http://www.postfix.org/download.html.
+ It works as follows. When a "public" network service runs into
+an "all server ports are busy" condition, the master(8) daemon logs
+a warning, restarts the service (without interrupting existing
+network sessions), and runs the service with "-o stress=yes" on the
+command line. Normally, it runs a stress-adaptive service with "-o
+stress=" on the command line (i.e. with an empty parameter value).
+Other services never have "-o stress" parameters on the command
+line, including services that listen on a loopback interface only.
+ The stress pseudo-parameter value is the key to making main.cf
+parameter settings stress adaptive: Translation:
+
+ Line 2: under conditions of stress, use an smtpd_timeout
+value of 10 seconds instead of the default 300 seconds,
+
+ Line 3: under conditions of stress, use an smtpd_hard_error_limit
+of 1 instead of the default 20. The syntax of ${name?value} and ${name:value} is explained at
+the beginning of the postconf(5) manual page. NOTE: Please keep in mind that the stress-adaptive feature is
+a fairly desperate measure to keep some legitimate mail
+flowing under overload conditions. If a site is reaching the SMTP
+server process limit when there isn't an attack or bot flood
+occurring, then either the process limit needs to be raised or more
+hardware needs to be added. To find out if your Postfix installation supports stress-adaptive
+behavior, use the "ps" command, and look for the smtpd processes.
+Postfix has stress-adaptive support when you see "-o stress=" or
+"-o stress=yes" command-line options. Remember that Postfix never
+enables stress-adaptive behavior on servers that listen on local
+addresses only. The following example is for FreeBSD or Linux. On Solaris, HP-UX
+and other System-V flavors, use "ps -ef" instead of "ps ax". You can't use postconf(1) to detect stress-adaptive support.
+The postconf(1) command ignores the existence of the stress parameter
+in main.cf, because the parameter has no effect there. Command-line
+"-o parameter" settings always take precedence over main.cf parameter
+settings.
+
+ If you configure stress-adaptive behavior in main.cf when it
+isn't supported, nothing bad will happen. The processes will run
+as if the stress parameter always has an empty value. You can manually force stress-adaptive behavior on, by adding
+a "-o stress=yes" command-line option in master.cf. This can be
+useful for testing overrides on the SMTP service. Issue "postfix
+reload" to make the change effective. To permanently force stress-adaptive behavior off with a specific
+service, specify "-o stress=" on its command line. This may be
+desirable for the "submission" service. Issue "postfix reload" to
+make the change effective.
diff --git a/postfix/html/STRESS_README.html b/postfix/html/STRESS_README.html
new file mode 100644
index 000000000..b63ebd23d
--- /dev/null
+++ b/postfix/html/STRESS_README.html
@@ -0,0 +1,469 @@
+
+
+
+
+
+
+
+
+
Postfix
+Stress-Dependent Configuration
+
+Overview
+
+
+
+
+
+ Symptoms of Postfix SMTP server overload
+
+
+
+
+
+
+Oct 3 20:39:27 spike postfix/master[28905]: warning: service "smtp"
+ (25) has reached its process limit "30": new clients may experience
+ noticeable delays
+Oct 3 20:39:27 spike postfix/master[28905]: warning: to avoid this
+ condition, increase the process count in master.cf or reduce the
+ service time per client
+
+
+
+
+
+
+ Service more SMTP clients at the same time
+
+
+
+
+
+
+ 1 /etc/postfix/main.cf:
+ 2 # Raise the global process limit, 100 since Postfix 2.0.
+ 3 default_process_limit = 200
+ 4
+ 5 /etc/postfix/master.cf:
+ 6 # =============================================================
+ 7 # service type private unpriv chroot wakeup maxproc command
+ 8 # =============================================================
+ 9 # Raise the SMTP service process limit only.
+10 smtp inet n - n - 200 smtpd
+
+
+
+1 /etc/postfix/master.cf:
+2 # =============================================================
+3 # service type private unpriv chroot wakeup maxproc command
+4 # =============================================================
+5 # Disable the policy service process limit.
+6 policy unix - n n - 0 spawn
+7 user=nobody argv=/some/where/policy-server
+
+
+ Spend less time per SMTP client
+
+
+
+
+
+
+ 1 /etc/postfix/header_checks:
+ 2 if /^Subject:/
+ 3 /^Subject: virus found in mail from you/ reject
+ 4 /^Subject: ..../ ....
+ 5 endif
+ 6
+ 7 if /^Received:/
+ 8 /^Received: from (postfix\.org) / reject forged client name in received header: $1
+ 9 /^Received: from .../ ....
+10 endif
+
+
+ Disconnect suspicious SMTP clients
+
+
+
+
+
+
+ 1 /etc/postfix/main.cf:
+ 2 smtpd_client_restrictions =
+ 3 permit_mynetworks
+ 4 reject_rbl_client zen.spamhaus.org=127.0.0.10
+ 5 reject_rbl_client zen.spamhaus.org=127.0.0.11
+ 6 reject_rbl_client zen.spamhaus.org
+ 7
+ 8 rbl_reply_maps = hash:/etc/postfix/rbl_reply_maps
+ 9
+10 /etc/postfix/rbl_reply_maps:
+11 zen.spamhaus.org=127.0.0.10 421 4.7.1 Service unavailable;
+12 $rbl_class [$rbl_what] blocked using
+13 $rbl_domain${rbl_reason?; $rbl_reason}
+14
+15 zen.spamhaus.org=127.0.0.11 421 4.7.1 Service unavailable;
+16 $rbl_class [$rbl_what] blocked using
+17 $rbl_domain${rbl_reason?; $rbl_reason}
+
+
+
+ 8 rbl_reply_maps = ${stress?hash:/etc/postfix/rbl_reply_maps}
+
+
+ Take desperate measures
+
+
+
+
+
+
+
+
+
+1 /etc/postfix/main.cf:
+2 smtpd_timeout = 10
+3 smtpd_hard_error_limit = 1
+4 # Caution: line 5 may trigger REJECTs by hostname-based access rules
+5 smtpd_peername_lookup = no
+
+ Make Postfix behavior stress-adaptive
+
+
+
+
+
+1 /etc/postfix/main.cf:
+2 smtpd_timeout = ${stress?10}${stress:300}
+3 smtpd_hard_error_limit = ${stress?1}${stress:20}
+
+
+
+
+
+ Detecting support for stress-adaptive behavior
+
+
+
+
+
+$ ps ax|grep smtpd
+83326 ?? S 0:00.28 smtpd -n smtp -t inet -u -c -o stress=
+84345 ?? Ss 0:00.11 /usr/bin/perl /usr/libexec/postfix/smtpd-policy.pl
+
+ Forcing stress-adaptive behavior on or off
+
+
+
+
+
+1 /etc/postfix/master.cf:
+2 # =============================================================
+3 # service type private unpriv chroot wakeup maxproc command
+4 # =============================================================
+5 #
+6 smtp inet n - n - - smtpd
+7 -o stress=yes
+8 -o . . .
+
+
+
+
+
+1 /etc/postfix/master.cf:
+2 # =============================================================
+3 # service type private unpriv chroot wakeup maxproc command
+4 # =============================================================
+5 #
+6 submission inet n - n - - smtpd
+7 -o stress=
+8 -o . . .
+
+ Credits
+
+
+
+
+
+
diff --git a/postfix/html/header_checks.5.html b/postfix/html/header_checks.5.html
index f0f68aa0e..f11df058e 100644
--- a/postfix/html/header_checks.5.html
+++ b/postfix/html/header_checks.5.html
@@ -93,19 +93,19 @@ HEADER_CHECKS(5) HEADER_CHECKS(5)
respectively.
/pattern/flags action
- When pattern matches the input string, execute the
- corresponding action. See below for a list of pos-
- sible actions.
+ When /pattern/ matches the input string, execute
+ the corresponding action. See below for a list of
+ possible actions.
!/pattern/flags action
- When pattern does not match the input string, exe-
- cute the corresponding action.
+ When /pattern/ does not match the input string,
+ execute the corresponding action.
if /pattern/flags
endif Match the input string against the patterns between
if and endif, if and only if the same input string
- also matches pattern. The if..endif can nest.
+ also matches /pattern/. The if..endif can nest.
Note: do not prepend whitespace to patterns inside
if..endif.
@@ -114,7 +114,7 @@ HEADER_CHECKS(5) HEADER_CHECKS(5)
endif Match the input string against the patterns between
if and endif, if and only if the same input string
- does not match pattern. The if..endif can nest.
+ does not match /pattern/. The if..endif can nest.
blank lines and comments
Empty lines and whitespace-only lines are ignored,
@@ -292,26 +292,31 @@ HEADER_CHECKS(5) HEADER_CHECKS(5)
a pattern before applying more drastic actions.
BUGS
- Many people overlook the main limitations of header and
+ Empty lines never match, because some map types mis-behave
+ when given a zero-length search string. This limitation
+ may be removed for regular expression tables in a future
+ release.
+
+ Many people overlook the main limitations of header and
body_checks rules.
- o These rules operate on one logical message header
+ o These rules operate on one logical message header
or one body line at a time. A decision made for one
line is not carried over to the next line.
- o If text in the message body is encoded (RFC 2045)
+ o If text in the message body is encoded (RFC 2045)
then the rules need to be specified for the encoded
form.
- o Likewise, when message headers are encoded (RFC
- 2047) then the rules need to be specified for the
+ o Likewise, when message headers are encoded (RFC
+ 2047) then the rules need to be specified for the
encoded form.
- Message headers added by the cleanup(8) daemon itself are
+ Message headers added by the cleanup(8) daemon itself are
excluded from inspection. Examples of such message headers
are From:, To:, Message-ID:, Date:.
- Message headers deleted by the cleanup(8) daemon will be
+ Message headers deleted by the cleanup(8) daemon will be
examined before they are deleted. Examples are: Bcc:, Con-
tent-Length:, Return-Path:.
@@ -319,11 +324,11 @@ HEADER_CHECKS(5) HEADER_CHECKS(5)
body_checks
Lookup tables with content filter rules for message
body lines. These filters see one physical line at
- a time, in chunks of at most $line_length_limit
+ a time, in chunks of at most $line_length_limit
bytes.
body_checks_size_limit
- The amount of content per message body segment
+ The amount of content per message body segment
(attachment) that is subjected to $body_checks fil-
tering.
@@ -333,32 +338,32 @@ HEADER_CHECKS(5) HEADER_CHECKS(5)
nested_header_checks (default: $header_checks)
Lookup tables with content filter rules for message
- header lines: respectively, these are applied to
- the initial message headers (not including MIME
- headers), to the MIME headers anywhere in the mes-
- sage, and to the initial headers of attached mes-
+ header lines: respectively, these are applied to
+ the initial message headers (not including MIME
+ headers), to the MIME headers anywhere in the mes-
+ sage, and to the initial headers of attached mes-
sages.
- Note: these filters see one logical message header
- at a time, even when a message header spans multi-
- ple lines. Message headers that are longer than
+ Note: these filters see one logical message header
+ at a time, even when a message header spans multi-
+ ple lines. Message headers that are longer than
$header_size_limit characters are truncated.
disable_mime_input_processing
- While receiving mail, give no special treatment to
- MIME related message headers; all text after the
+ While receiving mail, give no special treatment to
+ MIME related message headers; all text after the
initial message headers is considered to be part of
- the message body. This means that header_checks is
- applied to all the initial message headers, and
+ the message body. This means that header_checks is
+ applied to all the initial message headers, and
that body_checks is applied to the remainder of the
message.
- Note: when used in this manner, body_checks will
- process a multi-line message header one line at a
+ Note: when used in this manner, body_checks will
+ process a multi-line message header one line at a
time.
EXAMPLES
- Header pattern to block attachments with bad file name
+ Header pattern to block attachments with bad file name
extensions.
/etc/postfix/main.cf:
@@ -396,7 +401,7 @@ HEADER_CHECKS(5) HEADER_CHECKS(5)
BACKSCATTER_README, blocking returned forged mail
LICENSE
- The Secure Mailer license must be distributed with this
+ The Secure Mailer license must be distributed with this
software.
AUTHOR(S)
diff --git a/postfix/html/index.html b/postfix/html/index.html
index 4107fc0e1..b2e51c61e 100644
--- a/postfix/html/index.html
+++ b/postfix/html/index.html
@@ -54,6 +54,8 @@ configuration examples
Restricted body_checks(5) tables for the Postfix SMTP client. +These tables are searched while mail is being delivered. Actions +that change the delivery time or destination are not available. +
+ +This feature is available in Postfix 2.5 and later.
+ +This feature is available in Postfix 2.2 and later.
+ + +Restricted header_checks(5) tables for the Postfix SMTP client. +These tables are searched while mail is being delivered. Actions +that change the delivery time or destination are not available. +
+ +This feature is available in Postfix 2.5 and later.
+ +Restricted mime_header_checks(5) tables for the Postfix SMTP +client. These tables are searched while mail is being delivered. +Actions that change the delivery time or destination are not +available.
+ +This feature is available in Postfix 2.5 and later.
+ +This feature is available in Postfix 2.1 and later.
+ + +Restricted nested_header_checks(5) tables for the Postfix SMTP +client. These tables are searched while mail is being delivered. +Actions that change the delivery time or destination are not +available.
+ +This feature is available in Postfix 2.5 and later.
+ +This document describes features that require Postfix version +2.0 or later.
Topics covered in this document:
@@ -56,6 +53,11 @@ scanners +The examples use Perl Compatible Regular Expressions (Postfix +pcre: tables), but also provide a translation to POSIX regular +expressions (Postfix regexp: tables). PCRE is preferred primarily +because the implementation is often faster.
+When a spammer or worm sends mail with forged sender addresses, @@ -73,7 +75,7 @@ to=<yyyyyy@your.domain.here> proto=ESMTP helo=<zzzzzz> -
What you see are lots of "user unknown" errors with "from=<>". +
What you see are lots of "user unknown" errors with "from=<>". These are error reports from MAILER-DAEMONs elsewhere on the Internet.
diff --git a/postfix/proto/MILTER_README.html b/postfix/proto/MILTER_README.html index edef90fde..a28675041 100644 --- a/postfix/proto/MILTER_README.html +++ b/postfix/proto/MILTER_README.html @@ -501,7 +501,7 @@ EOH, BODY, EOM -Beware: 30s is not a lot for applications that do a lot of DNS +
Beware: 30s may be too short for applications doing lots of DNS lookups. However, if you increase the above timeouts too much, remote SMTP clients may hang up and mail may be delivered multiple times. This is an inherent problem with before-queue filtering.
@@ -624,7 +624,7 @@ TOContent filters may break domain key etc. signatures. If you +
Content filters may break DKIM etc. signatures. If you
use an SMTP-based content filter, then you should add a line to
master.cf with "-o disable_mime_output_conversion=yes" (note: no
spaces around the "="), as described in the
- And they may insert a message header with "unknown-msgid" like
-this: And they may insert an ugly message header with "unknown-msgid"
+like this: This document describes the symptoms of Postfix SMTP server
+overload, and how to avoid the condition under normal conditions.
+When the condition is caused by botnets or other malware, the
+document suggests configuration settings that help to minimize the
+impact on legitimate mail. Finally, the document introduces Postfix
+stress-adaptive behavior, and how it can be used to automatically
+switch configuration settings under overload. Topics covered in this document: Under normal conditions, Postfix responds immediately when a
+remote SMTP client connects. The time needed to deliver mail to
+Postfix may depend on how busy the CPU or disk are, but that should
+be noticeable only with very large messages. Performance degrades
+more dramatically when the number of remote SMTP clients exceeds
+the number of Postfix SMTP server processes. When a client connects
+while all server processes are busy, the client must wait until a
+server process becomes available. Overload may be caused by a legitimate mail (example: a DNS
+registrar opens a new zone for registrations), by mistake (mail
+explosion caused by a forwarding loop) or by illegitimate mail (worm
+outbreak, botnet, or other malware activity). Symptoms of Postfix
+SMTP mail server overload are: Postfix logs a warning that all server ports are busy: Remote SMTP clients experience a long delay before Postfix
+sends the "220 hostname.example.com ESMTP Postfix" greeting. If
+this affects end-user mail clients, enable the "submission" service
+entry in master.cf (present since Postfix 2.1), and tell users to
+connect to this instead of the public SMTP service. The Postfix SMTP server logs an increased number of "lost
+connection after CONNECT" events. This happens because remote SMTP
+clients disconnect before Postfix answers the connection. NOTE: The last two symptoms also happen without overload. Broken DNS also causes lengthy delays before "220
+hostname.example.com
+..." while the Postfix SMTP server tries to look up the client's
+hostname. A portscan for open SMTP ports also results in "lost
+connection ..." logfile messages. Legitimate mail that doesn't get through during an episode of
+overload is not necessarily lost. It should still arrive once the
+situation returns to normal, as long as the overload condition is
+temporary. To service more SMTP clients simultaneously, you need to increase
+the number of SMTP server processes. This will improve the
+responsiveness for remote SMTP clients, as long as the server machine
+has enough hardware and software resources to run the additional
+processes, and as long as the file system can keep up with the
+additional load. You increase the number of SMTP server processes either
+by increasing the default_process_limit in main.cf (line 3 below),
+or by increasing the SMTP server's "maxproc" field in master.cf
+(line 10 below). Either way, you need to issue a "postfix reload"
+command to make the change effective. Process limits above 1000 require Postfix version 2.4 or
+later, and an operating system that supports kernel-based event
+filters (BSD kqueue(2), Linux epoll(4), or Solaris /dev/poll).
+ You can reduce the Postfix memory footprint by using cdb:
+lookup tables instead of Berkeley DB. NOTE: older versions of the SMTPD_POLICY_README document
+contain a mistake: they configure a fixed number of policy daemon
+processes. When you raise the SMTP server's "maxproc" field in
+master.cf, SMTP server processes will report problems when connecting
+to policy server processes, because there aren't enough of them.
+Examples of errors are "connection refused" or "operation timed
+out". To fix, edit master.cf and specify a zero "maxproc" field
+in all policy server entries; see line 6 in the example below.
+Issue a "postfix reload" command to make the change effective. When increasing the number of SMTP server processes is not
+practical, you can improve Postfix server responsiveness by eliminating
+unnecessary work. When Postfix spends less time per SMTP session, the
+same number of SMTP server processes can service more clients in
+the same amount of time. Eliminate non-functional RBL lookups (blocklists that are
+no longer in operation). These lookups can degrade performance.
+Postfix logs a warning when an RBL server does not respond. Eliminate redundant RBL lookups (people often use multiple
+Spamhaus RBLs that include each other). To find out whether RBLs
+include other RBLs, look up the websites that document the RBL's
+policies. Eliminate header_checks and body_checks, and keep just a few
+emergency patterns to block the latest worm explosion or backscatter
+mail. See BACKSCATTER_README for examples of the latter.
+
+ Group your header_checks and body_checks patterns to avoid
+unnecessary pattern matching operations.
+
+ Under conditions of overload you can improve Postfix SMTP server
+responsiveness by hanging up on suspicious clients, so that other
+clients get a chance to talk to Postfix. Use "421" reply codes for botnet-related RBLs or for
+selected non-RBL restrictions. This causes Postfix 2.3 and later
+to disconnect immediately without waiting for the remote SMTP
+client to send a QUIT command. You can set individual reject codes for RBLs, and for individual
+responses from a specific RBL. We'll use zen.spamhaus.org as an
+example; by the time you read this document, details may have
+changed. Right now, their documents say that a response of 127.0.0.10
+or 127.0.0.11 indicates a dynamic client IP address, which means
+that the machine is probably running a bot of some kind. To give
+a 421 response instead of the default 554 response, use something
+like: Although the above shows three RBL lookups (lines 4-6), Postfix
+will still only do a single DNS query, so the performance difference
+is negligible. The down-side of sending 421 instead of the default 554 is that
+it works only for zombies and other malware. If the client is running
+a real MTA, then it may connect again several times until the mail
+expires in its queue. When this is a problem, stick with the default
+554 reply, and use "smtpd_hard_error_limit = 1" as described below.
+ With Postfix 2.5, or with earlier releases that contain the
+stress-adaptive behavior patch, you can turn on the above under
+overload by replacing line 8 with: More information about automatic stress-adaptive behavior is
+at the end of this document. The following measures will still allow most legitimate
+clients to connect and send mail, but may affect some legitimate
+clients. Reduce smtpd_timeout (default: 300s). Experience on the
+postfix-users list from a variety of sysadmins shows that reducing
+the "normal" smtpd_timeout to 60s is unlikely to affect legitimate
+clients. However, it is unlikely to become the Postfix default
+because it's not RFC compliant. Setting smtpd_timeout to 10s (line
+2 below) or even 5s under stress will still allow most
+legitimate clients to connect and send mail, but may delay mail
+from some clients. No mail should be lost, as long as this measure
+is used only temporarily. Reduce smtpd_hard_error_limit (default: 20). Setting this
+to 1 under stress (line 3 below) helps by disconnecting clients
+after a single error, giving other clients a chance to connect.
+However, this may cause significant delays with legitimate mail,
+such as a mailing list that contains a few no-longer-active user
+names that didn't bother to unsubscribe. No mail should be lost,
+as long as this measure is used only temporarily. Disable remote SMTP client hostname lookups, so that all
+SMTP client hostnames become "unknown" (line 5 below). This feature
+was introduced with Postfix 2.3. Unfortunately, this measure is
+more problematic than the other ones proposed sofar. First, this
+will result in loss of mail when you use hostname-based access rules
+that reject mail from "unknown" SMTP clients (examples:
+reject_unknown_client_hostname, reject_unknown_reverse_client_hostname).
+Second, this may result in loss of mail when you subject "unknown"
+SMTP clients to additional restrictions such as reject_unverified_sender.
+ Except with the last measure, no mail should be lost, as long
+as these measures are used only temporarily. The next section of
+this document introduces a way to automate this process. Postfix version 2.5 introduces automatic stress-adaptive behavior.
+This is also available as an add-on patch for Postfix versions 2.4
+and 2.3 from the mirrors listed at http://www.postfix.org/download.html.
+ It works as follows. When a "public" network service runs into
+an "all server ports are busy" condition, the master(8) daemon logs
+a warning, restarts the service (without interrupting existing
+network sessions), and runs the service with "-o stress=yes" on the
+command line. Normally, it runs a stress-adaptive service with "-o
+stress=" on the command line (i.e. with an empty parameter value).
+Other services never have "-o stress" parameters on the command
+line, including services that listen on a loopback interface only.
+ The stress pseudo-parameter value is the key to making main.cf
+parameter settings stress adaptive: Translation:
+
+ Line 2: under conditions of stress, use an smtpd_timeout
+value of 10 seconds instead of the default 300 seconds,
+
+ Line 3: under conditions of stress, use an smtpd_hard_error_limit
+of 1 instead of the default 20. The syntax of ${name?value} and ${name:value} is explained at
+the beginning of the postconf(5) manual page. NOTE: Please keep in mind that the stress-adaptive feature is
+a fairly desperate measure to keep some legitimate mail
+flowing under overload conditions. If a site is reaching the SMTP
+server process limit when there isn't an attack or bot flood
+occurring, then either the process limit needs to be raised or more
+hardware needs to be added. To find out if your Postfix installation supports stress-adaptive
+behavior, use the "ps" command, and look for the smtpd processes.
+Postfix has stress-adaptive support when you see "-o stress=" or
+"-o stress=yes" command-line options. Remember that Postfix never
+enables stress-adaptive behavior on servers that listen on local
+addresses only. The following example is for FreeBSD or Linux. On Solaris, HP-UX
+and other System-V flavors, use "ps -ef" instead of "ps ax". You can't use postconf(1) to detect stress-adaptive support.
+The postconf(1) command ignores the existence of the stress parameter
+in main.cf, because the parameter has no effect there. Command-line
+"-o parameter" settings always take precedence over main.cf parameter
+settings.
+
+ If you configure stress-adaptive behavior in main.cf when it
+isn't supported, nothing bad will happen. The processes will run
+as if the stress parameter always has an empty value. You can manually force stress-adaptive behavior on, by adding
+a "-o stress=yes" command-line option in master.cf. This can be
+useful for testing overrides on the SMTP service. Issue "postfix
+reload" to make the change effective. To permanently force stress-adaptive behavior off with a specific
+service, specify "-o stress=" on its command line. This may be
+desirable for the "submission" service. Issue "postfix reload" to
+make the change effective.
diff --git a/postfix/proto/Makefile.in b/postfix/proto/Makefile.in
index e2e855b78..ec7e0d6ae 100644
--- a/postfix/proto/Makefile.in
+++ b/postfix/proto/Makefile.in
@@ -34,6 +34,7 @@ HTML = ../html/ADDRESS_CLASS_README.html \
../html/SMTPD_POLICY_README.html \
../html/SMTPD_PROXY_README.html \
../html/STANDARD_CONFIGURATION_README.html \
+ ../html/STRESS_README.html \
../html/TLS_README.html ../html/TLS_LEGACY_README.html \
../html/TUNING_README.html \
../html/UUCP_README.html \
@@ -69,6 +70,7 @@ README = ../README_FILES/ADDRESS_CLASS_README \
../README_FILES/SMTPD_ACCESS_README \
../README_FILES/SMTPD_POLICY_README ../README_FILES/SMTPD_PROXY_README \
../README_FILES/STANDARD_CONFIGURATION_README \
+ ../README_FILES/STRESS_README \
../README_FILES/TLS_README ../README_FILES/TLS_LEGACY_README \
../README_FILES/TUNING_README \
../README_FILES/UUCP_README \
@@ -235,6 +237,9 @@ clobber:
../html/STANDARD_CONFIGURATION_README.html: STANDARD_CONFIGURATION_README.html
$(POSTLINK) $? >$@
+../html/STRESS_README.html: STRESS_README.html
+ $(POSTLINK) $? >$@
+
../html/TUNING_README.html: TUNING_README.html
$(POSTLINK) $? >$@
@@ -376,6 +381,9 @@ clobber:
../README_FILES/STANDARD_CONFIGURATION_README: STANDARD_CONFIGURATION_README.html
$(HT2READ) $? >$@
+../README_FILES/STRESS_README: STRESS_README.html
+ $(HT2READ) $? >$@
+
../README_FILES/TUNING_README: TUNING_README.html
$(HT2READ) $? >$@
diff --git a/postfix/proto/STRESS_README.html b/postfix/proto/STRESS_README.html
new file mode 100644
index 000000000..a313a0d7e
--- /dev/null
+++ b/postfix/proto/STRESS_README.html
@@ -0,0 +1,469 @@
+
+
+
+
+
+
+
+
+
Postfix
+Stress-Dependent Configuration
+
+Overview
+
+
+
+
+
+ Symptoms of Postfix SMTP server overload
+
+
+
+
+
+
+Oct 3 20:39:27 spike postfix/master[28905]: warning: service "smtp"
+ (25) has reached its process limit "30": new clients may experience
+ noticeable delays
+Oct 3 20:39:27 spike postfix/master[28905]: warning: to avoid this
+ condition, increase the process count in master.cf or reduce the
+ service time per client
+
+
+
+
+
+
+ Service more SMTP clients at the same time
+
+
+
+
+
+
+ 1 /etc/postfix/main.cf:
+ 2 # Raise the global process limit, 100 since Postfix 2.0.
+ 3 default_process_limit = 200
+ 4
+ 5 /etc/postfix/master.cf:
+ 6 # =============================================================
+ 7 # service type private unpriv chroot wakeup maxproc command
+ 8 # =============================================================
+ 9 # Raise the SMTP service process limit only.
+10 smtp inet n - n - 200 smtpd
+
+
+
+1 /etc/postfix/master.cf:
+2 # =============================================================
+3 # service type private unpriv chroot wakeup maxproc command
+4 # =============================================================
+5 # Disable the policy service process limit.
+6 policy unix - n n - 0 spawn
+7 user=nobody argv=/some/where/policy-server
+
+
+ Spend less time per SMTP client
+
+
+
+
+
+
+ 1 /etc/postfix/header_checks:
+ 2 if /^Subject:/
+ 3 /^Subject: virus found in mail from you/ reject
+ 4 /^Subject: ..../ ....
+ 5 endif
+ 6
+ 7 if /^Received:/
+ 8 /^Received: from (postfix\.org) / reject forged client name in received header: $1
+ 9 /^Received: from .../ ....
+10 endif
+
+
+ Disconnect suspicious SMTP clients
+
+
+
+
+
+
+ 1 /etc/postfix/main.cf:
+ 2 smtpd_client_restrictions =
+ 3 permit_mynetworks
+ 4 reject_rbl_client zen.spamhaus.org=127.0.0.10
+ 5 reject_rbl_client zen.spamhaus.org=127.0.0.11
+ 6 reject_rbl_client zen.spamhaus.org
+ 7
+ 8 rbl_reply_maps = hash:/etc/postfix/rbl_reply_maps
+ 9
+10 /etc/postfix/rbl_reply_maps:
+11 zen.spamhaus.org=127.0.0.10 421 4.7.1 Service unavailable;
+12 $rbl_class [$rbl_what] blocked using
+13 $rbl_domain${rbl_reason?; $rbl_reason}
+14
+15 zen.spamhaus.org=127.0.0.11 421 4.7.1 Service unavailable;
+16 $rbl_class [$rbl_what] blocked using
+17 $rbl_domain${rbl_reason?; $rbl_reason}
+
+
+
+ 8 rbl_reply_maps = ${stress?hash:/etc/postfix/rbl_reply_maps}
+
+
+ Take desperate measures
+
+
+
+
+
+
+
+
+
+1 /etc/postfix/main.cf:
+2 smtpd_timeout = 10
+3 smtpd_hard_error_limit = 1
+4 # Caution: line 5 may trigger REJECTs by hostname-based access rules
+5 smtpd_peername_lookup = no
+
+ Make Postfix behavior stress-adaptive
+
+
+
+
+
+1 /etc/postfix/main.cf:
+2 smtpd_timeout = ${stress?10}${stress:300}
+3 smtpd_hard_error_limit = ${stress?1}${stress:20}
+
+
+
+
+
+ Detecting support for stress-adaptive behavior
+
+
+
+
+
+$ ps ax|grep smtpd
+83326 ?? S 0:00.28 smtpd -n smtp -t inet -u -c -o stress=
+84345 ?? Ss 0:00.11 /usr/bin/perl /usr/libexec/postfix/smtpd-policy.pl
+
+ Forcing stress-adaptive behavior on or off
+
+
+
+
+
+1 /etc/postfix/master.cf:
+2 # =============================================================
+3 # service type private unpriv chroot wakeup maxproc command
+4 # =============================================================
+5 #
+6 smtp inet n - n - - smtpd
+7 -o stress=yes
+8 -o . . .
+
+
+
+
+
+1 /etc/postfix/master.cf:
+2 # =============================================================
+3 # service type private unpriv chroot wakeup maxproc command
+4 # =============================================================
+5 #
+6 submission inet n - n - - smtpd
+7 -o stress=
+8 -o . . .
+
+ Credits
+
+
+
+
+
+
diff --git a/postfix/proto/header_checks b/postfix/proto/header_checks
index ae73e1df0..33f428e5f 100644
--- a/postfix/proto/header_checks
+++ b/postfix/proto/header_checks
@@ -81,17 +81,17 @@
# For a discussion of specific pattern or flags syntax,
# see \fBpcre_table\fR(5) or \fBregexp_table\fR(5), respectively.
# .IP "\fB/\fIpattern\fB/\fIflags action\fR"
-# When \fIpattern\fR matches the input string, execute
+# When /\fIpattern\fR/ matches the input string, execute
# the corresponding \fIaction\fR. See below for a list
# of possible actions.
# .IP "\fB!/\fIpattern\fB/\fIflags action\fR"
-# When \fIpattern\fR does \fBnot\fR match the input string,
+# When /\fIpattern\fR/ does \fBnot\fR match the input string,
# execute the corresponding \fIaction\fR.
# .IP "\fBif /\fIpattern\fB/\fIflags\fR"
# .IP "\fBendif\fR"
# Match the input string against the patterns between \fBif\fR
# and \fBendif\fR, if and only if the same input string also
-# matches \fIpattern\fR. The \fBif\fR..\fBendif\fR can nest.
+# matches /\fIpattern\fR/. The \fBif\fR..\fBendif\fR can nest.
# .sp
# Note: do not prepend whitespace to patterns inside
# \fBif\fR..\fBendif\fR.
@@ -99,7 +99,7 @@
# .IP "\fBendif\fR"
# Match the input string against the patterns between \fBif\fR
# and \fBendif\fR, if and only if the same input string does
-# \fBnot\fR match \fIpattern\fR. The \fBif\fR..\fBendif\fR
+# \fBnot\fR match /\fIpattern\fR/. The \fBif\fR..\fBendif\fR
# can nest.
# .IP "blank lines and comments"
# Empty lines and whitespace-only lines are ignored, as
@@ -281,6 +281,10 @@
# action is useful for debugging and for testing a pattern
# before applying more drastic actions.
# BUGS
+# Empty lines never match, because some map types mis-behave
+# when given a zero-length search string. This limitation may
+# be removed for regular expression tables in a future release.
+#
# Many people overlook the main limitations of header and body_checks
# rules.
# .IP \(bu
diff --git a/postfix/proto/postconf.proto b/postfix/proto/postconf.proto
index bc5ed79b4..ce8fda2f1 100644
--- a/postfix/proto/postconf.proto
+++ b/postfix/proto/postconf.proto
@@ -2360,6 +2360,11 @@ and later.
This feature is available in Postfix 2.5 and later.
+ +%PARAM smtp_header_checks + +Restricted header_checks(5) tables for the Postfix SMTP client. +These tables are searched while mail is being delivered. Actions +that change the delivery time or destination are not available. +
+ +This feature is available in Postfix 2.5 and later.
+ +%PARAM smtp_mime_header_checks + +Restricted mime_header_checks(5) tables for the Postfix SMTP +client. These tables are searched while mail is being delivered. +Actions that change the delivery time or destination are not +available.
+ +This feature is available in Postfix 2.5 and later.
+ +%PARAM smtp_nested_header_checks + +Restricted nested_header_checks(5) tables for the Postfix SMTP +client. These tables are searched while mail is being delivered. +Actions that change the delivery time or destination are not +available.
+ +This feature is available in Postfix 2.5 and later.
+ +%PARAM smtp_body_checks + +Restricted body_checks(5) tables for the Postfix SMTP client. +These tables are searched while mail is being delivered. Actions +that change the delivery time or destination are not available. +
+ +This feature is available in Postfix 2.5 and later.
diff --git a/postfix/src/global/Makefile.in b/postfix/src/global/Makefile.in index 881440680..d077f3dc6 100644 --- a/postfix/src/global/Makefile.in +++ b/postfix/src/global/Makefile.in @@ -28,7 +28,7 @@ SRCS = abounce.c anvil_clnt.c been_here.c bounce.c bounce_log.c \ tok822_resolve.c tok822_rewrite.c tok822_tree.c trace.c \ user_acl.c valid_mailhost_addr.c verify.c verify_clnt.c \ verp_sender.c wildcard_inet_addr.c xtext.c delivered_hdr.c \ - fold_addr.c + fold_addr.c header_body_checks.c OBJS = abounce.o anvil_clnt.o been_here.o bounce.o bounce_log.o \ canon_addr.o cfg_parser.o cleanup_strerror.o cleanup_strflags.o \ clnt_stream.o conv_time.o db_common.o debug_peer.o debug_process.o \ @@ -58,7 +58,7 @@ OBJS = abounce.o anvil_clnt.o been_here.o bounce.o bounce_log.o \ tok822_resolve.o tok822_rewrite.o tok822_tree.o trace.o \ user_acl.o valid_mailhost_addr.o verify.o verify_clnt.o \ verp_sender.o wildcard_inet_addr.o xtext.o delivered_hdr.o \ - fold_addr.o + fold_addr.o header_body_checks.o HDRS = abounce.h anvil_clnt.h been_here.h bounce.h bounce_log.h \ canon_addr.h cfg_parser.h cleanup_user.h clnt_stream.h config.h \ conv_time.h db_common.h debug_peer.h debug_process.h defer.h \ @@ -82,7 +82,7 @@ HDRS = abounce.h anvil_clnt.h been_here.h bounce.h bounce_log.h \ string_list.h strip_addr.h sys_exits.h timed_ipc.h tok822.h \ trace.h user_acl.h valid_mailhost_addr.h verify.h verify_clnt.h \ verp_sender.h wildcard_inet_addr.h xtext.h delivered_hdr.h \ - fold_addr.h + fold_addr.h header_body_checks.h TESTSRC = rec2stream.c stream2rec.c recdump.c DEFS = -I. -I$(INC_DIR) -D$(SYSTYPE) CFLAGS = $(DEBUG) $(OPT) $(DEFS) @@ -94,7 +94,7 @@ TESTPROG= domain_list dot_lockfile mail_addr_crunch mail_addr_find \ resolve_local rewrite_clnt stream2rec string_list tok822_parse \ quote_821_local mail_conf_time mime_state strip_addr \ verify_clnt xtext anvil_clnt scache ehlo_mask \ - valid_mailhost_addr own_inet_addr + valid_mailhost_addr own_inet_addr header_body_checks LIBS = ../../lib/libutil.a LIB_DIR = ../../lib @@ -271,13 +271,20 @@ valid_mailhost_addr: valid_mailhost_addr.c $(LIB) $(LIBS) own_inet_addr: own_inet_addr.c $(LIB) $(LIBS) $(CC) $(CFLAGS) -DTEST -o $@ $@.c $(LIB) $(LIBS) $(SYSLIBS) +header_body_checks: header_body_checks.c $(LIB) $(LIBS) + $(CC) $(CFLAGS) -DTEST -o $@ $@.c $(LIB) $(LIBS) $(SYSLIBS) + tests: tok822_test mime_tests strip_addr_test tok822_limit_test \ xtext_test scache_multi_test ehlo_mask_test \ - namadr_list_test mail_conf_time_test + namadr_list_test mail_conf_time_test header_body_checks_tests mime_tests: mime_test mime_nest mime_8bit mime_dom mime_trunc mime_cvt \ mime_cvt2 mime_cvt3 mime_garb1 mime_garb2 mime_garb3 mime_garb4 +header_body_checks_tests: header_body_checks_null_test \ + header_body_checks_warn_test header_body_checks_prepend_test \ + header_body_checks_ignore_test header_body_checks_replace_test + root_tests: rewrite_clnt_test resolve_clnt_test tok822_test: tok822_parse tok822_parse.in tok822_parse.ref @@ -416,6 +423,56 @@ mail_conf_time_test: mail_conf_time mail_conf_time.ref diff mail_conf_time.ref mail_conf_time.tmp rm -f mail_conf_time.tmp +header_body_checks_null_test: header_body_checks header_body_checks_null.ref + ./header_body_checks "" "" "" "" \ +