From: Wietse Venema Date: Sun, 11 Nov 2007 05:00:00 +0000 (-0500) Subject: postfix-2.5-20071111 X-Git-Tag: v2.5.0-RC1~26 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=8130775303736274ac01825a0a4c36f5bb9ab833;p=thirdparty%2Fpostfix.git postfix-2.5-20071111 --- diff --git a/postfix/.indent.pro b/postfix/.indent.pro index 7fde1b246..96fd99799 100644 --- a/postfix/.indent.pro +++ b/postfix/.indent.pro @@ -99,6 +99,11 @@ -TEXPAND_ATTR -TFILE -TFORWARD_INFO +-THBC_OUTPUT_CALL_BACKS +-THBC_ACTION_CALL_BACKS +-THBC_CHECKS +-THBC_MAP_INFO +-THBC_TEST_CONTEXT -THEADER_OPTS -THEADER_TOKEN -THOST diff --git a/postfix/HISTORY b/postfix/HISTORY index 76ef2cb7f..59019d5b2 100644 --- a/postfix/HISTORY +++ b/postfix/HISTORY @@ -13708,10 +13708,11 @@ Apologies for any names omitted. 20070911 - Bugfix (introduced Postfix 2.2.11): certificate with - unparsable canonical name caused the SMTPD policy client - to allocate zero-length memory, triggering an assertion - that it shouldn't do such things. File: smtpd/smtpd_check.c. + Bugfix (introduced Postfix 2.2.11): TLS client certificate + with unparsable canonical name caused the SMTP server's + policy client to allocate zero-length memory, triggering + an assertion that it shouldn't do such things. File: + smtpd/smtpd_check.c. 20070912 @@ -13771,3 +13772,44 @@ Apologies for any names omitted. Cleanup: send client port information "0" instead of "unknown" to Milter applications. Files: smtpd/smtpd.c, smtpd/smtpd_milter.c, cleanup/cleanup_milter.c. + +20071025 + + Portability: on Linux we no longer need /proc to find out + local IPv6 interface address information. LaMont Jones. + Files: util/sys_defs.h. + +20071030 + + Bugfix: Postfix mistakenly enforced the 64kbyte limit (for + sending body parts TO Milter applications) also while + receiving packets FROM Milter applications. The limit is + now at least 1GB. File: milter/milter8.c. + +20071105 + + Feature: ORIGINAL_RECIPIENT environment variable. Corey + Hickey. File: local/local.c. + +20071108-10 + + Feature: general-purpose header/body_checks library module, + first used in the SMTP client. Actions that change the + message delivery time or destination can be implemented + with a simple extension mechanism (they make sense only in + before-queue filters). Configuration parameters: + smtp_header_checks, smtp_mime_header_checks, + smtp_nested_header_checks, smtp_body_checks. Unlike the + cleanup server, the mime and nested header checks don't by + default assume the header_checks value. Files: + global/header_body_checks.[hc], smtp/smtp_proto.c, + smtp/smtp_session.c. + +20071110 + + Feature: ${original_recipient} command-line macro. Corey + Hickey. File: pipe/pipe.c. + + Bugfix (introduced: 20071004) missing exception handling + in smtp-sink per-command delay feature. Victor Duchovni. + File: smtpstone/smtp-sink.c. diff --git a/postfix/README_FILES/AAAREADME b/postfix/README_FILES/AAAREADME index 44e307ca7..fb5d0a1d0 100644 --- a/postfix/README_FILES/AAAREADME +++ b/postfix/README_FILES/AAAREADME @@ -16,6 +16,7 @@ GGeenneerraall ccoonnffiigguurraattiioonn PPrroobblleemm ssoollvviinngg * QSHAPE_README: Bottleneck analysis + * STRESS_README: Stress-dependent configuration * TUNING_README: Performance tuning * DEBUG_README: Debugging strategies diff --git a/postfix/README_FILES/BACKSCATTER_README b/postfix/README_FILES/BACKSCATTER_README index 90a16bfa2..5a7db47e3 100644 --- a/postfix/README_FILES/BACKSCATTER_README +++ b/postfix/README_FILES/BACKSCATTER_README @@ -4,11 +4,7 @@ PPoossttffiixx BBaacckkssccaatttteerr HHoowwttoo OOvveerrvviieeww -This document describes features that require Postfix version 2.0 or later. The -examples use Perl Compatible Regular Expressions (Postfix pcre: tables), but -also provide a translation to POSIX regular expressions (Postfix regexp: -tables). PCRE is preferred primarily because the implementation is often -faster. +This document describes features that require Postfix version 2.0 or later. Topics covered in this document: @@ -21,6 +17,11 @@ Topics covered in this document: o Blocking backscatter mail with other forged information o Blocking backscatter mail from virus scanners +The examples use Perl Compatible Regular Expressions (Postfix pcre: tables), +but also provide a translation to POSIX regular expressions (Postfix regexp: +tables). PCRE is preferred primarily because the implementation is often +faster. + WWhhaatt iiss bbaacckkssccaatttteerr mmaaiill?? When a spammer or worm sends mail with forged sender addresses, innocent sites diff --git a/postfix/README_FILES/MILTER_README b/postfix/README_FILES/MILTER_README index 6f0ca58e3..39e50336f 100644 --- a/postfix/README_FILES/MILTER_README +++ b/postfix/README_FILES/MILTER_README @@ -300,7 +300,7 @@ message). |milter_content_timeout|300s |HEADER, EOH, BODY, EOM | |_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _|_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | -Beware: 30s is not a lot for applications that do a lot of DNS lookups. +Beware: 30s may be too short for applications doing lots of DNS lookups. However, if you increase the above timeouts too much, remote SMTP clients may hang up and mail may be delivered multiple times. This is an inherent problem with before-queue filtering. @@ -390,7 +390,7 @@ message). WWoorrkkaarroouunnddss -Content filters may break domain key etc. signatures. If you use an SMTP-based +Content filters may break DKIM etc. signatures. If you use an SMTP-based content filter, then you should add a line to master.cf with "- o disable_mime_output_conversion=yes" (note: no spaces around the "="), as described in the advanced content filter example. @@ -407,7 +407,7 @@ Milter applications make assumptions that aren't true in a Postfix environment. sid-filter[36540]: WARNING: sendmail symbol 'i' not available - And they may insert a message header with "unknown-msgid" like this: + And they may insert an ugly message header with "unknown-msgid" like this: X-SenderID: Sendmail Sender-ID Filter vx.y.z host.example.com diff --git a/postfix/README_FILES/STRESS_README b/postfix/README_FILES/STRESS_README new file mode 100644 index 000000000..b66be0224 --- /dev/null +++ b/postfix/README_FILES/STRESS_README @@ -0,0 +1,349 @@ +PPoossttffiixx SSttrreessss--DDeeppeennddeenntt CCoonnffiigguurraattiioonn + +------------------------------------------------------------------------------- + +OOvveerrvviieeww + +This document describes the symptoms of Postfix SMTP server overload, and how +to avoid the condition under normal conditions. When the condition is caused by +botnets or other malware, the document suggests configuration settings that +help to minimize the impact on legitimate mail. Finally, the document +introduces Postfix stress-adaptive behavior, and how it can be used to +automatically switch configuration settings under overload. + +Topics covered in this document: + + * Symptoms of Postfix SMTP server overload + * Service more SMTP clients at the same time + * Spend less time per SMTP client + * Disconnect suspicious SMTP clients + * Take desperate measures + * Make Postfix behavior stress-adaptive + * Detecting support for stress-adaptive behavior + * Forcing stress-adaptive behavior on or off + * Credits + +SSyymmppttoommss ooff PPoossttffiixx SSMMTTPP sseerrvveerr oovveerrllooaadd + +Under normal conditions, Postfix responds immediately when a remote SMTP client +connects. The time needed to deliver mail to Postfix may depend on how busy the +CPU or disk are, but that should be noticeable only with very large messages. +Performance degrades more dramatically when the number of remote SMTP clients +exceeds the number of Postfix SMTP server processes. When a client connects +while all server processes are busy, the client must wait until a server +process becomes available. + +Overload may be caused by a legitimate mail (example: a DNS registrar opens a +new zone for registrations), by mistake (mail explosion caused by a forwarding +loop) or by illegitimate mail (worm outbreak, botnet, or other malware +activity). Symptoms of Postfix SMTP mail server overload are: + + * Postfix logs a warning that all server ports are busy: + + Oct 3 20:39:27 spike postfix/master[28905]: warning: service "smtp" + (25) has reached its process limit "30": new clients may experience + noticeable delays + Oct 3 20:39:27 spike postfix/master[28905]: warning: to avoid this + condition, increase the process count in master.cf or reduce the + service time per client + + * Remote SMTP clients experience a long delay before Postfix sends the "220 + hostname.example.com ESMTP Postfix" greeting. If this affects end-user mail + clients, enable the "submission" service entry in master.cf (present since + Postfix 2.1), and tell users to connect to this instead of the public SMTP + service. + + * The Postfix SMTP server logs an increased number of "lost connection after + CONNECT" events. This happens because remote SMTP clients disconnect before + Postfix answers the connection. + +NOTE: The last two symptoms also happen without overload. + + * Broken DNS also causes lengthy delays before "220 hostname.example.com ..." + while the Postfix SMTP server tries to look up the client's hostname. + + * A portscan for open SMTP ports also results in "lost connection ..." + logfile messages. + +Legitimate mail that doesn't get through during an episode of overload is not +necessarily lost. It should still arrive once the situation returns to normal, +as long as the overload condition is temporary. + +SSeerrvviiccee mmoorree SSMMTTPP cclliieennttss aatt tthhee ssaammee ttiimmee + +To service more SMTP clients simultaneously, you need to increase the number of +SMTP server processes. This will improve the responsiveness for remote SMTP +clients, as long as the server machine has enough hardware and software +resources to run the additional processes, and as long as the file system can +keep up with the additional load. + + * You increase the number of SMTP server processes either by increasing the + default_process_limit in main.cf (line 3 below), or by increasing the SMTP + server's "maxproc" field in master.cf (line 10 below). Either way, you need + to issue a "postfix reload" command to make the change effective. + + * Process limits above 1000 require Postfix version 2.4 or later, and an + operating system that supports kernel-based event filters (BSD kqueue(2), + Linux epoll(4), or Solaris /dev/poll). + + * You can reduce the Postfix memory footprint by using cdb: lookup tables + instead of Berkeley DB. + + 1 /etc/postfix/main.cf: + 2 # Raise the global process limit, 100 since Postfix 2.0. + 3 default_process_limit = 200 + 4 + 5 /etc/postfix/master.cf: + 6 # ============================================================= + 7 # service type private unpriv chroot wakeup maxproc command + 8 # ============================================================= + 9 # Raise the SMTP service process limit only. + 10 smtp inet n - n - 200 smtpd + + * NOTE: older versions of the SMTPD_POLICY_README document contain a mistake: + they configure a fixed number of policy daemon processes. When you raise + the SMTP server's "maxproc" field in master.cf, SMTP server processes will + report problems when connecting to policy server processes, because there + aren't enough of them. Examples of errors are "connection refused" or + "operation timed out". To fix, edit master.cf and specify a zero "maxproc" + field in all policy server entries; see line 6 in the example below. Issue + a "postfix reload" command to make the change effective. + + 1 /etc/postfix/master.cf: + 2 # ============================================================= + 3 # service type private unpriv chroot wakeup maxproc command + 4 # ============================================================= + 5 # Disable the policy service process limit. + 6 policy unix - n n - 0 spawn + 7 user=nobody argv=/some/where/policy-server + +SSppeenndd lleessss ttiimmee ppeerr SSMMTTPP cclliieenntt + +When increasing the number of SMTP server processes is not practical, you can +improve Postfix server responsiveness by eliminating unnecessary work. When +Postfix spends less time per SMTP session, the same number of SMTP server +processes can service more clients in the same amount of time. + + * Eliminate non-functional RBL lookups (blocklists that are no longer in + operation). These lookups can degrade performance. Postfix logs a warning + when an RBL server does not respond. + + * Eliminate redundant RBL lookups (people often use multiple Spamhaus RBLs + that include each other). To find out whether RBLs include other RBLs, look + up the websites that document the RBL's policies. + + * Eliminate header_checks and body_checks, and keep just a few emergency + patterns to block the latest worm explosion or backscatter mail. See + BACKSCATTER_README for examples of the latter. + + * Group your header_checks and body_checks patterns to avoid unnecessary + pattern matching operations. + + 1 /etc/postfix/header_checks: + 2 if /^Subject:/ + 3 /^Subject: virus found in mail from you/ reject + 4 /^Subject: ..../ .... + 5 endif + 6 + 7 if /^Received:/ + 8 /^Received: from (postfix\.org) / reject forged client name in + received header: $1 + 9 /^Received: from .../ .... + 10 endif + +DDiissccoonnnneecctt ssuussppiicciioouuss SSMMTTPP cclliieennttss + +Under conditions of overload you can improve Postfix SMTP server responsiveness +by hanging up on suspicious clients, so that other clients get a chance to talk +to Postfix. + + * Use "421" reply codes for botnet-related RBLs or for selected non-RBL + restrictions. This causes Postfix 2.3 and later to disconnect immediately + without waiting for the remote SMTP client to send a QUIT command. + + You can set individual reject codes for RBLs, and for individual responses + from a specific RBL. We'll use zen.spamhaus.org as an example; by the time + you read this document, details may have changed. Right now, their + documents say that a response of 127.0.0.10 or 127.0.0.11 indicates a + dynamic client IP address, which means that the machine is probably running + a bot of some kind. To give a 421 response instead of the default 554 + response, use something like: + + 1 /etc/postfix/main.cf: + 2 smtpd_client_restrictions = + 3 permit_mynetworks + 4 reject_rbl_client zen.spamhaus.org=127.0.0.10 + 5 reject_rbl_client zen.spamhaus.org=127.0.0.11 + 6 reject_rbl_client zen.spamhaus.org + 7 + 8 rbl_reply_maps = hash:/etc/postfix/rbl_reply_maps + 9 + 10 /etc/postfix/rbl_reply_maps: + 11 zen.spamhaus.org=127.0.0.10 421 4.7.1 Service unavailable; + 12 $rbl_class [$rbl_what] blocked using + 13 $rbl_domain${rbl_reason?; $rbl_reason} + 14 + 15 zen.spamhaus.org=127.0.0.11 421 4.7.1 Service unavailable; + 16 $rbl_class [$rbl_what] blocked using + 17 $rbl_domain${rbl_reason?; $rbl_reason} + + Although the above shows three RBL lookups (lines 4-6), Postfix will still + only do a single DNS query, so the performance difference is negligible. + + The down-side of sending 421 instead of the default 554 is that it works + only for zombies and other malware. If the client is running a real MTA, + then it may connect again several times until the mail expires in its + queue. When this is a problem, stick with the default 554 reply, and use + "smtpd_hard_error_limit = 1" as described below. + + With Postfix 2.5, or with earlier releases that contain the stress-adaptive + behavior patch, you can turn on the above under overload by replacing line + 8 with: + + 8 rbl_reply_maps = ${stress?hash:/etc/postfix/rbl_reply_maps} + + More information about automatic stress-adaptive behavior is at the end of + this document. + +TTaakkee ddeessppeerraattee mmeeaassuurreess + +The following measures will still allow mmoosstt legitimate clients to connect and +send mail, but may affect some legitimate clients. + + * Reduce smtpd_timeout (default: 300s). Experience on the postfix-users list + from a variety of sysadmins shows that reducing the "normal" smtpd_timeout + to 60s is unlikely to affect legitimate clients. However, it is unlikely to + become the Postfix default because it's not RFC compliant. Setting + smtpd_timeout to 10s (line 2 below) or even 5s under stress will still + allow mmoosstt legitimate clients to connect and send mail, but may delay mail + from some clients. No mail should be lost, as long as this measure is used + only temporarily. + + * Reduce smtpd_hard_error_limit (default: 20). Setting this to 1 under stress + (line 3 below) helps by disconnecting clients after a single error, giving + other clients a chance to connect. However, this may cause significant + delays with legitimate mail, such as a mailing list that contains a few no- + longer-active user names that didn't bother to unsubscribe. No mail should + be lost, as long as this measure is used only temporarily. + + * Disable remote SMTP client hostname lookups, so that all SMTP client + hostnames become "unknown" (line 5 below). This feature was introduced with + Postfix 2.3. Unfortunately, this measure is more problematic than the other + ones proposed sofar. First, this will result in loss of mail when you use + hostname-based access rules that reject mail from "unknown" SMTP clients + (examples: reject_unknown_client_hostname, + reject_unknown_reverse_client_hostname). Second, this may result in loss of + mail when you subject "unknown" SMTP clients to additional restrictions + such as reject_unverified_sender. + + 1 /etc/postfix/main.cf: + 2 smtpd_timeout = 10 + 3 smtpd_hard_error_limit = 1 + 4 # Caution: line 5 may trigger REJECTs by hostname-based access rules + + 5 smtpd_peername_lookup = no + +Except with the last measure, no mail should be lost, as long as these measures +are used only temporarily. The next section of this document introduces a way +to automate this process. + +MMaakkee PPoossttffiixx bbeehhaavviioorr ssttrreessss--aaddaappttiivvee + +Postfix version 2.5 introduces automatic stress-adaptive behavior. This is also +available as an add-on patch for Postfix versions 2.4 and 2.3 from the mirrors +listed at http://www.postfix.org/download.html. + +It works as follows. When a "public" network service runs into an "all server +ports are busy" condition, the master(8) daemon logs a warning, restarts the +service (without interrupting existing network sessions), and runs the service +with "-o stress=yes" on the command line. Normally, it runs a stress-adaptive +service with "-o stress=" on the command line (i.e. with an empty parameter +value). Other services never have "-o stress" parameters on the command line, +including services that listen on a loopback interface only. + +The stress pseudo-parameter value is the key to making main.cf parameter +settings stress adaptive: + + 1 /etc/postfix/main.cf: + 2 smtpd_timeout = ${stress?10}${stress:300} + 3 smtpd_hard_error_limit = ${stress?1}${stress:20} + +Translation: + + * Line 2: under conditions of stress, use an smtpd_timeout value of 10 + seconds instead of the default 300 seconds, + + * Line 3: under conditions of stress, use an smtpd_hard_error_limit of 1 + instead of the default 20. + +The syntax of ${name?value} and ${name:value} is explained at the beginning of +the postconf(5) manual page. + +NOTE: Please keep in mind that the stress-adaptive feature is a fairly +desperate measure to keep ssoommee legitimate mail flowing under overload +conditions. If a site is reaching the SMTP server process limit when there +isn't an attack or bot flood occurring, then either the process limit needs to +be raised or more hardware needs to be added. + +DDeetteeccttiinngg ssuuppppoorrtt ffoorr ssttrreessss--aaddaappttiivvee bbeehhaavviioorr + +To find out if your Postfix installation supports stress-adaptive behavior, use +the "ps" command, and look for the smtpd processes. Postfix has stress-adaptive +support when you see "-o stress=" or "-o stress=yes" command-line options. +Remember that Postfix never enables stress-adaptive behavior on servers that +listen on local addresses only. + +The following example is for FreeBSD or Linux. On Solaris, HP-UX and other +System-V flavors, use "ps -ef" instead of "ps ax". + + $ ps ax|grep smtpd + 83326 ?? S 0:00.28 smtpd -n smtp -t inet -u -c -o stress= + 84345 ?? Ss 0:00.11 /usr/bin/perl /usr/libexec/postfix/smtpd- + policy.pl + +You can't use postconf(1) to detect stress-adaptive support. The postconf(1) +command ignores the existence of the stress parameter in main.cf, because the +parameter has no effect there. Command-line "-o parameter" settings always take +precedence over main.cf parameter settings. + +If you configure stress-adaptive behavior in main.cf when it isn't supported, +nothing bad will happen. The processes will run as if the stress parameter +always has an empty value. + +FFoorrcciinngg ssttrreessss--aaddaappttiivvee bbeehhaavviioorr oonn oorr ooffff + +You can manually force stress-adaptive behavior on, by adding a "-o stress=yes" +command-line option in master.cf. This can be useful for testing overrides on +the SMTP service. Issue "postfix reload" to make the change effective. + + 1 /etc/postfix/master.cf: + 2 # ============================================================= + 3 # service type private unpriv chroot wakeup maxproc command + 4 # ============================================================= + 5 # + 6 smtp inet n - n - - smtpd + 7 -o stress=yes + 8 -o . . . + +To permanently force stress-adaptive behavior off with a specific service, +specify "-o stress=" on its command line. This may be desirable for the +"submission" service. Issue "postfix reload" to make the change effective. + + 1 /etc/postfix/master.cf: + 2 # ============================================================= + 3 # service type private unpriv chroot wakeup maxproc command + 4 # ============================================================= + 5 # + 6 submission inet n - n - - smtpd + 7 -o stress= + 8 -o . . . + +CCrreeddiittss + + * Thanks to the postfix-users mailing list members for sharing early + experiences with the stress-adaptive feature. + * The RBL example and several other paragraphs of text were adapted from + postfix-users postings by Noel Jones. + * Wietse implemented stress-adaptive behavior as the smallest possible patch + while he should be working on other things. + diff --git a/postfix/RELEASE_NOTES b/postfix/RELEASE_NOTES index 7a6526d90..62f58225d 100644 --- a/postfix/RELEASE_NOTES +++ b/postfix/RELEASE_NOTES @@ -17,6 +17,15 @@ Incompatibility with Postfix 2.3 and earlier If you upgrade from Postfix 2.3 or earlier, read RELEASE_NOTES-2.4 before proceeding. +Major changes with Postfix snapshot 20071110 +============================================ + +Header/body checks are now available in the SMTP client, after the +implementation was moved from the cleanup server to a library module. +The SMTP client provides only actions that don't change the message +delivery time or destination: warn, replace, prepend, ignore, dunno, +ok. + Major changes with Postfix snapshot 20070911 ============================================ diff --git a/postfix/conf/header_checks b/postfix/conf/header_checks index 5ae99a5b7..5b1aaad04 100644 --- a/postfix/conf/header_checks +++ b/postfix/conf/header_checks @@ -87,19 +87,19 @@ # respectively. # # /pattern/flags action -# When pattern matches the input string, execute the -# corresponding action. See below for a list of pos- -# sible actions. +# When /pattern/ matches the input string, execute +# the corresponding action. See below for a list of +# possible actions. # # !/pattern/flags action -# When pattern does not match the input string, exe- -# cute the corresponding action. +# When /pattern/ does not match the input string, +# execute the corresponding action. # # if /pattern/flags # # endif Match the input string against the patterns between # if and endif, if and only if the same input string -# also matches pattern. The if..endif can nest. +# also matches /pattern/. The if..endif can nest. # # Note: do not prepend whitespace to patterns inside # if..endif. @@ -108,7 +108,7 @@ # # endif Match the input string against the patterns between # if and endif, if and only if the same input string -# does not match pattern. The if..endif can nest. +# does not match /pattern/. The if..endif can nest. # # blank lines and comments # Empty lines and whitespace-only lines are ignored, @@ -286,26 +286,31 @@ # a pattern before applying more drastic actions. # # BUGS -# Many people overlook the main limitations of header and +# Empty lines never match, because some map types mis-behave +# when given a zero-length search string. This limitation +# may be removed for regular expression tables in a future +# release. +# +# Many people overlook the main limitations of header and # body_checks rules. # -# o These rules operate on one logical message header +# o These rules operate on one logical message header # or one body line at a time. A decision made for one # line is not carried over to the next line. # -# o If text in the message body is encoded (RFC 2045) +# o If text in the message body is encoded (RFC 2045) # then the rules need to be specified for the encoded # form. # -# o Likewise, when message headers are encoded (RFC -# 2047) then the rules need to be specified for the +# o Likewise, when message headers are encoded (RFC +# 2047) then the rules need to be specified for the # encoded form. # -# Message headers added by the cleanup(8) daemon itself are +# Message headers added by the cleanup(8) daemon itself are # excluded from inspection. Examples of such message headers # are From:, To:, Message-ID:, Date:. # -# Message headers deleted by the cleanup(8) daemon will be +# Message headers deleted by the cleanup(8) daemon will be # examined before they are deleted. Examples are: Bcc:, Con- # tent-Length:, Return-Path:. # @@ -313,11 +318,11 @@ # body_checks # Lookup tables with content filter rules for message # body lines. These filters see one physical line at -# a time, in chunks of at most $line_length_limit +# a time, in chunks of at most $line_length_limit # bytes. # # body_checks_size_limit -# The amount of content per message body segment +# The amount of content per message body segment # (attachment) that is subjected to $body_checks fil- # tering. # @@ -327,32 +332,32 @@ # # nested_header_checks (default: $header_checks) # Lookup tables with content filter rules for message -# header lines: respectively, these are applied to -# the initial message headers (not including MIME -# headers), to the MIME headers anywhere in the mes- -# sage, and to the initial headers of attached mes- +# header lines: respectively, these are applied to +# the initial message headers (not including MIME +# headers), to the MIME headers anywhere in the mes- +# sage, and to the initial headers of attached mes- # sages. # -# Note: these filters see one logical message header -# at a time, even when a message header spans multi- -# ple lines. Message headers that are longer than +# Note: these filters see one logical message header +# at a time, even when a message header spans multi- +# ple lines. Message headers that are longer than # $header_size_limit characters are truncated. # # disable_mime_input_processing -# While receiving mail, give no special treatment to -# MIME related message headers; all text after the +# While receiving mail, give no special treatment to +# MIME related message headers; all text after the # initial message headers is considered to be part of -# the message body. This means that header_checks is -# applied to all the initial message headers, and +# the message body. This means that header_checks is +# applied to all the initial message headers, and # that body_checks is applied to the remainder of the # message. # -# Note: when used in this manner, body_checks will -# process a multi-line message header one line at a +# Note: when used in this manner, body_checks will +# process a multi-line message header one line at a # time. # # EXAMPLES -# Header pattern to block attachments with bad file name +# Header pattern to block attachments with bad file name # extensions. # # /etc/postfix/main.cf: @@ -384,7 +389,7 @@ # RFC 2047, message header encoding for non-ASCII text # # README FILES -# Use "postconf readme_directory" or "postconf html_direc- +# Use "postconf readme_directory" or "postconf html_direc- # tory" to locate this information. # DATABASE_README, Postfix lookup table overview # CONTENT_INSPECTION_README, Postfix content inspection overview @@ -392,7 +397,7 @@ # BACKSCATTER_README, blocking returned forged mail # # LICENSE -# The Secure Mailer license must be distributed with this +# The Secure Mailer license must be distributed with this # software. # # AUTHOR(S) diff --git a/postfix/conf/master.cf b/postfix/conf/master.cf index 1bdc00c1b..c1677f807 100644 --- a/postfix/conf/master.cf +++ b/postfix/conf/master.cf @@ -13,10 +13,12 @@ smtp inet n - n - - smtpd # -o smtpd_enforce_tls=yes # -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,reject +# -o milter_macro_daemon_name=ORIGINATING #smtps inet n - n - - smtpd # -o smtpd_tls_wrappermode=yes # -o smtpd_sasl_auth_enable=yes # -o smtpd_client_restrictions=permit_sasl_authenticated,reject +# -o milter_macro_daemon_name=ORIGINATING #628 inet n - n - - qmqpd pickup fifo n - n 60 1 pickup cleanup unix n - n - 0 cleanup diff --git a/postfix/conf/postfix-files b/postfix/conf/postfix-files index e637975c2..b63de9a35 100644 --- a/postfix/conf/postfix-files +++ b/postfix/conf/postfix-files @@ -262,6 +262,7 @@ $readme_directory/SMTPD_ACCESS_README:f:root:-:644 $readme_directory/SMTPD_POLICY_README:f:root:-:644 $readme_directory/SMTPD_PROXY_README:f:root:-:644 $readme_directory/STANDARD_CONFIGURATION_README:f:root:-:644 +$readme_directory/STRESS_README:f:root:-:644 $readme_directory/TLS_LEGACY_README:f:root:-:644 $readme_directory/TLS_README:f:root:-:644 $readme_directory/TUNING_README:f:root:-:644 @@ -309,6 +310,7 @@ $html_directory/SMTPD_ACCESS_README.html:f:root:-:644 $html_directory/SMTPD_POLICY_README.html:f:root:-:644 $html_directory/SMTPD_PROXY_README.html:f:root:-:644 $html_directory/STANDARD_CONFIGURATION_README.html:f:root:-:644 +$html_directory/STRESS_README.html:f:root:-:644 $html_directory/TLS_LEGACY_README.html:f:root:-:644 $html_directory/TLS_README.html:f:root:-:644 $html_directory/TUNING_README.html:f:root:-:644 diff --git a/postfix/html/BACKSCATTER_README.html b/postfix/html/BACKSCATTER_README.html index 2ef613f58..a10bc64fb 100644 --- a/postfix/html/BACKSCATTER_README.html +++ b/postfix/html/BACKSCATTER_README.html @@ -20,11 +20,8 @@ Backscatter Howto

Overview

-This document describes features that require Postfix version 2.0 -or later. The examples use Perl Compatible Regular Expressions -(Postfix pcre: tables), but also provide a translation to POSIX -regular expressions (Postfix regexp: tables). PCRE is preferred -primarily because the implementation is often faster.

+

This document describes features that require Postfix version +2.0 or later.

Topics covered in this document:

@@ -56,6 +53,11 @@ scanners +

The examples use Perl Compatible Regular Expressions (Postfix +pcre: tables), but also provide a translation to POSIX regular +expressions (Postfix regexp: tables). PCRE is preferred primarily +because the implementation is often faster.

+

What is backscatter mail?

When a spammer or worm sends mail with forged sender addresses, @@ -73,7 +75,7 @@ to=<yyyyyy@your.domain.here> proto=ESMTP helo=<zzzzzz> -

What you see are lots of "user unknown" errors with "from=<>". +

What you see are lots of "user unknown" errors with "from=<>". These are error reports from MAILER-DAEMONs elsewhere on the Internet.

diff --git a/postfix/html/CDB_README.html b/postfix/html/CDB_README.html index a7e51350a..ba1c5e8d6 100644 --- a/postfix/html/CDB_README.html +++ b/postfix/html/CDB_README.html @@ -27,7 +27,7 @@ updates: no single-record inserts or deletes are supported. CDB databases can be modified only by rebuilding them completely from scratch, hence the "constant" qualifier in the name.

-

Postfix CDB databases are specified as "cdb:name", where +

Postfix CDB databases are specified as "cdb:name", where name specifies the CDB file name without the ".cdb" suffix (another suffix, ".tmp", is used temporarily while a CDB file is under construction). CDB databases are maintained with the postmap(1) diff --git a/postfix/html/DATABASE_README.html b/postfix/html/DATABASE_README.html index d28def720..642ce78c4 100644 --- a/postfix/html/DATABASE_README.html +++ b/postfix/html/DATABASE_README.html @@ -267,7 +267,7 @@ without the ".db" suffix.

A read-optimized structure with no support for incremental updates. Database files are created with the postmap(1) or postalias(1) command. -The lookup table name as used in "cdb:table" is the database file name +The lookup table name as used in "cdb:table" is the database file name without the ".cdb" suffix. This feature is available with Postfix 2.2 and later.
diff --git a/postfix/html/MILTER_README.html b/postfix/html/MILTER_README.html index 4236a3bc6..d5b6a09ef 100644 --- a/postfix/html/MILTER_README.html +++ b/postfix/html/MILTER_README.html @@ -501,7 +501,7 @@ EOH, BODY, EOM -

Beware: 30s is not a lot for applications that do a lot of DNS +

Beware: 30s may be too short for applications doing lots of DNS lookups. However, if you increase the above timeouts too much, remote SMTP clients may hang up and mail may be delivered multiple times. This is an inherent problem with before-queue filtering.

@@ -624,7 +624,7 @@ TO

Workarounds

-

Content filters may break domain key etc. signatures. If you +

Content filters may break DKIM etc. signatures. If you use an SMTP-based content filter, then you should add a line to master.cf with "-o disable_mime_output_conversion=yes" (note: no spaces around the "="), as described in the -

And they may insert a message header with "unknown-msgid" like -this:

+

And they may insert an ugly message header with "unknown-msgid" +like this:

diff --git a/postfix/html/STRESS_README.html b/postfix/html/STRESS_README.html
new file mode 100644
index 000000000..b63ebd23d
--- /dev/null
+++ b/postfix/html/STRESS_README.html
@@ -0,0 +1,469 @@
+
+
+
+
+
+
+Postfix Stress-Dependent Configuration
+
+
+
+
+
+
+
+

Postfix +Stress-Dependent Configuration

+ +
+ +

Overview

+ +

This document describes the symptoms of Postfix SMTP server +overload, and how to avoid the condition under normal conditions. +When the condition is caused by botnets or other malware, the +document suggests configuration settings that help to minimize the +impact on legitimate mail. Finally, the document introduces Postfix +stress-adaptive behavior, and how it can be used to automatically +switch configuration settings under overload.

+ +

Topics covered in this document:

+ +
+ +

Symptoms of Postfix SMTP server overload

+ +

Under normal conditions, Postfix responds immediately when a +remote SMTP client connects. The time needed to deliver mail to +Postfix may depend on how busy the CPU or disk are, but that should +be noticeable only with very large messages. Performance degrades +more dramatically when the number of remote SMTP clients exceeds +the number of Postfix SMTP server processes. When a client connects +while all server processes are busy, the client must wait until a +server process becomes available.

+ +

Overload may be caused by a legitimate mail (example: a DNS +registrar opens a new zone for registrations), by mistake (mail +explosion caused by a forwarding loop) or by illegitimate mail (worm +outbreak, botnet, or other malware activity). Symptoms of Postfix +SMTP mail server overload are:

+ +
    + +
  • Postfix logs a warning that all server ports are busy:

    + +
    +Oct  3 20:39:27 spike postfix/master[28905]: warning: service "smtp"
    + (25) has reached its process limit "30": new clients may experience
    + noticeable delays
    +Oct  3 20:39:27 spike postfix/master[28905]: warning: to avoid this
    + condition, increase the process count in master.cf or reduce the
    + service time per client
    +
    + +
  • Remote SMTP clients experience a long delay before Postfix +sends the "220 hostname.example.com ESMTP Postfix" greeting. If +this affects end-user mail clients, enable the "submission" service +entry in master.cf (present since Postfix 2.1), and tell users to +connect to this instead of the public SMTP service.

    + +
  • The Postfix SMTP server logs an increased number of "lost +connection after CONNECT" events. This happens because remote SMTP +clients disconnect before Postfix answers the connection.

    + +
+ +

NOTE: The last two symptoms also happen without overload.

+ +
    + +
  • Broken DNS also causes lengthy delays before "220 +hostname.example.com +..." while the Postfix SMTP server tries to look up the client's +hostname.

    + +
  • A portscan for open SMTP ports also results in "lost +connection ..." logfile messages.

    + +
+ +

Legitimate mail that doesn't get through during an episode of +overload is not necessarily lost. It should still arrive once the +situation returns to normal, as long as the overload condition is +temporary.

+ +

Service more SMTP clients at the same time

+ +

To service more SMTP clients simultaneously, you need to increase +the number of SMTP server processes. This will improve the +responsiveness for remote SMTP clients, as long as the server machine +has enough hardware and software resources to run the additional +processes, and as long as the file system can keep up with the +additional load.

+ +
    + +
  • You increase the number of SMTP server processes either +by increasing the default_process_limit in main.cf (line 3 below), +or by increasing the SMTP server's "maxproc" field in master.cf +(line 10 below). Either way, you need to issue a "postfix reload" +command to make the change effective.

    + +
  • Process limits above 1000 require Postfix version 2.4 or +later, and an operating system that supports kernel-based event +filters (BSD kqueue(2), Linux epoll(4), or Solaris /dev/poll). +

    + +
  • You can reduce the Postfix memory footprint by using cdb: +lookup tables instead of Berkeley DB.

    + +
    + 1 /etc/postfix/main.cf:
    + 2     # Raise the global process limit, 100 since Postfix 2.0.
    + 3     default_process_limit = 200
    + 4
    + 5 /etc/postfix/master.cf:
    + 6     # =============================================================
    + 7     # service type  private unpriv  chroot  wakeup  maxproc command
    + 8     # =============================================================
    + 9     # Raise the SMTP service process limit only.
    +10     smtp      inet  n       -       n       -       200     smtpd
    +
    + +
  • NOTE: older versions of the SMTPD_POLICY_README document +contain a mistake: they configure a fixed number of policy daemon +processes. When you raise the SMTP server's "maxproc" field in +master.cf, SMTP server processes will report problems when connecting +to policy server processes, because there aren't enough of them. +Examples of errors are "connection refused" or "operation timed +out". To fix, edit master.cf and specify a zero "maxproc" field +in all policy server entries; see line 6 in the example below. +Issue a "postfix reload" command to make the change effective.

    + +
    +1 /etc/postfix/master.cf:
    +2     # =============================================================
    +3     # service type  private unpriv  chroot  wakeup  maxproc command
    +4     # =============================================================
    +5     # Disable the policy service process limit.
    +6     policy    unix  -       n       n       -       0       spawn
    +7         user=nobody argv=/some/where/policy-server
    +
    + +
+ +

Spend less time per SMTP client

+ +

When increasing the number of SMTP server processes is not +practical, you can improve Postfix server responsiveness by eliminating +unnecessary work. When Postfix spends less time per SMTP session, the +same number of SMTP server processes can service more clients in +the same amount of time.

+ +
    + +
  • Eliminate non-functional RBL lookups (blocklists that are +no longer in operation). These lookups can degrade performance. +Postfix logs a warning when an RBL server does not respond.

    + +
  • Eliminate redundant RBL lookups (people often use multiple +Spamhaus RBLs that include each other). To find out whether RBLs +include other RBLs, look up the websites that document the RBL's +policies.

    + +
  • Eliminate header_checks and body_checks, and keep just a few +emergency patterns to block the latest worm explosion or backscatter +mail. See BACKSCATTER_README for examples of the latter. + +

  • Group your header_checks and body_checks patterns to avoid +unnecessary pattern matching operations. + +

    + 1  /etc/postfix/header_checks:
    + 2      if /^Subject:/
    + 3      /^Subject: virus found in mail from you/ reject
    + 4      /^Subject: ..../ ....
    + 5      endif
    + 6  
    + 7      if /^Received:/
    + 8      /^Received: from (postfix\.org) / reject forged client name in received header: $1
    + 9      /^Received: from .../ ....
    +10      endif
    +
    + +
+ +

Disconnect suspicious SMTP clients

+ +

Under conditions of overload you can improve Postfix SMTP server +responsiveness by hanging up on suspicious clients, so that other +clients get a chance to talk to Postfix.

+ +
    + +
  • Use "421" reply codes for botnet-related RBLs or for +selected non-RBL restrictions. This causes Postfix 2.3 and later +to disconnect immediately without waiting for the remote SMTP +client to send a QUIT command.

    + +

    You can set individual reject codes for RBLs, and for individual +responses from a specific RBL. We'll use zen.spamhaus.org as an +example; by the time you read this document, details may have +changed. Right now, their documents say that a response of 127.0.0.10 +or 127.0.0.11 indicates a dynamic client IP address, which means +that the machine is probably running a bot of some kind. To give +a 421 response instead of the default 554 response, use something +like:

    + +
    + 1  /etc/postfix/main.cf:
    + 2      smtpd_client_restrictions =
    + 3         permit_mynetworks
    + 4         reject_rbl_client zen.spamhaus.org=127.0.0.10
    + 5         reject_rbl_client zen.spamhaus.org=127.0.0.11
    + 6         reject_rbl_client zen.spamhaus.org
    + 7  
    + 8      rbl_reply_maps = hash:/etc/postfix/rbl_reply_maps
    + 9  
    +10  /etc/postfix/rbl_reply_maps:
    +11      zen.spamhaus.org=127.0.0.10 421 4.7.1 Service unavailable;
    +12       $rbl_class [$rbl_what] blocked using
    +13       $rbl_domain${rbl_reason?; $rbl_reason}
    +14  
    +15      zen.spamhaus.org=127.0.0.11 421 4.7.1 Service unavailable;
    +16       $rbl_class [$rbl_what] blocked using
    +17       $rbl_domain${rbl_reason?; $rbl_reason}
    +
    + +

    Although the above shows three RBL lookups (lines 4-6), Postfix +will still only do a single DNS query, so the performance difference +is negligible.

    + +

    The down-side of sending 421 instead of the default 554 is that +it works only for zombies and other malware. If the client is running +a real MTA, then it may connect again several times until the mail +expires in its queue. When this is a problem, stick with the default +554 reply, and use "smtpd_hard_error_limit = 1" as described below. +

    + +

    With Postfix 2.5, or with earlier releases that contain the +stress-adaptive behavior patch, you can turn on the above under +overload by replacing line 8 with:

    + +
    + 8      rbl_reply_maps = ${stress?hash:/etc/postfix/rbl_reply_maps}
    +
    + +

    More information about automatic stress-adaptive behavior is +at the end of this document.

    + +
+ +

Take desperate measures

+ +

The following measures will still allow most legitimate +clients to connect and send mail, but may affect some legitimate +clients.

+ +
    + +
  • Reduce smtpd_timeout (default: 300s). Experience on the +postfix-users list from a variety of sysadmins shows that reducing +the "normal" smtpd_timeout to 60s is unlikely to affect legitimate +clients. However, it is unlikely to become the Postfix default +because it's not RFC compliant. Setting smtpd_timeout to 10s (line +2 below) or even 5s under stress will still allow most +legitimate clients to connect and send mail, but may delay mail +from some clients. No mail should be lost, as long as this measure +is used only temporarily.

    + +
  • Reduce smtpd_hard_error_limit (default: 20). Setting this +to 1 under stress (line 3 below) helps by disconnecting clients +after a single error, giving other clients a chance to connect. +However, this may cause significant delays with legitimate mail, +such as a mailing list that contains a few no-longer-active user +names that didn't bother to unsubscribe. No mail should be lost, +as long as this measure is used only temporarily.

    + +
  • Disable remote SMTP client hostname lookups, so that all +SMTP client hostnames become "unknown" (line 5 below). This feature +was introduced with Postfix 2.3. Unfortunately, this measure is +more problematic than the other ones proposed sofar. First, this +will result in loss of mail when you use hostname-based access rules +that reject mail from "unknown" SMTP clients (examples: +reject_unknown_client_hostname, reject_unknown_reverse_client_hostname). +Second, this may result in loss of mail when you subject "unknown" +SMTP clients to additional restrictions such as reject_unverified_sender. +

    + +
+ +
+
+1  /etc/postfix/main.cf:
+2      smtpd_timeout = 10
+3      smtpd_hard_error_limit = 1
+4      # Caution: line 5 may trigger REJECTs by hostname-based access rules 
+5      smtpd_peername_lookup = no
+
+
+ +

Except with the last measure, no mail should be lost, as long +as these measures are used only temporarily. The next section of +this document introduces a way to automate this process.

+ +

Make Postfix behavior stress-adaptive

+ +

Postfix version 2.5 introduces automatic stress-adaptive behavior. +This is also available as an add-on patch for Postfix versions 2.4 +and 2.3 from the mirrors listed at http://www.postfix.org/download.html. +

+ +

It works as follows. When a "public" network service runs into +an "all server ports are busy" condition, the master(8) daemon logs +a warning, restarts the service (without interrupting existing +network sessions), and runs the service with "-o stress=yes" on the +command line. Normally, it runs a stress-adaptive service with "-o +stress=" on the command line (i.e. with an empty parameter value). +Other services never have "-o stress" parameters on the command +line, including services that listen on a loopback interface only. +

+ +

The stress pseudo-parameter value is the key to making main.cf +parameter settings stress adaptive:

+ +
+
+1  /etc/postfix/main.cf:
+2      smtpd_timeout = ${stress?10}${stress:300}
+3      smtpd_hard_error_limit = ${stress?1}${stress:20}
+
+
+ +

Translation:

+ +

    + +
  • Line 2: under conditions of stress, use an smtpd_timeout +value of 10 seconds instead of the default 300 seconds, + +

  • Line 3: under conditions of stress, use an smtpd_hard_error_limit +of 1 instead of the default 20.

    + +
+ +

The syntax of ${name?value} and ${name:value} is explained at +the beginning of the postconf(5) manual page.

+ +

NOTE: Please keep in mind that the stress-adaptive feature is +a fairly desperate measure to keep some legitimate mail +flowing under overload conditions. If a site is reaching the SMTP +server process limit when there isn't an attack or bot flood +occurring, then either the process limit needs to be raised or more +hardware needs to be added.

+ +

Detecting support for stress-adaptive behavior

+ +

To find out if your Postfix installation supports stress-adaptive +behavior, use the "ps" command, and look for the smtpd processes. +Postfix has stress-adaptive support when you see "-o stress=" or +"-o stress=yes" command-line options. Remember that Postfix never +enables stress-adaptive behavior on servers that listen on local +addresses only.

+ +

The following example is for FreeBSD or Linux. On Solaris, HP-UX +and other System-V flavors, use "ps -ef" instead of "ps ax".

+ +
+
+$ ps ax|grep smtpd
+83326  ??  S      0:00.28 smtpd -n smtp -t inet -u -c -o stress=
+84345  ??  Ss     0:00.11 /usr/bin/perl /usr/libexec/postfix/smtpd-policy.pl
+
+
+ +

You can't use postconf(1) to detect stress-adaptive support. +The postconf(1) command ignores the existence of the stress parameter +in main.cf, because the parameter has no effect there. Command-line +"-o parameter" settings always take precedence over main.cf parameter +settings.

+ +

If you configure stress-adaptive behavior in main.cf when it +isn't supported, nothing bad will happen. The processes will run +as if the stress parameter always has an empty value.

+ +

Forcing stress-adaptive behavior on or off

+ +

You can manually force stress-adaptive behavior on, by adding +a "-o stress=yes" command-line option in master.cf. This can be +useful for testing overrides on the SMTP service. Issue "postfix +reload" to make the change effective.

+ +
+
+1 /etc/postfix/master.cf:
+2     # =============================================================
+3     # service type  private unpriv  chroot  wakeup  maxproc command
+4     # =============================================================
+5     # 
+6     smtp      inet  n       -       n       -       -       smtpd
+7         -o stress=yes
+8         -o . . .
+
+
+ +

To permanently force stress-adaptive behavior off with a specific +service, specify "-o stress=" on its command line. This may be +desirable for the "submission" service. Issue "postfix reload" to +make the change effective.

+ +
+
+1 /etc/postfix/master.cf:
+2     # =============================================================
+3     # service type  private unpriv  chroot  wakeup  maxproc command
+4     # =============================================================
+5     # 
+6     submission inet n       -       n       -       -       smtpd
+7         -o stress=
+8         -o . . .
+
+
+ +

Credits

+ +
    + +
  • Thanks to the postfix-users mailing list members for sharing +early experiences with the stress-adaptive feature. + +
  • The RBL example and several other paragraphs of text were +adapted from postfix-users postings by Noel Jones. + +
  • Wietse implemented stress-adaptive behavior as the smallest +possible patch while he should be working on other things. + +
+ + diff --git a/postfix/html/header_checks.5.html b/postfix/html/header_checks.5.html index f0f68aa0e..f11df058e 100644 --- a/postfix/html/header_checks.5.html +++ b/postfix/html/header_checks.5.html @@ -93,19 +93,19 @@ HEADER_CHECKS(5) HEADER_CHECKS(5) respectively. /pattern/flags action - When pattern matches the input string, execute the - corresponding action. See below for a list of pos- - sible actions. + When /pattern/ matches the input string, execute + the corresponding action. See below for a list of + possible actions. !/pattern/flags action - When pattern does not match the input string, exe- - cute the corresponding action. + When /pattern/ does not match the input string, + execute the corresponding action. if /pattern/flags endif Match the input string against the patterns between if and endif, if and only if the same input string - also matches pattern. The if..endif can nest. + also matches /pattern/. The if..endif can nest. Note: do not prepend whitespace to patterns inside if..endif. @@ -114,7 +114,7 @@ HEADER_CHECKS(5) HEADER_CHECKS(5) endif Match the input string against the patterns between if and endif, if and only if the same input string - does not match pattern. The if..endif can nest. + does not match /pattern/. The if..endif can nest. blank lines and comments Empty lines and whitespace-only lines are ignored, @@ -292,26 +292,31 @@ HEADER_CHECKS(5) HEADER_CHECKS(5) a pattern before applying more drastic actions. BUGS - Many people overlook the main limitations of header and + Empty lines never match, because some map types mis-behave + when given a zero-length search string. This limitation + may be removed for regular expression tables in a future + release. + + Many people overlook the main limitations of header and body_checks rules. - o These rules operate on one logical message header + o These rules operate on one logical message header or one body line at a time. A decision made for one line is not carried over to the next line. - o If text in the message body is encoded (RFC 2045) + o If text in the message body is encoded (RFC 2045) then the rules need to be specified for the encoded form. - o Likewise, when message headers are encoded (RFC - 2047) then the rules need to be specified for the + o Likewise, when message headers are encoded (RFC + 2047) then the rules need to be specified for the encoded form. - Message headers added by the cleanup(8) daemon itself are + Message headers added by the cleanup(8) daemon itself are excluded from inspection. Examples of such message headers are From:, To:, Message-ID:, Date:. - Message headers deleted by the cleanup(8) daemon will be + Message headers deleted by the cleanup(8) daemon will be examined before they are deleted. Examples are: Bcc:, Con- tent-Length:, Return-Path:. @@ -319,11 +324,11 @@ HEADER_CHECKS(5) HEADER_CHECKS(5) body_checks Lookup tables with content filter rules for message body lines. These filters see one physical line at - a time, in chunks of at most $line_length_limit + a time, in chunks of at most $line_length_limit bytes. body_checks_size_limit - The amount of content per message body segment + The amount of content per message body segment (attachment) that is subjected to $body_checks fil- tering. @@ -333,32 +338,32 @@ HEADER_CHECKS(5) HEADER_CHECKS(5) nested_header_checks (default: $header_checks) Lookup tables with content filter rules for message - header lines: respectively, these are applied to - the initial message headers (not including MIME - headers), to the MIME headers anywhere in the mes- - sage, and to the initial headers of attached mes- + header lines: respectively, these are applied to + the initial message headers (not including MIME + headers), to the MIME headers anywhere in the mes- + sage, and to the initial headers of attached mes- sages. - Note: these filters see one logical message header - at a time, even when a message header spans multi- - ple lines. Message headers that are longer than + Note: these filters see one logical message header + at a time, even when a message header spans multi- + ple lines. Message headers that are longer than $header_size_limit characters are truncated. disable_mime_input_processing - While receiving mail, give no special treatment to - MIME related message headers; all text after the + While receiving mail, give no special treatment to + MIME related message headers; all text after the initial message headers is considered to be part of - the message body. This means that header_checks is - applied to all the initial message headers, and + the message body. This means that header_checks is + applied to all the initial message headers, and that body_checks is applied to the remainder of the message. - Note: when used in this manner, body_checks will - process a multi-line message header one line at a + Note: when used in this manner, body_checks will + process a multi-line message header one line at a time. EXAMPLES - Header pattern to block attachments with bad file name + Header pattern to block attachments with bad file name extensions. /etc/postfix/main.cf: @@ -396,7 +401,7 @@ HEADER_CHECKS(5) HEADER_CHECKS(5) BACKSCATTER_README, blocking returned forged mail LICENSE - The Secure Mailer license must be distributed with this + The Secure Mailer license must be distributed with this software. AUTHOR(S) diff --git a/postfix/html/index.html b/postfix/html/index.html index 4107fc0e1..b2e51c61e 100644 --- a/postfix/html/index.html +++ b/postfix/html/index.html @@ -54,6 +54,8 @@ configuration examples
  • Bottleneck analysis +
  • Stress-dependent configuration +
  • Performance tuning
  • Debugging strategies diff --git a/postfix/html/lmtp.8.html b/postfix/html/lmtp.8.html index 26c6abba4..9e1659cde 100644 --- a/postfix/html/lmtp.8.html +++ b/postfix/html/lmtp.8.html @@ -262,6 +262,24 @@ SMTP(8) SMTP(8) riZation ID (authzid); send only the SASL authenti- Cation ID (authcid) plus the authcid's password. + Available in Postfix version 2.5 and later: + + smtp_header_checks (empty) + Restricted header_checks(5) tables for the Postfix + SMTP client. + + smtp_mime_header_checks (empty) + Restricted mime_header_checks(5) tables for the + Postfix SMTP client. + + smtp_nested_header_checks (empty) + Restricted nested_header_checks(5) tables for the + Postfix SMTP client. + + smtp_body_checks (empty) + Restricted body_checks(5) tables for the Postfix + SMTP client. + MIME PROCESSING CONTROLS Available in Postfix version 2.0 and later: @@ -732,6 +750,9 @@ SMTP(8) SMTP(8) that can't be found or that are unreachable. SEE ALSO + generic(5), output address rewriting + header_checks(5), message header content inspection + body_checks(5), body parts content inspection qmgr(8), queue manager bounce(8), delivery status reports scache(8), connection cache server diff --git a/postfix/html/local.8.html b/postfix/html/local.8.html index 26708b5df..6a5ff6c17 100644 --- a/postfix/html/local.8.html +++ b/postfix/html/local.8.html @@ -212,6 +212,10 @@ LOCAL(8) LOCAL(8) LOCAL The entire recipient address localpart (text to the left of the rightmost @ character). + ORIGINAL_RECIPIENT + The entire recipient address, before any address + rewriting or aliasing (Postfix 2.5 and later). + RECIPIENT The entire recipient address. @@ -221,97 +225,97 @@ LOCAL(8) LOCAL(8) the following environment variables: CLIENT_ADDRESS - Remote client network address. Available as of + Remote client network address. Available as of Postfix 2.2. CLIENT_HELO - Remote client EHLO command parameter. Available as + Remote client EHLO command parameter. Available as of Postfix 2.2. CLIENT_HOSTNAME - Remote client hostname. Available as of Postfix + Remote client hostname. Available as of Postfix 2.2. CLIENT_PROTOCOL - Remote client protocol. Available as of Postfix + Remote client protocol. Available as of Postfix 2.2. SASL_METHOD - SASL authentication method specified in the remote + SASL authentication method specified in the remote client AUTH command. Available as of Postfix 2.2. SASL_SENDER - SASL sender address specified in the remote client + SASL sender address specified in the remote client MAIL FROM command. Available as of Postfix 2.2. SASL_USERNAME - SASL username specified in the remote client AUTH + SASL username specified in the remote client AUTH command. Available as of Postfix 2.2. The PATH environment variable is always reset to a system- - dependent default path, and environment variables whose - names are blessed by the export_environment configuration + dependent default path, and environment variables whose + names are blessed by the export_environment configuration parameter are exported unchanged. The current working directory is the mail queue directory. - The local(8) daemon prepends a "From sender time_stamp" - envelope header to each message, prepends an X-Original- + The local(8) daemon prepends a "From sender time_stamp" + envelope header to each message, prepends an X-Original- To: header with the recipient address as given to Postfix, - prepends an optional Delivered-To: header with the final + prepends an optional Delivered-To: header with the final recipient envelope address, prepends a Return-Path: header - with the sender envelope address, and appends no empty + with the sender envelope address, and appends no empty line. EXTERNAL FILE DELIVERY - The delivery format depends on the destination filename - syntax. The default is to use UNIX-style mailbox format. - Specify a name ending in / for qmail-compatible maildir + The delivery format depends on the destination filename + syntax. The default is to use UNIX-style mailbox format. + Specify a name ending in / for qmail-compatible maildir delivery. - The allow_mail_to_files configuration parameter restricts - delivery to external files. The default setting (alias, + The allow_mail_to_files configuration parameter restricts + delivery to external files. The default setting (alias, forward) forbids file destinations in :include: files. - In the case of UNIX-style mailbox delivery, the local(8) + In the case of UNIX-style mailbox delivery, the local(8) daemon prepends a "From sender time_stamp" envelope header - to each message, prepends an X-Original-To: header with - the recipient address as given to Postfix, prepends an - optional Delivered-To: header with the final recipient - envelope address, prepends a > character to lines begin- - ning with "From ", and appends an empty line. The enve- - lope sender address is available in the Return-Path: - header. When the destination is a regular file, it is + to each message, prepends an X-Original-To: header with + the recipient address as given to Postfix, prepends an + optional Delivered-To: header with the final recipient + envelope address, prepends a > character to lines begin- + ning with "From ", and appends an empty line. The enve- + lope sender address is available in the Return-Path: + header. When the destination is a regular file, it is locked for exclusive access while delivery is in progress. In case of problems, an attempt is made to truncate a reg- ular file to its original length. In the case of maildir delivery, the local daemon prepends - an optional Delivered-To: header with the final envelope - recipient address, and prepends an X-Original-To: header + an optional Delivered-To: header with the final envelope + recipient address, and prepends an X-Original-To: header with the recipient address as given to Postfix. The enve- - lope sender address is available in the Return-Path: + lope sender address is available in the Return-Path: header. ADDRESS EXTENSION - The optional recipient_delimiter configuration parameter - specifies how to separate address extensions from local + The optional recipient_delimiter configuration parameter + specifies how to separate address extensions from local recipient names. - For example, with "recipient_delimiter = +", mail for - name+foo is delivered to the alias name+foo or to the - alias name, to the destinations listed in ~name/.for- + For example, with "recipient_delimiter = +", mail for + name+foo is delivered to the alias name+foo or to the + alias name, to the destinations listed in ~name/.for- ward+foo or in ~name/.forward, to the mailbox owned by the user name, or it is sent back as undeliverable. - In all cases the local(8) daemon prepends an optional - `Delivered-To: header line with the final recipient + In all cases the local(8) daemon prepends an optional + `Delivered-To: header line with the final recipient address. DELIVERY RIGHTS - Deliveries to external files and external commands are + Deliveries to external files and external commands are made with the rights of the receiving user on whose behalf - the delivery is made. In the absence of a user context, + the delivery is made. In the absence of a user context, the local(8) daemon uses the owner rights of the :include: file or alias database. When those files are owned by the superuser, delivery is made with the rights specified with @@ -322,48 +326,48 @@ LOCAL(8) LOCAL(8) RFC 3463 (Enhanced status codes) DIAGNOSTICS - Problems and transactions are logged to syslogd(8). Cor- - rupted message files are marked so that the queue manager + Problems and transactions are logged to syslogd(8). Cor- + rupted message files are marked so that the queue manager can move them to the corrupt queue afterwards. - Depending on the setting of the notify_classes parameter, - the postmaster is notified of bounces and of other trou- + Depending on the setting of the notify_classes parameter, + the postmaster is notified of bounces and of other trou- ble. SECURITY The local(8) delivery agent needs a dual personality 1) to access the private Postfix queue and IPC mechanisms, 2) to - impersonate the recipient and deliver to recipient-speci- - fied files or commands. It is therefore security sensi- + impersonate the recipient and deliver to recipient-speci- + fied files or commands. It is therefore security sensi- tive. - The local(8) delivery agent disallows regular expression - substitution of $1 etc. in alias_maps, because that would + The local(8) delivery agent disallows regular expression + substitution of $1 etc. in alias_maps, because that would open a security hole. - The local(8) delivery agent will silently ignore requests - to use the proxymap(8) server within alias_maps. Instead - it will open the table directly. Before Postfix version - 2.2, the local(8) delivery agent will terminate with a + The local(8) delivery agent will silently ignore requests + to use the proxymap(8) server within alias_maps. Instead + it will open the table directly. Before Postfix version + 2.2, the local(8) delivery agent will terminate with a fatal error. BUGS - For security reasons, the message delivery status of - external commands or of external files is never check- + For security reasons, the message delivery status of + external commands or of external files is never check- pointed to file. As a result, the program may occasionally deliver more than once to a command or external file. Bet- ter safe than sorry. - Mutually-recursive aliases or ~/.forward files are not - detected early. The resulting mail forwarding loop is + Mutually-recursive aliases or ~/.forward files are not + detected early. The resulting mail forwarding loop is broken by the use of the Delivered-To: message header. CONFIGURATION PARAMETERS - Changes to main.cf are picked up automatically, as - local(8) processes run for only a limited amount of time. + Changes to main.cf are picked up automatically, as + local(8) processes run for only a limited amount of time. Use the command "postfix reload" to speed up a change. - The text below provides only a parameter summary. See + The text below provides only a parameter summary. See postconf(5) for more details including examples. COMPATIBILITY CONTROLS @@ -373,13 +377,13 @@ LOCAL(8) LOCAL(8) expand_owner_alias (no) When delivering to an alias "aliasname" that has an "owner-aliasname" companion alias, set the envelope - sender address to the expansion of the "owner- + sender address to the expansion of the "owner- aliasname" alias. owner_request_special (yes) - Give special treatment to owner-listname and list- - name-request address localparts: don't split such - addresses when the recipient_delimiter is set to + Give special treatment to owner-listname and list- + name-request address localparts: don't split such + addresses when the recipient_delimiter is set to "-". sun_mailtool_compatibility (no) @@ -388,66 +392,66 @@ LOCAL(8) LOCAL(8) Available in Postfix version 2.3 and later: frozen_delivered_to (yes) - Update the local(8) delivery agent's idea of the - Delivered-To: address (see prepend_deliv- - ered_header) only once, at the start of a delivery - attempt; do not update the Delivered-To: address + Update the local(8) delivery agent's idea of the + Delivered-To: address (see prepend_deliv- + ered_header) only once, at the start of a delivery + attempt; do not update the Delivered-To: address while expanding aliases or .forward files. DELIVERY METHOD CONTROLS - The precedence of local(8) delivery methods from high to - low is: aliases, .forward files, mailbox_transport_maps, - mailbox_transport, mailbox_command_maps, mailbox_command, - home_mailbox, mail_spool_directory, fallback_trans- + The precedence of local(8) delivery methods from high to + low is: aliases, .forward files, mailbox_transport_maps, + mailbox_transport, mailbox_command_maps, mailbox_command, + home_mailbox, mail_spool_directory, fallback_trans- port_maps, fallback_transport, and luser_relay. alias_maps (see 'postconf -d' output) - The alias databases that are used for local(8) + The alias databases that are used for local(8) delivery. forward_path (see 'postconf -d' output) The local(8) delivery agent search list for finding - a .forward file with user-specified delivery meth- + a .forward file with user-specified delivery meth- ods. mailbox_transport_maps (empty) - Optional lookup tables with per-recipient message - delivery transports to use for local(8) mailbox - delivery, whether or not the recipients are found + Optional lookup tables with per-recipient message + delivery transports to use for local(8) mailbox + delivery, whether or not the recipients are found in the UNIX passwd database. mailbox_transport (empty) - Optional message delivery transport that the - local(8) delivery agent should use for mailbox - delivery to all local recipients, whether or not + Optional message delivery transport that the + local(8) delivery agent should use for mailbox + delivery to all local recipients, whether or not they are found in the UNIX passwd database. mailbox_command_maps (empty) - Optional lookup tables with per-recipient external + Optional lookup tables with per-recipient external commands to use for local(8) mailbox delivery. mailbox_command (empty) - Optional external command that the local(8) deliv- + Optional external command that the local(8) deliv- ery agent should use for mailbox delivery. home_mailbox (empty) - Optional pathname of a mailbox file relative to a + Optional pathname of a mailbox file relative to a local(8) user's home directory. mail_spool_directory (see 'postconf -d' output) - The directory where local(8) UNIX-style mailboxes + The directory where local(8) UNIX-style mailboxes are kept. fallback_transport_maps (empty) - Optional lookup tables with per-recipient message - delivery transports for recipients that the - local(8) delivery agent could not find in the + Optional lookup tables with per-recipient message + delivery transports for recipients that the + local(8) delivery agent could not find in the aliases(5) or UNIX password database. fallback_transport (empty) - Optional message delivery transport that the - local(8) delivery agent should use for names that - are not found in the aliases(5) or UNIX password + Optional message delivery transport that the + local(8) delivery agent should use for names that + are not found in the aliases(5) or UNIX password database. luser_relay (empty) @@ -457,7 +461,7 @@ LOCAL(8) LOCAL(8) Available in Postfix version 2.2 and later: command_execution_directory (empty) - The local(8) delivery agent working directory for + The local(8) delivery agent working directory for delivery to external command. MAILBOX LOCKING CONTROLS @@ -466,15 +470,15 @@ LOCAL(8) LOCAL(8) sive lock on a mailbox file or bounce(8) logfile. deliver_lock_delay (1s) - The time between attempts to acquire an exclusive + The time between attempts to acquire an exclusive lock on a mailbox file or bounce(8) logfile. stale_lock_time (500s) - The time after which a stale exclusive mailbox + The time after which a stale exclusive mailbox lockfile is removed. mailbox_delivery_lock (see 'postconf -d' output) - How to lock a UNIX-style local(8) mailbox before + How to lock a UNIX-style local(8) mailbox before attempting delivery. RESOURCE AND RATE CONTROLS @@ -482,17 +486,17 @@ LOCAL(8) LOCAL(8) Time limit for delivery to external commands. duplicate_filter_limit (1000) - The maximal number of addresses remembered by the - address duplicate filter for aliases(5) or vir- + The maximal number of addresses remembered by the + address duplicate filter for aliases(5) or vir- tual(5) alias expansion, or for showq(8) queue dis- plays. local_destination_concurrency_limit (2) - The maximal number of parallel deliveries via the + The maximal number of parallel deliveries via the local mail delivery transport to the same recipient - (when "local_destination_recipient_limit = 1") or - the maximal number of parallel deliveries to the - same local domain (when "local_destination_recipi- + (when "local_destination_recipient_limit = 1") or + the maximal number of parallel deliveries to the + same local domain (when "local_destination_recipi- ent_limit > 1"). local_destination_recipient_limit (1) @@ -505,49 +509,49 @@ LOCAL(8) LOCAL(8) SECURITY CONTROLS allow_mail_to_commands (alias, forward) - Restrict local(8) mail delivery to external com- + Restrict local(8) mail delivery to external com- mands. allow_mail_to_files (alias, forward) - Restrict local(8) mail delivery to external files. + Restrict local(8) mail delivery to external files. command_expansion_filter (see 'postconf -d' output) - Restrict the characters that the local(8) delivery - agent allows in $name expansions of $mailbox_com- + Restrict the characters that the local(8) delivery + agent allows in $name expansions of $mailbox_com- mand. default_privs (nobody) - The default rights used by the local(8) delivery + The default rights used by the local(8) delivery agent for delivery to external file or command. forward_expansion_filter (see 'postconf -d' output) - Restrict the characters that the local(8) delivery - agent allows in $name expansions of $forward_path. + Restrict the characters that the local(8) delivery + agent allows in $name expansions of $forward_path. Available in Postfix version 2.2 and later: execution_directory_expansion_filter (see 'postconf -d' output) - Restrict the characters that the local(8) delivery + Restrict the characters that the local(8) delivery agent allows in $name expansions of $command_execu- tion_directory. MISCELLANEOUS CONTROLS config_directory (see 'postconf -d' output) - The default location of the Postfix main.cf and + The default location of the Postfix main.cf and master.cf configuration files. daemon_timeout (18000s) - How much time a Postfix daemon process may take to - handle a request before it is terminated by a + How much time a Postfix daemon process may take to + handle a request before it is terminated by a built-in watchdog timer. delay_logging_resolution_limit (2) - The maximal number of digits after the decimal + The maximal number of digits after the decimal point when logging sub-second delay values. export_environment (see 'postconf -d' output) - The list of environment variables that a Postfix + The list of environment variables that a Postfix process will export to non-Postfix processes. ipc_timeout (3600s) @@ -555,39 +559,39 @@ LOCAL(8) LOCAL(8) over an internal communication channel. local_command_shell (empty) - Optional shell program for local(8) delivery to + Optional shell program for local(8) delivery to non-Postfix command. max_idle (100s) - The maximum amount of time that an idle Postfix - daemon process waits for an incoming connection + The maximum amount of time that an idle Postfix + daemon process waits for an incoming connection before terminating voluntarily. max_use (100) - The maximal number of incoming connections that a - Postfix daemon process will service before termi- + The maximal number of incoming connections that a + Postfix daemon process will service before termi- nating voluntarily. prepend_delivered_header (command, file, forward) - The message delivery contexts where the Postfix - local(8) delivery agent prepends a Delivered-To: - message header with the address that the mail was + The message delivery contexts where the Postfix + local(8) delivery agent prepends a Delivered-To: + message header with the address that the mail was delivered to. process_id (read-only) - The process ID of a Postfix command or daemon + The process ID of a Postfix command or daemon process. process_name (read-only) - The process name of a Postfix command or daemon + The process name of a Postfix command or daemon process. propagate_unmatched_extensions (canonical, virtual) - What address lookup tables copy an address exten- + What address lookup tables copy an address exten- sion from the lookup key to the lookup result. queue_directory (see 'postconf -d' output) - The location of the Postfix top-level queue direc- + The location of the Postfix top-level queue direc- tory. recipient_delimiter (empty) @@ -595,15 +599,15 @@ LOCAL(8) LOCAL(8) sions (user+foo). require_home_directory (no) - Whether or not a local(8) recipient's home direc- - tory must exist before mail delivery is attempted. + Whether or not a local(8) recipient's home direc- + tory must exist before mail delivery is attempted. syslog_facility (mail) The syslog facility of Postfix logging. syslog_name (postfix) - The mail system name that is prepended to the - process name in syslog records, so that "smtpd" + The mail system name that is prepended to the + process name in syslog records, so that "smtpd" becomes, for example, "postfix/smtpd". FILES @@ -623,14 +627,14 @@ LOCAL(8) LOCAL(8) syslogd(8), system logging LICENSE - The Secure Mailer license must be distributed with this + The Secure Mailer license must be distributed with this software. HISTORY The Delivered-To: message header appears in the qmail sys- tem by Daniel Bernstein. - The maildir structure appears in the qmail system by + The maildir structure appears in the qmail system by Daniel Bernstein. AUTHOR(S) diff --git a/postfix/html/pipe.8.html b/postfix/html/pipe.8.html index 7f26870f3..7f578a15b 100644 --- a/postfix/html/pipe.8.html +++ b/postfix/html/pipe.8.html @@ -124,34 +124,36 @@ PIPE(8) PIPE(8) This feature is available as of Postfix 2.5. - h Fold the command-line $recipient address - domain part (text to the right of the right- - most @ character) to lower case; fold the - entire command-line $domain and $nexthop - host or domain information to lower case. - This is recommended for delivery via UUCP. - - q Quote white space and other special charac- - ters in the command-line $sender and $recip- - ient address localparts (text to the left of - the right-most @ character), according to an - 8-bit transparent version of RFC 822. This - is recommended for delivery via UUCP or - BSMTP. - - The result is compatible with the address - parsing of command-line recipients by the + h Fold the command-line $original_recipient + and $recipient address domain part (text to + the right of the right-most @ character) to + lower case; fold the entire command-line + $domain and $nexthop host or domain informa- + tion to lower case. This is recommended for + delivery via UUCP. + + q Quote white space and other special charac- + ters in the command-line $sender, $origi- + nal_recipient and $recipient address local- + parts (text to the left of the right-most @ + character), according to an 8-bit transpar- + ent version of RFC 822. This is recommended + for delivery via UUCP or BSMTP. + + The result is compatible with the address + parsing of command-line recipients by the Postfix sendmail(1) mail submission command. - The q flag affects only entire addresses, + The q flag affects only entire addresses, not the partial address information from the - $user, $extension or $mailbox command-line + $user, $extension or $mailbox command-line macros. - u Fold the command-line $recipient address - localpart (text to the left of the right- - most @ character) to lower case. This is - recommended for delivery via UUCP. + u Fold the command-line $original_recipient + and $recipient address localpart (text to + the left of the right-most @ character) to + lower case. This is recommended for deliv- + ery via UUCP. . Prepend "." to lines starting with ".". This is needed by, for example, BSMTP software. @@ -294,6 +296,19 @@ PIPE(8) PIPE(8) This information is modified by the h flag for case folding. + ${original_recipient} + This macro expands to the complete recipient + address before any address rewriting or + aliasing. + + A command-line argument that contains + ${original_recipient} expands to as many + command-line arguments as there are recipi- + ents. + + This information is modified by the hqu + flags for quoting and case folding. + ${recipient} This macro expands to the complete recipient address. diff --git a/postfix/html/postconf.5.html b/postfix/html/postconf.5.html index 77e571316..5cef8f2da 100644 --- a/postfix/html/postconf.5.html +++ b/postfix/html/postconf.5.html @@ -4564,6 +4564,11 @@ and later.
    The recipient's username.
    +
    ORIGINAL_RECIPIENT
    + +
    The entire recipient address, before any address rewriting or +aliasing.
    +
    RECIPIENT
    The full recipient address.
    @@ -7029,6 +7034,19 @@ IP hosting, but can be a problem on multi-homed firewalls. See the but this form is not recommended here.

    + + +
    smtp_body_checks +(default: empty)
    + +

    Restricted body_checks(5) tables for the Postfix SMTP client. +These tables are searched while mail is being delivered. Actions +that change the delivery time or destination are not available. +

    + +

    This feature is available in Postfix 2.5 and later.

    + +
    smtp_cname_overrides_servername @@ -7418,6 +7436,19 @@ examples are shown in the ADDRESS_REWRIT

    This feature is available in Postfix 2.2 and later.

    + + +
    smtp_header_checks +(default: empty)
    + +

    Restricted header_checks(5) tables for the Postfix SMTP client. +These tables are searched while mail is being delivered. Actions +that change the delivery time or destination are not available. +

    + +

    This feature is available in Postfix 2.5 and later.

    + +
    smtp_helo_name @@ -7534,6 +7565,19 @@ The default time unit is s (seconds).

    + + +
    smtp_mime_header_checks +(default: empty)
    + +

    Restricted mime_header_checks(5) tables for the Postfix SMTP +client. These tables are searched while mail is being delivered. +Actions that change the delivery time or destination are not +available.

    + +

    This feature is available in Postfix 2.5 and later.

    + +
    smtp_mx_address_limit @@ -7564,6 +7608,19 @@ complete the EHLO and TLS handshake (Postfix version 2.3 and later).

    This feature is available in Postfix 2.1 and later.

    + + +
    smtp_nested_header_checks +(default: empty)
    + +

    Restricted nested_header_checks(5) tables for the Postfix SMTP +client. These tables are searched while mail is being delivered. +Actions that change the delivery time or destination are not +available.

    + +

    This feature is available in Postfix 2.5 and later.

    + +
    smtp_never_send_ehlo diff --git a/postfix/html/smtp.8.html b/postfix/html/smtp.8.html index 26c6abba4..9e1659cde 100644 --- a/postfix/html/smtp.8.html +++ b/postfix/html/smtp.8.html @@ -262,6 +262,24 @@ SMTP(8) SMTP(8) riZation ID (authzid); send only the SASL authenti- Cation ID (authcid) plus the authcid's password. + Available in Postfix version 2.5 and later: + + smtp_header_checks (empty) + Restricted header_checks(5) tables for the Postfix + SMTP client. + + smtp_mime_header_checks (empty) + Restricted mime_header_checks(5) tables for the + Postfix SMTP client. + + smtp_nested_header_checks (empty) + Restricted nested_header_checks(5) tables for the + Postfix SMTP client. + + smtp_body_checks (empty) + Restricted body_checks(5) tables for the Postfix + SMTP client. + MIME PROCESSING CONTROLS Available in Postfix version 2.0 and later: @@ -732,6 +750,9 @@ SMTP(8) SMTP(8) that can't be found or that are unreachable. SEE ALSO + generic(5), output address rewriting + header_checks(5), message header content inspection + body_checks(5), body parts content inspection qmgr(8), queue manager bounce(8), delivery status reports scache(8), connection cache server diff --git a/postfix/man/man5/header_checks.5 b/postfix/man/man5/header_checks.5 index a961476f8..1b2b11bec 100644 --- a/postfix/man/man5/header_checks.5 +++ b/postfix/man/man5/header_checks.5 @@ -91,17 +91,17 @@ given below. For a discussion of specific pattern or flags syntax, see \fBpcre_table\fR(5) or \fBregexp_table\fR(5), respectively. .IP "\fB/\fIpattern\fB/\fIflags action\fR" -When \fIpattern\fR matches the input string, execute +When /\fIpattern\fR/ matches the input string, execute the corresponding \fIaction\fR. See below for a list of possible actions. .IP "\fB!/\fIpattern\fB/\fIflags action\fR" -When \fIpattern\fR does \fBnot\fR match the input string, +When /\fIpattern\fR/ does \fBnot\fR match the input string, execute the corresponding \fIaction\fR. .IP "\fBif /\fIpattern\fB/\fIflags\fR" .IP "\fBendif\fR" Match the input string against the patterns between \fBif\fR and \fBendif\fR, if and only if the same input string also -matches \fIpattern\fR. The \fBif\fR..\fBendif\fR can nest. +matches /\fIpattern\fR/. The \fBif\fR..\fBendif\fR can nest. .sp Note: do not prepend whitespace to patterns inside \fBif\fR..\fBendif\fR. @@ -109,7 +109,7 @@ Note: do not prepend whitespace to patterns inside .IP "\fBendif\fR" Match the input string against the patterns between \fBif\fR and \fBendif\fR, if and only if the same input string does -\fBnot\fR match \fIpattern\fR. The \fBif\fR..\fBendif\fR +\fBnot\fR match /\fIpattern\fR/. The \fBif\fR..\fBendif\fR can nest. .IP "blank lines and comments" Empty lines and whitespace-only lines are ignored, as @@ -272,6 +272,10 @@ before applying more drastic actions. .SH BUGS .ad .fi +Empty lines never match, because some map types mis-behave +when given a zero-length search string. This limitation may +be removed for regular expression tables in a future release. + Many people overlook the main limitations of header and body_checks rules. .IP \(bu diff --git a/postfix/man/man5/postconf.5 b/postfix/man/man5/postconf.5 index 2c41ca371..d558846de 100644 --- a/postfix/man/man5/postconf.5 +++ b/postfix/man/man5/postconf.5 @@ -2461,6 +2461,9 @@ The recipient home directory. The recipient address localpart. .IP "\fBLOGNAME\fR" The recipient's username. +.IP "\fBORIGINAL_RECIPIENT\fR" +The entire recipient address, before any address rewriting or +aliasing. .IP "\fBRECIPIENT\fR" The full recipient address. .IP "\fBSASL_METHOD\fR" @@ -3911,6 +3914,12 @@ inet_interfaces documentation for more detail. .PP Note 2: address information may be enclosed inside [], but this form is not recommended here. +.SH smtp_body_checks (default: empty) +Restricted \fBbody_checks\fR(5) tables for the Postfix SMTP client. +These tables are searched while mail is being delivered. Actions +that change the delivery time or destination are not available. +.PP +This feature is available in Postfix 2.5 and later. .SH smtp_cname_overrides_servername (default: version dependent) Allow DNS CNAME records to override the servername that the Postfix SMTP client uses for logging, SASL password lookup, TLS @@ -4150,6 +4159,12 @@ examples are shown in the ADDRESS_REWRITING_README and STANDARD_CONFIGURATION_README documents. .PP This feature is available in Postfix 2.2 and later. +.SH smtp_header_checks (default: empty) +Restricted \fBheader_checks\fR(5) tables for the Postfix SMTP client. +These tables are searched while mail is being delivered. Actions +that change the delivery time or destination are not available. +.PP +This feature is available in Postfix 2.5 and later. .SH smtp_helo_name (default: $myhostname) The hostname to send in the SMTP EHLO or HELO command. .PP @@ -4204,6 +4219,13 @@ for receiving the server response. .PP Time units: s (seconds), m (minutes), h (hours), d (days), w (weeks). The default time unit is s (seconds). +.SH smtp_mime_header_checks (default: empty) +Restricted \fBmime_header_checks\fR(5) tables for the Postfix SMTP +client. These tables are searched while mail is being delivered. +Actions that change the delivery time or destination are not +available. +.PP +This feature is available in Postfix 2.5 and later. .SH smtp_mx_address_limit (default: 5) The maximal number of MX (mail exchanger) IP addresses that can result from mail exchanger lookups, or zero (no limit). Prior to @@ -4218,6 +4240,13 @@ SMTP initial handshake (Postfix version 2.2 and earlier) or that fail to complete the EHLO and TLS handshake (Postfix version 2.3 and later). .PP This feature is available in Postfix 2.1 and later. +.SH smtp_nested_header_checks (default: empty) +Restricted \fBnested_header_checks\fR(5) tables for the Postfix SMTP +client. These tables are searched while mail is being delivered. +Actions that change the delivery time or destination are not +available. +.PP +This feature is available in Postfix 2.5 and later. .SH smtp_never_send_ehlo (default: no) Never send EHLO at the start of an SMTP session. See also the smtp_always_send_ehlo parameter. diff --git a/postfix/man/man8/local.8 b/postfix/man/man8/local.8 index 445200745..2699ba6ee 100644 --- a/postfix/man/man8/local.8 +++ b/postfix/man/man8/local.8 @@ -226,6 +226,9 @@ The bare recipient name. .IP \fBLOCAL\fR The entire recipient address localpart (text to the left of the rightmost @ character). +.IP \fBORIGINAL_RECIPIENT\fR +The entire recipient address, before any address rewriting +or aliasing (Postfix 2.5 and later). .IP \fBRECIPIENT\fR The entire recipient address. .IP \fBSENDER\fR diff --git a/postfix/man/man8/pipe.8 b/postfix/man/man8/pipe.8 index 13c767013..bf8b4e890 100644 --- a/postfix/man/man8/pipe.8 +++ b/postfix/man/man8/pipe.8 @@ -118,14 +118,16 @@ from "relayed" into "delivered". .sp This feature is available as of Postfix 2.5. .IP \fBh\fR -Fold the command-line \fB$recipient\fR address domain part +Fold the command-line \fB$original_recipient\fR and +\fB$recipient\fR address domain part (text to the right of the right-most \fB@\fR character) to lower case; fold the entire command-line \fB$domain\fR and \fB$nexthop\fR host or domain information to lower case. This is recommended for delivery via \fBUUCP\fR. .IP \fBq\fR Quote white space and other special characters in the command-line -\fB$sender\fR and \fB$recipient\fR address localparts (text to the +\fB$sender\fR, \fB$original_recipient\fR and \fB$recipient\fR +address localparts (text to the left of the right-most \fB@\fR character), according to an 8-bit transparent version of RFC 822. This is recommended for delivery via \fBUUCP\fR or \fBBSMTP\fR. @@ -137,7 +139,8 @@ The \fBq\fR flag affects only entire addresses, not the partial address information from the \fB$user\fR, \fB$extension\fR or \fB$mailbox\fR command-line macros. .IP \fBu\fR -Fold the command-line \fB$recipient\fR address localpart (text to +Fold the command-line \fB$original_recipient\fR and +\fB$recipient\fR address localpart (text to the left of the right-most \fB@\fR character) to lower case. This is recommended for delivery via \fBUUCP\fR. .IP \fB.\fR @@ -254,6 +257,16 @@ This information is modified by the \fBu\fR flag for case folding. This macro expands to the next-hop hostname. .sp This information is modified by the \fBh\fR flag for case folding. +.IP \fB${\fBoriginal_recipient\fR}\fR +This macro expands to the complete recipient address before any +address rewriting or aliasing. +.sp +A command-line argument that contains +\fB${\fBoriginal_recipient\fR}\fR expands to as many +command-line arguments as there are recipients. +.sp +This information is modified by the \fBhqu\fR flags for quoting +and case folding. .IP \fB${\fBrecipient\fR}\fR This macro expands to the complete recipient address. .sp diff --git a/postfix/man/man8/smtp.8 b/postfix/man/man8/smtp.8 index a7390cd0f..40532a4db 100644 --- a/postfix/man/man8/smtp.8 +++ b/postfix/man/man8/smtp.8 @@ -233,6 +233,18 @@ Available in Postfix version 2.4.4 and later: When authenticating to a remote SMTP or LMTP server with the default setting "no", send no SASL authoriZation ID (authzid); send only the SASL authentiCation ID (authcid) plus the authcid's password. +.PP +Available in Postfix version 2.5 and later: +.IP "\fBsmtp_header_checks (empty)\fR" +Restricted \fBheader_checks\fR(5) tables for the Postfix SMTP client. +.IP "\fBsmtp_mime_header_checks (empty)\fR" +Restricted \fBmime_header_checks\fR(5) tables for the Postfix SMTP +client. +.IP "\fBsmtp_nested_header_checks (empty)\fR" +Restricted \fBnested_header_checks\fR(5) tables for the Postfix SMTP +client. +.IP "\fBsmtp_body_checks (empty)\fR" +Restricted \fBbody_checks\fR(5) tables for the Postfix SMTP client. .SH "MIME PROCESSING CONTROLS" .na .nf @@ -578,6 +590,9 @@ found or that are unreachable. .SH "SEE ALSO" .na .nf +generic(5), output address rewriting +header_checks(5), message header content inspection +body_checks(5), body parts content inspection qmgr(8), queue manager bounce(8), delivery status reports scache(8), connection cache server diff --git a/postfix/mantools/postlink b/postfix/mantools/postlink index 34421491a..cf1586db4 100755 --- a/postfix/mantools/postlink +++ b/postfix/mantools/postlink @@ -563,6 +563,10 @@ while (<>) { s;\bsmtp_tls_session_cache_database\b;$&;g; s;\bsmtp_tls_session_cache_timeout\b;$&;g; s;\bsmtp_use_tls\b;$&;g; + s;\bsmtp_header_checks\b;$&;g; + s;\bsmtp_mime_header_checks\b;$&;g; + s;\bsmtp_nested_header_checks\b;$&;g; + s;\bsmtp_body_checks\b;$&;g; s;\bsmtpd_enforce_tls\b;$&;g; s;\bsmtpd_sasl_tls_security_options\b;$&;g; s;\bsmtpd_sasl_type\b;$&;g; @@ -833,6 +837,7 @@ while (<>) { # Hyperlink map types. + s/\b(cdb):/$1<\/a>:/g; s/\b(cidr):/$1<\/a>:/g; s/\b(pcre):/$1<\/a>:/g; s/\b(proxy):/$1<\/a>:/g; diff --git a/postfix/proto/BACKSCATTER_README.html b/postfix/proto/BACKSCATTER_README.html index 376d896cf..ccd75b518 100644 --- a/postfix/proto/BACKSCATTER_README.html +++ b/postfix/proto/BACKSCATTER_README.html @@ -20,11 +20,8 @@ Backscatter Howto

    Overview

    -This document describes features that require Postfix version 2.0 -or later. The examples use Perl Compatible Regular Expressions -(Postfix pcre: tables), but also provide a translation to POSIX -regular expressions (Postfix regexp: tables). PCRE is preferred -primarily because the implementation is often faster.

    +

    This document describes features that require Postfix version +2.0 or later.

    Topics covered in this document:

    @@ -56,6 +53,11 @@ scanners
    +

    The examples use Perl Compatible Regular Expressions (Postfix +pcre: tables), but also provide a translation to POSIX regular +expressions (Postfix regexp: tables). PCRE is preferred primarily +because the implementation is often faster.

    +

    What is backscatter mail?

    When a spammer or worm sends mail with forged sender addresses, @@ -73,7 +75,7 @@ to=<yyyyyy@your.domain.here> proto=ESMTP helo=<zzzzzz>

  • -

    What you see are lots of "user unknown" errors with "from=<>". +

    What you see are lots of "user unknown" errors with "from=<>". These are error reports from MAILER-DAEMONs elsewhere on the Internet.

    diff --git a/postfix/proto/MILTER_README.html b/postfix/proto/MILTER_README.html index edef90fde..a28675041 100644 --- a/postfix/proto/MILTER_README.html +++ b/postfix/proto/MILTER_README.html @@ -501,7 +501,7 @@ EOH, BODY, EOM -

    Beware: 30s is not a lot for applications that do a lot of DNS +

    Beware: 30s may be too short for applications doing lots of DNS lookups. However, if you increase the above timeouts too much, remote SMTP clients may hang up and mail may be delivered multiple times. This is an inherent problem with before-queue filtering.

    @@ -624,7 +624,7 @@ TO

    Workarounds

    -

    Content filters may break domain key etc. signatures. If you +

    Content filters may break DKIM etc. signatures. If you use an SMTP-based content filter, then you should add a line to master.cf with "-o disable_mime_output_conversion=yes" (note: no spaces around the "="), as described in the -

    And they may insert a message header with "unknown-msgid" like -this:

    +

    And they may insert an ugly message header with "unknown-msgid" +like this:

    diff --git a/postfix/proto/Makefile.in b/postfix/proto/Makefile.in
    index e2e855b78..ec7e0d6ae 100644
    --- a/postfix/proto/Makefile.in
    +++ b/postfix/proto/Makefile.in
    @@ -34,6 +34,7 @@ HTML	= ../html/ADDRESS_CLASS_README.html \
     	../html/SMTPD_POLICY_README.html \
     	../html/SMTPD_PROXY_README.html \
     	../html/STANDARD_CONFIGURATION_README.html \
    +	../html/STRESS_README.html \
     	../html/TLS_README.html ../html/TLS_LEGACY_README.html \
     	../html/TUNING_README.html \
     	../html/UUCP_README.html \
    @@ -69,6 +70,7 @@ README	= ../README_FILES/ADDRESS_CLASS_README \
     	../README_FILES/SMTPD_ACCESS_README \
     	../README_FILES/SMTPD_POLICY_README ../README_FILES/SMTPD_PROXY_README \
     	../README_FILES/STANDARD_CONFIGURATION_README \
    +	../README_FILES/STRESS_README \
     	../README_FILES/TLS_README ../README_FILES/TLS_LEGACY_README \
     	../README_FILES/TUNING_README \
     	../README_FILES/UUCP_README \
    @@ -235,6 +237,9 @@ clobber:
     ../html/STANDARD_CONFIGURATION_README.html: STANDARD_CONFIGURATION_README.html
     	$(POSTLINK) $? >$@
     
    +../html/STRESS_README.html: STRESS_README.html
    +	$(POSTLINK) $? >$@
    +
     ../html/TUNING_README.html: TUNING_README.html
     	$(POSTLINK) $? >$@
     
    @@ -376,6 +381,9 @@ clobber:
     ../README_FILES/STANDARD_CONFIGURATION_README: STANDARD_CONFIGURATION_README.html
     	$(HT2READ) $? >$@
     
    +../README_FILES/STRESS_README: STRESS_README.html
    +	$(HT2READ) $? >$@
    +
     ../README_FILES/TUNING_README: TUNING_README.html
     	$(HT2READ) $? >$@
     
    diff --git a/postfix/proto/STRESS_README.html b/postfix/proto/STRESS_README.html
    new file mode 100644
    index 000000000..a313a0d7e
    --- /dev/null
    +++ b/postfix/proto/STRESS_README.html
    @@ -0,0 +1,469 @@
    +
    +
    +
    +
    +
    +
    +Postfix Stress-Dependent Configuration
    +
    +
    +
    +
    +
    +
    +
    +

    Postfix +Stress-Dependent Configuration

    + +
    + +

    Overview

    + +

    This document describes the symptoms of Postfix SMTP server +overload, and how to avoid the condition under normal conditions. +When the condition is caused by botnets or other malware, the +document suggests configuration settings that help to minimize the +impact on legitimate mail. Finally, the document introduces Postfix +stress-adaptive behavior, and how it can be used to automatically +switch configuration settings under overload.

    + +

    Topics covered in this document:

    + +
    + +

    Symptoms of Postfix SMTP server overload

    + +

    Under normal conditions, Postfix responds immediately when a +remote SMTP client connects. The time needed to deliver mail to +Postfix may depend on how busy the CPU or disk are, but that should +be noticeable only with very large messages. Performance degrades +more dramatically when the number of remote SMTP clients exceeds +the number of Postfix SMTP server processes. When a client connects +while all server processes are busy, the client must wait until a +server process becomes available.

    + +

    Overload may be caused by a legitimate mail (example: a DNS +registrar opens a new zone for registrations), by mistake (mail +explosion caused by a forwarding loop) or by illegitimate mail (worm +outbreak, botnet, or other malware activity). Symptoms of Postfix +SMTP mail server overload are:

    + +
      + +
    • Postfix logs a warning that all server ports are busy:

      + +
      +Oct  3 20:39:27 spike postfix/master[28905]: warning: service "smtp"
      + (25) has reached its process limit "30": new clients may experience
      + noticeable delays
      +Oct  3 20:39:27 spike postfix/master[28905]: warning: to avoid this
      + condition, increase the process count in master.cf or reduce the
      + service time per client
      +
      + +
    • Remote SMTP clients experience a long delay before Postfix +sends the "220 hostname.example.com ESMTP Postfix" greeting. If +this affects end-user mail clients, enable the "submission" service +entry in master.cf (present since Postfix 2.1), and tell users to +connect to this instead of the public SMTP service.

      + +
    • The Postfix SMTP server logs an increased number of "lost +connection after CONNECT" events. This happens because remote SMTP +clients disconnect before Postfix answers the connection.

      + +
    + +

    NOTE: The last two symptoms also happen without overload.

    + +
      + +
    • Broken DNS also causes lengthy delays before "220 +hostname.example.com +..." while the Postfix SMTP server tries to look up the client's +hostname.

      + +
    • A portscan for open SMTP ports also results in "lost +connection ..." logfile messages.

      + +
    + +

    Legitimate mail that doesn't get through during an episode of +overload is not necessarily lost. It should still arrive once the +situation returns to normal, as long as the overload condition is +temporary.

    + +

    Service more SMTP clients at the same time

    + +

    To service more SMTP clients simultaneously, you need to increase +the number of SMTP server processes. This will improve the +responsiveness for remote SMTP clients, as long as the server machine +has enough hardware and software resources to run the additional +processes, and as long as the file system can keep up with the +additional load.

    + +
      + +
    • You increase the number of SMTP server processes either +by increasing the default_process_limit in main.cf (line 3 below), +or by increasing the SMTP server's "maxproc" field in master.cf +(line 10 below). Either way, you need to issue a "postfix reload" +command to make the change effective.

      + +
    • Process limits above 1000 require Postfix version 2.4 or +later, and an operating system that supports kernel-based event +filters (BSD kqueue(2), Linux epoll(4), or Solaris /dev/poll). +

      + +
    • You can reduce the Postfix memory footprint by using cdb: +lookup tables instead of Berkeley DB.

      + +
      + 1 /etc/postfix/main.cf:
      + 2     # Raise the global process limit, 100 since Postfix 2.0.
      + 3     default_process_limit = 200
      + 4
      + 5 /etc/postfix/master.cf:
      + 6     # =============================================================
      + 7     # service type  private unpriv  chroot  wakeup  maxproc command
      + 8     # =============================================================
      + 9     # Raise the SMTP service process limit only.
      +10     smtp      inet  n       -       n       -       200     smtpd
      +
      + +
    • NOTE: older versions of the SMTPD_POLICY_README document +contain a mistake: they configure a fixed number of policy daemon +processes. When you raise the SMTP server's "maxproc" field in +master.cf, SMTP server processes will report problems when connecting +to policy server processes, because there aren't enough of them. +Examples of errors are "connection refused" or "operation timed +out". To fix, edit master.cf and specify a zero "maxproc" field +in all policy server entries; see line 6 in the example below. +Issue a "postfix reload" command to make the change effective.

      + +
      +1 /etc/postfix/master.cf:
      +2     # =============================================================
      +3     # service type  private unpriv  chroot  wakeup  maxproc command
      +4     # =============================================================
      +5     # Disable the policy service process limit.
      +6     policy    unix  -       n       n       -       0       spawn
      +7         user=nobody argv=/some/where/policy-server
      +
      + +
    + +

    Spend less time per SMTP client

    + +

    When increasing the number of SMTP server processes is not +practical, you can improve Postfix server responsiveness by eliminating +unnecessary work. When Postfix spends less time per SMTP session, the +same number of SMTP server processes can service more clients in +the same amount of time.

    + +
      + +
    • Eliminate non-functional RBL lookups (blocklists that are +no longer in operation). These lookups can degrade performance. +Postfix logs a warning when an RBL server does not respond.

      + +
    • Eliminate redundant RBL lookups (people often use multiple +Spamhaus RBLs that include each other). To find out whether RBLs +include other RBLs, look up the websites that document the RBL's +policies.

      + +
    • Eliminate header_checks and body_checks, and keep just a few +emergency patterns to block the latest worm explosion or backscatter +mail. See BACKSCATTER_README for examples of the latter. + +

    • Group your header_checks and body_checks patterns to avoid +unnecessary pattern matching operations. + +

      + 1  /etc/postfix/header_checks:
      + 2      if /^Subject:/
      + 3      /^Subject: virus found in mail from you/ reject
      + 4      /^Subject: ..../ ....
      + 5      endif
      + 6  
      + 7      if /^Received:/
      + 8      /^Received: from (postfix\.org) / reject forged client name in received header: $1
      + 9      /^Received: from .../ ....
      +10      endif
      +
      + +
    + +

    Disconnect suspicious SMTP clients

    + +

    Under conditions of overload you can improve Postfix SMTP server +responsiveness by hanging up on suspicious clients, so that other +clients get a chance to talk to Postfix.

    + +
      + +
    • Use "421" reply codes for botnet-related RBLs or for +selected non-RBL restrictions. This causes Postfix 2.3 and later +to disconnect immediately without waiting for the remote SMTP +client to send a QUIT command.

      + +

      You can set individual reject codes for RBLs, and for individual +responses from a specific RBL. We'll use zen.spamhaus.org as an +example; by the time you read this document, details may have +changed. Right now, their documents say that a response of 127.0.0.10 +or 127.0.0.11 indicates a dynamic client IP address, which means +that the machine is probably running a bot of some kind. To give +a 421 response instead of the default 554 response, use something +like:

      + +
      + 1  /etc/postfix/main.cf:
      + 2      smtpd_client_restrictions =
      + 3         permit_mynetworks
      + 4         reject_rbl_client zen.spamhaus.org=127.0.0.10
      + 5         reject_rbl_client zen.spamhaus.org=127.0.0.11
      + 6         reject_rbl_client zen.spamhaus.org
      + 7  
      + 8      rbl_reply_maps = hash:/etc/postfix/rbl_reply_maps
      + 9  
      +10  /etc/postfix/rbl_reply_maps:
      +11      zen.spamhaus.org=127.0.0.10 421 4.7.1 Service unavailable;
      +12       $rbl_class [$rbl_what] blocked using
      +13       $rbl_domain${rbl_reason?; $rbl_reason}
      +14  
      +15      zen.spamhaus.org=127.0.0.11 421 4.7.1 Service unavailable;
      +16       $rbl_class [$rbl_what] blocked using
      +17       $rbl_domain${rbl_reason?; $rbl_reason}
      +
      + +

      Although the above shows three RBL lookups (lines 4-6), Postfix +will still only do a single DNS query, so the performance difference +is negligible.

      + +

      The down-side of sending 421 instead of the default 554 is that +it works only for zombies and other malware. If the client is running +a real MTA, then it may connect again several times until the mail +expires in its queue. When this is a problem, stick with the default +554 reply, and use "smtpd_hard_error_limit = 1" as described below. +

      + +

      With Postfix 2.5, or with earlier releases that contain the +stress-adaptive behavior patch, you can turn on the above under +overload by replacing line 8 with:

      + +
      + 8      rbl_reply_maps = ${stress?hash:/etc/postfix/rbl_reply_maps}
      +
      + +

      More information about automatic stress-adaptive behavior is +at the end of this document.

      + +
    + +

    Take desperate measures

    + +

    The following measures will still allow most legitimate +clients to connect and send mail, but may affect some legitimate +clients.

    + +
      + +
    • Reduce smtpd_timeout (default: 300s). Experience on the +postfix-users list from a variety of sysadmins shows that reducing +the "normal" smtpd_timeout to 60s is unlikely to affect legitimate +clients. However, it is unlikely to become the Postfix default +because it's not RFC compliant. Setting smtpd_timeout to 10s (line +2 below) or even 5s under stress will still allow most +legitimate clients to connect and send mail, but may delay mail +from some clients. No mail should be lost, as long as this measure +is used only temporarily.

      + +
    • Reduce smtpd_hard_error_limit (default: 20). Setting this +to 1 under stress (line 3 below) helps by disconnecting clients +after a single error, giving other clients a chance to connect. +However, this may cause significant delays with legitimate mail, +such as a mailing list that contains a few no-longer-active user +names that didn't bother to unsubscribe. No mail should be lost, +as long as this measure is used only temporarily.

      + +
    • Disable remote SMTP client hostname lookups, so that all +SMTP client hostnames become "unknown" (line 5 below). This feature +was introduced with Postfix 2.3. Unfortunately, this measure is +more problematic than the other ones proposed sofar. First, this +will result in loss of mail when you use hostname-based access rules +that reject mail from "unknown" SMTP clients (examples: +reject_unknown_client_hostname, reject_unknown_reverse_client_hostname). +Second, this may result in loss of mail when you subject "unknown" +SMTP clients to additional restrictions such as reject_unverified_sender. +

      + +
    + +
    +
    +1  /etc/postfix/main.cf:
    +2      smtpd_timeout = 10
    +3      smtpd_hard_error_limit = 1
    +4      # Caution: line 5 may trigger REJECTs by hostname-based access rules 
    +5      smtpd_peername_lookup = no
    +
    +
    + +

    Except with the last measure, no mail should be lost, as long +as these measures are used only temporarily. The next section of +this document introduces a way to automate this process.

    + +

    Make Postfix behavior stress-adaptive

    + +

    Postfix version 2.5 introduces automatic stress-adaptive behavior. +This is also available as an add-on patch for Postfix versions 2.4 +and 2.3 from the mirrors listed at http://www.postfix.org/download.html. +

    + +

    It works as follows. When a "public" network service runs into +an "all server ports are busy" condition, the master(8) daemon logs +a warning, restarts the service (without interrupting existing +network sessions), and runs the service with "-o stress=yes" on the +command line. Normally, it runs a stress-adaptive service with "-o +stress=" on the command line (i.e. with an empty parameter value). +Other services never have "-o stress" parameters on the command +line, including services that listen on a loopback interface only. +

    + +

    The stress pseudo-parameter value is the key to making main.cf +parameter settings stress adaptive:

    + +
    +
    +1  /etc/postfix/main.cf:
    +2      smtpd_timeout = ${stress?10}${stress:300}
    +3      smtpd_hard_error_limit = ${stress?1}${stress:20}
    +
    +
    + +

    Translation:

    + +

      + +
    • Line 2: under conditions of stress, use an smtpd_timeout +value of 10 seconds instead of the default 300 seconds, + +

    • Line 3: under conditions of stress, use an smtpd_hard_error_limit +of 1 instead of the default 20.

      + +
    + +

    The syntax of ${name?value} and ${name:value} is explained at +the beginning of the postconf(5) manual page.

    + +

    NOTE: Please keep in mind that the stress-adaptive feature is +a fairly desperate measure to keep some legitimate mail +flowing under overload conditions. If a site is reaching the SMTP +server process limit when there isn't an attack or bot flood +occurring, then either the process limit needs to be raised or more +hardware needs to be added.

    + +

    Detecting support for stress-adaptive behavior

    + +

    To find out if your Postfix installation supports stress-adaptive +behavior, use the "ps" command, and look for the smtpd processes. +Postfix has stress-adaptive support when you see "-o stress=" or +"-o stress=yes" command-line options. Remember that Postfix never +enables stress-adaptive behavior on servers that listen on local +addresses only.

    + +

    The following example is for FreeBSD or Linux. On Solaris, HP-UX +and other System-V flavors, use "ps -ef" instead of "ps ax".

    + +
    +
    +$ ps ax|grep smtpd
    +83326  ??  S      0:00.28 smtpd -n smtp -t inet -u -c -o stress=
    +84345  ??  Ss     0:00.11 /usr/bin/perl /usr/libexec/postfix/smtpd-policy.pl
    +
    +
    + +

    You can't use postconf(1) to detect stress-adaptive support. +The postconf(1) command ignores the existence of the stress parameter +in main.cf, because the parameter has no effect there. Command-line +"-o parameter" settings always take precedence over main.cf parameter +settings.

    + +

    If you configure stress-adaptive behavior in main.cf when it +isn't supported, nothing bad will happen. The processes will run +as if the stress parameter always has an empty value.

    + +

    Forcing stress-adaptive behavior on or off

    + +

    You can manually force stress-adaptive behavior on, by adding +a "-o stress=yes" command-line option in master.cf. This can be +useful for testing overrides on the SMTP service. Issue "postfix +reload" to make the change effective.

    + +
    +
    +1 /etc/postfix/master.cf:
    +2     # =============================================================
    +3     # service type  private unpriv  chroot  wakeup  maxproc command
    +4     # =============================================================
    +5     # 
    +6     smtp      inet  n       -       n       -       -       smtpd
    +7         -o stress=yes
    +8         -o . . .
    +
    +
    + +

    To permanently force stress-adaptive behavior off with a specific +service, specify "-o stress=" on its command line. This may be +desirable for the "submission" service. Issue "postfix reload" to +make the change effective.

    + +
    +
    +1 /etc/postfix/master.cf:
    +2     # =============================================================
    +3     # service type  private unpriv  chroot  wakeup  maxproc command
    +4     # =============================================================
    +5     # 
    +6     submission inet n       -       n       -       -       smtpd
    +7         -o stress=
    +8         -o . . .
    +
    +
    + +

    Credits

    + +
      + +
    • Thanks to the postfix-users mailing list members for sharing +early experiences with the stress-adaptive feature. + +
    • The RBL example and several other paragraphs of text were +adapted from postfix-users postings by Noel Jones. + +
    • Wietse implemented stress-adaptive behavior as the smallest +possible patch while he should be working on other things. + +
    + + diff --git a/postfix/proto/header_checks b/postfix/proto/header_checks index ae73e1df0..33f428e5f 100644 --- a/postfix/proto/header_checks +++ b/postfix/proto/header_checks @@ -81,17 +81,17 @@ # For a discussion of specific pattern or flags syntax, # see \fBpcre_table\fR(5) or \fBregexp_table\fR(5), respectively. # .IP "\fB/\fIpattern\fB/\fIflags action\fR" -# When \fIpattern\fR matches the input string, execute +# When /\fIpattern\fR/ matches the input string, execute # the corresponding \fIaction\fR. See below for a list # of possible actions. # .IP "\fB!/\fIpattern\fB/\fIflags action\fR" -# When \fIpattern\fR does \fBnot\fR match the input string, +# When /\fIpattern\fR/ does \fBnot\fR match the input string, # execute the corresponding \fIaction\fR. # .IP "\fBif /\fIpattern\fB/\fIflags\fR" # .IP "\fBendif\fR" # Match the input string against the patterns between \fBif\fR # and \fBendif\fR, if and only if the same input string also -# matches \fIpattern\fR. The \fBif\fR..\fBendif\fR can nest. +# matches /\fIpattern\fR/. The \fBif\fR..\fBendif\fR can nest. # .sp # Note: do not prepend whitespace to patterns inside # \fBif\fR..\fBendif\fR. @@ -99,7 +99,7 @@ # .IP "\fBendif\fR" # Match the input string against the patterns between \fBif\fR # and \fBendif\fR, if and only if the same input string does -# \fBnot\fR match \fIpattern\fR. The \fBif\fR..\fBendif\fR +# \fBnot\fR match /\fIpattern\fR/. The \fBif\fR..\fBendif\fR # can nest. # .IP "blank lines and comments" # Empty lines and whitespace-only lines are ignored, as @@ -281,6 +281,10 @@ # action is useful for debugging and for testing a pattern # before applying more drastic actions. # BUGS +# Empty lines never match, because some map types mis-behave +# when given a zero-length search string. This limitation may +# be removed for regular expression tables in a future release. +# # Many people overlook the main limitations of header and body_checks # rules. # .IP \(bu diff --git a/postfix/proto/postconf.proto b/postfix/proto/postconf.proto index bc5ed79b4..ce8fda2f1 100644 --- a/postfix/proto/postconf.proto +++ b/postfix/proto/postconf.proto @@ -2360,6 +2360,11 @@ and later.
    The recipient's username.
    +
    ORIGINAL_RECIPIENT
    + +
    The entire recipient address, before any address rewriting or +aliasing.
    +
    RECIPIENT
    The full recipient address.
    @@ -10633,3 +10638,39 @@ the hostname and IP address. The logging format is "host[address]:port".

    This feature is available in Postfix 2.5 and later.

    + +%PARAM smtp_header_checks + +

    Restricted header_checks(5) tables for the Postfix SMTP client. +These tables are searched while mail is being delivered. Actions +that change the delivery time or destination are not available. +

    + +

    This feature is available in Postfix 2.5 and later.

    + +%PARAM smtp_mime_header_checks + +

    Restricted mime_header_checks(5) tables for the Postfix SMTP +client. These tables are searched while mail is being delivered. +Actions that change the delivery time or destination are not +available.

    + +

    This feature is available in Postfix 2.5 and later.

    + +%PARAM smtp_nested_header_checks + +

    Restricted nested_header_checks(5) tables for the Postfix SMTP +client. These tables are searched while mail is being delivered. +Actions that change the delivery time or destination are not +available.

    + +

    This feature is available in Postfix 2.5 and later.

    + +%PARAM smtp_body_checks + +

    Restricted body_checks(5) tables for the Postfix SMTP client. +These tables are searched while mail is being delivered. Actions +that change the delivery time or destination are not available. +

    + +

    This feature is available in Postfix 2.5 and later.

    diff --git a/postfix/src/global/Makefile.in b/postfix/src/global/Makefile.in index 881440680..d077f3dc6 100644 --- a/postfix/src/global/Makefile.in +++ b/postfix/src/global/Makefile.in @@ -28,7 +28,7 @@ SRCS = abounce.c anvil_clnt.c been_here.c bounce.c bounce_log.c \ tok822_resolve.c tok822_rewrite.c tok822_tree.c trace.c \ user_acl.c valid_mailhost_addr.c verify.c verify_clnt.c \ verp_sender.c wildcard_inet_addr.c xtext.c delivered_hdr.c \ - fold_addr.c + fold_addr.c header_body_checks.c OBJS = abounce.o anvil_clnt.o been_here.o bounce.o bounce_log.o \ canon_addr.o cfg_parser.o cleanup_strerror.o cleanup_strflags.o \ clnt_stream.o conv_time.o db_common.o debug_peer.o debug_process.o \ @@ -58,7 +58,7 @@ OBJS = abounce.o anvil_clnt.o been_here.o bounce.o bounce_log.o \ tok822_resolve.o tok822_rewrite.o tok822_tree.o trace.o \ user_acl.o valid_mailhost_addr.o verify.o verify_clnt.o \ verp_sender.o wildcard_inet_addr.o xtext.o delivered_hdr.o \ - fold_addr.o + fold_addr.o header_body_checks.o HDRS = abounce.h anvil_clnt.h been_here.h bounce.h bounce_log.h \ canon_addr.h cfg_parser.h cleanup_user.h clnt_stream.h config.h \ conv_time.h db_common.h debug_peer.h debug_process.h defer.h \ @@ -82,7 +82,7 @@ HDRS = abounce.h anvil_clnt.h been_here.h bounce.h bounce_log.h \ string_list.h strip_addr.h sys_exits.h timed_ipc.h tok822.h \ trace.h user_acl.h valid_mailhost_addr.h verify.h verify_clnt.h \ verp_sender.h wildcard_inet_addr.h xtext.h delivered_hdr.h \ - fold_addr.h + fold_addr.h header_body_checks.h TESTSRC = rec2stream.c stream2rec.c recdump.c DEFS = -I. -I$(INC_DIR) -D$(SYSTYPE) CFLAGS = $(DEBUG) $(OPT) $(DEFS) @@ -94,7 +94,7 @@ TESTPROG= domain_list dot_lockfile mail_addr_crunch mail_addr_find \ resolve_local rewrite_clnt stream2rec string_list tok822_parse \ quote_821_local mail_conf_time mime_state strip_addr \ verify_clnt xtext anvil_clnt scache ehlo_mask \ - valid_mailhost_addr own_inet_addr + valid_mailhost_addr own_inet_addr header_body_checks LIBS = ../../lib/libutil.a LIB_DIR = ../../lib @@ -271,13 +271,20 @@ valid_mailhost_addr: valid_mailhost_addr.c $(LIB) $(LIBS) own_inet_addr: own_inet_addr.c $(LIB) $(LIBS) $(CC) $(CFLAGS) -DTEST -o $@ $@.c $(LIB) $(LIBS) $(SYSLIBS) +header_body_checks: header_body_checks.c $(LIB) $(LIBS) + $(CC) $(CFLAGS) -DTEST -o $@ $@.c $(LIB) $(LIBS) $(SYSLIBS) + tests: tok822_test mime_tests strip_addr_test tok822_limit_test \ xtext_test scache_multi_test ehlo_mask_test \ - namadr_list_test mail_conf_time_test + namadr_list_test mail_conf_time_test header_body_checks_tests mime_tests: mime_test mime_nest mime_8bit mime_dom mime_trunc mime_cvt \ mime_cvt2 mime_cvt3 mime_garb1 mime_garb2 mime_garb3 mime_garb4 +header_body_checks_tests: header_body_checks_null_test \ + header_body_checks_warn_test header_body_checks_prepend_test \ + header_body_checks_ignore_test header_body_checks_replace_test + root_tests: rewrite_clnt_test resolve_clnt_test tok822_test: tok822_parse tok822_parse.in tok822_parse.ref @@ -416,6 +423,56 @@ mail_conf_time_test: mail_conf_time mail_conf_time.ref diff mail_conf_time.ref mail_conf_time.tmp rm -f mail_conf_time.tmp +header_body_checks_null_test: header_body_checks header_body_checks_null.ref + ./header_body_checks "" "" "" "" \ + header_body_checks_null.tmp 2>&1 + cmp header_body_checks_null.ref header_body_checks_null.tmp + ./header_body_checks static:dunno static:dunno static:dunno static:dunno \ + header_body_checks_null.tmp 2>&1 + cmp header_body_checks_null.ref header_body_checks_null.tmp + ./header_body_checks static:ok static:ok static:ok static:ok \ + header_body_checks_null.tmp 2>&1 + cmp header_body_checks_null.ref header_body_checks_null.tmp + rm -f header_body_checks_null.tmp + +header_body_checks_warn_test: header_body_checks header_body_checks_warn.ref + ./header_body_checks static:warn static:warn static:warn static:warn \ + header_body_checks_warn.tmp 2>&1 + cmp header_body_checks_warn.ref header_body_checks_warn.tmp + rm -f header_body_checks_warn.tmp + +header_body_checks_prepend_test: header_body_checks header_body_checks_prepend.ref + echo /./ prepend header: head >header_body_checks_head + echo /./ prepend header: mime >header_body_checks_mime + echo /./ prepend header: nest >header_body_checks_nest + echo /./ prepend body >header_body_checks_body + ./header_body_checks regexp:header_body_checks_head regexp:header_body_checks_mime \ + regexp:header_body_checks_nest regexp:header_body_checks_body \ + header_body_checks_prepend.tmp 2>&1 + cmp header_body_checks_prepend.ref header_body_checks_prepend.tmp + rm -f header_body_checks_prepend.tmp header_body_checks_head header_body_checks_mime header_body_checks_nest header_body_checks_body + +# Note: the IGNORE action will not strip empty lines. Postfix maps +# currently never see null query strings because some map types raise +# errors. We can eliminate this restriction by allowing individual +# map types to advertise whether they can handle null queries. +header_body_checks_ignore_test: header_body_checks header_body_checks_ignore.ref + ./header_body_checks static:ignore static:ignore static:ignore static:ignore \ + header_body_checks_ignore.tmp 2>&1 + cmp header_body_checks_ignore.ref header_body_checks_ignore.tmp + rm -f header_body_checks_ignore.tmp header_body_checks_head header_body_checks_mime header_body_checks_nest header_body_checks_body + +header_body_checks_replace_test: header_body_checks header_body_checks_replace.ref + echo /./ replace header: head >header_body_checks_head + echo /./ replace header: mime >header_body_checks_mime + echo /./ replace header: nest >header_body_checks_nest + echo /./ replace body >header_body_checks_body + ./header_body_checks regexp:header_body_checks_head regexp:header_body_checks_mime \ + regexp:header_body_checks_nest regexp:header_body_checks_body \ + header_body_checks_replace.tmp 2>&1 + cmp header_body_checks_replace.ref header_body_checks_replace.tmp + rm -f header_body_checks_replace.tmp header_body_checks_head header_body_checks_mime header_body_checks_nest header_body_checks_body + printfck: $(OBJS) $(PROG) rm -rf printfck mkdir printfck @@ -709,6 +766,21 @@ delivered_hdr.o: quote_822_local.h delivered_hdr.o: quote_flags.h delivered_hdr.o: rec_type.h delivered_hdr.o: record.h +header_body_checks.o: ../../include/argv.h +header_body_checks.o: ../../include/dict.h +header_body_checks.o: ../../include/msg.h +header_body_checks.o: ../../include/mymalloc.h +header_body_checks.o: ../../include/sys_defs.h +header_body_checks.o: ../../include/vbuf.h +header_body_checks.o: ../../include/vstream.h +header_body_checks.o: ../../include/vstring.h +header_body_checks.o: header_body_checks.c +header_body_checks.o: header_body_checks.h +header_body_checks.o: header_opts.h +header_body_checks.o: is_header.h +header_body_checks.o: maps.h +header_body_checks.o: mime_state.h +header_body_checks.o: rec_type.h dict_ldap.o: ../../include/argv.h dict_ldap.o: ../../include/binhash.h dict_ldap.o: ../../include/dict.h diff --git a/postfix/src/global/header_body_checks.c b/postfix/src/global/header_body_checks.c new file mode 100644 index 000000000..1ead675dd --- /dev/null +++ b/postfix/src/global/header_body_checks.c @@ -0,0 +1,637 @@ +/*++ +/* NAME +/* header_body_checks 3 +/* SUMMARY +/* header/body checks +/* SYNOPSIS +/* #include +/* +/* typedef struct { +/* void (*logger) (void *context, const char *action, +/* const char *where, const char *line, +/* const char *optional_text); +/* void (*prepend) (void *context, int rec_type, +/* const char *buf, ssize_t len, off_t offset); +/* char *(*extend) (void *context, const char *command, +/* int cmd_len, const char *cmd_args, +/* const char *where, const char *line, +/* ssize_t line_len, off_t offset); +/* } HBC_CALL_BACKS; +/* +/* HBC_CHECKS *hbc_header_checks_create( +/* header_checks_name, header_checks_value +/* mime_header_checks_name, mime_header_checks_value, +/* nested_header_checks_name, nested_header_checks_value, +/* call_backs) +/* const char *header_checks_name; +/* const char *header_checks_value; +/* const char *mime_header_checks_name; +/* const char *mime_header_checks_value; +/* const char *nested_header_checks_name; +/* const char *nested_header_checks_value; +/* HBC_CALL_BACKS *call_backs; +/* +/* HBC_CHECKS *hbc_body_checks_create( +/* body_checks_name, body_checks_value, +/* call_backs) +/* const char *body_checks_name; +/* const char *body_checks_value; +/* HBC_CALL_BACKS *call_backs; +/* +/* char *hbc_header_checks(context, hbc, header_class, hdr_opts, header) +/* void *context; +/* HBC_CHECKS *hbc; +/* int header_class; +/* HEADER_OPTS *hdr_opts; +/* VSTRING *header; +/* +/* char *hbc_body_checks(context, hbc, body_line, body_line_len) +/* void *context; +/* HBC_CHECKS *hbc; +/* const char *body_line; +/* ssize_t body_line_len; +/* +/* void hbc_header_checks_free(hbc) +/* HBC_CHECKS *hbc; +/* +/* void hbc_body_checks_free(hbc) +/* HBC_CHECKS *hbc; +/* DESCRIPTION +/* This module implements header_checks and body_checks. +/* Actions are executed while mail is being delivered. The +/* following actions are recognized: WARN, REPLACE, PREPEND, +/* IGNORE, DUNNO, and OK. These actions are safe for use in +/* delivery agents. +/* +/* Other actions may be supplied via the extension mechanism +/* described below. For example, actions that change the +/* message delivery time or destination. Such actions do not +/* make sense in delivery agents, but they can be appropriate +/* in, for example, before-queue filters. +/* +/* hbc_header_checks_create() creates a context for header +/* inspection. This function is typically called once during +/* program initialization. The result is a null pointer when +/* all _value arguments specify zero-length strings; in this +/* case, hbc_header_checks() and hbc_header_checks_free() must +/* not be called. +/* +/* hbc_header_checks() inspects the specified logical header. +/* The result is either the original header, HBC_CHECK_STAT_IGNORE +/* (meaning: discard the header) or a new header (meaning: +/* replace the header and destroy the new header with myfree()). +/* +/* hbc_header_checks_free() returns memory to the pool. +/* +/* hbc_body_checks_create(), dbhc_body_checks(), dbhc_body_free() +/* perform similar functions for body lines. +/* +/* Arguments: +/* .IP body_line +/* One line of body text. +/* .IP body_line_len +/* Body line length. +/* .IP call_backs +/* Table with call-back function pointers. This argument is +/* not copied. Note: the description below is not necessarily +/* in data structure order. +/* .RS +/* .IP logger +/* Call-back function for logging an action with the action's +/* name in lower case, a location within a message ("header" +/* or "body"), the content of the header or body line that +/* triggered the action, and optional text or a zero-length +/* string. This call-back feature must be specified. +/* .IP prepend +/* Call-back function for the PREPEND action. The arguments +/* are the same as those of mime_state(3) body output call-back +/* functions. Specify a null pointer to disable this action. +/* .IP extend +/* Call-back function that logs and executes other actions. +/* This function receives as arguments the command name and +/* name length, the command arguments if any, the location +/* within the message ("header" or "body"), the content and +/* length of the header or body line that triggered the action, +/* and the input byte offset within the current header or body +/* segment. The result value is either the original line +/* argument, HBC_CHECKS_STAT_IGNORE (delete the line from the +/* input stream) or HBC_CHECK_STAT_UNKNOWN (the command was +/* not recognized). Specify a null pointer to disable this +/* feature. +/* .RE +/* .IP context +/* Application context for call-back functions specified with the +/* call_backs argument. +/* .IP header +/* A logical message header. Lines within a multi-line header +/* are separated by a newline character. +/* .IP "header_checks_name, mime_header_checks_name" +/* .IP "nested_header_checks_name, body_checks_name" +/* The main.cf configuration parameter names for header and body +/* map lists. +/* .IP "header_checks_value, mime_header_checks_value" +/* .IP "nested_header_checks_value, body_checks_value" +/* The values of main.cf configuration parameters for header and body +/* map lists. Specify a zero-length string to disable a specific list. +/* .IP header_class +/* A number in the range MIME_HDR_FIRST..MIME_HDR_LAST. +/* .IP hbc +/* A handle created with hbc_header_checks_create() or +/* hbc_body_checks_create(). +/* .IP hdr_opts +/* Message header properties. +/* SEE ALSO +/* msg(3) diagnostics interface +/* DIAGNOSTICS +/* Fatal errors: memory allocation problem. +/* LICENSE +/* .ad +/* .fi +/* The Secure Mailer license must be distributed with this software. +/* AUTHOR(S) +/* Wietse Venema +/* IBM T.J. Watson Research +/* P.O. Box 704 +/* Yorktown Heights, NY 10598, USA +/*--*/ + +/* System library. */ + +#include +#include +#include +#ifdef STRCASECMP_IN_STRINGS_H +#include +#endif + +/* Utility library. */ + +#include +#include + +/* Global library. */ + +#include +#include +#include +#include +#include +#include + +/* Application-specific. */ + + /* + * Something that is guaranteed to be different from a real string result + * from header/body_checks. + */ +const char hbc_checks_unknown; + + /* + * Header checks are stored as an array of HBC_MAP_INFO structures, one + * structure for each header class (MIME_HDR_PRIMARY, MIME_HDR_MULTIPART, or + * MIME_HDR_NESTED). + * + * Body checks are stored as one single HBC_MAP_INFO structure, because we make + * no distinction between body segments. + */ +#define HBC_HEADER_INDEX(class) ((class) - MIME_HDR_FIRST) +#define HBC_BODY_INDEX (0) + +#define HBC_INIT(hbc, index, name, value) do { \ + HBC_MAP_INFO *_mp = (hbc)->map_info + (index); \ + if (*(value) != 0) { \ + _mp->map_class = (name); \ + _mp->maps = maps_create((name), (value), DICT_FLAG_LOCK); \ + } else { \ + _mp->map_class = 0; \ + _mp->maps = 0; \ + } \ + } while (0) + +/* How does the action routine know where we are? */ + +#define HBC_CTXT_HEADER "header" +#define HBC_CTXT_BODY "body" + +/* Silly little macros. */ + +#define STR(x) vstring_str(x) +#define LEN(x) VSTRING_LEN(x) + +/* hbc_action - act upon a header/body match */ + +static char *hbc_action(void *context, HBC_CALL_BACKS *cb, + const char *map_class, const char *where, + const char *cmd, const char *line, + ssize_t line_len, off_t offset) +{ + const char *cmd_args = cmd + strcspn(cmd, " \t"); + int cmd_len = cmd_args - cmd; + char *ret; + + /* + * XXX We don't delegate action logging to the action call-back + * functions, because actions without call-back must be logged here + * anyway. This means that some actions must report back whether the + * action should be logged. This is admittedly a little clumsy. + * + * XXX We don't use a hash table for action lookup. Mail rarely triggers an + * action, and mail that triggers multiple actions is even rarer. + */ + while (*cmd_args && ISSPACE(*cmd_args)) + cmd_args++; + +#define STREQUAL(x,y,l) (strncasecmp((x), (y), (l)) == 0 && (y)[l] == 0) + + if (cb->extend + && (ret = cb->extend(context, cmd, cmd_len, cmd_args, where, line, + line_len, offset)) != HBC_CHECKS_STAT_UNKNOWN) + return (ret); + + if (STREQUAL(cmd, "WARN", cmd_len)) { + cb->logger(context, "warning", where, line, cmd_args); + return ((char *) line); + } + if (STREQUAL(cmd, "REPLACE", cmd_len)) { + if (*cmd_args == 0) { + msg_warn("REPLACE action without text in %s map", map_class); + return ((char *) line); + } else if (strcmp(where, HBC_CTXT_HEADER) == 0 + && !is_header(cmd_args)) { + msg_warn("bad REPLACE header text \"%s\" in %s map -- " + "need \"headername: headervalue\"", cmd_args, map_class); + return ((char *) line); + } else { + cb->logger(context, "replace", where, line, cmd_args); + return (mystrdup(cmd_args)); + } + } + if (cb->prepend && STREQUAL(cmd, "PREPEND", cmd_len)) { + if (*cmd_args == 0) { + msg_warn("PREPEND action without text in %s map", map_class); + } else if (strcmp(where, HBC_CTXT_HEADER) == 0 + && !is_header(cmd_args)) { + msg_warn("bad PREPEND header text \"%s\" in %s map -- " + "need \"headername: headervalue\"", cmd_args, map_class); + } else { + cb->logger(context, "prepend", where, line, cmd_args); + cb->prepend(context, REC_TYPE_NORM, cmd_args, strlen(cmd_args), offset); + } + return ((char *) line); + } + /* Allow and ignore optional text after the action. */ + + if (STREQUAL(cmd, "IGNORE", cmd_len)) + /* XXX Not logged for compatibility with cleanup(8). */ + return (HBC_CHECKS_STAT_IGNORE); + + if (STREQUAL(cmd, "DUNNO", cmd_len) /* preferred */ + ||STREQUAL(cmd, "OK", cmd_len)) /* compatibility */ + return ((char *) line); + + msg_warn("unsupported command in %s map: %s", map_class, cmd); + return ((char *) line); +} + +/* hbc_header_checks - process one complete header line */ + +char *hbc_header_checks(void *context, HBC_CHECKS *hbc, int header_class, + HEADER_OPTS *hdr_opts, + VSTRING *header, off_t offset) +{ + const char *myname = "hbc_header_checks"; + const char *action; + HBC_MAP_INFO *mp; + + if (msg_verbose) + msg_info("%s: '%.30s'", myname, STR(header)); + + /* + * XXX This is for compatibility with the cleanup(8) server. + */ + if (hdr_opts && (hdr_opts->flags & HDR_OPT_MIME)) + header_class = MIME_HDR_MULTIPART; + + mp = hbc->map_info + HBC_HEADER_INDEX(header_class); + + if (mp->maps != 0 && (action = maps_find(mp->maps, STR(header), 0)) != 0) { + return (hbc_action(context, hbc->call_backs, + mp->map_class, HBC_CTXT_HEADER, action, + STR(header), LEN(header), offset)); + } else { + return (STR(header)); + } +} + +/* hbc_body_checks - inspect one body record */ + +char *hbc_body_checks(void *context, HBC_CHECKS *hbc, const char *line, + ssize_t len, off_t offset) +{ + const char *myname = "hbc_body_checks"; + const char *action; + HBC_MAP_INFO *mp; + + if (msg_verbose) + msg_info("%s: '%.30s'", myname, line); + + mp = hbc->map_info; + + if ((action = maps_find(mp->maps, line, 0)) != 0) { + return (hbc_action(context, hbc->call_backs, + mp->map_class, HBC_CTXT_BODY, action, + line, len, offset)); + } else { + return ((char *) line); + } +} + +/* hbc_header_checks_create - create header checking context */ + +HBC_CHECKS *hbc_header_checks_create(const char *header_checks_name, + const char *header_checks_value, + const char *mime_header_checks_name, + const char *mime_header_checks_value, + const char *nested_header_checks_name, + const char *nested_header_checks_value, + HBC_CALL_BACKS *call_backs) +{ + HBC_CHECKS *hbc; + + /* + * Optimize for the common case. + */ + if (*header_checks_value == 0 && *mime_header_checks_value == 0 + && *nested_header_checks_value == 0) { + return (0); + } else { + hbc = (HBC_CHECKS *) mymalloc(sizeof(*hbc) + + (MIME_HDR_LAST - MIME_HDR_FIRST) * sizeof(HBC_MAP_INFO)); + hbc->call_backs = call_backs; + HBC_INIT(hbc, HBC_HEADER_INDEX(MIME_HDR_PRIMARY), + header_checks_name, header_checks_value); + HBC_INIT(hbc, HBC_HEADER_INDEX(MIME_HDR_MULTIPART), + mime_header_checks_name, mime_header_checks_value); + HBC_INIT(hbc, HBC_HEADER_INDEX(MIME_HDR_NESTED), + nested_header_checks_name, nested_header_checks_value); + return (hbc); + } +} + +/* hbc_body_checks_create - create body checking context */ + +HBC_CHECKS *hbc_body_checks_create(const char *body_checks_name, + const char *body_checks_value, + HBC_CALL_BACKS *call_backs) +{ + HBC_CHECKS *hbc; + + /* + * Optimize for the common case. + */ + if (*body_checks_value == 0) { + return (0); + } else { + hbc = (HBC_CHECKS *) mymalloc(sizeof(*hbc)); + hbc->call_backs = call_backs; + HBC_INIT(hbc, HBC_BODY_INDEX, body_checks_name, body_checks_value); + return (hbc); + } +} + +/* _hbc_checks_free - destroy header/body checking context */ + +void _hbc_checks_free(HBC_CHECKS *hbc, ssize_t len) +{ + HBC_MAP_INFO *mp; + + for (mp = hbc->map_info; mp < hbc->map_info + len; mp++) + if (mp->maps) + maps_free(mp->maps); + myfree((char *) hbc); +} + + /* + * Test program. Specify the four maps on the command line, and feed a + * MIME-formatted message on stdin. + */ + +#ifdef TEST + +#include +#include +#include +#include +#include + +typedef struct { + HBC_CHECKS *header_checks; + HBC_CHECKS *body_checks; + HBC_CALL_BACKS *call_backs; + VSTREAM *fp; + VSTRING *buf; + const char *queueid; + int recno; +} HBC_TEST_CONTEXT; + +/*#define REC_LEN 40*/ +#define REC_LEN 1024 + +/* log_cb - log action with context */ + +static void log_cb(void *context, const char *action, const char *where, + const char *content, const char *text) +{ + const HBC_TEST_CONTEXT *dp = (HBC_TEST_CONTEXT *) context; + + if (*text) { + msg_info("%s: %s: %s %.200s: %s", + dp->queueid, action, where, content, text); + } else { + msg_info("%s: %s: %s %.200s", + dp->queueid, action, where, content); + } +} + +/* out_cb - output call-back */ + +static void out_cb(void *context, int rec_type, const char *buf, + ssize_t len, off_t offset) +{ + const HBC_TEST_CONTEXT *dp = (HBC_TEST_CONTEXT *) context; + + vstream_fwrite(dp->fp, buf, len); + VSTREAM_PUTC('\n', dp->fp); + vstream_fflush(dp->fp); +} + +/* head_out - MIME_STATE header call-back */ + +static void head_out(void *context, int header_class, HEADER_OPTS *header_info, + VSTRING *buf, off_t offset) +{ + HBC_TEST_CONTEXT *dp = (HBC_TEST_CONTEXT *) context; + char *out; + + if (dp->header_checks == 0 + || (out = hbc_header_checks(context, dp->header_checks, header_class, + header_info, buf, offset)) == STR(buf)) { + vstring_sprintf(dp->buf, "%d %s %ld\t|%s", + dp->recno, + header_class == MIME_HDR_PRIMARY ? "MAIN" : + header_class == MIME_HDR_MULTIPART ? "MULT" : + header_class == MIME_HDR_NESTED ? "NEST" : + "ERROR", (long) offset, STR(buf)); + out_cb(dp, REC_TYPE_NORM, STR(dp->buf), LEN(dp->buf), offset); + } else if (out != 0) { + vstring_sprintf(dp->buf, "%d %s %ld\t|%s", + dp->recno, + header_class == MIME_HDR_PRIMARY ? "MAIN" : + header_class == MIME_HDR_MULTIPART ? "MULT" : + header_class == MIME_HDR_NESTED ? "NEST" : + "ERROR", (long) offset, out); + out_cb(dp, REC_TYPE_NORM, STR(dp->buf), LEN(dp->buf), offset); + myfree(out); + } + dp->recno += 1; +} + +/* header_end - MIME_STATE end-of-header call-back */ + +static void head_end(void *context) +{ + HBC_TEST_CONTEXT *dp = (HBC_TEST_CONTEXT *) context; + + out_cb(dp, 0, "HEADER END", sizeof("HEADER END") - 1, 0); +} + +/* body_out - MIME_STATE body line call-back */ + +static void body_out(void *context, int rec_type, const char *buf, + ssize_t len, off_t offset) +{ + HBC_TEST_CONTEXT *dp = (HBC_TEST_CONTEXT *) context; + char *out; + + if (dp->body_checks == 0 + || (out = hbc_body_checks(context, dp->body_checks, + buf, len, offset)) == buf) { + vstring_sprintf(dp->buf, "%d BODY %c %ld\t|%s", + dp->recno, rec_type, (long) offset, buf); + out_cb(dp, rec_type, STR(dp->buf), LEN(dp->buf), offset); + } else if (out != 0) { + vstring_sprintf(dp->buf, "%d BODY %c %ld\t|%s", + dp->recno, rec_type, (long) offset, out); + out_cb(dp, rec_type, STR(dp->buf), LEN(dp->buf), offset); + myfree(out); + } + dp->recno += 1; +} + +/* body_end - MIME_STATE end-of-message call-back */ + +static void body_end(void *context) +{ + HBC_TEST_CONTEXT *dp = (HBC_TEST_CONTEXT *) context; + + out_cb(dp, 0, "BODY END", sizeof("BODY END") - 1, 0); +} + +/* err_print - print MIME_STATE errors */ + +static void err_print(void *unused_context, int err_flag, + const char *text, ssize_t len) +{ + msg_warn("%s: %.*s", mime_state_error(err_flag), + len < 100 ? (int) len : 100, text); +} + +int var_header_limit = 2000; +int var_mime_maxdepth = 20; +int var_mime_bound_len = 2000; + +int main(int argc, char **argv) +{ + int rec_type; + VSTRING *buf; + int err; + MIME_STATE *mime_state; + HBC_TEST_CONTEXT context; + static HBC_CALL_BACKS call_backs[1] = { + log_cb, /* logger */ + out_cb, /* prepend */ + }; + + /* + * Sanity check. + */ + if (argc != 5) + msg_fatal("usage: %s header_checks mime_header_checks nested_header_checks body_checks", argv[0]); + + /* + * Initialize. + */ +#define MIME_OPTIONS \ + (MIME_OPT_REPORT_8BIT_IN_7BIT_BODY \ + | MIME_OPT_REPORT_8BIT_IN_HEADER \ + | MIME_OPT_REPORT_ENCODING_DOMAIN \ + | MIME_OPT_REPORT_TRUNC_HEADER \ + | MIME_OPT_REPORT_NESTING \ + | MIME_OPT_DOWNGRADE) + msg_vstream_init(basename(argv[0]), VSTREAM_OUT); + buf = vstring_alloc(10); + mime_state = mime_state_alloc(MIME_OPTIONS, + head_out, head_end, + body_out, body_end, + err_print, + (void *) &context); + context.header_checks = + hbc_header_checks_create("header_checks", argv[1], + "mime_header_checks", argv[2], + "nested_header_checks", argv[3], + call_backs); + context.body_checks = + hbc_body_checks_create("body_checks", argv[4], call_backs); + context.buf = vstring_alloc(100); + context.fp = VSTREAM_OUT; + context.queueid = "test-queueID"; + context.recno = 0; + + /* + * Main loop. + */ + do { + rec_type = rec_streamlf_get(VSTREAM_IN, buf, REC_LEN); + VSTRING_TERMINATE(buf); + err = mime_state_update(mime_state, rec_type, STR(buf), LEN(buf)); + vstream_fflush(VSTREAM_OUT); + } while (rec_type > 0); + + /* + * Error reporting. + */ + if (err & MIME_ERR_TRUNC_HEADER) + msg_warn("message header length exceeds safety limit"); + if (err & MIME_ERR_NESTING) + msg_warn("MIME nesting exceeds safety limit"); + if (err & MIME_ERR_8BIT_IN_HEADER) + msg_warn("improper use of 8-bit data in message header"); + if (err & MIME_ERR_8BIT_IN_7BIT_BODY) + msg_warn("improper use of 8-bit data in message body"); + if (err & MIME_ERR_ENCODING_DOMAIN) + msg_warn("improper message/* or multipart/* encoding domain"); + + /* + * Cleanup. + */ + if (context.header_checks) + hbc_header_checks_free(context.header_checks); + if (context.body_checks) + hbc_body_checks_free(context.body_checks); + vstring_free(context.buf); + mime_state_free(mime_state); + vstring_free(buf); + exit(0); +} + +#endif diff --git a/postfix/src/global/header_body_checks.h b/postfix/src/global/header_body_checks.h new file mode 100644 index 000000000..2fc37677b --- /dev/null +++ b/postfix/src/global/header_body_checks.h @@ -0,0 +1,81 @@ +#ifndef _HBC_H_INCLUDED_ +#define _HBC_H_INCLUDED_ + +/*++ +/* NAME +/* header_body_checks 3h +/* SUMMARY +/* delivery agent header/body checks +/* SYNOPSIS +/* #include +/* DESCRIPTION +/* .nf + + /* + * Global library. + */ +#include +#include + + /* + * Postfix < 2.5 compatibility. + */ +#ifndef MIME_HDR_FIRST +#define MIME_HDR_FIRST (1) +#define MIME_HDR_LAST (3) +#endif + + /* + * External interface. + */ +typedef struct { + const char *map_class; /* parameter name */ + MAPS *maps; /* map handle */ +} HBC_MAP_INFO; + +typedef struct { + void (*logger) (void *, const char *, const char *, const char *, const char *); + void (*prepend) (void *, int, const char *, ssize_t, off_t); + char *(*extend) (void *, const char *, int, const char *, const char *, const char *, ssize_t, off_t); +} HBC_CALL_BACKS; + +typedef struct { + HBC_CALL_BACKS *call_backs; + HBC_MAP_INFO map_info[1]; /* actually, a bunch */ +} HBC_CHECKS; + +#define HBC_CHECKS_STAT_IGNORE ((char *) 0) +#define HBC_CHECKS_STAT_UNKNOWN (&hbc_checks_unknown) + +extern HBC_CHECKS *hbc_header_checks_create(const char *, const char *, + const char *, const char *, + const char *, const char *, + HBC_CALL_BACKS *); +extern HBC_CHECKS *hbc_body_checks_create(const char *, const char *, + HBC_CALL_BACKS *); +extern char *hbc_header_checks(void *, HBC_CHECKS *, int, HEADER_OPTS *, + VSTRING *, off_t); +extern char *hbc_body_checks(void *, HBC_CHECKS *, const char *, ssize_t, off_t); + +#define hbc_header_checks_free(hbc) _hbc_checks_free((hbc), HBC_HEADER_SIZE) +#define hbc_body_checks_free(hbc) _hbc_checks_free((hbc), 1) + + /* + * The following are NOT part of the external API. + */ +#define HBC_HEADER_SIZE (MIME_HDR_LAST - MIME_HDR_FIRST + 1) +extern void _hbc_checks_free(HBC_CHECKS *, ssize_t); +extern const char hbc_checks_unknown; + +/* LICENSE +/* .ad +/* .fi +/* The Secure Mailer license must be distributed with this software. +/* AUTHOR(S) +/* Wietse Venema +/* IBM T.J. Watson Research +/* P.O. Box 704 +/* Yorktown Heights, NY 10598, USA +/*--*/ + +#endif diff --git a/postfix/src/global/header_body_checks_ignore.ref b/postfix/src/global/header_body_checks_ignore.ref new file mode 100644 index 000000000..0d7283856 --- /dev/null +++ b/postfix/src/global/header_body_checks_ignore.ref @@ -0,0 +1,18 @@ +HEADER END +2 BODY N 0 | +4 BODY N 15 | +7 BODY N 0 | +header_body_checks: warning: invalid message/* or multipart/* encoding domain: base64 +11 BODY N 0 | +13 BODY N 13 | +16 BODY N 0 | +18 BODY N 19 | +21 BODY N 0 | +23 BODY N 19 | +26 BODY N 52 | +28 BODY N 67 | +31 BODY N 0 | +33 BODY N 21 | +35 BODY N 12 | +BODY END +header_body_checks: warning: improper message/* or multipart/* encoding domain diff --git a/postfix/src/global/header_body_checks_null.ref b/postfix/src/global/header_body_checks_null.ref new file mode 100644 index 000000000..ffe179c10 --- /dev/null +++ b/postfix/src/global/header_body_checks_null.ref @@ -0,0 +1,42 @@ +0 MAIN 0 |subject: primary subject +1 MAIN 71 |content-type: multipart/(co\m\)ment)mumble mumble; boundary = "ab\cd + ef" mumble +HEADER END +2 BODY N 0 | +3 BODY N 1 |abcdef prolog +4 BODY N 15 | +5 BODY N 16 |--abcd ef +6 MULT 0 |content-type: message/rfc822; mumble +7 BODY N 0 | +8 NEST 0 |subject: nested subject +9 NEST 57 |content-type: multipart/mumble; boundary(comment)="pqrs" +10 NEST 91 |content-transfer-encoding: base64 +header_body_checks: warning: invalid message/* or multipart/* encoding domain: base64 +11 BODY N 0 | +12 BODY N 1 |pqrs prolog +13 BODY N 13 | +14 BODY N 14 |--pqrs +15 MULT 0 |header: pqrs part 01 +16 BODY N 0 | +17 BODY N 1 |body pqrs part 01 +18 BODY N 19 | +19 BODY N 20 |--pqrs +20 MULT 0 |header: pqrs part 02 +21 BODY N 0 | +22 BODY N 1 |body pqrs part 02 +23 BODY N 19 | +24 BODY N 20 |--bogus-boundary +25 BODY N 37 |header: wietse +26 BODY N 52 | +27 BODY N 53 |body asdasads +28 BODY N 67 | +29 BODY N 68 |--abcd ef +30 MULT 0 |header: abcdef part 02 +31 BODY N 0 | +32 BODY N 1 |body abcdef part 02 +33 BODY N 21 | +34 BODY N 0 |--abcd ef-- +35 BODY N 12 | +36 BODY N 13 |epilog +BODY END +header_body_checks: warning: improper message/* or multipart/* encoding domain diff --git a/postfix/src/global/header_body_checks_prepend.ref b/postfix/src/global/header_body_checks_prepend.ref new file mode 100644 index 000000000..deaaefcc5 --- /dev/null +++ b/postfix/src/global/header_body_checks_prepend.ref @@ -0,0 +1,88 @@ +header_body_checks: test-queueID: prepend: header subject: primary subject: header: head +header: head +0 MAIN 0 |subject: primary subject +header_body_checks: test-queueID: prepend: header content-type: multipart/(co\m\)ment)mumble mumble; boundary = "ab\cd ? ef" mumble: header: mime +header: mime +1 MAIN 71 |content-type: multipart/(co\m\)ment)mumble mumble; boundary = "ab\cd + ef" mumble +HEADER END +2 BODY N 0 | +header_body_checks: test-queueID: prepend: body abcdef prolog: body +body +3 BODY N 1 |abcdef prolog +4 BODY N 15 | +header_body_checks: test-queueID: prepend: body --abcd ef: body +body +5 BODY N 16 |--abcd ef +header_body_checks: test-queueID: prepend: header content-type: message/rfc822; mumble: header: mime +header: mime +6 MULT 0 |content-type: message/rfc822; mumble +7 BODY N 0 | +header_body_checks: test-queueID: prepend: header subject: nested subject: header: nest +header: nest +8 NEST 0 |subject: nested subject +header_body_checks: test-queueID: prepend: header content-type: multipart/mumble; boundary(comment)="pqrs": header: mime +header: mime +9 NEST 57 |content-type: multipart/mumble; boundary(comment)="pqrs" +header_body_checks: test-queueID: prepend: header content-transfer-encoding: base64: header: mime +header: mime +10 NEST 91 |content-transfer-encoding: base64 +header_body_checks: warning: invalid message/* or multipart/* encoding domain: base64 +11 BODY N 0 | +header_body_checks: test-queueID: prepend: body pqrs prolog: body +body +12 BODY N 1 |pqrs prolog +13 BODY N 13 | +header_body_checks: test-queueID: prepend: body --pqrs: body +body +14 BODY N 14 |--pqrs +header_body_checks: test-queueID: prepend: header header: pqrs part 01: header: mime +header: mime +15 MULT 0 |header: pqrs part 01 +16 BODY N 0 | +header_body_checks: test-queueID: prepend: body body pqrs part 01: body +body +17 BODY N 1 |body pqrs part 01 +18 BODY N 19 | +header_body_checks: test-queueID: prepend: body --pqrs: body +body +19 BODY N 20 |--pqrs +header_body_checks: test-queueID: prepend: header header: pqrs part 02: header: mime +header: mime +20 MULT 0 |header: pqrs part 02 +21 BODY N 0 | +header_body_checks: test-queueID: prepend: body body pqrs part 02: body +body +22 BODY N 1 |body pqrs part 02 +23 BODY N 19 | +header_body_checks: test-queueID: prepend: body --bogus-boundary: body +body +24 BODY N 20 |--bogus-boundary +header_body_checks: test-queueID: prepend: body header: wietse: body +body +25 BODY N 37 |header: wietse +26 BODY N 52 | +header_body_checks: test-queueID: prepend: body body asdasads: body +body +27 BODY N 53 |body asdasads +28 BODY N 67 | +header_body_checks: test-queueID: prepend: body --abcd ef: body +body +29 BODY N 68 |--abcd ef +header_body_checks: test-queueID: prepend: header header: abcdef part 02: header: mime +header: mime +30 MULT 0 |header: abcdef part 02 +31 BODY N 0 | +header_body_checks: test-queueID: prepend: body body abcdef part 02: body +body +32 BODY N 1 |body abcdef part 02 +33 BODY N 21 | +header_body_checks: test-queueID: prepend: body --abcd ef--: body +body +34 BODY N 0 |--abcd ef-- +35 BODY N 12 | +header_body_checks: test-queueID: prepend: body epilog: body +body +36 BODY N 13 |epilog +BODY END +header_body_checks: warning: improper message/* or multipart/* encoding domain diff --git a/postfix/src/global/header_body_checks_replace.ref b/postfix/src/global/header_body_checks_replace.ref new file mode 100644 index 000000000..31bbeb352 --- /dev/null +++ b/postfix/src/global/header_body_checks_replace.ref @@ -0,0 +1,64 @@ +header_body_checks: test-queueID: replace: header subject: primary subject: header: head +0 MAIN 0 |header: head +header_body_checks: test-queueID: replace: header content-type: multipart/(co\m\)ment)mumble mumble; boundary = "ab\cd ? ef" mumble: header: mime +1 MAIN 71 |header: mime +HEADER END +2 BODY N 0 | +header_body_checks: test-queueID: replace: body abcdef prolog: body +3 BODY N 1 |body +4 BODY N 15 | +header_body_checks: test-queueID: replace: body --abcd ef: body +5 BODY N 16 |body +header_body_checks: test-queueID: replace: header content-type: message/rfc822; mumble: header: mime +6 MULT 0 |header: mime +7 BODY N 0 | +header_body_checks: test-queueID: replace: header subject: nested subject: header: nest +8 NEST 0 |header: nest +header_body_checks: test-queueID: replace: header content-type: multipart/mumble; boundary(comment)="pqrs": header: mime +9 NEST 57 |header: mime +header_body_checks: test-queueID: replace: header content-transfer-encoding: base64: header: mime +10 NEST 91 |header: mime +header_body_checks: warning: invalid message/* or multipart/* encoding domain: base64 +11 BODY N 0 | +header_body_checks: test-queueID: replace: body pqrs prolog: body +12 BODY N 1 |body +13 BODY N 13 | +header_body_checks: test-queueID: replace: body --pqrs: body +14 BODY N 14 |body +header_body_checks: test-queueID: replace: header header: pqrs part 01: header: mime +15 MULT 0 |header: mime +16 BODY N 0 | +header_body_checks: test-queueID: replace: body body pqrs part 01: body +17 BODY N 1 |body +18 BODY N 19 | +header_body_checks: test-queueID: replace: body --pqrs: body +19 BODY N 20 |body +header_body_checks: test-queueID: replace: header header: pqrs part 02: header: mime +20 MULT 0 |header: mime +21 BODY N 0 | +header_body_checks: test-queueID: replace: body body pqrs part 02: body +22 BODY N 1 |body +23 BODY N 19 | +header_body_checks: test-queueID: replace: body --bogus-boundary: body +24 BODY N 20 |body +header_body_checks: test-queueID: replace: body header: wietse: body +25 BODY N 37 |body +26 BODY N 52 | +header_body_checks: test-queueID: replace: body body asdasads: body +27 BODY N 53 |body +28 BODY N 67 | +header_body_checks: test-queueID: replace: body --abcd ef: body +29 BODY N 68 |body +header_body_checks: test-queueID: replace: header header: abcdef part 02: header: mime +30 MULT 0 |header: mime +31 BODY N 0 | +header_body_checks: test-queueID: replace: body body abcdef part 02: body +32 BODY N 1 |body +33 BODY N 21 | +header_body_checks: test-queueID: replace: body --abcd ef--: body +34 BODY N 0 |body +35 BODY N 12 | +header_body_checks: test-queueID: replace: body epilog: body +36 BODY N 13 |body +BODY END +header_body_checks: warning: improper message/* or multipart/* encoding domain diff --git a/postfix/src/global/header_body_checks_warn.ref b/postfix/src/global/header_body_checks_warn.ref new file mode 100644 index 000000000..50afd352b --- /dev/null +++ b/postfix/src/global/header_body_checks_warn.ref @@ -0,0 +1,65 @@ +header_body_checks: test-queueID: warning: header subject: primary subject +0 MAIN 0 |subject: primary subject +header_body_checks: test-queueID: warning: header content-type: multipart/(co\m\)ment)mumble mumble; boundary = "ab\cd ? ef" mumble +1 MAIN 71 |content-type: multipart/(co\m\)ment)mumble mumble; boundary = "ab\cd + ef" mumble +HEADER END +2 BODY N 0 | +header_body_checks: test-queueID: warning: body abcdef prolog +3 BODY N 1 |abcdef prolog +4 BODY N 15 | +header_body_checks: test-queueID: warning: body --abcd ef +5 BODY N 16 |--abcd ef +header_body_checks: test-queueID: warning: header content-type: message/rfc822; mumble +6 MULT 0 |content-type: message/rfc822; mumble +7 BODY N 0 | +header_body_checks: test-queueID: warning: header subject: nested subject +8 NEST 0 |subject: nested subject +header_body_checks: test-queueID: warning: header content-type: multipart/mumble; boundary(comment)="pqrs" +9 NEST 57 |content-type: multipart/mumble; boundary(comment)="pqrs" +header_body_checks: test-queueID: warning: header content-transfer-encoding: base64 +10 NEST 91 |content-transfer-encoding: base64 +header_body_checks: warning: invalid message/* or multipart/* encoding domain: base64 +11 BODY N 0 | +header_body_checks: test-queueID: warning: body pqrs prolog +12 BODY N 1 |pqrs prolog +13 BODY N 13 | +header_body_checks: test-queueID: warning: body --pqrs +14 BODY N 14 |--pqrs +header_body_checks: test-queueID: warning: header header: pqrs part 01 +15 MULT 0 |header: pqrs part 01 +16 BODY N 0 | +header_body_checks: test-queueID: warning: body body pqrs part 01 +17 BODY N 1 |body pqrs part 01 +18 BODY N 19 | +header_body_checks: test-queueID: warning: body --pqrs +19 BODY N 20 |--pqrs +header_body_checks: test-queueID: warning: header header: pqrs part 02 +20 MULT 0 |header: pqrs part 02 +21 BODY N 0 | +header_body_checks: test-queueID: warning: body body pqrs part 02 +22 BODY N 1 |body pqrs part 02 +23 BODY N 19 | +header_body_checks: test-queueID: warning: body --bogus-boundary +24 BODY N 20 |--bogus-boundary +header_body_checks: test-queueID: warning: body header: wietse +25 BODY N 37 |header: wietse +26 BODY N 52 | +header_body_checks: test-queueID: warning: body body asdasads +27 BODY N 53 |body asdasads +28 BODY N 67 | +header_body_checks: test-queueID: warning: body --abcd ef +29 BODY N 68 |--abcd ef +header_body_checks: test-queueID: warning: header header: abcdef part 02 +30 MULT 0 |header: abcdef part 02 +31 BODY N 0 | +header_body_checks: test-queueID: warning: body body abcdef part 02 +32 BODY N 1 |body abcdef part 02 +33 BODY N 21 | +header_body_checks: test-queueID: warning: body --abcd ef-- +34 BODY N 0 |--abcd ef-- +35 BODY N 12 | +header_body_checks: test-queueID: warning: body epilog +36 BODY N 13 |epilog +BODY END +header_body_checks: warning: improper message/* or multipart/* encoding domain diff --git a/postfix/src/global/mail_params.h b/postfix/src/global/mail_params.h index bbae895c7..7b9110107 100644 --- a/postfix/src/global/mail_params.h +++ b/postfix/src/global/mail_params.h @@ -2802,6 +2802,34 @@ extern bool var_smtpd_client_port_log; #define DEF_QMQPD_CLIENT_PORT_LOG 0 extern bool var_qmqpd_client_port_log; + /* + * Header/body checks in delivery agents. + */ +#define VAR_SMTP_HEAD_CHKS "smtp_header_checks" +#define DEF_SMTP_HEAD_CHKS "" +extern char *var_smtp_head_chks; + +#define VAR_SMTP_MIME_CHKS "smtp_mime_header_checks" +#define DEF_SMTP_MIME_CHKS "" +extern char *var_smtp_mime_chks; + +#define VAR_SMTP_NEST_CHKS "smtp_nested_header_checks" +#define DEF_SMTP_NEST_CHKS "" +extern char *var_smtp_nest_chks; + +#define VAR_SMTP_BODY_CHKS "smtp_body_checks" +#define DEF_SMTP_BODY_CHKS "" +extern char *var_smtp_body_chks; + +#define VAR_LMTP_HEAD_CHKS "lmtp_header_checks" +#define DEF_LMTP_HEAD_CHKS "" +#define VAR_LMTP_MIME_CHKS "lmtp_mime_header_checks" +#define DEF_LMTP_MIME_CHKS "" +#define VAR_LMTP_NEST_CHKS "lmtp_nested_header_checks" +#define DEF_LMTP_NEST_CHKS "" +#define VAR_LMTP_BODY_CHKS "lmtp_body_checks" +#define DEF_LMTP_BODY_CHKS "" + /* LICENSE /* .ad /* .fi diff --git a/postfix/src/global/mail_version.h b/postfix/src/global/mail_version.h index ec6dc4381..b4d627ed7 100644 --- a/postfix/src/global/mail_version.h +++ b/postfix/src/global/mail_version.h @@ -20,7 +20,7 @@ * Patches change both the patchlevel and the release date. Snapshots have no * patchlevel; they change the release date only. */ -#define MAIL_RELEASE_DATE "20071006" +#define MAIL_RELEASE_DATE "20071111" #define MAIL_VERSION_NUMBER "2.5" #ifdef SNAPSHOT diff --git a/postfix/src/global/mime_state.c b/postfix/src/global/mime_state.c index 4b4bf69f9..bea9e0ad4 100644 --- a/postfix/src/global/mime_state.c +++ b/postfix/src/global/mime_state.c @@ -152,8 +152,12 @@ /* At the start of a nested (e.g., message/rfc822) message. /* .RE /* .sp +/* For convenience, the macros MIME_HDR_FIRST and MIME_HDR_LAST +/* specify the range of MIME_HDR_MUMBLE macros. +/* .sp /* To find out if something is a MIME header at the beginning -/* of an RFC 822 message, look at the header_info argument. +/* of an RFC 822 message or an attached message, look at the +/* header_info argument. /* .IP header_info /* Null pointer or information about the message header, see /* header_opts(3). diff --git a/postfix/src/global/mime_state.h b/postfix/src/global/mime_state.h index 78454a23e..35f5c827f 100644 --- a/postfix/src/global/mime_state.h +++ b/postfix/src/global/mime_state.h @@ -73,12 +73,14 @@ extern MIME_STATE_DETAIL *mime_state_detail(int); extern const char *mime_state_error(int); /* - * Header classes. Look at the header_opts argument to find out if something - * is a MIME header in a primary or nested section. + * With header classes, look at the header_opts argument to recognize MIME + * headers in primary or nested sections. */ +#define MIME_HDR_FIRST (1) /* first class */ #define MIME_HDR_PRIMARY (1) /* initial headers */ #define MIME_HDR_MULTIPART (2) /* headers after multipart boundary */ #define MIME_HDR_NESTED (3) /* attached message initial headers */ +#define MIME_HDR_LAST (3) /* last class */ /* LICENSE /* .ad diff --git a/postfix/src/local/command.c b/postfix/src/local/command.c index 8873d15bc..7952583c2 100644 --- a/postfix/src/local/command.c +++ b/postfix/src/local/command.c @@ -165,6 +165,9 @@ int deliver_command(LOCAL_STATE state, USER_ATTR usr_attr, const char *comma argv_add(env, "DOMAIN", state.msg_attr.domain, ARGV_END); if (state.msg_attr.extension) argv_add(env, "EXTENSION", state.msg_attr.extension, ARGV_END); + if (state.msg_attr.rcpt.orig_addr && state.msg_attr.rcpt.orig_addr[0]) + argv_add(env, "ORIGINAL_RECIPIENT", state.msg_attr.rcpt.orig_addr, + ARGV_END); #define EXPORT_REQUEST(name, value) \ if ((value)[0]) argv_add(env, (name), (value), ARGV_END); diff --git a/postfix/src/local/local.c b/postfix/src/local/local.c index b2992d7c4..6788fc80f 100644 --- a/postfix/src/local/local.c +++ b/postfix/src/local/local.c @@ -210,6 +210,9 @@ /* .IP \fBLOCAL\fR /* The entire recipient address localpart (text to the left of the /* rightmost @ character). +/* .IP \fBORIGINAL_RECIPIENT\fR +/* The entire recipient address, before any address rewriting +/* or aliasing (Postfix 2.5 and later). /* .IP \fBRECIPIENT\fR /* The entire recipient address. /* .IP \fBSENDER\fR diff --git a/postfix/src/milter/milter8.c b/postfix/src/milter/milter8.c index d69024059..2aad9612c 100644 --- a/postfix/src/milter/milter8.c +++ b/postfix/src/milter/milter8.c @@ -63,6 +63,7 @@ #include #include #include +#include /* INT_MAX */ #ifndef SHUT_RDWR #define SHUT_RDWR 2 @@ -281,7 +282,7 @@ typedef struct { /* * We don't accept insane amounts of data. */ -#define XXX_MAX_DATA (MILTER_CHUNK_SIZE * 2) +#define XXX_MAX_DATA (INT_MAX / 2) #define XXX_TIMEOUT 10 #ifndef USE_LIBMILTER_INCLUDES @@ -533,8 +534,8 @@ static int milter8_read_resp(MILTER8 *milter, int event, unsigned char *command, milter->m.name, (long) pkt_len); return (milter8_comm_error(milter)); } else if (pkt_len > XXX_MAX_DATA) { - msg_warn("milter %s: unreasonable packet length: %ld", - milter->m.name, (long) pkt_len); + msg_warn("milter %s: unreasonable packet length: %ld > %ld", + milter->m.name, (long) pkt_len, (long) XXX_MAX_DATA); return (milter8_comm_error(milter)); } diff --git a/postfix/src/pipe/pipe.c b/postfix/src/pipe/pipe.c index b768d6fc7..e5c37b13f 100644 --- a/postfix/src/pipe/pipe.c +++ b/postfix/src/pipe/pipe.c @@ -108,14 +108,16 @@ /* .sp /* This feature is available as of Postfix 2.5. /* .IP \fBh\fR -/* Fold the command-line \fB$recipient\fR address domain part +/* Fold the command-line \fB$original_recipient\fR and +/* \fB$recipient\fR address domain part /* (text to the right of the right-most \fB@\fR character) to /* lower case; fold the entire command-line \fB$domain\fR and /* \fB$nexthop\fR host or domain information to lower case. /* This is recommended for delivery via \fBUUCP\fR. /* .IP \fBq\fR /* Quote white space and other special characters in the command-line -/* \fB$sender\fR and \fB$recipient\fR address localparts (text to the +/* \fB$sender\fR, \fB$original_recipient\fR and \fB$recipient\fR +/* address localparts (text to the /* left of the right-most \fB@\fR character), according to an 8-bit /* transparent version of RFC 822. /* This is recommended for delivery via \fBUUCP\fR or \fBBSMTP\fR. @@ -127,7 +129,8 @@ /* address information from the \fB$user\fR, \fB$extension\fR or /* \fB$mailbox\fR command-line macros. /* .IP \fBu\fR -/* Fold the command-line \fB$recipient\fR address localpart (text to +/* Fold the command-line \fB$original_recipient\fR and +/* \fB$recipient\fR address localpart (text to /* the left of the right-most \fB@\fR character) to lower case. /* This is recommended for delivery via \fBUUCP\fR. /* .IP \fB.\fR @@ -244,6 +247,16 @@ /* This macro expands to the next-hop hostname. /* .sp /* This information is modified by the \fBh\fR flag for case folding. +/* .IP \fB${\fBoriginal_recipient\fR}\fR +/* This macro expands to the complete recipient address before any +/* address rewriting or aliasing. +/* .sp +/* A command-line argument that contains +/* \fB${\fBoriginal_recipient\fR}\fR expands to as many +/* command-line arguments as there are recipients. +/* .sp +/* This information is modified by the \fBhqu\fR flags for quoting +/* and case folding. /* .IP \fB${\fBrecipient\fR}\fR /* This macro expands to the complete recipient address. /* .sp @@ -470,6 +483,7 @@ #define PIPE_DICT_TABLE "pipe_command" /* table name */ #define PIPE_DICT_NEXTHOP "nexthop" /* key */ #define PIPE_DICT_RCPT "recipient" /* key */ +#define PIPE_DICT_ORIG_RCPT "original_recipient" /* key */ #define PIPE_DICT_SENDER "sender"/* key */ #define PIPE_DICT_USER "user" /* key */ #define PIPE_DICT_EXTENSION "extension" /* key */ @@ -494,6 +508,7 @@ #define PIPE_FLAG_EXTENSION (1<<2) #define PIPE_FLAG_MAILBOX (1<<3) #define PIPE_FLAG_DOMAIN (1<<4) +#define PIPE_FLAG_ORIG_RCPT (1<<5) /* * Additional flags. These are colocated with mail_copy() flags. Allow some @@ -567,6 +582,7 @@ static int parse_callback(int type, VSTRING *buf, char *context) static struct cmd_flags cmd_flags[] = { PIPE_DICT_NEXTHOP, 0, PIPE_DICT_RCPT, PIPE_FLAG_RCPT, + PIPE_DICT_ORIG_RCPT, PIPE_FLAG_ORIG_RCPT, PIPE_DICT_SENDER, 0, PIPE_DICT_USER, PIPE_FLAG_USER, PIPE_DICT_EXTENSION, PIPE_FLAG_EXTENSION, @@ -672,6 +688,14 @@ static ARGV *expand_argv(const char *service, char **argv, dict_update(PIPE_DICT_TABLE, PIPE_DICT_RCPT, STR(buf)); } + /* + * This argument contains $original_recipient. + */ + if (state.expand_flag & PIPE_FLAG_ORIG_RCPT) { + morph_recipient(buf, rcpt_list->info[i].orig_addr, flags); + dict_update(PIPE_DICT_TABLE, PIPE_DICT_ORIG_RCPT, STR(buf)); + } + /* * This argument contains $user. Extract the plain user name. * Either anything to the left of the extension delimiter or, diff --git a/postfix/src/smtp/Makefile.in b/postfix/src/smtp/Makefile.in index 8e578347d..48c9f8b5b 100644 --- a/postfix/src/smtp/Makefile.in +++ b/postfix/src/smtp/Makefile.in @@ -95,6 +95,8 @@ smtp.o: ../../include/dsn.h smtp.o: ../../include/dsn_buf.h smtp.o: ../../include/ext_prop.h smtp.o: ../../include/flush_clnt.h +smtp.o: ../../include/header_body_checks.h +smtp.o: ../../include/header_opts.h smtp.o: ../../include/htable.h smtp.o: ../../include/mail_conf.h smtp.o: ../../include/mail_params.h @@ -103,6 +105,7 @@ smtp.o: ../../include/mail_version.h smtp.o: ../../include/maps.h smtp.o: ../../include/match_list.h smtp.o: ../../include/match_ops.h +smtp.o: ../../include/mime_state.h smtp.o: ../../include/msg.h smtp.o: ../../include/msg_stats.h smtp.o: ../../include/mymalloc.h @@ -131,6 +134,8 @@ smtp_addr.o: ../../include/dict.h smtp_addr.o: ../../include/dns.h smtp_addr.o: ../../include/dsn.h smtp_addr.o: ../../include/dsn_buf.h +smtp_addr.o: ../../include/header_body_checks.h +smtp_addr.o: ../../include/header_opts.h smtp_addr.o: ../../include/htable.h smtp_addr.o: ../../include/inet_addr_list.h smtp_addr.o: ../../include/inet_proto.h @@ -138,6 +143,7 @@ smtp_addr.o: ../../include/mail_params.h smtp_addr.o: ../../include/maps.h smtp_addr.o: ../../include/match_list.h smtp_addr.o: ../../include/match_ops.h +smtp_addr.o: ../../include/mime_state.h smtp_addr.o: ../../include/msg.h smtp_addr.o: ../../include/msg_stats.h smtp_addr.o: ../../include/myaddrinfo.h @@ -168,6 +174,8 @@ smtp_chat.o: ../../include/dict.h smtp_chat.o: ../../include/dsn.h smtp_chat.o: ../../include/dsn_buf.h smtp_chat.o: ../../include/dsn_util.h +smtp_chat.o: ../../include/header_body_checks.h +smtp_chat.o: ../../include/header_opts.h smtp_chat.o: ../../include/htable.h smtp_chat.o: ../../include/int_filt.h smtp_chat.o: ../../include/line_wrap.h @@ -177,6 +185,7 @@ smtp_chat.o: ../../include/mail_params.h smtp_chat.o: ../../include/maps.h smtp_chat.o: ../../include/match_list.h smtp_chat.o: ../../include/match_ops.h +smtp_chat.o: ../../include/mime_state.h smtp_chat.o: ../../include/msg.h smtp_chat.o: ../../include/msg_stats.h smtp_chat.o: ../../include/mymalloc.h @@ -205,6 +214,8 @@ smtp_connect.o: ../../include/dict.h smtp_connect.o: ../../include/dns.h smtp_connect.o: ../../include/dsn.h smtp_connect.o: ../../include/dsn_buf.h +smtp_connect.o: ../../include/header_body_checks.h +smtp_connect.o: ../../include/header_opts.h smtp_connect.o: ../../include/host_port.h smtp_connect.o: ../../include/htable.h smtp_connect.o: ../../include/inet_addr_list.h @@ -215,6 +226,7 @@ smtp_connect.o: ../../include/mail_proto.h smtp_connect.o: ../../include/maps.h smtp_connect.o: ../../include/match_list.h smtp_connect.o: ../../include/match_ops.h +smtp_connect.o: ../../include/mime_state.h smtp_connect.o: ../../include/msg.h smtp_connect.o: ../../include/msg_stats.h smtp_connect.o: ../../include/myaddrinfo.h @@ -247,11 +259,14 @@ smtp_map11.o: ../../include/deliver_request.h smtp_map11.o: ../../include/dict.h smtp_map11.o: ../../include/dsn.h smtp_map11.o: ../../include/dsn_buf.h +smtp_map11.o: ../../include/header_body_checks.h +smtp_map11.o: ../../include/header_opts.h smtp_map11.o: ../../include/htable.h smtp_map11.o: ../../include/mail_addr_map.h smtp_map11.o: ../../include/maps.h smtp_map11.o: ../../include/match_list.h smtp_map11.o: ../../include/match_ops.h +smtp_map11.o: ../../include/mime_state.h smtp_map11.o: ../../include/msg.h smtp_map11.o: ../../include/msg_stats.h smtp_map11.o: ../../include/name_code.h @@ -283,6 +298,7 @@ smtp_proto.o: ../../include/dsn_buf.h smtp_proto.o: ../../include/dsn_mask.h smtp_proto.o: ../../include/ehlo_mask.h smtp_proto.o: ../../include/ext_prop.h +smtp_proto.o: ../../include/header_body_checks.h smtp_proto.o: ../../include/header_opts.h smtp_proto.o: ../../include/htable.h smtp_proto.o: ../../include/iostuff.h @@ -334,10 +350,13 @@ smtp_rcpt.o: ../../include/dict.h smtp_rcpt.o: ../../include/dsn.h smtp_rcpt.o: ../../include/dsn_buf.h smtp_rcpt.o: ../../include/dsn_mask.h +smtp_rcpt.o: ../../include/header_body_checks.h +smtp_rcpt.o: ../../include/header_opts.h smtp_rcpt.o: ../../include/htable.h smtp_rcpt.o: ../../include/maps.h smtp_rcpt.o: ../../include/match_list.h smtp_rcpt.o: ../../include/match_ops.h +smtp_rcpt.o: ../../include/mime_state.h smtp_rcpt.o: ../../include/msg.h smtp_rcpt.o: ../../include/msg_stats.h smtp_rcpt.o: ../../include/mymalloc.h @@ -363,11 +382,14 @@ smtp_reuse.o: ../../include/deliver_request.h smtp_reuse.o: ../../include/dict.h smtp_reuse.o: ../../include/dsn.h smtp_reuse.o: ../../include/dsn_buf.h +smtp_reuse.o: ../../include/header_body_checks.h +smtp_reuse.o: ../../include/header_opts.h smtp_reuse.o: ../../include/htable.h smtp_reuse.o: ../../include/mail_params.h smtp_reuse.o: ../../include/maps.h smtp_reuse.o: ../../include/match_list.h smtp_reuse.o: ../../include/match_ops.h +smtp_reuse.o: ../../include/mime_state.h smtp_reuse.o: ../../include/msg.h smtp_reuse.o: ../../include/msg_stats.h smtp_reuse.o: ../../include/mymalloc.h @@ -393,12 +415,15 @@ smtp_sasl_glue.o: ../../include/deliver_request.h smtp_sasl_glue.o: ../../include/dict.h smtp_sasl_glue.o: ../../include/dsn.h smtp_sasl_glue.o: ../../include/dsn_buf.h +smtp_sasl_glue.o: ../../include/header_body_checks.h +smtp_sasl_glue.o: ../../include/header_opts.h smtp_sasl_glue.o: ../../include/htable.h smtp_sasl_glue.o: ../../include/mail_addr_find.h smtp_sasl_glue.o: ../../include/mail_params.h smtp_sasl_glue.o: ../../include/maps.h smtp_sasl_glue.o: ../../include/match_list.h smtp_sasl_glue.o: ../../include/match_ops.h +smtp_sasl_glue.o: ../../include/mime_state.h smtp_sasl_glue.o: ../../include/msg.h smtp_sasl_glue.o: ../../include/msg_stats.h smtp_sasl_glue.o: ../../include/mymalloc.h @@ -426,11 +451,14 @@ smtp_sasl_proto.o: ../../include/deliver_request.h smtp_sasl_proto.o: ../../include/dict.h smtp_sasl_proto.o: ../../include/dsn.h smtp_sasl_proto.o: ../../include/dsn_buf.h +smtp_sasl_proto.o: ../../include/header_body_checks.h +smtp_sasl_proto.o: ../../include/header_opts.h smtp_sasl_proto.o: ../../include/htable.h smtp_sasl_proto.o: ../../include/mail_params.h smtp_sasl_proto.o: ../../include/maps.h smtp_sasl_proto.o: ../../include/match_list.h smtp_sasl_proto.o: ../../include/match_ops.h +smtp_sasl_proto.o: ../../include/mime_state.h smtp_sasl_proto.o: ../../include/msg.h smtp_sasl_proto.o: ../../include/msg_stats.h smtp_sasl_proto.o: ../../include/mymalloc.h @@ -457,6 +485,7 @@ smtp_session.o: ../../include/deliver_request.h smtp_session.o: ../../include/dict.h smtp_session.o: ../../include/dsn.h smtp_session.o: ../../include/dsn_buf.h +smtp_session.o: ../../include/header_body_checks.h smtp_session.o: ../../include/header_opts.h smtp_session.o: ../../include/htable.h smtp_session.o: ../../include/mail_params.h @@ -490,11 +519,14 @@ smtp_state.o: ../../include/deliver_request.h smtp_state.o: ../../include/dict.h smtp_state.o: ../../include/dsn.h smtp_state.o: ../../include/dsn_buf.h +smtp_state.o: ../../include/header_body_checks.h +smtp_state.o: ../../include/header_opts.h smtp_state.o: ../../include/htable.h smtp_state.o: ../../include/mail_params.h smtp_state.o: ../../include/maps.h smtp_state.o: ../../include/match_list.h smtp_state.o: ../../include/match_ops.h +smtp_state.o: ../../include/mime_state.h smtp_state.o: ../../include/msg.h smtp_state.o: ../../include/msg_stats.h smtp_state.o: ../../include/mymalloc.h @@ -522,11 +554,14 @@ smtp_trouble.o: ../../include/deliver_request.h smtp_trouble.o: ../../include/dict.h smtp_trouble.o: ../../include/dsn.h smtp_trouble.o: ../../include/dsn_buf.h +smtp_trouble.o: ../../include/header_body_checks.h +smtp_trouble.o: ../../include/header_opts.h smtp_trouble.o: ../../include/htable.h smtp_trouble.o: ../../include/mail_error.h smtp_trouble.o: ../../include/maps.h smtp_trouble.o: ../../include/match_list.h smtp_trouble.o: ../../include/match_ops.h +smtp_trouble.o: ../../include/mime_state.h smtp_trouble.o: ../../include/msg.h smtp_trouble.o: ../../include/msg_stats.h smtp_trouble.o: ../../include/name_code.h @@ -552,10 +587,13 @@ smtp_unalias.o: ../../include/dict.h smtp_unalias.o: ../../include/dns.h smtp_unalias.o: ../../include/dsn.h smtp_unalias.o: ../../include/dsn_buf.h +smtp_unalias.o: ../../include/header_body_checks.h +smtp_unalias.o: ../../include/header_opts.h smtp_unalias.o: ../../include/htable.h smtp_unalias.o: ../../include/maps.h smtp_unalias.o: ../../include/match_list.h smtp_unalias.o: ../../include/match_ops.h +smtp_unalias.o: ../../include/mime_state.h smtp_unalias.o: ../../include/msg.h smtp_unalias.o: ../../include/msg_stats.h smtp_unalias.o: ../../include/myaddrinfo.h diff --git a/postfix/src/smtp/lmtp_params.c b/postfix/src/smtp/lmtp_params.c index 3a40791c6..0ebb292b3 100644 --- a/postfix/src/smtp/lmtp_params.c +++ b/postfix/src/smtp/lmtp_params.c @@ -47,6 +47,10 @@ VAR_LMTP_PIX_BUG_WORDS, DEF_LMTP_PIX_BUG_WORDS, &var_smtp_pix_bug_words, 0, 0, VAR_LMTP_PIX_BUG_MAPS, DEF_LMTP_PIX_BUG_MAPS, &var_smtp_pix_bug_maps, 0, 0, VAR_CYRUS_CONF_PATH, DEF_CYRUS_CONF_PATH, &var_cyrus_conf_path, 0, 0, + VAR_LMTP_HEAD_CHKS, DEF_LMTP_HEAD_CHKS, &var_smtp_head_chks, 0, 0, + VAR_LMTP_MIME_CHKS, DEF_LMTP_MIME_CHKS, &var_smtp_mime_chks, 0, 0, + VAR_LMTP_NEST_CHKS, DEF_LMTP_NEST_CHKS, &var_smtp_nest_chks, 0, 0, + VAR_LMTP_BODY_CHKS, DEF_LMTP_BODY_CHKS, &var_smtp_body_chks, 0, 0, 0, }; static CONFIG_TIME_TABLE lmtp_time_table[] = { diff --git a/postfix/src/smtp/smtp.c b/postfix/src/smtp/smtp.c index 2f5abaf6a..356967c7c 100644 --- a/postfix/src/smtp/smtp.c +++ b/postfix/src/smtp/smtp.c @@ -211,6 +211,18 @@ /* When authenticating to a remote SMTP or LMTP server with the /* default setting "no", send no SASL authoriZation ID (authzid); send /* only the SASL authentiCation ID (authcid) plus the authcid's password. +/* .PP +/* Available in Postfix version 2.5 and later: +/* .IP "\fBsmtp_header_checks (empty)\fR" +/* Restricted \fBheader_checks\fR(5) tables for the Postfix SMTP client. +/* .IP "\fBsmtp_mime_header_checks (empty)\fR" +/* Restricted \fBmime_header_checks\fR(5) tables for the Postfix SMTP +/* client. +/* .IP "\fBsmtp_nested_header_checks (empty)\fR" +/* Restricted \fBnested_header_checks\fR(5) tables for the Postfix SMTP +/* client. +/* .IP "\fBsmtp_body_checks (empty)\fR" +/* Restricted \fBbody_checks\fR(5) tables for the Postfix SMTP client. /* MIME PROCESSING CONTROLS /* .ad /* .fi @@ -538,6 +550,9 @@ /* Optional list of relay hosts for SMTP destinations that can't be /* found or that are unreachable. /* SEE ALSO +/* generic(5), output address rewriting +/* header_checks(5), message header content inspection +/* body_checks(5), body parts content inspection /* qmgr(8), queue manager /* bounce(8), delivery status reports /* scache(8), connection cache server @@ -719,6 +734,10 @@ bool var_smtp_cname_overr; char *var_smtp_pix_bug_words; char *var_smtp_pix_bug_maps; char *var_cyrus_conf_path; +char *var_smtp_head_chks; +char *var_smtp_mime_chks; +char *var_smtp_nest_chks; +char *var_smtp_body_chks; /* * Global variables. @@ -730,6 +749,8 @@ MAPS *smtp_ehlo_dis_maps; MAPS *smtp_generic_maps; int smtp_ext_prop_mask; MAPS *smtp_pix_bug_maps; +HBC_CHECKS *smtp_header_checks; /* limited header checks */ +HBC_CHECKS *smtp_body_checks; /* limited body checks */ #ifdef USE_TLS @@ -944,6 +965,18 @@ static void pre_init(char *unused_name, char **unused_argv) smtp_generic_maps = maps_create(VAR_SMTP_GENERIC_MAPS, var_smtp_generic_maps, DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX); + + /* + * Header/body checks. + */ + smtp_header_checks = hbc_header_checks_create( + VAR_SMTP_HEAD_CHKS, var_smtp_head_chks, + VAR_SMTP_MIME_CHKS, var_smtp_mime_chks, + VAR_SMTP_NEST_CHKS, var_smtp_nest_chks, + smtp_hbc_callbacks); + smtp_body_checks = hbc_body_checks_create( + VAR_SMTP_BODY_CHKS, var_smtp_body_chks, + smtp_hbc_callbacks); } /* pre_accept - see if tables have changed */ diff --git a/postfix/src/smtp/smtp.h b/postfix/src/smtp/smtp.h index 72005bfc0..d1e027cf1 100644 --- a/postfix/src/smtp/smtp.h +++ b/postfix/src/smtp/smtp.h @@ -30,6 +30,7 @@ #include #include #include +#include /* * Postfix TLS library. @@ -175,6 +176,9 @@ extern SSL_CTX *smtp_tls_ctx; /* client-side TLS engine */ #endif +extern HBC_CHECKS *smtp_header_checks; /* limited header checks */ +extern HBC_CHECKS *smtp_body_checks; /* limited body checks */ + /* * smtp_session.c */ @@ -259,6 +263,8 @@ extern int smtp_xfer(SMTP_STATE *); extern int smtp_rset(SMTP_STATE *); extern int smtp_quit(SMTP_STATE *); +extern HBC_CALL_BACKS smtp_hbc_callbacks[]; + /* * A connection is re-usable if session->expire_time is > 0 and the * expiration time has not been reached. This is subtle because the timer diff --git a/postfix/src/smtp/smtp_params.c b/postfix/src/smtp/smtp_params.c index 463c20813..947f5299c 100644 --- a/postfix/src/smtp/smtp_params.c +++ b/postfix/src/smtp/smtp_params.c @@ -48,6 +48,10 @@ VAR_SMTP_PIX_BUG_WORDS, DEF_SMTP_PIX_BUG_WORDS, &var_smtp_pix_bug_words, 0, 0, VAR_SMTP_PIX_BUG_MAPS, DEF_SMTP_PIX_BUG_MAPS, &var_smtp_pix_bug_maps, 0, 0, VAR_CYRUS_CONF_PATH, DEF_CYRUS_CONF_PATH, &var_cyrus_conf_path, 0, 0, + VAR_SMTP_HEAD_CHKS, DEF_SMTP_HEAD_CHKS, &var_smtp_head_chks, 0, 0, + VAR_SMTP_MIME_CHKS, DEF_SMTP_MIME_CHKS, &var_smtp_mime_chks, 0, 0, + VAR_SMTP_NEST_CHKS, DEF_SMTP_NEST_CHKS, &var_smtp_nest_chks, 0, 0, + VAR_SMTP_BODY_CHKS, DEF_SMTP_BODY_CHKS, &var_smtp_body_chks, 0, 0, 0, }; static CONFIG_TIME_TABLE smtp_time_table[] = { diff --git a/postfix/src/smtp/smtp_proto.c b/postfix/src/smtp/smtp_proto.c index a4acfa0cb..ddee19f7c 100644 --- a/postfix/src/smtp/smtp_proto.c +++ b/postfix/src/smtp/smtp_proto.c @@ -238,6 +238,18 @@ char *xfer_request[SMTP_STATE_LAST] = { static int smtp_start_tls(SMTP_STATE *); + /* + * Call-back information for header/body checks. We don't provide call-backs + * for actions that change the message delivery time or destination. + */ +static void smtp_hbc_logger(void *, const char *, const char *, const char *, const char *); +static void smtp_text_out(void *, int, const char *, ssize_t, off_t); + +HBC_CALL_BACKS smtp_hbc_callbacks[1] = { + smtp_hbc_logger, + smtp_text_out, +}; + /* smtp_helo - perform initial handshake with SMTP server */ int smtp_helo(SMTP_STATE *state) @@ -810,6 +822,23 @@ static int smtp_start_tls(SMTP_STATE *state) #endif +/* smtp_hbc_logger - logging call-back for header/body checks */ + +static void smtp_hbc_logger(void *context, const char *action, + const char *where, const char *content, + const char *text) +{ + const SMTP_STATE *state = (SMTP_STATE *) context; + + if (*text) { + msg_info("%s: %s: %s %.60s: %s", + state->request->queue_id, action, where, content, text); + } else { + msg_info("%s: %s: %s %.60s", + state->request->queue_id, action, where, content); + } +} + /* smtp_text_out - output one header/body record */ static void smtp_text_out(void *context, int rec_type, @@ -906,6 +935,21 @@ static void smtp_header_rewrite(void *context, int header_class, char *start; char *next_line; char *end_line; + char *result; + + /* + * Apply optional header filtering. + */ + if (smtp_header_checks) { + result = hbc_header_checks(context, smtp_header_checks, header_class, + header_info, buf, offset); + if (result == 0) + return; + if (result != STR(buf)) { + vstring_strcpy(buf, result); + myfree(result); + } + } /* * Rewrite primary header addresses that match the smtp_generic_maps. The @@ -913,7 +957,7 @@ static void smtp_header_rewrite(void *context, int header_class, * and that all addresses are in proper form, so we don't have to repeat * that. */ - if (header_info && header_class == MIME_HDR_PRIMARY + if (smtp_generic_maps && header_info && header_class == MIME_HDR_PRIMARY && (header_info->flags & (HDR_OPT_SENDER | HDR_OPT_RECIP)) != 0) { TOK822 *tree; TOK822 **addr_list; @@ -982,6 +1026,29 @@ static void smtp_header_rewrite(void *context, int header_class, } } +/* smtp_body_rewrite - rewrite message body before output */ + +static void smtp_body_rewrite(void *context, int type, + const char *buf, ssize_t len, + off_t offset) +{ + SMTP_STATE *state = (SMTP_STATE *) context; + char *result; + + /* + * Apply optional body filtering. + */ + if (smtp_body_checks) { + result = hbc_body_checks(context, smtp_body_checks, buf, len, offset); + if (result == buf) { + smtp_text_out(state, type, buf, len, offset); + } else if (result != 0) { + smtp_text_out(state, type, result, strlen(result), offset); + myfree(result); + } + } +} + /* smtp_mime_fail - MIME problem */ static void smtp_mime_fail(SMTP_STATE *state, int mime_errs) @@ -1706,15 +1773,19 @@ static int smtp_loop(SMTP_STATE *state, NOCLOBBER int send_state, * XXX Don't downgrade just because generic_maps is turned * on. */ - if (downgrading || smtp_generic_maps) + if (downgrading || smtp_generic_maps || smtp_header_checks + || smtp_body_checks) session->mime_state = mime_state_alloc(downgrading ? MIME_OPT_DOWNGRADE | MIME_OPT_REPORT_NESTING : MIME_OPT_DISABLE_MIME, - smtp_generic_maps ? + smtp_generic_maps + || smtp_header_checks ? smtp_header_rewrite : smtp_header_out, (MIME_STATE_ANY_END) 0, + smtp_body_checks ? + smtp_body_rewrite : smtp_text_out, (MIME_STATE_ANY_END) 0, (MIME_STATE_ERR_PRINT) 0, diff --git a/postfix/src/smtpstone/smtp-sink.c b/postfix/src/smtpstone/smtp-sink.c index 242756124..4b5842b23 100644 --- a/postfix/src/smtpstone/smtp-sink.c +++ b/postfix/src/smtpstone/smtp-sink.c @@ -784,14 +784,35 @@ static void delay_event(int unused_event, char *context) { SINK_STATE *state = (SINK_STATE *) context; - state->delayed_response(state, state->delayed_args); - myfree(state->delayed_args); - state->delayed_args = 0; + switch (vstream_setjmp(state->stream)) { + + default: + msg_panic("unknown read/write error"); + /* NOTREACHED */ + + case SMTP_ERR_TIME: + msg_warn("write timeout"); + disconnect(state); + return; + + case SMTP_ERR_EOF: + msg_warn("lost connection"); + disconnect(state); + return; + + case 0: + state->delayed_response(state, state->delayed_args); + myfree(state->delayed_args); + state->delayed_args = 0; + break; + } if (state->delayed_response == quit_response) { disconnect(state); return; } + state->delayed_response = 0; + /* Resume input event handling after the delayed response. */ event_enable_read(vstream_fileno(state->stream), read_event, (char *) state); event_request_timer(read_timeout, (char *) state, var_tmout); @@ -1311,7 +1332,7 @@ static void connect_event(int unused_event, char *unused_context) case 0: if (command_resp(state, command_table, "connect", "") < 0) disconnect(state); - else { + else if (command_table->delay == 0) { event_enable_read(fd, read_event, (char *) state); event_request_timer(read_timeout, (char *) state, var_tmout); } diff --git a/postfix/src/util/events.c b/postfix/src/util/events.c index d0545aad1..0aa9eee52 100644 --- a/postfix/src/util/events.c +++ b/postfix/src/util/events.c @@ -108,7 +108,8 @@ /* event_drain() repeatedly calls event_loop() until no more timer /* events or I/O events are pending or until the time limit is reached. /* This routine must not be called from an event_whatever() callback -/* routine. +/* routine. Note: this function ignores pending timer events, and +/* assumes that no new I/O events will be registered. /* DIAGNOSTICS /* Panics: interface violations. Fatal errors: out of memory, /* system call failure. Warnings: the number of available diff --git a/postfix/src/util/inet_addr_local.c b/postfix/src/util/inet_addr_local.c index 6d6e93eaa..1ebf3fc6e 100644 --- a/postfix/src/util/inet_addr_local.c +++ b/postfix/src/util/inet_addr_local.c @@ -99,9 +99,10 @@ * With SIOCGLIFNETMASK we can obtain the netmask for either address family. * Again, this is not present in all major operating systems. * - * - On Linux, get IPv4 interface information with SIOCGIFCONF, and read IPv6 - * address/prefix information from a file in the /proc filesystem. Linux - * does not return IPv6 addresses with SIOCGIFCONF. + * - On Linux, glibc's getifaddrs(3) has returned IPv4 information for some + * time, but IPv6 information was not returned until 2.3.3. With older Linux + * versions we get IPv4 interface information with SIOCGIFCONF, and read + * IPv6 address/prefix information from a file in the /proc filesystem. * * - On other systems we expect SIOCGIFCONF to return IPv6 addresses. Since * SIOCGIFNETMASK does not work reliably for IPv6 addresses, we always set @@ -437,7 +438,7 @@ static int ial_siocgif(INET_ADDR_LIST *addr_list, #ifdef HAS_PROCNET_IFINET6 /* - * Linux does not provide proper calls to retrieve IPv6 interface + * Older Linux versions lack proper calls to retrieve IPv6 interface * addresses. Instead, the addresses can be read from a file in the * /proc tree. The most important issue with this approach however * is that the /proc tree may not always be available, for example diff --git a/postfix/src/util/sys_defs.h b/postfix/src/util/sys_defs.h index 3576d6900..af3b1fd17 100644 --- a/postfix/src/util/sys_defs.h +++ b/postfix/src/util/sys_defs.h @@ -724,9 +724,14 @@ extern int initgroups(const char *, int); #endif #ifndef NO_IPV6 # define HAS_IPV6 +#if defined(__GLIBC_PREREQ) && __GLIBC_PREREQ(2,4) +/* Really 2.3.3 or later, but there's no __GLIBC_MICRO version macro. */ +# define HAVE_GETIFADDRS +#else # define HAS_PROCNET_IFINET6 # define _PATH_PROCNET_IFINET6 "/proc/net/if_inet6" #endif +#endif #include #if !defined(KERNEL_VERSION) # define KERNEL_VERSION(a,b,c) (LINUX_VERSION_CODE + 1)