command before you can use the file, and that it does not
detect changes after the file is read. All information is
read into memory. Files: util/dict_open.c, util/dict_thash.[hc],
- proto/DATABSE_README.html, postconf/postconf.c
+ proto/DATABASE_README.html, postconf/postconf.c
+
+20100912
+
+ Feature: bare newline detection in postscreen. Real spambots
+ don't make this mistake but poorly-written software often does.
+ File: postscreen/smtpd.c.
+
+ Documentation: POSTSCREEN_README including instructions for
+ turning postscreen(8) on without blocking mail, and more.
+ Trimmed the text in the postscreen(8) manpage. File:
+ proto/POSTSCREEN_README.html, postscreen/postscreen.c.
(8)'s DNSBL lookups is that DNS records are already cached before the Postfix
SMTP server looks them up later.
-postscreen(8) maintains a temporary whitelist of positive decisions. Once an
-SMTP client is whitelisted, it is immediately forwarded to a real Postfix SMTP
-server process without further checking.
+postscreen(8) maintains a temporary whitelist for clients that have passed a
+number of tests. When an SMTP client IP address is whitelisted, postscreen(8)
+hands off the connection immediately to a Postfix SMTP server process. This
+minimizes the overhead for legitimate mail.
-By default, the program logs only statistics, and it does not run any checks on
-clients in mynetworks (primarily, to avoid problems with buggy SMTP
-implementations in network appliances).
-
-Many of the ideas in postscreen(8) have been explored in earlier work by
-Michael Tokarev, in OpenBSD spamd, and in MailChannels Traffic Control.
+By default, postscreen(8) logs statistics and hands off every connection to a
+Postfix SMTP server process, while excluding clients in mynetworks from all
+tests (primarily, to avoid problems with non-standard SMTP implementations in
+network appliances). This mode is useful for non-destructive testing.
Topics in this document:
* Other errors
* When all tests succeed
* Configuring the postscreen(8) service
+ * Historical notes and credits
T\bTh\bhe\be b\bba\bas\bsi\bic\bc i\bid\bde\bea\ba b\bbe\beh\bhi\bin\bnd\bd p\bpo\bos\bst\bts\bsc\bcr\bre\bee\ben\bn(\b(8\b8)\b)
Spambots have a limited amount of time to send out spam before they become
blacklisted. For this reason, spambots make compromises in their SMTP protocol
implementation to speed up spam deliveries. For example, they speak before
-their turn.
+their turn, or they ignore responses from SMTP servers.
-Many spambots avoid spamming the same site repeatedly. Thus, postscreen(8) must
-make a long-term decision after a single measurement. For example, allow a good
-client to skip the DNSBL test for 24 hours.
+Many spambots avoid spamming the same site repeatedly, in an attempt to fly
+under the radar. Thus, postscreen(8) must make a long-term decision after a
+single measurement. For example, allow a good client to skip the "pregreet"
+test for 24 hours.
To recognize spambots, postscreen(8) measures properties of the client IP
address and of the client SMTP protocol implementation (the protocol
typically 24 hours for simple tests or 1 week for complex tests. This minimizes
the impact of the tests on legitimate mail clients.
-After logging the result of its tests, postscreen(8) by default forwards all
-connections to a real SMTP server process. This mode is useful for non-
-destructive testing.
+After logging its findings, postscreen(8) by default hands off all connections
+to a Postfix SMTP server process. This mode is useful for non-destructive
+testing.
In a typical production setting, postscreen(8) is configured to reject mail
-from clients that fail one or more tests, after logging the sender and
+from clients that fail one or more tests, after logging the helo, sender and
recipient information.
Note: postscreen(8) is not an SMTP proxy; this is intentional. The purpose is
-to prioritize legitimate clients with as little overhead as possible.
+to keep spambots away from Postfix, with minimal overhead for legitimate
+clients.
Q\bQu\bui\bic\bck\bk t\bte\bes\bst\bts\bs b\bbe\bef\bfo\bor\bre\be e\bev\bve\ber\bry\byt\bth\bhi\bin\bng\bg e\bel\bls\bse\be
W\bWH\bHI\bIT\bTE\bEL\bLI\bIS\bST\bTE\bED\bD address
-The action is not configurable: immediately forward the connection to a real
-SMTP server process.
+The action is not configurable: immediately hand off the connection to a
+Postfix SMTP server process.
P\bPe\ber\brm\bma\ban\bne\ben\bnt\bt b\bbl\bla\bac\bck\bkl\bli\bis\bst\bt t\bte\bes\bst\bt
P\bPA\bAS\bSS\bS O\bOL\bLD\bD address
-The action is not configurable: immediately forward the connection to a real
-SMTP server process. The client is excluded from further tests until its
-temporary whitelist entry expires, as controlled with the postscreen_*_ttl
+The action is not configurable: immediately hand off the connection to a
+Postfix SMTP server process. The client is excluded from further tests until
+its temporary whitelist entry expires, as controlled with the postscreen_*_ttl
parameters. Expired entries are silently renewed if possible.
T\bTe\bes\bst\bts\bs b\bbe\bef\bfo\bor\bre\be t\bth\bhe\be 2\b22\b20\b0 S\bSM\bMT\bTP\bP s\bse\ber\brv\bve\ber\br g\bgr\bre\bee\bet\bti\bin\bng\bg
parallel.
When a good client passes these tests, and no "deep protocol tests" are
-configured, postscreen(8) adds the client to the temporary whitelist and passes
-the "live" connection to a Postfix SMTP server process. The client can then
+configured, postscreen(8) adds the client to the temporary whitelist and hands
+off the "live" connection to a Postfix SMTP server process. The client can then
continue as if postscreen(8) never even existed (except of course for the short
postscreen_greet_wait delay).
be queried in parallel with the reverse client IP address. This test is
disabled by default.
- CAUTION: when postscreen rejects mail, it replies with the DNSBL domain
- name. Use the postscreen_dnsbl_reply_map feature to hide "password"
+ CAUTION: when postscreen rejects mail, it's SMTP reply contains the DNSBL
+ domain name. Use the postscreen_dnsbl_reply_map feature to hide "password"
information in DNSBL domain names.
When the postscreen_greet_wait time has elapsed, and the combined DNSBL score
T\bTe\bes\bst\bts\bs a\baf\bft\bte\ber\br t\bth\bhe\be 2\b22\b20\b0 S\bSM\bMT\bTP\bP s\bse\ber\brv\bve\ber\br g\bgr\bre\bee\bet\bti\bin\bng\bg
-The tests in this phase use an SMTP protocol engine that is built into the
+In this phase of the protocol, postscreen(8) implements a number of "deep
+protocol" tests. These tests use an SMTP protocol engine that is built into the
postscreen(8) server.
-Important notes:
-
- * These tests are disabled by default, because they are more intrusive than
- the pregreet and DNSBL tests.
-
- When a good client passes the deep protocol tests, postscreen(8) adds the
- client to the temporary whitelist but it cannot pass the "live" connection
- to a Postfix SMTP server process in the middle of the session. Instead,
- postscreen(8) defers mail delivery attempts with a 4XX status, logs the
- helo/sender/recipient information, and waits for the client to disconnect.
+Important note: deep protocol tests are disabled by default. They are more
+intrusive than the pregreet and DNSBL tests, and they have limitations as
+discussed next.
- The next time the client connects it will be allowed to talk to a real SMTP
- server process to deliver its mail.
+ * When a good client passes the deep protocol tests, postscreen(8) adds the
+ client to the temporary whitelist but it cannot hand off the "live"
+ connection to a Postfix SMTP server process in the middle of the session.
+ Instead, postscreen(8) defers mail delivery attempts with a 4XX status,
+ logs the helo/sender/recipient information, and waits for the client to
+ disconnect.
- To minimize the impact of these tests, postscreen(8) gives them relatively
- long expiration times.
+ The next time the client connects it will be allowed to talk to a Postfix
+ SMTP server process to deliver its mail. To minimize the impact of this
+ limitation, postscreen(8) gives deep protocol tests a relatively long
+ expiration time.
- * postscreen(8) does not implement the AUTH, STARTTLS, XCLIENT, and XFORWARD
- features. STARTTLS support may be added in a future version.
+ * postscreen(8)'s built-in SMTP engine does not implement the AUTH, STARTTLS,
+ XCLIENT, and XFORWARD features. STARTTLS and AUTH support may be added in a
+ future version.
End-user clients should connect directly to the submission service. Other
systems that require the above features should directly connect to a Postfix
C\bCo\bom\bmm\bma\ban\bnd\bd p\bpi\bip\bpe\bel\bli\bin\bni\bin\bng\bg t\bte\bes\bst\bt
By default, SMTP is a half-duplex protocol: the sender and receiver send one
-command and one response at a time. Unlike the real Postfix SMTP server,
-postscreen(8) does not announce support for ESMTP command pipelining.
-Therefore, clients are not allowed to send multiple commands. This test is
-disabled by default.
+command and one response at a time. Unlike the Postfix SMTP server, postscreen
+(8) does not announce support for ESMTP command pipelining. Therefore, clients
+are not allowed to send multiple commands. postscreen(8)'s deep protocol test
+for this is disabled by default.
With "postscreen_pipelining_enable = yes", postscreen(8) detects spambots that
send multiple commands, instead of sending one command and waiting for the
server to reply.
-This test is opportunistically enabled when enabled when postscreen(8) has to
-use the built-in SMTP engine anyway, to make postscreen(8) logging more
-informative.
+This test is opportunistically enabled when postscreen(8) has to use the built-
+in SMTP engine anyway. This is to make postscreen(8) logging more informative.
When a client sends multiple commands, postscreen(8) logs this as:
-C\bCO\bOM\bMM\bMA\bAN\bND\bD P\bPI\bIP\bPE\bEL\bLI\bIN\bNI\bIN\bNG\bG a\baf\bft\bte\ber\br time f\bfr\bro\bom\bm address
+ C\bCO\bOM\bMM\bMA\bAN\bND\bD P\bPI\bIP\bPE\bEL\bLI\bIN\bNI\bIN\bNG\bG a\baf\bft\bte\ber\br time f\bfr\bro\bom\bm address
Translation: the SMTP client at address sent multiple SMTP commands, instead of
sending one command and then waiting for the server to reply. This happened
N\bNo\bon\bn-\b-S\bSM\bMT\bTP\bP c\bco\bom\bmm\bma\ban\bnd\bd t\bte\bes\bst\bt
+Some spambots send their mail through open proxies. A symptom of this is the
+usage of commands such as CONNECT and other non-SMTP commands. Just like the
+Postfix SMTP server's smtpd_forbidden_commands feature, postscreen(8) has an
+equivalent postscreen_forbidden_commands feature to block these clients.
+postscreen(8)'s deep protocol test for this is disabled by default.
+
With "postscreen_non_smtp_command_enable = yes", postscreen(8) detects spambots
-that send non-SMTP commands, such as commands specified with the
-postscreen_forbidden_commands parameter, and commands that have the syntax of a
-message header label.
+that send commands specified with the postscreen_forbidden_commands parameter.
+This also detects commands with the syntax of a message header label. The
+latter is a symptom that the client is sending message content after ignoring
+all the responses from postscreen(8) that reject mail.
-This test is disabled by default. The test is opportunistically enabled when
-postscreen(8) has to use the built-in SMTP engine anyway, to make postscreen(8)
-logging more informative.
+This test is opportunistically enabled when postscreen(8) has to use the built-
+in SMTP engine anyway. This is to make postscreen(8) logging more informative.
When a client sends non-SMTP commands, postscreen(8) logs this as:
B\bBa\bar\bre\be n\bne\bew\bwl\bli\bin\bne\be t\bte\bes\bst\bt
SMTP is a line-oriented protocol: lines have a limited length, and are
-terminated with <CR><LF>.
+terminated with <CR><LF>. Lines ending in a "bare" <LF>, that is newline not
+preceded by carriage return, are not allowed in SMTP. postscreen(8)'s deep
+protocol test for this is disabled by default.
-With "postscreen_bare_newline_enable = yes", postscreen(8) detects spambots
-that send lines ending in bare newline characters, that is newline not preceded
-by carriage return.
+With "postscreen_bare_newline_enable = yes", postscreen(8) detects clients that
+send lines ending in bare newline characters.
-This test is disabled by default. The test is opportunistically enabled when
-postscreen(8) has to use the built-in SMTP engine anyway, to make postscreen(8)
-logging more informative.
+This test is opportunistically enabled when postscreen(8) has to use the built-
+in SMTP engine anyway. This is to make postscreen(8) logging more informative.
When a client sends bare newline characters, postscreen(8) logs this as:
the temporary whitelist entry expires, as controlled with the postscreen_*_ttl
parameters.
-When no "deep procol tests" are configured, postscreen(8) passes the "live"
+When no "deep protocol tests" are configured, postscreen(8) passes the "live"
connection to a Postfix SMTP server process. The client can then continue as if
postscreen(8) never even existed (except for the short postscreen_greet_wait
delay).
-When any "deep procol tests" are configured, postscreen(8) cannot pass the
-"live" connection to a Postfix SMTP server process. Instead, postscreen(8)
-defers mail delivery attempts with a 4XX status, logs the helo/sender/recipient
-information, and waits for the client to disconnect. The next time the client
-connects it will be allowed to talk to a Postfix SMTP server process to deliver
-its mail.
+When any "deep protocol tests" are configured, postscreen(8) cannot hand off
+the "live" connection to a Postfix SMTP server process in the middle of the
+session. Instead, postscreen(8) defers mail delivery attempts with a 4XX
+status, logs the helo/sender/recipient information, and waits for the client to
+disconnect. The next time the client connects it will be allowed to talk to a
+Postfix SMTP server process to deliver its mail. postscreen(8) mitigates the
+impact of this limitation by giving deep protocol tests a long expiration time.
C\bCo\bon\bnf\bfi\big\bgu\bur\bri\bin\bng\bg t\bth\bhe\be p\bpo\bos\bst\bts\bsc\bcr\bre\bee\ben\bn(\b(8\b8)\b) s\bse\ber\brv\bvi\bic\bce\be
Notes:
+ * Some postscreen(8) configuration parameters implement stress-dependent
+ behavior. This is supported only when the default value is stress-dependent
+ (that is, it looks like ${stress?X}${stress:Y}). Other parameters always
+ evaluate as if the stress value is the empty string.
+
* See "Tests before the 220 SMTP server greeting" for details about the
logging from these postscreen(8) tests.
than the pregreet or DNSBL tests.
When a good client passes the "deep protocol tests", postscreen(8) adds the
- client to the temporary whitelist but it cannot pass the "live" connection
- to a Postfix SMTP server process in the middle of the session. Instead,
- postscreen(8) defers mail delivery attempts with a 4XX status, logs the
- helo/sender/recipient information, and waits for the client to disconnect.
+ client to the temporary whitelist but it cannot hand off the "live"
+ connection to a Postfix SMTP server process in the middle of the session.
+ Instead, postscreen(8) defers mail delivery attempts with a 4XX status,
+ logs the helo/sender/recipient information, and waits for the client to
+ disconnect.
- When the client comes back in a later session, it is allowed to talk
+ When the good client comes back in a later session, it is allowed to talk
directly to a Postfix SMTP server. See "after_220 Tests after the 220 SMTP
server greeting above for limitations with STARTTLS, AUTH and other
- features that clients may need. Wietse enables "deep protocol tests" on his
- own internet-facing mail server.
+ features that clients may need.
+
+ An unexpected benefit from "deep protocol tests" is that some "good"
+ clients don't return after the 4XX reply; these clients were not so good
+ after all. Wietse enables "deep protocol tests" on his own internet-facing
+ mail server.
* There is also support for permanent blacklists and whitelists; see the
description of the postscreen_whitelist_networks and
5. Read the new configuration with "postfix reload".
+H\bHi\bis\bst\bto\bor\bri\bic\bca\bal\bl n\bno\bot\bte\bes\bs a\ban\bnd\bd c\bcr\bre\bed\bdi\bit\bts\bs
+
+Many ideas in postscreen(8) were explored in earlier work by Michael Tokarev,
+in OpenBSD spamd, and in MailChannels Traffic Control.
+
+Wietse threw together a crude prototype with pregreet and dnsbl support in June
+2009, because he needed something new for a Mailserver conference presentation
+in July. Ralf Hildebrandt ran this code on several servers to collect real-
+world evidence. This version used the dnsblog(8) ad-hoc DNS client program.
+
+Wietse needed new material for a LISA conference presentation in November 2010,
+so he added support for DNSBL weights and filters in August, followed by a
+major code rewrite, deep protocol tests, helo/sender/recipient logging, and
+stress-adaptive behavior in September. Ralf Hildebrandt ran this code on
+several servers to collect real-world evidence. This version still used the
+same delay for pregreet and DNBL tests, as well as the embarrassing dnsblog(8)
+ad-hoc DNS client.
+
Incompatibility with snapshot 20100912
======================================
+- If your DNSBL queries have a "secret" in the domain name, you
+ must now censor this information from the postscreen(8) SMTP
+ replies. For example:
+
+ /etc/postfix/main.cf:
+ postscreen_dnsbl_reply_map = texthash:/etc/postfix/dnsbl_reply
+
+ /etc/postfix/dnsbl_reply:
+ # Secret DNSBL name Name in postscreen(8) replies
+ secret.zen.spamhaus.org zen.spamhaus.org
+
+ The texthash: format is similar to hash: except that there is no need to
+ run postmap(1) before the file can be used, and that it does not detect
+ changes after the file is read. It is new with Postfix version 2.8.
+
- The postscreen "continue" action is now called "ignore". The old
name is still supported but no longer documented.
already cached before the Postfix SMTP server looks them up later.
</p>
-<p> <a href="postscreen.8.html">postscreen(8)</a> maintains a temporary whitelist of positive
-decisions. Once an SMTP client is whitelisted, it is immediately
-forwarded to a real Postfix SMTP server process without further
-checking. </p>
-
-<p> By default, the program logs only statistics, and it does not
-run any checks on clients in <a href="postconf.5.html#mynetworks">mynetworks</a> (primarily, to avoid problems
-with buggy SMTP implementations in network appliances). </p>
-
-<p> Many of the ideas in <a href="postscreen.8.html">postscreen(8)</a> have been explored in earlier
-work by Michael Tokarev, in OpenBSD spamd, and in MailChannels
-Traffic Control. </p>
+<p> <a href="postscreen.8.html">postscreen(8)</a> maintains a temporary whitelist for clients that
+have passed a number of tests. When an SMTP client IP address is
+whitelisted, <a href="postscreen.8.html">postscreen(8)</a> hands off the connection immediately to
+a Postfix SMTP server process. This minimizes the overhead for
+legitimate mail. </p>
+
+<p> By default, <a href="postscreen.8.html">postscreen(8)</a> logs statistics and hands off every
+connection to a Postfix SMTP server process, while excluding clients
+in <a href="postconf.5.html#mynetworks">mynetworks</a> from all tests (primarily, to avoid problems with
+non-standard SMTP implementations in network appliances). This mode
+is useful for non-destructive testing. </p>
<p> Topics in this document: </p>
<li> <a href="#config"> Configuring the postscreen(8) service</a>
+<li> <a href="#historical"> Historical notes and credits </a>
+
</ul>
<h2> <a name="basic">The basic idea behind postscreen(8)</a> </h2>
<p> Spambots have a limited amount of time to send out spam before
they become blacklisted. For this reason, spambots make compromises
in their SMTP protocol implementation to speed up spam deliveries.
-For example, they speak before their turn. </p>
+For example, they speak before their turn, or they ignore responses
+from SMTP servers. </p>
-<p> Many spambots avoid spamming the same site repeatedly. Thus,
-<a href="postscreen.8.html">postscreen(8)</a> must make a long-term decision after a single
-measurement. For example, allow a good client to skip the DNSBL
-test for 24 hours. </p>
+<p> Many spambots avoid spamming the same site repeatedly, in an
+attempt to fly under the radar. Thus, <a href="postscreen.8.html">postscreen(8)</a> must make a
+long-term decision after a single measurement. For example, allow
+a good client to skip the "<a href="#pregreet">pregreet</a>" test
+for 24 hours. </p>
<p> To recognize spambots, <a href="postscreen.8.html">postscreen(8)</a> measures properties of the
client IP address and of the client SMTP protocol implementation
tests or 1 week for complex tests. This minimizes the impact of
the tests on legitimate mail clients. </p>
-<p> After logging the result of its tests, <a href="postscreen.8.html">postscreen(8)</a> by default
-forwards all connections to a real SMTP server process. This mode
-is useful for non-destructive testing. </p>
+<p> After logging its findings, <a href="postscreen.8.html">postscreen(8)</a> by default hands off
+all connections to a Postfix SMTP server process. This mode is
+useful for non-destructive testing. </p>
<p> In a typical production setting, <a href="postscreen.8.html">postscreen(8)</a> is configured
to reject mail from clients that fail one or more tests, after
-logging the sender and recipient information. </p>
+logging the helo, sender and recipient information. </p>
<p> Note: <a href="postscreen.8.html">postscreen(8)</a> is not an SMTP proxy; this is intentional.
-The purpose is to prioritize legitimate clients with as little
-overhead as possible. </p>
+The purpose is to keep spambots away from Postfix, with minimal
+overhead for legitimate clients. </p>
<h2> <a name="quick">Quick tests before everything else</a> </h2>
<b>WHITELISTED</b> <i>address</i>
</pre>
-<p> The action is not configurable: immediately forward the
-connection to a real SMTP server process. </p>
+<p> The action is not configurable: immediately hand off the
+connection to a Postfix SMTP server process. </p>
<h3> <a name="perm_black"> Permanent blacklist test </a> </h3>
<b>PASS OLD</b> <i>address</i>
</pre>
-<p> The action is not configurable: immediately forward the
-connection to a real SMTP server process. The client is
+<p> The action is not configurable: immediately hand off the
+connection to a Postfix SMTP server process. The client is
excluded from further tests until its temporary whitelist
entry expires, as controlled with the postscreen_*_ttl
parameters. Expired entries are silently renewed if possible. </p>
<p> When a good client passes these tests, and no "<a
href="#after_220">deep protocol tests</a>" are configured, postscreen(8)
-adds the client to the temporary whitelist and passes the "live"
+adds the client to the temporary whitelist and hands off the "live"
connection to a Postfix SMTP server process. The client can then
continue as if <a href="postscreen.8.html">postscreen(8)</a> never even existed (except of course
for the short <a href="postconf.5.html#postscreen_greet_wait">postscreen_greet_wait</a> delay). </p>
<blockquote>
<p>
-CAUTION: when postscreen rejects mail, it replies with the DNSBL
-domain name. Use the <a href="postconf.5.html#postscreen_dnsbl_reply_map">postscreen_dnsbl_reply_map</a> feature to hide
-"password" information in DNSBL domain names.
+CAUTION: when postscreen rejects mail, it's SMTP reply contains the
+DNSBL domain name. Use the <a href="postconf.5.html#postscreen_dnsbl_reply_map">postscreen_dnsbl_reply_map</a> feature to
+hide "password" information in DNSBL domain names.
</p>
</blockquote>
<h2> <a name="after_220">Tests after the 220 SMTP server greeting</a> </h2>
-<p> The tests in this phase use an SMTP protocol engine that is
-built into the <a href="postscreen.8.html">postscreen(8)</a> server. </p>
+<p> In this phase of the protocol, <a href="postscreen.8.html">postscreen(8)</a> implements a
+number of "deep protocol" tests. These tests use an SMTP protocol
+engine that is built into the <a href="postscreen.8.html">postscreen(8)</a> server. </p>
-<p> Important notes: </p>
+<p> Important note: deep protocol tests are disabled by default.
+They are more intrusive than the pregreet and DNSBL tests, and they
+have limitations as discussed next. </p>
<ul>
-<li> <p> These tests are disabled by default, because they
-are more intrusive than the pregreet and DNSBL tests. </p>
-
-<p> When a good client passes the <a href="#after_220">deep
-protocol tests </a>, postscreen(8) adds the client to the temporary
-whitelist but it cannot pass the "live" connection to a Postfix
+<li> <p> When a good client passes the <a href="#after_220">deep
+protocol tests</a>, postscreen(8) adds the client to the temporary
+whitelist but it cannot hand off the "live" connection to a Postfix
SMTP server process in the middle of the session. Instead, <a href="postscreen.8.html">postscreen(8)</a>
defers mail delivery attempts with a 4XX status, logs the
helo/sender/recipient information, and waits for the client to
disconnect. </p>
<p> The next time the client connects it will be allowed to talk
-to a real SMTP server process to deliver its mail. </p>
-
-<p> To minimize the impact of these tests, <a href="postscreen.8.html">postscreen(8)</a> gives them
-relatively long expiration times. </p>
+to a Postfix SMTP server process to deliver its mail. To minimize the
+impact of this limitation, <a href="postscreen.8.html">postscreen(8)</a> gives deep protocol tests
+a relatively long expiration time. </p>
-<li> <p> <a href="postscreen.8.html">postscreen(8)</a> does not implement the AUTH, STARTTLS,
-XCLIENT, and XFORWARD features. STARTTLS support may be added in
-a future version. </p>
+<li> <p> <a href="postscreen.8.html">postscreen(8)</a>'s built-in SMTP engine does not implement
+the AUTH, STARTTLS, XCLIENT, and XFORWARD features. STARTTLS and
+AUTH support may be added in a future version. </p>
</ul>
<p> By default, SMTP is a half-duplex protocol: the sender and
receiver send one command and one response at a time. Unlike the
-
real Postfix SMTP server, <a href="postscreen.8.html">postscreen(8)</a> does not announce support
+Postfix SMTP server, <a href="postscreen.8.html">postscreen(8)</a> does not announce support
for ESMTP command pipelining. Therefore, clients are not allowed
-to send multiple commands. This test is disabled by default. </p>
+to send multiple commands. postscreen(8)'s <a href="#after_220">deep
+protocol test</a> for this is disabled by default. </p>
<p> With "<a href="postconf.5.html#postscreen_pipelining_enable">postscreen_pipelining_enable</a> = yes", <a href="postscreen.8.html">postscreen(8)</a> detects
spambots that send multiple commands, instead of sending one command
-and waiting for the server to reply. </p>
+and waiting for the server to reply. </p>
-<p> This test is opportunistically enabled when enabled when
-<a href="postscreen.8.html">postscreen(8)</a> has to use the built-in SMTP engine anyway, to make
-<a href="postscreen.8.html">postscreen(8)</a> logging more informative. </p>
+<p> This test is opportunistically enabled when <a href="postscreen.8.html">postscreen(8)</a> has
+to use the built-in SMTP engine anyway. This is to make <a href="postscreen.8.html">postscreen(8)</a>
+logging more informative. </p>
<p> When a client sends multiple commands, <a href="postscreen.8.html">postscreen(8)</a> logs this
as: </p>
-<b>COMMAND PIPELINING after</b> <i>time</i> <b>from</b> <i>address</i>
+<pre>
+ <b>COMMAND PIPELINING after</b> <i>time</i> <b>from</b> <i>address</i>
+</pre>
<p> Translation: the SMTP client at <i>address</i> sent multiple
SMTP commands, instead of sending one command and then waiting for
<h3> <a name="non_smtp">Non-SMTP command test</a> </h3>
+<p> Some spambots send their mail through open proxies. A symptom
+of this is the usage of commands such as CONNECT and other non-SMTP
+commands. Just like the Postfix SMTP server's <a href="postconf.5.html#smtpd_forbidden_commands">smtpd_forbidden_commands</a>
+feature, <a href="postscreen.8.html">postscreen(8)</a> has an equivalent <a href="postconf.5.html#postscreen_forbidden_commands">postscreen_forbidden_commands</a>
+feature to block these clients. postscreen(8)'s <a href="#after_220">deep
+protocol test</a> for this is disabled by default. </p>
+
<p> With "<a href="postconf.5.html#postscreen_non_smtp_command_enable">postscreen_non_smtp_command_enable</a> = yes", <a href="postscreen.8.html">postscreen(8)</a>
-detects spambots that send non-SMTP commands, such as commands
-specified with the <a href="postconf.5.html#postscreen_forbidden_commands">postscreen_forbidden_commands</a> parameter, and
-commands that have the syntax of a message header label. </p>
+detects spambots that send commands specified with the
+<a href="postconf.5.html#postscreen_forbidden_commands">postscreen_forbidden_commands</a> parameter. This also detects commands
+with the syntax of a message header label. The latter is a symptom
+that the client is sending message content after ignoring all the
+responses from <a href="postscreen.8.html">postscreen(8)</a> that reject mail. </p>
-<p> This test is disabled by default. The test is opportunistically
-enabled when <a href="postscreen.8.html">postscreen(8)</a> has to use the built-in SMTP engine
-anyway, to make <a href="postscreen.8.html">postscreen(8)</a> logging more informative. </p>
+<p> This test is opportunistically enabled when <a href="postscreen.8.html">postscreen(8)</a> has
+to use the built-in SMTP engine anyway. This is to make <a href="postscreen.8.html">postscreen(8)</a>
+logging more informative. </p>
<p> When a client sends non-SMTP commands, <a href="postscreen.8.html">postscreen(8)</a> logs this
as: </p>
<h3> <a name="barelf">Bare newline test</a> </h3>
-<p> SMTP is a line-oriented protocol: lines have a limited
-length, and are terminated with <CR><LF>. </p>
+<p> SMTP is a line-oriented protocol: lines have a limited length,
+and are terminated with <CR><LF>. Lines ending in a
+"bare" <LF>, that is newline not preceded by carriage return,
+are not allowed in SMTP. postscreen(8)'s <a href="#after_220">deep
+protocol test</a> for this is disabled by default. </p>
<p> With "<a href="postconf.5.html#postscreen_bare_newline_enable">postscreen_bare_newline_enable</a> = yes", <a href="postscreen.8.html">postscreen(8)</a>
-detects spambots that send lines ending in bare newline
-characters, that is newline not preceded by carriage return. </p>
+detects clients that send lines ending in bare newline characters.
+</p>
-<p> This test is disabled by default. The test is opportunistically
-enabled when <a href="postscreen.8.html">postscreen(8)</a> has to use the built-in SMTP engine
-anyway, to make <a href="postscreen.8.html">postscreen(8)</a> logging more informative. </p>
+<p> This test is opportunistically enabled when <a href="postscreen.8.html">postscreen(8)</a> has
+to use the built-in SMTP engine anyway. This is to make <a href="postscreen.8.html">postscreen(8)</a>
+logging more informative. </p>
<p> When a client sends bare newline characters, <a href="postscreen.8.html">postscreen(8)</a> logs
this as:
address from further tests until the temporary whitelist entry
expires, as controlled with the postscreen_*_ttl parameters. </p>
-<p> When no "<a href="#after_220">deep procol tests</a>" are
+<p> When no "<a href="#after_220">deep protocol tests</a>" are
configured, <a href="postscreen.8.html">postscreen(8)</a> passes the "live" connection to a Postfix
SMTP server process. The client can then continue as if <a href="postscreen.8.html">postscreen(8)</a>
never even existed (except for the short <a href="postconf.5.html#postscreen_greet_wait">postscreen_greet_wait</a> delay).
</p>
-<p> When any "<a href="#after_220">deep procol tests</a>" are
-configured, <a href="postscreen.8.html">postscreen(8)</a> cannot pass the "live" connection to a
-Postfix SMTP server process. Instead, <a href="postscreen.8.html">postscreen(8)</a> defers mail
-delivery attempts with a 4XX status, logs the helo/sender/recipient
-information, and waits for the client to disconnect. The next time
-the client connects it will be allowed to talk to a Postfix SMTP
-server process to deliver its mail. </p>
+<p> When any "<a href="#after_220">deep protocol tests</a>" are
+configured, <a href="postscreen.8.html">postscreen(8)</a> cannot hand off the "live" connection to
+a Postfix SMTP server process in the middle of the session. Instead,
+<a href="postscreen.8.html">postscreen(8)</a> defers mail delivery attempts with a 4XX status, logs
+the helo/sender/recipient information, and waits for the client to
+disconnect. The next time the client connects it will be allowed
+to talk to a Postfix SMTP server process to deliver its mail.
+<a href="postscreen.8.html">postscreen(8)</a> mitigates the impact of this limitation by giving
+<a href="#after_220">deep protocol tests</a> a long expiration
+time. </p>
<h2> <a name="config"> Configuring the postscreen(8) service</a>
</h2>
<ul>
+<li> <p> Some <a href="postscreen.8.html">postscreen(8)</a> configuration parameters implement
+stress-dependent behavior. This is supported only when the default
+value is stress-dependent (that is, it looks like ${stress?X}${stress:Y}).
+Other parameters always evaluate as if the stress value is the empty
+string. </p>
+
<li> <p> See "<a href="#before_220">Tests before the 220 SMTP server
greeting</a>" for details about the logging from these postscreen(8)
tests. </p>
<p> When a good client passes the "<a href="#after_220">deep
protocol tests</a>", postscreen(8) adds the client to the temporary
-whitelist but it cannot pass the "live" connection to a Postfix
+whitelist but it cannot hand off the "live" connection to a Postfix
SMTP server process in the middle of the session. Instead, <a href="postscreen.8.html">postscreen(8)</a>
defers mail delivery attempts with a 4XX status, logs the
helo/sender/recipient information, and waits for the client to
disconnect. </p>
-<p> When the client comes back in a later session, it is allowed
+<p> When the good client comes back in a later session, it is allowed
to talk directly to a Postfix SMTP server. See "after_220 <a
href="#after_220">Tests after the 220 SMTP server greeting</a> above
for limitations with STARTTLS, AUTH and other features that clients
-may need. Wietse enables "<a href="#after_220">deep protocol
-tests</a>" on his own internet-facing mail server. </p>
+may need. </p>
+
+<p> An unexpected benefit from "<a href="#after_220">deep protocol
+tests</a>" is that some "good" clients don't return after the 4XX
+reply; these clients were not so good after all. Wietse enables
+"<a href="#after_220">deep protocol tests</a>" on his own internet-facing
+mail server. </p>
<li> <p> There is also support for permanent blacklists and whitelists;
see the description of the <a href="postconf.5.html#postscreen_whitelist_networks">postscreen_whitelist_networks</a> and
</ol>
+<h2> <a name="historical"> Historical notes and credits </a> </h2>
+
+<p> Many ideas in <a href="postscreen.8.html">postscreen(8)</a> were explored in earlier work by
+Michael Tokarev, in OpenBSD spamd, and in MailChannels Traffic
+Control. </p>
+
+<p> Wietse threw together a crude prototype with pregreet and dnsbl
+support in June 2009, because he needed something new for a Mailserver
+conference presentation in July. Ralf Hildebrandt ran this code on
+several servers to collect real-world evidence. This version used
+the <a href="dnsblog.8.html">dnsblog(8)</a> ad-hoc DNS client program. </p>
+
+<p> Wietse needed new material for a LISA conference presentation
+in November 2010, so he added support for DNSBL weights and filters
+in August, followed by a major code rewrite, deep protocol tests,
+helo/sender/recipient logging, and stress-adaptive behavior in
+September. Ralf Hildebrandt ran this code on several servers to
+collect real-world evidence. This version still used the same delay
+for pregreet and DNBL tests, as well as the embarrassing <a href="dnsblog.8.html">dnsblog(8)</a>
+ad-hoc DNS client. </p>
+
</body>
</html>
processes, more Postfix SMTP server processes remain
available for legitimate clients.
- <a href="postscreen.8.html"><b>postscreen</b>(8)</a> maintains a temporary whitelist of positive
- decisions. Once an SMTP client is whitelisted, it is imme-
- diately forwarded to a real Postfix SMTP server process
- without further checking.
-
- By default, the program logs only statistics, and it does
- not run any tests against clients in <a href="postconf.5.html#mynetworks">mynetworks</a> (primar-
- ily, to avoid problems with non-standard SMTP implementa-
- tions in network appliances and test programs).
- <a href="postscreen.8.html"><b>postscreen</b>(8)</a> by default forwards all connections to a
- real SMTP server process. This mode is useful for non-
+ <a href="postscreen.8.html"><b>postscreen</b>(8)</a> maintains a temporary whitelist for clients
+ that have passed a number of tests. When an SMTP client
+ IP address is whitelisted, <a href="postscreen.8.html"><b>postscreen</b>(8)</a> hands off the
+ connection immediately to a Postfix SMTP server process.
+ This minimizes the overhead for legitimate mail.
+
+ By default, <a href="postscreen.8.html"><b>postscreen</b>(8)</a> logs statistics and hands off
+ every connection to a Postfix SMTP server process, while
+ excluding clients in <a href="postconf.5.html#mynetworks">mynetworks</a> from all tests (primarily,
+ to avoid problems with non-standard SMTP implementations
+ in network appliances). This mode is useful for non-
destructive testing.
In a typical production setting, <a href="postscreen.8.html"><b>postscreen</b>(8)</a> is config-
ured to reject mail from clients that fail one or more
- tests, after logging the helo, sender and recipient infor-
- mation.
+ tests. <a href="postscreen.8.html"><b>postscreen</b>(8)</a> logs rejected mail with the client
+ address, helo, sender and recipient information.
<a href="postscreen.8.html"><b>postscreen</b>(8)</a> is not an SMTP proxy; this is intentional.
The purpose is to keep spambots away from Postfix SMTP
- server processes, not to control traffic flows.
+ server processes, while minimizing overhead for legitimate
+ traffic.
<b>SECURITY</b>
The <a href="postscreen.8.html"><b>postscreen</b>(8)</a> server is moderately security-sensitive.
- It talks to untrusted clients on the network. The process
+ It talks to untrusted clients on the network. The process
can be run chrooted at fixed low privilege.
<b>STANDARDS</b>
Problems and transactions are logged to <b>syslogd</b>(8).
<b>BUGS</b>
- When successful tests involve <a href="postscreen.8.html"><b>postscreen</b>(8)</a>'s built-in
- SMTP protocol engine, <a href="postscreen.8.html"><b>postscreen</b>(8)</a> adds the client to the
- temporary whitelist but it cannot not hand off the "live"
- connection from a good SMTP client to a Postfix SMTP
- server process. Instead, <a href="postscreen.8.html"><b>postscreen</b>(8)</a> defers attempts to
+ Some of the non-default protocol tests involve
+ <a href="postscreen.8.html"><b>postscreen</b>(8)</a>'s built-in SMTP protocol engine. When these
+ tests succeed, <a href="postscreen.8.html"><b>postscreen</b>(8)</a> adds the client to the tempo-
+ rary whitelist but it cannot not hand off the "live" con-
+ nection to a Postfix SMTP server process in the middle of
+ a session. Instead, <a href="postscreen.8.html"><b>postscreen</b>(8)</a> defers attempts to
deliver mail with a 4XX status, and waits for the client
to disconnect. The next time a good client connects, it
will be allowed to talk to a Postfix SMTP server process
- to deliver mail.
+ to deliver mail. <a href="postscreen.8.html"><b>postscreen</b>(8)</a> mitigates the impact of
+ this limitation by giving such tests a long expiration
+ time.
+
+ The <a href="postscreen.8.html"><b>postscreen</b>(8)</a> built-in SMTP protocol engine does not
+ announce support for STARTTLS, AUTH, XCLIENT or XFORWARD
+ (support for STARTTLS and AUTH may be added in the
+ future). End-user clients should connect directly to the
+ submission service; other systems that require the above
+ features should directly connect to a Postfix SMTP server,
+ or they should be placed on the <a href="postscreen.8.html"><b>postscreen</b>(8)</a> whitelist.
<b>CONFIGURATION PARAMETERS</b>
Changes to <a href="postconf.5.html">main.cf</a> are not picked up automatically, as
The text below provides only a parameter summary. See
<a href="postconf.5.html"><b>postconf</b>(5)</a> for more details including examples.
- NOTE: Some parameters implement stress-dependent behavior.
- This is supported only when the default value is stress-
- dependent (${stress?X}${stress:Y}). Other parameters
- always evaluate as if the stress value is the empty
- string.
+ NOTE: Some <a href="postscreen.8.html"><b>postscreen</b>(8)</a> parameters implement stress-
+ dependent behavior. This is supported only when the
+ default value is stress-dependent (that is, it looks like
+ ${stress?X}${stress:Y}). Other parameters always evaluate
+ as if the stress value is the empty string.
<b>TRIAGE PARAMETERS</b>
<b><a href="postconf.5.html#postscreen_bare_newline_action">postscreen_bare_newline_action</a> (ignore)</b>
The Secure Mailer license must be distributed with this
software.
+<b>HISTORY</b>
+ Many ideas in <a href="postscreen.8.html"><b>postscreen</b>(8)</a> were explored in earlier work
+ by Michael Tokarev, in OpenBSD spamd, and in MailChannels
+ Traffic Control.
+
<b>AUTHOR(S)</b>
Wietse Venema
IBM T.J. Watson Research
server processes, more Postfix SMTP server processes remain
available for legitimate clients.
-\fBpostscreen\fR(8) maintains a temporary whitelist of
-positive decisions. Once an SMTP client is whitelisted, it
-is immediately forwarded to a real Postfix SMTP server
-process without further checking.
+\fBpostscreen\fR(8) maintains a temporary whitelist for
+clients that have passed a number of tests. When an SMTP
+client IP address is whitelisted, \fBpostscreen\fR(8) hands
+off the connection immediately to a Postfix SMTP server
+process. This minimizes the overhead for legitimate mail.
-By default, the program logs only statistics, and it does
-not run any tests against clients in mynetworks (primarily,
+By default, \fBpostscreen\fR(8) logs statistics and hands
+off every connection to a Postfix SMTP server process, while
+excluding clients in mynetworks from all tests (primarily,
to avoid problems with non-standard SMTP implementations
-in network appliances and test programs). \fBpostscreen\fR(8)
-by default forwards all connections to a real SMTP server
-process. This mode is useful for non-destructive testing.
+in network appliances). This mode is useful for non-destructive
+testing.
In a typical production setting, \fBpostscreen\fR(8) is
configured to reject mail from clients that fail one or
-more tests, after logging the helo, sender and recipient
-information.
+more tests. \fBpostscreen\fR(8) logs rejected mail with the
+client address, helo, sender and recipient information.
\fBpostscreen\fR(8) is not an SMTP proxy; this is intentional.
The purpose is to keep spambots away from Postfix SMTP
-server processes, not to control traffic flows.
+server processes, while minimizing overhead for legitimate
+traffic.
.SH "SECURITY"
.na
.nf
.SH BUGS
.ad
.fi
-When successful tests involve \fBpostscreen\fR(8)'s built-in
-SMTP protocol engine, \fBpostscreen\fR(8) adds the client
+Some of the non-default protocol tests involve
+\fBpostscreen\fR(8)'s built-in SMTP protocol engine. When
+these tests succeed, \fBpostscreen\fR(8) adds the client
to the temporary whitelist but it cannot not hand off the
-"live" connection from a good SMTP client to a Postfix SMTP
-server process. Instead, \fBpostscreen\fR(8) defers attempts
-to deliver mail with a 4XX status, and waits for the client
-to disconnect. The next time a good client connects, it
-will be allowed to talk to a Postfix SMTP server process
-to deliver mail.
+"live" connection to a Postfix SMTP server process in the
+middle of a session. Instead, \fBpostscreen\fR(8) defers
+attempts to deliver mail with a 4XX status, and waits for
+the client to disconnect. The next time a good client
+connects, it will be allowed to talk to a Postfix SMTP
+server process to deliver mail. \fBpostscreen\fR(8) mitigates
+the impact of this limitation by giving such tests a long
+expiration time.
+
+The \fBpostscreen\fR(8) built-in SMTP protocol engine does
+not announce support for STARTTLS, AUTH, XCLIENT or XFORWARD
+(support for STARTTLS and AUTH may be added in the future).
+End-user clients should connect directly to the submission
+service; other systems that require the above features
+should directly connect to a Postfix SMTP server, or they
+should be placed on the \fBpostscreen\fR(8) whitelist.
.SH "CONFIGURATION PARAMETERS"
.na
.nf
The text below provides only a parameter summary. See
\fBpostconf\fR(5) for more details including examples.
-NOTE: Some parameters implement stress-dependent behavior.
-This is supported only when the default value is stress-dependent
-(${stress?X}${stress:Y}). Other parameters always evaluate
+NOTE: Some \fBpostscreen\fR(8) parameters implement
+stress-dependent behavior. This is supported only when the
+default value is stress-dependent (that is, it looks like
+${stress?X}${stress:Y}). Other parameters always evaluate
as if the stress value is the empty string.
.SH "TRIAGE PARAMETERS"
.na
.ad
.fi
The Secure Mailer license must be distributed with this software.
+.SH "HISTORY"
+.na
+.nf
+.ad
+.fi
+Many ideas in \fBpostscreen\fR(8) were explored in earlier
+work by Michael Tokarev, in OpenBSD spamd, and in MailChannels
+Traffic Control.
.SH "AUTHOR(S)"
.na
.nf
already cached before the Postfix SMTP server looks them up later.
</p>
-<p> postscreen(8) maintains a temporary whitelist of positive
-decisions. Once an SMTP client is whitelisted, it is immediately
-forwarded to a real Postfix SMTP server process without further
-checking. </p>
-
-<p> By default, the program logs only statistics, and it does not
-run any checks on clients in mynetworks (primarily, to avoid problems
-with buggy SMTP implementations in network appliances). </p>
-
-<p> Many of the ideas in postscreen(8) have been explored in earlier
-work by Michael Tokarev, in OpenBSD spamd, and in MailChannels
-Traffic Control. </p>
+<p> postscreen(8) maintains a temporary whitelist for clients that
+have passed a number of tests. When an SMTP client IP address is
+whitelisted, postscreen(8) hands off the connection immediately to
+a Postfix SMTP server process. This minimizes the overhead for
+legitimate mail. </p>
+
+<p> By default, postscreen(8) logs statistics and hands off every
+connection to a Postfix SMTP server process, while excluding clients
+in mynetworks from all tests (primarily, to avoid problems with
+non-standard SMTP implementations in network appliances). This mode
+is useful for non-destructive testing. </p>
<p> Topics in this document: </p>
<li> <a href="#config"> Configuring the postscreen(8) service</a>
+<li> <a href="#historical"> Historical notes and credits </a>
+
</ul>
<h2> <a name="basic">The basic idea behind postscreen(8)</a> </h2>
<p> Spambots have a limited amount of time to send out spam before
they become blacklisted. For this reason, spambots make compromises
in their SMTP protocol implementation to speed up spam deliveries.
-For example, they speak before their turn. </p>
+For example, they speak before their turn, or they ignore responses
+from SMTP servers. </p>
-<p> Many spambots avoid spamming the same site repeatedly. Thus,
-postscreen(8) must make a long-term decision after a single
-measurement. For example, allow a good client to skip the DNSBL
-test for 24 hours. </p>
+<p> Many spambots avoid spamming the same site repeatedly, in an
+attempt to fly under the radar. Thus, postscreen(8) must make a
+long-term decision after a single measurement. For example, allow
+a good client to skip the "<a href="#pregreet">pregreet</a>" test
+for 24 hours. </p>
<p> To recognize spambots, postscreen(8) measures properties of the
client IP address and of the client SMTP protocol implementation
tests or 1 week for complex tests. This minimizes the impact of
the tests on legitimate mail clients. </p>
-<p> After logging the result of its tests, postscreen(8) by default
-forwards all connections to a real SMTP server process. This mode
-is useful for non-destructive testing. </p>
+<p> After logging its findings, postscreen(8) by default hands off
+all connections to a Postfix SMTP server process. This mode is
+useful for non-destructive testing. </p>
<p> In a typical production setting, postscreen(8) is configured
to reject mail from clients that fail one or more tests, after
-logging the sender and recipient information. </p>
+logging the helo, sender and recipient information. </p>
<p> Note: postscreen(8) is not an SMTP proxy; this is intentional.
-The purpose is to prioritize legitimate clients with as little
-overhead as possible. </p>
+The purpose is to keep spambots away from Postfix, with minimal
+overhead for legitimate clients. </p>
<h2> <a name="quick">Quick tests before everything else</a> </h2>
<b>WHITELISTED</b> <i>address</i>
</pre>
-<p> The action is not configurable: immediately forward the
-connection to a real SMTP server process. </p>
+<p> The action is not configurable: immediately hand off the
+connection to a Postfix SMTP server process. </p>
<h3> <a name="perm_black"> Permanent blacklist test </a> </h3>
<b>PASS OLD</b> <i>address</i>
</pre>
-<p> The action is not configurable: immediately forward the
-connection to a real SMTP server process. The client is
+<p> The action is not configurable: immediately hand off the
+connection to a Postfix SMTP server process. The client is
excluded from further tests until its temporary whitelist
entry expires, as controlled with the postscreen_*_ttl
parameters. Expired entries are silently renewed if possible. </p>
<p> When a good client passes these tests, and no "<a
href="#after_220">deep protocol tests</a>" are configured, postscreen(8)
-adds the client to the temporary whitelist and passes the "live"
+adds the client to the temporary whitelist and hands off the "live"
connection to a Postfix SMTP server process. The client can then
continue as if postscreen(8) never even existed (except of course
for the short postscreen_greet_wait delay). </p>
<blockquote>
<p>
-CAUTION: when postscreen rejects mail, it replies with the DNSBL
-domain name. Use the postscreen_dnsbl_reply_map feature to hide
-"password" information in DNSBL domain names.
+CAUTION: when postscreen rejects mail, it's SMTP reply contains the
+DNSBL domain name. Use the postscreen_dnsbl_reply_map feature to
+hide "password" information in DNSBL domain names.
</p>
</blockquote>
<h2> <a name="after_220">Tests after the 220 SMTP server greeting</a> </h2>
-<p> The tests in this phase use an SMTP protocol engine that is
-built into the postscreen(8) server. </p>
+<p> In this phase of the protocol, postscreen(8) implements a
+number of "deep protocol" tests. These tests use an SMTP protocol
+engine that is built into the postscreen(8) server. </p>
-<p> Important notes: </p>
+<p> Important note: deep protocol tests are disabled by default.
+They are more intrusive than the pregreet and DNSBL tests, and they
+have limitations as discussed next. </p>
<ul>
-<li> <p> These tests are disabled by default, because they
-are more intrusive than the pregreet and DNSBL tests. </p>
-
-<p> When a good client passes the <a href="#after_220">deep
-protocol tests </a>, postscreen(8) adds the client to the temporary
-whitelist but it cannot pass the "live" connection to a Postfix
+<li> <p> When a good client passes the <a href="#after_220">deep
+protocol tests</a>, postscreen(8) adds the client to the temporary
+whitelist but it cannot hand off the "live" connection to a Postfix
SMTP server process in the middle of the session. Instead, postscreen(8)
defers mail delivery attempts with a 4XX status, logs the
helo/sender/recipient information, and waits for the client to
disconnect. </p>
<p> The next time the client connects it will be allowed to talk
-to a real SMTP server process to deliver its mail. </p>
-
-<p> To minimize the impact of these tests, postscreen(8) gives them
-relatively long expiration times. </p>
+to a Postfix SMTP server process to deliver its mail. To minimize the
+impact of this limitation, postscreen(8) gives deep protocol tests
+a relatively long expiration time. </p>
-<li> <p> postscreen(8) does not implement the AUTH, STARTTLS,
-XCLIENT, and XFORWARD features. STARTTLS support may be added in
-a future version. </p>
+<li> <p> postscreen(8)'s built-in SMTP engine does not implement
+the AUTH, STARTTLS, XCLIENT, and XFORWARD features. STARTTLS and
+AUTH support may be added in a future version. </p>
</ul>
<p> By default, SMTP is a half-duplex protocol: the sender and
receiver send one command and one response at a time. Unlike the
-real Postfix SMTP server, postscreen(8) does not announce support
+Postfix SMTP server, postscreen(8) does not announce support
for ESMTP command pipelining. Therefore, clients are not allowed
-to send multiple commands. This test is disabled by default. </p>
+to send multiple commands. postscreen(8)'s <a href="#after_220">deep
+protocol test</a> for this is disabled by default. </p>
<p> With "postscreen_pipelining_enable = yes", postscreen(8) detects
spambots that send multiple commands, instead of sending one command
-and waiting for the server to reply. </p>
+and waiting for the server to reply. </p>
-<p> This test is opportunistically enabled when enabled when
-postscreen(8) has to use the built-in SMTP engine anyway, to make
-postscreen(8) logging more informative. </p>
+<p> This test is opportunistically enabled when postscreen(8) has
+to use the built-in SMTP engine anyway. This is to make postscreen(8)
+logging more informative. </p>
<p> When a client sends multiple commands, postscreen(8) logs this
as: </p>
-<b>COMMAND PIPELINING after</b> <i>time</i> <b>from</b> <i>address</i>
+<pre>
+ <b>COMMAND PIPELINING after</b> <i>time</i> <b>from</b> <i>address</i>
+</pre>
<p> Translation: the SMTP client at <i>address</i> sent multiple
SMTP commands, instead of sending one command and then waiting for
<h3> <a name="non_smtp">Non-SMTP command test</a> </h3>
+<p> Some spambots send their mail through open proxies. A symptom
+of this is the usage of commands such as CONNECT and other non-SMTP
+commands. Just like the Postfix SMTP server's smtpd_forbidden_commands
+feature, postscreen(8) has an equivalent postscreen_forbidden_commands
+feature to block these clients. postscreen(8)'s <a href="#after_220">deep
+protocol test</a> for this is disabled by default. </p>
+
<p> With "postscreen_non_smtp_command_enable = yes", postscreen(8)
-detects spambots that send non-SMTP commands, such as commands
-specified with the postscreen_forbidden_commands parameter, and
-commands that have the syntax of a message header label. </p>
+detects spambots that send commands specified with the
+postscreen_forbidden_commands parameter. This also detects commands
+with the syntax of a message header label. The latter is a symptom
+that the client is sending message content after ignoring all the
+responses from postscreen(8) that reject mail. </p>
-<p> This test is disabled by default. The test is opportunistically
-enabled when postscreen(8) has to use the built-in SMTP engine
-anyway, to make postscreen(8) logging more informative. </p>
+<p> This test is opportunistically enabled when postscreen(8) has
+to use the built-in SMTP engine anyway. This is to make postscreen(8)
+logging more informative. </p>
<p> When a client sends non-SMTP commands, postscreen(8) logs this
as: </p>
<h3> <a name="barelf">Bare newline test</a> </h3>
-<p> SMTP is a line-oriented protocol: lines have a limited
-length, and are terminated with <CR><LF>. </p>
+<p> SMTP is a line-oriented protocol: lines have a limited length,
+and are terminated with <CR><LF>. Lines ending in a
+"bare" <LF>, that is newline not preceded by carriage return,
+are not allowed in SMTP. postscreen(8)'s <a href="#after_220">deep
+protocol test</a> for this is disabled by default. </p>
<p> With "postscreen_bare_newline_enable = yes", postscreen(8)
-detects spambots that send lines ending in bare newline
-characters, that is newline not preceded by carriage return. </p>
+detects clients that send lines ending in bare newline characters.
+</p>
-<p> This test is disabled by default. The test is opportunistically
-enabled when postscreen(8) has to use the built-in SMTP engine
-anyway, to make postscreen(8) logging more informative. </p>
+<p> This test is opportunistically enabled when postscreen(8) has
+to use the built-in SMTP engine anyway. This is to make postscreen(8)
+logging more informative. </p>
<p> When a client sends bare newline characters, postscreen(8) logs
this as:
address from further tests until the temporary whitelist entry
expires, as controlled with the postscreen_*_ttl parameters. </p>
-<p> When no "<a href="#after_220">deep procol tests</a>" are
+<p> When no "<a href="#after_220">deep protocol tests</a>" are
configured, postscreen(8) passes the "live" connection to a Postfix
SMTP server process. The client can then continue as if postscreen(8)
never even existed (except for the short postscreen_greet_wait delay).
</p>
-<p> When any "<a href="#after_220">deep procol tests</a>" are
-configured, postscreen(8) cannot pass the "live" connection to a
-Postfix SMTP server process. Instead, postscreen(8) defers mail
-delivery attempts with a 4XX status, logs the helo/sender/recipient
-information, and waits for the client to disconnect. The next time
-the client connects it will be allowed to talk to a Postfix SMTP
-server process to deliver its mail. </p>
+<p> When any "<a href="#after_220">deep protocol tests</a>" are
+configured, postscreen(8) cannot hand off the "live" connection to
+a Postfix SMTP server process in the middle of the session. Instead,
+postscreen(8) defers mail delivery attempts with a 4XX status, logs
+the helo/sender/recipient information, and waits for the client to
+disconnect. The next time the client connects it will be allowed
+to talk to a Postfix SMTP server process to deliver its mail.
+postscreen(8) mitigates the impact of this limitation by giving
+<a href="#after_220">deep protocol tests</a> a long expiration
+time. </p>
<h2> <a name="config"> Configuring the postscreen(8) service</a>
</h2>
<ul>
+<li> <p> Some postscreen(8) configuration parameters implement
+stress-dependent behavior. This is supported only when the default
+value is stress-dependent (that is, it looks like ${stress?X}${stress:Y}).
+Other parameters always evaluate as if the stress value is the empty
+string. </p>
+
<li> <p> See "<a href="#before_220">Tests before the 220 SMTP server
greeting</a>" for details about the logging from these postscreen(8)
tests. </p>
<p> When a good client passes the "<a href="#after_220">deep
protocol tests</a>", postscreen(8) adds the client to the temporary
-whitelist but it cannot pass the "live" connection to a Postfix
+whitelist but it cannot hand off the "live" connection to a Postfix
SMTP server process in the middle of the session. Instead, postscreen(8)
defers mail delivery attempts with a 4XX status, logs the
helo/sender/recipient information, and waits for the client to
disconnect. </p>
-<p> When the client comes back in a later session, it is allowed
+<p> When the good client comes back in a later session, it is allowed
to talk directly to a Postfix SMTP server. See "after_220 <a
href="#after_220">Tests after the 220 SMTP server greeting</a> above
for limitations with STARTTLS, AUTH and other features that clients
-may need. Wietse enables "<a href="#after_220">deep protocol
-tests</a>" on his own internet-facing mail server. </p>
+may need. </p>
+
+<p> An unexpected benefit from "<a href="#after_220">deep protocol
+tests</a>" is that some "good" clients don't return after the 4XX
+reply; these clients were not so good after all. Wietse enables
+"<a href="#after_220">deep protocol tests</a>" on his own internet-facing
+mail server. </p>
<li> <p> There is also support for permanent blacklists and whitelists;
see the description of the postscreen_whitelist_networks and
</ol>
+<h2> <a name="historical"> Historical notes and credits </a> </h2>
+
+<p> Many ideas in postscreen(8) were explored in earlier work by
+Michael Tokarev, in OpenBSD spamd, and in MailChannels Traffic
+Control. </p>
+
+<p> Wietse threw together a crude prototype with pregreet and dnsbl
+support in June 2009, because he needed something new for a Mailserver
+conference presentation in July. Ralf Hildebrandt ran this code on
+several servers to collect real-world evidence. This version used
+the dnsblog(8) ad-hoc DNS client program. </p>
+
+<p> Wietse needed new material for a LISA conference presentation
+in November 2010, so he added support for DNSBL weights and filters
+in August, followed by a major code rewrite, deep protocol tests,
+helo/sender/recipient logging, and stress-adaptive behavior in
+September. Ralf Hildebrandt ran this code on several servers to
+collect real-world evidence. This version still used the same delay
+for pregreet and DNBL tests, as well as the embarrassing dnsblog(8)
+ad-hoc DNS client. </p>
+
</body>
</html>
* Patches change both the patchlevel and the release date. Snapshots have no
* patchlevel; they change the release date only.
*/
-#define MAIL_RELEASE_DATE "20100912"
+#define MAIL_RELEASE_DATE "20100913"
#define MAIL_VERSION_NUMBER "2.8"
#ifdef SNAPSHOT
/* server processes, more Postfix SMTP server processes remain
/* available for legitimate clients.
/*
-/* \fBpostscreen\fR(8) maintains a temporary whitelist of
-/* positive decisions. Once an SMTP client is whitelisted, it
-/* is immediately forwarded to a real Postfix SMTP server
-/* process without further checking.
+/* \fBpostscreen\fR(8) maintains a temporary whitelist for
+/* clients that have passed a number of tests. When an SMTP
+/* client IP address is whitelisted, \fBpostscreen\fR(8) hands
+/* off the connection immediately to a Postfix SMTP server
+/* process. This minimizes the overhead for legitimate mail.
/*
-/* By default, the program logs only statistics, and it does
-/* not run any tests against clients in mynetworks (primarily,
+/* By default, \fBpostscreen\fR(8) logs statistics and hands
+/* off every connection to a Postfix SMTP server process, while
+/* excluding clients in mynetworks from all tests (primarily,
/* to avoid problems with non-standard SMTP implementations
-/* in network appliances and test programs). \fBpostscreen\fR(8)
-/* by default forwards all connections to a real SMTP server
-/* process. This mode is useful for non-destructive testing.
+/* in network appliances). This mode is useful for non-destructive
+/* testing.
/*
/* In a typical production setting, \fBpostscreen\fR(8) is
/* configured to reject mail from clients that fail one or
-/* more tests, after logging the helo, sender and recipient
-/* information.
+/* more tests. \fBpostscreen\fR(8) logs rejected mail with the
+/* client address, helo, sender and recipient information.
/*
/* \fBpostscreen\fR(8) is not an SMTP proxy; this is intentional.
/* The purpose is to keep spambots away from Postfix SMTP
-/* server processes, not to control traffic flows.
+/* server processes, while minimizing overhead for legitimate
+/* traffic.
/* SECURITY
/* .ad
/* .fi
/* DIAGNOSTICS
/* Problems and transactions are logged to \fBsyslogd\fR(8).
/* BUGS
-/* When successful tests involve \fBpostscreen\fR(8)'s built-in
-/* SMTP protocol engine, \fBpostscreen\fR(8) adds the client
+/* Some of the non-default protocol tests involve
+/* \fBpostscreen\fR(8)'s built-in SMTP protocol engine. When
+/* these tests succeed, \fBpostscreen\fR(8) adds the client
/* to the temporary whitelist but it cannot not hand off the
-/* "live" connection from a good SMTP client to a Postfix SMTP
-/* server process. Instead, \fBpostscreen\fR(8) defers attempts
-/* to deliver mail with a 4XX status, and waits for the client
-/* to disconnect. The next time a good client connects, it
-/* will be allowed to talk to a Postfix SMTP server process
-/* to deliver mail.
+/* "live" connection to a Postfix SMTP server process in the
+/* middle of a session. Instead, \fBpostscreen\fR(8) defers
+/* attempts to deliver mail with a 4XX status, and waits for
+/* the client to disconnect. The next time a good client
+/* connects, it will be allowed to talk to a Postfix SMTP
+/* server process to deliver mail. \fBpostscreen\fR(8) mitigates
+/* the impact of this limitation by giving such tests a long
+/* expiration time.
+/*
+/* The \fBpostscreen\fR(8) built-in SMTP protocol engine does
+/* not announce support for STARTTLS, AUTH, XCLIENT or XFORWARD
+/* (support for STARTTLS and AUTH may be added in the future).
+/* End-user clients should connect directly to the submission
+/* service; other systems that require the above features
+/* should directly connect to a Postfix SMTP server, or they
+/* should be placed on the \fBpostscreen\fR(8) whitelist.
/* CONFIGURATION PARAMETERS
/* .ad
/* .fi
/* The text below provides only a parameter summary. See
/* \fBpostconf\fR(5) for more details including examples.
/*
-/* NOTE: Some parameters implement stress-dependent behavior.
-/* This is supported only when the default value is stress-dependent
-/* (${stress?X}${stress:Y}). Other parameters always evaluate
+/* NOTE: Some \fBpostscreen\fR(8) parameters implement
+/* stress-dependent behavior. This is supported only when the
+/* default value is stress-dependent (that is, it looks like
+/* ${stress?X}${stress:Y}). Other parameters always evaluate
/* as if the stress value is the empty string.
/* TRIAGE PARAMETERS
/* .ad
/* .ad
/* .fi
/* The Secure Mailer license must be distributed with this software.
+/* HISTORY
+/* .ad
+/* .fi
+/* Many ideas in \fBpostscreen\fR(8) were explored in earlier
+/* work by Michael Tokarev, in OpenBSD spamd, and in MailChannels
+/* Traffic Control.
/* AUTHOR(S)
/* Wietse Venema
/* IBM T.J. Watson Research
projects / thirdparty / postfix.git / commitdiff
