From: Wietse Venema The Postfix postscreen(8) server performs triage on multiple
inbound SMTP connections in parallel. While a single postscreen(8)
-process keeps spambots away from Postfix SMTP server processes,
+process keeps zombies away from Postfix SMTP server processes,
more Postfix SMTP server processes remain available for legitimate
clients. By doing these checks in a single postscreen(8) process, Postfix
-can avoid wasting one SMTP server process per connection. A side
-benefit of postscreen(8)'s DNSBL lookups is that DNS records are
-already cached before the Postfix SMTP server looks them up later.
-
Topics in this document:
@@ -57,30 +56,39 @@ already cached before the Postfix SMTP server looks them up later.Spambots have a limited amount of time to send out spam before -they become blacklisted. For this reason, spambots make compromises -in their SMTP protocol implementation to speed up spam deliveries. -For example, they speak before their turn, or they ignore responses -from SMTP servers.
- -Many spambots avoid spamming the same site repeatedly, in an -attempt to fly under the radar. Thus, postscreen(8) must make a -long-term decision after a single measurement. For example, allow -a good client to skip the "pregreet" test -for 24 hours.
- -To recognize spambots, postscreen(8) measures properties of the -client IP address and of the client SMTP protocol implementation -(the protocol compromises that were made to speed up delivery). -These properties don't change with delivery attempts, and are -therefore suitable for making a long-term decision after a single -measurement.
- -postscreen(8) does not inspect message content. The reason is -that content can change with each delivery attempt, especially with -legitimate clients. Message content is not good for making a long-term -decision after a single measurement, and that is the problem that -postscreen(8) is focused on.
+Most email is spam, and most spam is sent out by zombies (malware +on compromised end-user computers). Wietse expects that the zombie +problem will get worse before things improve, if ever. Without a +tool like postscreen(8) that keeps the zombies away, Postfix would be +spending most of its resources not receiving email.
+ +The main challenge for postscreen(8) is to make an is-it-a-zombie +decision based on a single measurement. This is necessary because +many zombies avoid spamming the same site repeatedly, in an attempt +to fly under the radar. Once postscreen(8) decides that a client +is not-a-zombie, it whitelists the client temporarily to avoid +further delays for legitimate mail.
+ +Zombies have challenges too: they have only a limited amount +of time to deliver spam before their IP address becomes blacklisted. +To speed up spam deliveries, zombies make compromises in their SMTP +protocol implementation. For example, they speak before their turn, +or they ignore responses from SMTP servers and continue sending +mail even when the server tells them to go away.
+ +postscreen(8) uses a variety of measurements to recognize +zombies. First, postscreen(8) determines if the remote SMTP client +IP address is blacklisted. Second, postscreen(8) looks for protocol +compromises that are made to speed up delivery. The results of +such measurements don't change with each delivery attempt, and are +therefore good for making an is-it-a-zombie decision based on a +single measurement.
+ +postscreen(8) does not inspect message content. Message content +can vary widely with each delivery attempt, especially with clients +that (also) send legitimate email. Content is therefore not good +for making an is-it-a-zombie decision based on a single measurement, +and that is the problem that postscreen(8) is focused on.
Note: postscreen(8) is not an SMTP proxy; this is intentional. -The purpose is to keep spambots away from Postfix, with minimal +The purpose is to keep zombies away from Postfix, with minimal overhead for legitimate clients.
The SMTP protocol is a classic example of a protocol where the -server speaks before the client. postscreen(8) detects spambots +server speaks before the client. postscreen(8) detects zombies that are in a hurry and that speak before their turn. This test is enabled by default.
@@ -205,7 +213,7 @@ portion of a "220-text..." teaser banner (default: $postscreen(8) daemon sends this before the postscreen_greet_wait timer is started. The purpose of the teaser banner is to confuse -spambots so that they speak before their turn. It has no effect on +zombies so that they speak before their turn. It has no effect on SMTP clients that correctly implement the protocol.To avoid problems with poorly-implemented SMTP engines in network @@ -262,7 +270,9 @@ hide "password" information in DNSBL domain names. DNSBL score is equal to or greater than the postscreen_dnsbl_threshold parameter value, postscreen(8) logs this as:
-DNSBL rank count for address ++ DNSBL rank count for address +
Translation: the SMTP client at address has a combined DNSBL score of count.
@@ -359,7 +369,7 @@ to send multiple commands. postscreen(8)'s deep protocol test for this is disabled by default.With "postscreen_pipelining_enable = yes", postscreen(8) detects -spambots that send multiple commands, instead of sending one command +zombies that send multiple commands, instead of sending one command and waiting for the server to reply.
This test is opportunistically enabled when postscreen(8) has @@ -392,7 +402,7 @@ feature to block these clients. postscreen(8)'s deep protocol test for this is disabled by default.
With "postscreen_non_smtp_command_enable = yes", postscreen(8) -detects spambots that send commands specified with the +detects zombies that send commands specified with the postscreen_forbidden_commands parameter. This also detects commands with the syntax of a message header label. The latter is a symptom that the client is sending message content after ignoring all the @@ -695,7 +705,7 @@ Postfix SMTP servers dramatically.
clients that talk before their turn, and to log the helo/sender/recipient information. This stops over half of all known-to-be illegitimate connections to Wietse's mail server. It is backup protection for -spambots that haven't yet been blacklisted. +zombies that haven't yet been blacklisted.You can also enable "deep protocol tests", but these are more intrusive than the pregreet or DNSBL diff --git a/postfix/proto/POSTSCREEN_README.html b/postfix/proto/POSTSCREEN_README.html index 40d9be062..90eeeeaed 100644 --- a/postfix/proto/POSTSCREEN_README.html +++ b/postfix/proto/POSTSCREEN_README.html @@ -19,15 +19,14 @@
The Postfix postscreen(8) server performs triage on multiple inbound SMTP connections in parallel. While a single postscreen(8) -process keeps spambots away from Postfix SMTP server processes, +process keeps zombies away from Postfix SMTP server processes, more Postfix SMTP server processes remain available for legitimate clients.
By doing these checks in a single postscreen(8) process, Postfix -can avoid wasting one SMTP server process per connection. A side -benefit of postscreen(8)'s DNSBL lookups is that DNS records are -already cached before the Postfix SMTP server looks them up later. -
+can avoid wasting one SMTP server process per zombie. A side benefit +of postscreen(8)'s DNSBL lookups is that DNS records will already be +cached before the Postfix SMTP server looks them up later.Topics in this document:
@@ -57,30 +56,39 @@ already cached before the Postfix SMTP server looks them up later.Spambots have a limited amount of time to send out spam before -they become blacklisted. For this reason, spambots make compromises -in their SMTP protocol implementation to speed up spam deliveries. -For example, they speak before their turn, or they ignore responses -from SMTP servers.
- -Many spambots avoid spamming the same site repeatedly, in an -attempt to fly under the radar. Thus, postscreen(8) must make a -long-term decision after a single measurement. For example, allow -a good client to skip the "pregreet" test -for 24 hours.
- -To recognize spambots, postscreen(8) measures properties of the -client IP address and of the client SMTP protocol implementation -(the protocol compromises that were made to speed up delivery). -These properties don't change with delivery attempts, and are -therefore suitable for making a long-term decision after a single -measurement.
- -postscreen(8) does not inspect message content. The reason is -that content can change with each delivery attempt, especially with -legitimate clients. Message content is not good for making a long-term -decision after a single measurement, and that is the problem that -postscreen(8) is focused on.
+Most email is spam, and most spam is sent out by zombies (malware +on compromised end-user computers). Wietse expects that the zombie +problem will get worse before things improve, if ever. Without a +tool like postscreen(8) that keeps the zombies away, Postfix would be +spending most of its resources not receiving email.
+ +The main challenge for postscreen(8) is to make an is-it-a-zombie +decision based on a single measurement. This is necessary because +many zombies avoid spamming the same site repeatedly, in an attempt +to fly under the radar. Once postscreen(8) decides that a client +is not-a-zombie, it whitelists the client temporarily to avoid +further delays for legitimate mail.
+ +Zombies have challenges too: they have only a limited amount +of time to deliver spam before their IP address becomes blacklisted. +To speed up spam deliveries, zombies make compromises in their SMTP +protocol implementation. For example, they speak before their turn, +or they ignore responses from SMTP servers and continue sending +mail even when the server tells them to go away.
+ +postscreen(8) uses a variety of measurements to recognize +zombies. First, postscreen(8) determines if the remote SMTP client +IP address is blacklisted. Second, postscreen(8) looks for protocol +compromises that are made to speed up delivery. The results of +such measurements don't change with each delivery attempt, and are +therefore good for making an is-it-a-zombie decision based on a +single measurement.
+ +postscreen(8) does not inspect message content. Message content +can vary widely with each delivery attempt, especially with clients +that (also) send legitimate email. Content is therefore not good +for making an is-it-a-zombie decision based on a single measurement, +and that is the problem that postscreen(8) is focused on.
Note: postscreen(8) is not an SMTP proxy; this is intentional. -The purpose is to keep spambots away from Postfix, with minimal +The purpose is to keep zombies away from Postfix, with minimal overhead for legitimate clients.
The SMTP protocol is a classic example of a protocol where the -server speaks before the client. postscreen(8) detects spambots +server speaks before the client. postscreen(8) detects zombies that are in a hurry and that speak before their turn. This test is enabled by default.
@@ -205,7 +213,7 @@ portion of a "220-text..." teaser banner (default: $smtpd_banner). Note that this becomes the first part of a multi-line server greeting. The postscreen(8) daemon sends this before the postscreen_greet_wait timer is started. The purpose of the teaser banner is to confuse -spambots so that they speak before their turn. It has no effect on +zombies so that they speak before their turn. It has no effect on SMTP clients that correctly implement the protocol.To avoid problems with poorly-implemented SMTP engines in network @@ -262,7 +270,9 @@ hide "password" information in DNSBL domain names. DNSBL score is equal to or greater than the postscreen_dnsbl_threshold parameter value, postscreen(8) logs this as:
-DNSBL rank count for address ++ DNSBL rank count for address +
Translation: the SMTP client at address has a combined DNSBL score of count.
@@ -359,7 +369,7 @@ to send multiple commands. postscreen(8)'s deep protocol test for this is disabled by default.With "postscreen_pipelining_enable = yes", postscreen(8) detects -spambots that send multiple commands, instead of sending one command +zombies that send multiple commands, instead of sending one command and waiting for the server to reply.
This test is opportunistically enabled when postscreen(8) has @@ -392,7 +402,7 @@ feature to block these clients. postscreen(8)'s deep protocol test for this is disabled by default.
With "postscreen_non_smtp_command_enable = yes", postscreen(8) -detects spambots that send commands specified with the +detects zombies that send commands specified with the postscreen_forbidden_commands parameter. This also detects commands with the syntax of a message header label. The latter is a symptom that the client is sending message content after ignoring all the @@ -695,7 +705,7 @@ Postfix SMTP servers dramatically.
clients that talk before their turn, and to log the helo/sender/recipient information. This stops over half of all known-to-be illegitimate connections to Wietse's mail server. It is backup protection for -spambots that haven't yet been blacklisted. +zombies that haven't yet been blacklisted.You can also enable "deep protocol tests", but these are more intrusive than the pregreet or DNSBL diff --git a/postfix/src/global/mail_version.h b/postfix/src/global/mail_version.h index 1d8d21624..3a92662dc 100644 --- a/postfix/src/global/mail_version.h +++ b/postfix/src/global/mail_version.h @@ -20,7 +20,7 @@ * Patches change both the patchlevel and the release date. Snapshots have no * patchlevel; they change the release date only. */ -#define MAIL_RELEASE_DATE "20100918" +#define MAIL_RELEASE_DATE "20100923" #define MAIL_VERSION_NUMBER "2.8" #ifdef SNAPSHOT diff --git a/postfix/src/postscreen/postscreen.c b/postfix/src/postscreen/postscreen.c index d2671ee07..98e32443c 100644 --- a/postfix/src/postscreen/postscreen.c +++ b/postfix/src/postscreen/postscreen.c @@ -560,7 +560,7 @@ static void ps_service(VSTREAM *smtp_client_stream, /* Not: PS_PASS_SESSION_STATE. Repeat this test the next time. */ break; default: - msg_panic("%s: unknown pregreet action value %d", + msg_panic("%s: unknown blacklist action value %d", myname, ps_blist_action); } } @@ -580,7 +580,7 @@ static void ps_service(VSTREAM *smtp_client_stream, if (msg_verbose) msg_info("%s: cached + recent flags: %s", myname, ps_print_state_flags(state->flags, myname)); - if ((state->flags & PS_STATE_FLAG_ANY_TODO) == 0) { + if ((state->flags & PS_STATE_FLAG_ANY_TODO_FAIL) == 0) { msg_info("PASS OLD %s", state->smtp_client_addr); ps_conclude(state); return; diff --git a/postfix/src/postscreen/postscreen.h b/postfix/src/postscreen/postscreen.h index 4dc351c6a..4eb3adf64 100644 --- a/postfix/src/postscreen/postscreen.h +++ b/postfix/src/postscreen/postscreen.h @@ -186,6 +186,12 @@ typedef struct { #define PS_STATE_FLAG_ANY_TODO \ (PS_STATE_FLAG_EARLY_TODO | PS_STATE_FLAG_SMTPD_TODO) +#define PS_STATE_FLAG_ANY_TODO_FAIL \ + (PS_STATE_FLAG_ANY_TODO | PS_STATE_FLAG_ANY_FAIL) + +#define PS_STATE_FLAG_ANY_UPDATE \ + (PS_STATE_FLAG_ANY_PASS) + /* * See log_adhoc.c for discussion. */ @@ -383,6 +389,7 @@ extern void ps_dnsbl_request(const char *, void (*) (int, char *), char *); (dst)->pregr_stamp = PS_TIME_STAMP_INVALID; \ (dst)->dnsbl_stamp = PS_TIME_STAMP_INVALID; \ (dst)->pipel_stamp = PS_TIME_STAMP_INVALID; \ + (dst)->barlf_stamp = PS_TIME_STAMP_INVALID; \ } while (0) #define PS_BEGIN_TESTS(state, name) do { \ (state)->test_name = (name); \ @@ -411,7 +418,7 @@ extern void ps_smtpd_init(void); /* * postscreen_misc.c */ -extern char *ps_format_delta_time(VSTRING *, struct timeval, int *); +extern char *ps_format_delta_time(VSTRING *, struct timeval, DELTA_TIME *); extern void ps_conclude(PS_STATE *); extern void ps_hangup_event(PS_STATE *); diff --git a/postfix/src/postscreen/postscreen_early.c b/postfix/src/postscreen/postscreen_early.c index 4131fa60a..8fc8fade8 100644 --- a/postfix/src/postscreen/postscreen_early.c +++ b/postfix/src/postscreen/postscreen_early.c @@ -58,7 +58,7 @@ static void ps_early_event(int event, char *context) char read_buf[PS_READ_BUF_SIZE]; int read_count; int dnsbl_score; - int elapsed; + DELTA_TIME elapsed; const char *dnsbl_name; if (msg_verbose > 1) @@ -206,13 +206,13 @@ static void ps_early_event(int event, char *context) * EVENT_TIME, instead of calling ps_early_event recursively. */ state->flags |= PS_STATE_FLAG_PREGR_DONE; - if (elapsed >= PS_EFF_GREET_WAIT + if (elapsed.dt_sec >= PS_EFF_GREET_WAIT || ((state->flags & PS_STATE_FLAG_EARLY_DONE) == PS_STATE_FLAGS_TODO_TO_DONE(state->flags & PS_STATE_FLAG_EARLY_TODO))) ps_early_event(EVENT_TIME, context); else event_request_timer(ps_early_event, context, - PS_EFF_GREET_WAIT - elapsed); + PS_EFF_GREET_WAIT - elapsed.dt_sec); return; } } diff --git a/postfix/src/postscreen/postscreen_misc.c b/postfix/src/postscreen/postscreen_misc.c index de129cec2..443db5fee 100644 --- a/postfix/src/postscreen/postscreen_misc.c +++ b/postfix/src/postscreen/postscreen_misc.c @@ -9,7 +9,7 @@ /* char *ps_format_delta_time(buf, tv, delta) /* VSTRING *buf; /* struct timeval tv; -/* int *delta; +/* DELTA_TIME *delta; /* /* void ps_conclude(state) /* PS_STATE *state; @@ -66,7 +66,7 @@ /* ps_format_delta_time - pretty-formatted delta time */ -char *ps_format_delta_time(VSTRING *buf, struct timeval tv, int *delta) +char *ps_format_delta_time(VSTRING *buf, struct timeval tv, DELTA_TIME *delta) { DELTA_TIME pdelay; struct timeval now; @@ -75,7 +75,7 @@ char *ps_format_delta_time(VSTRING *buf, struct timeval tv, int *delta) PS_CALC_DELTA(pdelay, now, tv); VSTRING_RESET(buf); format_tv(buf, pdelay.dt_sec, pdelay.dt_usec, SIG_DIGS, var_delay_max_res); - *delta = pdelay.dt_sec; + *delta = pdelay; return (STR(buf)); } @@ -95,26 +95,26 @@ void ps_conclude(PS_STATE *state) * blacklisting. There may still be unfinished tests; those tests will * need to be completed when the client returns in a later session. */ - if ((state->flags & PS_STATE_FLAG_ANY_PASS) != 0 - && (state->flags & PS_STATE_FLAG_ANY_FAIL) == 0) { - - /* - * Log our final blessing when all unfinished tests were completed. - */ - if ((state->flags & PS_STATE_FLAG_ANY_PASS) == - PS_STATE_FLAGS_TODO_TO_PASS(state->flags & PS_STATE_FLAG_ANY_TODO)) - msg_info("PASS %s %s", (state->flags & PS_STATE_FLAG_NEW) == 0 ? - "OLD" : "NEW", state->smtp_client_addr); - - /* - * Update the postscreen cache. This still supports a scenario where - * a client gets whitelisted in the course of multiple sessions, as - * long as that client does not "fail" any test. - */ - if (ps_cache_map != 0) { - ps_print_tests(ps_temp, state); - ps_cache_update(ps_cache_map, state->smtp_client_addr, STR(ps_temp)); - } + if (state->flags & PS_STATE_FLAG_ANY_FAIL) + state->flags &= ~PS_STATE_FLAG_ANY_PASS; + + /* + * Log our final blessing when all unfinished tests were completed. + */ + if ((state->flags & PS_STATE_FLAG_ANY_PASS) == + PS_STATE_FLAGS_TODO_TO_PASS(state->flags & PS_STATE_FLAG_ANY_TODO)) + msg_info("PASS %s %s", (state->flags & PS_STATE_FLAG_NEW) == 0 ? + "OLD" : "NEW", state->smtp_client_addr); + + /* + * Update the postscreen cache. This still supports a scenario where a + * client gets whitelisted in the course of multiple sessions, as long as + * that client does not "fail" any test. + */ + if ((state->flags & PS_STATE_FLAG_ANY_UPDATE) != 0 + && ps_cache_map != 0) { + ps_print_tests(ps_temp, state); + ps_cache_update(ps_cache_map, state->smtp_client_addr, STR(ps_temp)); } /* @@ -123,7 +123,7 @@ void ps_conclude(PS_STATE *state) if ((state->flags & PS_STATE_FLAG_NOFORWARD) == 0) { ps_send_socket(state); } else { - if ((state->flags & PS_STATE_FLAG_HANGUP) == 0) + if ((state->flags & PS_STATE_FLAG_HANGUP) == 0) (void) ps_send_reply(vstream_fileno(state->smtp_client_stream), state->smtp_client_addr, state->smtp_client_port, state->final_reply); @@ -136,7 +136,7 @@ void ps_conclude(PS_STATE *state) void ps_hangup_event(PS_STATE *state) { - int elapsed; + DELTA_TIME elapsed; /* * Sessions can break at any time, even after the client passes all tests diff --git a/postfix/src/postscreen/postscreen_tests.c b/postfix/src/postscreen/postscreen_tests.c index a2baddabf..181f02a0b 100644 --- a/postfix/src/postscreen/postscreen_tests.c +++ b/postfix/src/postscreen/postscreen_tests.c @@ -191,11 +191,17 @@ void ps_parse_tests(PS_STATE *state, default: break; } - if ((state->pregr_stamp = pregr_stamp) == PS_TIME_STAMP_NEW - || (state->dnsbl_stamp = dnsbl_stamp) == PS_TIME_STAMP_NEW - || (state->pipel_stamp = pipel_stamp) == PS_TIME_STAMP_NEW - || (state->nsmtp_stamp = nsmtp_stamp) == PS_TIME_STAMP_NEW - || (state->barlf_stamp = barlf_stamp) == PS_TIME_STAMP_NEW) + state->pregr_stamp = pregr_stamp; + state->dnsbl_stamp = dnsbl_stamp; + state->pipel_stamp = pipel_stamp; + state->nsmtp_stamp = nsmtp_stamp; + state->barlf_stamp = barlf_stamp; + + if (pregr_stamp == PS_TIME_STAMP_NEW + || dnsbl_stamp == PS_TIME_STAMP_NEW + || pipel_stamp == PS_TIME_STAMP_NEW + || nsmtp_stamp == PS_TIME_STAMP_NEW + || barlf_stamp == PS_TIME_STAMP_NEW) state->flags |= PS_STATE_FLAG_NEW; /* @@ -258,8 +264,8 @@ char *ps_print_tests(VSTRING *buf, PS_STATE *state) /* * Sanity check. */ - if ((state->flags & PS_STATE_FLAG_ANY_PASS) == 0) - msg_panic("%s: attempt to save a no-pass record", myname); + if ((state->flags & PS_STATE_FLAG_ANY_UPDATE) == 0) + msg_panic("%s: attempt to save a no-update record", myname); /* * Give disabled tests a dummy time stamp so that we don't log a client