handling of out-of-range numbers. Files: global/cfg_parser.c,
global/conv_time.c, global/mail_conf_int.c,
global/mail_conf_long.c, global/mail_conf_nint.c.
+
+20101217
+
+ Cleanup: eliminated the code that copied TLS protocol
+ messages between the OpenSSL TLS engine and the network.
+ This change hopefully simplifies the TLS library enough
+ that it can be used in an event-driven TLS proxy in front
+ of postscreen. Files: tls/tls_bio.c, tls/tls_server.c,
+ tls/tls_client.c.
+
+ This change eliminates an obscure bug where the SMTP server
+ would wait for another $smtpd_timeout seconds after sending
+ the "421 Error: timeout exceeded" message to the client.
away from Postfix SMTP server processes, more Postfix SMTP server processes
remain available for legitimate clients.
-By doing these checks in a single postscreen(8) process, Postfix can avoid
-wasting one SMTP server process per zombie. A side benefit of postscreen(8)'s
-DNSBL lookups is that DNS records will already be cached before the Postfix
-SMTP server looks them up later.
+postscreen(8) is the first layer in a multi-layer defense.
+
+ * The postscreen(8) layer blocks connections from zombies and other spambots
+ that are responsible for about 90% of all spam. It is implemented as a
+ single process to make this defense as cheap as possible.
+
+ * The second layer implements more complex SMTP-level access checks that are
+ available with Postfix SMTP servers, policy daemons, and Milter
+ applications.
+
+ * The third layer performs light-weight content inspection with the Postfix
+ built-in header_checks and body_checks. This can block unacceptable
+ attachments such as executable programs, and worms or viruses with easy-to-
+ recognize signatures.
+
+ * The fourth layer provides heavy-weight content inspection with external
+ content filters. Typical examples are Amavisd-new, SpamAssassin, and Milter
+ applications.
+
+Each layer reduces the spam volume. The general strategy is to eliminate spam
+early with the less expensive defenses and to use the more expensive defenses
+for the spam that remains.
Topics in this document:
based on a single measurement. This is necessary because many zombies avoid
spamming the same site repeatedly, in an attempt to fly under the radar. Once
postscreen(8) decides that a client is not-a-zombie, it whitelists the client
-temporarily to avoid further delays for legitimate mail. Clients that pass
-postscreen(8) are still subject to the checks that are built into Postfix smtpd
-(8), Postfix built-in content filters, and external content filters.
+temporarily to avoid further delays for legitimate mail.
Zombies have challenges too: they have only a limited amount of time to deliver
spam before their IP address becomes blacklisted. To speed up spam deliveries,
more Postfix SMTP server processes remain available for legitimate
clients. </p>
-<p> By doing these checks in a single <a href="postscreen.8.html">postscreen(8)</a> process, Postfix
-can avoid wasting one SMTP server process per zombie. A side benefit
-of <a href="postscreen.8.html">postscreen(8)</a>'s DNSBL lookups is that DNS records will already be
-cached before the Postfix SMTP server looks them up later. </p>
+<p> <a href="postscreen.8.html">postscreen(8)</a> is the first layer in a multi-layer defense. <p>
+
+<ul>
+
+<li> <p> The <a href="postscreen.8.html">postscreen(8)</a> layer blocks connections from zombies
+and other spambots that are responsible for about 90% of all spam.
+It is implemented as a single process to make this defense as cheap
+as possible. </p>
+
+<li> <p> The second layer implements more complex SMTP-level access
+checks that are available with Postfix SMTP servers, policy daemons,
+and Milter applications. </p>
+
+<li> <p> The third layer performs light-weight content inspection
+with the Postfix built-in <a href="postconf.5.html#header_checks">header_checks</a> and <a href="postconf.5.html#body_checks">body_checks</a>. This can
+block unacceptable attachments such as executable programs, and
+worms or viruses with easy-to-recognize signatures. </p>
+
+<li> <p> The fourth layer provides heavy-weight content inspection
+with external content filters. Typical examples are Amavisd-new,
+SpamAssassin, and Milter applications. </p>
+
+</ul>
+
+<p> Each layer reduces the spam volume. The general strategy is to
+eliminate spam early with the less expensive defenses and to use
+the more expensive defenses for the spam that remains. </p>
<p> Topics in this document: </p>
many zombies avoid spamming the same site repeatedly, in an attempt
to fly under the radar. Once <a href="postscreen.8.html">postscreen(8)</a> decides that a client
is not-a-zombie, it whitelists the client temporarily to avoid
-further delays for legitimate mail. Clients that pass <a href="postscreen.8.html">postscreen(8)</a>
-are still subject to the checks that are built into Postfix <a href="smtpd.8.html">smtpd(8)</a>,
-Postfix built-in content filters, and external content filters.
-</p>
+further delays for legitimate mail. </p>
<p> Zombies have challenges too: they have only a limited amount
of time to deliver spam before their IP address becomes blacklisted.
more Postfix SMTP server processes remain available for legitimate
clients. </p>
-<p> By doing these checks in a single postscreen(8) process, Postfix
-can avoid wasting one SMTP server process per zombie. A side benefit
-of postscreen(8)'s DNSBL lookups is that DNS records will already be
-cached before the Postfix SMTP server looks them up later. </p>
+<p> postscreen(8) is the first layer in a multi-layer defense. <p>
+
+<ul>
+
+<li> <p> The postscreen(8) layer blocks connections from zombies
+and other spambots that are responsible for about 90% of all spam.
+It is implemented as a single process to make this defense as cheap
+as possible. </p>
+
+<li> <p> The second layer implements more complex SMTP-level access
+checks that are available with Postfix SMTP servers, policy daemons,
+and Milter applications. </p>
+
+<li> <p> The third layer performs light-weight content inspection
+with the Postfix built-in header_checks and body_checks. This can
+block unacceptable attachments such as executable programs, and
+worms or viruses with easy-to-recognize signatures. </p>
+
+<li> <p> The fourth layer provides heavy-weight content inspection
+with external content filters. Typical examples are Amavisd-new,
+SpamAssassin, and Milter applications. </p>
+
+</ul>
+
+<p> Each layer reduces the spam volume. The general strategy is to
+eliminate spam early with the less expensive defenses and to use
+the more expensive defenses for the spam that remains. </p>
<p> Topics in this document: </p>
many zombies avoid spamming the same site repeatedly, in an attempt
to fly under the radar. Once postscreen(8) decides that a client
is not-a-zombie, it whitelists the client temporarily to avoid
-further delays for legitimate mail. Clients that pass postscreen(8)
-are still subject to the checks that are built into Postfix smtpd(8),
-Postfix built-in content filters, and external content filters.
-</p>
+further delays for legitimate mail. </p>
<p> Zombies have challenges too: they have only a limited amount
of time to deliver spam before their IP address becomes blacklisted.
* Patches change both the patchlevel and the release date. Snapshots have no
* patchlevel; they change the release date only.
*/
-#define MAIL_RELEASE_DATE "20101210"
+#define MAIL_RELEASE_DATE "20101217"
#define MAIL_VERSION_NUMBER "2.8"
#ifdef SNAPSHOT
/*
* This program handles all incoming connections, so it must not block.
* We use event-driven code for all operations that introduce latency.
+ *
+ * Note: instead of using VSTREAM-level timeouts, we enforce limits on the
+ * total amount of time to receive a complete SMTP command line.
*/
non_blocking(vstream_fileno(smtp_client_stream), NON_BLOCKING);
/*
* Try to match the current character desired by the state
* machine. If that fails, try to restart the machine with a
- * match for its first state. smtpd(8) incompatibility: we
- * require that lines end in <CR><LF>, while smtpd(8) allows
- * lines ending in <CR><LF> and bare <LF>.
+ * match for its first state. Like smtpd(8), we understand lines
+ * ending in <CR><LF> and bare <LF>. Unlike smtpd(8), we may
+ * treat lines ending in bare <LF> as an offense.
*/
for (transp = cmd_trans; transp->state != state->read_state; transp++)
if (transp->want == 0)
tls_certkey.o: tls.h
tls_certkey.o: tls_certkey.c
tls_client.o: ../../include/argv.h
+tls_client.o: ../../include/iostuff.h
tls_client.o: ../../include/mail_params.h
tls_client.o: ../../include/msg.h
tls_client.o: ../../include/mymalloc.h
tls_server.o: ../../include/argv.h
tls_server.o: ../../include/dict.h
tls_server.o: ../../include/hex_code.h
+tls_server.o: ../../include/iostuff.h
tls_server.o: ../../include/mail_params.h
tls_server.o: ../../include/msg.h
tls_server.o: ../../include/mymalloc.h
int cipher_algbits;
/* Private. */
SSL *con;
- BIO *internal_bio; /* postfix/TLS side of pair */
- BIO *network_bio; /* network side of pair */
char *cache_type; /* tlsmgr(8) cache type if enabled */
char *serverid; /* unique server identifier */
char *namaddr; /* nam[addr] for logging */
/* NAME
/* tls_bio_ops 3
/* SUMMARY
-/* TLS network BIO management
+/* TLS network basic I/O management
/* SYNOPSIS
/* #define TLS_INTERNAL
/* #include <tls.h>
/* int timeout;
/* TLS_SESS_STATE *context;
/* DESCRIPTION
-/* This layer synchronizes the TLS network buffers with the network
-/* while performing TLS handshake or input/output operations.
+/* This module enforces timeouts on non-blocking I/O while
+/* performing TLS handshake or input/output operations.
/*
-/* When the TLS layer is active, it converts plain-text
-/* data from Postfix into encrypted network data and vice versa.
-/* However, to handle network timeout conditions, Postfix
-/* needs to maintain control over network input/output. This
-/* rules out the usual approach of placing the TLS layer
-/* between the application and the network socket.
+/* The Postfix VSTREAM read/write routines invoke the
+/* tls_bio_read/write routines to send and receive plain-text
+/* data. In addition, this module provides tls_bio_connect/accept
+/* routines that trigger the initial TLS handshake. The
+/* tls_bio_xxx routines invoke the corresponding SSL routines
+/* that translate the requests into TLS protocol messages.
/*
-/* As shown below, Postfix reads/writes plain-text data from/to
-/* the TLS layer. The TLS layer informs Postfix when it needs
-/* to read/write encrypted data from/to the network; Postfix
-/* then reads/writes encrypted data from/to the TLS layer and
-/* takes care of the network socket I/O.
+/* Whenever an SSL operation indicates that network input (or
+/* output) needs to happen, the tls_bio_xxx routines wait for
+/* the network to become readable (or writable) within the
+/* timeout limit, then retry the SSL operation. This works
+/* because the network socket is in non-blocking mode.
/*
-/* The TLS layer to network interface is realized with a BIO pair:
+/* tls_bio_connect() performs the SSL_connect() operation.
/*
-/* Postfix SMTP layer | TLS layer
-/* |
-/* smtp/smtpd |
-/* /\ || |
-/* || \/ |
-/* vstream read/write <===> TLS read/write/etc
-/* | /\ ||
-/* | || \/
-/* | BIO pair (internal_bio)
-/* | BIO pair (network_bio)
-/* Postfix socket layer | /\ ||
-/* | || \/
-/* socket read/write <===> BIO read/write
-/* /\ || |
-/* || \/ |
-/* network |
+/* tls_bio_accept() performs the SSL_accept() operation.
/*
-/* The Postfix VSTREAM read/write operations invoke the SSL
-/* read/write operations to send and retrieve plain-text data. Inside
-/* the TLS layer the data are converted to/from TLS protocol.
+/* tls_bio_shutdown() performs the SSL_shutdown() operation.
/*
-/* Whenever an SSL operation reports success, or whenever it
-/* indicates that network input/output needs to happen, Postfix
-/* uses the BIO read/write routines to synchronize the
-/* network_bio buffer with the network. Writing data to the
-/* network has precedence over reading from the network. This
-/* is necessary to avoid deadlock.
+/* tls_bio_read() performs the SSL_read() operation.
/*
-/* The BIO pair buffer size is set to 8192 bytes. This is much
-/* larger than the typical Path MTU, and avoids sending tiny TCP
-/* segments. It is also larger than the default VSTREAM_BUFSIZE
-/* (4096, see vstream.h), so that large write operations can
-/* be handled within one request. The internal buffer in the
-/* network/network_bio handling layer is set to the same
-/* value, since this seems to be reasonable. The code is
-/* however able to handle arbitrary values smaller or larger
-/* than the buffer size in the BIO pair.
-/*
-/* tls_bio_connect() performs the SSL_connect() operation while
-/* synchronizing the network_bio buffer with the network.
-/*
-/* tls_bio_accept() performs the SSL_accept() operation while
-/* synchronizing the network_bio buffer with the network.
-/*
-/* tls_bio_shutdown() performs the SSL_shutdown() operation while
-/* synchronizing the network_bio buffer with the network.
-/*
-/* tls_bio_read() performs the SSL_read() operation while
-/* synchronizing the network_bio buffer with the network.
-/*
-/* tls_bio_write() performs the SSL_write() operation while
-/* synchronizing the network_bio buffer with the network.
+/* tls_bio_write() performs the SSL_write() operation.
/*
/* Arguments:
/* .IP fd
#define TLS_INTERNAL
#include <tls.h>
-/* Application-specific. */
-
-#define NETLAYER_BUFFERSIZE 8192
-
-/* network_biopair_interop - synchronize network with BIO pair */
-
-static int network_biopair_interop(int fd, int timeout, BIO *network_bio)
-{
- const char *myname = "network_biopair_interop";
- int want_write;
- int num_write;
- int write_pos;
- int from_bio;
- int want_read;
- int num_read;
- int to_bio;
- char buffer[NETLAYER_BUFFERSIZE];
-
- /*
- * To avoid deadlock, write all pending data to the network before
- * attempting to read from the network.
- */
- while ((want_write = BIO_ctrl_pending(network_bio)) > 0) {
- if (want_write > sizeof(buffer))
- want_write = sizeof(buffer);
- from_bio = BIO_read(network_bio, buffer, want_write);
-
- /*
- * Write the complete buffer contents to the network.
- */
- for (write_pos = 0; write_pos < from_bio; /* see below */ ) {
- if (timeout > 0 && write_wait(fd, timeout) < 0)
- return (-1);
- num_write = write(fd, buffer + write_pos, from_bio - write_pos);
- if (num_write <= 0) {
- if ((num_write < 0) && (timeout > 0) && (errno == EAGAIN)) {
- msg_warn("write() returns EAGAIN on a writable file descriptor!");
- msg_warn("pausing to avoid going into a tight select/write loop!");
- sleep(1);
- } else {
- msg_warn("%s: error writing %d bytes to the network: %m",
- myname, from_bio - write_pos);
- return (-1);
- }
- } else {
- write_pos += num_write;
- }
- }
- }
-
- /*
- * Read data from the network into the BIO pair.
- */
- while ((want_read = BIO_ctrl_get_read_request(network_bio)) > 0) {
- if (want_read > sizeof(buffer))
- want_read = sizeof(buffer);
- if (timeout > 0 && read_wait(fd, timeout) < 0)
- return (-1);
- num_read = read(fd, buffer, want_read);
- if (num_read == 0)
- /* FIX 200412 Cannot return a zero read count. */
- return (-1);
- if (num_read < 0) {
- if ((num_read < 0) && (timeout > 0) && (errno == EAGAIN)) {
- msg_warn("read() returns EAGAIN on a readable file descriptor!");
- msg_warn("pausing to avoid going into a tight select/write loop!");
- sleep(1);
- } else {
- msg_warn("%s: error reading %d bytes from the network: %m",
- myname, want_read);
- return (-1);
- }
- } else {
- to_bio = BIO_write(network_bio, buffer, num_read);
- if (to_bio != num_read)
- msg_panic("%s: BIO_write error: to_bio != num_read", myname);
- }
- }
- return (0);
-}
-
/* tls_bio - perform SSL input/output operation with extreme prejudice */
int tls_bio(int fd, int timeout, TLS_SESS_STATE *TLScontext,
int status;
int err;
int retval = 0;
- int biop_retval;
int done;
/*
case SSL_ERROR_NONE: /* success */
retval = status;
done = 1;
- /* FALLTHROUGH */
- case SSL_ERROR_WANT_WRITE: /* flush/update buffers */
+ break;
+ case SSL_ERROR_WANT_WRITE:
+ if (write_wait(fd, timeout) < 0)
+ return (-1); /* timeout error */
+ break;
case SSL_ERROR_WANT_READ:
- biop_retval =
- network_biopair_interop(fd, timeout, TLScontext->network_bio);
- if (biop_retval < 0)
- return (-1); /* network read/write error */
+ if (read_wait(fd, timeout) < 0)
+ return (-1); /* timeout error */
break;
/*
#include <vstream.h>
#include <stringops.h>
#include <msg.h>
+#include <iostuff.h> /* non-blocking */
/* Global library. */
| ((protomask & TLS_PROTOCOL_SSLv3) ? SSL_OP_NO_SSLv3 : 0L)
| ((protomask & TLS_PROTOCOL_SSLv2) ? SSL_OP_NO_SSLv2 : 0L));
- /*
- * The TLS connection is realized by a BIO_pair, so obtain the pair.
- *
- * XXX There is no need to make internal_bio a member of the TLScontext
- * structure. It will be attached to TLScontext->con, and destroyed along
- * with it. The network_bio, however, needs to be freed explicitly.
- */
- if (!BIO_new_bio_pair(&TLScontext->internal_bio, TLS_BIO_BUFSIZE,
- &TLScontext->network_bio, TLS_BIO_BUFSIZE)) {
- msg_warn("Could not obtain BIO_pair");
- tls_print_errors();
- tls_free_context(TLScontext);
- return (0);
- }
-
/*
* XXX To avoid memory leaks we must always call SSL_SESSION_free() after
* calling SSL_set_session(), regardless of whether or not the session
SSL_set_connect_state(TLScontext->con);
/*
- * Connect the SSL connection with the Postfix side of the BIO-pair for
- * reading and writing.
+ * Connect the SSL connection with the network socket.
+ */
+ if (SSL_set_fd(TLScontext->con, vstream_fileno(props->stream)) != 1) {
+ msg_info("SSL_set_fd error to %s: %d", props->namaddr, sts);
+ tls_print_errors();
+ uncache_session(app_ctx->ssl_ctx, TLScontext);
+ tls_free_context(TLScontext);
+ return (0);
+ }
+
+ /*
+ * Turn on non-blocking I/O so that we can enforce timeouts on network
+ * I/O.
*/
- SSL_set_bio(TLScontext->con, TLScontext->internal_bio,
- TLScontext->internal_bio);
+ non_blocking(vstream_fileno(props->stream), NON_BLOCKING);
/*
* If the debug level selected is high enough, all of the data is dumped:
TLScontext = (TLS_SESS_STATE *) mymalloc(sizeof(TLS_SESS_STATE));
memset((char *) TLScontext, 0, sizeof(*TLScontext));
TLScontext->con = 0;
- TLScontext->internal_bio = 0;
- TLScontext->network_bio = 0;
TLScontext->cache_type = 0;
TLScontext->serverid = 0;
TLScontext->peer_CN = 0;
*/
if (TLScontext->con != 0)
SSL_free(TLScontext->con);
- if (TLScontext->network_bio)
- BIO_free(TLScontext->network_bio);
if (TLScontext->namaddr)
myfree(TLScontext->namaddr);
#include <stringops.h>
#include <msg.h>
#include <hex_code.h>
+#include <iostuff.h> /* non-blocking */
/* Global library. */
return (0);
}
- /*
- * The TLS connection is realized by a BIO_pair, so obtain the pair.
- *
- * XXX There is no need to store the internal_bio handle in the TLScontext
- * structure. It will be attached to and destroyed with TLScontext->con.
- * The network_bio, however, needs to be freed explicitly, so we need to
- * store its handle in TLScontext.
- */
- if (!BIO_new_bio_pair(&TLScontext->internal_bio, TLS_BIO_BUFSIZE,
- &TLScontext->network_bio, TLS_BIO_BUFSIZE)) {
- msg_warn("Could not obtain BIO_pair");
- tls_print_errors();
- tls_free_context(TLScontext);
- return (0);
- }
-
/*
* Before really starting anything, try to seed the PRNG a little bit
* more.
SSL_set_accept_state(TLScontext->con);
/*
- * Connect the SSL connection with the Postfix side of the BIO-pair for
- * reading and writing.
+ * Connect the SSL connection with the network socket.
+ */
+ if (SSL_set_fd(TLScontext->con, vstream_fileno(props->stream)) != 1) {
+ msg_info("SSL_set_fd error to %s: %d", props->namaddr, sts);
+ tls_print_errors();
+ uncache_session(app_ctx->ssl_ctx, TLScontext);
+ tls_free_context(TLScontext);
+ return (0);
+ }
+
+ /*
+ * Turn on non-blocking I/O so that we can enforce timeouts on network
+ * I/O.
*/
- SSL_set_bio(TLScontext->con, TLScontext->internal_bio,
- TLScontext->internal_bio);
+ non_blocking(vstream_fileno(props->stream), NON_BLOCKING);
/*
* If the debug level selected is high enough, all of the data is dumped:
projects / thirdparty / postfix.git / commitdiff
