Documentation: added smtp_balance_inet_protocols to the
text with smtp_address_preference caveats. File:
proto/postconf.proto.
+
+20230926
+
+ Documentation: added a section to smtp_balance_inet_protocols
+ to address the problem that servers may flag mail received
+ over IPv6 as more spammy. File: proto/postconf.proto.
+
+20231006
+
+ Cleanup: attempt to log the SASL username after authentication
+ failure. This appends ", sasl_username=xxx" to SASL authentication
+ failure logging. Based on code by Jozsef Kadlecsik. Files:
+ xsasl/xsasl_server.c, xsasl/xsasl_cyrus_server.c,
+ smtpd/smtpd_sasl_glue.c.
+
+20231008
+
+ Cleanup: enforce stricter UTF8 checks in printable(). Factor
+ out the UTF8 parser, so that it can be shared between
+ valid_utf8_string() and printable(). Wietse Venema, with
+ tests by Viktor Dukhovni. Files: util/valid_utf8_string.c,
+ util/printable.c, util/parse_utf8_char.c, util/printable.in,
+ util/printable.ref.
spam before their IP address becomes denylisted. To speed up spam deliveries,
zombies make compromises in their SMTP protocol implementation. For example,
they speak before their turn, or they ignore responses from SMTP servers and
-continue sending mail even when the server tells them to go away.
+continue sending commands even when the server tells them to go away.
postscreen(8) uses a variety of measurements to recognize zombies. First,
postscreen(8) determines if the remote SMTP client IP address is denylisted.
To speed up spam deliveries, zombies make compromises in their SMTP
protocol implementation. For example, they speak before their turn,
or they ignore responses from SMTP servers and continue sending
-mail even when the server tells them to go away. </p>
+commands even when the server tells them to go away. </p>
<p> <a href="postscreen.8.html">postscreen(8)</a> uses a variety of measurements to recognize
zombies. First, <a href="postscreen.8.html">postscreen(8)</a> determines if the remote SMTP client
<ul>
<li> <p> The setting "<a href="postconf.5.html#smtp_address_preference">smtp_address_preference</a> = ipv6" is unsafe.
-All deliveries will suffer delays when IPv6 is not available even
+All deliveries will suffer delays during an IPv6 outage, even
while the destination is still reachable over IPv4. Mail may be
stuck in the queue with Postfix versions < 3.3 that do not
implement "<a href="postconf.5.html#smtp_balance_inet_protocols">smtp_balance_inet_protocols</a>". For similar reasons, the
half of deliveries will suffer delays if there is an outage
that affects IPv6 or IPv4, as long as it does not affect both. </p>
+<li> <p> The setting "<a href="postconf.5.html#smtp_address_preference">smtp_address_preference</a> = ipv4" is not a
+solution for remote servers that flag email received over IPv6 as
+more 'spammy' (the client IPv6 address has a bad or missing PTR or
+AAAA record, bad network neighbors, etc.). Instead, configure Postfix
+to receive mail over both IPv4 and IPv6, and to deliver mail over
+only IPv4. </p>
+
+<blockquote>
+<pre>
+/etc/postfix/<a href="postconf.5.html">main.cf</a>:
+ <a href="postconf.5.html#inet_protocols">inet_protocols</a> = all
+</pre>
+</blockquote>
+
+<blockquote>
+<pre>
+/etc/postfix/<a href="master.5.html">master.cf</a>
+ smtp ...other fields... smtp -o <a href="postconf.5.html#inet_protocols">inet_protocols</a>=ipv4
+</pre>
+</blockquote>
+
</ul>
<p> This feature is available in Postfix 2.8 and later. </p>
IPv6 connectivity:
.IP \(bu
The setting "smtp_address_preference = ipv6" is unsafe.
-All deliveries will suffer delays when IPv6 is not available even
+All deliveries will suffer delays during an IPv6 outage, even
while the destination is still reachable over IPv4. Mail may be
stuck in the queue with Postfix versions < 3.3 that do not
implement "smtp_balance_inet_protocols". For similar reasons, the
this, and "smtp_balance_inet_protocols = yes" (the default), only
half of deliveries will suffer delays if there is an outage
that affects IPv6 or IPv4, as long as it does not affect both.
+.IP \(bu
+The setting "smtp_address_preference = ipv4" is not a
+solution for remote servers that flag email received over IPv6 as
+more 'spammy' (the client IPv6 address has a bad or missing PTR or
+AAAA record, bad network neighbors, etc.). Instead, configure Postfix
+to receive mail over both IPv4 and IPv6, and to deliver mail over
+only IPv4.
+.sp
+.in +4
+.nf
+.na
+.ft C
+/etc/postfix/main.cf:
+ inet_protocols = all
+.fi
+.ad
+.ft R
+.in -4
+.sp
+.in +4
+.nf
+.na
+.ft C
+/etc/postfix/master.cf
+ smtp ...other fields... smtp \-o inet_protocols=ipv4
+.fi
+.ad
+.ft R
+.in -4
.br
.PP
This feature is available in Postfix 2.8 and later.
To speed up spam deliveries, zombies make compromises in their SMTP
protocol implementation. For example, they speak before their turn,
or they ignore responses from SMTP servers and continue sending
-mail even when the server tells them to go away. </p>
+commands even when the server tells them to go away. </p>
<p> postscreen(8) uses a variety of measurements to recognize
zombies. First, postscreen(8) determines if the remote SMTP client
<ul>
<li> <p> The setting "smtp_address_preference = ipv6" is unsafe.
-All deliveries will suffer delays when IPv6 is not available even
+All deliveries will suffer delays during an IPv6 outage, even
while the destination is still reachable over IPv4. Mail may be
stuck in the queue with Postfix versions < 3.3 that do not
implement "smtp_balance_inet_protocols". For similar reasons, the
half of deliveries will suffer delays if there is an outage
that affects IPv6 or IPv4, as long as it does not affect both. </p>
+<li> <p> The setting "smtp_address_preference = ipv4" is not a
+solution for remote servers that flag email received over IPv6 as
+more 'spammy' (the client IPv6 address has a bad or missing PTR or
+AAAA record, bad network neighbors, etc.). Instead, configure Postfix
+to receive mail over both IPv4 and IPv6, and to deliver mail over
+only IPv4. </p>
+
+<blockquote>
+<pre>
+/etc/postfix/main.cf:
+ inet_protocols = all
+</pre>
+</blockquote>
+
+<blockquote>
+<pre>
+/etc/postfix/master.cf
+ smtp ...other fields... smtp -o inet_protocols=ipv4
+</pre>
+</blockquote>
+
</ul>
<p> This feature is available in Postfix 2.8 and later. </p>
Serg
Kinzler
smtpstone
+spammy
wraptls
api
MinProtocol
+spammy
* Patches change both the patchlevel and the release date. Snapshots have no
* patchlevel; they change the release date only.
*/
-#define MAIL_RELEASE_DATE "20230924"
+#define MAIL_RELEASE_DATE "20231008"
#define MAIL_VERSION_NUMBER "3.9"
#ifdef SNAPSHOT
/* Google, Inc.
/* 111 8th Avenue
/* New York, NY 10011, USA
+/*
+/* Wietse Venema
+/* porcupine.org
+/* Amawalk, NY 10501, USA
/*--*/
/* System library. */
}
}
if (status != XSASL_AUTH_DONE) {
- msg_warn("%s: SASL %s authentication failed: %s",
+ sasl_username = xsasl_server_get_username(state->sasl_server);
+ msg_warn("%s: SASL %s authentication failed: %s, sasl_username=%s",
state->namaddr, sasl_method,
- STR(state->sasl_reply));
+ STR(state->sasl_reply),
+ sasl_username ? sasl_username : "(unavailable)");
/* RFC 4954 Section 6. */
if (status == XSASL_AUTH_TEMP)
smtpd_chat_reply(state, "454 4.7.0 Temporary authentication failure: %s",
--- /dev/null
+printable.in binary
vstream timecmp dict_cache midna_domain casefold strcasecmp_utf8 \
vbuf_print split_qnameval vstream msg_logger byte_mask \
known_tcp_ports dict_stream find_inet binhash hash_fnv argv \
- clean_env inet_prefix_top
+ clean_env inet_prefix_top printable
PLUGIN_MAP_SO = $(LIB_PREFIX)pcre$(LIB_SUFFIX) $(LIB_PREFIX)lmdb$(LIB_SUFFIX) \
$(LIB_PREFIX)cdb$(LIB_SUFFIX) $(LIB_PREFIX)sdbm$(LIB_SUFFIX)
HTABLE_FIX = NORANDOMIZE=1
$(CC) $(CFLAGS) -DTEST -o $@ $@.c $(LIB) $(SYSLIBS)
mv junk $@.o
+printable: $(LIB)
+ mv $@.o junk
+ $(CC) $(CFLAGS) -DTEST -o $@ $@.c $(LIB) $(SYSLIBS)
+ mv junk $@.o
+
hex_quote: $(LIB)
mv $@.o junk
$(CC) $(CFLAGS) -DTEST -o $@ $@.c $(LIB) $(SYSLIBS)
strcasecmp_utf8_test vbuf_print_test miss_endif_cidr_test \
miss_endif_regexp_test split_qnameval_test vstring_test \
vstream_test byte_mask_tests mystrtok_test known_tcp_ports_test \
- binhash_test argv_test inet_prefix_top_test
+ binhash_test argv_test inet_prefix_top_test printable_test
dict_tests: all dict_test \
dict_pcre_tests dict_cidr_test dict_thash_test dict_static_test \
# diff unescape.in unescape.tmp
rm -f unescape.tmp
+printable_test: printable printable.in
+ $(SHLIB_ENV) ${VALGRIND} ./printable <printable.in > printable.tmp
+ diff -b printable.ref printable.tmp
+ rm -f printable.tmp
+
hex_quote_test: hex_quote
$(SHLIB_ENV) ${VALGRIND} ./hex_quote <hex_quote.c | od -cb >hex_quote.tmp
od -cb <hex_quote.c >hex_quote.ref
posix_signals.o: posix_signals.h
posix_signals.o: sys_defs.h
printable.o: check_arg.h
+printable.o: parse_utf8_char.h
printable.o: printable.c
printable.o: stringops.h
printable.o: sys_defs.h
valid_utf8_hostname.o: vbuf.h
valid_utf8_hostname.o: vstring.h
valid_utf8_string.o: check_arg.h
+valid_utf8_string.o: parse_utf8_char.h
valid_utf8_string.o: stringops.h
valid_utf8_string.o: sys_defs.h
valid_utf8_string.o: valid_utf8_string.c
--- /dev/null
+/*++
+/* NAME
+/* parse_utf8_char 3h
+/* SUMMARY
+/* parse one UTF-8 multibyte character
+/* SYNOPSIS
+/* #include <parse_utf8_char.h>
+/*
+/* char *parse_utf8_char(str, len)
+/* const char *str;
+/* ssize_t len;
+/* DESCRIPTION
+/* parse_utf8_char() determines if the \fBlen\fR bytes starting
+/* at \fBstr\fR begin with a complete UTF-8 multi-byte character
+/* as defined in RFC 3629. That is, it contains a proper
+/* encoding of code points U+0000..U+10FFFF, excluding over-long
+/* encodings and excluding U+D800..U+DFFF surrogates.
+/*
+/* When the \fBlen\fR bytes starting at \fBstr\fR begin with
+/* a complete UTF-8 multi-byte character, this function returns
+/* a pointer to the last byte in that character. Otherwise,
+/* it returns a null pointer.
+/* BUGS
+/* Code points in the range U+FDD0..U+FDEF and ending in FFFE
+/* or FFFF are non-characters in UNICODE. This function does
+/* not reject these.
+/* LICENSE
+/* .ad
+/* .fi
+/* The Secure Mailer license must be distributed with this software.
+/* AUTHOR(S)
+/* Wietse Venema
+/* IBM T.J. Watson Research
+/* P.O. Box 704
+/* Yorktown Heights, NY 10598, USA
+/*
+/* Wietse Venema
+/* porcupine.org
+/* Amawalk, NY 10501, USA
+/*--*/
+
+ /*
+ * System library.
+ */
+#include <sys_defs.h>
+
+#ifdef NO_INLINE
+#define inline /* */
+#endif
+
+/* parse_utf8_char - parse and validate one UTF8 multibyte sequence */
+
+static inline char *parse_utf8_char(const char *str, const char *end)
+{
+ const unsigned char *cp = (const unsigned char *) str;
+ const unsigned char *ep = (const unsigned char *) end;
+ unsigned char c0, ch;
+
+ /*
+ * Optimized for correct input, time, space, and for CPUs that have a
+ * decent number of registers.
+ */
+ /* Single-byte encodings. */
+ if (EXPECTED((c0 = *cp) <= 0x7f) /* we know that c0 >= 0x0 */ ) {
+ return ((char *) cp);
+ }
+ /* Two-byte encodings. */
+ else if (EXPECTED(c0 <= 0xdf) /* we know that c0 >= 0x80 */ ) {
+ /* Exclude over-long encodings. */
+ if (UNEXPECTED(c0 < 0xc2)
+ || UNEXPECTED(cp + 1 >= ep)
+ /* Require UTF-8 tail byte. */
+ || UNEXPECTED(((ch = *++cp) & 0xc0) != 0x80))
+ return (0);
+ return ((char *) cp);
+ }
+ /* Three-byte encodings. */
+ else if (EXPECTED(c0 <= 0xef) /* we know that c0 >= 0xe0 */ ) {
+ if (UNEXPECTED(cp + 2 >= ep)
+ /* Exclude over-long encodings. */
+ || UNEXPECTED((ch = *++cp) < (c0 == 0xe0 ? 0xa0 : 0x80))
+ /* Exclude U+D800..U+DFFF. */
+ || UNEXPECTED(ch > (c0 == 0xed ? 0x9f : 0xbf))
+ /* Require UTF-8 tail byte. */
+ || UNEXPECTED(((ch = *++cp) & 0xc0) != 0x80))
+ return (0);
+ return ((char *) cp);
+ }
+ /* Four-byte encodings. */
+ else if (EXPECTED(c0 <= 0xf4) /* we know that c0 >= 0xf0 */ ) {
+ if (UNEXPECTED(cp + 3 >= ep)
+ /* Exclude over-long encodings. */
+ || UNEXPECTED((ch = *++cp) < (c0 == 0xf0 ? 0x90 : 0x80))
+ /* Exclude code points above U+10FFFF. */
+ || UNEXPECTED(ch > (c0 == 0xf4 ? 0x8f : 0xbf))
+ /* Require UTF-8 tail byte. */
+ || UNEXPECTED(((ch = *++cp) & 0xc0) != 0x80)
+ /* Require UTF-8 tail byte. */
+ || UNEXPECTED(((ch = *++cp) & 0xc0) != 0x80))
+ return (0);
+ return ((char *) cp);
+ }
+ /* Invalid: c0 >= 0xf5 */
+ else {
+ return (0);
+ }
+}
+
+#undef inline
/* Google, Inc.
/* 111 8th Avenue
/* New York, NY 10011, USA
+/*
+/* Wietse Venema
+/* porcupine.org
+/* Amawalk, NY 10501, USA
/*--*/
/* System library. */
/* Utility library. */
#include "stringops.h"
+#include "parse_utf8_char.h"
int util_utf8_enable = 0;
char *printable_except(char *string, int replacement, const char *except)
{
- unsigned char *cp;
+ char *cp;
+ char *ep = string + strlen(string);
+ char *last;
int ch;
/*
- * XXX Replace invalid UTF8 sequences (too short, over-long encodings,
- * out-of-range code points, etc). See valid_utf8_string.c.
+ * In case of a non-UTF8 sequence (bad leader byte, bad non-leader byte,
+ * over-long encodings, out-of-range code points, etc), replace the first
+ * byte, and try to resynchronize at the next byte.
*/
- cp = (unsigned char *) string;
- while ((ch = *cp) != 0) {
- if (ISASCII(ch) && (ISPRINT(ch) || (except && strchr(except, ch)))) {
- /* ok */
- } else if (util_utf8_enable && ch >= 194 && ch <= 254
- && cp[1] >= 128 && cp[1] < 192) {
- /* UTF8; skip the rest of the bytes in the character. */
- while (cp[1] >= 128 && cp[1] < 192)
- cp++;
- } else {
- /* Not ASCII and not UTF8. */
- *cp = replacement;
+#define PRINT_OR_EXCEPT(ch) (ISPRINT(ch) || (except && strchr(except, ch)))
+
+ for (cp = string; (ch = *(unsigned char *) cp) != 0; cp++) {
+ if (util_utf8_enable == 0) {
+ if (ISASCII(ch) && PRINT_OR_EXCEPT(ch))
+ continue;
+ } else if ((last = parse_utf8_char(cp, ep)) == cp) { /* ASCII */
+ if (PRINT_OR_EXCEPT(ch))
+ continue;
+ } else if (last > cp) { /* Other UTF8 */
+ cp = last;
+ continue;
}
- cp++;
+ *cp = replacement;
}
return (string);
}
+
+#ifdef TEST
+
+#include <stdlib.h>
+#include <string.h>
+#include <msg.h>
+#include <vstring_vstream.h>
+
+int main(int argc, char **argv)
+{
+ VSTRING *in = vstring_alloc(10);
+
+ util_utf8_enable = 1;
+
+ while (vstring_fgets_nonl(in, VSTREAM_IN)) {
+ printable(vstring_str(in), '?');
+ vstream_fwrite(VSTREAM_OUT, vstring_str(in), VSTRING_LEN(in));
+ VSTREAM_PUTC('\n', VSTREAM_OUT);
+ }
+ vstream_fflush(VSTREAM_OUT);
+ exit(0);
+}
+
+#endif
--- /dev/null
+printable
+non\bn-printable
+naïve
+naÃve
+виктор
+вÐкºÑ\82оÑ\80
+ויקטוּר
+×\95\99ק×\98×\95ּר
+中国互联网络发展状况统计报告
+ä¸åäº\92è\94ç½\91ç»\9cå\8f\91å±\95ç\8a¶å\86µç»\9f计æ\8a¥å
--- /dev/null
+printable
+non?n-printable
+naïve
+na?ve
+виктор
+в?к?тор
+ויקטוּר
+ו?קטוּר
+中国互联网络发展状况统计报告
+中?互??网络发展状况统计报?
/* const char *str;
/* ssize_t len;
/* DESCRIPTION
-/* valid_utf8_string() determines if a string satisfies the UTF-8
-/* definition in RFC 3629. That is, it contains proper encodings
-/* of code points U+0000..U+10FFFF, excluding over-long encodings
-/* and excluding U+D800..U+DFFF surrogates.
+/* valid_utf8_string() determines if all bytes in a string
+/* satisfy parse_utf8_char(3h) checks. See there for any
+/* implementation limitations.
/*
/* A zero-length string is considered valid.
/* DIAGNOSTICS
/* The result value is zero when the caller specifies a negative
-/* length, or a string that violates RFC 3629, for example a
-/* string that is truncated in the middle of a multi-byte
-/* sequence.
-/* BUGS
-/* But wait, there is more. Code points in the range U+FDD0..U+FDEF
-/* and ending in FFFE or FFFF are non-characters in UNICODE. This
-/* function does not block these.
+/* length, or a string that does not pass parse_utf8_char(3h) checks.
/* SEE ALSO
-/* RFC 3629
+/* parse_utf8_char(3h), parse one UTF-8 multibyte character
/* LICENSE
/* .ad
/* .fi
/* IBM T.J. Watson Research
/* P.O. Box 704
/* Yorktown Heights, NY 10598, USA
+/*
+/* Wietse Venema
+/* porcupine.org
+/* Amawalk, NY 10501, USA
/*--*/
/* System library. */
/* Utility library. */
#include <stringops.h>
+#include <parse_utf8_char.h>
/* valid_utf8_string - validate string according to RFC 3629 */
int valid_utf8_string(const char *str, ssize_t len)
{
- const unsigned char *end = (const unsigned char *) str + len;
- const unsigned char *cp;
- unsigned char c0, ch;
+ const char *ep = str + len;
+ const char *cp;
+ const char *last;
if (len < 0)
return (0);
return (1);
/*
- * Optimized for correct input, time, space, and for CPUs that have a
- * decent number of registers.
+ * Ideally, the compiler will inline parse_utf8_char().
*/
- for (cp = (const unsigned char *) str; cp < end; cp++) {
- /* Single-byte encodings. */
- if (EXPECTED((c0 = *cp) <= 0x7f) /* we know that c0 >= 0x0 */ ) {
- /* void */ ;
- }
- /* Two-byte encodings. */
- else if (EXPECTED(c0 <= 0xdf) /* we know that c0 >= 0x80 */ ) {
- /* Exclude over-long encodings. */
- if (UNEXPECTED(c0 < 0xc2)
- || UNEXPECTED(cp + 1 >= end)
- /* Require UTF-8 tail byte. */
- || UNEXPECTED(((ch = *++cp) & 0xc0) != 0x80))
- return (0);
- }
- /* Three-byte encodings. */
- else if (EXPECTED(c0 <= 0xef) /* we know that c0 >= 0xe0 */ ) {
- if (UNEXPECTED(cp + 2 >= end)
- /* Exclude over-long encodings. */
- || UNEXPECTED((ch = *++cp) < (c0 == 0xe0 ? 0xa0 : 0x80))
- /* Exclude U+D800..U+DFFF. */
- || UNEXPECTED(ch > (c0 == 0xed ? 0x9f : 0xbf))
- /* Require UTF-8 tail byte. */
- || UNEXPECTED(((ch = *++cp) & 0xc0) != 0x80))
- return (0);
- }
- /* Four-byte encodings. */
- else if (EXPECTED(c0 <= 0xf4) /* we know that c0 >= 0xf0 */ ) {
- if (UNEXPECTED(cp + 3 >= end)
- /* Exclude over-long encodings. */
- || UNEXPECTED((ch = *++cp) < (c0 == 0xf0 ? 0x90 : 0x80))
- /* Exclude code points above U+10FFFF. */
- || UNEXPECTED(ch > (c0 == 0xf4 ? 0x8f : 0xbf))
- /* Require UTF-8 tail byte. */
- || UNEXPECTED(((ch = *++cp) & 0xc0) != 0x80)
- /* Require UTF-8 tail byte. */
- || UNEXPECTED(((ch = *++cp) & 0xc0) != 0x80))
- return (0);
- }
- /* Invalid: c0 >= 0xf5 */
- else {
+ for (cp = str; cp < ep; cp++) {
+ if ((last = parse_utf8_char(cp, ep)) != 0)
+ cp = last;
+ else
return (0);
- }
}
return (1);
}
/* Google, Inc.
/* 111 8th Avenue
/* New York, NY 10011, USA
+/*
+/* Wietse Venema
+/* porcupine.org
+/* Amawalk, NY 10501, USA
/*--*/
/* System library. */
/*
* XXX Do not free(serverout).
*/
+ if (server->username)
+ myfree(server->username);
sasl_status = sasl_getprop(server->sasl_conn, SASL_USERNAME, &serverout);
if (sasl_status != SASL_OK || serverout == 0) {
- msg_warn("%s: sasl_getprop SASL_USERNAME botch: %s",
- myname, xsasl_cyrus_strerror(sasl_status));
- return (0);
+ server->username = 0;
+ } else {
+ server->username = mystrdup(serverout);
+ printable(server->username, '?');
}
- if (server->username)
- myfree(server->username);
- server->username = mystrdup(serverout);
- printable(server->username, '?');
return (server->username);
}
/* reply.
/*
/* xsasl_server_get_username() returns the stored username
-/* after successful authentication.
+/* after successful authentication. The username may be null
+/* after authentication failure, depending on the kind of
+/* failure and on authentication backend implementation
+/* details. A non-null result is converted to printable text.
/*
/* Arguments:
/* .IP addr_family
/* Google, Inc.
/* 111 8th Avenue
/* New York, NY 10011, USA
+/*
+/* Wietse Venema
+/* porcupine.org
+/* Amawalk, NY 10501, USA
/*--*/
/* System library. */
projects / thirdparty / postfix.git / commitdiff
