From: Wietse Venema Date: Mon, 15 Jun 2015 05:00:00 +0000 (-0500) Subject: postfix-3.1-20150615-nonprod X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=89059c519bf065708ac872fd9303d62c96556b68;p=thirdparty%2Fpostfix.git postfix-3.1-20150615-nonprod --- diff --git a/postfix/HISTORY b/postfix/HISTORY index 8ac693bcd..5bbac2a8b 100644 --- a/postfix/HISTORY +++ b/postfix/HISTORY @@ -21757,3 +21757,28 @@ Apologies for any names omitted. Cleanup: updated the examples in MILTER_README. File: proto/MILTER_README.html + +20150529 + + Support for DNS reply TTL values in dnsblog and postscreen. + Files: dbsblog/dnsblog.c, postscreen/postscreen_early.c, + postscreen/postscreen_dnsbl.c. + +20150607 + + Support for DNS reply TTL values for "not found" responses + (negative reply caching). The postscreen daemon needs this to + accurately whitelist an SMTP client that is not found on any + DNSBL. Files: dns/dns_lookup.c, dns/dns_strrecord.c, dns/dns.h, + dns/test_dns_lookup.c. + +20150615 + + Two new parameters to limit how long a DNSBL or DNSWL lookup + result remains valid: postscreen_dnsbl_max_ttl is an upper + limit for the TTL from a DNS query, and postscreen_dnsbl_min_ttl + is a lower limit. The old postscreen_dnsbl_ttl provides a + backwards-compatible default for postscreen_dnsbl_max_ttl. + Files: global/mail_params.h, postscreen/postscreen.c, + postscreen/postscreen_early.c, mantools/postlink, + proto/postconf.proto. diff --git a/postfix/RELEASE_NOTES b/postfix/RELEASE_NOTES index c47c503b2..183994687 100644 --- a/postfix/RELEASE_NOTES +++ b/postfix/RELEASE_NOTES @@ -15,3 +15,29 @@ specifies the release date of a stable release or snapshot release. If you upgrade from Postfix 2.11 or earlier, read RELEASE_NOTES-3.0 before proceeding. + +Major change with snaphot 20106XX + +postscreen support for the TTL of DNS lookup results, including +TTLs of "not found" results (as described in RFC 2308). + +Historically, the default setting "postscreen_dnsbl_ttl = 1h" assumes +that a "not found" result from a DNSBL server will be valid for one hour. +This may have been adequate five years ago, but nowadays, this will +result in missed opportunities to block spambots. + +Support for DNS reply TTL values is controlled by two parameters: + +postscreen_dnsbl_ttl_minimum (default: 1 minute). + + This parameter specifies a minimum for the amount of time that a + result will be cached in the postscreen_cache_map. This prevents an + excessive number of postscreen cache updates when a DNSBNL or DNSWL + server specifies a very small reply TTL. + +The postscreen_dnsbl_ttl (default: 1 hour) + + This parameter already exists. It now specifies an upper bound + for for the amount of time that a result will be cached in the + postscreen_cache_map. This prevents cache pollution when a DNSBNL or + DNSWL server specifies a very large replt TTL. diff --git a/postfix/html/postconf.5.html b/postfix/html/postconf.5.html index c14c4ea3b..08990cb32 100644 --- a/postfix/html/postconf.5.html +++ b/postfix/html/postconf.5.html @@ -7746,6 +7746,43 @@ this test the next time the client connects.

This feature is available in Postfix 2.8.

+ + +
postscreen_dnsbl_max_ttl +(default: ${postscreen_dnsbl_ttl?{$postscreen_dnsbl_ttl}:{1h}})
+ +

The maximum amount of time that postscreen(8) will use the +result from a successful DNS blocklist or whitelist test before a +client IP address is required to pass that test again. If the DNS +reply specifies a shorter TTL value, that value will be used unless +it would be smaller than postscreen_dnsbl_min_ttl.

+ +

Specify a non-zero time value (an integral value plus an optional +one-letter suffix that specifies the time unit). Time units: s +(seconds), m (minutes), h (hours), d (days), w (weeks).

+ +

This feature is available in Postfix 3.1. The default setting +is backwards-compatible with older Postfix versions.

+ + +
+ +
postscreen_dnsbl_min_ttl +(default: 100s)
+ +

The minimum amount of time that postscreen(8) will use the +result from a successful DNS blocklist or whitelist test before a +client IP address is required to pass that test again. If the DNS +reply specifies a larger TTL value, that value will be used unless +it would be larger than postscreen_dnsbl_max_ttl.

+ +

Specify a non-zero time value (an integral value plus an optional +one-letter suffix that specifies the time unit). Time units: s +(seconds), m (minutes), h (hours), d (days), w (weeks).

+ +

This feature is available in Postfix 3.1.

+ +
postscreen_dnsbl_reply_map @@ -7869,16 +7906,15 @@ resolver(3) routines.

(default: 1h)

The amount of time that postscreen(8) will use the result from -a successful DNS blocklist test. During this time, the client IP address -is excluded from this test. The default is relatively short, because a -good client can immediately talk to a real Postfix SMTP server. -

+a successful DNS blocklist or whitelist lookup before a client +IP address is required to pass that test again.

Specify a non-zero time value (an integral value plus an optional one-letter suffix that specifies the time unit). Time units: s (seconds), m (minutes), h (hours), d (days), w (weeks).

-

This feature is available in Postfix 2.8.

+

This feature is available in Postfix 2.8-3.0. It was +replaced by postscreen_dnsbl_max_ttl in Postfix 3.1.

diff --git a/postfix/html/postscreen.8.html b/postfix/html/postscreen.8.html index 85761ea45..6590a3313 100644 --- a/postfix/html/postscreen.8.html +++ b/postfix/html/postscreen.8.html @@ -292,9 +292,16 @@ POSTSCREEN(8) POSTSCREEN(8) The amount of time that postscreen(8) will use the result from a successful "bare newline" SMTP protocol test. - postscreen_dnsbl_ttl (1h) - The amount of time that postscreen(8) will use the result from a - successful DNS blocklist test. + postscreen_dnsbl_max_ttl + (${postscreen_dnsbl_ttl?{$postscreen_dnsbl_ttl}:{1h}}) + The maximum amount of time that postscreen(8) will use the + result from a successful DNS blocklist or whitelist test before + a client IP address is required to pass that test again. + + postscreen_dnsbl_min_ttl (100s) + The minimum amount of time that postscreen(8) will use the + result from a successful DNS blocklist or whitelist test before + a client IP address is required to pass that test again. postscreen_greet_ttl (1d) The amount of time that postscreen(8) will use the result from a @@ -310,34 +317,34 @@ POSTSCREEN(8) POSTSCREEN(8) RESOURCE CONTROLS line_length_limit (2048) - Upon input, long lines are chopped up into pieces of at most + Upon input, long lines are chopped up into pieces of at most this length; upon delivery, long lines are reconstructed. postscreen_client_connection_count_limit ($smtpd_client_connec- tion_count_limit) - How many simultaneous connections any remote SMTP client is + How many simultaneous connections any remote SMTP client is allowed to have with the postscreen(8) daemon. postscreen_command_count_limit (20) - The limit on the total number of commands per SMTP session for + The limit on the total number of commands per SMTP session for postscreen(8)'s built-in SMTP protocol engine. postscreen_command_time_limit (normal: 300s, overload: 10s) - The time limit to read an entire command line with + The time limit to read an entire command line with postscreen(8)'s built-in SMTP protocol engine. postscreen_post_queue_limit ($default_process_limit) - The number of clients that can be waiting for service from a + The number of clients that can be waiting for service from a real Postfix SMTP server process. postscreen_pre_queue_limit ($default_process_limit) - The number of non-whitelisted clients that can be waiting for a - decision whether they will receive service from a real Postfix + The number of non-whitelisted clients that can be waiting for a + decision whether they will receive service from a real Postfix SMTP server process. postscreen_watchdog_timeout (10s) - How much time a postscreen(8) process may take to respond to a - remote SMTP client command or to perform a cache operation + How much time a postscreen(8) process may take to respond to a + remote SMTP client command or to perform a cache operation before it is terminated by a built-in watchdog timer. STARTTLS CONTROLS @@ -350,11 +357,11 @@ POSTSCREEN(8) POSTSCREEN(8) The name of the tlsproxy(8) service entry in master.cf. OBSOLETE STARTTLS SUPPORT CONTROLS - These parameters are supported for compatibility with smtpd(8) legacy + These parameters are supported for compatibility with smtpd(8) legacy parameters. postscreen_use_tls ($smtpd_use_tls) - Opportunistic TLS: announce STARTTLS support to remote SMTP + Opportunistic TLS: announce STARTTLS support to remote SMTP clients, but do not require that clients use TLS encryption. postscreen_enforce_tls ($smtpd_enforce_tls) @@ -363,18 +370,18 @@ POSTSCREEN(8) POSTSCREEN(8) MISCELLANEOUS CONTROLS config_directory (see 'postconf -d' output) - The default location of the Postfix main.cf and master.cf con- + The default location of the Postfix main.cf and master.cf con- figuration files. delay_logging_resolution_limit (2) - The maximal number of digits after the decimal point when log- + The maximal number of digits after the decimal point when log- ging sub-second delay values. command_directory (see 'postconf -d' output) The location of all postfix administrative commands. max_idle (100s) - The maximum amount of time that an idle Postfix daemon process + The maximum amount of time that an idle Postfix daemon process waits for an incoming connection before terminating voluntarily. process_id (read-only) @@ -387,8 +394,8 @@ POSTSCREEN(8) POSTSCREEN(8) The syslog facility of Postfix logging. syslog_name (see 'postconf -d' output) - The mail system name that is prepended to the process name in - syslog records, so that "smtpd" becomes, for example, "post- + The mail system name that is prepended to the process name in + syslog records, so that "smtpd" becomes, for example, "post- fix/smtpd". SEE ALSO @@ -406,7 +413,7 @@ POSTSCREEN(8) POSTSCREEN(8) HISTORY This service was introduced with Postfix version 2.8. - Many ideas in postscreen(8) were explored in earlier work by Michael + Many ideas in postscreen(8) were explored in earlier work by Michael Tokarev, in OpenBSD spamd, and in MailChannels Traffic Control. AUTHOR(S) diff --git a/postfix/makedefs b/postfix/makedefs index 052cf9b98..2ce0774db 100644 --- a/postfix/makedefs +++ b/postfix/makedefs @@ -817,7 +817,7 @@ CCARGS="$CCARGS -DSNAPSHOT" # Non-production: needs thorough testing, or major changes are still # needed before the code stabilizes. -#CCARGS="$CCARGS -DNONPROD" +CCARGS="$CCARGS -DNONPROD" # Workaround: prepend Postfix include files before other include files. CCARGS="-I. -I../../include $CCARGS" diff --git a/postfix/man/man5/postconf.5 b/postfix/man/man5/postconf.5 index 4c303432a..e618bf228 100644 --- a/postfix/man/man5/postconf.5 +++ b/postfix/man/man5/postconf.5 @@ -4719,6 +4719,31 @@ this test the next time the client connects. .br .PP This feature is available in Postfix 2.8. +.SH postscreen_dnsbl_max_ttl (default: ${postscreen_dnsbl_ttl?{$postscreen_dnsbl_ttl}:{1h}}) +The maximum amount of time that \fBpostscreen\fR(8) will use the +result from a successful DNS blocklist or whitelist test before a +client IP address is required to pass that test again. If the DNS +reply specifies a shorter TTL value, that value will be used unless +it would be smaller than postscreen_dnsbl_min_ttl. +.PP +Specify a non\-zero time value (an integral value plus an optional +one\-letter suffix that specifies the time unit). Time units: s +(seconds), m (minutes), h (hours), d (days), w (weeks). +.PP +This feature is available in Postfix 3.1. The default setting +is backwards\-compatible with older Postfix versions. +.SH postscreen_dnsbl_min_ttl (default: 100s) +The minimum amount of time that \fBpostscreen\fR(8) will use the +result from a successful DNS blocklist or whitelist test before a +client IP address is required to pass that test again. If the DNS +reply specifies a larger TTL value, that value will be used unless +it would be larger than postscreen_dnsbl_max_ttl. +.PP +Specify a non\-zero time value (an integral value plus an optional +one\-letter suffix that specifies the time unit). Time units: s +(seconds), m (minutes), h (hours), d (days), w (weeks). +.PP +This feature is available in Postfix 3.1. .SH postscreen_dnsbl_reply_map (default: empty) A mapping from actual DNSBL domain name which includes a secret password, to the DNSBL domain name that postscreen will reply with @@ -4825,15 +4850,15 @@ the timeouts in the \fBdnsblog\fR(8) daemon which are defined by system This feature is available in Postfix 3.0. .SH postscreen_dnsbl_ttl (default: 1h) The amount of time that \fBpostscreen\fR(8) will use the result from -a successful DNS blocklist test. During this time, the client IP address -is excluded from this test. The default is relatively short, because a -good client can immediately talk to a real Postfix SMTP server. +a successful DNS blocklist or whitelist lookup before a client +IP address is required to pass that test again. .PP Specify a non\-zero time value (an integral value plus an optional one\-letter suffix that specifies the time unit). Time units: s (seconds), m (minutes), h (hours), d (days), w (weeks). .PP -This feature is available in Postfix 2.8. +This feature is available in Postfix 2.8\-3.0. It was +replaced by postscreen_dnsbl_max_ttl in Postfix 3.1. .SH postscreen_dnsbl_whitelist_threshold (default: 0) Allow a remote SMTP client to skip "before" and "after 220 greeting" protocol tests, based on its combined DNSBL score as diff --git a/postfix/man/man8/postscreen.8 b/postfix/man/man8/postscreen.8 index bf92fbe00..9494568c5 100644 --- a/postfix/man/man8/postscreen.8 +++ b/postfix/man/man8/postscreen.8 @@ -307,9 +307,14 @@ temporary whitelist entry before it is removed. .IP "\fBpostscreen_bare_newline_ttl (30d)\fR" The amount of time that \fBpostscreen\fR(8) will use the result from a successful "bare newline" SMTP protocol test. -.IP "\fBpostscreen_dnsbl_ttl (1h)\fR" -The amount of time that \fBpostscreen\fR(8) will use the result from -a successful DNS blocklist test. +.IP "\fBpostscreen_dnsbl_max_ttl (${postscreen_dnsbl_ttl?{$postscreen_dnsbl_ttl}:{1h}})\fR" +The maximum amount of time that \fBpostscreen\fR(8) will use the +result from a successful DNS blocklist or whitelist test before a +client IP address is required to pass that test again. +.IP "\fBpostscreen_dnsbl_min_ttl (100s)\fR" +The minimum amount of time that \fBpostscreen\fR(8) will use the +result from a successful DNS blocklist or whitelist test before a +client IP address is required to pass that test again. .IP "\fBpostscreen_greet_ttl (1d)\fR" The amount of time that \fBpostscreen\fR(8) will use the result from a successful PREGREET test. diff --git a/postfix/mantools/postlink b/postfix/mantools/postlink index fd55bb7a0..1755fea96 100755 --- a/postfix/mantools/postlink +++ b/postfix/mantools/postlink @@ -1004,6 +1004,8 @@ while (<>) { s;\bpostscreen_dnsbl_thresh[-]*\n* *[]*old\b;$&;g; s;\bpostscreen_dnsbl_whitelist_thresh[-]*\n* *[]*old\b;$&;g; s;\bpostscreen_dnsbl_action\b;$&;g; + s;\bpostscreen_dnsbl_max_ttl\b;$&;g; + s;\bpostscreen_dnsbl_min_ttl\b;$&;g; s;\bpostscreen_dnsbl_ttl\b;$&;g; s;\bpostscreen_dnsbl_timeout\b;$&;g; s;\bpostscreen_for[-]*\n*[ ]*bid[-]*\n* *[]*den_commands\b;$&;g; diff --git a/postfix/proto/postconf.proto b/postfix/proto/postconf.proto index d710b6d9a..65aee46a5 100644 --- a/postfix/proto/postconf.proto +++ b/postfix/proto/postconf.proto @@ -14305,16 +14305,44 @@ built-in SMTP protocol engine.

%PARAM postscreen_dnsbl_ttl 1h

The amount of time that postscreen(8) will use the result from -a successful DNS blocklist test. During this time, the client IP address -is excluded from this test. The default is relatively short, because a -good client can immediately talk to a real Postfix SMTP server. -

+a successful DNS blocklist or whitelist lookup before a client +IP address is required to pass that test again.

Specify a non-zero time value (an integral value plus an optional one-letter suffix that specifies the time unit). Time units: s (seconds), m (minutes), h (hours), d (days), w (weeks).

-

This feature is available in Postfix 2.8.

+

This feature is available in Postfix 2.8-3.0. It was +replaced by postscreen_dnsbl_max_ttl in Postfix 3.1.

+ +%PARAM postscreen_dnsbl_min_ttl 100s + +

The minimum amount of time that postscreen(8) will use the +result from a successful DNS blocklist or whitelist test before a +client IP address is required to pass that test again. If the DNS +reply specifies a larger TTL value, that value will be used unless +it would be larger than postscreen_dnsbl_max_ttl.

+ +

Specify a non-zero time value (an integral value plus an optional +one-letter suffix that specifies the time unit). Time units: s +(seconds), m (minutes), h (hours), d (days), w (weeks).

+ +

This feature is available in Postfix 3.1.

+ +%PARAM postscreen_dnsbl_max_ttl ${postscreen_dnsbl_ttl?{$postscreen_dnsbl_ttl}:{1h}} + +

The maximum amount of time that postscreen(8) will use the +result from a successful DNS blocklist or whitelist test before a +client IP address is required to pass that test again. If the DNS +reply specifies a shorter TTL value, that value will be used unless +it would be smaller than postscreen_dnsbl_min_ttl.

+ +

Specify a non-zero time value (an integral value plus an optional +one-letter suffix that specifies the time unit). Time units: s +(seconds), m (minutes), h (hours), d (days), w (weeks).

+ +

This feature is available in Postfix 3.1. The default setting +is backwards-compatible with older Postfix versions.

%PARAM postscreen_pipelining_action enforce diff --git a/postfix/src/dns/Makefile.in b/postfix/src/dns/Makefile.in index d8551b0f8..878fd692c 100644 --- a/postfix/src/dns/Makefile.in +++ b/postfix/src/dns/Makefile.in @@ -30,7 +30,33 @@ test: $(TESTPROG) tests: test dns_rr_to_pa_test dns_rr_to_sa_test dns_sa_to_rr_test \ dns_rr_eq_sa_test no-a-test no-aaaa-test no-mx-test \ - error-filter-test nullmx_test nxdomain_test mxonly_test + error-filter-test nullmx_test nxdomain_test mxonly_test \ + dnsbl_tests + +dnsbl_tests: \ + dnsbl_ttl_127.0.0.2_bind_plain_test \ + dnsbl_ttl_127.0.0.2_bind_ncache_test \ + dnsbl_ttl_127.0.0.2_priv_plain_test \ + dnsbl_ttl_127.0.0.2_priv_ncache_test \ + dnsbl_ttl_127.0.0.1_bind_plain_test \ + dnsbl_ttl_127.0.0.1_bind_ncache_test \ + dnsbl_ttl_127.0.0.1_priv_plain_test \ + dnsbl_ttl_127.0.0.1_priv_ncache_test + +DNSBL_NEXIST_REPLY_FIX = \ + sed -e 's/ [0-9][0-9]* IN SOA / TTL IN SOA /' \ + -e 's/len=[0-9][0-9]* /len=LEN /' \ + -e 's/nscount=[1-9][0-9]*/nscount=N/' \ + -e 's/ [0-9]* [0-9]* [0-9]* [0-9]* [0-9]*/ D D D D D/' + +DNSBL_EXIST_REPLY_FIX = \ + sed -e 's/ [0-9][0-9]* IN A / TTL IN A /' \ + -e 's/len=[0-9][0-9]* /len=LEN /' \ + -e 's/ancount=[1-9][0-9]*/ancount=N/' \ + -e 's/nscount=[1-9][0-9]*/nscount=N/' \ + -e 's/ [0-9]* [0-9]* [0-9]* [0-9]* [0-9]*/ D D D D D/' \ + -e 's/127.0.0.[0-9]*$$/127.0.0.D/' \ + | uniq root_tests: @@ -134,6 +160,94 @@ mxonly_test: test_dns_lookup mxonly_test.ref diff mxonly_test.ref mxonly_test.tmp rm -f mxonly_test.tmp +# Non-existent record, libbind API, RFC 2308 disabled. + +dnsbl_ttl_127.0.0.1_bind_plain_test: test_dns_lookup dnsbl_ttl_127.0.0.1_bind_plain.ref + (set -e; \ + $(SHLIB_ENV) ./test_dns_lookup a 1.0.0.127.zen.spamhaus.org; \ + $(SHLIB_ENV) ./test_dns_lookup a 1.0.0.127.b.barracudacentral.org; \ + $(SHLIB_ENV) ./test_dns_lookup a 1.0.0.127.bl.spamcop.net; \ + ) 2>&1 | $(DNSBL_NEXIST_REPLY_FIX) >dnsbl_ttl_127.0.0.1_bind_plain.tmp + diff dnsbl_ttl_127.0.0.1_bind_plain.ref dnsbl_ttl_127.0.0.1_bind_plain.tmp + rm -f dnsbl_ttl_127.0.0.1_bind_plain.tmp + +# Non-existent record, private API, RFC 2308 disabled. + +dnsbl_ttl_127.0.0.1_priv_plain_test: test_dns_lookup dnsbl_ttl_127.0.0.1_bind_plain.ref + (set -e; \ + $(SHLIB_ENV) ./test_dns_lookup -p a 1.0.0.127.zen.spamhaus.org; \ + $(SHLIB_ENV) ./test_dns_lookup -p a 1.0.0.127.b.barracudacentral.org; \ + $(SHLIB_ENV) ./test_dns_lookup -p a 1.0.0.127.bl.spamcop.net; \ + ) 2>&1 | $(DNSBL_NEXIST_REPLY_FIX) >dnsbl_ttl_127.0.0.1_priv_plain.tmp + diff dnsbl_ttl_127.0.0.1_bind_plain.ref dnsbl_ttl_127.0.0.1_priv_plain.tmp + rm -f dnsbl_ttl_127.0.0.1_priv_plain.tmp + +# Non-existent record, libbind API, RFC 2308 enabled. + +dnsbl_ttl_127.0.0.1_bind_ncache_test: test_dns_lookup dnsbl_ttl_127.0.0.1_bind_ncache.ref + (set -e; \ + $(SHLIB_ENV) ./test_dns_lookup -n a 1.0.0.127.zen.spamhaus.org; \ + $(SHLIB_ENV) ./test_dns_lookup -n a 1.0.0.127.b.barracudacentral.org; \ + $(SHLIB_ENV) ./test_dns_lookup -n a 1.0.0.127.bl.spamcop.net; \ + ) 2>&1 | $(DNSBL_NEXIST_REPLY_FIX) >dnsbl_ttl_127.0.0.1_bind_ncache.tmp + diff dnsbl_ttl_127.0.0.1_bind_ncache.ref dnsbl_ttl_127.0.0.1_bind_ncache.tmp + rm -f dnsbl_ttl_127.0.0.1_bind_ncache.tmp + +# Non-existent record, private API, RFC 2308 enabled. + +dnsbl_ttl_127.0.0.1_priv_ncache_test: test_dns_lookup dnsbl_ttl_127.0.0.1_bind_ncache.ref + (set -e; \ + $(SHLIB_ENV) ./test_dns_lookup -n -p a 1.0.0.127.zen.spamhaus.org; \ + $(SHLIB_ENV) ./test_dns_lookup -n -p a 1.0.0.127.b.barracudacentral.org; \ + $(SHLIB_ENV) ./test_dns_lookup -n -p a 1.0.0.127.bl.spamcop.net; \ + ) 2>&1 | $(DNSBL_NEXIST_REPLY_FIX) >dnsbl_ttl_127.0.0.1_priv_ncache.tmp + diff dnsbl_ttl_127.0.0.1_bind_ncache.ref dnsbl_ttl_127.0.0.1_priv_ncache.tmp + rm -f dnsbl_ttl_127.0.0.1_priv_ncache.tmp + +# Existing record, libbind API, RFC 2308 disabled. + +dnsbl_ttl_127.0.0.2_bind_plain_test: test_dns_lookup dnsbl_ttl_127.0.0.2_bind_plain.ref + (set -e; \ + $(SHLIB_ENV) ./test_dns_lookup a 2.0.0.127.zen.spamhaus.org; \ + $(SHLIB_ENV) ./test_dns_lookup a 2.0.0.127.b.barracudacentral.org; \ + $(SHLIB_ENV) ./test_dns_lookup a 2.0.0.127.bl.spamcop.net; \ + ) 2>&1 | $(DNSBL_EXIST_REPLY_FIX) >dnsbl_ttl_127.0.0.2_bind_plain.tmp + diff dnsbl_ttl_127.0.0.2_bind_plain.ref dnsbl_ttl_127.0.0.2_bind_plain.tmp + rm -f dnsbl_ttl_127.0.0.2_bind_plain.tmp + +# Existing record, private API, RFC 2308 disabled. + +dnsbl_ttl_127.0.0.2_priv_plain_test: test_dns_lookup dnsbl_ttl_127.0.0.2_bind_plain.ref + (set -e; \ + $(SHLIB_ENV) ./test_dns_lookup -p a 2.0.0.127.zen.spamhaus.org; \ + $(SHLIB_ENV) ./test_dns_lookup -p a 2.0.0.127.b.barracudacentral.org; \ + $(SHLIB_ENV) ./test_dns_lookup -p a 2.0.0.127.bl.spamcop.net; \ + ) 2>&1 | $(DNSBL_EXIST_REPLY_FIX) >dnsbl_ttl_127.0.0.2_priv_plain.tmp + diff dnsbl_ttl_127.0.0.2_bind_plain.ref dnsbl_ttl_127.0.0.2_priv_plain.tmp + rm -f dnsbl_ttl_127.0.0.2_priv_plain.tmp + +# Existing record, libbind API, RFC 2308 enabled. + +dnsbl_ttl_127.0.0.2_bind_ncache_test: test_dns_lookup dnsbl_ttl_127.0.0.2_bind_plain.ref + (set -e; \ + $(SHLIB_ENV) ./test_dns_lookup -n a 2.0.0.127.zen.spamhaus.org; \ + $(SHLIB_ENV) ./test_dns_lookup -n a 2.0.0.127.b.barracudacentral.org; \ + $(SHLIB_ENV) ./test_dns_lookup -n a 2.0.0.127.bl.spamcop.net; \ + ) 2>&1 | $(DNSBL_EXIST_REPLY_FIX) >dnsbl_ttl_127.0.0.2_bind_ncache.tmp + diff dnsbl_ttl_127.0.0.2_bind_plain.ref dnsbl_ttl_127.0.0.2_bind_ncache.tmp + rm -f dnsbl_ttl_127.0.0.2_bind_ncache.tmp + +# Existing record, private API, RFC 2308 enabled. + +dnsbl_ttl_127.0.0.2_priv_ncache_test: test_dns_lookup dnsbl_ttl_127.0.0.2_bind_plain.ref + (set -e; \ + $(SHLIB_ENV) ./test_dns_lookup -n -p a 2.0.0.127.zen.spamhaus.org; \ + $(SHLIB_ENV) ./test_dns_lookup -n -p a 2.0.0.127.b.barracudacentral.org; \ + $(SHLIB_ENV) ./test_dns_lookup -n -p a 2.0.0.127.bl.spamcop.net; \ + ) 2>&1 | $(DNSBL_EXIST_REPLY_FIX) >dnsbl_ttl_127.0.0.2_priv_ncache.tmp + diff dnsbl_ttl_127.0.0.2_bind_plain.ref dnsbl_ttl_127.0.0.2_priv_ncache.tmp + rm -f dnsbl_ttl_127.0.0.2_priv_ncache.tmp + printfck: $(OBJS) $(PROG) rm -rf printfck mkdir printfck diff --git a/postfix/src/dns/dns.h b/postfix/src/dns/dns.h index 7cfc58108..2938ac3d6 100644 --- a/postfix/src/dns/dns.h +++ b/postfix/src/dns/dns.h @@ -212,15 +212,20 @@ extern int dns_rr_eq_sa(DNS_RR *, struct sockaddr *); /* * dns_lookup.c */ -extern int dns_lookup_r(const char *, unsigned, unsigned, DNS_RR **, - VSTRING *, VSTRING *, int *); +extern int dns_lookup_x(const char *, unsigned, unsigned, DNS_RR **, + VSTRING *, VSTRING *, int *, unsigned); extern int dns_lookup_rl(const char *, unsigned, DNS_RR **, VSTRING *, VSTRING *, int *, int,...); extern int dns_lookup_rv(const char *, unsigned, DNS_RR **, VSTRING *, VSTRING *, int *, int, unsigned *); +extern int dns_ncache_ttl_fix_enable; #define dns_lookup(name, type, rflags, list, fqdn, why) \ - dns_lookup_r((name), (type), (rflags), (list), (fqdn), (why), (int *) 0) + dns_lookup_x((name), (type), (rflags), (list), (fqdn), (why), (int *) 0, \ + (unsigned) 0) +#define dns_lookup_r(name, type, rflags, list, fqdn, why, rcode) \ + dns_lookup_x((name), (type), (rflags), (list), (fqdn), (why), (rcode), \ + (unsigned) 0) #define dns_lookup_l(name, rflags, list, fqdn, why, lflags, ...) \ dns_lookup_rl((name), (rflags), (list), (fqdn), (why), (int *) 0, \ (lflags), __VA_ARGS__) @@ -235,6 +240,7 @@ extern int dns_lookup_rv(const char *, unsigned, DNS_RR **, VSTRING *, #define DNS_REQ_FLAG_STOP_INVAL (1<<1) #define DNS_REQ_FLAG_STOP_NULLMX (1<<2) #define DNS_REQ_FLAG_STOP_MX_POLICY (1<<3) +#define DNS_REQ_FLAG_NCACHE_TTL (1<<4) #define DNS_REQ_FLAG_NONE (0) /* diff --git a/postfix/src/dns/dns_lookup.c b/postfix/src/dns/dns_lookup.c index 3838ea6b1..a1ad04a31 100644 --- a/postfix/src/dns/dns_lookup.c +++ b/postfix/src/dns/dns_lookup.c @@ -32,6 +32,8 @@ /* int lflags; /* unsigned *ltype; /* AUXILIARY FUNCTIONS +/* int dns_ncache_ttl_fix_enable; +/* /* int dns_lookup_r(name, type, rflags, list, fqdn, why, rcode) /* const char *name; /* unsigned type; @@ -62,6 +64,16 @@ /* int *rcode; /* int lflags; /* unsigned *ltype; +/* +/* int dns_lookup_x(name, type, rflags, list, fqdn, why, rcode, lflags) +/* const char *name; +/* unsigned type; +/* unsigned rflags; +/* DNS_RR **list; +/* VSTRING *fqdn; +/* VSTRING *why; +/* int *rcode; +/* unsigned lflags; /* DESCRIPTION /* dns_lookup() looks up DNS resource records. When requested to /* look up data other than type CNAME, it will follow a limited @@ -74,8 +86,14 @@ /* dns_lookup_l() and dns_lookup_v() allow the user to specify /* a list of resource types. /* -/* dns_lookup_r(), dns_lookup_rl() and dns_lookup_rv() provide -/* additional information. +/* dns_lookup_x, dns_lookup_r(), dns_lookup_rl() and dns_lookup_rv() +/* accept or return additional information. +/* +/* The dns_ncache_ttl_fix_enable variable controls a workaround +/* for res_search(3) implementations that break the +/* DNS_REQ_FLAG_NCACHE_TTL feature. The workaround does not +/* support EDNS0 or DNSSEC, but it should be sufficient for +/* DNSBL/DNSWL lookups. /* INPUTS /* .ad /* .fi @@ -100,25 +118,33 @@ /* implement DNSSEC. /* .RE /* .IP lflags -/* Multi-type request control for dns_lookup_l() and dns_lookup_v(). -/* For convenience, DNS_REQ_FLAG_NONE requests no special -/* processing. Invoke dns_lookup() for all specified resource -/* record types in the specified order, and merge their results. +/* Flags that control the operation of the dns_lookup*() +/* functions. DNS_REQ_FLAG_NONE requests no special processing. /* Otherwise, specify one or more of the following: /* .RS /* .IP DNS_REQ_FLAG_STOP_INVAL +/* This flag is used by dns_lookup_l() and dns_lookup_v(). /* Invoke dns_lookup() for the resource types in the order as /* specified, and return when dns_lookup() returns DNS_INVAL. /* .IP DNS_REQ_FLAG_STOP_NULLMX +/* This flag is used by dns_lookup_l() and dns_lookup_v(). /* Invoke dns_lookup() for the resource types in the order as /* specified, and return when dns_lookup() returns DNS_NULLMX. /* .IP DNS_REQ_FLAG_STOP_MX_POLICY +/* This flag is used by dns_lookup_l() and dns_lookup_v(). /* Invoke dns_lookup() for the resource types in the order as /* specified, and return when dns_lookup() returns DNS_POLICY /* for an MX query. /* .IP DNS_REQ_FLAG_STOP_OK +/* This flag is used by dns_lookup_l() and dns_lookup_v(). /* Invoke dns_lookup() for the resource types in the order as /* specified, and return when dns_lookup() returns DNS_OK. +/* .IP DNS_REQ_FLAG_NCACHE_TTL +/* When the lookup result status is DNS_NOTFOUND, return the +/* SOA record(s) from the authority section in the reply, if +/* available. The per-record record TTL specifies how long the +/* DNS_NOTFOUND answer is valid. The caller should pass the +/* record(s) to dns_rr_free(). /* .RE /* .IP ltype /* The resource record types to be looked up. In the case of @@ -215,6 +241,7 @@ */ #define DEF_DNS_REPLY_SIZE 4096 /* in case we're using TCP */ #define MAX_DNS_REPLY_SIZE 65536 /* in case we're using TCP */ +#define MAX_DNS_QUERY_SIZE 2048 /* XXX */ typedef struct DNS_REPLY { unsigned char *buf; /* raw reply data */ @@ -223,22 +250,161 @@ typedef struct DNS_REPLY { int dnssec_ad; /* DNSSEC AD bit */ int query_count; /* number of queries */ int answer_count; /* number of answers */ + int auth_count; /* number of authority records */ unsigned char *query_start; /* start of query data */ unsigned char *answer_start; /* start of answer data */ unsigned char *end; /* first byte past reply */ } DNS_REPLY; + /* + * Test/set primitives to determine if the reply buffer contains a server + * response. We use this when the caller requests DNS_REQ_FLAG_NCACHE_TTL, + * and the DNS server replies that the requested record does not exist. + */ +#define TEST_HAVE_DNS_REPLY_PACKET(r) ((r)->end > (r)->buf) +#define SET_HAVE_DNS_REPLY_PACKET(r, l) ((r)->end = (r)->buf + (l)) +#define SET_NO_DNS_REPLY_PACKET(r) ((r)->end = (r)->buf) + #define INET_ADDR_LEN 4 /* XXX */ #define INET6_ADDR_LEN 16 /* XXX */ + /* + * To improve postscreen's whitelisting support, we need to know how long a + * DNSBL "not found" answer is valid. The 2010 implementation assumed it was + * valid for 3600 seconds. That is too long by 2015 standards. + * + * Instead of guessing, Postfix 3.1 and later implement RFC 2308 (DNS NCACHE), + * where a DNS server provides the TTL of a "not found" response as the TTL + * of an SOA record in the authority section. + * + * Unfortunately, the res_search() and res_query() API gets in the way. These + * functions overload their result value, the server reply length, and + * return -1 when the requested record does not exist. With libbind-based + * res_search() implementations, the server response is still available in a + * caller-supplied buffer, thanks to a promise made by re_send(). With some + * creativity we can still use the server response. + * + * If this should stop working (for example, res_search() does not call + * res_send(), but some non-libbind implementation that updates the + * caller-supplied buffer only when the requested record exists), then we + * have a way out by setting the dns_ncache_ttl_fix_enable variable. This + * enables a limited res_query() clone that should be sufficient for DNSBL / + * DNSWL lookups. + * + * The libunbound API does not comingle the reply length and reply status + * information, but that will have to wait until it is safe to make + * libunbound a mandatory dependency for Postfix. + */ +int dns_ncache_ttl_fix_enable = 0; + +/* dns_res_query - res_query() that can return negative replies */ + +static int dns_res_query(const char *name, int class, int type, + unsigned char *answer, int anslen) +{ + unsigned char msg_buf[MAX_DNS_QUERY_SIZE]; + HEADER *reply_header = (HEADER *) answer; + int len; + + /* + * Differences with res_query(): + * + * - This function returns a positive server reply length not only in case + * of success, but in all cases where a server reply is available and + * passes the preliminary checks in res_send(). + * + * - This function clears h_errno in case of success. The caller must use + * h_errno instead of the return value to decide if the lookup was + * successful. + * + * - No support for EDNS0 and DNSSEC (including turning off EDNS0 after + * error). That should be sufficient for DNSXL lookups where the reply + * contains a small number of IP addresses. + */ + reply_header->rcode = NOERROR; + +#define NO_MKQUERY_DATA_BUF ((unsigned char *) 0) +#define NO_MKQUERY_DATA_LEN ((int) 0) +#define NO_MKQUERY_NEWRR ((unsigned char *) 0) + + if ((len = res_mkquery(QUERY, name, class, type, NO_MKQUERY_DATA_BUF, + NO_MKQUERY_DATA_LEN, NO_MKQUERY_NEWRR, + msg_buf, sizeof(msg_buf))) < 0) { + SET_H_ERRNO(NO_RECOVERY); + if (msg_verbose) + msg_info("res_mkquery() failed"); + return (len); + } else if ((len = res_send(msg_buf, len, answer, anslen)) < 0) { + SET_H_ERRNO(TRY_AGAIN); + if (msg_verbose) + msg_info("res_send() failed"); + return (len); + } else { + switch (reply_header->rcode) { + case NXDOMAIN: + SET_H_ERRNO(HOST_NOT_FOUND); + break; + case NOERROR: + if (reply_header->ancount != 0) + SET_H_ERRNO(0); + else + SET_H_ERRNO(NO_DATA); + break; + case SERVFAIL: + SET_H_ERRNO(TRY_AGAIN); + break; + default: + SET_H_ERRNO(NO_RECOVERY); + break; + } + return (len); + } +} + +/* dns_res_search - res_search() that can return negative replies */ + +static int dns_res_search(const char *name, int class, int type, + unsigned char *answer, int anslen, int keep_notfound) +{ + int len; + + /* + * Differences with res_search(): + * + * - With a non-zero keep_notfound argument, this function returns a + * positive server reply length not only in case of success, but also in + * case of a "notfound" reply status. The keep_notfound argument is + * usually zero, which allows us to avoid an unnecessary memset() call in + * the most common use case. + * + * - This function clears h_errno in case of success. The caller must use + * h_errno instead of the return value to decide if a lookup was + * successful. + */ +#define NOT_FOUND_H_ERRNO(he) ((he) == HOST_NOT_FOUND || (he) == NO_DATA) + + if (keep_notfound) + /* Prepare for returning a null-padded server reply. */ + memset(answer, 0, anslen); + len = res_query(name, class, type, answer, anslen); + if (len > 0) { + SET_H_ERRNO(0); + } else if (keep_notfound && NOT_FOUND_H_ERRNO(h_errno)) { + /* Expect to return a null-padded server reply. */ + len = anslen; + } + return (len); +} + /* dns_query - query name server and pre-parse the reply */ -static int dns_query(const char *name, int type, int flags, - DNS_REPLY *reply, VSTRING *why) +static int dns_query(const char *name, int type, unsigned flags, + DNS_REPLY *reply, VSTRING *why, unsigned lflags) { HEADER *reply_header; int len; unsigned long saved_options; + int keep_notfound = (lflags & DNS_REQ_FLAG_NCACHE_TTL); /* * Initialize the reply buffer. @@ -289,12 +455,18 @@ static int dns_query(const char *name, int type, int flags, for (;;) { _res.options &= ~saved_options; _res.options |= flags; - len = res_search((char *) name, C_IN, type, reply->buf, reply->buf_len); + if (keep_notfound && dns_ncache_ttl_fix_enable) { + len = dns_res_query((char *) name, C_IN, type, reply->buf, + reply->buf_len); + } else { + len = dns_res_search((char *) name, C_IN, type, reply->buf, + reply->buf_len, keep_notfound); + } _res.options &= ~flags; _res.options |= saved_options; reply_header = (HEADER *) reply->buf; reply->rcode = reply_header->rcode; - if (len < 0) { + if (h_errno != 0) { if (why) vstring_sprintf(why, "Host or domain name not found. " "Name service error for name=%s type=%s: %s", @@ -307,13 +479,17 @@ static int dns_query(const char *name, int type, int flags, return (DNS_FAIL); case HOST_NOT_FOUND: case NO_DATA: + if (keep_notfound) + break; + SET_NO_DNS_REPLY_PACKET(reply); return (DNS_NOTFOUND); default: return (DNS_RETRY); } + } else { + if (msg_verbose) + msg_info("dns_query: %s (%s): OK", name, dns_strtype(type)); } - if (msg_verbose) - msg_info("dns_query: %s (%s): OK", name, dns_strtype(type)); if (reply_header->tc == 0 || reply->buf_len >= MAX_DNS_REPLY_SIZE) break; @@ -322,6 +498,14 @@ static int dns_query(const char *name, int type, int flags, reply->buf_len *= 2; } + /* + * Future proofing. If this reaches the panic call, then some code change + * introduced a bug. + */ + if (len < 0) + msg_panic("dns_query: bad length %d (h_errno=%s)", + len, dns_strerror(h_errno)); + /* * Paranoia. */ @@ -340,12 +524,28 @@ static int dns_query(const char *name, int type, int flags, #else reply->dnssec_ad = 0; #endif - reply->end = reply->buf + len; + SET_HAVE_DNS_REPLY_PACKET(reply, len); reply->query_start = reply->buf + sizeof(HEADER); reply->answer_start = 0; reply->query_count = ntohs(reply_header->qdcount); reply->answer_count = ntohs(reply_header->ancount); - return (DNS_OK); + reply->auth_count = ntohs(reply_header->nscount); + if (msg_verbose) + msg_info("dns_query: reply len=%d ancount=%d nscount=%d", + len, reply->answer_count, reply->auth_count); + + /* + * Future proofing. If this reaches the panic call, then some code change + * introduced a bug. + */ + if (h_errno == 0) { + return (DNS_OK); + } else if (keep_notfound) { + return (DNS_NOTFOUND); + } else { + msg_panic("dns_query: unexpected reply status: %s", + dns_strerror(h_errno)); + } } /* dns_skip_query - skip query data in name server reply */ @@ -354,7 +554,6 @@ static int dns_skip_query(DNS_REPLY *reply) { int query_count = reply->query_count; unsigned char *pos = reply->query_start; - char temp[DNS_NAME_LEN]; int len; /* @@ -364,7 +563,7 @@ static int dns_skip_query(DNS_REPLY *reply) while (query_count-- > 0) { if (pos >= reply->end) return DNS_RETRY; - len = dn_expand(reply->buf, reply->end, pos, temp, DNS_NAME_LEN); + len = dn_skipname(pos, reply->end); if (len < 0) return (DNS_RETRY); pos += len + QFIXEDSZ; @@ -441,6 +640,8 @@ static int dns_get_rr(DNS_RR **list, const char *orig_name, DNS_REPLY *reply, { char temp[DNS_NAME_LEN]; char *tempbuf = temp; + UINT32_TYPE soa_buf[5]; + int comp_len; ssize_t data_len; unsigned pref = 0; unsigned char *src; @@ -532,6 +733,35 @@ static int dns_get_rr(DNS_RR **list, const char *orig_name, DNS_REPLY *reply, data_len = fixed->length; tempbuf = (char *) pos; break; + + /* + * We use the SOA record TTL to determine the negative reply TTL. We + * save the time fields in the SOA record for debugging, but for now + * we don't bother saving the source host and mailbox information, as + * that would require changes to the DNS_RR structure. See also code + * in dns_strrecord(). + */ + case T_SOA: + comp_len = dn_skipname(pos, reply->end); + if (comp_len < 0) + return (DNS_RETRY); + pos += comp_len; + comp_len = dn_skipname(pos, reply->end); + if (comp_len < 0) + return (DNS_RETRY); + pos += comp_len; + if (reply->end - pos < sizeof(soa_buf)) { + msg_warn("extract_answer: bad SOA length: %d", fixed->length); + return (DNS_RETRY); + } + GETLONG(soa_buf[0], pos); /* Serial */ + GETLONG(soa_buf[1], pos); /* Refresh */ + GETLONG(soa_buf[2], pos); /* Retry */ + GETLONG(soa_buf[3], pos); /* Expire */ + GETLONG(soa_buf[4], pos); /* Ncache TTL */ + tempbuf = (char *) soa_buf; + data_len = sizeof(soa_buf); + break; } *list = dns_rr_create(orig_name, rr_name, fixed->type, fixed->class, fixed->ttl, pref, tempbuf, data_len); @@ -576,8 +806,6 @@ static int dns_get_answer(const char *orig_name, DNS_REPLY *reply, int type, if ((status = dns_skip_query(reply)) < 0) return (status); pos = reply->answer_start; - if (rrlist) - *rrlist = 0; /* * Either this, or use a GOTO for emergency exits. The purpose is to @@ -663,11 +891,11 @@ static int dns_get_answer(const char *orig_name, DNS_REPLY *reply, int type, return (not_found_status); } -/* dns_lookup_r - DNS lookup user interface */ +/* dns_lookup_x - DNS lookup user interface */ -int dns_lookup_r(const char *name, unsigned type, unsigned flags, +int dns_lookup_x(const char *name, unsigned type, unsigned flags, DNS_RR **rrlist, VSTRING *fqdn, VSTRING *why, - int *rcode) + int *rcode, unsigned lflags) { char cname[DNS_NAME_LEN]; int c_len = sizeof(cname); @@ -677,6 +905,13 @@ int dns_lookup_r(const char *name, unsigned type, unsigned flags, int maybe_secure = 1; /* Query name presumed secure */ const char *orig_name = name; + /* + * Reset results early. DNS_OK is not the only status that returns + * resource records; DNS_NOTFOUND will do that too, if requested. + */ + if (rrlist) + *rrlist = 0; + /* * DJBDNS produces a bogus A record when given a numerical hostname. */ @@ -685,6 +920,8 @@ int dns_lookup_r(const char *name, unsigned type, unsigned flags, vstring_sprintf(why, "Name service error for %s: invalid host or domain name", name); + if (rcode) + *rcode = NXDOMAIN; SET_H_ERRNO(HOST_NOT_FOUND); return (DNS_NOTFOUND); } @@ -697,6 +934,8 @@ int dns_lookup_r(const char *name, unsigned type, unsigned flags, vstring_sprintf(why, "Name service error for %s: invalid host or domain name", name); + if (rcode) + *rcode = NXDOMAIN; SET_H_ERRNO(HOST_NOT_FOUND); return (DNS_NOTFOUND); } @@ -710,11 +949,25 @@ int dns_lookup_r(const char *name, unsigned type, unsigned flags, /* * Perform the DNS lookup, and pre-parse the name server reply. */ - status = dns_query(name, type, flags, &reply, why); + status = dns_query(name, type, flags, &reply, why, lflags); if (rcode) *rcode = reply.rcode; - if (status != DNS_OK) + if (status != DNS_OK) { + + /* + * If the record does not exist, and we have a copy of the server + * response, try to extract the negative caching TTL for the SOA + * record in the authority section. DO NOT return an error if an + * SOA record is malformed. + */ + if (status == DNS_NOTFOUND && TEST_HAVE_DNS_REPLY_PACKET(&reply) + && reply.auth_count > 0) { + reply.answer_count = reply.auth_count; /* XXX TODO: Fix API */ + (void) dns_get_answer(orig_name, &reply, T_SOA, rrlist, fqdn, + cname, c_len, &maybe_secure); + } return (status); + } /* * Extract resource records of the requested type. Pick up CNAME @@ -733,10 +986,10 @@ int dns_lookup_r(const char *name, unsigned type, unsigned flags, if (why) vstring_sprintf(why, "Domain %s does not accept mail (nullMX)", name); - h_errno = NO_DATA; + SET_H_ERRNO(NO_DATA); return (status); case DNS_OK: - if (dns_rr_filter_maps) { + if (rrlist && dns_rr_filter_maps) { if (dns_rr_filter_execute(rrlist) < 0) { if (why) vstring_sprintf(why, @@ -790,6 +1043,7 @@ int dns_lookup_rl(const char *name, unsigned flags, DNS_RR **rrlist, int hpref_status = INT_MIN; VSTRING *hpref_rtext = 0; int hpref_rcode; + int hpref_h_errno; DNS_RR *rr; /* Save intermediate highest-priority result. */ @@ -801,6 +1055,7 @@ int dns_lookup_rl(const char *name, unsigned flags, DNS_RR **rrlist, vstring_strcpy(hpref_rtext ? hpref_rtext : \ (hpref_rtext = vstring_alloc(VSTRING_LEN(why))), \ vstring_str(why)); \ + hpref_h_errno = h_errno; \ } while (0) /* Restore intermediate highest-priority result. */ @@ -810,6 +1065,7 @@ int dns_lookup_rl(const char *name, unsigned flags, DNS_RR **rrlist, *rcode = hpref_rcode; \ if (why && status != DNS_OK) \ vstring_strcpy(why, vstring_str(hpref_rtext)); \ + SET_H_ERRNO(hpref_h_errno); \ } while (0) if (rrlist) @@ -820,8 +1076,8 @@ int dns_lookup_rl(const char *name, unsigned flags, DNS_RR **rrlist, if (msg_verbose) msg_info("lookup %s type %s flags %d", name, dns_strtype(type), flags); - status = dns_lookup_r(name, type, flags, rrlist ? &rr : (DNS_RR **) 0, - fqdn, why, rcode); + status = dns_lookup_x(name, type, flags, rrlist ? &rr : (DNS_RR **) 0, + fqdn, why, rcode, lflags); if (status == DNS_OK) { if (rrlist) *rrlist = dns_rr_append(*rrlist, rr); @@ -862,6 +1118,7 @@ int dns_lookup_rv(const char *name, unsigned flags, DNS_RR **rrlist, int hpref_status = INT_MIN; VSTRING *hpref_rtext = 0; int hpref_rcode; + int hpref_h_errno; DNS_RR *rr; if (rrlist) @@ -871,8 +1128,8 @@ int dns_lookup_rv(const char *name, unsigned flags, DNS_RR **rrlist, if (msg_verbose) msg_info("lookup %s type %s flags %d", name, dns_strtype(type), flags); - status = dns_lookup_r(name, type, flags, rrlist ? &rr : (DNS_RR **) 0, - fqdn, why, rcode); + status = dns_lookup_x(name, type, flags, rrlist ? &rr : (DNS_RR **) 0, + fqdn, why, rcode, lflags); if (status == DNS_OK) { if (rrlist) *rrlist = dns_rr_append(*rrlist, rr); diff --git a/postfix/src/dns/dns_strrecord.c b/postfix/src/dns/dns_strrecord.c index 318cdb90b..370850917 100644 --- a/postfix/src/dns/dns_strrecord.c +++ b/postfix/src/dns/dns_strrecord.c @@ -29,6 +29,7 @@ /* System library. */ #include +#include /* memcpy */ /* Utility library. */ @@ -45,6 +46,7 @@ char *dns_strrecord(VSTRING *buf, DNS_RR *rr) { const char myname[] = "dns_strrecord"; MAI_HOSTADDR_STR host; + UINT32_TYPE soa_buf[5]; vstring_sprintf(buf, "%s. %u IN %s ", rr->rname, rr->ttl, dns_strtype(rr->type)); @@ -88,6 +90,19 @@ char *dns_strrecord(VSTRING *buf, DNS_RR *rr) } else { vstring_sprintf_append(buf, "[truncated record]"); } + + /* + * We use the SOA record TTL to determine the negative reply TTL. We + * save the time fields in the SOA record for debugging, but for now + * we don't bother saving the source host and mailbox information, as + * that would require changes to the DNS_RR structure. See also code + * in dns_get_rr(). + */ + case T_SOA: + memcpy(soa_buf, rr->data, sizeof(soa_buf)); + vstring_sprintf_append(buf, "- - %u %u %u %u %u", + soa_buf[0], soa_buf[1], soa_buf[2], + soa_buf[3], soa_buf[4]); break; default: msg_fatal("%s: don't know how to print type %s", diff --git a/postfix/src/dns/dnsbl_ttl_127.0.0.1_bind_ncache.ref b/postfix/src/dns/dnsbl_ttl_127.0.0.1_bind_ncache.ref new file mode 100644 index 000000000..f4259d7f7 --- /dev/null +++ b/postfix/src/dns/dnsbl_ttl_127.0.0.1_bind_ncache.ref @@ -0,0 +1,15 @@ +./test_dns_lookup: dns_query: 1.0.0.127.zen.spamhaus.org (A): Host not found +./test_dns_lookup: dns_query: reply len=LEN ancount=0 nscount=N +./test_dns_lookup: dns_get_answer: type SOA for zen.spamhaus.org +./test_dns_lookup: warning: Host or domain name not found. Name service error for name=1.0.0.127.zen.spamhaus.org type=A: Host not found (rcode=3) +1.0.0.127.zen.spamhaus.org: fqdn: zen.spamhaus.org +ad: 0, rr: zen.spamhaus.org. TTL IN SOA - - D D D D D +./test_dns_lookup: dns_query: 1.0.0.127.b.barracudacentral.org (A): Host not found +./test_dns_lookup: dns_query: reply len=LEN ancount=0 nscount=0 +./test_dns_lookup: warning: Host or domain name not found. Name service error for name=1.0.0.127.b.barracudacentral.org type=A: Host not found (rcode=3) +./test_dns_lookup: dns_query: 1.0.0.127.bl.spamcop.net (A): Host not found +./test_dns_lookup: dns_query: reply len=LEN ancount=0 nscount=N +./test_dns_lookup: dns_get_answer: type SOA for bl.spamcop.net +./test_dns_lookup: warning: Host or domain name not found. Name service error for name=1.0.0.127.bl.spamcop.net type=A: Host not found (rcode=3) +1.0.0.127.bl.spamcop.net: fqdn: bl.spamcop.net +ad: 0, rr: bl.spamcop.net. TTL IN SOA - - D D D D D diff --git a/postfix/src/dns/dnsbl_ttl_127.0.0.1_bind_plain.ref b/postfix/src/dns/dnsbl_ttl_127.0.0.1_bind_plain.ref new file mode 100644 index 000000000..9a57f881b --- /dev/null +++ b/postfix/src/dns/dnsbl_ttl_127.0.0.1_bind_plain.ref @@ -0,0 +1,6 @@ +./test_dns_lookup: dns_query: 1.0.0.127.zen.spamhaus.org (A): Host not found +./test_dns_lookup: warning: Host or domain name not found. Name service error for name=1.0.0.127.zen.spamhaus.org type=A: Host not found (rcode=3) +./test_dns_lookup: dns_query: 1.0.0.127.b.barracudacentral.org (A): Host not found +./test_dns_lookup: warning: Host or domain name not found. Name service error for name=1.0.0.127.b.barracudacentral.org type=A: Host not found (rcode=3) +./test_dns_lookup: dns_query: 1.0.0.127.bl.spamcop.net (A): Host not found +./test_dns_lookup: warning: Host or domain name not found. Name service error for name=1.0.0.127.bl.spamcop.net type=A: Host not found (rcode=3) diff --git a/postfix/src/dns/dnsbl_ttl_127.0.0.2_bind_plain.ref b/postfix/src/dns/dnsbl_ttl_127.0.0.2_bind_plain.ref new file mode 100644 index 000000000..2308937ae --- /dev/null +++ b/postfix/src/dns/dnsbl_ttl_127.0.0.2_bind_plain.ref @@ -0,0 +1,15 @@ +./test_dns_lookup: dns_query: 2.0.0.127.zen.spamhaus.org (A): OK +./test_dns_lookup: dns_query: reply len=LEN ancount=N nscount=N +./test_dns_lookup: dns_get_answer: type A for 2.0.0.127.zen.spamhaus.org +2.0.0.127.zen.spamhaus.org: fqdn: 2.0.0.127.zen.spamhaus.org +ad: 0, rr: 2.0.0.127.zen.spamhaus.org. TTL IN A 127.0.0.D +./test_dns_lookup: dns_query: 2.0.0.127.b.barracudacentral.org (A): OK +./test_dns_lookup: dns_query: reply len=LEN ancount=N nscount=N +./test_dns_lookup: dns_get_answer: type A for 2.0.0.127.b.barracudacentral.org +2.0.0.127.b.barracudacentral.org: fqdn: 2.0.0.127.b.barracudacentral.org +ad: 0, rr: 2.0.0.127.b.barracudacentral.org. TTL IN A 127.0.0.D +./test_dns_lookup: dns_query: 2.0.0.127.bl.spamcop.net (A): OK +./test_dns_lookup: dns_query: reply len=LEN ancount=N nscount=N +./test_dns_lookup: dns_get_answer: type A for 2.0.0.127.bl.spamcop.net +2.0.0.127.bl.spamcop.net: fqdn: 2.0.0.127.bl.spamcop.net +ad: 0, rr: 2.0.0.127.bl.spamcop.net. TTL IN A 127.0.0.D diff --git a/postfix/src/dns/test_dns_lookup.c b/postfix/src/dns/test_dns_lookup.c index e27250b63..ce3bb2643 100644 --- a/postfix/src/dns/test_dns_lookup.c +++ b/postfix/src/dns/test_dns_lookup.c @@ -51,13 +51,13 @@ static void print_rr(VSTRING *buf, DNS_RR *rr) static NORETURN usage(char **argv) { - msg_fatal("usage: %s [-v] [-f filter] types name", argv[0]); + msg_fatal("usage: %s [-npv] [-f filter] types name", argv[0]); } int main(int argc, char **argv) { ARGV *types_argv; - unsigned *types; + unsigned type; char *name; VSTRING *fqdn = vstring_alloc(100); VSTRING *why = vstring_alloc(100); @@ -66,42 +66,49 @@ int main(int argc, char **argv) DNS_RR *rr; int i; int ch; + int status; + int lflags = 0; msg_vstream_init(argv[0], VSTREAM_ERR); - while ((ch = GETOPT(argc, argv, "vf:")) > 0) { + while ((ch = GETOPT(argc, argv, "f:npv")) > 0) { switch (ch) { + case 'v': msg_verbose++; break; case 'f': dns_rr_filter_compile("DNS reply filter", optarg); break; + case 'n': + lflags |= DNS_REQ_FLAG_NCACHE_TTL; + break; + case 'p': + dns_ncache_ttl_fix_enable = 1; + break; default: usage(argv); } } if (argc != optind + 2) usage(argv); + buf = vstring_alloc(100); types_argv = argv_split(argv[optind], CHARS_COMMA_SP); - types = (unsigned *) mymalloc(sizeof(*types) * (types_argv->argc + 1)); - for (i = 0; i < types_argv->argc; i++) - if ((types[i] = dns_type(types_argv->argv[i])) == 0) + for (i = 0; i < types_argv->argc; i++) { + if ((type = dns_type(types_argv->argv[i])) == 0) msg_fatal("invalid query type: %s", types_argv->argv[i]); - types[i] = 0; - argv_free(types_argv); - name = argv[optind + 1]; - msg_verbose = 1; - switch (dns_lookup_rv(name, RES_USE_DNSSEC, &rr, fqdn, why, - &rcode, DNS_REQ_FLAG_NONE, types)) { - default: - msg_fatal("%s (rcode=%d)", vstring_str(why), rcode); - case DNS_OK: - vstream_printf("%s: fqdn: %s\n", name, vstring_str(fqdn)); - buf = vstring_alloc(100); - print_rr(buf, rr); - dns_rr_free(rr); - vstring_free(buf); + name = argv[optind + 1]; + msg_verbose = 1; + status = dns_lookup_x(name, type, RES_USE_DNSSEC, &rr, fqdn, why, + &rcode, lflags); + if (status != DNS_OK) + msg_warn("%s (rcode=%d)", vstring_str(why), rcode); + if (rr) { + vstream_printf("%s: fqdn: %s\n", name, vstring_str(fqdn)); + print_rr(buf, rr); + dns_rr_free(rr); + } vstream_fflush(VSTREAM_OUT); } - myfree((void *) types); + argv_free(types_argv); + vstring_free(buf); exit(0); } diff --git a/postfix/src/dnsblog/dnsblog.c b/postfix/src/dnsblog/dnsblog.c index 6b5c46a7d..d96f78ba9 100644 --- a/postfix/src/dnsblog/dnsblog.c +++ b/postfix/src/dnsblog/dnsblog.c @@ -78,6 +78,7 @@ /* System library. */ #include +#include /* Utility library. */ @@ -128,7 +129,8 @@ static VSTRING *result; /* static void dnsblog_query - query DNSBL for client address */ -static VSTRING *dnsblog_query(VSTRING *result, const char *dnsbl_domain, +static VSTRING *dnsblog_query(VSTRING *result, int *result_ttl, + const char *dnsbl_domain, const char *addr) { const char *myname = "dnsblog_query"; @@ -183,8 +185,10 @@ static VSTRING *dnsblog_query(VSTRING *result, const char *dnsbl_domain, * Tack on the RBL domain name and query the DNS for an A record. */ vstring_strcat(query, dnsbl_domain); - dns_status = dns_lookup(STR(query), T_A, 0, &addr_list, (VSTRING *) 0, why); + dns_status = dns_lookup_x(STR(query), T_A, 0, &addr_list, (VSTRING *) 0, + why, (int *) 0, DNS_REQ_FLAG_NCACHE_TTL); VSTRING_RESET(result); + *result_ttl = INT_MAX; if (dns_status == DNS_OK) { for (rr = addr_list; rr != 0; rr = rr->next) { if (dns_rr_to_pa(rr, &hostaddr) == 0) { @@ -196,6 +200,9 @@ static VSTRING *dnsblog_query(VSTRING *result, const char *dnsbl_domain, if (LEN(result) > 0) vstring_strcat(result, " "); vstring_strcat(result, hostaddr.buf); + /* Grab the positive reply TTL. */ + if (*result_ttl > rr->ttl) + *result_ttl = rr->ttl; } } dns_rr_free(addr_list); @@ -203,6 +210,12 @@ static VSTRING *dnsblog_query(VSTRING *result, const char *dnsbl_domain, if (msg_verbose) msg_info("%s: addr %s not listed by domain %s", myname, addr, dnsbl_domain); + /* Grab the negative reply TTL. */ + for (rr = addr_list; rr != 0; rr = rr->next) { + if (rr->type == T_SOA && *result_ttl > rr->ttl) + *result_ttl = rr->ttl; + } + dns_rr_free(addr_list); } else { msg_warn("%s: lookup error for DNS query %s: %s", myname, STR(query), STR(why)); @@ -217,6 +230,7 @@ static void dnsblog_service(VSTREAM *client_stream, char *unused_service, char **argv) { int request_id; + int result_ttl; /* * Sanity check. This service takes no command-line arguments. @@ -235,7 +249,7 @@ static void dnsblog_service(VSTREAM *client_stream, char *unused_service, RECV_ATTR_STR(MAIL_ATTR_ACT_CLIENT_ADDR, addr), RECV_ATTR_INT(MAIL_ATTR_LABEL, &request_id), ATTR_TYPE_END) == 3) { - (void) dnsblog_query(result, STR(rbl_domain), STR(addr)); + (void) dnsblog_query(result, &result_ttl, STR(rbl_domain), STR(addr)); if (var_dnsblog_delay > 0) sleep(var_dnsblog_delay); attr_print(client_stream, ATTR_FLAG_NONE, @@ -243,6 +257,7 @@ static void dnsblog_service(VSTREAM *client_stream, char *unused_service, SEND_ATTR_STR(MAIL_ATTR_ACT_CLIENT_ADDR, STR(addr)), SEND_ATTR_INT(MAIL_ATTR_LABEL, request_id), SEND_ATTR_STR(MAIL_ATTR_RBL_ADDR, STR(result)), + SEND_ATTR_INT(MAIL_ATTR_TTL, result_ttl), ATTR_TYPE_END); vstream_fflush(client_stream); } diff --git a/postfix/src/global/mail_params.h b/postfix/src/global/mail_params.h index 823fc0c85..cdda12855 100644 --- a/postfix/src/global/mail_params.h +++ b/postfix/src/global/mail_params.h @@ -3523,9 +3523,13 @@ extern char *var_psc_dnsbl_enable; #define DEF_PSC_DNSBL_ACTION "ignore" extern char *var_psc_dnsbl_action; -#define VAR_PSC_DNSBL_TTL "postscreen_dnsbl_ttl" -#define DEF_PSC_DNSBL_TTL "1h" -extern int var_psc_dnsbl_ttl; +#define VAR_PSC_DNSBL_MIN_TTL "postscreen_dnsbl_min_ttl" +#define DEF_PSC_DNSBL_MIN_TTL "100s" +extern int var_psc_dnsbl_min_ttl; + +#define VAR_PSC_DNSBL_MAX_TTL "postscreen_dnsbl_max_ttl" +#define DEF_PSC_DNSBL_MAX_TTL "${postscreen_dnsbl_ttl?{$postscreen_dnsbl_ttl}:{1h}}" +extern int var_psc_dnsbl_max_ttl; #define VAR_PSC_DNSBL_REPLY "postscreen_dnsbl_reply_map" #define DEF_PSC_DNSBL_REPLY "" diff --git a/postfix/src/global/mail_version.h b/postfix/src/global/mail_version.h index b663dfcda..ad6afd18f 100644 --- a/postfix/src/global/mail_version.h +++ b/postfix/src/global/mail_version.h @@ -20,7 +20,7 @@ * Patches change both the patchlevel and the release date. Snapshots have no * patchlevel; they change the release date only. */ -#define MAIL_RELEASE_DATE "20150523" +#define MAIL_RELEASE_DATE "20150615" #define MAIL_VERSION_NUMBER "3.1" #ifdef SNAPSHOT diff --git a/postfix/src/postconf/postconf_builtin.c b/postfix/src/postconf/postconf_builtin.c index a966ebaec..9b06c25f0 100644 --- a/postfix/src/postconf/postconf_builtin.c +++ b/postfix/src/postconf/postconf_builtin.c @@ -136,6 +136,7 @@ static const CONFIG_STR_TABLE pcf_legacy_str_table[] = { {"fallback_relay", ""}, {"authorized_verp_clients", ""}, {"smtpd_client_connection_limit_exceptions", ""}, + {"postscreen_dnsbl_ttl", ""}, 0, }; diff --git a/postfix/src/postscreen/postscreen.c b/postfix/src/postscreen/postscreen.c index 95c70b44e..2cc2b9f81 100644 --- a/postfix/src/postscreen/postscreen.c +++ b/postfix/src/postscreen/postscreen.c @@ -275,9 +275,14 @@ /* .IP "\fBpostscreen_bare_newline_ttl (30d)\fR" /* The amount of time that \fBpostscreen\fR(8) will use the result from /* a successful "bare newline" SMTP protocol test. -/* .IP "\fBpostscreen_dnsbl_ttl (1h)\fR" -/* The amount of time that \fBpostscreen\fR(8) will use the result from -/* a successful DNS blocklist test. +/* .IP "\fBpostscreen_dnsbl_max_ttl (${postscreen_dnsbl_ttl?{$postscreen_dnsbl_ttl}:{1h}})\fR" +/* The maximum amount of time that \fBpostscreen\fR(8) will use the +/* result from a successful DNS blocklist or whitelist test before a +/* client IP address is required to pass that test again. +/* .IP "\fBpostscreen_dnsbl_min_ttl (100s)\fR" +/* The minimum amount of time that \fBpostscreen\fR(8) will use the +/* result from a successful DNS blocklist or whitelist test before a +/* client IP address is required to pass that test again. /* .IP "\fBpostscreen_greet_ttl (1d)\fR" /* The amount of time that \fBpostscreen\fR(8) will use the result from /* a successful PREGREET test. @@ -476,7 +481,8 @@ char *var_psc_dnsbl_reply; int var_psc_dnsbl_thresh; int var_psc_dnsbl_wthresh; char *var_psc_dnsbl_action; -int var_psc_dnsbl_ttl; +int var_psc_dnsbl_min_ttl; +int var_psc_dnsbl_max_ttl; int var_psc_dnsbl_tmout; bool var_psc_pipel_enable; @@ -524,7 +530,6 @@ int psc_pipel_action; /* PSC_ACT_DROP/ENFORCE/etc */ int psc_nsmtp_action; /* PSC_ACT_DROP/ENFORCE/etc */ int psc_barlf_action; /* PSC_ACT_DROP/ENFORCE/etc */ int psc_min_ttl; /* Update with new tests! */ -int psc_max_ttl; /* Update with new tests! */ STRING_LIST *psc_forbid_cmds; /* CONNECT GET POST */ int psc_stress_greet_wait; /* stressed greet wait */ int psc_normal_greet_wait; /* stressed greet wait */ @@ -1022,13 +1027,9 @@ static void post_jail_init(char *unused_name, char **unused_argv) * Pre-compute the minimal and maximal TTL. */ psc_min_ttl = - PSC_MIN(PSC_MIN(var_psc_pregr_ttl, var_psc_dnsbl_ttl), + PSC_MIN(PSC_MIN(var_psc_pregr_ttl, var_psc_dnsbl_min_ttl), PSC_MIN(PSC_MIN(var_psc_pipel_ttl, var_psc_nsmtp_ttl), var_psc_barlf_ttl)); - psc_max_ttl = - PSC_MAX(PSC_MAX(var_psc_pregr_ttl, var_psc_dnsbl_ttl), - PSC_MAX(PSC_MAX(var_psc_pipel_ttl, var_psc_nsmtp_ttl), - var_psc_barlf_ttl)); /* * Pre-compute the stress and normal command time limits. @@ -1122,7 +1123,8 @@ int main(int argc, char **argv) static const CONFIG_TIME_TABLE time_table[] = { VAR_PSC_GREET_WAIT, DEF_PSC_GREET_WAIT, &var_psc_greet_wait, 1, 0, VAR_PSC_PREGR_TTL, DEF_PSC_PREGR_TTL, &var_psc_pregr_ttl, 1, 0, - VAR_PSC_DNSBL_TTL, DEF_PSC_DNSBL_TTL, &var_psc_dnsbl_ttl, 1, 0, + VAR_PSC_DNSBL_MIN_TTL, DEF_PSC_DNSBL_MIN_TTL, &var_psc_dnsbl_min_ttl, 1, 0, + VAR_PSC_DNSBL_MAX_TTL, DEF_PSC_DNSBL_MAX_TTL, &var_psc_dnsbl_max_ttl, 1, 0, VAR_PSC_PIPEL_TTL, DEF_PSC_PIPEL_TTL, &var_psc_pipel_ttl, 1, 0, VAR_PSC_NSMTP_TTL, DEF_PSC_NSMTP_TTL, &var_psc_nsmtp_ttl, 1, 0, VAR_PSC_BARLF_TTL, DEF_PSC_BARLF_TTL, &var_psc_barlf_ttl, 1, 0, diff --git a/postfix/src/postscreen/postscreen.h b/postfix/src/postscreen/postscreen.h index 130b52770..ceb2a2571 100644 --- a/postfix/src/postscreen/postscreen.h +++ b/postfix/src/postscreen/postscreen.h @@ -75,6 +75,7 @@ typedef struct { time_t expire_time[PSC_TINDX_COUNT]; /* per-test expiration */ VSTRING *dnsbl_reply; /* dnsbl reject text */ int dnsbl_score; /* saved DNSBL score */ + int dnsbl_ttl; /* saved DNSBL TTL */ const char *dnsbl_name; /* DNSBL name with largest weight */ int dnsbl_index; /* dnsbl request index */ const char *rcpt_reply; /* how to reject recipients */ @@ -372,7 +373,6 @@ extern int psc_pipel_action; /* PSC_ACT_DROP etc. */ extern int psc_nsmtp_action; /* PSC_ACT_DROP etc. */ extern int psc_barlf_action; /* PSC_ACT_DROP etc. */ extern int psc_min_ttl; /* Update with new tests! */ -extern int psc_max_ttl; /* Update with new tests! */ extern STRING_LIST *psc_forbid_cmds; /* CONNECT GET POST */ extern int psc_stress_greet_wait; /* stressed greet wait */ extern int psc_normal_greet_wait; /* stressed greet wait */ @@ -480,7 +480,7 @@ const char *psc_maps_find(MAPS *, const char *, int); * postscreen_dnsbl.c */ extern void psc_dnsbl_init(void); -extern int psc_dnsbl_retrieve(const char *, const char **, int); +extern int psc_dnsbl_retrieve(const char *, const char **, int, int *); extern int psc_dnsbl_request(const char *, void (*) (int, void *), void *); /* diff --git a/postfix/src/postscreen/postscreen_dnsbl.c b/postfix/src/postscreen/postscreen_dnsbl.c index 76c21ed0d..d56dca2df 100644 --- a/postfix/src/postscreen/postscreen_dnsbl.c +++ b/postfix/src/postscreen/postscreen_dnsbl.c @@ -13,10 +13,12 @@ /* void (*callback)(int, char *); /* char *context; /* -/* int psc_dnsbl_retrieve(client_addr, dnsbl_name, dnsbl_index) +/* int psc_dnsbl_retrieve(client_addr, dnsbl_name, dnsbl_index, +/* dnsbl_ttl) /* char *client_addr; /* const char **dnsbl_name; /* int dnsbl_index; +/* int *dnsbl_ttl; /* DESCRIPTION /* This module implements preliminary support for DNSBL lookups. /* Multiple requests for the same information are handled with @@ -37,9 +39,10 @@ /* The result value is the index for the psc_dnsbl_retrieve() /* call. /* -/* psc_dnsbl_retrieve() retrieves the result score requested with -/* psc_dnsbl_request() and decrements the reference count. It -/* is an error to retrieve a score without requesting it first. +/* psc_dnsbl_retrieve() retrieves the result score and reply +/* TTL requested with psc_dnsbl_request(), and decrements the +/* reference count. It is an error to retrieve a score without +/* requesting it first. /* LICENSE /* .ad /* .fi @@ -58,6 +61,7 @@ #include /* inet_pton() */ #include /* inet_pton() */ #include /* sscanf */ +#include /* Utility library. */ @@ -141,6 +145,7 @@ typedef struct { const char *dnsbl_name; /* DNSBL with largest contribution */ int dnsbl_weight; /* weight of largest contribution */ int total; /* combined blocklist score */ + int dnsbl_ttl; /* DNS reply ttl */ int refcount; /* score reference count */ int pending_lookups; /* nr of DNS requests in flight */ int request_id; /* duplicate suppression */ @@ -306,7 +311,7 @@ static int psc_dnsbl_match(const char *filter, ARGV *reply) /* psc_dnsbl_retrieve - retrieve blocklist score, decrement reference count */ int psc_dnsbl_retrieve(const char *client_addr, const char **dnsbl_name, - int dnsbl_index) + int dnsbl_index, int *dnsbl_ttl) { const char *myname = "psc_dnsbl_retrieve"; PSC_DNSBL_SCORE *score; @@ -329,6 +334,7 @@ int psc_dnsbl_retrieve(const char *client_addr, const char **dnsbl_name, */ result_score = score->total; *dnsbl_name = score->dnsbl_name; + *dnsbl_ttl = score->dnsbl_ttl; score->refcount -= 1; if (score->refcount < 1) { if (msg_verbose > 1) @@ -349,6 +355,7 @@ static void psc_dnsbl_receive(int event, void *context) PSC_DNSBL_SITE *site; ARGV *reply_argv; int request_id; + int dnsbl_ttl; PSC_CLEAR_EVENT_REQUEST(vstream_fileno(stream), psc_dnsbl_receive, context); @@ -374,7 +381,8 @@ static void psc_dnsbl_receive(int event, void *context) RECV_ATTR_STR(MAIL_ATTR_ACT_CLIENT_ADDR, reply_client), RECV_ATTR_INT(MAIL_ATTR_LABEL, &request_id), RECV_ATTR_STR(MAIL_ATTR_RBL_ADDR, reply_addr), - ATTR_TYPE_END) == 4 + RECV_ATTR_INT(MAIL_ATTR_TTL, &dnsbl_ttl), + ATTR_TYPE_END) == 5 && (score = (PSC_DNSBL_SCORE *) htable_find(dnsbl_score_cache, STR(reply_client))) != 0 && score->request_id == request_id) { @@ -387,9 +395,9 @@ static void psc_dnsbl_receive(int event, void *context) * server may be messed up. */ if (msg_verbose > 1) - msg_info("%s: client=\"%s\" score=%d domain=\"%s\" reply=\"%s\"", + msg_info("%s: client=\"%s\" score=%d domain=\"%s\" reply=\"%d %s\"", myname, STR(reply_client), score->total, - STR(reply_dnsbl), STR(reply_addr)); + STR(reply_dnsbl), dnsbl_ttl, STR(reply_addr)); if (*STR(reply_addr) != 0) { head = (PSC_DNSBL_HEAD *) htable_find(dnsbl_site_cache, STR(reply_dnsbl)); @@ -414,6 +422,14 @@ static void psc_dnsbl_receive(int event, void *context) argv_free(reply_argv); } + /* + * For now, don't try to be clever, and play safe: use the smallest + * TTL even if that particular DNS reply would not change the final + * decision. + */ + if (score->dnsbl_ttl > dnsbl_ttl) + score->dnsbl_ttl = dnsbl_ttl; + /* * Notify the requestor(s) that the result is ready to be picked up. * If this call isn't made, clients have to sit out the entire @@ -485,6 +501,7 @@ int psc_dnsbl_request(const char *client_addr, score->request_id = request_count++; score->dnsbl_name = 0; score->dnsbl_weight = 0; + score->dnsbl_ttl = INT_MAX; score->total = 0; score->refcount = 1; score->pending_lookups = 0; diff --git a/postfix/src/postscreen/postscreen_early.c b/postfix/src/postscreen/postscreen_early.c index e46487bcb..99ba9c157 100644 --- a/postfix/src/postscreen/postscreen_early.c +++ b/postfix/src/postscreen/postscreen_early.c @@ -58,6 +58,11 @@ static void psc_whitelist_non_dnsbl(PSC_STATE *state) time_t now; int tindx; +#define PSC_EFF_DNSBL_TTL(state) \ + (var_psc_dnsbl_max_ttl < (state)->dnsbl_ttl ? var_psc_dnsbl_max_ttl : \ + var_psc_dnsbl_min_ttl > (state)->dnsbl_ttl ? var_psc_dnsbl_min_ttl : \ + (state)->dnsbl_ttl) + /* * If no tests failed (we can't undo those), and if the whitelist * threshold is met, flag non-dnsbl tests that are pending or disabled as @@ -87,8 +92,8 @@ static void psc_whitelist_non_dnsbl(PSC_STATE *state) state->flags |= PSC_STATE_FLAG_BYTINDX_PASS(tindx); } /* Update expiration even if the test was completed or disabled. */ - if (state->expire_time[tindx] < now + var_psc_dnsbl_ttl) - state->expire_time[tindx] = now + var_psc_dnsbl_ttl; + if (state->expire_time[tindx] < now + PSC_EFF_DNSBL_TTL(state)) + state->expire_time[tindx] = now + PSC_EFF_DNSBL_TTL(state); } } } @@ -164,12 +169,13 @@ static void psc_early_event(int event, void *context) state->dnsbl_score = psc_dnsbl_retrieve(state->smtp_client_addr, &state->dnsbl_name, - state->dnsbl_index); + state->dnsbl_index, + &state->dnsbl_ttl); if (var_psc_dnsbl_wthresh < 0) psc_whitelist_non_dnsbl(state); } if (state->dnsbl_score < var_psc_dnsbl_thresh) { - state->dnsbl_stamp = event_time() + var_psc_dnsbl_ttl; + state->dnsbl_stamp = event_time() + PSC_EFF_DNSBL_TTL(state); PSC_PASS_SESSION_STATE(state, "dnsbl test", PSC_STATE_FLAG_DNSBL_PASS); } else { @@ -228,7 +234,8 @@ static void psc_early_event(int event, void *context) && (state->flags & PSC_STATE_FLAG_DNSBL_TODO)) (void) psc_dnsbl_retrieve(state->smtp_client_addr, &state->dnsbl_name, - state->dnsbl_index); + state->dnsbl_index, + &state->dnsbl_ttl); /* XXX Wait for DNS replies to come in. */ psc_hangup_event(state); return; @@ -246,7 +253,8 @@ static void psc_early_event(int event, void *context) && (state->flags & PSC_STATE_FLAG_DNSBL_TODO)) (void) psc_dnsbl_retrieve(state->smtp_client_addr, &state->dnsbl_name, - state->dnsbl_index); + state->dnsbl_index, + &state->dnsbl_ttl); PSC_DROP_SESSION_STATE(state, "521 5.5.1 Protocol error\r\n"); return; case PSC_ACT_ENFORCE: @@ -298,7 +306,7 @@ static void psc_early_dnsbl_event(int unused_event, void *context) */ state->dnsbl_score = psc_dnsbl_retrieve(state->smtp_client_addr, &state->dnsbl_name, - state->dnsbl_index); + state->dnsbl_index, &state->dnsbl_ttl); if (var_psc_dnsbl_wthresh < 0) psc_whitelist_non_dnsbl(state);