From: Wietse Venema Date: Sun, 18 Aug 2013 05:00:00 +0000 (-0500) Subject: postfix-2.11-20130818 X-Git-Tag: v2.11.0-RC1~26 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=f66cfedc154b1e61ed0f187b143ddbecb67a9ffa;p=thirdparty%2Fpostfix.git postfix-2.11-20130818 --- diff --git a/postfix/HISTORY b/postfix/HISTORY index a2de98587..347b75ba1 100644 --- a/postfix/HISTORY +++ b/postfix/HISTORY @@ -18796,3 +18796,46 @@ Apologies for any names omitted. instead of overstrike sequences. To make matters worse, it uses the ESC[0m sequence sometimes for end-of-bold and sometimes for end-of-italic. File: mantools/man2html. + +20130714 + + Cleanup: added smtpd_relay_restrictions entries to the + default master.cf file, so that main.cf settings won't + affect the submission and smtps services. Simon Matter. + File: conf/master.cf. + +20130728 + + Cleanup: wrong function name in error message. John Fawcett. + File: util/vstring_vstream.c. + +20130801 + + Cleanup: with ``make makefiles CCARGS="-DHAS_DB...'', the + makedefs script no longer tries to locate the Linux Berkeley + DB include and library files. Instead it assumes that the + locations are given on the command line, as shown in the + DB_README examples. Leo Baltus. File: makedefs. + +20130805 + + Documentation: clarified reject_non_fqdn_helo_hostname. + File: proto/postconf.proto. + +20130809 + + Cleanup: the lmdb_map_size parameter is now a long integer. + Howard Chu. Files: global/mail_params.[hc]. + +20130815 + + Documentation: added pointer to Dovecot 2 configuration. + File: proto/SASL_README.html + +20130818 + + Update: LMDB client updated to LMDB 0.9.7, which hopefully + fixes the unrecoverable "transaction full" error. With a + new MDB_MAP_FULL workaround by Howard Chu that ensures that + postfix will make progress as long as the disk is not full. + File: util/dict_lmdb.c. diff --git a/postfix/README_FILES/LMDB_README b/postfix/README_FILES/LMDB_README index 09f8635d0..4d0281481 100644 --- a/postfix/README_FILES/LMDB_README +++ b/postfix/README_FILES/LMDB_README @@ -4,20 +4,11 @@ PPoossttffiixx OOppeennLLDDAAPP LLMMDDBB HHoowwttoo IInnttrroodduuccttiioonn -Note: - Postfix support for LMDB databases is suspended due to the existence of a - hard limit (an "out of storage" failure mode that cannot be resolved by - increasing the database size). - - Postfix may support LMDB again when it no longer limits the size of Postfix - transactions, whether the limit is built into LMDB itself, or implicit by - requiring an unbounded amount of memory to handle a large transaction. - Postfix uses databases of various kinds to store and look up information. Postfix databases are specified as "type:name". OpenLDAP LMDB implements the Postfix database type "lmdb". The name of a Postfix OpenLDAP LMDB database is the name of the database file without the ".lmdb" suffix. OpenLDAP LMDB -databases are maintained with the postmap(1) command. +databases are maintained with the postmap(1) and postalias(1) commands. This document describes: @@ -51,7 +42,7 @@ CCoonnffiigguurree LLMMDDBB sseettttiinnggss Postfix provides a configuration parameter that controls how large an OpenLDAP LMDB database may grow. - * lmdb_map_size (default: 10 MBytes per table). This setting controls how + * lmdb_map_size (default: 16 MBytes per table). This setting controls how large any OpenLDAP LMDB database may grow. It must be set large enough to accommodate the largest table that Postfix will use. @@ -77,20 +68,6 @@ that don't exist with other Postfix databases. UUnneexxppeecctteedd ppoossttmmaapp((11))//ppoossttaalliiaass((11)) ""ddaattaabbaassee ffuullll"" eerrrroorrss.. -Problem: - The "postmap lmdb:filename" command fails with an MDB_TXN_FULL error. This - problem does not exist with other Postfix databases. - -Background: - The LMDB implementation has a hard limit on the total transaction size. - This limit is independent of the LMDB database size. Therefore, the problem - cannot be resolved by increasing the lmdb_map_size value. - - This symptom is indicative of a flawed design. All LMDB data structures - should share the same storage pool so that they can scale with the database - size, and so that all "out of storage" errors are resolved by increasing - the database size. - Problem: The "postmap lmdb:filename" command fails with an MDB_MAP_FULL error. This problem does not exist with other Postfix databases. @@ -109,16 +86,20 @@ Impact: still exists in the database. Mitigation: + When the postmap(1) or postalias(1) command fails with an MDB_MAP_FULL + error, it expands the database file size to the current LMDB map size limit + before terminating. + When the postmap(1) or postalias(1) command opens an LMDB file larger than lmdb_map_size/3, it logs a warning and uses a larger size limit instead: warning: filename.lmdb: file size 15024128 >= (lmdb map size limit 16777216)/3 -- using a larger map size limit - This can be used to automate recovery and avoid the need for human - intervention. Just keep running "postmap lmdb:filename". After each failure - it will use a 3x larger size limit, and eventually the "database full" - error will disappear. + The two steps above can be used to automate recovery and avoid the need for + human intervention. Just repeat "postmap lmdb:filename" (up to some limit). + After each failure it will use a 3x larger size limit, and eventually the + "database full" error will disappear. Prevention: Monitor your LMDB files and make sure that lmdb_map_size > 3x the largest @@ -161,7 +142,7 @@ Problem: Background: The Postfix LMDB database client does not truncate the database file. - Instead it attempts to create a transaction for a "drop" request and + Instead it attempts to create a transaction for a "drop" request plus subsequent "store" requests. That is obviously not possible with a corrupted database file. @@ -176,3 +157,9 @@ Recovery: Prevention: Arrange your file systems such that they never run out of free space. + Use ECC memory to detect and correct silent corruption of in-memory file + system data and metadata. + + Use a file system such as ZFS to detect and correct silent corruption of + on-disk file system data and metadata. + diff --git a/postfix/README_FILES/SASL_README b/postfix/README_FILES/SASL_README index c35481991..309ee9ac4 100644 --- a/postfix/README_FILES/SASL_README +++ b/postfix/README_FILES/SASL_README @@ -120,6 +120,9 @@ specified on the Dovecot server side in dovecot.conf. The following example assumes that the Postfix queue is under /var/spool/ postfix/. +Note: the example uses Dovecot 1 syntax, See http://www.dovecot.org/ for newer +syntax. + 1 /etc/dovecot.conf: 2 auth default { 3 mechanisms = plain login @@ -152,6 +155,9 @@ on the Dovecot server side in 10-auth.conf and 10-master.conf. The following examples assume that Postfix should communicate with Dovecot on TCP port 12345. +Note: the examples use Dovecot 1 syntax, See http://www.dovecot.org/ for newer +syntax. + 1 /etc/dovecot/conf.d/10-auth.conf: 2 auth_mechanisms = plain login diff --git a/postfix/WISHLIST b/postfix/WISHLIST index a5d39ed74..927544ef0 100644 --- a/postfix/WISHLIST +++ b/postfix/WISHLIST @@ -11,6 +11,8 @@ Wish list: Things to do after the stable release: + Why does postlink no longer hyperlink static:all? + Begin code revision, after DANE support stabilizes. This should be one pass that changes only names and no code. diff --git a/postfix/conf/master.cf b/postfix/conf/master.cf index acc67d726..1369918c3 100644 --- a/postfix/conf/master.cf +++ b/postfix/conf/master.cf @@ -21,7 +21,8 @@ smtp inet n - n - - smtpd # -o smtpd_client_restrictions=$mua_client_restrictions # -o smtpd_helo_restrictions=$mua_helo_restrictions # -o smtpd_sender_restrictions=$mua_sender_restrictions -# -o smtpd_recipient_restrictions=permit_sasl_authenticated,reject +# -o smtpd_recipient_restrictions= +# -o smtpd_relay_restrictions=permit_sasl_authenticated,reject # -o milter_macro_daemon_name=ORIGINATING #smtps inet n - n - - smtpd # -o syslog_name=postfix/smtps @@ -31,7 +32,8 @@ smtp inet n - n - - smtpd # -o smtpd_client_restrictions=$mua_client_restrictions # -o smtpd_helo_restrictions=$mua_helo_restrictions # -o smtpd_sender_restrictions=$mua_sender_restrictions -# -o smtpd_recipient_restrictions=permit_sasl_authenticated,reject +# -o smtpd_recipient_restrictions= +# -o smtpd_relay_restrictions=permit_sasl_authenticated,reject # -o milter_macro_daemon_name=ORIGINATING #628 inet n - n - - qmqpd pickup unix n - n 60 1 pickup diff --git a/postfix/html/LMDB_README.html b/postfix/html/LMDB_README.html index b5414f12e..e9938aa53 100644 --- a/postfix/html/LMDB_README.html +++ b/postfix/html/LMDB_README.html @@ -19,6 +19,8 @@

Introduction

+ +

Postfix uses databases of various kinds to store and look up information. Postfix databases are specified as "type:name". OpenLDAP LMDB implements the Postfix database type "lmdb". The name of a Postfix OpenLDAP LMDB database is the name of the database file without the ".lmdb" suffix. OpenLDAP LMDB databases -are maintained with the postmap(1) command.

+are maintained with the postmap(1) and postalias(1) commands.

This document describes:

@@ -83,10 +87,10 @@ large an OpenLDAP LMDB database may grow.

@@ -126,6 +130,8 @@ errors.

+ +
Problem:

The "postmap lmdb:filename" command fails with an MDB_MAP_FULL error. This problem does not exist with other Postfix databases.

@@ -167,18 +175,25 @@ reused.

availability, because the old data still exists in the database.

-
Mitigation:

When the postmap(1) or postalias(1) -command opens an LMDB file larger than lmdb_map_size/3, it logs a -warning and uses a larger size limit instead:

+
Mitigation:
+ +

When the postmap(1) or postalias(1) command fails with an +MDB_MAP_FULL error, it expands the database file size to the current +LMDB map size limit before terminating.

+ +

When the postmap(1) or postalias(1) command opens an LMDB file +larger than lmdb_map_size/3, it logs a warning and uses a larger +size limit instead:

warning: filename.lmdb: file size 15024128 ≥ (lmdb map size limit 16777216)/3 -- using a larger map size limit

-

This can be used to automate recovery and avoid the need for -human intervention. Just keep running "postmap lmdb:filename". -After each failure it will use a 3x larger size limit, and eventually -the "database full" error will disappear.

+

The two steps above can be used to automate recovery and avoid +the need for human intervention. Just repeat "postmap lmdb:filename" +(up to some limit). After each failure it will use a 3x larger +size limit, and eventually the "database full" error will disappear. +

Prevention:

Monitor your LMDB files and make sure that lmdb_map_size > 3x the largest LMDB file size.

@@ -215,8 +230,6 @@ full" error will disappear, at least for a while.

sure that lmdb_map_size > 3x the largest LMDB file size.

- -

Non-obvious recovery with postmap(1)/postalias(1)/tlsmgr(8) from a corrupted database.

@@ -229,7 +242,7 @@ does not exist with other Postfix databases.

Background:

The Postfix LMDB database client does not truncate the database file. Instead it attempts to create -a transaction for a "drop" request and subsequent "store" requests. +a transaction for a "drop" request plus subsequent "store" requests. That is obviously not possible with a corrupted database file.

@@ -241,7 +254,15 @@ then rebuild the file with the postmap(1) or tlsmgr(8) daemon restarts automatically.

-
Prevention:

Arrange your file systems such that -they never run out of free space.

+
Prevention:
+ +

Arrange your file systems such that they never run out of free +space.

- +

Use ECC memory to detect and correct silent corruption of +in-memory file system data and metadata.

+ +

Use a file system such as ZFS to detect and correct silent +corruption of on-disk file system data and metadata.

+ +
diff --git a/postfix/html/SASL_README.html b/postfix/html/SASL_README.html index f39feefcd..ec5cb84ca 100644 --- a/postfix/html/SASL_README.html +++ b/postfix/html/SASL_README.html @@ -198,6 +198,9 @@ specified on the Dovecot server side in dovecot.conf.

The following example assumes that the Postfix queue is under /var/spool/postfix/.

+

Note: the example uses Dovecot 1 syntax, See http://www.dovecot.org/ +for newer syntax.

+ 
  1 /etc/dovecot.conf:
@@ -238,6 +241,9 @@ and 10-master.conf. 

 
 The following examples assume that Postfix should communicate
 with Dovecot on TCP port 12345. 

 
+
 Note: the examples use Dovecot 1 syntax, See http://www.dovecot.org/
+for newer syntax. 

+
 

 
 1 /etc/dovecot/conf.d/10-auth.conf:
diff --git a/postfix/html/master.5.html b/postfix/html/master.5.html
index 51ad7b755..01cc4ef5a 100644
--- a/postfix/html/master.5.html
+++ b/postfix/html/master.5.html
@@ -74,8 +74,8 @@ MASTER(5)                                                            MASTER(5)
                      port  may be given in symbolic form (host or
                      service name) or in numeric form (IP address
                      or  port  number).   Host information may be
-                     enclosed inside "[]", but this form  is  not
-                     necessary.
+                     enclosed inside "[]"; this form is necessary
+                     only with IPv6 addresses.
 
                      Examples:  a service named 127.0.0.1:smtp or
                      ::1:smtp  receives  mail  via  the  loopback
@@ -201,7 +201,7 @@ MASTER(5)                                                            MASTER(5)
                      in  the  main.cf  configuration  file.   See
                      DEBUG_README for hints and tips.
 
-              -o name=value
+              -o name=value
                      Override  the  named  main.cf  configuration
                      parameter.  The parameter value can refer to
                      other parameters as $name etc., just like in
diff --git a/postfix/html/pcre_table.5.html b/postfix/html/pcre_table.5.html
index 004b0a152..eac6faa92 100644
--- a/postfix/html/pcre_table.5.html
+++ b/postfix/html/pcre_table.5.html
@@ -10,9 +10,9 @@ PCRE_TABLE(5)                                                    PCRE_TABLE(5)
        pcre_table - format of Postfix PCRE tables
 
 SYNOPSIS
-       postmap -q "string" pcre:/etc/postfix/filename
+       postmap -q "string" pcre:/etc/postfix/filename
 
-       postmap -q - pcre:/etc/postfix/filename <inputfile
+       postmap -q - pcre:/etc/postfix/filename <inputfile
 
 DESCRIPTION
        The  Postfix  mail system uses optional tables for address
@@ -39,15 +39,15 @@ PCRE_TABLE(5)                                                    PCRE_TABLE(5)
 TABLE FORMAT
        The general form of a PCRE table is:
 
-       /pattern/flags result
+       /pattern/flags result
               When pattern matches the input string, use the cor-
               responding result value.
 
-       !/pattern/flags result
+       !/pattern/flags result
               When pattern does not match the input  string,  use
               the corresponding result value.
 
-       if /pattern/flags
+       if /pattern/flags
 
        endif  Match the input string against the patterns between
               if and endif, if and only if that same input string
@@ -58,7 +58,7 @@ PCRE_TABLE(5)                                                    PCRE_TABLE(5)
 
               This feature is available in Postfix 2.1 and later.
 
-       if !/pattern/flags
+       if !/pattern/flags
 
        endif  Match the input string against the patterns between
               if and endif, if and only if that same input string
@@ -115,7 +115,7 @@ PCRE_TABLE(5)                                                    PCRE_TABLE(5)
               a whitespace character  as  part  of  the  pattern,
               escape it with backslash.
 
-              Note: do not use #comment after patterns.
+              Note: do not use #comment after patterns.
 
        A (default: off)
               Toggles  the PCRE_ANCHORED flag.  When this flag is
@@ -164,12 +164,12 @@ PCRE_TABLE(5)                                                    PCRE_TABLE(5)
        nor is user+foo broken up into user and foo.
 
 TEXT SUBSTITUTION
-       Substitution of substrings  from  the  matched  expression
-       into  the result string is possible using the conventional
-       perl syntax ($1, $2, etc.); specify  $$  to  produce  a  $
-       character  as output.  The macros in the result string may
-       need to be written as ${n} or $(n) if they aren't followed
-       by whitespace.
+       Substitution of substrings  (text  that  matches  patterns
+       inside  "()")  from the matched expression into the result
+       string is requested with $1, $2, etc.; specify $$ to  pro-
+       duce  a  $  character as output.  The macros in the result
+       string may need to be written as  ${n}  or  $(n)  if  they
+       aren't followed by whitespace.
 
        Note:  since negated patterns (those preceded by !) return
        a result when the expression does not match, substitutions
diff --git a/postfix/html/postconf.5.html b/postfix/html/postconf.5.html
index 322bf6995..8d83dea6a 100644
--- a/postfix/html/postconf.5.html
+++ b/postfix/html/postconf.5.html
@@ -13463,7 +13463,8 @@ for rejected requests (default: 501).
 
reject_non_fqdn_helo_hostname (with Postfix < 2.3: reject_non_fqdn_hostname)

 
 
Reject the request when the HELO or EHLO hostname is not in
-fully-qualified domain form, as required by the RFC. Note: specify
+fully-qualified domain or address literal form, as required by the
+RFC. Note: specify
 "smtpd_helo_required = yes" to fully enforce this restriction
 (without "smtpd_helo_required = yes", a client can simply skip
 reject_non_fqdn_helo_hostname by not sending HELO or EHLO).  

diff --git a/postfix/html/regexp_table.5.html b/postfix/html/regexp_table.5.html
index cff60ba8a..8a2bb3b20 100644
--- a/postfix/html/regexp_table.5.html
+++ b/postfix/html/regexp_table.5.html
@@ -10,9 +10,9 @@ REGEXP_TABLE(5)                                                REGEXP_TABLE(5)
        regexp_table - format of Postfix regular expression tables
 
 SYNOPSIS
-       postmap -q "string" regexp:/etc/postfix/filename
+       postmap -q "string" regexp:/etc/postfix/filename
 
-       postmap -q - regexp:/etc/postfix/filename <inputfile
+       postmap -q - regexp:/etc/postfix/filename <inputfile
 
 DESCRIPTION
        The Postfix mail system uses optional tables  for  address
@@ -39,15 +39,15 @@ REGEXP_TABLE(5)                                                REGEXP_TABLE(5)
 TABLE FORMAT
        The general form of a Postfix regular expression table is:
 
-       /pattern/flags result
+       /pattern/flags result
               When pattern matches the input string, use the cor-
               responding result value.
 
-       !/pattern/flags result
+       !/pattern/flags result
               When pattern does not match the input  string,  use
               the corresponding result value.
 
-       if /pattern/flags
+       if /pattern/flags
 
        endif  Match the input string against the patterns between
               if and endif, if and only if that same input string
@@ -58,7 +58,7 @@ REGEXP_TABLE(5)                                                REGEXP_TABLE(5)
 
               This feature is available in Postfix 2.1 and later.
 
-       if !/pattern/flags
+       if !/pattern/flags
 
        endif  Match the input string against the patterns between
               if and endif, if and only if that same input string
@@ -125,13 +125,14 @@ REGEXP_TABLE(5)                                                REGEXP_TABLE(5)
        nor is user+foo broken up into user and foo.
 
 TEXT SUBSTITUTION
-       Substitution of substrings  from  the  matched  expression
-       into  the  result  string  is possible using $1, $2, etc.;
-       specify $$ to produce a $ character as output.  The macros
-       in  the  result  string  may need to be written as ${n} or
-       $(n) if they aren't followed by whitespace.
-
-       Note: since negated patterns (those preceded by !)  return
+       Substitution of substrings  (text  that  matches  patterns
+       inside  "()")  from the matched expression into the result
+       string is requested with $1, $2, etc.; specify $$ to  pro-
+       duce  a  $  character as output.  The macros in the result
+       string may need to be written as  ${n}  or  $(n)  if  they
+       aren't followed by whitespace.
+
+       Note:  since negated patterns (those preceded by !) return
        a result when the expression does not match, substitutions
        are not available for negated patterns.
 
diff --git a/postfix/makedefs b/postfix/makedefs
index 2ceff2e4c..a731c04c2 100644
--- a/postfix/makedefs
+++ b/postfix/makedefs
@@ -278,12 +278,14 @@ case "$SYSTEM.$RELEASE" in
     Linux.1*)	SYSTYPE=LINUX1
 		case "$CCARGS" in
 		 *-DNO_DB*) ;;
+		 *-DHAS_DB*) ;;
 		 *) SYSLIBS="-ldb";;
 		esac
 		;;
     Linux.2*)	SYSTYPE=LINUX2
 		case "$CCARGS" in
 		 *-DNO_DB*) ;;
+		 *-DHAS_DB*) ;;
 		 *) if [ -f /usr/include/db.h ]
 		    then
 			: we are all set
@@ -356,6 +358,7 @@ EOF
     Linux.3*)	SYSTYPE=LINUX3
 		case "$CCARGS" in
 		 *-DNO_DB*) ;;
+		 *-DHAS_DB*) ;;
 		 *) if [ -f /usr/include/db.h ]
 		    then
 			: we are all set
diff --git a/postfix/man/man5/master.5 b/postfix/man/man5/master.5
index efc6d55e1..644ce57ea 100644
--- a/postfix/man/man5/master.5
+++ b/postfix/man/man5/master.5
@@ -69,8 +69,8 @@ the host and port on which new connections should be
 accepted. The host part (and colon) may be omitted.  Either
 host or port may be given in symbolic form (host or service
 name) or in numeric form (IP address or port number).
-Host information may be enclosed inside "[]", but this form
-is not necessary.
+Host information may be enclosed inside "[]"; this form
+is necessary only with IPv6 addresses.
 .sp
 Examples: a service named \fB127.0.0.1:smtp\fR or \fB::1:smtp\fR
 receives
diff --git a/postfix/man/man5/pcre_table.5 b/postfix/man/man5/pcre_table.5
index 9399450be..37a65dd79 100644
--- a/postfix/man/man5/pcre_table.5
+++ b/postfix/man/man5/pcre_table.5
@@ -152,11 +152,12 @@ broken up into \fIuser\fR and \fIfoo\fR.
 .nf
 .ad
 .fi
-Substitution of substrings from the matched expression into the result
-string is possible using the conventional perl syntax ($1, $2, etc.);
-specify $$ to produce a $ character as output.
-The macros in the result string may need to be written as ${n}
-or $(n) if they aren't followed by whitespace.
+Substitution of substrings (text that matches patterns
+inside "()") from the matched expression into the result
+string is requested with $1, $2, etc.; specify $$ to produce
+a $ character as output.
+The macros in the result string may need to be written as
+${n} or $(n) if they aren't followed by whitespace.
 
 Note: since negated patterns (those preceded by \fB!\fR) return a
 result when the expression does not match, substitutions are not
diff --git a/postfix/man/man5/postconf.5 b/postfix/man/man5/postconf.5
index 3759615fc..2a69e1c39 100644
--- a/postfix/man/man5/postconf.5
+++ b/postfix/man/man5/postconf.5
@@ -8873,7 +8873,8 @@ for rejected requests (default: 501).
 .br
 .IP "\fBreject_non_fqdn_helo_hostname\fR (with Postfix < 2.3: reject_non_fqdn_hostname)"
 Reject the request when the HELO or EHLO hostname is not in
-fully-qualified domain form, as required by the RFC. Note: specify
+fully-qualified domain or address literal form, as required by the
+RFC. Note: specify
 "smtpd_helo_required = yes" to fully enforce this restriction
 (without "smtpd_helo_required = yes", a client can simply skip
 reject_non_fqdn_helo_hostname by not sending HELO or EHLO).
diff --git a/postfix/man/man5/regexp_table.5 b/postfix/man/man5/regexp_table.5
index 991088d33..cc8979c70 100644
--- a/postfix/man/man5/regexp_table.5
+++ b/postfix/man/man5/regexp_table.5
@@ -120,12 +120,12 @@ broken up into \fIuser\fR and \fIfoo\fR.
 .nf
 .ad
 .fi
-Substitution of substrings from the matched expression into the result
-string is possible using $1, $2, etc.;
-specify $$ to produce a $ character as output.
-The macros in the result string
-may need to be written as ${n} or $(n) if they aren't followed
-by whitespace.
+Substitution of substrings (text that matches patterns
+inside "()") from the matched expression into the result
+string is requested with $1, $2, etc.; specify $$ to produce
+a $ character as output.
+The macros in the result string may need to be written as
+${n} or $(n) if they aren't followed by whitespace.
 
 Note: since negated patterns (those preceded by \fB!\fR) return a
 result when the expression does not match, substitutions are not
diff --git a/postfix/proto/LMDB_README.html b/postfix/proto/LMDB_README.html
index 3fd4b9b01..292e9920f 100644
--- a/postfix/proto/LMDB_README.html
+++ b/postfix/proto/LMDB_README.html
@@ -19,6 +19,8 @@
 
 
Introduction

 
+
+
 
 Postfix uses databases of various kinds to store and look up
 information. Postfix databases are specified as "type:name".
 OpenLDAP LMDB implements the Postfix database type "lmdb".
 The name of a Postfix OpenLDAP LMDB database is the name
 of the database file without the ".lmdb" suffix. OpenLDAP LMDB databases
-are maintained with the postmap(1) command.  

+are maintained with the postmap(1) and postalias(1) commands.  

 
 
 This document describes: 

 
@@ -83,10 +87,10 @@ large an OpenLDAP LMDB database may grow. 

 
 
    
 
-
  •  
     lmdb_map_size (default: 10 MBytes per
-table).  This setting controls how large any OpenLDAP LMDB database
-may grow. It must be set large enough to accommodate the largest
-table that Postfix will use. 
    
+
  •  
     lmdb_map_size (default: 16 MBytes per table).  This setting
+controls how large any OpenLDAP LMDB database may grow. It must be
+set large enough to accommodate the largest table that Postfix will
+use. 
    
 
 

 
@@ -126,6 +130,8 @@ errors.  

 
 

 
+
+
 
 Problem: 
 
 
 The "postmap lmdb:filename" command
 fails with an MDB_MAP_FULL error.  This problem does not exist with
 other Postfix databases. 
 

@@ -167,18 +175,25 @@ reused.  

 availability, because the old data still exists in the database.
 
 

 
-
 Mitigation: 
 
 
 When the postmap(1) or postalias(1)
-command opens an LMDB file larger than lmdb_map_size/3, it logs a
-warning and uses a larger size limit instead: 

+
 Mitigation: 
 

+
+
 When the postmap(1) or postalias(1) command fails with an
+MDB_MAP_FULL error, it expands the database file size to the current
+LMDB map size limit before terminating.  

+
+
 When the postmap(1) or postalias(1) command opens an LMDB file
+larger than lmdb_map_size/3, it logs a warning and uses a larger
+size limit instead: 

 
 
  warning: filename.lmdb: file size 15024128 ≥
 (lmdb map size limit 16777216)/3 -- using a larger map size limit
 

 
-
 This can be used to automate recovery and avoid the need for
-human intervention. Just keep running "postmap lmdb:filename".
-After each failure it will use a 3x larger size limit, and eventually
-the "database full" error will disappear. 

+
 The two steps above can be used to automate recovery and avoid
+the need for human intervention. Just repeat "postmap lmdb:filename"
+(up to some limit).  After each failure it will use a 3x larger
+size limit, and eventually the "database full" error will disappear.
+

 
 
 Prevention: 
 
 
 Monitor your LMDB files and make
 sure that lmdb_map_size > 3x the largest LMDB file size. 

@@ -215,8 +230,6 @@ full" error will disappear, at least for a while.  

 sure that lmdb_map_size > 3x the largest LMDB file size. 

 
 
 
-
-
 
 Non-obvious recovery with postmap(1)/postalias(1)/tlsmgr(8)
 from a corrupted database.  

 
@@ -229,7 +242,7 @@ does not exist with other Postfix databases.  
 
 
 
 Background: 
 
 
 The Postfix LMDB database client
 does not truncate the database file.  Instead it attempts to create
-a transaction for a "drop" request and subsequent "store" requests.
+a transaction for a "drop" request plus subsequent "store" requests.
 That is obviously not possible with a corrupted database file. 

 

 
@@ -241,7 +254,15 @@ then rebuild the file with the postmap(1) or postalias(1) command,
 or wait until the tlsmgr(8) daemon restarts automatically.  

 
 
-
 Prevention: 
 
 
 Arrange your file systems such that
-they never run out of free space.  
 
 
+
 Prevention: 
 
 
+
+
 Arrange your file systems such that they never run out of free
+space. 

 
-
+
 Use ECC memory to detect and correct silent corruption of
+in-memory file system data and metadata. 

+
+
 Use a file system such as ZFS to detect and correct silent
+corruption of on-disk file system data and metadata. 

+
+
 
diff --git a/postfix/proto/SASL_README.html b/postfix/proto/SASL_README.html
index 1146a36a1..49e7fb7cf 100644
--- a/postfix/proto/SASL_README.html
+++ b/postfix/proto/SASL_README.html
@@ -198,6 +198,9 @@ specified on the Dovecot server side in dovecot.conf.
 
 The following example assumes that the Postfix queue is under
 /var/spool/postfix/. 

 
+
 Note: the example uses Dovecot 1 syntax, See http://www.dovecot.org/
+for newer syntax. 

+
 

 
  1 /etc/dovecot.conf:
@@ -238,6 +241,9 @@ and 10-master.conf. 

 
 The following examples assume that Postfix should communicate
 with Dovecot on TCP port 12345. 

 
+
 Note: the examples use Dovecot 1 syntax, See http://www.dovecot.org/
+for newer syntax. 

+
 

 
 1 /etc/dovecot/conf.d/10-auth.conf:
diff --git a/postfix/proto/master b/postfix/proto/master
index d4f2dc297..acd381caf 100644
--- a/postfix/proto/master
+++ b/postfix/proto/master
@@ -63,8 +63,8 @@
 #	accepted. The host part (and colon) may be omitted.  Either
 #	host or port may be given in symbolic form (host or service
 #	name) or in numeric form (IP address or port number).
-#	Host information may be enclosed inside "[]", but this form
-#	is not necessary.
+#	Host information may be enclosed inside "[]"; this form
+#	is necessary only with IPv6 addresses.
 # .sp
 #	Examples: a service named \fB127.0.0.1:smtp\fR or \fB::1:smtp\fR
 #	receives
diff --git a/postfix/proto/pcre_table b/postfix/proto/pcre_table
index 5b7e2b206..7cf6ea4e7 100644
--- a/postfix/proto/pcre_table
+++ b/postfix/proto/pcre_table
@@ -138,11 +138,12 @@
 # TEXT SUBSTITUTION
 # .ad
 # .fi
-#	Substitution of substrings from the matched expression into the result
-#	string is possible using the conventional perl syntax ($1, $2, etc.);
-#	specify $$ to produce a $ character as output.
-#	The macros in the result string may need to be written as ${n}
-#	or $(n) if they aren't followed by whitespace.
+#	Substitution of substrings (text that matches patterns
+#	inside "()") from the matched expression into the result
+#	string is requested with $1, $2, etc.; specify $$ to produce
+#	a $ character as output.
+#	The macros in the result string may need to be written as
+#	${n} or $(n) if they aren't followed by whitespace.
 #
 #	Note: since negated patterns (those preceded by \fB!\fR) return a
 #	result when the expression does not match, substitutions are not
diff --git a/postfix/proto/postconf.proto b/postfix/proto/postconf.proto
index 213e50c4e..5a1608c66 100644
--- a/postfix/proto/postconf.proto
+++ b/postfix/proto/postconf.proto
@@ -5529,7 +5529,8 @@ for rejected requests (default: 501).
 
reject_non_fqdn_helo_hostname (with Postfix < 2.3: reject_non_fqdn_hostname)

 
 
Reject the request when the HELO or EHLO hostname is not in
-fully-qualified domain form, as required by the RFC. Note: specify
+fully-qualified domain or address literal form, as required by the
+RFC. Note: specify
 "smtpd_helo_required = yes" to fully enforce this restriction
 (without "smtpd_helo_required = yes", a client can simply skip
 reject_non_fqdn_helo_hostname by not sending HELO or EHLO).  

diff --git a/postfix/proto/regexp_table b/postfix/proto/regexp_table
index ae062f675..1f1de40a1 100644
--- a/postfix/proto/regexp_table
+++ b/postfix/proto/regexp_table
@@ -106,12 +106,12 @@
 # TEXT SUBSTITUTION
 # .ad
 # .fi
-#	Substitution of substrings from the matched expression into the result
-#	string is possible using $1, $2, etc.;
-#	specify $$ to produce a $ character as output.
-#	The macros in the result string
-#	may need to be written as ${n} or $(n) if they aren't followed
-#	by whitespace.
+#	Substitution of substrings (text that matches patterns
+#	inside "()") from the matched expression into the result
+#	string is requested with $1, $2, etc.; specify $$ to produce
+#	a $ character as output.
+#	The macros in the result string may need to be written as
+#	${n} or $(n) if they aren't followed by whitespace.
 #
 #	Note: since negated patterns (those preceded by \fB!\fR) return a
 #	result when the expression does not match, substitutions are not
diff --git a/postfix/src/global/mail_params.c b/postfix/src/global/mail_params.c
index 5bc95c659..108b4a69d 100644
--- a/postfix/src/global/mail_params.c
+++ b/postfix/src/global/mail_params.c
@@ -96,7 +96,7 @@
 /*	char   *var_proxywrite_service;
 /*	int	var_db_create_buf;
 /*	int	var_db_read_buf;
-/*	int	var_lmdb_map_size;
+/*	long	var_lmdb_map_size;
 /*	int	var_mime_maxdepth;
 /*	int	var_mime_bound_len;
 /*	int	var_header_limit;
@@ -288,7 +288,7 @@ char   *var_proxymap_service;
 char   *var_proxywrite_service;
 int     var_db_create_buf;
 int     var_db_read_buf;
-int     var_lmdb_map_size;
+long    var_lmdb_map_size;
 int     var_mime_maxdepth;
 int     var_mime_bound_len;
 int     var_header_limit;
@@ -601,7 +601,6 @@ void    mail_params_init()
 	VAR_FAULT_INJ_CODE, DEF_FAULT_INJ_CODE, &var_fault_inj_code, 0, 0,
 	VAR_DB_CREATE_BUF, DEF_DB_CREATE_BUF, &var_db_create_buf, 1, 0,
 	VAR_DB_READ_BUF, DEF_DB_READ_BUF, &var_db_read_buf, 1, 0,
-	VAR_LMDB_MAP_SIZE, DEF_LMDB_MAP_SIZE, &var_lmdb_map_size, 1, 0,
 	VAR_HEADER_LIMIT, DEF_HEADER_LIMIT, &var_header_limit, 1, 0,
 	VAR_TOKEN_LIMIT, DEF_TOKEN_LIMIT, &var_token_limit, 1, 0,
 	VAR_MIME_MAXDEPTH, DEF_MIME_MAXDEPTH, &var_mime_maxdepth, 1, 0,
@@ -612,6 +611,7 @@ void    mail_params_init()
     };
     static const CONFIG_LONG_TABLE long_defaults[] = {
 	VAR_MESSAGE_LIMIT, DEF_MESSAGE_LIMIT, &var_message_limit, 0, 0,
+	VAR_LMDB_MAP_SIZE, DEF_LMDB_MAP_SIZE, &var_lmdb_map_size, 1, 0,
 	0,
     };
     static const CONFIG_TIME_TABLE time_defaults[] = {
diff --git a/postfix/src/global/mail_params.h b/postfix/src/global/mail_params.h
index 5716aa4c9..7bd987da6 100644
--- a/postfix/src/global/mail_params.h
+++ b/postfix/src/global/mail_params.h
@@ -2762,7 +2762,7 @@ extern int var_db_read_buf;
   */
 #define VAR_LMDB_MAP_SIZE		"lmdb_map_size"
 #define DEF_LMDB_MAP_SIZE		(16 * 1024 *1024)
-extern int var_lmdb_map_size;
+extern long var_lmdb_map_size;
 
  /*
   * Named queue file attributes.
diff --git a/postfix/src/global/mail_version.h b/postfix/src/global/mail_version.h
index c3d77779a..7efe5848c 100644
--- a/postfix/src/global/mail_version.h
+++ b/postfix/src/global/mail_version.h
@@ -20,7 +20,7 @@
   * Patches change both the patchlevel and the release date. Snapshots have no
   * patchlevel; they change the release date only.
   */
-#define MAIL_RELEASE_DATE	"20130710"
+#define MAIL_RELEASE_DATE	"20130818"
 #define MAIL_VERSION_NUMBER	"2.11"
 
 #ifdef SNAPSHOT
diff --git a/postfix/src/util/dict_lmdb.c b/postfix/src/util/dict_lmdb.c
index 3b692facb..e9d07458c 100644
--- a/postfix/src/util/dict_lmdb.c
+++ b/postfix/src/util/dict_lmdb.c
@@ -27,6 +27,12 @@
 /*	file size, so that there is room for the file to grow. This
 /*	ensures that a process can recover from a "table full" error
 /*	with a simple terminate-and-restart.
+/*
+/*	As a second safety measure, when an update or delete operation
+/*	runs into an MDB_MAP_FULL error, Postfix will extend the
+/*	database file to the current ".lmdb" file size, so that the
+/*	above workaround will be triggered the next time the database
+/*	is opened.
 /* DIAGNOSTICS
 /*	Fatal errors: cannot open file, file write error, out of memory.
 /* SEE ALSO
@@ -90,6 +96,43 @@ size_t  dict_lmdb_map_size = (10 * 1024 * 1024);	/* 10MB default mmap
 unsigned int dict_lmdb_max_readers = 216;	/* 200 postfix processes,
 						 * plus some extra */
 
+ /*
+  * Out-of-space safety net. When an update or delete operation fails with
+  * MDB_MAP_FULL, extend the database file size so that the next
+  * dict_lmdb_open() call will force a 3x over-allocation.
+  * 
+  * XXX This strategy assumes that a bogus file size will not affect LMDB
+  * operation. In private communication on August 18, 2013, Howard Chu
+  * confirmed this as follows: "It will have no effect. LMDB internally
+  * accounts for the last used page#, the filesystem's notion of filesize
+  * isn't used for any purpose."
+  * 
+  * We make no assumptions about which LMDB operations may fail with
+  * MDB_MAP_FULL. Instead we wrap all LMDB operations inside a Postfix
+  * function that may change a database.
+  */
+#define DICT_LMDB_WRAPPER(dict, status, operation) \
+    ((status = operation) == MDB_MAP_FULL ? \
+	(dict_lmdb_grow(dict), status) : status)
+
+/* dict_lmdb_grow - grow the DB file if the last txn failed to grow it */
+
+static void dict_lmdb_grow(DICT *dict)
+{
+    struct stat st;
+    char   *mdb_path = concatenate(dict->name, "." DICT_TYPE_LMDB, (char *) 0);
+
+    /*
+     * After MDB_MAP_FULL error, expand the file size to trigger the 3x size
+     * limit workaround on the next open() attempt.
+     */
+    if (stat(mdb_path, &st) == 0 && st.st_size < dict_lmdb_map_size
+	&& truncate(mdb_path, dict_lmdb_map_size) < 0)
+	msg_warn("dict_lmdb_grow: cannot grow database file %s:%s: %m",
+		 dict->type, dict->name);
+    myfree(mdb_path);
+}
+
 /* dict_lmdb_lookup - find database entry */
 
 static const char *dict_lmdb_lookup(DICT *dict, const char *name)
@@ -223,14 +266,16 @@ static int dict_lmdb_update(DICT *dict, const char *name, const char *value)
      */
     if (dict_lmdb->txn)
 	txn = dict_lmdb->txn;
-    else if ((status = mdb_txn_begin(dict_lmdb->env, NULL, 0, &txn)))
+    else if (DICT_LMDB_WRAPPER(dict, status,
+			       mdb_txn_begin(dict_lmdb->env, NULL, 0, &txn)))
 	msg_fatal("%s: txn_begin(write) dictionary: %s", dict_lmdb->dict.name, mdb_strerror(status));
 
     /*
      * Do the update.
      */
-    status = mdb_put(txn, dict_lmdb->dbi, &mdb_key, &mdb_value,
-	       (dict->flags & DICT_FLAG_DUP_REPLACE) ? 0 : MDB_NOOVERWRITE);
+    (void) DICT_LMDB_WRAPPER(dict, status,
+			  mdb_put(txn, dict_lmdb->dbi, &mdb_key, &mdb_value,
+	      (dict->flags & DICT_FLAG_DUP_REPLACE) ? 0 : MDB_NOOVERWRITE));
     if (status) {
 	if (status == MDB_KEYEXIST) {
 	    if (dict->flags & DICT_FLAG_DUP_IGNORE)
@@ -247,7 +292,7 @@ static int dict_lmdb_update(DICT *dict, const char *name, const char *value)
     /*
      * Commit the transaction if it's not the global txn.
      */
-    if (!dict_lmdb->txn && ((status = mdb_txn_commit(txn))))
+    if (!dict_lmdb->txn && DICT_LMDB_WRAPPER(dict, status, mdb_txn_commit(txn)))
 	msg_fatal("error committing LMDB database %s: %s", dict_lmdb->dict.name, mdb_strerror(status));
 
     return (status);
@@ -286,7 +331,8 @@ static int dict_lmdb_delete(DICT *dict, const char *name)
      */
     if (dict_lmdb->txn)
 	txn = dict_lmdb->txn;
-    else if ((status = mdb_txn_begin(dict_lmdb->env, NULL, 0, &txn)))
+    else if (DICT_LMDB_WRAPPER(dict, status,
+			       mdb_txn_begin(dict_lmdb->env, NULL, 0, &txn)))
 	msg_fatal("%s: txn_begin(write) dictionary: %s", dict_lmdb->dict.name, mdb_strerror(status));
 
     /*
@@ -296,7 +342,8 @@ static int dict_lmdb_delete(DICT *dict, const char *name)
     if (dict->flags & DICT_FLAG_TRY1NULL) {
 	mdb_key.mv_data = (void *) name;
 	mdb_key.mv_size = klen + 1;
-	status = mdb_del(txn, dict_lmdb->dbi, &mdb_key, NULL);
+	(void) DICT_LMDB_WRAPPER(dict, status,
+			      mdb_del(txn, dict_lmdb->dbi, &mdb_key, NULL));
 	if (status) {
 	    if (status == MDB_NOTFOUND)
 		status = 1;
@@ -314,7 +361,8 @@ static int dict_lmdb_delete(DICT *dict, const char *name)
     if (status > 0 && (dict->flags & DICT_FLAG_TRY0NULL)) {
 	mdb_key.mv_data = (void *) name;
 	mdb_key.mv_size = klen;
-	status = mdb_del(txn, dict_lmdb->dbi, &mdb_key, NULL);
+	(void) DICT_LMDB_WRAPPER(dict, status,
+			      mdb_del(txn, dict_lmdb->dbi, &mdb_key, NULL));
 	if (status) {
 	    if (status == MDB_NOTFOUND)
 		status = 1;
@@ -328,7 +376,7 @@ static int dict_lmdb_delete(DICT *dict, const char *name)
     /*
      * Commit the transaction if it's not the global txn.
      */
-    if (!dict_lmdb->txn && ((rc = mdb_txn_commit(txn))))
+    if (!dict_lmdb->txn && DICT_LMDB_WRAPPER(dict, rc, mdb_txn_commit(txn)))
 	msg_fatal("error committing LMDB database %s: %s", dict_lmdb->dict.name, mdb_strerror(rc));
 
     return (status);
@@ -425,8 +473,9 @@ static void dict_lmdb_close(DICT *dict)
     DICT_LMDB *dict_lmdb = (DICT_LMDB *) dict;
 
     if (dict_lmdb->txn) {
-	int     status = mdb_txn_commit(dict_lmdb->txn);
+	int     status;
 
+	(void) DICT_LMDB_WRAPPER(dict, status, mdb_txn_commit(dict_lmdb->txn));
 	if (status)
 	    msg_fatal("%s: closing dictionary: %s", dict_lmdb->dict.name, mdb_strerror(status));
 	dict_lmdb->cursor = NULL;
@@ -461,7 +510,7 @@ DICT   *dict_lmdb_open(const char *path, int open_flags, int dict_flags)
     char   *mdb_path;
     int     env_flags, status;
 
-    mdb_path = concatenate(path, ".lmdb", (char *) 0);
+    mdb_path = concatenate(path, "." DICT_TYPE_LMDB, (char *) 0);
 
     env_flags = MDB_NOSUBDIR;
     if (open_flags == O_RDONLY)
diff --git a/postfix/src/util/vstring_vstream.c b/postfix/src/util/vstring_vstream.c
index 36c913472..bec981699 100644
--- a/postfix/src/util/vstring_vstream.c
+++ b/postfix/src/util/vstring_vstream.c
@@ -166,7 +166,7 @@ int     vstring_get_null_bound(VSTRING *vp, VSTREAM *fp, ssize_t bound)
     int     c;
 
     if (bound <= 0)
-	msg_panic("vstring_get_nonl_bound: invalid bound %ld", (long) bound);
+	msg_panic("vstring_get_null_bound: invalid bound %ld", (long) bound);
 
     VSTRING_RESET(vp);
     while (bound-- > 0 && (c = VSTREAM_GETC(fp)) != VSTREAM_EOF && c != 0)