From: Wietse Venema Date: Sat, 16 Mar 2013 05:00:00 +0000 (-0500) Subject: postfix-2.11-20130316 X-Git-Tag: v2.11.0-RC1~51 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=2f62ba28458b110dc8b07507561bae69e9393228;p=thirdparty%2Fpostfix.git postfix-2.11-20130316 --- diff --git a/postfix/HISTORY b/postfix/HISTORY index d0d817ed3..1f3a451f6 100644 --- a/postfix/HISTORY +++ b/postfix/HISTORY @@ -18248,12 +18248,20 @@ Apologies for any names omitted. Bugfix: an error handler for smtp_tls_policy_maps lookups was never invoked. File: smtp/smtp_session.c. +20130212 + + Cleanup: logfile message formatting (X: subject_CN=X, + issuer_CN=X, fingerprint=X, pkey_fingerprint=X). File: + tls/tls_client.c. + 20130315 Feature: preliminary LMDB (memory-mapped persistent file) - support by Howard Chu. This feature has a number of unexpected - limitations and is therefore "snapshot only", i.e. it will - not be part of a stable release. Files: proto/postconf.proto, + support by Howard Chu. This feature has unexpected limitations + that don't exist with other Postfix databases, and is + therefore "snapshot only", i.e. it will not be part of a + stable release without further changes to the Postfix LMDB + client or the Postfix dictionary API. Files: proto/postconf.proto, proto/LMDB_README.html, proto/DATABASE_README.html, - util/dict_lmdb.[hc], util/dict_open.c, global/mkmap_lmdb.[hc], - global/mkmap_open.c, postconf/postconf.c. + proto/INSTALL.html util/dict_lmdb.[hc], util/dict_open.c, + global/mkmap_lmdb.[hc], global/mkmap_open.c, postconf/postconf.c. diff --git a/postfix/README_FILES/AAAREADME b/postfix/README_FILES/AAAREADME index 7a93a6ba4..0017b3d81 100644 --- a/postfix/README_FILES/AAAREADME +++ b/postfix/README_FILES/AAAREADME @@ -48,6 +48,7 @@ LLooookkuupp ttaabblleess ((ddaattaabbaasseess)) * DB_README: Berkeley DB Howto * CDB_README: CDB Howto * LDAP_README: LDAP Howto + * LMDB_README: LMDB Howto * MEMCACHE_README: Memcache Howto * MYSQL_README: MySQL Howto * PCRE_README: PCRE Howto diff --git a/postfix/README_FILES/LMDB_README b/postfix/README_FILES/LMDB_README index 7d422e0a3..dfd131373 100644 --- a/postfix/README_FILES/LMDB_README +++ b/postfix/README_FILES/LMDB_README @@ -18,8 +18,11 @@ This document describes: 3. Missing pthread library trouble. - Caution: the current Postfix LMDB client has unexpected limitations. For - this reason, LMDB support is not part of the stable Postfix release. +Caution: + The current Postfix LMDB client has unexpected limitations that don't exist + with other Postfix databases. For this reason, LMDB support will not be + part of the stable Postfix release without further changes to the Postfix + LMDB client or the Postfix dictionary API. BBuuiillddiinngg PPoossttffiixx wwiitthh OOppeennLLDDAAPP LLMMDDBB ssuuppppoorrtt @@ -64,62 +67,90 @@ information is available at http://highlandsun.com/hyc/mdb/. LLiimmiittaattiioonnss ooff PPoossttffiixx LLMMDDBB ddaattaabbaasseess.. - * UUnneexxppeecctteedd ppoossttmmaapp//ppoossttaalliiaass ""ddaattaabbaassee ffuullll"" eerrrroorrss.. +UUnneexxppeecctteedd ppoossttmmaapp((11))//ppoossttaalliiaass((11)) ""ddaattaabbaassee ffuullll"" eerrrroorrss.. - Even if the "postmap filename" command succeeds, the exact same command, - with the exact same input data, may fail subsequently with an MDB_MAP_FULL - error. The reason is that unlike other Postfix databases such as "hash" or - "btree", +Problem: + Even if the "postmap lmdb:filename" command succeeds, the exact same + command (with the exact same input data) may fail subsequently with an + MDB_MAP_FULL error. This problem does not exist with other Postfix + databases. - o LMDB databases have a hard size limit (configured with the - lmdb_map_size configuration parameter). +Background: + LMDB databases have a hard size limit (configured with the lmdb_map_size + configuration parameter). - o The Postfix LMDB database client does not implement the "truncate" - operation. Instead it saves all store requests to a transaction (which - takes up space in addition to the existing data), and commits the - transaction when the database is closed. Only then can the space for - old data be reused. + When executing "postmap lmdb:filename", the Postfix LMDB database client + does not truncate the database file. Instead it saves the "drop" request + and subsequent "store" requests to a transaction (which takes up space in + addition to the existing data), and commits the transaction when it closes + the database. Only then can the space for old data be reused. - This postmap(1) or postalias(1) command failure does not affect Postfix - availability, because the old data still exists in the database. +Impact: + This failure does not affect Postfix availability, because the old data + still exists in the database. - To recover, increase the lmdb_map_size limit in main.cf, and retry the - postmap(1) or postalias(1) command. +Recovery: + Increase the lmdb_map_size limit in main.cf, and retry the postmap(1) or + postalias(1) command. + +PPoossttffiixx ddaaeemmoonn ""ddaattaabbaassee ffuullll"" eerrrroorrss.. - * PPoossttffiixx ddaaeemmoonn ""ddaattaabbaassee ffuullll"" eerrrroorrss.. +Problem: + "database full" errors with daemon programs such as postscreen(8), tlsmgr + (8) or verify(8). This problem does not exist with other Postfix databases. - Unfortunately, "database full" problems will affect Postfix availability - with daemon programs such as postscreen(8), tlsmgr(8) or verify(8). These - daemon programs will continue to fail until someone increases the - lmdb_map_size parameter value. Meanwhile, mail processing comes to a halt. +Impact: + Postfix does not process mail until someone fixes the problem. - * NNoonn--oobbvviioouuss rreeccoovveerryy ffrroomm aa ccoorrrruupptteedd ddaattaabbaassee.. +Recovery: + Increase the lmdb_map_size limit in main.cf, and "reload" Postfix. - Unlike other Postfix databases such as "hash" or "btree", you can't rebuild - a corrupted LMDB database simply by running postmap(1) or postalias(1), as - those commands will crash, too. +NNoonn--oobbvviioouuss rreeccoovveerryy wwiitthh ppoossttmmaapp((11))//ppoossttaalliiaass((11)) ffrroomm aa ccoorrrruupptteedd ddaattaabbaassee.. +Problem: + You cannot rebuild a corrupted LMDB database simply by running postmap(1) + or postalias(1). This problem does not exist with other Postfix databases. + +Background: The reason for this limitation is that the Postfix LMDB database client - does not implement the database "truncate" operation. Instead it tries to - save all store requests to a transaction for later processing. That is - obviously not possible with a corrupted database file. - - To recover, you must first delete the ".lmdb" file by hand, and only then - you can retry the postmap(1) or postalias(1) command. - - * IInnccoommppaattiibbiilliittyy wwiitthh ttllssmmggrr((88)).. - - The Postfix LMDB database client breaks tlsmgr(8) TLS session cache - management. Specifically, it breaks how tlsmgr(8) clears its TLS session - cache databases upon start-up, it breaks how tlsmgr(8) looks up new TLS - session cache entries, and it breaks how tlsmgr(8) automatically recovers - from a corrupted database file. - - The reason for these limitations is that the Postfix LMDB database client - does not implement the database "truncate" operation. Instead it saves all - store requests to a transaction which it commits only when the database is - closed. Therefore, tlsmgr(8) will never find any entries that it stores - after opening its TLS session cache databases. And when the database - becomes corrupted, tlsmgr(8) will keep crashing until someone removes the - file ".lmdb" file by hand. + does not truncate the database file. Instead it attempts to save the "drop" + request and subsequent "store" requests to a transaction for later + processing. That is obviously not possible with a corrupted database file. + +Recovery: + First delete the ".lmdb" file by hand, then rebuild the file with the + postmap(1) or postalias(1) command. + +IInnccoommppaattiibbiilliittyy wwiitthh ttllssmmggrr((88)).. + +Problem: + The Postfix LMDB database client never commits any tlsmgr(8) transaction. + This problem does not exist with other Postfix databases. + +Background: + Instead, it creates a single transaction that accumulates a "drop" request + and all tlsmgr(8) "store" requests that are made during the lifetime of the + process. + +Solution: + This requires changes to the Postfix dictionary API, or to the Postfix LMDB + database client. + +Problem: + The Postfix LMDB database client breaks how tlsmgr(8) automatically + recovers from a corrupted database file. This problem does not exist with + other Postfix databases. + +Background: + The Postfix LMDB database client does not truncate the database file. + Instead it attempts to create a transaction which obviously is not possible + when the database file is corrupted. + +Impact: + The tlsmgr(8) process will keep crashing until someone removes the ".lmdb" + file. + +Recovery: + Remove the the ".lmdb" file by hand, and wait until the tlsmgr(8) process + restarts. diff --git a/postfix/RELEASE_NOTES b/postfix/RELEASE_NOTES index 94a1a46ea..64fee86b6 100644 --- a/postfix/RELEASE_NOTES +++ b/postfix/RELEASE_NOTES @@ -17,7 +17,8 @@ before proceeding. Major changes with snapshot 20130315 ==================================== -Perliminary LMDB support by Howard Chu. This implementation has -unexpected limitations that make it a poor fit, and for this reason -the code is "snapshot only", i.e. it will not be part of the stable -release. See LMDB_README for details. +Preliminary LMDB support by Howard Chu. This implementation has +unexpected limitations that don't exist with other Postfix databases, +and therefore the code is "snapshot only", i.e. it will not be part +of the stable release without further changes to the Postfix LMDB +client or the Postfix dictionary API. See LMDB_README for details. diff --git a/postfix/WISHLIST b/postfix/WISHLIST index 1f5052d1e..8ef201348 100644 --- a/postfix/WISHLIST +++ b/postfix/WISHLIST @@ -8,6 +8,15 @@ Wish list: Spellcheck and double-word check. + It would be nice if the result from one table lookup could + serve as input for another (e.g. virtual aliases before the + list of valid recipients). For this to work the magical + (bare user, domain only, etc.) lookups need to become a + table property, not a property of the client context. + + It would be nice if "bare username" lookup is not hard-coded + for domains in the local address class. + Don't forget Apple's code donation for fetching mail from IMAP server. diff --git a/postfix/html/LMDB_README.html b/postfix/html/LMDB_README.html index df4e32d83..e6f1bb984 100644 --- a/postfix/html/LMDB_README.html +++ b/postfix/html/LMDB_README.html @@ -39,10 +39,12 @@ LMDB support.

-

Caution: the current Postfix LMDB client has unexpected limitations. For this reason, -LMDB support is not part of the stable Postfix release.

-
+
Caution:

The current Postfix LMDB client +has unexpected limitations that don't +exist with other Postfix databases. For this reason, LMDB support +will not be part of the stable Postfix release without further +changes to the Postfix LMDB client or the Postfix dictionary API. +

Building Postfix with OpenLDAP LMDB support

@@ -109,81 +111,114 @@ undefined reference to `pthread_mutex_lock' More information is available at http://highlandsun.com/hyc/mdb/.

-

Limitations of Postfix LMDB databases.

- - +
Impact:

Postfix does not process mail until +someone fixes the problem.

+ +
Recovery:

Increase the lmdb_map_size limit in +main.cf, and "reload" Postfix.

+ + + +

Non-obvious recovery with postmap(1)/postalias(1) +from a corrupted database.

+ +
+ +
Problem:

You cannot rebuild a corrupted LMDB +database simply by running postmap(1) or postalias(1). This problem +does not exist with other Postfix databases.

+ +
Background:

The reason for this limitation is +that the Postfix LMDB database client does not truncate the database +file. Instead it attempts to save the "drop" request and subsequent +"store" requests to a transaction for later processing. That is +obviously not possible with a corrupted database file.

+ +
Recovery:

First delete the ".lmdb" file by hand, +then rebuild the file with the postmap(1) or postalias(1) command. +

+ +
+ +

Incompatibility with tlsmgr(8).

+ +
+ +
Problem:

The Postfix LMDB database client never +commits any tlsmgr(8) transaction. This problem does not exist with +other Postfix databases.

+ +
Background:

Instead, it creates a single +transaction that accumulates a "drop" request and all tlsmgr(8) +"store" requests that are made during the lifetime of the process. +

+ +
Solution:

This requires changes to the Postfix +dictionary API, or to the Postfix LMDB database client.

+ +
Problem:

The Postfix LMDB database client breaks +how tlsmgr(8) automatically recovers from a corrupted database file. +This problem does not exist with other Postfix databases.

+ +
Background:

The Postfix LMDB database client +does not truncate the database file. Instead it attempts to create +a transaction which obviously is not possible when the database +file is corrupted.

+ +
Impact:

The tlsmgr(8) process will keep crashing +until someone removes the ".lmdb" file.

+ +
Recovery:

Remove the the ".lmdb" file by hand, +and wait until the tlsmgr(8) process restarts.

+ +
diff --git a/postfix/html/index.html b/postfix/html/index.html index 77544f5ad..57c6f78b1 100644 --- a/postfix/html/index.html +++ b/postfix/html/index.html @@ -132,6 +132,8 @@ Per-client/user/etc. access
  • LDAP Howto +
  • LMDB Howto +
  • Memcache Howto
  • MySQL Howto diff --git a/postfix/proto/LMDB_README.html b/postfix/proto/LMDB_README.html index 2f7b4248f..adf44346f 100644 --- a/postfix/proto/LMDB_README.html +++ b/postfix/proto/LMDB_README.html @@ -39,10 +39,12 @@ LMDB support.

    -

    Caution: the current Postfix LMDB client has unexpected limitations. For this reason, -LMDB support is not part of the stable Postfix release.

    -
    +
    Caution:

    The current Postfix LMDB client +has unexpected limitations that don't +exist with other Postfix databases. For this reason, LMDB support +will not be part of the stable Postfix release without further +changes to the Postfix LMDB client or the Postfix dictionary API. +

    Building Postfix with OpenLDAP LMDB support

    @@ -109,81 +111,114 @@ http://www.openldap.org. More information is available at http://highlandsun.com/hyc/mdb/.

    -

    Limitations of Postfix LMDB databases.

    - -
      +

      Limitations of Postfix LMDB databases. +

      -

    • Unexpected postmap/postalias "database full" errors. +

      Unexpected postmap(1)/postalias(1) "database full" errors.

      -

      Even if the "postmap filename" command succeeds, the exact same -command, with the exact same input data, may fail subsequently with -an MDB_MAP_FULL error. The reason is that unlike other Postfix -databases such as "hash" or "btree",

      +
      -
        +
        Problem:

        Even if the "postmap lmdb:filename" +command succeeds, the exact same command (with the exact same input +data) may fail subsequently with an MDB_MAP_FULL error. This problem +does not exist with other Postfix databases.

        -

      • LMDB databases have a hard size limit (configured with the +

        Background:
        + +
        + +

        LMDB databases have a hard size limit (configured with the lmdb_map_size configuration parameter).

        -

      • The Postfix LMDB database client does not implement the -"truncate" operation. Instead it saves all store requests to a -transaction (which takes up space in addition to the existing data), -and commits the transaction when the database is closed. Only then -can the space for old data be reused.

        +

        When executing "postmap lmdb:filename", the Postfix LMDB database +client does not truncate the database file. Instead it saves the +"drop" request and subsequent "store" requests to a transaction +(which takes up space in addition to the existing data), and commits +the transaction when it closes the database. Only then can the +space for old data be reused.

        -
      + -

      This postmap(1) or postalias(1) command failure does not affect -Postfix availability, because the old data still exists in the -database.

      +
      Impact:

      This failure does not affect Postfix +availability, because the old data still exists in the database. +

      -

      To recover, increase the lmdb_map_size limit in main.cf, and -retry the postmap(1) or postalias(1) command.

      +
      Recovery:

      Increase the lmdb_map_size limit in +main.cf, and retry the postmap(1) or postalias(1) command.

      +
      -

    • Postfix daemon "database full" errors.

      +
      • -

      Unfortunately, "database full" problems will affect Postfix -availability with daemon programs such as postscreen(8), tlsmgr(8) -or verify(8). These daemon programs will continue to fail until -someone increases the lmdb_map_size parameter value. Meanwhile, -mail processing comes to a halt.

      +

      Postfix daemon "database full" errors.

      -

    • Non-obvious recovery from a corrupted database. -

      +
      -

      Unlike other Postfix databases such as "hash" or "btree", you -can't rebuild a corrupted LMDB database simply by running postmap(1) -or postalias(1), as those commands will crash, too.

      - -

      The reason for this limitation is that the Postfix LMDB database -client does not implement the database "truncate" operation. Instead -it tries to save all store requests to a transaction for later -processing. That is obviously not possible with a corrupted database -file.

      - -

      To recover, you must first delete the ".lmdb" file by hand, and -only then you can retry the postmap(1) or postalias(1) command. -

      - -

    • Incompatibility with tlsmgr(8).

      - -

      The Postfix LMDB database client breaks tlsmgr(8) TLS session -cache management. Specifically, it breaks how tlsmgr(8) clears its -TLS session cache databases upon start-up, it breaks how tlsmgr(8) -looks up new TLS session cache entries, and it breaks how tlsmgr(8) -automatically recovers from a corrupted database file.

      - -

      The reason for these limitations is that the Postfix LMDB -database client does not implement the database "truncate" operation. -Instead it saves all store requests to a transaction which it commits -only when the database is closed. Therefore, tlsmgr(8) will never -find any entries that it stores after opening its TLS session cache -databases. And when the database becomes corrupted, tlsmgr(8) will -keep crashing until someone removes the file ".lmdb" file by hand. -

      +
      Problem:

      "database full" errors with daemon +programs such as postscreen(8), tlsmgr(8) or verify(8). This problem +does not exist with other Postfix databases.

      -
    +
    Impact:

    Postfix does not process mail until +someone fixes the problem.

    + +
    Recovery:

    Increase the lmdb_map_size limit in +main.cf, and "reload" Postfix.

    + + + +

    Non-obvious recovery with postmap(1)/postalias(1) +from a corrupted database.

    + +
    + +
    Problem:

    You cannot rebuild a corrupted LMDB +database simply by running postmap(1) or postalias(1). This problem +does not exist with other Postfix databases.

    + +
    Background:

    The reason for this limitation is +that the Postfix LMDB database client does not truncate the database +file. Instead it attempts to save the "drop" request and subsequent +"store" requests to a transaction for later processing. That is +obviously not possible with a corrupted database file.

    + +
    Recovery:

    First delete the ".lmdb" file by hand, +then rebuild the file with the postmap(1) or postalias(1) command. +

    + +
    + +

    Incompatibility with tlsmgr(8).

    + +
    + +
    Problem:

    The Postfix LMDB database client never +commits any tlsmgr(8) transaction. This problem does not exist with +other Postfix databases.

    + +
    Background:

    Instead, it creates a single +transaction that accumulates a "drop" request and all tlsmgr(8) +"store" requests that are made during the lifetime of the process. +

    + +
    Solution:

    This requires changes to the Postfix +dictionary API, or to the Postfix LMDB database client.

    + +
    Problem:

    The Postfix LMDB database client breaks +how tlsmgr(8) automatically recovers from a corrupted database file. +This problem does not exist with other Postfix databases.

    + +
    Background:

    The Postfix LMDB database client +does not truncate the database file. Instead it attempts to create +a transaction which obviously is not possible when the database +file is corrupted.

    + +
    Impact:

    The tlsmgr(8) process will keep crashing +until someone removes the ".lmdb" file.

    + +
    Recovery:

    Remove the the ".lmdb" file by hand, +and wait until the tlsmgr(8) process restarts.

    + +
    diff --git a/postfix/src/global/mail_version.h b/postfix/src/global/mail_version.h index 702a68240..848be5bb0 100644 --- a/postfix/src/global/mail_version.h +++ b/postfix/src/global/mail_version.h @@ -20,7 +20,7 @@ * Patches change both the patchlevel and the release date. Snapshots have no * patchlevel; they change the release date only. */ -#define MAIL_RELEASE_DATE "20130315" +#define MAIL_RELEASE_DATE "20130316" #define MAIL_VERSION_NUMBER "2.11" #ifdef SNAPSHOT