-Tssize_t
-Tssl_cipher_stack_t
-Tssl_comp_stack_t
--Tstat
-Ttime_t
-Ttlsa_filter
-Tx509_extension_stack_t
Cleanup: Unify/simplify reporting of configuration or other
conditions that prevent DANE security. Viktor Dukhovni.
Files: global/dsn_buf.[hc], tls/tls_dane.c, smtp/smtp_tls_policy.c.
+
+20140110-15
+
+ Miscellaneous documentation cleanups.
Forward secrecy does not protect against active attacks such as forged DNS
replies or forged TLS server certificates. If such attacks are a concern, then
the SMTP client will need to authenticate the remote SMTP server in a
-sufficiently-secure manner. For example, by the fingerprint of the public key
-or certificate. Conventional PKI relies on many trusted parties and is easily
-subverted by a state-funded adversary.
+sufficiently-secure manner. For example, by the fingerprint of a (CA or leaf)
+public key or certificate. Conventional PKI relies on many trusted parties and
+is easily subverted by a state-funded adversary.
-B\bBa\bac\bck\bkg\bgr\bro\bou\bun\bnd\bd
+O\bOv\bve\ber\brv\bvi\bie\bew\bw
Postfix supports forward secrecy of TLS network communication since version
2.2. This support was adopted from Lutz Jänicke's "Postfix TLS patch" for
the Postfix SMTP client and server. See TLS_README for a general description of
Postfix TLS support.
+Topics covered in this document:
+
+ * Give me some background on forward secrecy in Postfix
+
+ o What is Forward Secrecy
+ o Forward Secrecy in TLS
+ o Forward Secrecy in the Postfix SMTP Server
+ o Forward Secrecy in the Postfix SMTP Client
+
+ * Never mind, just show me what it takes to get forward secrecy
+
+ o Getting started, quick and dirty
+ o How can I see that a connection has forward secrecy?
+ o What ciphers provide forward secrecy?
+ o What do "Anonymous", "Untrusted", etc. in Postfix logging mean?
+
+ * Credits
+
W\bWh\bha\bat\bt i\bis\bs F\bFo\bor\brw\bwa\bar\brd\bd S\bSe\bec\bcr\bre\bec\bcy\by
The term "Forward Secrecy" (or sometimes "Perfect Forward Secrecy") is used to
traffic is generally infeasible, and even recovery of individual sessions may
be infeasible given a sufficiently-strong key agreement method.
-Topics covered in this document:
-
- * Forward Secrecy in TLS
- * Forward Secrecy in the Postfix SMTP Server
- * Forward Secrecy in the Postfix SMTP Client
- * Getting started, quick and dirty
- * How can I see that a connection has forward secrecy?
- * What ciphers provide forward secrecy?
- * What do "Anonymous", "Untrusted", etc. in Postfix logging mean?
- * Credits
-
-And last but not least, for the impatient:
-
- * Configuring forward secrecy, quick and dirty
-
F\bFo\bor\brw\bwa\bar\brd\bd S\bSe\bec\bcr\bre\bec\bcy\by i\bin\bn T\bTL\bLS\bS
Early implementations of the SSL protocol do not provide forward secrecy (some
# Postfix 2.6 or 2.7 only. This is default with Postfix 2.8 and later.
smtpd_tls_eecdh_grade = strong
-E\bED\bDH\bH C\bCl\bli\bie\ben\bnt\bt s\bsu\bup\bpp\bpo\bor\brt\bt (\b(P\bPo\bos\bst\btf\bfi\bix\bx >\b>=\b= 2\b2.\b.2\b2)\b)
+E\bED\bDH\bH C\bCl\bli\bie\ben\bnt\bt s\bsu\bup\bpp\bpo\bor\brt\bt (\b(P\bPo\bos\bst\btf\bfi\bix\bx >\b>=\b= 2\b2.\b.2\b2,\b, a\bal\bll\bl s\bsu\bup\bpp\bpo\bor\brt\bte\bed\bd O\bOp\bpe\ben\bnS\bSS\bSL\bL v\bve\ber\brs\bsi\bio\bon\bns\bs)\b)
This works "out of the box" without additional configuration.
-E\bED\bDH\bH S\bSe\ber\brv\bve\ber\br s\bsu\bup\bpp\bpo\bor\brt\bt (\b(P\bPo\bos\bst\btf\bfi\bix\bx >\b>=\b= 2\b2.\b.2\b2)\b)
+E\bED\bDH\bH S\bSe\ber\brv\bve\ber\br s\bsu\bup\bpp\bpo\bor\brt\bt (\b(P\bPo\bos\bst\btf\bfi\bix\bx >\b>=\b= 2\b2.\b.2\b2,\b, a\bal\bll\bl s\bsu\bup\bpp\bpo\bor\brt\bte\bed\bd O\bOp\bpe\ben\bnS\bSS\bSL\bL v\bve\ber\brs\bsi\bio\bon\bns\bs)\b)
Optionally generate non-default Postfix SMTP server EDH parameters for improved
security against pre-computation attacks and for compatibility with Debian-
The verification levels below are subject to man-in-the-middle attacks to
different degrees. If such attacks are a concern, then the SMTP client will
need to authenticate the remote SMTP server in a sufficiently-secure manner.
-For example, by the fingerprint of the public key or certificate. Remember that
-conventional PKI relies on many trusted parties and is easily subverted by a
-state-funded adversary.
+For example, by the fingerprint of a (CA or leaf) public key or certificate.
+Remember that conventional PKI relies on many trusted parties and is easily
+subverted by a state-funded adversary.
A\bAn\bno\bon\bny\bym\bmo\bou\bus\bs (no peer certificate)
P\bPo\bos\bst\btf\bfi\bix\bx S\bSM\bMT\bTP\bP c\bcl\bli\bie\ben\bnt\bt:\b: With opportunistic TLS (the "may" security level) the
P\bPo\bos\bst\btf\bfi\bix\bx S\bSM\bMT\bTP\bP s\bse\ber\brv\bve\ber\br:\b: The remote SMTP client certificate was signed by a CA
that the Postfix SMTP server trusts. The Postfix SMTP server never verifies
- the remote SMTP client name against the names in the certificate. Since the
- client chooses to connect to the server, the Postfix SMTP server has no
- expectation of a particular client hostname.
+ the remote SMTP client name against the names in the client certificate.
+ Since the client chooses to connect to the server, the Postfix SMTP server
+ has no expectation of a particular client hostname.
-V\bVe\ber\bri\bif\bfi\bie\bed\bd (peer certificate signed by trusted CA, verified peer name)
+V\bVe\ber\bri\bif\bfi\bie\bed\bd (peer certificate signed by trusted CA and verified peer name; or:
+peer certificate with expected public-key or certificate fingerprint)
P\bPo\bos\bst\btf\bfi\bix\bx S\bSM\bMT\bTP\bP c\bcl\bli\bie\ben\bnt\bt:\b: The remote SMTP server's certificate was signed by a
- CA that the Postfix SMTP client trusts, and it matches one of the expected
- server names. This implies that the Postfix SMTP client enforced
- verification for the destination server name, otherwise the verification
- status would have been just "Trusted".
-
- P\bPo\bos\bst\btf\bfi\bix\bx S\bSM\bMT\bTP\bP s\bse\ber\brv\bve\ber\br:\b: The status is never "Verified", as the Postfix SMTP
- server never verifies the remote SMTP client name against the names in the
- certificate.
+ CA that the Postfix SMTP client trusts, and the certificate name matches
+ the destination or server name(s). The Postfix SMTP client was configured
+ to require a verified name, otherwise the verification status would have
+ been just "Trusted".
+
+ P\bPo\bos\bst\btf\bfi\bix\bx S\bSM\bMT\bTP\bP c\bcl\bli\bie\ben\bnt\bt:\b: The "Verified" status may also mean that the Postfix
+ SMTP client successfully matched the expected fingerprint against the
+ remote SMTP server public key or certificate. The expected fingerprint may
+ come from smtp_tls_policy_maps or from TLSA (secure) DNS records. The
+ Postfix SMTP client ignores the CA signature.
+
+ P\bPo\bos\bst\btf\bfi\bix\bx S\bSM\bMT\bTP\bP s\bse\ber\brv\bve\ber\br:\b: The status is never "Verified", because the Postfix
+ SMTP server never verifies the remote SMTP client name against the names in
+ the client certificate, and because the Postfix SMTP does not expect a
+ specific fingerprint in the client public key or certificate.
C\bCr\bre\bed\bdi\bit\bts\bs
* Configuring LMDB settings.
- * Supported minimum LMDB patchlevel.
+ * Using LMDB maps with non-Postfix programs.
+
+ * Required minimum LMDB patchlevel.
+
+ * Credits.
B\bBu\bui\bil\bld\bdi\bin\bng\bg P\bPo\bos\bst\btf\bfi\bix\bx w\bwi\bit\bth\bh L\bLM\bMD\bDB\bB s\bsu\bup\bpp\bpo\bor\brt\bt
behavior.
* lmdb_map_size (default: 16777216). This setting specifies the initial LMDB
- database size limit in bytes. Each time a database becomes full, its size
+ database size limit in bytes. Each time a database becomes "full", its size
limit is doubled. The maximum size is the largest signed integer value of
"long".
-S\bSu\bup\bpp\bpo\bor\brt\bte\bed\bd m\bmi\bin\bni\bim\bmu\bum\bm L\bLM\bMD\bDB\bB p\bpa\bat\btc\bch\bhl\ble\bev\bve\bel\bl
+U\bUs\bsi\bin\bng\bg L\bLM\bMD\bDB\bB m\bma\bap\bps\bs w\bwi\bit\bth\bh n\bno\bon\bn-\b-P\bPo\bos\bst\btf\bfi\bix\bx p\bpr\bro\bog\bgr\bra\bam\bms\bs
+
+Programs that use LMDB's built-in locking protocol will corrupt a Postfix LMDB
+database or will read garbage.
+
+Postfix does not use LMDB's built-in locking protocol, because that would
+require world-writable lockfiles, and would violate Postfix security policy.
+Instead, Postfix uses external locks based on fcntl(2) to prevent writers from
+corrupting the database, and to prevent readers from receiving garbage.
+
+See lmdb_table(5) for a detailed description of the locking protocol that all
+programs must use when they access a Postfix LMDB database.
-Currently, Postfix supports LMDB 0.9.11 or later. The supported minimum LMDB
-patchlevel has evolved over time, as the result of deployment experience with
-Postfix.
+R\bRe\beq\bqu\bui\bir\bre\bed\bd m\bmi\bin\bni\bim\bmu\bum\bm L\bLM\bMD\bDB\bB p\bpa\bat\btc\bch\bhl\ble\bev\bve\bel\bl
+
+Currently, Postfix requires LMDB 0.9.11 or later. The required minimum LMDB
+patchlevel has evolved over time, as the result of Postfix deployment
+experience:
* LMDB 0.9.11 allows Postfix daemons to log an LMDB error message, instead of
falling out of the sky without any notification.
- * LMDB 0.9.10 closes an information leak where LMDB was writing up to 4kbyte-
- chunks of uninitialized heap memory to the database, persisting information
- that was not meant to be persisted, or sharing information that was not
- meant to be shared.
+ * LMDB 0.9.10 closes an information leak where LMDB was writing up to 4-kbyte
+ chunks of uninitialized heap memory to the database. This would persist
+ information that was not meant to be persisted, or share information that
+ was not meant to be shared.
+
+ * LMDB 0.9.9 allows Postfix to use external (fcntl()-based) locks, instead of
+ having to use world-writable LMDB lock files, violating the Postfix
+ security model in multiple ways.
+
+ * LMDB 0.9.8 allows Postfix to recover from a "database full" error without
+ having to close the database. This version adds support to update the
+ database size limit on-the-fly. This is necessary because Postfix database
+ sizes vary with mail server load.
+
+ * LMDB 0.9.7 allows the postmap(1) and postalias(1) commands to use a bulk-
+ mode transaction larger than the amount of physical memory. This is
+ necessary because LMDB supports databases larger than physical memory.
+
+C\bCr\bre\bed\bdi\bit\bts\bs
- * LMDB 0.9.8 allows Postfix to use external (fcntl()-based) locks, instead of
- having to use world-writable LMDB lock files.
+ * Howard Chu contributed the initial Postfix dict_lmdb driver.
- * LMDB 0.9.8 allows an application to recover from a "database full" error
- without having to close the database, by adding support to update the
- database size limit on-the-fly; and it adds support for an application to
- adopt someone elses change to the database size limit, without having to
- close the database.
+ * Wietse Venema wrote an abstraction layer (slmdb) that behaves more like
+ Berkeley DB, NDBM, etc. This layer automatically retries an LMDB request
+ when a database needs to be resized, or after a database was resized by a
+ different process.
- * LMDB 0.9.7 allows the postmap and postalias commands to use a bulk-mode
- transaction larger than the amount of physical memory. This is necessary
- because LMDB supports databases larger than physical memory.
+ * Howard and Wietse went through many iterations with changes to both LMDB
+ and Postfix, with input from Viktor Dukhovni.
protocol" tests. These tests use an SMTP protocol engine that is built into the
postscreen(8) server.
-Important note: deep protocol tests are disabled by default. They are more
+Important note: these protocol tests are disabled by default. They are more
intrusive than the pregreet and DNSBL tests, and they have limitations as
discussed next.
- * When a good client passes the deep protocol tests, postscreen(8) adds the
- client to the temporary whitelist but it cannot hand off the "live"
- connection to a Postfix SMTP server process in the middle of the session.
- Instead, postscreen(8) defers mail delivery attempts with a 4XX status,
- logs the helo/sender/recipient information, and waits for the client to
- disconnect.
+ * The main limitation of "after 220 greeting" tests is that a new client must
+ disconnect after passing these tests (reason: postscreen is not a proxy).
+ Then the client must reconnect from the same IP address before it can
+ deliver mail. The following measures may help to avoid email delays:
+
+ o Allow "good" clients to skip tests with the
+ postscreen_dnsbl_whitelist_threshold feature (Postfix 2.11 and later).
+ This is especially effective for sites such as Google that never retry
+ immediately from the same IP address.
- The next time the client connects it will be allowed to talk to a Postfix
- SMTP server process to deliver its mail. To minimize the impact of this
- limitation, postscreen(8) gives deep protocol tests a relatively long
- expiration time.
+ o Small sites: Configure postscreen(8) to listen on multiple IP
+ addresses, published in DNS as different IP addresses for the same MX
+ hostname or for different MX hostnames. This avoids mail delivery
+ delays with clients that reconnect immediately from the same IP
+ address.
+
+ o Large sites: Share the postscreen(8) cache between different Postfix
+ MTAs with a large-enough memcache_table(5). Again, this avoids mail
+ delivery delays with clients that reconnect immediately from the same
+ IP address.
* postscreen(8)'s built-in SMTP engine does not implement the AUTH, XCLIENT,
- and XFORWARD features. AUTH support may be added in a future version. In
- the mean time, if you need to make these services available on port 25,
- then do not enable the tests after the 220 server greeting.
+ and XFORWARD features. If you need to make these services available on port
+ 25, then do not enable the tests after the 220 server greeting.
+
+ * End-user clients should connect directly to the submission service, so that
+ they never have to deal with postscreen(8)'s tests.
-End-user clients should connect directly to the submission service, so that
-they never have to deal with postscreen(8)'s tests.
+The following "after 220 greeting" tests are available:
* Command pipelining test
* Non-SMTP command test
verification, where the CA public key or the server certificate is
identified via DNSSEC lookup.
-This feature introduces a new TLS security level called "dane"
-(DNS-based Authentication of Named Entities) that uses DNSSEC to
-look up CA information for a server TLS certificate. The details
+This feature introduces new TLS security levels called "dane" and
+"dane-only" (DNS-based Authentication of Named Entities) that use
+DNSSEC to look up CA or server certificate information. The details
of DANE core protocols are still evolving, as are the details of
how DANE should be used in the context of SMTP. Postfix implements
-what appears to be a "rational" subset of the DANE profiles.
-
-The problem with PKI is that there are literally hundreds of
-organizations world-wide that can provide a certificate in anyone's
-name. There have been widely-published incidents in recent history
-where a certificate authority gave out an inappropriate certificate
-(e.g., a certificate in the name of Microsoft to someone who did
-not represent Microsoft), where a CA was compromised (e.g., DigiNotar,
-Comodo), or where a CA made operational mistakes (e.g., TURKTRUST).
-Another concern is that a legitimate CA might be coerced to provide
-a certificate that allows its government to play man-in-the-middle
-on TLS traffic and observe the plaintext.
+what appears to be a "rational" subset of the DANE profiles that
+is suitable for SMTP.
+
+The problem with conventional PKI is that there are literally
+hundreds of organizations world-wide that can provide a certificate
+in anyone's name. There have been widely-published incidents in
+recent history where a certificate authority gave out an inappropriate
+certificate (e.g., a certificate in the name of Microsoft to someone
+who did not represent Microsoft), where a CA was compromised (e.g.,
+DigiNotar, Comodo), or where a CA made operational mistakes (e.g.,
+TURKTRUST). Another concern is that a legitimate CA might be coerced
+to provide a certificate that allows its government to play
+man-in-the-middle on TLS traffic and observe the plaintext.
Major changes - LMDB database support
-------------------------------------
LMDB is a memory-mapped database that was originally developed as
-part of OpenLDAP. The Postfix LMDB driver was originally written
-by Howard Chu, LMDB's creator. Support for LMDB has evolved throughout
-the Postfix 2.11 development cycle.
+part of OpenLDAP. The Postfix LMDB driver was originally contributed
+by Howard Chu, LMDB's creator.
-LMDB can be used for all Postfix lookup table and cache storage.
-It is the first persistent Postfix database that can be shared among
+LMDB can be used for all Postfix lookup tables and caches. It is
+the first persistent Postfix database that can be shared among
multiple writers such as postscreen daemons (Postfix already supported
-non-persistent memcached caches). See lmdb_table(5) and LMDB_README
-for further information.
+shared non-persistent memcached caches). See lmdb_table(5) and
+LMDB_README for further information, including how to access Postfix
+LMDB databases with non-Postfix programs.
-Postfix currently supports LMDB version 0.9.11 and later. The minimum
+Postfix currently requires LMDB version 0.9.11 or later. The minimum
version may change over time in the light of deployment experience.
Major changes - postscreen whitelisting
---------------------------------------
-[Feature 20130512] Allow an SMTP client to skip postscreen(8) tests
-based on its postscreen_dnsbl_sites score.
+[Feature 20130512] Allow a remote SMTP client to skip postscreen(8)
+tests based on its postscreen_dnsbl_sites score.
-Specify a negative "postscreen_dnsbl_whitelist_threshold" to enable
-this feature. When a client passes the threshold value without
-having failed other tests, all pending or disabled tests are flagged
-as completed.
+Specify a negative "postscreen_dnsbl_whitelist_threshold" value to
+enable this feature. When a client passes the threshold value
+without having failed other tests, all pending or disabled tests
+are flagged as completed.
+
+This feature can mitigate the email delays due to "after 220 greeting"
+protocol tests, which otherwise require that a client reconnects
+before it can deliver mail. Some providers such as Google don't
+retry from the same IP address. This can result in large email
+delivery delays.
Major changes - recipient_delimiter
-----------------------------------
Major changes - milter
----------------------
-[Feature 20131126] Support for ESMTP parameters NOTIFY and ORCPT
-in the SMFIR_ADDRCPT_PAR (add recipient) request. Credits: Andrew
-Ayer.
+[Feature 20131126] Support for ESMTP parameters "NOTIFY" and "ORCPT"
+in the SMFIR_ADDRCPT_PAR (add recipient with parameters) request.
+Credits: Andrew Ayer.
Major changes - mysql
---------------------
as forged DNS replies or forged TLS server certificates. If such
attacks are a concern, then the SMTP client will need to authenticate
the remote SMTP server in a sufficiently-secure manner. For example,
-by the fingerprint of the public key or certificate. Conventional
-PKI relies on many trusted parties and is easily subverted by a
-state-funded adversary. </p>
+by the fingerprint of a (CA or leaf) public key or certificate.
+Conventional PKI relies on many trusted parties and is easily
+subverted by a state-funded adversary. </p>
-<h2> Background </h2>
+<h2> Overview </h2>
<p> Postfix supports forward secrecy of TLS network communication
since version 2.2. This support was adopted from Lutz Jänicke's
server. See <a href="TLS_README.html">TLS_README</a> for a general
description of Postfix TLS support. </p>
-<h2><a name="dfn_fs">What is Forward Secrecy</a></h2>
+<p> Topics covered in this document: </p>
-<p> The term "Forward Secrecy" (or sometimes "Perfect Forward Secrecy")
-is used to describe security protocols in which the confidentiality
-of past traffic is not compromised when long-term keys used by either
-or both sides are later disclosed. </p>
+<ul>
-<p> Forward secrecy is accomplished by negotiating session keys
-using per-session cryptographically-strong random numbers that are
-not saved, and signing the exchange with long-term authentication
-keys. Later disclosure of the long-term keys allows impersonation
-of the key holder from that point on, but not recovery of prior
-traffic, since with forward secrecy, the discarded random key
-agreement inputs are not available to the attacker. </p>
+<li> <p> Give me some background on forward secrecy in Postfix </p>
-<p> Forward secrecy is only "perfect" when brute-force attacks on
-the key agreement algorithm are impractical even for the best-funded
-adversary and the random-number generators used by both parties are
-sufficiently strong. Otherwise, forward secrecy leaves the attacker
-with the challenge of cracking the key-agreement protocol, which
-is likely quite computationally intensive, but may be feasible for
-sessions of sufficiently high value. Thus forward secrecy places
-cost constraints on the efficacy of bulk surveillance, recovering
-all past traffic is generally infeasible, and even recovery of
-individual sessions may be infeasible given a sufficiently-strong
-key agreement method. </p>
-
-<p> Topics covered in this document: </p>
+<ul>
-<ul>
+<li><a href="#dfn_fs">What is Forward Secrecy</a>
<li><a href="#tls_fs">Forward Secrecy in TLS</a>
<li><a href="#client_fs">Forward Secrecy in the Postfix SMTP Client</a>
+</ul>
+
+<li> <p> Never mind, just show me what it takes to get forward
+secrecy </p>
+
+<ul>
+
<li><a href="#quick-start">Getting started, quick and dirty</a>
<li><a href="#test">How can I see that a connection has forward secrecy?</a>
<li><a href="#status"> What do "Anonymous", "Untrusted", etc. in
Postfix logging mean? </a>
-<li><a href="#credits"> Credits </a>
+</ul>
+
+<li> <p> <a href="#credits"> Credits </a> </p>
</ul>
-<p> And last but not least, for the impatient: </p>
+<h2><a name="dfn_fs">What is Forward Secrecy</a></h2>
-<ul>
+<p> The term "Forward Secrecy" (or sometimes "Perfect Forward Secrecy")
+is used to describe security protocols in which the confidentiality
+of past traffic is not compromised when long-term keys used by either
+or both sides are later disclosed. </p>
-<li><a href="#quick-start">Configuring forward secrecy, quick and dirty</a>
+<p> Forward secrecy is accomplished by negotiating session keys
+using per-session cryptographically-strong random numbers that are
+not saved, and signing the exchange with long-term authentication
+keys. Later disclosure of the long-term keys allows impersonation
+of the key holder from that point on, but not recovery of prior
+traffic, since with forward secrecy, the discarded random key
+agreement inputs are not available to the attacker. </p>
-</ul>
+<p> Forward secrecy is only "perfect" when brute-force attacks on
+the key agreement algorithm are impractical even for the best-funded
+adversary and the random-number generators used by both parties are
+sufficiently strong. Otherwise, forward secrecy leaves the attacker
+with the challenge of cracking the key-agreement protocol, which
+is likely quite computationally intensive, but may be feasible for
+sessions of sufficiently high value. Thus forward secrecy places
+cost constraints on the efficacy of bulk surveillance, recovering
+all past traffic is generally infeasible, and even recovery of
+individual sessions may be infeasible given a sufficiently-strong
+key agreement method. </p>
<h2><a name="tls_fs">Forward Secrecy in TLS</a></h2>
</pre>
</blockquote>
-<h3> EDH Client support (Postfix ≥ 2.2) </h3>
+<h3> EDH Client support (Postfix ≥ 2.2, all supported OpenSSL
+versions) </h3>
<p> This works "out of the box" without additional configuration. </p>
-<h3> EDH Server support (Postfix ≥ 2.2) </h3>
+<h3> EDH Server support (Postfix ≥ 2.2, all supported OpenSSL
+versions) </h3>
<p> Optionally generate non-default Postfix SMTP server EDH parameters
for improved security against pre-computation attacks and for
attacks to different degrees. If such attacks are a concern, then
the SMTP client will need to authenticate the remote SMTP server
in a sufficiently-secure manner. For example, by the fingerprint
-of the public key or certificate. Remember that conventional PKI
-relies on many trusted parties and is easily subverted by a
-state-funded adversary. </p>
+of a (CA or leaf) public key or certificate. Remember that
+conventional PKI relies on many trusted parties and is easily
+subverted by a state-funded adversary. </p>
<dl>
<p> <b> Postfix SMTP server:</b> The remote SMTP client certificate
was signed by a CA that the Postfix SMTP server trusts. The Postfix
SMTP server never verifies the remote SMTP client name against the
-names in the certificate. Since the client chooses to connect to
-the server, the Postfix SMTP server has no expectation of a particular
-client hostname. </p>
+names in the client certificate. Since the client chooses to connect
+to the server, the Postfix SMTP server has no expectation of a
+particular client hostname. </p>
</dd>
-<dt><b>Verified</b> (peer certificate signed by trusted CA, verified
-peer name)</dt>
+<dt><b>Verified</b> (peer certificate signed by trusted CA and
+verified peer name; or: peer certificate with expected public-key
+or certificate fingerprint)</dt>
<dd>
<p> <b> Postfix SMTP client:</b> The remote SMTP server's certificate
-was signed by a CA that the Postfix SMTP client trusts, and it
-matches one of the expected server names. This implies that the
-Postfix SMTP client enforced verification for the destination server
-name, otherwise the verification status would have been just
-"Trusted". </p>
+was signed by a CA that the Postfix SMTP client trusts, and the
+certificate name matches the destination or server name(s). The
+Postfix SMTP client was configured to require a verified name,
+otherwise the verification status would have been just "Trusted".
+</p>
+
+<p> <b> Postfix SMTP client:</b> The "Verified" status may also
+mean that the Postfix SMTP client successfully matched the expected
+fingerprint against the remote SMTP server public key or certificate.
+The expected fingerprint may come from <a href="postconf.5.html#smtp_tls_policy_maps">smtp_tls_policy_maps</a> or from
+TLSA (secure) DNS records. The Postfix SMTP client ignores the CA
+signature. </p>
<p> <b> Postfix SMTP server:</b> The status is never "Verified",
-as the Postfix SMTP server never verifies the remote SMTP client
-name against the names in the certificate. </p>
+because the Postfix SMTP server never verifies the remote SMTP
+client name against the names in the client certificate, and because
+the Postfix SMTP does not expect a specific fingerprint in the
+client public key or certificate. </p>
</dd>
<li> <p> <a href="#configure">Configuring LMDB settings</a>. </p>
-<li> <p> <a href="#supported"> Supported minimum LMDB patchlevel</a>. </p>
+<li> <p> <a href="#locking">Using LMDB maps with non-Postfix programs</a>. </p>
+
+<li> <p> <a href="#supported"> Required minimum LMDB patchlevel</a>. </p>
+
+<li> <p> <a href="#credits"> Credits</a>. </p>
</ul>
<ul>
<li> <p> <a href="postconf.5.html#lmdb_map_size">lmdb_map_size</a> (default: 16777216). This setting specifies
-the initial LMDB database size limit in bytes. Each time
-a database becomes full, its size limit is doubled. The maximum
-size is the largest signed integer value of "long". </p>
+the initial LMDB database size limit in bytes. Each time a database
+becomes "full", its size limit is doubled. The maximum size is the
+largest signed integer value of "long". </p>
</ul>
-<h2> <a name="supported"> Supported minimum LMDB patchlevel </a> </h2>
+<h2> <a name="locking">Using LMDB maps with non-Postfix programs</a> </h2>
+
+<p> Programs that use LMDB's built-in locking protocol will corrupt
+a Postfix LMDB database or will read garbage. </p>
+
+<p> Postfix does not use LMDB's built-in locking protocol, because
+that would require world-writable lockfiles, and would violate
+Postfix security policy. Instead, Postfix uses external locks based
+on fcntl(2) to prevent writers from corrupting the database, and
+to prevent readers from receiving garbage. </p>
+
+<p> See <a href="lmdb_table.5.html">lmdb_table(5)</a> for a detailed description of the locking
+protocol that all programs must use when they access a Postfix LMDB
+database. </p>
-<p> Currently, Postfix supports LMDB 0.9.11 or later. The supported
+<h2> <a name="supported"> Required minimum LMDB patchlevel </a> </h2>
+
+<p> Currently, Postfix requires LMDB 0.9.11 or later. The required
minimum LMDB patchlevel has evolved over time, as the result of
-deployment experience with Postfix. </p>
+Postfix deployment experience: </p>
<ul>
</p>
<li> <p> LMDB 0.9.10 closes an information leak where LMDB was
-writing up to 4kbyte-chunks of uninitialized heap memory to the
-database, persisting information that was not meant to be persisted,
-or sharing information that was not meant to be shared. </p>
-
-<li> <p> LMDB 0.9.8 allows Postfix to use external (fcntl()-based)
-locks, instead of having to use world-writable LMDB lock files.
+writing up to 4-kbyte chunks of uninitialized heap memory to the
+database. This would persist information that was not meant to be
+persisted, or share information that was not meant to be shared.
</p>
-<li> <p> LMDB 0.9.8 allows an application to recover from a "database
-full" error without having to close the database, by adding support
-to update the database size limit on-the-fly; and it adds support
-for an application to adopt someone elses change to the database
-size limit, without having to close the database. </p>
+<li> <p> LMDB 0.9.9 allows Postfix to use external (fcntl()-based)
+locks, instead of having to use world-writable LMDB lock files,
+violating the Postfix security model in multiple ways. </p>
-<li> <p> LMDB 0.9.7 allows the postmap and postalias commands to
-use a bulk-mode transaction larger than the amount of physical
+<li> <p> LMDB 0.9.8 allows Postfix to recover from a "database full"
+error without having to close the database. This version adds support
+to update the database size limit on-the-fly. This is necessary
+because Postfix database sizes vary with mail server load. </p>
+
+<li> <p> LMDB 0.9.7 allows the <a href="postmap.1.html">postmap(1)</a> and <a href="postalias.1.html">postalias(1)</a> commands
+to use a bulk-mode transaction larger than the amount of physical
memory. This is necessary because LMDB supports databases larger
than physical memory. </p>
</ul>
+<h2> <a name="credits"> Credits</a> </h2>
+
+<ul>
+
+<li> <p> Howard Chu contributed the initial Postfix dict_lmdb driver.
+</p>
+
+<li> <p> Wietse Venema wrote an abstraction layer (slmdb) that
+behaves more like Berkeley DB, NDBM, etc. This layer automatically
+retries an LMDB request when a database needs to be resized, or
+after a database was resized by a different process. </p>
+
+<li> <p> Howard and Wietse went through many iterations with changes
+to both LMDB and Postfix, with input from Viktor Dukhovni. </p>
+
+</ul>
+
<!--
<h2><a name="limitations">Unexpected failure modes of Postfix LMDB
number of "deep protocol" tests. These tests use an SMTP protocol
engine that is built into the <a href="postscreen.8.html">postscreen(8)</a> server. </p>
-<p> Important note: deep protocol tests are disabled by default.
+<p> Important note: these protocol tests are disabled by default.
They are more intrusive than the pregreet and DNSBL tests, and they
have limitations as discussed next. </p>
<ul>
-<li> <p> When a good client passes the <a href="#after_220">deep
-protocol tests</a>, postscreen(8) adds the client to the temporary
-whitelist but it cannot hand off the "live" connection to a Postfix
-SMTP server process in the middle of the session. Instead, <a href="postscreen.8.html">postscreen(8)</a>
-defers mail delivery attempts with a 4XX status, logs the
-helo/sender/recipient information, and waits for the client to
-disconnect. </p>
+<li> <p> The main limitation of "after 220 greeting" tests is that
+a new client must disconnect after passing these tests (reason:
+postscreen is not a proxy). Then the client must reconnect from
+the same IP address before it can deliver mail. The following
+measures may help to avoid email delays: </p>
-<p> The next time the client connects it will be allowed to talk
-to a Postfix SMTP server process to deliver its mail. To minimize the
-impact of this limitation, <a href="postscreen.8.html">postscreen(8)</a> gives deep protocol tests
-a relatively long expiration time. </p>
+<ul>
-<li> <p> <a href="postscreen.8.html">postscreen(8)</a>'s built-in SMTP engine does not implement
-the AUTH, XCLIENT, and XFORWARD features. AUTH support may be added
-in a future version. In the mean time, if you need to make these
-services available on port 25, then do not enable the tests after
-the 220 server greeting. </p>
+<li> <p> Allow "good" clients to skip tests with the
+<a href="postconf.5.html#postscreen_dnsbl_whitelist_threshold">postscreen_dnsbl_whitelist_threshold</a> feature (Postfix 2.11 and
+later). This is especially effective for sites such as Google that
+never retry immediately from the same IP address. </p>
+
+<li> <p> Small sites: Configure <a href="postscreen.8.html">postscreen(8)</a> to listen on multiple
+IP addresses, published in DNS as different IP addresses for the
+same MX hostname or for different MX hostnames. This avoids mail
+delivery delays with clients that reconnect immediately from the
+same IP address. </p>
+
+<li> <p> Large sites: Share the <a href="postscreen.8.html">postscreen(8)</a> cache between different
+Postfix MTAs with a large-enough <a href="memcache_table.5.html">memcache_table(5)</a>. Again, this
+avoids mail delivery delays with clients that reconnect immediately
+from the same IP address. </p>
</ul>
-<p> End-user clients should connect directly to the submission
+<li> <p> <a href="postscreen.8.html">postscreen(8)</a>'s built-in SMTP engine does not implement the
+AUTH, XCLIENT, and XFORWARD features. If you need to make these
+services available on port 25, then do not enable the tests after
+the 220 server greeting. </p>
+
+<li> <p> End-user clients should connect directly to the submission
service, so that they never have to deal with <a href="postscreen.8.html">postscreen(8)</a>'s tests.
</p>
+</ul>
+
+<p> The following "after 220 greeting" tests are available: </p>
+
<ul>
<li> <a href="#pipelining">Command pipelining test</a>
<b>DESCRIPTION</b>
The Postfix LMDB adapter provides access to a persistent, memory-
mapped, key-value store. The database size is limited only by the size
- of the memory address space (typically 32 or 64 bits) and by the avail-
- able file system space.
+ of the memory address space (typically 31 or 47 bits on 32-bit or
+ 64-bit CPUs, respectively) and by the available file system space.
<b>REQUESTS</b>
The LMDB adapter supports all Postfix lookup table operations. This
When a transaction fails due to a full database, Postfix resizes the
database and retries the transaction.
- Postfix access, address mapping and routing table lookups may generate
- partial search keys such as domain names without one or more subdo-
- mains, network addresses without one or more least-significant octets,
- or email addresses without the localpart, address extension or domain
- portion. This behavior is also found with <a href="DATABASE_README.html#types">btree</a>:, <a href="DATABASE_README.html#types">hash</a>:, or <a href="ldap_table.5.html">ldap</a>:
- tables.
+ Postfix table lookups may generate partial search keys such as domain
+ names without one or more subdomains, network addresses without one or
+ more least-significant octets, or email addresses without the local-
+ part, address extension or domain portion. This behavior is also found
+ with, for example, <a href="DATABASE_README.html#types">btree</a>:, <a href="DATABASE_README.html#types">hash</a>:, or <a href="ldap_table.5.html">ldap</a>: tables.
- Unlike other flat-file based Postfix databases, changes to an LMDB
- database do not trigger automatic daemon program restart.
+ Unlike other flat-file Postfix databases, changes to an LMDB database
+ do not trigger automatic daemon program restart, and do not require
+ "<b>postfix reload</b>".
<b>RELIABILITY</b>
- LMDB's copy-on-write architecture achieves reliable updates, at the
- cost of using more space than some other flat-file databases. Read
- operations are memory-mapped for speed. Write operations are not mem-
- ory-mapped to avoid silent curruption due stray pointer bugs.
-
- The Postfix LMDB adapter implements locking with fcntl(2) locks at
- whole-file granularity. LMDB's native locking scheme would require
- world-writable lockfiles and would therefore violate the Postfix secu-
- rity model.
+ LMDB's copy-on-write architecture provides safe updates, at the cost of
+ using more space than some other flat-file databases. Read operations
+ are memory-mapped for speed. Write operations are not memory-mapped to
+ avoid silent curruption due to stray pointer bugs.
Multiple processes can safely update an LMDB database without serializ-
ing requests through the <a href="proxymap.8.html">proxymap(8)</a> service. This makes LMDB suitable
as a shared cache for <a href="verify.8.html">verify(8)</a> or <a href="postscreen.8.html">postscreen(8)</a> services.
+<b>SYNCHRONIZATION</b>
+ The Postfix LMDB adapter does not use LMDB's built-in locking scheme,
+ because that would require world-writable lockfiles and would violate
+ the Postfix security model. Instead, Postfix uses fcntl(2) locks with
+ whole-file granularity. Programs that use LMDB's built-in locking pro-
+ tocol will corrupt a Postfix LMDB database or will read garbage.
+
+ Every Postfix LMDB database read or write transaction must be protected
+ from start to end with a shared or exclusive fcntl(2) lock. A writer
+ may atomically downgrade an exclusive lock to a shared lock, but it
+ must acquire an exclusive lock between updating the database and start-
+ ing another write transaction.
+
+ Note that fcntl(2) locks do not protect transactions within the same
+ process against each other. If a program cannot avoid making simulta-
+ neous database requests, then it must protect its transactions with in-
+ process locks, in addition to the per-process fcntl(2) locks.
+
<b>CONFIGURATION PARAMETERS</b>
Short-lived programs automatically pick up changes to <a href="postconf.5.html">main.cf</a>. With
long-running daemon programs, Use the command "<b>postfix reload</b>" after a
change queue file names that don't match their message file inode
number). </p>
-<p> Note: see below for how to prepare long queue file names
-for migration to Postfix ≤ 2.8. </p>
+<p> Note: see below for how to convert long queue file names to
+Postfix ≤ 2.8. </p>
<p> Changing the parameter value to "yes" has the following effects:
</p>
.fi
The Postfix LMDB adapter provides access to a persistent,
memory-mapped, key-value store. The database size is limited
-only by the size of the memory address space (typically 32
-or 64 bits) and by the available file system space.
+only by the size of the memory address space (typically 31
+or 47 bits on 32-bit or 64-bit CPUs, respectively) and by
+the available file system space.
.SH "REQUESTS"
.na
.nf
When a transaction fails due to a full database, Postfix
resizes the database and retries the transaction.
-Postfix access, address mapping and routing table lookups
-may generate partial search keys such as domain names without
-one or more subdomains, network addresses without one or
-more least-significant octets, or email addresses without
-the localpart, address extension or domain portion.
-This behavior is also found with btree:, hash:, or ldap:
-tables.
+Postfix table lookups may generate partial search keys such
+as domain names without one or more subdomains, network
+addresses without one or more least-significant octets, or
+email addresses without the localpart, address extension
+or domain portion. This behavior is also found with, for
+example, btree:, hash:, or ldap: tables.
-Unlike other flat-file based Postfix databases, changes to
+Unlike other flat-file Postfix databases, changes to
an LMDB database do not trigger automatic daemon program
-restart.
+restart, and do not require "\fBpostfix reload\fR".
.SH "RELIABILITY"
.na
.nf
.ad
.fi
-LMDB's copy-on-write architecture achieves reliable updates,
+LMDB's copy-on-write architecture provides safe updates,
at the cost of using more space than some other flat-file
databases. Read operations are memory-mapped for speed.
Write operations are not memory-mapped to avoid silent
-curruption due stray pointer bugs.
-
-The Postfix LMDB adapter implements locking with fcntl(2)
-locks at whole-file granularity. LMDB's native locking
-scheme would require world-writable lockfiles and would
-therefore violate the Postfix security model.
+curruption due to stray pointer bugs.
Multiple processes can safely update an LMDB database without
serializing requests through the proxymap(8) service. This
makes LMDB suitable as a shared cache for verify(8) or
postscreen(8) services.
+.SH "SYNCHRONIZATION"
+.na
+.nf
+.ad
+.fi
+The Postfix LMDB adapter does not use LMDB's built-in locking
+scheme, because that would require world-writable lockfiles
+and would violate the Postfix security model. Instead,
+Postfix uses fcntl(2) locks with whole-file granularity.
+Programs that use LMDB's built-in locking protocol will
+corrupt a Postfix LMDB database or will read garbage.
+
+Every Postfix LMDB database read or write transaction must
+be protected from start to end with a shared or exclusive
+fcntl(2) lock. A writer may atomically downgrade an exclusive
+lock to a shared lock, but it must acquire an exclusive
+lock between updating the database and starting another
+write transaction.
+
+Note that fcntl(2) locks do not protect transactions within
+the same process against each other. If a program cannot
+avoid making simultaneous database requests, then it must
+protect its transactions with in-process locks, in addition
+to the per-process fcntl(2) locks.
.SH "CONFIGURATION PARAMETERS"
.na
.nf
change queue file names that don't match their message file inode
number).
.PP
-Note: see below for how to prepare long queue file names
-for migration to Postfix <= 2.8.
+Note: see below for how to convert long queue file names to
+Postfix <= 2.8.
.PP
Changing the parameter value to "yes" has the following effects:
.IP \(bu
as forged DNS replies or forged TLS server certificates. If such
attacks are a concern, then the SMTP client will need to authenticate
the remote SMTP server in a sufficiently-secure manner. For example,
-by the fingerprint of the public key or certificate. Conventional
-PKI relies on many trusted parties and is easily subverted by a
-state-funded adversary. </p>
+by the fingerprint of a (CA or leaf) public key or certificate.
+Conventional PKI relies on many trusted parties and is easily
+subverted by a state-funded adversary. </p>
-<h2> Background </h2>
+<h2> Overview </h2>
<p> Postfix supports forward secrecy of TLS network communication
since version 2.2. This support was adopted from Lutz Jänicke's
server. See <a href="TLS_README.html">TLS_README</a> for a general
description of Postfix TLS support. </p>
-<h2><a name="dfn_fs">What is Forward Secrecy</a></h2>
+<p> Topics covered in this document: </p>
-<p> The term "Forward Secrecy" (or sometimes "Perfect Forward Secrecy")
-is used to describe security protocols in which the confidentiality
-of past traffic is not compromised when long-term keys used by either
-or both sides are later disclosed. </p>
+<ul>
-<p> Forward secrecy is accomplished by negotiating session keys
-using per-session cryptographically-strong random numbers that are
-not saved, and signing the exchange with long-term authentication
-keys. Later disclosure of the long-term keys allows impersonation
-of the key holder from that point on, but not recovery of prior
-traffic, since with forward secrecy, the discarded random key
-agreement inputs are not available to the attacker. </p>
+<li> <p> Give me some background on forward secrecy in Postfix </p>
-<p> Forward secrecy is only "perfect" when brute-force attacks on
-the key agreement algorithm are impractical even for the best-funded
-adversary and the random-number generators used by both parties are
-sufficiently strong. Otherwise, forward secrecy leaves the attacker
-with the challenge of cracking the key-agreement protocol, which
-is likely quite computationally intensive, but may be feasible for
-sessions of sufficiently high value. Thus forward secrecy places
-cost constraints on the efficacy of bulk surveillance, recovering
-all past traffic is generally infeasible, and even recovery of
-individual sessions may be infeasible given a sufficiently-strong
-key agreement method. </p>
-
-<p> Topics covered in this document: </p>
+<ul>
-<ul>
+<li><a href="#dfn_fs">What is Forward Secrecy</a>
<li><a href="#tls_fs">Forward Secrecy in TLS</a>
<li><a href="#client_fs">Forward Secrecy in the Postfix SMTP Client</a>
+</ul>
+
+<li> <p> Never mind, just show me what it takes to get forward
+secrecy </p>
+
+<ul>
+
<li><a href="#quick-start">Getting started, quick and dirty</a>
<li><a href="#test">How can I see that a connection has forward secrecy?</a>
<li><a href="#status"> What do "Anonymous", "Untrusted", etc. in
Postfix logging mean? </a>
-<li><a href="#credits"> Credits </a>
+</ul>
+
+<li> <p> <a href="#credits"> Credits </a> </p>
</ul>
-<p> And last but not least, for the impatient: </p>
+<h2><a name="dfn_fs">What is Forward Secrecy</a></h2>
-<ul>
+<p> The term "Forward Secrecy" (or sometimes "Perfect Forward Secrecy")
+is used to describe security protocols in which the confidentiality
+of past traffic is not compromised when long-term keys used by either
+or both sides are later disclosed. </p>
-<li><a href="#quick-start">Configuring forward secrecy, quick and dirty</a>
+<p> Forward secrecy is accomplished by negotiating session keys
+using per-session cryptographically-strong random numbers that are
+not saved, and signing the exchange with long-term authentication
+keys. Later disclosure of the long-term keys allows impersonation
+of the key holder from that point on, but not recovery of prior
+traffic, since with forward secrecy, the discarded random key
+agreement inputs are not available to the attacker. </p>
-</ul>
+<p> Forward secrecy is only "perfect" when brute-force attacks on
+the key agreement algorithm are impractical even for the best-funded
+adversary and the random-number generators used by both parties are
+sufficiently strong. Otherwise, forward secrecy leaves the attacker
+with the challenge of cracking the key-agreement protocol, which
+is likely quite computationally intensive, but may be feasible for
+sessions of sufficiently high value. Thus forward secrecy places
+cost constraints on the efficacy of bulk surveillance, recovering
+all past traffic is generally infeasible, and even recovery of
+individual sessions may be infeasible given a sufficiently-strong
+key agreement method. </p>
<h2><a name="tls_fs">Forward Secrecy in TLS</a></h2>
</pre>
</blockquote>
-<h3> EDH Client support (Postfix ≥ 2.2) </h3>
+<h3> EDH Client support (Postfix ≥ 2.2, all supported OpenSSL
+versions) </h3>
<p> This works "out of the box" without additional configuration. </p>
-<h3> EDH Server support (Postfix ≥ 2.2) </h3>
+<h3> EDH Server support (Postfix ≥ 2.2, all supported OpenSSL
+versions) </h3>
<p> Optionally generate non-default Postfix SMTP server EDH parameters
for improved security against pre-computation attacks and for
attacks to different degrees. If such attacks are a concern, then
the SMTP client will need to authenticate the remote SMTP server
in a sufficiently-secure manner. For example, by the fingerprint
-of the public key or certificate. Remember that conventional PKI
-relies on many trusted parties and is easily subverted by a
-state-funded adversary. </p>
+of a (CA or leaf) public key or certificate. Remember that
+conventional PKI relies on many trusted parties and is easily
+subverted by a state-funded adversary. </p>
<dl>
<p> <b> Postfix SMTP server:</b> The remote SMTP client certificate
was signed by a CA that the Postfix SMTP server trusts. The Postfix
SMTP server never verifies the remote SMTP client name against the
-names in the certificate. Since the client chooses to connect to
-the server, the Postfix SMTP server has no expectation of a particular
-client hostname. </p>
+names in the client certificate. Since the client chooses to connect
+to the server, the Postfix SMTP server has no expectation of a
+particular client hostname. </p>
</dd>
-<dt><b>Verified</b> (peer certificate signed by trusted CA, verified
-peer name)</dt>
+<dt><b>Verified</b> (peer certificate signed by trusted CA and
+verified peer name; or: peer certificate with expected public-key
+or certificate fingerprint)</dt>
<dd>
<p> <b> Postfix SMTP client:</b> The remote SMTP server's certificate
-was signed by a CA that the Postfix SMTP client trusts, and it
-matches one of the expected server names. This implies that the
-Postfix SMTP client enforced verification for the destination server
-name, otherwise the verification status would have been just
-"Trusted". </p>
+was signed by a CA that the Postfix SMTP client trusts, and the
+certificate name matches the destination or server name(s). The
+Postfix SMTP client was configured to require a verified name,
+otherwise the verification status would have been just "Trusted".
+</p>
+
+<p> <b> Postfix SMTP client:</b> The "Verified" status may also
+mean that the Postfix SMTP client successfully matched the expected
+fingerprint against the remote SMTP server public key or certificate.
+The expected fingerprint may come from smtp_tls_policy_maps or from
+TLSA (secure) DNS records. The Postfix SMTP client ignores the CA
+signature. </p>
<p> <b> Postfix SMTP server:</b> The status is never "Verified",
-as the Postfix SMTP server never verifies the remote SMTP client
-name against the names in the certificate. </p>
+because the Postfix SMTP server never verifies the remote SMTP
+client name against the names in the client certificate, and because
+the Postfix SMTP does not expect a specific fingerprint in the
+client public key or certificate. </p>
</dd>
<li> <p> <a href="#configure">Configuring LMDB settings</a>. </p>
-<li> <p> <a href="#supported"> Supported minimum LMDB patchlevel</a>. </p>
+<li> <p> <a href="#locking">Using LMDB maps with non-Postfix programs</a>. </p>
+
+<li> <p> <a href="#supported"> Required minimum LMDB patchlevel</a>. </p>
+
+<li> <p> <a href="#credits"> Credits</a>. </p>
</ul>
<ul>
<li> <p> lmdb_map_size (default: 16777216). This setting specifies
-the initial LMDB database size limit in bytes. Each time
-a database becomes full, its size limit is doubled. The maximum
-size is the largest signed integer value of "long". </p>
+the initial LMDB database size limit in bytes. Each time a database
+becomes "full", its size limit is doubled. The maximum size is the
+largest signed integer value of "long". </p>
</ul>
-<h2> <a name="supported"> Supported minimum LMDB patchlevel </a> </h2>
+<h2> <a name="locking">Using LMDB maps with non-Postfix programs</a> </h2>
+
+<p> Programs that use LMDB's built-in locking protocol will corrupt
+a Postfix LMDB database or will read garbage. </p>
+
+<p> Postfix does not use LMDB's built-in locking protocol, because
+that would require world-writable lockfiles, and would violate
+Postfix security policy. Instead, Postfix uses external locks based
+on fcntl(2) to prevent writers from corrupting the database, and
+to prevent readers from receiving garbage. </p>
+
+<p> See lmdb_table(5) for a detailed description of the locking
+protocol that all programs must use when they access a Postfix LMDB
+database. </p>
-<p> Currently, Postfix supports LMDB 0.9.11 or later. The supported
+<h2> <a name="supported"> Required minimum LMDB patchlevel </a> </h2>
+
+<p> Currently, Postfix requires LMDB 0.9.11 or later. The required
minimum LMDB patchlevel has evolved over time, as the result of
-deployment experience with Postfix. </p>
+Postfix deployment experience: </p>
<ul>
</p>
<li> <p> LMDB 0.9.10 closes an information leak where LMDB was
-writing up to 4kbyte-chunks of uninitialized heap memory to the
-database, persisting information that was not meant to be persisted,
-or sharing information that was not meant to be shared. </p>
-
-<li> <p> LMDB 0.9.8 allows Postfix to use external (fcntl()-based)
-locks, instead of having to use world-writable LMDB lock files.
+writing up to 4-kbyte chunks of uninitialized heap memory to the
+database. This would persist information that was not meant to be
+persisted, or share information that was not meant to be shared.
</p>
-<li> <p> LMDB 0.9.8 allows an application to recover from a "database
-full" error without having to close the database, by adding support
-to update the database size limit on-the-fly; and it adds support
-for an application to adopt someone elses change to the database
-size limit, without having to close the database. </p>
+<li> <p> LMDB 0.9.9 allows Postfix to use external (fcntl()-based)
+locks, instead of having to use world-writable LMDB lock files,
+violating the Postfix security model in multiple ways. </p>
-<li> <p> LMDB 0.9.7 allows the postmap and postalias commands to
-use a bulk-mode transaction larger than the amount of physical
+<li> <p> LMDB 0.9.8 allows Postfix to recover from a "database full"
+error without having to close the database. This version adds support
+to update the database size limit on-the-fly. This is necessary
+because Postfix database sizes vary with mail server load. </p>
+
+<li> <p> LMDB 0.9.7 allows the postmap(1) and postalias(1) commands
+to use a bulk-mode transaction larger than the amount of physical
memory. This is necessary because LMDB supports databases larger
than physical memory. </p>
</ul>
+<h2> <a name="credits"> Credits</a> </h2>
+
+<ul>
+
+<li> <p> Howard Chu contributed the initial Postfix dict_lmdb driver.
+</p>
+
+<li> <p> Wietse Venema wrote an abstraction layer (slmdb) that
+behaves more like Berkeley DB, NDBM, etc. This layer automatically
+retries an LMDB request when a database needs to be resized, or
+after a database was resized by a different process. </p>
+
+<li> <p> Howard and Wietse went through many iterations with changes
+to both LMDB and Postfix, with input from Viktor Dukhovni. </p>
+
+</ul>
+
<!--
<h2><a name="limitations">Unexpected failure modes of Postfix LMDB
number of "deep protocol" tests. These tests use an SMTP protocol
engine that is built into the postscreen(8) server. </p>
-<p> Important note: deep protocol tests are disabled by default.
+<p> Important note: these protocol tests are disabled by default.
They are more intrusive than the pregreet and DNSBL tests, and they
have limitations as discussed next. </p>
<ul>
-<li> <p> When a good client passes the <a href="#after_220">deep
-protocol tests</a>, postscreen(8) adds the client to the temporary
-whitelist but it cannot hand off the "live" connection to a Postfix
-SMTP server process in the middle of the session. Instead, postscreen(8)
-defers mail delivery attempts with a 4XX status, logs the
-helo/sender/recipient information, and waits for the client to
-disconnect. </p>
+<li> <p> The main limitation of "after 220 greeting" tests is that
+a new client must disconnect after passing these tests (reason:
+postscreen is not a proxy). Then the client must reconnect from
+the same IP address before it can deliver mail. The following
+measures may help to avoid email delays: </p>
-<p> The next time the client connects it will be allowed to talk
-to a Postfix SMTP server process to deliver its mail. To minimize the
-impact of this limitation, postscreen(8) gives deep protocol tests
-a relatively long expiration time. </p>
+<ul>
-<li> <p> postscreen(8)'s built-in SMTP engine does not implement
-the AUTH, XCLIENT, and XFORWARD features. AUTH support may be added
-in a future version. In the mean time, if you need to make these
-services available on port 25, then do not enable the tests after
-the 220 server greeting. </p>
+<li> <p> Allow "good" clients to skip tests with the
+postscreen_dnsbl_whitelist_threshold feature (Postfix 2.11 and
+later). This is especially effective for sites such as Google that
+never retry immediately from the same IP address. </p>
+
+<li> <p> Small sites: Configure postscreen(8) to listen on multiple
+IP addresses, published in DNS as different IP addresses for the
+same MX hostname or for different MX hostnames. This avoids mail
+delivery delays with clients that reconnect immediately from the
+same IP address. </p>
+
+<li> <p> Large sites: Share the postscreen(8) cache between different
+Postfix MTAs with a large-enough memcache_table(5). Again, this
+avoids mail delivery delays with clients that reconnect immediately
+from the same IP address. </p>
</ul>
-<p> End-user clients should connect directly to the submission
+<li> <p> postscreen(8)'s built-in SMTP engine does not implement the
+AUTH, XCLIENT, and XFORWARD features. If you need to make these
+services available on port 25, then do not enable the tests after
+the 220 server greeting. </p>
+
+<li> <p> End-user clients should connect directly to the submission
service, so that they never have to deal with postscreen(8)'s tests.
</p>
+</ul>
+
+<p> The following "after 220 greeting" tests are available: </p>
+
<ul>
<li> <a href="#pipelining">Command pipelining test</a>
# DESCRIPTION
# The Postfix LMDB adapter provides access to a persistent,
# memory-mapped, key-value store. The database size is limited
-# only by the size of the memory address space (typically 32
-# or 64 bits) and by the available file system space.
+# only by the size of the memory address space (typically 31
+# or 47 bits on 32-bit or 64-bit CPUs, respectively) and by
+# the available file system space.
# REQUESTS
# .ad
# .fi
# When a transaction fails due to a full database, Postfix
# resizes the database and retries the transaction.
#
-# Postfix access, address mapping and routing table lookups
-# may generate partial search keys such as domain names without
-# one or more subdomains, network addresses without one or
-# more least-significant octets, or email addresses without
-# the localpart, address extension or domain portion.
-# This behavior is also found with btree:, hash:, or ldap:
-# tables.
+# Postfix table lookups may generate partial search keys such
+# as domain names without one or more subdomains, network
+# addresses without one or more least-significant octets, or
+# email addresses without the localpart, address extension
+# or domain portion. This behavior is also found with, for
+# example, btree:, hash:, or ldap: tables.
#
-# Unlike other flat-file based Postfix databases, changes to
+# Unlike other flat-file Postfix databases, changes to
# an LMDB database do not trigger automatic daemon program
-# restart.
+# restart, and do not require "\fBpostfix reload\fR".
# RELIABILITY
# .ad
# .fi
-# LMDB's copy-on-write architecture achieves reliable updates,
+# LMDB's copy-on-write architecture provides safe updates,
# at the cost of using more space than some other flat-file
# databases. Read operations are memory-mapped for speed.
# Write operations are not memory-mapped to avoid silent
-# curruption due stray pointer bugs.
-#
-# The Postfix LMDB adapter implements locking with fcntl(2)
-# locks at whole-file granularity. LMDB's native locking
-# scheme would require world-writable lockfiles and would
-# therefore violate the Postfix security model.
+# curruption due to stray pointer bugs.
#
# Multiple processes can safely update an LMDB database without
# serializing requests through the proxymap(8) service. This
# makes LMDB suitable as a shared cache for verify(8) or
# postscreen(8) services.
+# SYNCHRONIZATION
+# .ad
+# .fi
+# The Postfix LMDB adapter does not use LMDB's built-in locking
+# scheme, because that would require world-writable lockfiles
+# and would violate the Postfix security model. Instead,
+# Postfix uses fcntl(2) locks with whole-file granularity.
+# Programs that use LMDB's built-in locking protocol will
+# corrupt a Postfix LMDB database or will read garbage.
+#
+# Every Postfix LMDB database read or write transaction must
+# be protected from start to end with a shared or exclusive
+# fcntl(2) lock. A writer may atomically downgrade an exclusive
+# lock to a shared lock, but it must acquire an exclusive
+# lock between updating the database and starting another
+# write transaction.
+#
+# Note that fcntl(2) locks do not protect transactions within
+# the same process against each other. If a program cannot
+# avoid making simultaneous database requests, then it must
+# protect its transactions with in-process locks, in addition
+# to the per-process fcntl(2) locks.
# CONFIGURATION PARAMETERS
# .ad
# .fi
change queue file names that don't match their message file inode
number). </p>
-<p> Note: see below for how to prepare long queue file names
-for migration to Postfix ≤ 2.8. </p>
+<p> Note: see below for how to convert long queue file names to
+Postfix ≤ 2.8. </p>
<p> Changing the parameter value to "yes" has the following effects:
</p>
nlnetlabs
resolver's
tempfails
+Chu
+LMDB's
+NDBM
+dict
+kbyte
+llmdb
+lockfiles
+slmdb
* Patches change both the patchlevel and the release date. Snapshots have no
* patchlevel; they change the release date only.
*/
-#define MAIL_RELEASE_DATE "20140109"
+#define MAIL_RELEASE_DATE "20140115"
#define MAIL_VERSION_NUMBER "2.12"
#ifdef SNAPSHOT
#include <dict_lmdb.h>
#include <warn_stat.h>
- /*
- * Supported LMDB versions.
- *
- * LMDB 0.9.9 allows the application to manage locks. This elimimates multiple
- * problems:
- *
- * - The need for a (world-)writable lockfile, which was a show-stopper for
- * multiprogrammed applications such as Postfix that consist of privileged
- * writer processes and unprivileged reader processes.
- *
- * - Hard-coded inode numbers (from ftok() output) in lockfile content that
- * could prevent automatic crash recovery, and related to that, sub-optimal
- * semaphore performance on BSD systems.
- */
-#if MDB_VERSION_FULL < MDB_VERINT(0, 9, 9)
-#error "Build with LMDB version 0.9.9 or later"
-#endif
-
/* Application-specific. */
typedef struct {
/* .IP lmdb_flags
/* Flags that control the LMDB environment. If MDB_NOLOCK is
/* specified, then each slmdb_get() or slmdb_cursor_get() call
-/* must be protected with a shared (or stronger) external lock,
+/* must be protected with a shared (or exclusive) external lock,
/* and each slmdb_put() or slmdb_del() call must be protected
-/* with an exclusive external lock. The locks may be released
-/* after the call returns.
+/* with an exclusive external lock. A lock may be released
+/* after the call returns. A writer may atomically downgrade
+/* an exclusive lock to shared, but it must obtain an exclusive
+/* lock before making another slmdb(3) write request.
+/* .sp
+/* Note: when a database is opened with MDB_NOLOCK, external
+/* locks such as fcntl() do not protect slmdb(3) requests
+/* within the same process against each other. If a program
+/* cannot avoid making simultaneous slmdb(3) requests, then
+/* it must synchronize these requests with in-process locks,
+/* in addition to the per-process fcntl(2) locks.
/* .IP slmdb_flags
/* Bit-wise OR of zero or more of the following:
/* .RS
/* Open the database and create a "bulk" transaction that is
/* committed when the database is closed. If MDB_NOLOCK is
/* specified, then the entire transaction must be protected
-/* with a persistent exclusive external lock (the lock may be
-/* atomically downgraded to a shared lock to permit concurrent
-/* read-only access). All slmdb_get(), slmdb_put() and slmdb_del()
-/* requests will be directed to the "bulk" transaction. The effect
-/* of calling slmdb_cursor_get() is undefined.
+/* with a persistent external lock. All slmdb_get(), slmdb_put()
+/* and slmdb_del() requests will be directed to the "bulk"
+/* transaction.
/* .RE
/* .IP mdb_key
/* Pointer to caller-provided lookup key storage.
/* memory mapping. According to LMDB documentation this
/* requires that there is no concurrent activity in the same
/* database by other threads in the same memory address space.
-/*
-/* When a database is opened with MDB_NOLOCK, and the external
-/* lock is based on fcntl() or the like, there is no protection
-/* against concurrent activity in the same process.
/* SEE ALSO
/* lmdb(3) API manpage (currently, non-existent).
/* AUTHOR(S)
* violation because it made information persistent that was not meant to be
* persisted, or it was sharing information that was not meant to be shared.
*
- * LMDB 0.9.8 allows the application to update the database size limit
- * on-the-fly, so that it can recover from an MDB_MAP_FULL error without
- * having to close the database. It also allows an application to "pick up"
- * a new database size limit on-the-fly, so that it can recover from an
- * MDB_MAP_RESIZED error without having to close the database. Finally, it
- * avoids the need for world-writable lockfiles, by using MDB_NOLOCK.
+ * LMDB 0.9.9 allows Postfix to use external (fcntl()-based) locks, instead of
+ * having to use world-writable LMDB lock files.
+ *
+ * LMDB 0.9.8 allows Postfix to update the database size limit on-the-fly, so
+ * that it can recover from an MDB_MAP_FULL error without having to close
+ * the database. It also allows an application to "pick up" a new database
+ * size limit on-the-fly, so that it can recover from an MDB_MAP_RESIZED
+ * error without having to close the database.
*
- * The database size limit that remains is imposed by the hardware address
- * space. The implementation is supposed to handle databases larger than
- * physical memory. However, this is not necessarily guaranteed for (bulk)
+ * The database size limit that remains is imposed by the hardware memory
+ * address space (31 or 47 bits, typically) or file system. The LMDB
+ * implementation is supposed to handle databases larger than physical
+ * memory. However, this is not necessarily guaranteed for (bulk)
* transactions larger than physical memory.
*/
#if MDB_VERSION_FULL < MDB_VERINT(0, 9, 11)
} while (0)
/*
- * With MDB_NOLOCK, the iterator must close the cursor's read transaction
- * before returning the (key, value) to the caller. See ITS#7774 and
- * followups.
+ * With MDB_NOLOCK, the application uses an external lock for inter-process
+ * synchronization. Because the caller may release the external lock after
+ * an SLMDB API call, each SLMDB API function must use a short-lived
+ * transaction unless the transaction is a bulk-mode transaction.
*/
/* slmdb_cursor_close - close cursor and its read transaction */
MDB_envinfo info;
/*
- * This may be needed in non-MDB_NOLOCK mode. Recovery is rare enough that
- * we don't care about a few wasted cycles.
+ * This may be needed in non-MDB_NOLOCK mode. Recovery is rare enough
+ * that we don't care about a few wasted cycles.
*/
if (slmdb->cursor != 0)
slmdb_cursor_close(slmdb);
/*
* If a bulk-transaction error is recoverable, build a new bulk
* transaction from scratch, by making a long jump back into the caller
- * at some pre-arranged point.
+ * at some pre-arranged point. In MDB_NOLOCK mode, there is no need to
+ * upgrade the lock to "exclusive", because the failed write transaction
+ * has no side effects.
*/
if (slmdb->txn != 0 && status == 0 && slmdb->longjmp_fn != 0
&& (slmdb->bulk_retry_count += 1) <= slmdb->bulk_retry_limit) {
*/
if (HAVE_SLMDB_SAVED_KEY(slmdb) && op != MDB_FIRST)
status = mdb_cursor_get(slmdb->cursor, &slmdb->saved_key,
- (MDB_val *) 0, MDB_SET);
+ (MDB_val *) 0, MDB_SET);
}
/*
projects / thirdparty / postfix.git / commitdiff
