+Introduction
+============
+
This is a very first implementation of Postfix content filtering.
A Postfix content filter receives unfiltered mail from Postfix and
-either bounces the mail or re-injects filtered mail back into Postfix.
+does one of the following:
+
+- re-injects the mail back into Postfix, perhaps after changing content
+- rejects the mail (by sending a suitable status code back to
+ Postfix) so that it is returned to sender.
+- sends the mail somewhere else
+
+This document describes two approaches to content filtering: simple
+and advanced. Both filter all the mail by default.
-This document describes two approaches to content filtering.
+At the end are examples that show how to filter only mail from
+users, about using different filters for different domains that
+you provide MX service for, and about selective filtering on the
+basis of message envelope and/or header/body patterns.
Simple content filtering example
================================
The first example is simple to set up. It uses a shell script that
receives unfiltered mail from the Postfix pipe delivery agent, and
that feeds filtered mail back into the Postfix sendmail command.
+
Only mail arriving via SMTP will be content filtered.
..................................
| |
+-Postfix sendmail<----filter script<--+
-The /some/where/filter program can be a simple shell script like this:
+Mail is filtered by a /some/where/filter program. This can be a
+simple shell script like this:
#!/bin/sh
exit $?
The idea is to first capture the message to file and then run the
-content through a third-party content filter program. If the
-mail cannot be captured to file, mail delivery is deferred by
-terminating with exit status 75 (EX_TEMPFAIL). If the content
-filter program finds a problem, the mail is bounced by terminating
-the shell script with exit status 69 (EX_UNAVAILABLE). If the
-content is OK, it is given as input to Postfix sendmail, and the
-exit status of the filter command is whatever exit status Postfix
-sendmail produces.
-
-I suggest that you play with this script for a while until you are
-satisfied with the results. Run it with a real message (headers+body)
-as input:
+content through a third-party content filter program.
+
+- If the mail cannot be captured to file, mail delivery is deferred
+ by terminating with exit status 75 (EX_TEMPFAIL). Postfix will
+ try again after some delay.
+
+- If the content filter program finds a problem, the mail is bounced
+ by terminating with exit status 69 (EX_UNAVAILABLE). Postfix
+ will return the message to the sender as undeliverable.
+
+- If the content is OK, it is given as input to the Postfix sendmail
+ command, and the exit status of the filter command is whatever
+ exit status the Postfix sendmail command produces. Postfix will
+ deliver the message as usual.
+
+I suggest that you run this script by hand until you are satisfied
+with the results. Run it with a real message (headers+body) as
+input:
% /some/where/filter -f sender recipient... <message-file
flags=Rq user=filter argv=/somewhere/filter -f ${sender} -- ${recipient}
To turn on content filtering for mail arriving via SMTP only, append
-"-o content_filter=filter:" to the master.cf entry that defines
+"-o content_filter=filter:dummy" to the master.cf entry that defines
the Postfix SMTP server:
/etc/postfix/master.cf:
- smtp inet ...stuff... smtpd
- -o content_filter=filter:
+ smtp inet ...stuff... smtpd
+ -o content_filter=filter:dummy
-Note the ":" at the end!! The content_filter configuration parameter
-accepts the same syntax as the right-hand side in a Postfix transport
-table. Execute "postfix reload" to complete the change.
+The content_filter configuration parameter accepts the same syntax
+as the right-hand side in a Postfix transport table. Execute
+"postfix reload" to complete the change.
To turn off content filtering, edit the master.cf file, remove the
-"-o content_filter=filter:" text from the entry that defines the
-Postfix SMTP server, and execute another "postfix reload".
+"-o content_filter=filter:dummy" text from the entry that defines
+the Postfix SMTP server, and execute another "postfix reload".
With the shell script as shown above you will lose a factor of four
in Postfix performance for transit mail that arrives and leaves
=================================
The problem with content filters like the one above is that they
-are not very robust, because the software does not talk a well-defined
-protocol with Postfix. If the filter shell script aborts because
-the shell runs into some memory allocation problem, the script will
-not produce a nice exit status as per /usr/include/sysexits.h and
-mail will probably bounce. The same lack of robustness is possible
+are not very robust. The reason is that the software does not talk
+a well-defined protocol with Postfix. If the filter shell script
+aborts because the shell runs into some memory allocation problem,
+the script will not produce a nice exit status as defined in the
+file /usr/include/sysexits.h. Instead of going to the deferred
+queue, mail will bounce. The same lack of robustness can happen
when the content filtering software itself runs into a resource
problem.
Advanced content filtering example
===================================
-The second example is considerably more complex, but can give much
-better performance, and is less likely to bounce mail when the
-machine runs into a resource problem. This approach uses content
-filtering software that can receive and deliver mail via SMTP.
+The second example is more complex, but can give much better
+performance, and is less likely to bounce mail when the machine
+runs into a resource problem. This approach uses content filtering
+software that can receive and deliver mail via SMTP.
+
+Some Anti-virus software is built to receive and deliver mail via
+SMTP and is ready to use as an advanced Postfix content filter.
+For non-SMTP capable content filtering software, Bennett Todd's
+SMTP proxy implements a nice PERL/SMTP content filtering framework.
+See: http://bent.latency.net/smtpprox/
+
+The example given here filters all mail, including mail that arrives
+via SMTP and mail that is locally submitted via the Postfix sendmail
+command.
You can expect to lose about a factor of two in Postfix performance
for transit mail that arrives and leaves via SMTP, provided that
new parameter:
/etc/postfix/main.cf:
- content_filter = scan:localhost:10025
+ content_filter = scan:localhost:10025
This causes Postfix to add one extra content filtering record to
each incoming mail message, with content scan:localhost:10025.
/etc/postfix/master.cf:
scan unix - - n - 10 smtp
- -o disable_dns_lookups=yes
-
-Turning off DNS lookups at this point can make a great difference
-in content filtering performance. It also isolates the content
-filtering process from temporary outages in DNS service.
-Instead of 10, use whatever limit is feasible for your machine.
-Content inspection software can gobble up a lot of system resources,
-so you don't want to have too much of it running at the same time.
+Instead of a limit of 10 concurrent processes, use whatever process
+limit is feasible for your machine. Content inspection software
+can gobble up a lot of system resources, so you don't want to have
+too much of it running at the same time.
The content filter can be set up with the Postfix spawn service,
which is the Postfix equivalent of inetd. For example, to instantiate
up to 10 content filtering processes on demand:
/etc/postfix/master.cf:
- localhost:10025 inet n n n - 10 spawn
- user=filter argv=/some/where/filter localhost 10026
+ localhost:10025 inet n n n - 10 spawn
+ user=filter argv=/some/where/filter localhost 10026
"filter" is a dedicated local user account. The user will never
log in, and can be given a "*" password and non-existent shell and
program.
Note: the localhost port 10025 SMTP server filter should announce
-itself as "220 localhost...", to silence warnings in the log.
+itself as "220 localhost...". Postfix aborts delivery when it
+connects to an SMTP server that uses the same hostname, because
+that normally means you have a mail delivery loop problem.
-The /some/where/filter command is most likely a PERL script. PERL
-has modules that make talking SMTP easy. The command-line specifies
-that mail should be sent back into Postfix via localhost port 10026.
-
-For now, it is left up to the Postfix users to come up with a
-PERL/SMTP framework for Postfix content filtering. If done well,
-it can be used with other mailers too, which is a nice spin-off.
+The example here assumes that the /some/where/filter command is a
+PERL script. PERL has modules that make talking SMTP easy. The
+command-line specifies that mail should be sent back into Postfix
+via localhost port 10026.
The simplest content filter just copies SMTP commands and data
between its inputs and outputs. If it has a problem, all it has to
/etc/postfix/master.cf:
localhost:10026 inet n - n - 10 smtpd
- -o content_filter=
- -o local_recipient_maps=
- -o myhostname=localhost.domain.tld
- -o smtpd_helo_restrictions=
- -o smtpd_client_restrictions=
- -o smtpd_sender_restrictions=
- -o smtpd_recipient_restrictions=permit_mynetworks,reject
- -o mynetworks=127.0.0.0/8
-
-This is just another SMTP server. The "-o content_filter=" requests
-no content filtering for incoming mail. The server has the same
-process limit as the "filter" master.cf entry.
-
-The "-o local_recipient_maps=" is a safety in case you have specified
-local_recipient_maps in the main.cf file. That could interfere with
-content filtering.
+ -o content_filter=
+ -o local_recipient_maps=
+ -o relay_recipient_maps=
+ -o myhostname=localhost.domain.tld
+ -o smtpd_helo_restrictions=
+ -o smtpd_client_restrictions=
+ -o smtpd_sender_restrictions=
+ -o smtpd_recipient_restrictions=permit_mynetworks,reject
+ -o mynetworks=127.0.0.0/8
+
+Warning for Postfix version 2 users: in this SMTP server after the
+content filter, do not override main.cf settings for virtual_alias_maps
+or virtual_alias_domains. That would cause mail to be rejected with
+"User unknown".
+
+This SMTP server has the same process limit as the "filter" master.cf
+entry.
+
+The "-o content_filter=" requests no content filtering for incoming
+mail.
+
+The "-o local_recipient_maps=" and "-o relay_recipient_maps=" avoid
+unnecessary table lookups.
The "-o myhostname=localhost.domain.tld" avoids a possible problem
-if the content filter is picky about the hostname that Postfix
-sends in SMTP server replies.
+if your content filter is based on a proxy that simply relays SMTP
+commands.
The "-o smtpd_xxx_restrictions" and "-o mynetworks=127.0.0.0/8"
turn off UCE controls that would only waste time here.
Many refinements are possible, such as running a specially-configured
smtp delivery agent for feeding mail into the content filter, and
-turning off address rewriting before or after content filtering.
+turning off address rewriting before content filtering.
As the example below shows, things quickly become very complex,
because a lot of main.cf like information gets listed in the
If you need to squeeze out more performance, it is probably simpler
to run multiple Postfix instances, one before and one after the
content filter. That way, each instance can have simple main.cf
-and master.cf files, and the system will be easier to understand.
+and master.cf files, each instance can have its own mail queue,
+and the system will be easier to understand.
As before, we will set up a content filtering program that receives
SMTP mail via localhost port 10025, and that submits SMTP mail back
#
smtp inet n - n - - smtpd
#
+# ------------------------------------------------------------------
+#
# This is the cleanup daemon that handles messages in front of
-# the content filter, it does header_checks and body_checks (if
-# any), but does not do any address rewriting.
+# the content filter. It does header_checks and body_checks (if
+# any), but does no virtual alias or canonical address mapping.
#
-# The address rewriting happens in the second cleanup phase after
-# the content filter. This gives the content_filter access to
-# *largely* unmodified addresses for maximum flexibility.
+# Virtual alias or canonical address mapping happens in the second
+# cleanup phase after the content filter. This gives the content_filter
+# access to *largely* unmodified addresses for maximum flexibility.
#
-# The trivial-rewrite daemon handles the logic of append_myorigin
-# and append_dot_mydomain, turning these off requires two
-# trivial-rewrite services, by which point (if you are not
-# already) you are much better off with two complete Postfix
-# instances one in front of and one behind the content filter.
+# Turning off append_myorigin/append_dot_mydomain address rewriting
+# before the content filter would require two instances of the
+# trivial-rewrite daemon. If you want to go to this trouble then
+# you're clearly better off with two complete Postfix instances: one
+# in front of and one behind the content filter.
#
# Note that some sites may specifically want to do the opposite:
# perform rewrites in front of the content_filter which would
-# then see only cleaned up addresses, in this case the parameter
-# settings below should be moved to the second "cleanup"
-# instance.
+# then see only cleaned up addresses. In that case the "-o" parameter
+# settings below should be moved to the second "cleanup" instance.
#
cleanup unix n - n - 0 cleanup
- -o canonical_maps=
- -o sender_canonical_maps=
- -o recipient_canonical_maps=
- -o masquerade_domains=
- -o virtual_maps=
+ -o canonical_maps=
+ -o sender_canonical_maps=
+ -o recipient_canonical_maps=
+ -o masquerade_domains=
+ -o virtual_alias_maps=
+#
+# ------------------------------------------------------------------
#
# This is the delivery agent that injects mail into the content
-# filter. It is tuned for low latency and low concurrency, most
-# content filters burn CPU and high concurrency causes thrashing.
-# The process limit of 10 reenforces the effect of
-# $default_destination_concurrency_limit, even without an
-# explicit process limit, the concurrency is bounded because all
-# messages heading into the content filter have the same
-# destination. The "disable_dns_lookups" setting prevents the
-# delivery agent from consuming precious "bandwidth" in the
-# narrow deliver channel waiting for slow DNS responses. It also
-# ensures that the original envelope recipient is seen by the
-# content filter.
+# filter. It is tuned for low concurrency, because most content
+# filters burn CPU and use lots of memory. The process limit of 10
+# re-enforces the effect of $default_destination_concurrency_limit.
+# Even without an explicit process limit, the concurrency is bounded
+# because all messages heading into the content filter have the same
+# destination.
#
scan unix - - n - 10 smtp
- -o disable_dns_lookups=yes
+#
+# ------------------------------------------------------------------
#
# This is the SMTP listener that receives filtered messages from
# the content filter. It *MUST* clear the content_filter
# parameter to avoid loops, and use a different hostname to avoid
# triggering the Postfix SMTP loop detection code.
#
-#
-# Since all recipients have been validated by the first "smtpd",
-# clear local_recipient_maps, virtual_maps and
-# virtual_mailbox_maps.
-#
# This "smtpd" uses a separate cleanup that does no header or
# body checks, but does do the various address rewrites disabled
# in the first cleanup.
-o content_filter=
-o myhostname=localhost.domain.tld
-o local_recipient_maps=
- -o virtual_maps=
- -o virtual_mailbox_maps=
+ -o relay_recipient_maps=
-o cleanup_service_name=cleanup2
-o mynetworks=127.0.0.0/8
-o mynetworks_style=host
-o smtpd_sender_restrictions=
-o smtpd_recipient_restrictions=permit_mynetworks,reject
#
-# This is the second cleanup daemon. No header or body checks.
-# If preferable, enable rewrites in the first cleanup daemon, and
-# disable them here.
+# Do not override main.cf settings here for virtual_alias_maps or
+# virtual_mailbox_maps. This causes mail to be rejected with "User
+# unknown in virtual (alias|mailbox) recipient table".
+#
+# ------------------------------------------------------------------
+#
+# This is the second cleanup daemon. No header or body checks,
+# because those have already been taken care of by the cleanup instance
+# before the content filter. The second cleanup instance does all the
+# virtual alias and canonical address mapping that was disabled in
+# the first cleanup instance.
+#
+# If it is preferable to do the virtual alias and canonical address
+# mapping before the content filter, delete the "-o" lines that
+# disable canonical and virtual mappings in the above cleanup daemon
+# instance and insert them here.
#
cleanup2 unix n - n - 0 cleanup
-o header_checks=
-o nested_header_checks=
-o body_checks=
#
+# ------------------------------------------------------------------
+#
# The normal "smtp" delivery agent for contrast with "scan".
-# Definitely do not set "disable_dns_lookups = yes" here if you
-# send mail to the Internet.
#
smtp unix - - n - - smtp
-This causes Postfix to add one extra content filtering record to
-each incoming mail message, with content scan:localhost:10025.
+The above example causes Postfix to add one content filtering record
+to each incoming mail message, with content scan:localhost:10025.
You can use the same syntax as in the right-hand side of a Postfix
transport table. The content filtering records are added by the
smtpd and pickup servers.
See the previous example for setting up the content filter with
the Postfix spawn service; you can of course use any server that
can be run stand-alone outside the Postfix environment.
+
+Filtering mail from outside users only
+======================================
+
+The easiest approach is to configure ONE Postfix instance with TWO
+SMTP server addresses in master.cf:
+
+- One SMTP server address for inside users only that never invokes
+ content filtering.
+
+- One SMTP server address for outside users that always invokes
+ content filtering.
+
+/etc/postfix.master.cf:
+ # SMTP service for internal users only, no content filtering.
+ 1.2.3.4:smtp inet n - n - - smtpd
+ -o smtpd_client_restrictions=permit_mynetworks,reject
+ 127.0.0.1:smtp inet n - n - - smtpd
+ -o smtpd_client_restrictions=permit_mynetworks,reject
+
+ # SMTP service for external users, with content filtering.
+ 1.2.3.5:smtp inet n - n - - smtpd
+ -o content_filter=foo:bar
+
+Different content filters for different MX domains
+==================================================
+
+This is a variant on the previous example. You configure ONE
+Postfix instance with multiple SMTP server addresses. Each
+SMTP server invokes a different content filter.
+
+/etc/postfix.master.cf:
+ # MX server for destinations that use the foo:bar content filter.
+ 1.2.3.5:smtp inet n - n - - smtpd
+ -o content_filter=foo:bar
+ -o relay_domains=/etc/postfix/foo-bar-domains
+ -o smtpd_recipient_restrictions=reject_unauth_destination
+
+ # MX server for destinations that use the bar:baz content filter.
+ 1.2.3.6:smtp inet n - n - - smtpd
+ -o content_filter=bar:baz
+ -o relay_domains=/etc/postfix/bar-baz-domains
+ -o smtpd_recipient_restrictions=reject_unauth_destination
+
+ # SMTP servers for internal users only.
+ 1.2.3.4:smtp inet n - n - - smtpd
+ -o smtpd_recipient_restrictions=permit_mynetworks,reject
+ 127.0.0.1:smtp inet n - n - - smtpd
+ -o smtpd_recipient_restrictions=permit_mynetworks,reject
+
+Getting really nasty
+====================
+
+The above filtering configurations are static. Mail that follows
+a given path is either always filtered or it is never filtered. As
+of Postfix 2.0 you can also turn on content filtering on the fly.
+The Postfix UCE features allow you to specify a filtering action
+on the fly:
+
+ FILTER foo:bar
+
+You can do this in smtpd access maps as well as the cleanup server's
+header/body_checks. This feature must be used with great care:
+you must disable all the UCE features in the after-filter smtpd
+and cleanup daemons or else you will have a content filtering loop.
+
+Limitations:
+
+- There can be only one content filter action per message.
+
+- FILTER actions from smtpd access maps and header/body_checks take
+ precedence over filters specified with the main.cf content_filter
+ parameter.
+
+- Only the last FILTER action from smtpd access maps or in
+ header/body_checks takes effect.
+
+- The same content filter is applied to all the recipients of a
+ given message.
projects / thirdparty / postfix.git / commitdiff
