-TDICT_THASH
-TDICT_UNION
-TDICT_UNIX
+-TDICT_UTF8_BACKUP
-TDNS_FIXED
-TDNS_REPLY
-TDNS_RR
As a first step, with "smtputf8_enable = yes" all features
based on Postfix matchlists enable UTF-8 syntax checks and
- casefolding support for table queries and results. That
- includes mynetworks, mydestination, relay_domains,
- virtual_alias_domains, and virtual_mailbox_domains.
-
- The next step is to turn on UTF-8 syntax checks and casefolding
- support for access maps, address rewriting and routing.
+ UTF-8 casefolding for table patterns, but NOT YET for string
+ patterns. The list of features includes authorized_flush_users,
+ authorized_mailq_users, authorized_submit_users, debug_peer_list,
+ fast_flush_domains, mydestination, permit_mx_backup_networks,
+ qmqpd_authorized_clients, smtp_connection_cache_destinations,
+ smtpd_authorized_verp_clients, smtpd_authorized_xclient_hosts,
+ smtpd_authorized_xforward_hosts,
+ smtpd_client_event_limit_exceptions,
+ smtpd_log_access_permit_actions, smtpd_sasl_exceptions_networks,
+ the "domains" feature in ldap_table(5), memcache_table(5)
+ mysql_table(5), pgsql_table(5) and sqlite_table(5),
+ virtual_alias_domains, virtual_mailbox_domains.
+
+20140111
+
+ Cleanup: simplified the interposition layer that adds UTF-8
+ support to Postfix lookup tables. Files: util/dict_utf8.c.
+
+ With "smtputf8_enable = yes", Enable UTF-8 syntax checks
+ and UTF-8 casefolding for SMTP server access maps, alias_maps,
+ canonical_maps, fallback_transport_maps,
+ lmtp_tls_session_cache_database, local_recipient_maps,
+ mailbox_command_maps, mailbox_transport_maps, rbl_reply_maps,
+ recipient_bcc_maps, recipient_canonical_maps, relay_recipient_maps,
+ relocated_maps, sender_bcc_maps, sender_canonical_maps,
+ sender_dependent_relayhost_maps, sender_dependent_transport_maps,
+ smtp_generic_maps, smtp_sasl_auth_cache_name,
+ smtp_sasl_password_maps, smtp_tls_per_site, smtp_tls_policy_maps,
+ smtp_tls_session_cache_database, smtpd_sender_login_maps,
+ smtpd_tls_session_cache_database, transport_maps,
+ virtual_alias_maps, virtual_gid_maps, virtual_mailbox_maps,
+ virtual_uid_maps.
Postfix releases:
* UTF-8 is permitted in the myorigin parameter value. However, the myhostname
- and mydomain parameters must specify ASCII-only domain names. This
- limitation may be removed later.
+ and mydomain parameters must currently specify ASCII-only domain names.
+ This limitation may be removed later.
+
+ * UTF-8 is the only form of non-ASCII text that Postfix supports in access
+ tables, address rewriting tables, and other tables that are indexed with an
+ email address, hostname, or domain name.
+
+ * The header_checks-like and body_checks-like features are not UTF-8 enabled,
+ and therefore they do not enforce UTF-8 syntax rules on inputs and outputs.
+ The reason is that non-ASCII text may be sent in encodings other than UTF-
+ 8, and that real email sometimes contains malformed headers. Instead of
+ skipping non-UTF-8 content, Postfix should be able to filter it. You may
+ try to enable UTF-8 processing by starting a PCRE pattern with the sequence
+ (*UTF8), but this is will result in "message not accepted, try again later"
+ errors when the PCRE pattern matcher encounters non-UTF-8 input. Other
+ features that are not UTF-8 enabled are smtpd_command_filter,
+ smtp_reply_filter, the *_delivery_status_filter features, and the
+ *_dns_reply_filter features (the latter because DNS is by definition an
+ ASCII protocol).
* The Postfix SMTP server announces SMTPUTF8 support in the EHLO response.
performs no automatic conversions on UTF8 strings beyond what is needed to
perform DNS lookups.
-N\bNo\bo c\bch\bha\bar\bra\bac\bct\bte\ber\brs\bse\bet\bt c\bca\ban\bno\bon\bni\bic\bca\bal\bli\biz\bza\bat\bti\bio\bon\bn f\bfo\bor\br n\bno\bon\bn-\b-A\bAS\bSC\bCI\bII\bI d\bdo\bom\bma\bai\bin\bn n\bna\bam\bme\bes\bs.\b.
+N\bNo\bo a\bau\but\bto\bom\bma\bat\bti\bic\bc c\bco\bon\bnv\bve\ber\brs\bsi\bio\bon\bns\bs b\bbe\bet\btw\bwe\bee\ben\bn A\bAS\bSC\bCI\bII\bI a\ban\bnd\bd U\bUT\bTF\bF-\b-8\b8 d\bdo\bom\bma\bai\bin\bn n\bna\bam\bme\bes\bs.\b.
Postfix currently does not translate domain names from UTF-8 into ASCII (or
ASCII into UTF-8) before looking up the domain name in mydestination,
configure both UTF-8 and ASCII forms in Postfix configuration files; and both
forms will have to be handled by logfile tools, policy daemons and Milters.
-N\bNo\bo c\bca\bas\bse\be c\bca\ban\bno\bon\bni\bic\bca\bal\bli\biz\bza\bat\bti\bio\bon\bn f\bfo\bor\br n\bno\bon\bn-\b-A\bAS\bSC\bCI\bII\bI c\bch\bha\bar\bra\bac\bct\bte\ber\brs\bs.\b.
+I\bIm\bmp\bpl\ble\bem\bme\ben\bnt\bte\bed\bd:\b: c\bca\bas\bse\be-\b-i\bin\bns\bse\ben\bns\bsi\bit\bti\biv\bve\be t\bta\bab\bbl\ble\be s\bse\bea\bar\brc\bch\bh w\bwi\bit\bth\bh n\bno\bon\bn-\b-A\bAS\bSC\bCI\bII\bI t\bte\bex\bxt\bt.\b.
-Postfix currently does not case-fold non-ASCII characters when looking up an
-"Internationalized" domain name in mydestination, relay_domains, access maps,
-etc. Some non-ASCII scripts do not distinguish between upper and lower case,
-some have different numbers of upper and lower case characters.
+Postfix will casefold UTF-8 when searching with an "Internationalized" domain
+name or email address in mydestination, relay_domains, access maps,
+transport_maps, etc., and when maintaining tables with the postmap(1) and
+postalias(1) commands.
+
+N\bNo\bo c\bca\bas\bse\be-\b-i\bin\bns\bse\ben\bns\bsi\bit\bti\biv\bve\be m\bma\bat\btc\bch\bhi\bin\bng\bg o\bof\bf n\bno\bon\bn-\b-A\bAS\bSC\bCI\bII\bI s\bst\btr\bri\bin\bng\bg p\bpa\bat\btt\bte\ber\brn\bns\bs i\bin\bn m\bma\bat\btc\bch\bhl\bli\bis\bst\bts\bs.\b.
+
+Postfix currently does not yet implement case-insensitive string comparison for
+non-ASCII string patterns in list features such as mydestination,
+relay_domains, etc. For now, use "inline:{string}" instead of "string". This
+limitation will be removed before the stable release.
C\bCo\bom\bmp\bpa\bat\bti\bib\bbi\bil\bli\bit\bty\by w\bwi\bit\bth\bh p\bpr\bre\be-\b-S\bSM\bMT\bTP\bPU\bUT\bTF\bF8\b8 e\ben\bnv\bvi\bir\bro\bon\bnm\bme\ben\bnt\bts\bs
Things to do after the stable release:
+ Expose UTF8 flag in server_acl API. Some applications such
+ as postscreen don't need UTF8 support.
+
+ Expose UTF8 flag in match_list API. Some applications
+ such as address lists don't need UTF8 support.
+
+ Implement UTF8 casefolding in match_list for non-table
+ patterns.
+
+ UTF8 DNS[BW]L domain name.
+
+ Consolidate maps flags in mail_params.h instead of having
+ multiple copies scattered across programs.
+
Try to allow UTF-8 myhostname/mydomain, at least in bounce
template expansion.
<ul>
<li> <p> UTF-8 is permitted in the <a href="postconf.5.html#myorigin">myorigin</a> parameter value. However,
-the <a href="postconf.5.html#myhostname">myhostname</a> and <a href="postconf.5.html#mydomain">mydomain</a> parameters must specify ASCII-only
-domain names. This limitation may be removed later. </p>
+the <a href="postconf.5.html#myhostname">myhostname</a> and <a href="postconf.5.html#mydomain">mydomain</a> parameters must currently specify
+ASCII-only domain names. This limitation may be removed later. </p>
+
+<li> <p> UTF-8 is the only form of non-ASCII text that Postfix
+supports in access tables, address rewriting tables, and other
+tables that are indexed with an email address, hostname, or domain
+name. </p>
+
+<li> <p> The <a href="postconf.5.html#header_checks">header_checks</a>-like and <a href="postconf.5.html#body_checks">body_checks</a>-like features are
+not UTF-8 enabled, and therefore they do not enforce UTF-8 syntax
+rules on inputs and outputs. The reason is that non-ASCII text may
+be sent in encodings other than UTF-8, and that real email sometimes
+contains malformed headers. Instead of skipping non-UTF-8 content,
+Postfix should be able to filter it. You may try to enable UTF-8
+processing by starting a PCRE pattern with the sequence (*UTF8),
+but this is will result in "message not accepted, try again later"
+errors when the PCRE pattern matcher encounters non-UTF-8 input.
+Other features that are not UTF-8 enabled are <a href="postconf.5.html#smtpd_command_filter">smtpd_command_filter</a>,
+<a href="postconf.5.html#smtp_reply_filter">smtp_reply_filter</a>, the *_delivery_status_filter features, and the
+*_dns_reply_filter features (the latter because DNS is by definition
+an ASCII protocol). </p>
<li> <p> The Postfix SMTP server announces SMTPUTF8 support in the
EHLO response. </p>
SMTPUTF8 implementation performs no automatic conversions on UTF8
strings beyond what is needed to perform DNS lookups. </p>
-<h3> No characterset canonicalization for non-ASCII domain names.
-</h3>
+<h3> No automatic conversions between ASCII and UTF-8 domain names. </h3>
<p> Postfix currently does not translate domain names from UTF-8
into ASCII (or ASCII into UTF-8) before looking up the domain name
forms in Postfix configuration files; and both forms will have to
be handled by logfile tools, policy daemons and Milters. </p>
-<h3> No case canonicalization for non-ASCII characters. </h3>
+<h3> Implemented: case-insensitive table search with non-ASCII text. </h3>
-<p> Postfix currently does not case-fold non-ASCII characters when
-looking up an "Internationalized" domain name in <a href="postconf.5.html#mydestination">mydestination</a>,
-<a href="postconf.5.html#relay_domains">relay_domains</a>, access maps, etc. Some non-ASCII scripts do not
-distinguish between upper and lower case, some have different numbers
-of upper and lower case characters. </p>
+<p> Postfix will casefold UTF-8 when searching with an "Internationalized"
+domain name or email address in <a href="postconf.5.html#mydestination">mydestination</a>, <a href="postconf.5.html#relay_domains">relay_domains</a>, access
+maps, <a href="postconf.5.html#transport_maps">transport_maps</a>, etc., and when maintaining tables with the
+<a href="postmap.1.html">postmap(1)</a> and <a href="postalias.1.html">postalias(1)</a> commands. </p>
+
+<h3> No case-insensitive matching of non-ASCII string patterns in matchlists. </h3>
+
+<p> Postfix currently does not yet implement case-insensitive string
+comparison for non-ASCII string patterns in list features such as
+<a href="postconf.5.html#mydestination">mydestination</a>, <a href="postconf.5.html#relay_domains">relay_domains</a>, etc. For now, use "<a href="DATABASE_README.html#types">inline</a>:{string}"
+instead of "string". This limitation will be removed before the
+stable release. </p>
<h2> <a name="compatibility">Compatibility with pre-SMTPUTF8
environments</a> </h2>
<ul>
<li> <p> UTF-8 is permitted in the myorigin parameter value. However,
-the myhostname and mydomain parameters must specify ASCII-only
-domain names. This limitation may be removed later. </p>
+the myhostname and mydomain parameters must currently specify
+ASCII-only domain names. This limitation may be removed later. </p>
+
+<li> <p> UTF-8 is the only form of non-ASCII text that Postfix
+supports in access tables, address rewriting tables, and other
+tables that are indexed with an email address, hostname, or domain
+name. </p>
+
+<li> <p> The header_checks-like and body_checks-like features are
+not UTF-8 enabled, and therefore they do not enforce UTF-8 syntax
+rules on inputs and outputs. The reason is that non-ASCII text may
+be sent in encodings other than UTF-8, and that real email sometimes
+contains malformed headers. Instead of skipping non-UTF-8 content,
+Postfix should be able to filter it. You may try to enable UTF-8
+processing by starting a PCRE pattern with the sequence (*UTF8),
+but this is will result in "message not accepted, try again later"
+errors when the PCRE pattern matcher encounters non-UTF-8 input.
+Other features that are not UTF-8 enabled are smtpd_command_filter,
+smtp_reply_filter, the *_delivery_status_filter features, and the
+*_dns_reply_filter features (the latter because DNS is by definition
+an ASCII protocol). </p>
<li> <p> The Postfix SMTP server announces SMTPUTF8 support in the
EHLO response. </p>
SMTPUTF8 implementation performs no automatic conversions on UTF8
strings beyond what is needed to perform DNS lookups. </p>
-<h3> No characterset canonicalization for non-ASCII domain names.
-</h3>
+<h3> No automatic conversions between ASCII and UTF-8 domain names. </h3>
<p> Postfix currently does not translate domain names from UTF-8
into ASCII (or ASCII into UTF-8) before looking up the domain name
forms in Postfix configuration files; and both forms will have to
be handled by logfile tools, policy daemons and Milters. </p>
-<h3> No case canonicalization for non-ASCII characters. </h3>
+<h3> Implemented: case-insensitive table search with non-ASCII text. </h3>
-<p> Postfix currently does not case-fold non-ASCII characters when
-looking up an "Internationalized" domain name in mydestination,
-relay_domains, access maps, etc. Some non-ASCII scripts do not
-distinguish between upper and lower case, some have different numbers
-of upper and lower case characters. </p>
+<p> Postfix will casefold UTF-8 when searching with an "Internationalized"
+domain name or email address in mydestination, relay_domains, access
+maps, transport_maps, etc., and when maintaining tables with the
+postmap(1) and postalias(1) commands. </p>
+
+<h3> No case-insensitive matching of non-ASCII string patterns in matchlists. </h3>
+
+<p> Postfix currently does not yet implement case-insensitive string
+comparison for non-ASCII string patterns in list features such as
+mydestination, relay_domains, etc. For now, use "inline:{string}"
+instead of "string". This limitation will be removed before the
+stable release. </p>
<h2> <a name="compatibility">Compatibility with pre-SMTPUTF8
environments</a> </h2>
if (*var_canonical_maps)
cleanup_comm_canon_maps =
maps_create(VAR_CANONICAL_MAPS, var_canonical_maps,
- DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX);
+ DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX
+ | DICT_FLAG_UTF8_REQUEST);
if (*var_send_canon_maps)
cleanup_send_canon_maps =
maps_create(VAR_SEND_CANON_MAPS, var_send_canon_maps,
- DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX);
+ DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX
+ | DICT_FLAG_UTF8_REQUEST);
if (*var_rcpt_canon_maps)
cleanup_rcpt_canon_maps =
maps_create(VAR_RCPT_CANON_MAPS, var_rcpt_canon_maps,
- DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX);
+ DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX
+ | DICT_FLAG_UTF8_REQUEST);
if (*var_virt_alias_maps)
cleanup_virt_alias_maps = maps_create(VAR_VIRT_ALIAS_MAPS,
var_virt_alias_maps,
DICT_FLAG_LOCK
- | DICT_FLAG_FOLD_FIX);
+ | DICT_FLAG_FOLD_FIX
+ | DICT_FLAG_UTF8_REQUEST);
if (*var_canon_classes)
cleanup_comm_canon_flags =
name_mask(VAR_CANON_CLASSES, canon_class_table,
if (*var_send_bcc_maps)
cleanup_send_bcc_maps =
maps_create(VAR_SEND_BCC_MAPS, var_send_bcc_maps,
- DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX);
+ DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX
+ | DICT_FLAG_UTF8_REQUEST);
if (*var_rcpt_bcc_maps)
cleanup_rcpt_bcc_maps =
maps_create(VAR_RCPT_BCC_MAPS, var_rcpt_bcc_maps,
- DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX);
+ DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX
+ | DICT_FLAG_UTF8_REQUEST);
if (*var_cleanup_milters)
cleanup_milters = milter_create(var_cleanup_milters,
var_milt_conn_time,
mkmap_open.o: ../../include/myflock.h
mkmap_open.o: ../../include/mymalloc.h
mkmap_open.o: ../../include/sigdelay.h
+mkmap_open.o: ../../include/stringops.h
mkmap_open.o: ../../include/sys_defs.h
mkmap_open.o: ../../include/vbuf.h
mkmap_open.o: ../../include/vstream.h
/*
* Don't frustrate future attempts to make Postfix UTF-8 transparent.
*/
- if (DICT_IS_ENABLE_UTF8(dict->flags) == 0
+ if ((dict->flags & DICT_FLAG_UTF8_ACTIVE) == 0
&& !valid_utf8_string(name, strlen(name))) {
if (msg_verbose)
msg_info("%s: %s: Skipping lookup of non-UTF-8 key '%s'",
/*
* Don't frustrate future attempts to make Postfix UTF-8 transparent.
*/
- if (DICT_IS_ENABLE_UTF8(dict->flags) == 0
+ if ((dict->flags & DICT_FLAG_UTF8_ACTIVE) == 0
&& !valid_utf8_string(name, strlen(name))) {
if (msg_verbose)
msg_info("%s: %s: Skipping lookup of non-UTF-8 key '%s'",
* Initialize.
*/
mail_conf_read();
- path = maps_create(argv[0], argv[1], DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX);
+ path = maps_create(argv[0], argv[1], DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX \
+ | DICT_FLAG_UTF8_REQUEST);
while (vstring_fgets_nonl(buffer, VSTREAM_IN)) {
extent = 0;
result = mail_addr_find(path, STR(buffer), &extent);
msg_verbose = 1;
if (chdir(var_queue_dir) < 0)
msg_fatal("chdir %s: %m", var_queue_dir);
- path = maps_create(argv[0], argv[1], DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX);
+ path = maps_create(argv[0], argv[1], DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX \
+ | DICT_FLAGS_UTF8_REQUEST);
while (vstring_fgets_nonl(buffer, VSTREAM_IN)) {
msg_info("=== Address extension on, extension propagation on ===");
UPDATE(var_rcpt_delim, "+");
VAR_DAEMON_OPEN_FATAL, DEF_DAEMON_OPEN_FATAL, &var_daemon_open_fatal,
0,
};
+ static const CONFIG_NBOOL_TABLE first_nbool_defaults[] = {
+ /* read and process the following before opening tables. */
+ VAR_SMTPUTF8_ENABLE, DEF_SMTPUTF8_ENABLE, &var_smtputf8_enable,
+ 0,
+ };
static const CONFIG_STR_FN_TABLE function_str_defaults[] = {
VAR_MYHOSTNAME, check_myhostname, &var_myhostname, 1, 0,
VAR_MYDOMAIN, check_mydomainname, &var_mydomain, 1, 0,
VAR_STRICT_SMTPUTF8, DEF_STRICT_SMTPUTF8, &var_strict_smtputf8,
0,
};
- static const CONFIG_NBOOL_TABLE nbool_defaults[] = {
- VAR_SMTPUTF8_ENABLE, DEF_SMTPUTF8_ENABLE, &var_smtputf8_enable,
- 0,
- };
const char *cp;
/*
if (var_daemon_open_fatal)
dict_allow_surrogate = 0;
+ /*
+ * Should we open tables with UTF8 support, or in the legacy 8-bit clean
+ * mode with ASCII-only casefolding?
+ */
+ get_mail_conf_nbool_table(first_nbool_defaults);
+
+ /*
+ * Report run-time versus compile-time discrepancies.
+ */
+#ifdef NO_EAI
+ if (var_smtputf8_enable)
+ msg_warn("%s is true, but EAI support is not compiled in",
+ VAR_SMTPUTF8_ENABLE);
+ var_smtputf8_enable = 0;
+#endif
+ util_utf8_enable = var_smtputf8_enable;
+
/*
* What protocols should we attempt to support? The result is stored in
* the global inet_proto_table variable.
get_mail_conf_int_table(other_int_defaults);
get_mail_conf_long_table(long_defaults);
get_mail_conf_bool_table(bool_defaults);
- get_mail_conf_nbool_table(nbool_defaults);
get_mail_conf_time_table(time_defaults);
check_default_privs();
check_mail_owner();
dict_lmdb_map_size = var_lmdb_map_size;
inet_windowsize = var_inet_windowsize;
- /*
- * Report run-time versus compile-time discrepancies.
- */
-#ifdef NO_EAI
- if (var_smtputf8_enable)
- msg_warn("%s is true, but EAI support is not compiled in",
- VAR_SMTPUTF8_ENABLE);
- var_smtputf8_enable = 0;
-#endif
- util_utf8_enable = var_smtputf8_enable;
-
/*
* Variables whose defaults are determined at runtime, after other
* variables have been set. This dependency is admittedly a bit tricky.
* Patches change both the patchlevel and the release date. Snapshots have no
* patchlevel; they change the release date only.
*/
-#define MAIL_RELEASE_DATE "20141228"
+#define MAIL_RELEASE_DATE "20150111"
#define MAIL_VERSION_NUMBER "2.12"
#ifdef SNAPSHOT
#include <dict_fail.h>
#include <sigdelay.h>
#include <mymalloc.h>
+#include <stringops.h>
/* Global library. */
/*
* Wrap the dictionary for UTF-8 syntax checks and casefolding.
*/
- if ((mkmap->dict->flags & DICT_FLAG_UTF8_PROXY) == 0
- && DICT_IS_ENABLE_UTF8(dict_flags))
- mkmap->dict = dict_utf8_encapsulate(mkmap->dict);
+ if ((mkmap->dict->flags & DICT_FLAG_UTF8_ACTIVE) == 0
+ && DICT_NEED_UTF8_ACTIVATION(util_utf8_enable, dict_flags))
+ mkmap->dict = dict_utf8_activate(mkmap->dict);
/*
* Resume signal delivery if multi-writer safe.
} else {
if (dict_handle(acl) == 0)
dict_register(acl, dict_open(acl, O_RDONLY, DICT_FLAG_LOCK
- | DICT_FLAG_FOLD_FIX));
+ | DICT_FLAG_FOLD_FIX
+ | DICT_FLAG_UTF8_REQUEST));
}
}
argv_add(intern_acl, acl, (char *) 0);
}
alias_maps = maps_create("aliases", var_alias_maps,
DICT_FLAG_LOCK | DICT_FLAG_PARANOID
- | DICT_FLAG_FOLD_FIX);
+ | DICT_FLAG_FOLD_FIX
+ | DICT_FLAG_UTF8_REQUEST);
flush_init();
}
*/
if (*var_mbox_transp_maps && transp_maps == 0)
transp_maps = maps_create(VAR_MBOX_TRANSP_MAPS, var_mbox_transp_maps,
- DICT_FLAG_LOCK | DICT_FLAG_NO_REGSUB);
+ DICT_FLAG_LOCK | DICT_FLAG_NO_REGSUB
+ | DICT_FLAG_UTF8_REQUEST);
/* The -1 is a hint for the down-stream deliver_completed() function. */
if (transp_maps
&& (map_transport = maps_find(transp_maps, state.msg_attr.user,
if (*var_mailbox_cmd_maps && cmd_maps == 0)
cmd_maps = maps_create(VAR_MAILBOX_CMD_MAPS, var_mailbox_cmd_maps,
- DICT_FLAG_LOCK | DICT_FLAG_PARANOID);
+ DICT_FLAG_LOCK | DICT_FLAG_PARANOID
+ | DICT_FLAG_UTF8_REQUEST);
if (cmd_maps && (map_command = maps_find(cmd_maps, state.msg_attr.user,
- DICT_FLAG_NONE)) != 0) {
+ DICT_FLAG_NONE)) != 0) {
status = deliver_command(state, usr_attr, map_command);
} else if (cmd_maps && cmd_maps->error != 0) {
/* Details in the logfile. */
*/
if (*var_fbck_transp_maps && transp_maps == 0)
transp_maps = maps_create(VAR_FBCK_TRANSP_MAPS, var_fbck_transp_maps,
- DICT_FLAG_LOCK | DICT_FLAG_NO_REGSUB);
+ DICT_FLAG_LOCK | DICT_FLAG_NO_REGSUB
+ | DICT_FLAG_UTF8_REQUEST);
/* The -1 is a hint for the down-stream deliver_completed() function. */
if (transp_maps
&& (map_transport = maps_find(transp_maps, state.msg_attr.user,
/*
* First some UTF-8 checks sans casefolding.
*/
- if (DICT_IS_ENABLE_UTF8(dict_flags)
+ if ((mkmap->dict->flags & DICT_FLAG_UTF8_ACTIVE)
&& !allascii(STR(line_buffer))
&& !valid_utf8_string(STR(line_buffer), LEN(line_buffer))) {
msg_warn("%s, line %d: non-UTF-8 input \"%s\"",
int postalias_flags = POSTALIAS_FLAG_AS_OWNER | POSTALIAS_FLAG_SAVE_PERM;
int open_flags = O_RDWR | O_CREAT | O_TRUNC;
int dict_flags = (DICT_FLAG_DUP_WARN | DICT_FLAG_FOLD_FIX
- | DICT_FLAG_UTF8_ENABLE);
+ | DICT_FLAG_UTF8_REQUEST);
char *query = 0;
char *delkey = 0;
int sequence = 0;
sequence = 1;
break;
case 'u':
- dict_flags &= ~DICT_FLAG_UTF8_ENABLE;
+ dict_flags &= ~DICT_FLAG_UTF8_REQUEST;
break;
case 'v':
msg_verbose++;
/*
* First some UTF-8 checks sans casefolding.
*/
- if (DICT_IS_ENABLE_UTF8(dict_flags)
+ if ((mkmap->dict->flags & DICT_FLAG_UTF8_ACTIVE)
&& !allascii(STR(line_buffer))
&& !valid_utf8_string(STR(line_buffer), LEN(line_buffer))) {
msg_warn("%s, line %d: non-UTF-8 input \"%s\"",
int postmap_flags = POSTMAP_FLAG_AS_OWNER | POSTMAP_FLAG_SAVE_PERM;
int open_flags = O_RDWR | O_CREAT | O_TRUNC;
int dict_flags = (DICT_FLAG_DUP_WARN | DICT_FLAG_FOLD_FIX
- | DICT_FLAG_UTF8_ENABLE);
+ | DICT_FLAG_UTF8_REQUEST);
char *query = 0;
char *delkey = 0;
int sequence = 0;
int found;
- int force_utf8 = 0;
+ int force_utf8 = 0;
/*
* Fingerprint executables and core dumps.
sequence = 1;
break;
case 'u':
- dict_flags &= ~DICT_FLAG_UTF8_ENABLE;
+ dict_flags &= ~DICT_FLAG_UTF8_REQUEST;
break;
case 'U':
force_utf8 = 1;
&& (postmap_flags & POSTMAP_FLAG_ANY_KEY)
== (postmap_flags & POSTMAP_FLAG_MIME_KEY))
msg_warn("ignoring -m option without -b or -h");
- if ((postmap_flags & (POSTMAP_FLAG_ANY_KEY & ~POSTMAP_FLAG_MIME_KEY))
+ if ((postmap_flags & (POSTMAP_FLAG_ANY_KEY & ~POSTMAP_FLAG_MIME_KEY))
&& force_utf8 == 0)
dict_flags &= ~DICT_FLAG_UTF8_MASK;
-
+
/*
* Use the map type specified by the user, or fall back to a default
* database type.
if (*var_smtp_generic_maps)
smtp_generic_maps =
maps_create(VAR_LMTP_SMTP(GENERIC_MAPS), var_smtp_generic_maps,
- DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX);
+ DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX
+ | DICT_FLAG_UTF8_REQUEST);
/*
* Header/body checks.
if (argc < 3)
msg_fatal("usage: %s maptype:mapname address...", argv[0]);
- maps = maps_create(argv[1], argv[1], DICT_FLAG_FOLD_FIX);
+ util_utf8_enable = 1;
+ maps = maps_create(argv[1], argv[1], DICT_FLAG_FOLD_FIX
+ | DICT_FLAG_UTF8_REQUEST);
mail_params_init();
if (chdir(var_queue_dir) < 0)
msg_fatal("chdir(%s): %m", var_queue_dir);
* dict_proxy module one level down in the build dependency hierachy.
*/
#define CACHE_DICT_OPEN_FLAGS \
- (DICT_FLAG_DUP_REPLACE | DICT_FLAG_SYNC_UPDATE)
+ (DICT_FLAG_DUP_REPLACE | DICT_FLAG_SYNC_UPDATE | DICT_FLAG_UTF8_REQUEST)
#define PROXY_COLON DICT_TYPE_PROXY ":"
#define PROXY_COLON_LEN (sizeof(PROXY_COLON) - 1)
* Open the per-host password table and initialize the SASL library. Use
* shared locks for reading, just in case someone updates the table.
*/
- smtp_sasl_passwd_map = maps_create("smtp_sasl_passwd",
+ smtp_sasl_passwd_map = maps_create(VAR_LMTP_SMTP(SASL_PASSWD),
var_smtp_sasl_passwd,
- DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX);
+ DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX
+ | DICT_FLAG_UTF8_REQUEST);
if ((smtp_sasl_impl = xsasl_client_init(var_smtp_sasl_type,
var_smtp_sasl_path)) == 0)
msg_fatal("SASL library initialization");
var_smtp_sasl_auth_cache_time);
#else
msg_warn("not compiled with TLS support -- "
- "ignoring the %s setting", VAR_LMTP_SMTP(SASL_AUTH_CACHE_NAME));
+ "ignoring the %s setting", VAR_LMTP_SMTP(SASL_AUTH_CACHE_NAME));
#endif
}
}
if (*var_smtp_tls_policy) {
tls_policy = maps_create(VAR_LMTP_SMTP(TLS_POLICY),
var_smtp_tls_policy,
- DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX);
+ DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX
+ | DICT_FLAG_UTF8_REQUEST);
if (*var_smtp_tls_per_site)
msg_warn("%s ignored when %s is not empty.",
VAR_LMTP_SMTP(TLS_PER_SITE), VAR_LMTP_SMTP(TLS_POLICY));
if (*var_smtp_tls_per_site) {
tls_per_site = maps_create(VAR_LMTP_SMTP(TLS_PER_SITE),
var_smtp_tls_per_site,
- DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX);
+ DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX
+ | DICT_FLAG_UTF8_REQUEST);
}
}
else if ((flags & SMTPD_CHECK_PARSE_MAPS)
&& strchr(name, ':') && dict_handle(name) == 0) {
dict_register(name, dict_open(name, O_RDONLY, DICT_FLAG_LOCK
- | DICT_FLAG_FOLD_FIX));
+ | DICT_FLAG_FOLD_FIX
+ | DICT_FLAG_UTF8_REQUEST));
}
last = name;
}
* Pre-parse and pre-open the recipient maps.
*/
local_rcpt_maps = maps_create(VAR_LOCAL_RCPT_MAPS, var_local_rcpt_maps,
- DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX);
+ DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX
+ | DICT_FLAG_UTF8_REQUEST);
rcpt_canon_maps = maps_create(VAR_RCPT_CANON_MAPS, var_rcpt_canon_maps,
- DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX);
+ DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX
+ | DICT_FLAG_UTF8_REQUEST);
canonical_maps = maps_create(VAR_CANONICAL_MAPS, var_canonical_maps,
- DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX);
+ DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX
+ | DICT_FLAG_UTF8_REQUEST);
virt_alias_maps = maps_create(VAR_VIRT_ALIAS_MAPS, var_virt_alias_maps,
- DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX);
+ DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX
+ | DICT_FLAG_UTF8_REQUEST);
virt_mailbox_maps = maps_create(VAR_VIRT_MAILBOX_MAPS,
var_virt_mailbox_maps,
- DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX);
+ DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX
+ | DICT_FLAG_UTF8_REQUEST);
relay_rcpt_maps = maps_create(VAR_RELAY_RCPT_MAPS, var_relay_rcpt_maps,
- DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX);
+ DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX
+ | DICT_FLAG_UTF8_REQUEST);
#ifdef TEST
virt_alias_doms = string_list_init(MATCH_FLAG_NONE, var_virt_alias_doms);
* Templates for RBL rejection replies.
*/
rbl_reply_maps = maps_create(VAR_RBL_REPLY_MAPS, var_rbl_reply_maps,
- DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX);
+ DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX
+ | DICT_FLAG_UTF8_REQUEST);
/*
* Sender to login name mapping.
*/
smtpd_sender_login_maps = maps_create(VAR_SMTPD_SND_AUTH_MAPS,
var_smtpd_snd_auth_maps,
- DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX);
+ DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX
+ | DICT_FLAG_UTF8_REQUEST);
/*
* error_text is used for returning error responses.
char **ext)
{
const char *result;
-
+
if ((result = mail_addr_find(maps, key, ext)) != 0 || maps->error == 0)
return (result);
if (maps->error == DICT_ERR_RETRY)
UPDATE_STRING(var_virt_alias_maps, args->argv[1]);
UPDATE_MAPS(virt_alias_maps, VAR_VIRT_ALIAS_MAPS,
var_virt_alias_maps, DICT_FLAG_LOCK
- | DICT_FLAG_FOLD_FIX);
+ | DICT_FLAG_FOLD_FIX | DICT_FLAG_UTF8_REQUEST);
resp = 0;
break;
}
UPDATE_STRING(var_virt_mailbox_maps, args->argv[1]);
UPDATE_MAPS(virt_mailbox_maps, VAR_VIRT_MAILBOX_MAPS,
var_virt_mailbox_maps, DICT_FLAG_LOCK
- | DICT_FLAG_FOLD_FIX);
+ | DICT_FLAG_FOLD_FIX | DICT_FLAG_UTF8_REQUEST);
resp = 0;
break;
}
UPDATE_STRING(var_local_rcpt_maps, args->argv[1]);
UPDATE_MAPS(local_rcpt_maps, VAR_LOCAL_RCPT_MAPS,
var_local_rcpt_maps, DICT_FLAG_LOCK
- | DICT_FLAG_FOLD_FIX);
+ | DICT_FLAG_FOLD_FIX | DICT_FLAG_UTF8_REQUEST);
resp = 0;
break;
}
UPDATE_STRING(var_relay_rcpt_maps, args->argv[1]);
UPDATE_MAPS(relay_rcpt_maps, VAR_RELAY_RCPT_MAPS,
var_relay_rcpt_maps, DICT_FLAG_LOCK
- | DICT_FLAG_FOLD_FIX);
+ | DICT_FLAG_FOLD_FIX | DICT_FLAG_UTF8_REQUEST);
resp = 0;
break;
}
UPDATE_STRING(var_canonical_maps, args->argv[1]);
UPDATE_MAPS(canonical_maps, VAR_CANONICAL_MAPS,
var_canonical_maps, DICT_FLAG_LOCK
- | DICT_FLAG_FOLD_FIX);
+ | DICT_FLAG_FOLD_FIX | DICT_FLAG_UTF8_REQUEST);
resp = 0;
break;
}
UPDATE_STRING(var_rbl_reply_maps, args->argv[1]);
UPDATE_MAPS(rbl_reply_maps, VAR_RBL_REPLY_MAPS,
var_rbl_reply_maps, DICT_FLAG_LOCK
- | DICT_FLAG_FOLD_FIX);
+ | DICT_FLAG_FOLD_FIX | DICT_FLAG_UTF8_REQUEST);
resp = 0;
break;
}
>>> # Expect: REJECT (temporary lookup failure)
>>> helo foobar
./smtpd_check: warning: fail:1_helo_access: table lookup problem
-./smtpd_check: <queue id>: reject: HELO from localhost[127.0.0.1]: 451 4.3.5 <foobar>: Helo command rejected: Server configuration error; proto=SMTP helo=<foobar>
-451 4.3.5 <foobar>: Helo command rejected: Server configuration error
+./smtpd_check: <queue id>: reject: HELO from localhost[127.0.0.1]: 451 4.3.0 <foobar>: Temporary lookup failure; proto=SMTP helo=<foobar>
+451 4.3.0 <foobar>: Temporary lookup failure
>>> #
>>> # Test check_namadr_access()
>>> #
>>> # Expect: REJECT (temporary lookup failure)
>>> client foo.dunno.com 131.155.210.17
./smtpd_check: warning: fail:1_client_access: table lookup problem
-./smtpd_check: <queue id>: reject: CONNECT from foo.dunno.com[131.155.210.17]: 451 4.3.5 <foo.dunno.com[131.155.210.17]>: Client host rejected: Server configuration error; proto=SMTP helo=<foobar>
-451 4.3.5 <foo.dunno.com[131.155.210.17]>: Client host rejected: Server configuration error
+./smtpd_check: <queue id>: reject: CONNECT from foo.dunno.com[131.155.210.17]: 451 4.3.0 <foo.dunno.com[131.155.210.17]>: Temporary lookup failure; proto=SMTP helo=<foobar>
+451 4.3.0 <foo.dunno.com[131.155.210.17]>: Temporary lookup failure
>>> #
>>> # Test check_mail_access()
>>> #
>>> # Expect: REJECT (temporary lookup failure)
>>> mail reject@dunno.domain
./smtpd_check: warning: fail:1_sender_access: table lookup problem
-./smtpd_check: <queue id>: reject: MAIL from foo.dunno.com[131.155.210.17]: 451 4.3.5 <reject@dunno.domain>: Sender address rejected: Server configuration error; from=<reject@dunno.domain> proto=SMTP helo=<foobar>
-451 4.3.5 <reject@dunno.domain>: Sender address rejected: Server configuration error
+./smtpd_check: <queue id>: reject: MAIL from foo.dunno.com[131.155.210.17]: 451 4.3.0 <reject@dunno.domain>: Temporary lookup failure; from=<reject@dunno.domain> proto=SMTP helo=<foobar>
+451 4.3.0 <reject@dunno.domain>: Temporary lookup failure
>>> #
>>> # Test check_rcpt_access()
>>> #
>>> # Expect: REJECT (temporary lookup failure)
>>> rcpt reject@dunno.domain
./smtpd_check: warning: fail:1_rcpt_access: table lookup problem
-./smtpd_check: <queue id>: reject: RCPT from foo.dunno.com[131.155.210.17]: 451 4.3.5 <reject@dunno.domain>: Recipient address rejected: Server configuration error; from=<reject@dunno.domain> to=<reject@dunno.domain> proto=SMTP helo=<foobar>
-451 4.3.5 <reject@dunno.domain>: Recipient address rejected: Server configuration error
+./smtpd_check: <queue id>: reject: RCPT from foo.dunno.com[131.155.210.17]: 451 4.3.0 <reject@dunno.domain>: Temporary lookup failure; from=<reject@dunno.domain> to=<reject@dunno.domain> proto=SMTP helo=<foobar>
+451 4.3.0 <reject@dunno.domain>: Temporary lookup failure
>>> # Expect: OK
>>> rcpt postmaster
OK
OK
>>> mail <>
./smtpd_check: warning: fail:1_sender_access: table lookup problem
-./smtpd_check: <queue id>: reject: MAIL from foo.dunno.com[131.155.210.17]: 451 4.3.5 <>: Sender address rejected: Server configuration error; from=<> proto=SMTP helo=<foobar>
-451 4.3.5 <>: Sender address rejected: Server configuration error
+./smtpd_check: <queue id>: reject: MAIL from foo.dunno.com[131.155.210.17]: 451 4.3.0 <>: Temporary lookup failure; from=<> proto=SMTP helo=<foobar>
+451 4.3.0 <>: Temporary lookup failure
>>> #
>>> # Test permit_tls_client_certs in generic_restrictions
>>> #
* opening a damaged file after some process terminated abnormally.
*/
#ifdef SINGLE_UPDATER
-#define DICT_FLAGS (DICT_FLAG_DUP_REPLACE | DICT_FLAG_OPEN_LOCK)
+#define DICT_FLAGS (DICT_FLAG_DUP_REPLACE | DICT_FLAG_OPEN_LOCK \
+ | DICT_FLAG_UTF8_REQUEST)
#else
#define DICT_FLAGS \
- (DICT_FLAG_DUP_REPLACE | DICT_FLAG_LOCK | DICT_FLAG_SYNC_UPDATE)
+ (DICT_FLAG_DUP_REPLACE | DICT_FLAG_LOCK | DICT_FLAG_SYNC_UPDATE \
+ | DICT_FLAG_UTF8_REQUEST)
#endif
dict = dict_open(dbname, O_RDWR | O_CREAT | O_TRUNC, DICT_FLAGS);
if (*var_relocated_maps)
relocated_maps =
maps_create(VAR_RELOCATED_MAPS, var_relocated_maps,
- DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX);
+ DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX
+ | DICT_FLAG_UTF8_REQUEST);
}
tp = (TRANSPORT_INFO *) mymalloc(sizeof(*tp));
tp->transport_path = maps_create(transport_maps_name, transport_maps,
DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX
- | DICT_FLAG_NO_REGSUB);
+ | DICT_FLAG_NO_REGSUB
+ | DICT_FLAG_UTF8_REQUEST);
tp->wildcard_channel = tp->wildcard_nexthop = 0;
tp->wildcard_errno = 0;
tp->expire = 0;
maps_create(resolve_regular.snd_relay_maps_name,
RES_PARAM_VALUE(resolve_regular.snd_relay_maps),
DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX
- | DICT_FLAG_NO_REGSUB);
+ | DICT_FLAG_NO_REGSUB | DICT_FLAG_UTF8_REQUEST);
if (*RES_PARAM_VALUE(resolve_verify.snd_relay_maps))
resolve_verify.snd_relay_info =
maps_create(resolve_verify.snd_relay_maps_name,
RES_PARAM_VALUE(resolve_verify.snd_relay_maps),
DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX
- | DICT_FLAG_NO_REGSUB);
+ | DICT_FLAG_NO_REGSUB | DICT_FLAG_UTF8_REQUEST);
if (*RES_PARAM_VALUE(resolve_regular.snd_def_xp_maps))
resolve_regular.snd_def_xp_info =
maps_create(resolve_regular.snd_def_xp_maps_name,
RES_PARAM_VALUE(resolve_regular.snd_def_xp_maps),
DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX
- | DICT_FLAG_NO_REGSUB);
+ | DICT_FLAG_NO_REGSUB | DICT_FLAG_UTF8_REQUEST);
if (*RES_PARAM_VALUE(resolve_verify.snd_def_xp_maps))
resolve_verify.snd_def_xp_info =
maps_create(resolve_verify.snd_def_xp_maps_name,
RES_PARAM_VALUE(resolve_verify.snd_def_xp_maps),
DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX
- | DICT_FLAG_NO_REGSUB);
+ | DICT_FLAG_NO_REGSUB | DICT_FLAG_UTF8_REQUEST);
}
/* post_jail_init - initialize after entering chroot jail */
$(SHLIB_ENV) ./dict_open inline:'{ foo=xx {x=y}x}' read </dev/null; \
(echo get foo; echo get bar; echo get baz) | $(SHLIB_ENV) \
./dict_open inline:'{ foo=XX, { bAr = lotsa stuff }}' read fold_fix; \
+ (echo get foo; echo get bar; echo get baz) | $(SHLIB_ENV) \
+ ./dict_open inline:'{ foo=XX, { bAr = lotsa stuff }}' read 'fold_fix,utf8_request'; \
) >dict_inline.tmp 2>&1
diff dict_inline.ref dict_inline.tmp
rm -f dict_inline.tmp
"open_lock", DICT_FLAG_OPEN_LOCK, /* permanent lock upon open */
"bulk_update", DICT_FLAG_BULK_UPDATE, /* bulk update if supported */
"multi_writer", DICT_FLAG_MULTI_WRITER, /* multi-writer safe */
- "utf8_enable", DICT_FLAG_UTF8_ENABLE, /* enable UTF-8 checks/fold */
- "utf8_proxy", DICT_FLAG_UTF8_PROXY, /* UTF-8 proxy is present */
+ "utf8_request", DICT_FLAG_UTF8_REQUEST, /* request UTF-8 activation */
+ "utf8_active", DICT_FLAG_UTF8_ACTIVE, /* UTF-8 is activated */
0,
};
#define DICT_FLAG_OPEN_LOCK (1<<16) /* perm lock if not multi-writer safe */
#define DICT_FLAG_BULK_UPDATE (1<<17) /* optimize for bulk updates */
#define DICT_FLAG_MULTI_WRITER (1<<18) /* multi-writer safe map */
-#define DICT_FLAG_UTF8_ENABLE (1<<19) /* enable UTF-8 checks */
-#define DICT_FLAG_UTF8_PROXY (1<<20) /* UTF-8 proxy layer is present */
+#define DICT_FLAG_UTF8_REQUEST (1<<19) /* activate UTF-8 if possible */
+#define DICT_FLAG_UTF8_ACTIVE (1<<20) /* UTF-8 proxy layer is present */
-#define DICT_FLAG_UTF8_MASK (DICT_FLAG_UTF8_ENABLE)
+#define DICT_FLAG_UTF8_MASK (DICT_FLAG_UTF8_REQUEST)
/* IMPORTANT: Update the dict_mask[] table when the above changes */
/*
* Feature tests.
*/
-extern int util_utf8_enable;
-
-#define DICT_IS_ENABLE_UTF8(flags) \
- (util_utf8_enable && (flags & DICT_FLAG_UTF8_MASK))
+#define DICT_NEED_UTF8_ACTIVATION(enable, flags) \
+ ((enable) && ((flags) & DICT_FLAG_UTF8_MASK))
/*
* dict->error values. Errors must be negative; smtpd_check depends on this.
/*
* Check and convert UTF-8 keys and values.
*/
-extern DICT *dict_utf8_encapsulate(DICT *);
+extern DICT *dict_utf8_activate(DICT *);
extern char *dict_utf8_check_fold(DICT *, const char *, CONST_CHAR_STAR *);
extern int dict_utf8_check(const char *, CONST_CHAR_STAR *);
/*
* UTF-8 syntax check.
*/
- if (DICT_IS_ENABLE_UTF8(dict_flags)
+ if (DICT_NEED_UTF8_ACTIVATION(util_utf8_enable, dict_flags)
&& allascii(name) == 0
&& valid_utf8_string(name, strlen(name)) == 0)
DICT_INLINE_RETURN(dict_surrogate(DICT_TYPE_INLINE, name,
bar=lotsa stuff
> get baz
baz: not found
+owner=trusted (uid=2147483647)
+> get foo
+foo=XX
+> get bar
+bar=lotsa stuff
+> get baz
+baz: not found
dict_type, dict_name);
}
/* Last step: insert proxy for UTF-8 syntax checks and casefolding. */
- if ((dict->flags & DICT_FLAG_UTF8_PROXY) == 0
- && DICT_IS_ENABLE_UTF8(dict_flags))
- dict = dict_utf8_encapsulate(dict);
+ if ((dict->flags & DICT_FLAG_UTF8_ACTIVE) == 0
+ && DICT_NEED_UTF8_ACTIVATION(util_utf8_enable, dict_flags))
+ dict = dict_utf8_activate(dict);
return (dict);
}
/* dict_type_override - disguise a dictionary type */
-void dict_type_override(DICT *dict, const char *type)
+void dict_type_override(DICT *dict, const char *type)
{
myfree(dict->type);
dict->type = mystrdup(type);
dict_flags |= DICT_FLAG_LOCK;
if ((dict_flags & (DICT_FLAG_DUP_WARN | DICT_FLAG_DUP_IGNORE)) == 0)
dict_flags |= DICT_FLAG_DUP_REPLACE;
- dict_flags |= DICT_FLAG_UTF8_ENABLE;
+ dict_flags |= DICT_FLAG_UTF8_REQUEST;
vstream_fflush(VSTREAM_OUT);
dict_name = argv[optind];
dict_allow_surrogate = 1;
/*
* First some UTF-8 checks sans casefolding.
*/
- if (DICT_IS_ENABLE_UTF8(dict_flags)
+ if ((dict->flags & DICT_FLAG_UTF8_ACTIVE)
&& allascii(STR(line_buffer)) == 0
&& valid_utf8_string(STR(line_buffer), LEN(line_buffer)) == 0) {
msg_warn("%s, line %d: non-UTF-8 input \"%s\""
/* SYNOPSIS
/* #include <dict.h>
/*
-/* DICT *dict_utf8_encapsulate(
+/* DICT *dict_utf8_activate(
/* DICT *dict)
/*
/* char *dict_utf8_check_fold(
/* const char *string,
/* CONST_CHAR_STAR *err)
/* DESCRIPTION
-/* dict_utf8_encapsulate() wraps a dictionary's lookup/update/delete
+/* dict_utf8_activate() wraps a dictionary's lookup/update/delete
/* methods with code that enforces UTF-8 checks on keys and
/* values, and that logs a warning when incorrect UTF-8 is
/* encountered. The original dictionary handle becomes invalid.
/*
/* The wrapper code enforces a policy that maximizes application
-/* robustness. Attempts to store non-UTF-8 keys or values are
-/* skipped while reporting success, attempts to look up or
-/* delete non-UTF-8 keys are skipped while reporting success,
-/* and attempts to look up a non-UTF-8 value are flagged while
-/* reporting a configuration error.
+/* robustness (it avoids the need for new error-handling code
+/* paths in application code). Attempts to store non-UTF-8
+/* keys or values are skipped while reporting a non-error
+/* status, attempts to look up or delete non-UTF-8 keys are
+/* skipped while reporting a non-error status, and attempts
+/* to look up a non-UTF-8 value are flagged while reporting a
+/* configuration error.
/*
/* The dict_utf8_check* functions may be invoked to perform
/* UTF-8 validity checks when util_utf8_enable is non-zero and
/*
/* dict_utf8_check() checks a string for UTF-8 validity. The
/* result is zero in case of error.
+/* BUGS
+/* dict_utf8_activate() does not nest.
/* LICENSE
/* .ad
/* .fi
#include <mymalloc.h>
#include <msg.h>
+ /*
+ * Backed-up accessor function pointers.
+ */
+typedef struct {
+ const char *(*lookup) (struct DICT *, const char *);
+ int (*update) (struct DICT *, const char *, const char *);
+ int (*delete) (struct DICT *, const char *);
+} DICT_UTF8_BACKUP;
+
/*
* The goal is to maximize robustness: bad UTF-8 should not appear in keys,
* because those are derived from controlled inputs, and values should be
/*
* Casefold and implicitly validate UTF-8.
*/
- if (fold_flag != 0 && (fold_flag == (dict->flags & DICT_FLAG_FIXED) ?
+ if (fold_flag != 0 && (fold_flag & (dict->flags & DICT_FLAG_FIXED) ?
DICT_FLAG_FOLD_FIX : DICT_FLAG_FOLD_MUL)) {
if (dict->fold_buf == 0)
dict->fold_buf = vstring_alloc(10);
/* dict_utf8_lookup - UTF-8 lookup method wrapper */
-static const char *dict_utf8_lookup(DICT *self, const char *key)
+static const char *dict_utf8_lookup(DICT *dict, const char *key)
{
- DICT *dict;
+ DICT_UTF8_BACKUP *backup;
const char *utf8_err;
const char *fold_res;
const char *value;
+ int saved_flags;
/*
* Validate and optionally fold the key, and if invalid skip the request.
*/
- if ((fold_res = dict_utf8_check_fold(self, key, &utf8_err)) == 0) {
+ if ((fold_res = dict_utf8_check_fold(dict, key, &utf8_err)) == 0) {
msg_warn("%s:%s: non-UTF-8 key \"%s\": %s",
- self->type, self->name, key, utf8_err);
- self->error = DICT_ERR_NONE;
+ dict->type, dict->name, key, utf8_err);
+ dict->error = DICT_ERR_NONE;
return (0);
}
/*
- * Proxy the request.
+ * Proxy the request with casefolding turned off.
*/
- dict = (void *) self - self->size;
- dict->flags = self->flags;
- value = dict->lookup(dict, fold_res);
- self->flags = dict->flags;
- self->error = dict->error;
+ saved_flags = (dict->flags & DICT_FLAG_FOLD_ANY);
+ dict->flags &= ~DICT_FLAG_FOLD_ANY;
+ backup = (void *) dict + dict->size;
+ value = backup->lookup(dict, fold_res);
+ dict->flags |= saved_flags;
/*
* Validate the result, and if invalid fail the request.
*/
if (value != 0 && dict_utf8_check(value, &utf8_err) == 0) {
msg_warn("%s:%s: key \"%s\": non-UTF-8 value \"%s\": %s",
- self->type, self->name, key, value, utf8_err);
- self->error = DICT_ERR_CONFIG;
+ dict->type, dict->name, key, value, utf8_err);
+ dict->error = DICT_ERR_CONFIG;
return (0);
} else {
return (value);
/* dict_utf8_update - UTF-8 update method wrapper */
-static int dict_utf8_update(DICT *self, const char *key, const char *value)
+static int dict_utf8_update(DICT *dict, const char *key, const char *value)
{
- DICT *dict;
+ DICT_UTF8_BACKUP *backup;
const char *utf8_err;
const char *fold_res;
+ int saved_flags;
int status;
/*
* Validate or fold the key, and if invalid skip the request.
*/
- if ((fold_res = dict_utf8_check_fold(self, key, &utf8_err)) == 0) {
+ if ((fold_res = dict_utf8_check_fold(dict, key, &utf8_err)) == 0) {
msg_warn("%s:%s: non-UTF-8 key \"%s\": %s",
- self->type, self->name, key, utf8_err);
- self->error = DICT_ERR_NONE;
+ dict->type, dict->name, key, utf8_err);
+ dict->error = DICT_ERR_NONE;
return (DICT_STAT_SUCCESS);
}
*/
else if (dict_utf8_check(value, &utf8_err) == 0) {
msg_warn("%s:%s: key \"%s\": non-UTF-8 value \"%s\": %s",
- self->type, self->name, key, value, utf8_err);
- self->error = DICT_ERR_NONE;
+ dict->type, dict->name, key, value, utf8_err);
+ dict->error = DICT_ERR_NONE;
return (DICT_STAT_SUCCESS);
}
/*
- * Proxy the request.
+ * Proxy the request with casefolding turned off.
*/
else {
- dict = (void *) self - self->size;
- dict->flags = self->flags;
- status = dict->update(dict, fold_res, value);
- self->flags = dict->flags;
- self->error = dict->error;
+ saved_flags = (dict->flags & DICT_FLAG_FOLD_ANY);
+ dict->flags &= ~DICT_FLAG_FOLD_ANY;
+ backup = (void *) dict + dict->size;
+ status = backup->update(dict, fold_res, value);
+ dict->flags |= saved_flags;
return (status);
}
}
/* dict_utf8_delete - UTF-8 delete method wrapper */
-static int dict_utf8_delete(DICT *self, const char *key)
+static int dict_utf8_delete(DICT *dict, const char *key)
{
- DICT *dict;
+ DICT_UTF8_BACKUP *backup;
const char *utf8_err;
const char *fold_res;
+ int saved_flags;
int status;
/*
* Validate and optionally fold the key, and if invalid skip the request.
*/
- if ((fold_res = dict_utf8_check_fold(self, key, &utf8_err)) == 0) {
+ if ((fold_res = dict_utf8_check_fold(dict, key, &utf8_err)) == 0) {
msg_warn("%s:%s: non-UTF-8 key \"%s\": %s",
- self->type, self->name, key, utf8_err);
- self->error = DICT_ERR_NONE;
+ dict->type, dict->name, key, utf8_err);
+ dict->error = DICT_ERR_NONE;
return (DICT_STAT_SUCCESS);
}
/*
- * Proxy the request.
+ * Proxy the request with casefolding turned off.
*/
else {
- dict = (void *) self - self->size;
- dict->flags = self->flags;
- status = dict->delete(dict, fold_res);
- self->flags = dict->flags;
- self->error = dict->error;
+ saved_flags = (dict->flags & DICT_FLAG_FOLD_ANY);
+ dict->flags &= ~DICT_FLAG_FOLD_ANY;
+ backup = (void *) dict + dict->size;
+ status = backup->delete(dict, fold_res);
+ dict->flags |= saved_flags;
return (status);
}
}
-/* dict_utf8_close - dummy */
+/* dict_utf8_activate - wrap a legacy dict object for UTF-8 processing */
-static void dict_utf8_close(DICT *self)
+DICT *dict_utf8_activate(DICT *dict)
{
- DICT *dict;
-
- /*
- * Destroy the dict object that we are appended to, and thereby destroy
- * ourselves.
- */
- dict = (void *) self - self->size;
- dict->close(dict);
-}
-
-/* dict_utf8_encapsulate - wrap a legacy dict object for UTF-8 processing */
-
-DICT *dict_utf8_encapsulate(DICT *dict)
-{
- DICT *self;
+ DICT_UTF8_BACKUP *backup;
/*
* Sanity check.
*/
- if (dict->flags & DICT_FLAG_UTF8_PROXY)
- msg_panic("dict_utf8_encapsulate: %s:%s is already encapsulated",
+ if (dict->flags & DICT_FLAG_UTF8_ACTIVE)
+ msg_panic("dict_utf8_activate: %s:%s is already encapsulated",
dict->type, dict->name);
/*
- * Append ourselves to the dict object, so that dict_close(dict) will do
- * the right thing. dict->size is based on the actual size of the dict
- * object's subclass, so we don't have to worry about alignment problems.
+ * Unlike dict_debug(3) we do not put a proxy dict object in front of the
+ * encapsulated object, because then we would have to bidirectionally
+ * propagate changes in the data members (errors, flags, jbuf, and so on)
+ * between proxy object and encapsulated object.
*
- * XXX Add dict_flags argument to dict_alloc() so that it can allocate the
- * right memory amount, and we can avoid having to resize an object.
+ * Instead we append ourselves to the encapsulated dict object itself, and
+ * redirect some function pointers. This approach does not yet generalize
+ * to arbitrary levels of encapsulation. That is, it does not co-exist
+ * with dict_debug(3) which is broken for the reasons stated above.
*/
- dict = myrealloc(dict, dict->size + sizeof(*self));
- self = (void *) dict + dict->size;
- *self = *dict;
+ dict = myrealloc(dict, dict->size + sizeof(*backup));
+ backup = (void *) dict + dict->size;
/*
- * Interpose on the lookup/update/delete/close methods. In particular we
- * do not interpose on the iterator. Invalid keys are not stored, and we
- * want to be able to delete an invalid value.
+ * Interpose on the lookup/update/delete methods. It is a conscious
+ * decision not to tinker with the iterator or destructor.
*/
- self->lookup = dict_utf8_lookup;
- self->update = dict_utf8_update;
- self->delete = dict_utf8_delete;
- self->close = dict_utf8_close;
+ backup->lookup = dict->lookup;
+ backup->update = dict->update;
+ backup->delete = dict->delete;
+
+ dict->lookup = dict_utf8_lookup;
+ dict->update = dict_utf8_update;
+ dict->delete = dict_utf8_delete;
/*
- * Finally, disable casefolding in the dict object. It now happens in the
- * lookup/update/delete wrappers.
+ * Leave our mark. See sanity check above.
*/
- dict->flags &= ~DICT_FLAG_FOLD_ANY;
- self->flags |= DICT_FLAG_UTF8_PROXY;
+ dict->flags |= DICT_FLAG_UTF8_ACTIVE;
- return (self);
+ return (dict);
}
printf "put xxx %c%c%c\n", 128, 128, 128
printf "get xxx\n"
exit
-}' | ./dict_open internal:whatever write utf8_enable
+}' | ./dict_open internal:whatever write utf8_request
owner=trusted (uid=2147483647)
> flags
-dict flags fixed|lock|replace|utf8_enable|utf8_proxy
+dict flags fixed|lock|replace|utf8_request|utf8_active
> verbose
> get foo
foo: not found
#define OPEN_FLAGS O_RDONLY
#define DICT_FLAGS (DICT_FLAG_LOCK | DICT_FLAG_FOLD_FIX \
- | DICT_FLAG_UTF8_ENABLE)
+ | DICT_FLAG_UTF8_REQUEST)
#define STR(x) vstring_str(x)
/*
* Start the cache cleanup thread after permanently dropping privileges.
*/
#define VERIFY_DICT_OPEN_FLAGS (DICT_FLAG_DUP_REPLACE | DICT_FLAG_SYNC_UPDATE \
- | DICT_FLAG_OPEN_LOCK)
+ | DICT_FLAG_OPEN_LOCK | DICT_FLAG_UTF8_REQUEST)
saved_mask = umask(022);
verify_map =
*/
virtual_mailbox_maps =
maps_create(VAR_VIRT_MAILBOX_MAPS, var_virt_mailbox_maps,
- DICT_FLAG_LOCK | DICT_FLAG_PARANOID);
+ DICT_FLAG_LOCK | DICT_FLAG_PARANOID
+ | DICT_FLAG_UTF8_REQUEST);
virtual_uid_maps =
maps_create(VAR_VIRT_UID_MAPS, var_virt_uid_maps,
- DICT_FLAG_LOCK | DICT_FLAG_PARANOID);
+ DICT_FLAG_LOCK | DICT_FLAG_PARANOID
+ | DICT_FLAG_UTF8_REQUEST);
virtual_gid_maps =
maps_create(VAR_VIRT_GID_MAPS, var_virt_gid_maps,
- DICT_FLAG_LOCK | DICT_FLAG_PARANOID);
+ DICT_FLAG_LOCK | DICT_FLAG_PARANOID
+ | DICT_FLAG_UTF8_REQUEST);
virtual_mbox_lock_mask = mbox_lock_mask(var_virt_mailbox_lock);
}
projects / thirdparty / postfix.git / commitdiff
