From: Wietse Venema Date: Sun, 6 Jan 2019 05:00:00 +0000 (-0500) Subject: postfix-3.4-20190106 X-Git-Tag: v3.4.0-RC1~6 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=f442aefe04c5f975365d7939d13620c619fd3f19;p=thirdparty%2Fpostfix.git postfix-3.4-20190106 --- diff --git a/postfix/HISTORY b/postfix/HISTORY index b3a77cf05..033625768 100644 --- a/postfix/HISTORY +++ b/postfix/HISTORY @@ -23928,3 +23928,8 @@ Apologies for any names omitted. into the maps_find() method. Files: util/dict.h. util/dict_file.c, global/maps.[hc], postmap/postmap.c. +20190106 + + Documentation: documented the SRC_RHS_IS_FILE flag in + dict_open.c, and updated the -F description in the postmap + manpage. Files: util/dict_open.c, postmap/postmap.c. diff --git a/postfix/WISHLIST b/postfix/WISHLIST index edc77926a..67a0abb8e 100644 --- a/postfix/WISHLIST +++ b/postfix/WISHLIST @@ -1,8 +1,5 @@ Wish list: - With DICT_FLAG_RHS_IS_FILE, should dict_update() open a - file? base64-encode the value? - In smtpd(8) and postscreen(8), set the ehlo_discard_mask to ~0 so that STARTTLS, BDAT, DSN, etc. work only for clients that send EHLO. diff --git a/postfix/html/postmap.1.html b/postfix/html/postmap.1.html index d7a205660..5e8445472 100644 --- a/postfix/html/postmap.1.html +++ b/postfix/html/postmap.1.html @@ -41,143 +41,146 @@ POSTMAP(1) POSTMAP(1) space is stripped off. Whitespace in lookup keys is supported as of Postfix 3.2. - When the -F option is given, the value must specify a filename; - postmap(1) will store the base64-encoded content of that file instead - of the value itself. + When the -F option is given, the value must specify one or more file- + names separated by comma and/or whitespace; postmap(1) will concatenate + the file content (with a newline character inserted between files) and + will store the base64-encoded result instead of the value. - When the key specifies email address information, the localpart should + When the key specifies email address information, the localpart should be enclosed with double quotes if required by RFC 5322. For example, an address localpart that contains ";", or a localpart that starts or ends with ".". - By default the lookup key is mapped to lowercase to make the lookups + By default the lookup key is mapped to lowercase to make the lookups case insensitive; as of Postfix 2.3 this case folding happens only with tables whose lookup keys are fixed-case strings such as btree:, dbm: or hash:. With earlier versions, the lookup key is folded even with tables - where a lookup field can match both upper and lower case text, such as - regexp: and pcre:. This resulted in loss of information with $number + where a lookup field can match both upper and lower case text, such as + regexp: and pcre:. This resulted in loss of information with $number substitutions. COMMAND-LINE ARGUMENTS - -b Enable message body query mode. When reading lookup keys from - standard input with "-q -", process the input as if it is an - email message in RFC 5322 format. Each line of body content + -b Enable message body query mode. When reading lookup keys from + standard input with "-q -", process the input as if it is an + email message in RFC 5322 format. Each line of body content becomes one lookup key. - By default, the -b option starts generating lookup keys at the - first non-header line, and stops when the end of the message is - reached. To simulate body_checks(5) processing, enable MIME - parsing with -m. With this, the -b option generates no - body-style lookup keys for attachment MIME headers and for + By default, the -b option starts generating lookup keys at the + first non-header line, and stops when the end of the message is + reached. To simulate body_checks(5) processing, enable MIME + parsing with -m. With this, the -b option generates no + body-style lookup keys for attachment MIME headers and for attached message/* headers. - NOTE: with "smtputf8_enable = yes", the -b option option dis- - ables UTF-8 syntax checks on query keys and lookup results. + NOTE: with "smtputf8_enable = yes", the -b option option dis- + ables UTF-8 syntax checks on query keys and lookup results. Specify the -U option to force UTF-8 syntax checks anyway. This feature is available in Postfix version 2.6 and later. -c config_dir - Read the main.cf configuration file in the named directory + Read the main.cf configuration file in the named directory instead of the default configuration directory. - -d key Search the specified maps for key and remove one entry per map. - The exit status is zero when the requested information was + -d key Search the specified maps for key and remove one entry per map. + The exit status is zero when the requested information was found. - If a key value of - is specified, the program reads key values - from the standard input stream. The exit status is zero when at + If a key value of - is specified, the program reads key values + from the standard input stream. The exit status is zero when at least one of the requested keys was found. - -f Do not fold the lookup key to lower case while creating or + -f Do not fold the lookup key to lower case while creating or querying a table. - With Postfix version 2.3 and later, this option has no effect + With Postfix version 2.3 and later, this option has no effect for regular expression tables. There, case folding is controlled by appending a flag to a pattern. - -F When creating a map from source file, replace each value with - the base64-encoded content of the named file. When querying a - map, or listing a map, base64-decode each value. + -F When querying a map, or listing a map, base64-decode each value. + When creating a map from source file, process each value as a + list of filenames, concatenate the content of those files, and + store the base64-encoded result instead of the value (see INPUT + FORMAT for details). - -h Enable message header query mode. When reading lookup keys from - standard input with "-q -", process the input as if it is an - email message in RFC 5322 format. Each logical header line - becomes one lookup key. A multi-line header becomes one lookup + -h Enable message header query mode. When reading lookup keys from + standard input with "-q -", process the input as if it is an + email message in RFC 5322 format. Each logical header line + becomes one lookup key. A multi-line header becomes one lookup key with one or more embedded newline characters. - By default, the -h option generates lookup keys until the first - non-header line is reached. To simulate header_checks(5) pro- - cessing, enable MIME parsing with -m. With this, the -h option - also generates header-style lookup keys for attachment MIME + By default, the -h option generates lookup keys until the first + non-header line is reached. To simulate header_checks(5) pro- + cessing, enable MIME parsing with -m. With this, the -h option + also generates header-style lookup keys for attachment MIME headers and for attached message/* headers. - NOTE: with "smtputf8_enable = yes", the -b option option dis- - ables UTF-8 syntax checks on query keys and lookup results. + NOTE: with "smtputf8_enable = yes", the -b option option dis- + ables UTF-8 syntax checks on query keys and lookup results. Specify the -U option to force UTF-8 syntax checks anyway. This feature is available in Postfix version 2.6 and later. - -i Incremental mode. Read entries from standard input and do not - truncate an existing database. By default, postmap(1) creates a + -i Incremental mode. Read entries from standard input and do not + truncate an existing database. By default, postmap(1) creates a new database from the entries in file_name. -m Enable MIME parsing with "-b" and "-h". This feature is available in Postfix version 2.6 and later. - -N Include the terminating null character that terminates lookup - keys and values. By default, postmap(1) does whatever is the + -N Include the terminating null character that terminates lookup + keys and values. By default, postmap(1) does whatever is the default for the host operating system. - -n Don't include the terminating null character that terminates - lookup keys and values. By default, postmap(1) does whatever is + -n Don't include the terminating null character that terminates + lookup keys and values. By default, postmap(1) does whatever is the default for the host operating system. - -o Do not release root privileges when processing a non-root input - file. By default, postmap(1) drops root privileges and runs as + -o Do not release root privileges when processing a non-root input + file. By default, postmap(1) drops root privileges and runs as the source file owner instead. - -p Do not inherit the file access permissions from the input file - when creating a new file. Instead, create a new file with + -p Do not inherit the file access permissions from the input file + when creating a new file. Instead, create a new file with default access permissions (mode 0644). - -q key Search the specified maps for key and write the first value - found to the standard output stream. The exit status is zero + -q key Search the specified maps for key and write the first value + found to the standard output stream. The exit status is zero when the requested information was found. - Note: this performs a single query with the key as specified, - and does not make iterative queries with substrings of the key - as described for access(5), canonical(5), transport(5), vir- + Note: this performs a single query with the key as specified, + and does not make iterative queries with substrings of the key + as described for access(5), canonical(5), transport(5), vir- tual(5) and other Postfix table-driven features. - If a key value of - is specified, the program reads key values - from the standard input stream and writes one line of key value + If a key value of - is specified, the program reads key values + from the standard input stream and writes one line of key value output for each key that was found. The exit status is zero when at least one of the requested keys was found. - -r When updating a table, do not complain about attempts to update + -r When updating a table, do not complain about attempts to update existing entries, and make those updates anyway. - -s Retrieve all database elements, and write one line of key value - output for each element. The elements are printed in database - order, which is not necessarily the same as the original input + -s Retrieve all database elements, and write one line of key value + output for each element. The elements are printed in database + order, which is not necessarily the same as the original input order. - This feature is available in Postfix version 2.2 and later, and + This feature is available in Postfix version 2.2 and later, and is not available for all database types. - -u Disable UTF-8 support. UTF-8 support is enabled by default when - "smtputf8_enable = yes". It requires that keys and values are + -u Disable UTF-8 support. UTF-8 support is enabled by default when + "smtputf8_enable = yes". It requires that keys and values are valid UTF-8 strings. -U With "smtputf8_enable = yes", force UTF-8 syntax checks with the -b and -h options. - -v Enable verbose logging for debugging purposes. Multiple -v + -v Enable verbose logging for debugging purposes. Multiple -v options make the software increasingly verbose. - -w When updating a table, do not complain about attempts to update + -w When updating a table, do not complain about attempts to update existing entries, and ignore those attempts. Arguments: @@ -189,32 +192,32 @@ POSTMAP(1) POSTMAP(1) The postmap(1) command can query any supported file type, but it can create only the following file types: - btree The output file is a btree file, named file_name.db. - This is available on systems with support for db data- + btree The output file is a btree file, named file_name.db. + This is available on systems with support for db data- bases. - cdb The output consists of one file, named file_name.cdb. - This is available on systems with support for cdb data- + cdb The output consists of one file, named file_name.cdb. + This is available on systems with support for cdb data- bases. dbm The output consists of two files, named file_name.pag and file_name.dir. This is available on systems with support for dbm databases. - hash The output file is a hashed file, named file_name.db. - This is available on systems with support for db data- + hash The output file is a hashed file, named file_name.db. + This is available on systems with support for db data- bases. - fail A table that reliably fails all requests. The lookup ta- - ble name is used for logging only. This table exists to + fail A table that reliably fails all requests. The lookup ta- + ble name is used for logging only. This table exists to simplify Postfix error tests. sdbm The output consists of two files, named file_name.pag and file_name.dir. This is available on systems with support for sdbm databases. - When no file_type is specified, the software uses the database - type specified via the default_database_type configuration + When no file_type is specified, the software uses the database + type specified via the default_database_type configuration parameter. file_name @@ -223,11 +226,11 @@ POSTMAP(1) POSTMAP(1) DIAGNOSTICS Problems are logged to the standard error stream and to syslogd(8). No - output means that no problems were detected. Duplicate entries are + output means that no problems were detected. Duplicate entries are skipped and are flagged with a warning. postmap(1) terminates with zero exit status in case of success (includ- - ing successful "postmap -q" lookup) and terminates with non-zero exit + ing successful "postmap -q" lookup) and terminates with non-zero exit status in case of failure. ENVIRONMENT @@ -238,12 +241,12 @@ POSTMAP(1) POSTMAP(1) Enable verbose logging for debugging purposes. CONFIGURATION PARAMETERS - The following main.cf parameters are especially relevant to this pro- - gram. The text below provides only a parameter summary. See post- + The following main.cf parameters are especially relevant to this pro- + gram. The text below provides only a parameter summary. See post- conf(5) for more details including examples. berkeley_db_create_buffer_size (16777216) - The per-table I/O buffer size for programs that create Berkeley + The per-table I/O buffer size for programs that create Berkeley DB hash or btree tables. berkeley_db_read_buffer_size (131072) @@ -251,7 +254,7 @@ POSTMAP(1) POSTMAP(1) hash or btree tables. config_directory (see 'postconf -d' output) - The default location of the Postfix main.cf and master.cf con- + The default location of the Postfix main.cf and master.cf con- figuration files. default_database_type (see 'postconf -d' output) @@ -259,19 +262,19 @@ POSTMAP(1) POSTMAP(1) and postmap(1) commands. import_environment (see 'postconf -d' output) - The list of environment parameters that a privileged Postfix - process will import from a non-Postfix parent process, or + The list of environment parameters that a privileged Postfix + process will import from a non-Postfix parent process, or name=value environment overrides. smtputf8_enable (yes) - Enable preliminary SMTPUTF8 support for the protocols described + Enable preliminary SMTPUTF8 support for the protocols described in RFC 6531..6533. syslog_facility (mail) The syslog facility of Postfix logging. syslog_name (see 'postconf -d' output) - A prefix that is prepended to the process name in syslog + A prefix that is prepended to the process name in syslog records, so that, for example, "smtpd" becomes "prefix/smtpd". SEE ALSO diff --git a/postfix/man/man1/postmap.1 b/postfix/man/man1/postmap.1 index a6f32c4db..bfb06328d 100644 --- a/postfix/man/man1/postmap.1 +++ b/postfix/man/man1/postmap.1 @@ -49,9 +49,11 @@ surrounding white space is stripped off. Whitespace in lookup keys is supported as of Postfix 3.2. When the \fB\-F\fR option is given, the \fIvalue\fR must -specify a filename; \fBpostmap\fR(1) will store the -base64\-encoded content of that file instead of the \fIvalue\fR -itself. +specify one or more filenames separated by comma and/or +whitespace; \fBpostmap\fR(1) will concatenate the file +content (with a newline character inserted between files) +and will store the base64\-encoded result instead of the +\fIvalue\fR. When the \fIkey\fR specifies email address information, the localpart should be enclosed with double quotes if required @@ -109,9 +111,11 @@ With Postfix version 2.3 and later, this option has no effect for regular expression tables. There, case folding is controlled by appending a flag to a pattern. .IP \fB\-F\fR -When creating a map from source file, replace each value -with the base64\-encoded content of the named file. When -querying a map, or listing a map, base64\-decode each value. +When querying a map, or listing a map, base64\-decode each +value. When creating a map from source file, process each +value as a list of filenames, concatenate the content of +those files, and store the base64\-encoded result instead +of the value (see INPUT FORMAT for details). .IP \fB\-h\fR Enable message header query mode. When reading lookup keys from standard input with "\fB\-q \-\fR", process the input diff --git a/postfix/src/global/mail_version.h b/postfix/src/global/mail_version.h index 012f65a77..f4091897b 100644 --- a/postfix/src/global/mail_version.h +++ b/postfix/src/global/mail_version.h @@ -20,7 +20,7 @@ * Patches change both the patchlevel and the release date. Snapshots have no * patchlevel; they change the release date only. */ -#define MAIL_RELEASE_DATE "20181229" +#define MAIL_RELEASE_DATE "20190106" #define MAIL_VERSION_NUMBER "3.4" #ifdef SNAPSHOT diff --git a/postfix/src/postmap/postmap.c b/postfix/src/postmap/postmap.c index 9f368abb5..db40a38c3 100644 --- a/postfix/src/postmap/postmap.c +++ b/postfix/src/postmap/postmap.c @@ -41,9 +41,11 @@ /* keys is supported as of Postfix 3.2. /* /* When the \fB-F\fR option is given, the \fIvalue\fR must -/* specify a filename; \fBpostmap\fR(1) will store the -/* base64-encoded content of that file instead of the \fIvalue\fR -/* itself. +/* specify one or more filenames separated by comma and/or +/* whitespace; \fBpostmap\fR(1) will concatenate the file +/* content (with a newline character inserted between files) +/* and will store the base64-encoded result instead of the +/* \fIvalue\fR. /* /* When the \fIkey\fR specifies email address information, the /* localpart should be enclosed with double quotes if required @@ -99,9 +101,11 @@ /* effect for regular expression tables. There, case folding /* is controlled by appending a flag to a pattern. /* .IP \fB-F\fR -/* When creating a map from source file, replace each value -/* with the base64-encoded content of the named file. When -/* querying a map, or listing a map, base64-decode each value. +/* When querying a map, or listing a map, base64-decode each +/* value. When creating a map from source file, process each +/* value as a list of filenames, concatenate the content of +/* those files, and store the base64-encoded result instead +/* of the value (see INPUT FORMAT for details). /* .IP \fB-h\fR /* Enable message header query mode. When reading lookup keys /* from standard input with "\fB-q -\fR", process the input @@ -884,9 +888,9 @@ static void postmap_seq(const char *map_type, const char *map_name, if ((unb64 = dict_file_from_b64(dict, value)) == 0) { err = dict_file_get_error(dict); msg_warn("table %s:%s: key %s: %s", - dict->type, dict->name, - key, err); + dict->type, dict->name, key, err); myfree(err); + /* dict->error = DICT_ERR_CONFIG; */ continue; } value = STR(unb64); diff --git a/postfix/src/util/dict_file.c b/postfix/src/util/dict_file.c index ca4924c85..eb86b3522 100644 --- a/postfix/src/util/dict_file.c +++ b/postfix/src/util/dict_file.c @@ -222,9 +222,7 @@ const char *dict_file_lookup(DICT *dict, const char *key) return (0); if ((unb64 = dict_file_from_b64(dict, res)) == 0) { err = dict_file_get_error(dict); - msg_warn("table %s:%s: key %s: %s", - dict->type, dict->name, - key, err); + msg_warn("table %s:%s: key %s: %s", dict->type, dict->name, key, err); myfree(err); dict->error = DICT_ERR_CONFIG; return (0); diff --git a/postfix/src/util/dict_open.c b/postfix/src/util/dict_open.c index ca8df67ee..afea3917f 100644 --- a/postfix/src/util/dict_open.c +++ b/postfix/src/util/dict_open.c @@ -162,6 +162,26 @@ /* request with a non-UTF-8 key, skip an update request with /* a non-UTF-8 value, and fail a lookup request with a non-UTF-8 /* value. +/* .IP DICT_FLAG_SRC_RHS_IS_FILE +/* With dictionaries that are created from source text, each +/* value in the source of a dictionary specifies a list of +/* file names separated by comma and/or whitespace. The file +/* contents are concatenated with a newline inserted between +/* files, and the base64-encoded result is stored under the +/* key. +/* .sp +/* NOTE 1: it is up to the application to decode lookup results +/* with dict_file_lookup() or equivalent (this requires that +/* the dictionary is opened with DICT_FLAG_SRC_RHS_IS_FILE). +/* Decoding is not built into the normal dictionary lookup +/* method, because that would complicate dictionary nesting, +/* pipelining, and proxying. +/* .sp +/* NOTE 2: it is up to the application to convert file names +/* into base64-encoded file content before calling the dictionary +/* update method (see dict_file(3) for support). Automatic +/* file content encoding is available only when a dictionary +/* is created from source text. /* .PP /* Specify DICT_FLAG_NONE for no special processing. /*