git-send-email: add --transfer-encoding option

The thread at http://thread.gmane.org/gmane.comp.version-control.git/257392
details problems when applying patches with "git am" in a repository with
CRLF line endings.  In the example in the thread, the repository originated
from "git-svn" so it is not possible to use core.eol and friends on it.

Right now, the best option is to use "git am --keep-cr".  However, when
a patch create new files, the patch application process will reject the
new file because it finds a "/dev/null\r" string instead of "/dev/null".

The problem is that SMTP transport is CRLF-unsafe.  Sending a patch by
email is the same as passing it through "dos2unix | unix2dos".  The newly
introduced CRLFs are normally transparent because git-am strips them. The
keepcr=true setting preserves them, but it is mostly working by chance
and it would be very problematic to have a "git am" workflow in a
repository with mixed LF and CRLF line endings.

The MIME solution to this is the quoted-printable transfer enconding.
This is not something that we want to enable by default, since it makes
received emails horrible to look at.  However, it is a very good match
for projects that store CRLF line endings in the repository.

The only disadvantage of quoted-printable is that quoted-printable
patches fail to apply if the maintainer uses "git am --keep-cr".  This
is because the decoded patch will have two carriage returns at the end
of the line.  Therefore, add support for base64 transfer encoding too,
which makes received emails downright impossible to look at outside
a MUA, but really just works.

The patch covers all bases, including users that still live in the late
80s, by also providing a 7bit content transfer encoding that refuses
to send emails with non-ASCII character in them.  And finally, "8bit"
will add a Content-Transfer-Encoding header but otherwise do nothing.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 922072596fdd80eaaa47746fba5e5591223c5108..cc2ff201be1d53cda4d35afe15b8a140b145bd6e 100644 (file)
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -2303,6 +2303,7 @@ sendemail.smtpserverport::
 sendemail.smtpserveroption::
 sendemail.smtpuser::
 sendemail.thread::
+sendemail.transferencoding::
 sendemail.validate::
        See linkgit:git-send-email[1] for description.
 

diff --git a/Documentation/git-send-email.txt b/Documentation/git-send-email.txt
index a60776eb579e10507814c643cc292dfa99750aa9..a9efa5c2ec18776e12c5bdbb250df5b8e0f41581 100644 (file)
--- a/Documentation/git-send-email.txt
+++ b/Documentation/git-send-email.txt
@@ -131,6 +131,16 @@ Note that no attempts whatsoever are made to validate the encoding.
        Specify encoding of compose message. Default is the value of the
        'sendemail.composeencoding'; if that is unspecified, UTF-8 is assumed.
 
+--transfer-encoding=(7bit|8bit|quoted-printable|base64)::
+       Specify the transfer encoding to be used to send the message over SMTP.
+       7bit will fail upon encountering a non-ASCII message.  quoted-printable
+       can be useful when the repository contains files that contain carriage
+       returns, but makes the raw patch email file (as saved from a MUA) much
+       harder to inspect manually.  base64 is even more fool proof, but also
+       even more opaque.  Default is the value of the 'sendemail.transferEncoding'
+       configuration value; if that is unspecified, git will use 8bit and not
+       add a Content-Transfer-Encoding header.
+
 
 Sending
 ~~~~~~~

diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 2fece98c603e24ed5ce08156e0e44199a149ba83..b154f7066bbcb51e789c269600019d296cba04f4 100644 (file)
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -1875,6 +1875,10 @@ _git_config ()
                __gitcomp "$__git_send_email_suppresscc_options"
                return
                ;;
+       sendemail.transferencoding)
+               __gitcomp "7bit 8bit quoted-printable base64"
+               return
+               ;;
        --get|--get-all|--unset|--unset-all)
                __gitcomp_nl "$(__git_config_get_set_variables)"
                return

diff --git a/git-send-email.perl b/git-send-email.perl
index b29a3045b7d91eb6c9543559e0cb909219a826dd..82c6feaa4640e993051bfbe57df593c0f012a9ba 100755 (executable)
--- a/git-send-email.perl
+++ b/git-send-email.perl
@@ -58,6 +58,7 @@ git send-email [options] <file | directory | rev-list options >
     --compose                      * Open an editor for introduction.
     --compose-encoding      <str>  * Encoding to assume for introduction.
     --8bit-encoding         <str>  * Encoding to assume 8bit mails if undeclared
+    --transfer-encoding     <str>  * Transfer encoding to use (quoted-printable, 8bit, base64)
 
   Sending:
     --envelope-sender       <str>  * Email envelope sender.
@@ -206,6 +207,7 @@ my ($validate, $confirm);
 my (@suppress_cc);
 my ($auto_8bit_encoding);
 my ($compose_encoding);
+my ($target_xfer_encoding);
 
 my ($debug_net_smtp) = 0;              # Net::SMTP, see send_message()
 
@@ -242,6 +244,7 @@ my %config_settings = (
     "from" => \$sender,
     "assume8bitencoding" => \$auto_8bit_encoding,
     "composeencoding" => \$compose_encoding,
+    "transferencoding" => \$target_xfer_encoding,
 );
 
 my %config_path_settings = (
@@ -314,6 +317,7 @@ my $rc = GetOptions("h" => \$help,
                    "envelope-sender=s" => \$envelope_sender,
                    "thread!" => \$thread,
                    "validate!" => \$validate,
+                   "transfer-encoding=s" => \$target_xfer_encoding,
                    "format-patch!" => \$format_patch,
                    "8bit-encoding=s" => \$auto_8bit_encoding,
                    "compose-encoding=s" => \$compose_encoding,
@@ -1482,6 +1486,12 @@ foreach my $t (@files) {
                        }
                }
        }
+       if (defined $target_xfer_encoding) {
+               $xfer_encoding = '8bit' if not defined $xfer_encoding;
+               $message = apply_transfer_encoding(
+                       $message, $xfer_encoding, $target_xfer_encoding);
+               $xfer_encoding = $target_xfer_encoding;
+       }
        if (defined $xfer_encoding) {
                push @xh, "Content-Transfer-Encoding: $xfer_encoding";
        }
@@ -1556,6 +1566,32 @@ sub cleanup_compose_files {
 
 $smtp->quit if $smtp;
 
+sub apply_transfer_encoding {
+       my $message = shift;
+       my $from = shift;
+       my $to = shift;
+
+       return $message if ($from eq $to and $from ne '7bit');
+
+       require MIME::QuotedPrint;
+       require MIME::Base64;
+
+       $message = MIME::QuotedPrint::decode($message)
+               if ($from eq 'quoted-printable');
+       $message = MIME::Base64::decode($message)
+               if ($from eq 'base64');
+
+       die "cannot send message as 7bit"
+               if ($to eq '7bit' and $message =~ /[^[:ascii:]]/);
+       return $message
+               if ($to eq '7bit' or $to eq '8bit');
+       return MIME::QuotedPrint::encode($message, "\n", 0)
+               if ($to eq 'quoted-printable');
+       return MIME::Base64::encode($message, "\n")
+               if ($to eq 'base64');
+       die "invalid transfer encoding";
+}
+
 sub unique_email_list {
        my %seen;
        my @emails;

diff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh
index 19a3ced600a20d805d430533b275a8fa4269b835..e5016f457398a0770ebfc0e98d6f60fdfa4be2fc 100755 (executable)
--- a/t/t9001-send-email.sh
+++ b/t/t9001-send-email.sh
@@ -1298,6 +1298,163 @@ test_expect_success $PREREQ '--8bit-encoding also treats subject' '
        test_cmp expected actual
 '
 
+test_expect_success $PREREQ 'setup expect' '
+cat >email-using-8bit <<EOF
+From fe6ecc66ece37198fe5db91fa2fc41d9f4fe5cc4 Mon Sep 17 00:00:00 2001
+Message-Id: <bogus-message-id@example.com>
+From: A U Thor <author@example.com>
+Date: Sat, 12 Jun 2010 15:53:58 +0200
+Content-Type: text/plain; charset=UTF-8
+Subject: Nothing to see here.
+
+Dieser Betreff enthält auch einen Umlaut!
+EOF
+'
+
+test_expect_success $PREREQ 'sendemail.transferencoding=7bit fails on 8bit data' '
+       clean_fake_sendmail &&
+       git config sendemail.transferEncoding 7bit &&
+       test_must_fail git send-email \
+         --transfer-encoding=7bit \
+         --smtp-server="$(pwd)/fake.sendmail" \
+         email-using-8bit \
+         2>errors >out &&
+       grep "cannot send message as 7bit" errors &&
+       test -z "$(ls msgtxt*)"
+'
+
+test_expect_success $PREREQ '--transfer-encoding overrides sendemail.transferEncoding' '
+       clean_fake_sendmail &&
+       git config sendemail.transferEncoding 8bit
+       test_must_fail git send-email \
+         --transfer-encoding=7bit \
+         --smtp-server="$(pwd)/fake.sendmail" \
+         email-using-8bit \
+         2>errors >out &&
+       grep "cannot send message as 7bit" errors &&
+       test -z "$(ls msgtxt*)"
+'
+
+test_expect_success $PREREQ 'sendemail.transferencoding=8bit' '
+       clean_fake_sendmail &&
+       git send-email \
+         --transfer-encoding=8bit \
+         --smtp-server="$(pwd)/fake.sendmail" \
+         email-using-8bit \
+         2>errors >out &&
+       sed '1,/^$/d' msgtxt1 >actual &&
+       sed '1,/^$/d' email-using-8bit >expected &&
+       test_cmp expected actual
+'
+
+test_expect_success $PREREQ 'setup expect' '
+cat >expected <<EOF
+Dieser Betreff enth=C3=A4lt auch einen Umlaut!
+EOF
+'
+
+test_expect_success $PREREQ '8-bit and sendemail.transferencoding=quoted-printable' '
+       clean_fake_sendmail &&
+       git send-email \
+         --transfer-encoding=quoted-printable \
+         --smtp-server="$(pwd)/fake.sendmail" \
+         email-using-8bit \
+         2>errors >out &&
+       sed '1,/^$/d' msgtxt1 >actual &&
+       test_cmp expected actual
+'
+
+test_expect_success $PREREQ 'setup expect' '
+cat >expected <<EOF
+RGllc2VyIEJldHJlZmYgZW50aMOkbHQgYXVjaCBlaW5lbiBVbWxhdXQhCg==
+EOF
+'
+
+test_expect_success $PREREQ '8-bit and sendemail.transferencoding=base64' '
+       clean_fake_sendmail &&
+       git send-email \
+         --transfer-encoding=base64 \
+         --smtp-server="$(pwd)/fake.sendmail" \
+         email-using-8bit \
+         2>errors >out &&
+       sed '1,/^$/d' msgtxt1 >actual &&
+       test_cmp expected actual
+'
+
+test_expect_success $PREREQ 'setup expect' '
+cat >email-using-qp <<EOF
+From fe6ecc66ece37198fe5db91fa2fc41d9f4fe5cc4 Mon Sep 17 00:00:00 2001
+Message-Id: <bogus-message-id@example.com>
+From: A U Thor <author@example.com>
+Date: Sat, 12 Jun 2010 15:53:58 +0200
+MIME-Version: 1.0
+Content-Transfer-Encoding: quoted-printable
+Content-Type: text/plain; charset=UTF-8
+Subject: Nothing to see here.
+
+Dieser Betreff enth=C3=A4lt auch einen Umlaut!
+EOF
+'
+
+test_expect_success $PREREQ 'convert from quoted-printable to base64' '
+       clean_fake_sendmail &&
+       git send-email \
+         --transfer-encoding=base64 \
+         --smtp-server="$(pwd)/fake.sendmail" \
+         email-using-qp \
+         2>errors >out &&
+       sed '1,/^$/d' msgtxt1 >actual &&
+       test_cmp expected actual
+'
+
+test_expect_success $PREREQ 'setup expect' "
+tr -d '\\015' | tr '%' '\\015' > email-using-crlf <<EOF
+From fe6ecc66ece37198fe5db91fa2fc41d9f4fe5cc4 Mon Sep 17 00:00:00 2001
+Message-Id: <bogus-message-id@example.com>
+From: A U Thor <author@example.com>
+Date: Sat, 12 Jun 2010 15:53:58 +0200
+Content-Type: text/plain; charset=UTF-8
+Subject: Nothing to see here.
+
+Look, I have a CRLF and an = sign!%
+EOF
+"
+
+test_expect_success $PREREQ 'setup expect' '
+cat >expected <<EOF
+Look, I have a CRLF and an =3D sign!=0D
+EOF
+'
+
+test_expect_success $PREREQ 'CRLF and sendemail.transferencoding=quoted-printable' '
+       clean_fake_sendmail &&
+       git send-email \
+         --transfer-encoding=quoted-printable \
+         --smtp-server="$(pwd)/fake.sendmail" \
+         email-using-crlf \
+         2>errors >out &&
+       sed '1,/^$/d' msgtxt1 >actual &&
+       test_cmp expected actual
+'
+
+test_expect_success $PREREQ 'setup expect' '
+cat >expected <<EOF
+TG9vaywgSSBoYXZlIGEgQ1JMRiBhbmQgYW4gPSBzaWduIQ0K
+EOF
+'
+
+test_expect_success $PREREQ 'CRLF and sendemail.transferencoding=base64' '
+       clean_fake_sendmail &&
+       git send-email \
+         --transfer-encoding=base64 \
+         --smtp-server="$(pwd)/fake.sendmail" \
+         email-using-crlf \
+         2>errors >out &&
+       sed '1,/^$/d' msgtxt1 >actual &&
+       test_cmp expected actual
+'
+
+
 # Note that the patches in this test are deliberately out of order; we
 # want to make sure it works even if the cover-letter is not in the
 # first mail.