mbox: do not copy Content-Type into exported mbox

Daniel reports a patch + comment combination that breaks in
git am. The patch reports a Content-Type of US-ASCII, while
the comment adds a Ack with UTF-8 characters. The exported
mbox contains both the original Content-Type, and a UTF-8
Content-Type that we set. However, because the US-ASCII one
occurs later, git am honours it instead of ours, and chokes
on the UTF-8 characters.

Strip out any subsequent Content-Type:s. We normalise things
to UTF-8 and should not allow it to be overridden.

Add a test for this, based on the original report.

Reported-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Daniel Axtens <dja@axtens.net>

diff --git a/patchwork/tests/series/bugs-multiple-content-types.mbox b/patchwork/tests/series/bugs-multiple-content-types.mbox
new file mode 100644 (file)
index 0000000..f7006b4
--- /dev/null
+++ b/patchwork/tests/series/bugs-multiple-content-types.mbox
@@ -0,0 +1,172 @@
+From mboxrd@z Thu Jan  1 00:00:00 1970
+Return-Path: <SRS0=vqEd=WQ=vger.kernel.org=netdev-owner@kernel.org>
+X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
+       aws-us-west-2-korg-lkml-1.web.codeaurora.org
+X-Spam-Level: 
+X-Spam-Status: No, score=-9.7 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS,
+       INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,
+       URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no
+       version=3.4.0
+Received: from mail.kernel.org (mail.kernel.org [198.145.29.99])
+       by smtp.lore.kernel.org (Postfix) with ESMTP id A702DC3A5A2
+       for <netdev@archiver.kernel.org>; Tue, 20 Aug 2019 01:33:12 +0000 (UTC)
+Received: from vger.kernel.org (vger.kernel.org [209.132.180.67])
+       by mail.kernel.org (Postfix) with ESMTP id 8717B22DA7
+       for <netdev@archiver.kernel.org>; Tue, 20 Aug 2019 01:33:12 +0000 (UTC)
+Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
+        id S1728887AbfHTBdL (ORCPT <rfc822;netdev@archiver.kernel.org>);
+        Mon, 19 Aug 2019 21:33:11 -0400
+Received: from szxga05-in.huawei.com ([45.249.212.191]:4731 "EHLO huawei.com"
+        rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP
+        id S1728627AbfHTBdL (ORCPT <rfc822;netdev@vger.kernel.org>);
+        Mon, 19 Aug 2019 21:33:11 -0400
+Received: from DGGEMS411-HUB.china.huawei.com (unknown [172.30.72.59])
+        by Forcepoint Email with ESMTP id EF227A58CA1FC4ADCFA3;
+        Tue, 20 Aug 2019 09:33:03 +0800 (CST)
+Received: from localhost.localdomain.localdomain (10.175.113.25) by
+ DGGEMS411-HUB.china.huawei.com (10.3.19.211) with Microsoft SMTP Server id
+ 14.3.439.0; Tue, 20 Aug 2019 09:32:56 +0800
+From:   YueHaibing <yuehaibing@huawei.com>
+To:     <bjorn.topel@intel.com>, <magnus.karlsson@intel.com>,
+        <jonathan.lemon@gmail.com>, <ast@kernel.org>,
+        <daniel@iogearbox.net>, <kafai@fb.com>, <songliubraving@fb.com>,
+        <yhs@fb.com>, <john.fastabend@gmail.com>
+CC:     YueHaibing <yuehaibing@huawei.com>, <netdev@vger.kernel.org>,
+        <bpf@vger.kernel.org>, <kernel-janitors@vger.kernel.org>
+Subject: [PATCH -next] bpf: Use PTR_ERR_OR_ZERO in xsk_map_inc()
+Date:   Tue, 20 Aug 2019 01:36:52 +0000
+Message-ID: <20190820013652.147041-1-yuehaibing@huawei.com>
+X-Mailer: git-send-email 2.20.1
+MIME-Version: 1.0
+Content-Type:   text/plain; charset=US-ASCII
+Content-Transfer-Encoding: 7BIT
+X-Originating-IP: [10.175.113.25]
+X-CFilter-Loop: Reflected
+Sender: netdev-owner@vger.kernel.org
+Precedence: bulk
+List-ID: <netdev.vger.kernel.org>
+X-Mailing-List: netdev@vger.kernel.org
+Archived-At: <https://lore.kernel.org/netdev/20190820013652.147041-1-yuehaibing@huawei.com/>
+List-Archive: <https://lore.kernel.org/netdev/>
+List-Post: <mailto:netdev@vger.kernel.org>
+
+Use PTR_ERR_OR_ZERO rather than if(IS_ERR(...)) + PTR_ERR
+
+Signed-off-by: YueHaibing <yuehaibing@huawei.com>
+---
+ kernel/bpf/xskmap.c | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/kernel/bpf/xskmap.c b/kernel/bpf/xskmap.c
+index 4cc28e226398..942c662e2eed 100644
+--- a/kernel/bpf/xskmap.c
++++ b/kernel/bpf/xskmap.c
+@@ -21,7 +21,7 @@ int xsk_map_inc(struct xsk_map *map)
+       struct bpf_map *m = &map->map;
+ 
+       m = bpf_map_inc(m, false);
+-      return IS_ERR(m) ? PTR_ERR(m) : 0;
++      return PTR_ERR_OR_ZERO(m);
+ }
+ 
+ void xsk_map_put(struct xsk_map *map)
+
+
+From mboxrd@z Thu Jan  1 00:00:00 1970
+Return-Path: <SRS0=vqEd=WQ=vger.kernel.org=netdev-owner@kernel.org>
+X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
+       aws-us-west-2-korg-lkml-1.web.codeaurora.org
+X-Spam-Level: 
+X-Spam-Status: No, score=-8.2 required=3.0 tests=FROM_EXCESS_BASE64,
+       HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,
+       SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=unavailable
+       autolearn_force=no version=3.4.0
+Received: from mail.kernel.org (mail.kernel.org [198.145.29.99])
+       by smtp.lore.kernel.org (Postfix) with ESMTP id 02AB1C3A59E
+       for <netdev@archiver.kernel.org>; Tue, 20 Aug 2019 07:28:35 +0000 (UTC)
+Received: from vger.kernel.org (vger.kernel.org [209.132.180.67])
+       by mail.kernel.org (Postfix) with ESMTP id CC7942082F
+       for <netdev@archiver.kernel.org>; Tue, 20 Aug 2019 07:28:34 +0000 (UTC)
+Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
+        id S1729348AbfHTH2e (ORCPT <rfc822;netdev@archiver.kernel.org>);
+        Tue, 20 Aug 2019 03:28:34 -0400
+Received: from mga18.intel.com ([134.134.136.126]:58020 "EHLO mga18.intel.com"
+        rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
+        id S1729047AbfHTH2e (ORCPT <rfc822;netdev@vger.kernel.org>);
+        Tue, 20 Aug 2019 03:28:34 -0400
+X-Amp-Result: SKIPPED(no attachment in message)
+X-Amp-File-Uploaded: False
+Received: from orsmga007.jf.intel.com ([10.7.209.58])
+  by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 20 Aug 2019 00:28:33 -0700
+X-ExtLoop1: 1
+X-IronPort-AV: E=Sophos;i="5.64,408,1559545200"; 
+   d="scan'208";a="169001452"
+Received: from arappl-mobl2.ger.corp.intel.com (HELO btopel-mobl.ger.intel.com) ([10.252.53.140])
+  by orsmga007.jf.intel.com with ESMTP; 20 Aug 2019 00:28:27 -0700
+Subject: Re: [PATCH -next] bpf: Use PTR_ERR_OR_ZERO in xsk_map_inc()
+To:     YueHaibing <yuehaibing@huawei.com>, magnus.karlsson@intel.com,
+        jonathan.lemon@gmail.com, ast@kernel.org, daniel@iogearbox.net,
+        kafai@fb.com, songliubraving@fb.com, yhs@fb.com,
+        john.fastabend@gmail.com
+Cc:     netdev@vger.kernel.org, bpf@vger.kernel.org,
+        kernel-janitors@vger.kernel.org
+References: <20190820013652.147041-1-yuehaibing@huawei.com>
+From:   =?UTF-8?B?QmrDtnJuIFTDtnBlbA==?= <bjorn.topel@intel.com>
+Message-ID: <93fafdab-8fb3-0f2b-8f36-0cf297db3cd9@intel.com>
+Date:   Tue, 20 Aug 2019 09:28:26 +0200
+User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
+ Thunderbird/60.8.0
+MIME-Version: 1.0
+In-Reply-To: <20190820013652.147041-1-yuehaibing@huawei.com>
+Content-Type: text/plain; charset=utf-8; format=flowed
+Content-Language: en-US
+Content-Transfer-Encoding: 8bit
+Sender: netdev-owner@vger.kernel.org
+Precedence: bulk
+List-ID: <netdev.vger.kernel.org>
+X-Mailing-List: netdev@vger.kernel.org
+Archived-At: <https://lore.kernel.org/netdev/93fafdab-8fb3-0f2b-8f36-0cf297db3cd9@intel.com/>
+List-Archive: <https://lore.kernel.org/netdev/>
+List-Post: <mailto:netdev@vger.kernel.org>
+
+On 2019-08-20 03:36, YueHaibing wrote:
+> Use PTR_ERR_OR_ZERO rather than if(IS_ERR(...)) + PTR_ERR
+> 
+> Signed-off-by: YueHaibing <yuehaibing@huawei.com>
+> ---
+>   kernel/bpf/xskmap.c | 2 +-
+>   1 file changed, 1 insertion(+), 1 deletion(-)
+> 
+> diff --git a/kernel/bpf/xskmap.c b/kernel/bpf/xskmap.c
+> index 4cc28e226398..942c662e2eed 100644
+> --- a/kernel/bpf/xskmap.c
+> +++ b/kernel/bpf/xskmap.c
+> @@ -21,7 +21,7 @@ int xsk_map_inc(struct xsk_map *map)
+>      struct bpf_map *m = &map->map;
+>   
+>      m = bpf_map_inc(m, false);
+> -    return IS_ERR(m) ? PTR_ERR(m) : 0;
+> +    return PTR_ERR_OR_ZERO(m);
+>   }
+>   
+>   void xsk_map_put(struct xsk_map *map)
+>
+
+Acked-by: Björn Töpel <bjorn.topel@intel.com>
+
+Thanks for the patch!
+
+For future patches: Prefix AF_XDP socket work with "xsk:" and use "PATCH
+bpf-next" to let the developers know what tree you're aiming for.
+
+
+
+Cheers!
+Björn
+
+
+> 
+> 
+
+
+

diff --git a/patchwork/tests/test_series.py b/patchwork/tests/test_series.py
index 9b5c012916215190b6f3214a4441827453847785..515a5bd14ff1620abffee15b0f9cb296ca5cfeb0 100644 (file)
--- a/patchwork/tests/test_series.py
+++ b/patchwork/tests/test_series.py
@@ -25,6 +25,7 @@ from django.test import TestCase
 from patchwork import models
 from patchwork import parser
 from patchwork.tests import utils
+from patchwork.views.utils import patch_to_mbox
 
 
 TEST_SERIES_DIR = os.path.join(os.path.dirname(__file__), 'series')
@@ -273,6 +274,16 @@ class BaseSeriesTest(_BaseTestCase):
         self.assertSerialized(patches, [2])
         self.assertSerialized(covers, [1])
 
+    def test_multiple_content_types(self):
+        """Test what happens when a patch and comment have different
+        Content-Type headers."""
+
+        _, patches, _ = self._parse_mbox(
+            'bugs-multiple-content-types.mbox', [0, 1, 1])
+
+        patch = patches[0]
+        self.assertEqual(patch_to_mbox(patch).count('Content-Type:'), 1)
+
 
 class RevisedSeriesTest(_BaseTestCase):
     """Tests for a series plus a single revision.

diff --git a/patchwork/views/utils.py b/patchwork/views/utils.py
index fb0195ceb08c13f04e8896a791be9bae8a183187..7376bae8beca2b5723c058d7ec9b9eb31188bf9d 100644 (file)
--- a/patchwork/views/utils.py
+++ b/patchwork/views/utils.py
@@ -99,8 +99,8 @@ def _submission_to_mbox(submission):
 
     orig_headers = HeaderParser().parsestr(str(submission.headers))
     for key, val in orig_headers.items():
-        # we set this ourselves
-        if key == 'Content-Transfer-Encoding':
+        # we set these ourselves
+        if key in ['Content-Type', 'Content-Transfer-Encoding']:
             continue
         # we don't save GPG signatures described in RFC1847 [1] so this
         # Content-Type value is invalid