From: Niklas Hambüchen <mail@nh2.me>
Date: Mon, 27 Jan 2020 23:37:22 +0000 (+0100)
Subject: Fix unicode printing error on Python 2 without TTY.
X-Git-Tag: v2.9.0~18^2
X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=refs%2Fpull%2F691%2Fhead;p=thirdparty%2Fbabel.git

Fix unicode printing error on Python 2 without TTY.

Until now, on Python 2.7, `python setup.py test | cat` crashed in the test
runner with

    ======================================================================
    ERROR: test_abort_invalid_po_file (tests.messages.test_pofile.ReadPoTestCase)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "src/babel/tests/messages/test_pofile.py", line 458, in test_abort_invalid_po_file
        output = pofile.read_po(buf, locale='fr', abort_invalid=False)
      File "src/babel/babel/messages/pofile.py", line 377, in read_po
        parser.parse(fileobj)
      File "src/babel/babel/messages/pofile.py", line 310, in parse
        self._process_message_line(lineno, line)
      File "src/babel/babel/messages/pofile.py", line 210, in _process_message_line
        self._process_keyword_line(lineno, line, obsolete)
      File "src/babel/babel/messages/pofile.py", line 222, in _process_keyword_line
        self._invalid_pofile(line, lineno, "Start of line didn't match any expected keyword.")
      File "src/babel/babel/messages/pofile.py", line 325, in _invalid_pofile
        print(u"WARNING: Problem on line {0}: {1}".format(lineno + 1, line))
    UnicodeEncodeError: 'ascii' codec can't encode character u'\xe0' in position 84: ordinal not in range(128)

The test suite would show this when printing the `à` in the test pofile contents

    Pour toute question, veuillez communiquer avec Fulano  à nadie@blah.com

But this bug is not confined to the test suite only. Any call to `read_po()`
with invalid .po file could trigger it in non-test code when
`sys.stdout.encoding` is `None`, which is the default for Python 2 when
`sys.stdout.isatty()` is false (as induced e.g. by `| cat`).

The fix is to `repr()` the line when printing the WARNING.
---

diff --git a/babel/messages/pofile.py b/babel/messages/pofile.py
index f6771bed..b86dd405 100644
--- a/babel/messages/pofile.py
+++ b/babel/messages/pofile.py
@@ -323,7 +323,10 @@ class PoFileParser(object):
         if self.abort_invalid:
             raise PoFileError(msg, self.catalog, line, lineno)
         print("WARNING:", msg)
-        print(u"WARNING: Problem on line {0}: {1}".format(lineno + 1, line))
+        # `line` is guaranteed to be unicode so u"{}"-interpolating would always
+        # succeed, but on Python < 2 if not in a TTY, `sys.stdout.encoding`
+        # is `None`, unicode may not be printable so we `repr()` to ASCII.
+        print(u"WARNING: Problem on line {0}: {1}".format(lineno + 1, repr(line)))
 
 
 def read_po(fileobj, locale=None, domain=None, ignore_obsolete=False, charset=None, abort_invalid=False):