]> git.ipfire.org Git - thirdparty/git.git/commit
git-p4: improve encoding handling to support inconsistent encodings
authorTao Klerks <tao@klerks.biz>
Sat, 30 Apr 2022 19:26:52 +0000 (19:26 +0000)
committerJunio C Hamano <gitster@pobox.com>
Wed, 4 May 2022 17:30:01 +0000 (10:30 -0700)
commitf7b5ff607fa1a62a480399afb6ccb9691735bf79
treefbf3bc4cda3b6e879c0d2652c6ac32e96767d447
parent6cd33dceed60949e2dbc32e3f0f5e67c4c882e1e
git-p4: improve encoding handling to support inconsistent encodings

git-p4 is designed to run correctly under python2.7 and python3, but
its functional behavior wrt importing user-entered text differs across
these environments:

Under python2, git-p4 "naively" writes the Perforce bytestream into git
metadata (and does not set an "encoding" header on the commits); this
means that any non-utf-8 byte sequences end up creating invalidly-encoded
commit metadata in git.

Under python3, git-p4 attempts to decode the Perforce bytestream as utf-8
data, and fails badly (with an unhelpful error) when non-utf-8 data is
encountered.

Perforce clients (especially p4v) encourage user entry of changelist
descriptions (and user full names) in OS-local encoding, and store the
resulting bytestream to the server unmodified - such that different
clients can end up creating mutually-unintelligible messages. The most
common inconsistency, in many Perforce environments, is likely to be utf-8
(typical in linux) vs cp-1252 (typical in windows).

Make the changelist-description- and user-fullname-handling code
python-runtime-agnostic, introducing three "strategies" selectable via
config:
- 'passthrough', behaving as previously under python2,
- 'strict', behaving as previously under python3, and
- 'fallback', favoring utf-8 but supporting a secondary encoding when
utf-8 decoding fails, and finally escaping high-range bytes if the
decoding with the secondary encoding also fails.

Keep the python2 default behavior as-is ('legacy' strategy), but switch
the python3 default strategy to 'fallback' with default fallback encoding
'cp1252'.

Also include tests exercising these encoding strategies, documentation for
the new config, and improve the user-facing error messages when decoding
does fail.

Signed-off-by: Tao Klerks <tao@klerks.biz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Documentation/git-p4.txt
git-p4.py
t/lib-git-p4.sh
t/t9835-git-p4-metadata-encoding-python2.sh [new file with mode: 0755]
t/t9836-git-p4-metadata-encoding-python3.sh [new file with mode: 0755]