From: Miss Islington (bot) <31488909+miss-islington@users.noreply.github.com>
Date: Fri, 24 May 2024 14:52:20 +0000 (+0200)
Subject: [3.13] GH-119496: accept UTF-8 BOM in .pth files (GH-119508)
X-Git-Tag: v3.13.0b2~123
X-Git-Url: http://git.ipfire.org/gitweb.cgi?a=commitdiff_plain;h=217d57fc3c9a8ec45dfccd3aab9a05dbf6656da0;p=thirdparty%2FPython%2Fcpython.git

[3.13] GH-119496: accept UTF-8 BOM in .pth files (GH-119508)

`Out-File -Encoding utf8` and similar commands in Windows Powershell 5.1 emit
UTF-8 with a BOM marker, which the regular `utf-8` codec decodes incorrectly.

`utf-8-sig` accepts a BOM, but also works correctly without one.

This change also makes .pth files match the way Python source files are handled.

(cherry picked from commit bf5b6467f8cc06759f3396ab1a8ad64fe7d1db2e)

Co-authored-by: Alyssa Coghlan <ncoghlan@gmail.com>
Co-authored-by: Inada Naoki <songofacandy@gmail.com>
---

diff --git a/Lib/site.py b/Lib/site.py
index f1a6d9cf66fd..7eace190f5ab 100644
--- a/Lib/site.py
+++ b/Lib/site.py
@@ -185,7 +185,9 @@ def addpackage(sitedir, name, known_paths):
         return
 
     try:
-        pth_content = pth_content.decode()
+        # Accept BOM markers in .pth files as we do in source files
+        # (Windows PowerShell 5.1 makes it hard to emit UTF-8 files without a BOM)
+        pth_content = pth_content.decode("utf-8-sig")
     except UnicodeDecodeError:
         # Fallback to locale encoding for backward compatibility.
         # We will deprecate this fallback in the future.