]> git.ipfire.org Git - thirdparty/Python/cpython.git/commit
gh-63161: Fix tokenize.detect_encoding() (GH-139446)
authorSerhiy Storchaka <storchaka@gmail.com>
Mon, 20 Oct 2025 17:08:47 +0000 (20:08 +0300)
committerGitHub <noreply@github.com>
Mon, 20 Oct 2025 17:08:47 +0000 (20:08 +0300)
commit38d4b436ca767351db834189b3a5379406cd52a8
tree8bb7da5ea2edf04b34d51218fd70a37850bb0d2b
parent3222ea0f1444cac0733f89b36f1d48ae67a08ffb
gh-63161: Fix tokenize.detect_encoding() (GH-139446)

* Support non-UTF-8 shebang and comments if non-UTF-8 encoding is specified.
* Detect decoding error for non-UTF-8 encoding.
* Detect null bytes in source code.
Lib/test/test_tokenize.py
Lib/tokenize.py
Misc/NEWS.d/next/Library/2025-09-30-12-52-54.gh-issue-63161.mECM1A.rst [new file with mode: 0644]