]> git.ipfire.org Git - thirdparty/Python/cpython.git/commit
gh-79638: Restore "Treat an unreachable robots.txt as disallow all" (GH-152525)
authorSerhiy Storchaka <storchaka@gmail.com>
Sun, 5 Jul 2026 11:13:01 +0000 (14:13 +0300)
committerGitHub <noreply@github.com>
Sun, 5 Jul 2026 11:13:01 +0000 (14:13 +0300)
commit70100b9ea0b43a95226eee45652a5735bb91e9c7
tree2734e13c6570908b3e4803dc47999a72415270e0
parent19b5e8e1f510eb2fdd3f6261006b61f8cf441101
gh-79638: Restore "Treat an unreachable robots.txt as disallow all" (GH-152525)

This change was accidentally reverted by f0daba1652c (gh-106693,
GH-149514), which only intended to revert the ob_sval change.
The tests were already restored by GH-149569.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Lib/urllib/robotparser.py
Misc/NEWS.d/next/Library/2025-09-05-20-50-35.gh-issue-79638.Y-JfaH.rst [new file with mode: 0644]