]> git.ipfire.org Git - thirdparty/Python/cpython.git/commit
gh-129005: Align FileIO.readall between _pyio and _io (#129705)
authorCody Maloney <cmaloney@users.noreply.github.com>
Fri, 7 Feb 2025 11:06:11 +0000 (03:06 -0800)
committerGitHub <noreply@github.com>
Fri, 7 Feb 2025 11:06:11 +0000 (12:06 +0100)
commita3d5aab9a89e311cded9c724ce7d5a873e4d680d
treea9414f49045a51f457a3ea93e4841d4a65e8212f
parentae132edc296d27c6ed04fe4d400c67e3cfb622e8
gh-129005: Align FileIO.readall between _pyio and _io (#129705)

Utilize `bytearray.resize()` and `os.readinto()` to reduce copies
and match behavior of `_io.FileIO.readall()`.

There is still an extra copy which means twice the memory required
compared to FileIO because there isn't a zero-copy  path from
`bytearray` -> `bytes` currently.

On my system reading a 2 GB file:
`./python -m test -M8g -uall test_largefile -m test.test_largefile.PyLargeFileTest.test_large_read -v`

Goes from ~2.7 seconds -> ~2.2 seconds

Co-authored-by: Victor Stinner <vstinner@python.org>
Lib/_pyio.py
Misc/NEWS.d/next/Library/2025-02-05-13-19-15.gh-issue-129005.Sb69L_.rst [new file with mode: 0644]