lib/sstatesig: use the SHA256 fastpath in bitbake/utils.py

Instead of iterating through a file by hand to hash it, call into the
hashing code in bitbake's utils.py.  This uses mmap() instead of plain
file operations which is ~30% faster in benchmarks[1], and will speed
up hash calculations when writing sstate.

[1] On my build machine, hashing a 300MB kernel image 100 times takes
    34s with open() and 26s with mmap()

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

diff --git a/meta/lib/oe/sstatesig.py b/meta/lib/oe/sstatesig.py
index af6df60271da85b3d5689a5ab65b8e38ca95d596..f016fc677040e562d8d6f3e4a268abec56e1eb95 100644 (file)
--- a/meta/lib/oe/sstatesig.py
+++ b/meta/lib/oe/sstatesig.py
@@ -684,10 +684,10 @@ def OEOuthashBasic(path, sigfile, task, d):
                     update_hash(" " * 10)
 
                 update_hash(" ")
-                fh = hashlib.sha256()
                 if stat.S_ISREG(s.st_mode):
                     # Hash file contents
                     if filterfile:
+                        fh = hashlib.sha256()
                         # Need to ignore paths in crossscripts and postinst-useradd files.
                         with open(path, 'rb') as d:
                             chunk = d.read()
@@ -701,13 +701,13 @@ def OEOuthashBasic(path, sigfile, task, d):
                                     else:
                                         chunk = chunk.replace(bytes(r, encoding='utf8'), b'')
                             fh.update(chunk)
+                        update_hash(fh.hexdigest())
                     else:
-                        with open(path, 'rb') as d:
-                            for chunk in iter(lambda: d.read(4096), b""):
-                                fh.update(chunk)
-                    update_hash(fh.hexdigest())
+                        # Plain file that we're not filtering, use the fastpath in bb.utils
+                        update_hash(bb.utils.sha256_file(path))
                 else:
-                    update_hash(" " * len(fh.hexdigest()))
+                    # SHA256 has a 64 character hex digest
+                    update_hash(" " * 64)
 
                 update_hash(" %s" % path)