From: Richard Purdie Date: Sat, 18 Jan 2014 23:23:02 +0000 (+0000) Subject: siggen: Remove fsync that is causing performance issues X-Git-Url: http://git.ipfire.org/gitweb.cgi?a=commitdiff_plain;h=46fd841319479f6079d850b3813e64bd8c2680a3;p=thirdparty%2Fopenembedded%2Fopenembedded-core-contrib.git siggen: Remove fsync that is causing performance issues This fsync was added for belt and braces protection for things like sstate on NFS. To be honest, it probably doesn't buy much, if the rename isn't atomic, all bets are off anyway and there are bigger issues to worry about. The issue is that at the end of every task, the dump_sig() code is triggered to save out information about the task and this was triggering an fsync(fd). Whilst it may select the file descriptor, on file systems like ext4, it will require large parts of the journal to be written out so it can have significant impact. latencytop showed an average fsync() call overhead of about 2s and if that happens for 5000 tasks, the time mounts up. This blocks the next task execution by that time. We therefore drop the fsync since in reality its causing problems and is unlikely to buy much. Signed-off-by: Richard Purdie --- diff --git a/lib/bb/siggen.py b/lib/bb/siggen.py index bc96fd3b7a4..bb8203df053 100644 --- a/lib/bb/siggen.py +++ b/lib/bb/siggen.py @@ -238,7 +238,6 @@ class SignatureGeneratorBasic(SignatureGenerator): with os.fdopen(fd, "wb") as stream: p = pickle.dump(data, stream, -1) stream.flush() - os.fsync(fd) os.chmod(tmpfile, 0664) os.rename(tmpfile, sigfile) except (OSError, IOError) as err: