]> git.ipfire.org Git - thirdparty/gcc.git/commit - contrib/compare_two_ftime_report_sets
Add a contrib script for comparing the performance of two sets of
authorLawrence Crowl <crowl@google.com>
Wed, 7 Nov 2012 00:36:04 +0000 (00:36 +0000)
committerLawrence Crowl <crowl@gcc.gnu.org>
Wed, 7 Nov 2012 00:36:04 +0000 (00:36 +0000)
commitbff0e5295af23c7947f7b4804a72e86aeca0ce56
tree8ab0c916dd01a2cfac2710a91391bbc30eaeb98e
parent8f7a6877dbcd89fd253fd7a6e9ffdb2c7949063e
Add a contrib script for comparing the performance of two sets of
compiler runs.

Usage documentation is in the script.

The script produces output of the form:

$ compare_two_ftime_report_sets "Log0/*perf" "Log3/*perf"

Arithmetic sample for timevar log files
"Log0/*perf"
and selecting lines containing "TOTAL" with desired confidence 95 is
trial count is 4, mean is 443.022 (95% confidence in 440.234 to 445.811),
std.deviation is 1.75264, std.error is 0.876322

Arithmetic sample for timevar log files
"Log3/*perf"
and selecting lines containing "TOTAL" with desired confidence 95 is
trial count is 4, mean is 441.302 (95% confidence in 436.671 to 445.934),
std.deviation is 2.91098, std.error is 1.45549

The first sample appears to be 0.39% larger,
with 60% confidence of being larger.
To reach 95% confidence, you need roughly 14 trials,
assuming the standard deviation is stable, which is iffy.

Tested on x86_64 builds.

Index: contrib/ChangeLog

2012-11-05  Lawrence Crowl  <crowl@google.com>

* compare_two_ftime_report_sets: New.

From-SVN: r193277
contrib/ChangeLog
contrib/compare_two_ftime_report_sets [new file with mode: 0755]