From: drh Date: Sat, 18 Jan 2003 22:01:06 +0000 (+0000) Subject: Update the speed comparison page. (CVS 840) X-Git-Tag: version-3.6.10~5233 X-Git-Url: http://git.ipfire.org/?a=commitdiff_plain;h=79104c9d2a4a336c8b31728cfa050d12ae0adfad;p=thirdparty%2Fsqlite.git Update the speed comparison page. (CVS 840) FossilOrigin-Name: 424cb2edb0c51b911791422ce7a9f5284a57f9ce --- diff --git a/manifest b/manifest index 72236041eb..a52e81abd0 100644 --- a/manifest +++ b/manifest @@ -1,5 +1,5 @@ -C Fix\sdatatype\sreporting\sand\scollating\ssequence\sselection\sso\sthat\sit\sworks\ncorrectly\son\sviews\sand\swith\sthe\sUNION,\sEXCEPT,\sand\sINTERCEPT\soperators.\s(CVS\s839) -D 2003-01-18T20:11:06 +C Update\sthe\sspeed\scomparison\spage.\s(CVS\s840) +D 2003-01-18T22:01:07 F Makefile.in 6606854b1512f185b8e8c779b8d7fc2750463d64 F Makefile.linux-gcc b86a99c493a5bfb402d1d9178dcdc4bd4b32f906 F README f1de682fbbd94899d50aca13d387d1b3fd3be2dd @@ -143,18 +143,18 @@ F www/dynload.tcl 02eb8273aa78cfa9070dd4501dca937fb22b466c F www/faq.tcl 06276ff6c3e369374bb83034cc9d4a7d3a2a34a1 F www/fileformat.tcl a4b5c2c6e89b7d42d09f97fd4d7bbd39cbf24936 F www/formatchng.tcl b4449e065d2da38b6563bdf12cf46cfe1d4d765e -F www/index.tcl 374d30b92a753428baed933623fb9f4259fc5a5c +F www/index.tcl b5265ca54a5124ec40bffb7c7943e072e074d61a F www/lang.tcl 1ea38a9fe867e4fed2220f519f5c0862e7022c53 F www/mingw.tcl f1c7c0a7f53387dd9bb4f8c7e8571b7561510ebc F www/nulls.tcl 29497dac2bc5b437aa7e2e94577dad4d8933ed26 F www/omitted.tcl 118062f40a203fcb88b8d68ef1d7c0073ac191ec F www/opcode.tcl 33c5f2061a05c5d227c72b84c080b3bf74c74f8b F www/quickstart.tcl fde79aa2de20074842b60f780800cdeee6a5dec2 -F www/speed.tcl a20a792738475b68756ea7a19321600f23d1d803 +F www/speed.tcl 52759968401d81760fc01f9d3ab6242f6d2a7066 F www/sqlite.tcl ae3dcfb077e53833b59d4fcc94d8a12c50a44098 F www/tclsqlite.tcl 1db15abeb446aad0caf0b95b8b9579720e4ea331 F www/vdbe.tcl 2013852c27a02a091d39a766bc87cff329f21218 -P 32a8e6e9771d636c0ad3042632d35865bc08585b -R 04e23a1126d9810e8620977dfdd8daac +P 71cc292dce59cf8224b205d1cdbff59ad12f1043 +R 843340c3fef7e6ee0ac6b3187554a951 U drh -Z d5604eb0c9b6db0f87d9ac88ab2eea6a +Z 66212ef777c3998af30d067a457e17bb diff --git a/manifest.uuid b/manifest.uuid index 8cd846a518..49b0a5ce51 100644 --- a/manifest.uuid +++ b/manifest.uuid @@ -1 +1 @@ -71cc292dce59cf8224b205d1cdbff59ad12f1043 \ No newline at end of file +424cb2edb0c51b911791422ce7a9f5284a57f9ce \ No newline at end of file diff --git a/www/index.tcl b/www/index.tcl index ae1402cf27..03dbd82fa4 100644 --- a/www/index.tcl +++ b/www/index.tcl @@ -1,7 +1,7 @@ # # Run this TCL script to generate HTML for the index.html file. # -set rcsid {$Id: index.tcl,v 1.71 2003/01/18 20:11:07 drh Exp $} +set rcsid {$Id: index.tcl,v 1.72 2003/01/18 22:01:07 drh Exp $} puts { SQLite: An Embeddable SQL Database Engine @@ -57,8 +57,9 @@ puts {

Features

different byte orders.
  • Supports databases up to 2 terabytes (2^41 bytes) in size.
  • Small memory footprint: less than 25K lines of C code.
    • -
  • Four times faster than PostgreSQL. - Twice as fast as SQLite 1.0.
    • +
  • 10x faster than PostgreSQL and + 2x faster than MySQL for many common + operations.
  • Very simple C/C++ interface requires the use of only three functions and one opaque structure.
    • diff --git a/www/speed.tcl b/www/speed.tcl index 3d3e9c7ec4..e71c8dd84f 100644 --- a/www/speed.tcl +++ b/www/speed.tcl @@ -1,7 +1,7 @@ # # Run this Tcl script to generate the speed.html file. # -set rcsid {$Id: speed.tcl,v 1.8 2002/08/24 18:24:58 drh Exp $ } +set rcsid {$Id: speed.tcl,v 1.9 2003/01/18 22:01:07 drh Exp $ } puts { @@ -19,25 +19,42 @@ puts {

    Executive Summary

    A series of tests were run to measure the relative performance of -SQLite 2.7.0, PostgreSQL 7.1.3, and MySQL 3.23.41. +SQLite 2.7.6, PostgreSQL 7.1.3, and MySQL 3.23.41. The following are general conclusions drawn from these experiments:

    + +

    +The results presented here come with the following caveats: +

    + +

    Test Environment

    @@ -110,10 +127,10 @@ INSERT INTO t1 VALUES(999,24322,'twenty four thousand three hundred twenty two') INSERT INTO t1 VALUES(1000,94142,'ninety four thousand one hundred forty two');
    - - - - + + + +
    PostgreSQL:   3.613
    MySQL:   0.086
    SQLite 2.7.0:   8.672
    SQLite 2.7.0 (nosync):   0.286
    PostgreSQL:   3.658
    MySQL:   0.109
    SQLite 2.7.6:   7.177
    SQLite 2.7.6 (nosync):   0.266

    SQLite must close and reopen the database file, and thus invalidate @@ -122,6 +139,7 @@ version of SQLite is still nearly as fast as MySQL. Notice how much slower the synchronous version is, however. This is due to the necessity of calling fsync() after each SQL statement.

    +

    Test 2: 25000 INSERTs in a transaction

    BEGIN;
    @@ -133,10 +151,10 @@ INSERT INTO t2 VALUES(25000,473330,'four hundred seventy three thousand three hu COMMIT;
    - - - - + + + +
    PostgreSQL:   4.430
    MySQL:   2.025
    SQLite 2.7.0:   0.885
    SQLite 2.7.0 (nosync):   0.753
    PostgreSQL:   5.058
    MySQL:   2.271
    SQLite 2.7.6:   0.912
    SQLite 2.7.6 (nosync):   0.798

    @@ -148,62 +166,66 @@ this way, SQLite is much faster than either PostgreSQL and MySQL.

    Test 3: 100 SELECTs without an index

    +BEGIN;
    SELECT count(*), avg(b) FROM t2 WHERE b>=0 AND b<1000;
    SELECT count(*), avg(b) FROM t2 WHERE b>=100 AND b<1100;
    -SELECT count(*), avg(b) FROM t2 WHERE b>=200 AND b<1200;
    -... 94 lines omitted
    -SELECT count(*), avg(b) FROM t2 WHERE b>=9700 AND b<10700;
    +... 96 lines omitted
    SELECT count(*), avg(b) FROM t2 WHERE b>=9800 AND b<10800;
    SELECT count(*), avg(b) FROM t2 WHERE b>=9900 AND b<10900;
    +COMMIT;
    - - - - + + + +
    PostgreSQL:   3.274
    MySQL:   2.624
    SQLite 2.7.0:   5.585
    SQLite 2.7.0 (nosync):   5.443
    PostgreSQL:   3.657
    MySQL:   3.368
    SQLite 2.7.6:   4.386
    SQLite 2.7.6 (nosync):   4.314

    This test does 100 queries on a 25000 entry table without an index, -thus requiring a full table scan. SQLite is about half the speed of -PostgreSQL and MySQL. This is because SQLite stores all data as strings -and must therefore call strtod() 5 million times in the +thus requiring a full table scan. SQLite is about 20% or 30% slower +than PostgreSQL and MySQL. The reason for this is believed to be +because SQLite stores all data as strings +and must therefore do 5 million string-to-number conversions in the course of evaluating the WHERE clauses. Both PostgreSQL and MySQL store data as binary values where appropriate and can forego this conversion effort.

    +

    Test 4: 100 SELECTs on a string comparison

    +BEGIN;
    SELECT count(*), avg(b) FROM t2 WHERE c LIKE '%one%';
    SELECT count(*), avg(b) FROM t2 WHERE c LIKE '%two%';
    -SELECT count(*), avg(b) FROM t2 WHERE c LIKE '%three%';
    -... 94 lines omitted
    -SELECT count(*), avg(b) FROM t2 WHERE c LIKE '%ninety eight%';
    +... 96 lines omitted
    SELECT count(*), avg(b) FROM t2 WHERE c LIKE '%ninety nine%';
    SELECT count(*), avg(b) FROM t2 WHERE c LIKE '%one hundred%';
    +COMMIT;
    - - - - + + + +
    PostgreSQL:   14.511
    MySQL:   4.616
    SQLite 2.7.0:   5.966
    SQLite 2.7.0 (nosync):   5.918
    PostgreSQL:   15.967
    MySQL:   5.088
    SQLite 2.7.6:   5.419
    SQLite 2.7.6 (nosync):   5.367

    -This set of 100 queries uses string comparisons instead of -numerical comparisions. As a result, the speed of SQLite is -compariable to or better then PostgreSQL and MySQL. +This test still does 100 full table scans but it uses +uses string comparisons instead of numerical comparisions. +SQLite is almost three times faster than PostgreSQL here. But it is +still 15% slower than MySQL. MySQL appears to be very good +at doing full table scans.

    Test 5: Creating an index

    CREATE INDEX i2a ON t2(a);
    CREATE INDEX i2b ON t2(b);
    - - - - + + + +
    PostgreSQL:   0.483
    MySQL:   0.304
    SQLite 2.7.0:   0.779
    SQLite 2.7.0 (nosync):   0.637
    PostgreSQL:   0.431
    MySQL:   0.340
    SQLite 2.7.6:   0.814
    SQLite 2.7.6 (nosync):   0.675

    @@ -223,15 +245,15 @@ SELECT count(*), avg(b) FROM t2 WHERE b>=499800 AND b<499900;
    SELECT count(*), avg(b) FROM t2 WHERE b>=499900 AND b<500000;
    - - - - + + + +
    PostgreSQL:   4.939
    MySQL:   1.335
    SQLite 2.7.0:   1.165
    SQLite 2.7.0 (nosync):   1.144
    PostgreSQL:   5.369
    MySQL:   1.489
    SQLite 2.7.6:   1.423
    SQLite 2.7.6 (nosync):   1.358

    This test runs a set of 5000 queries that are similar in form to -those in test 3. But now instead of being half as fast, SQLite +those in test 3. But now instead of being slower, SQLite is faster than both PostgreSQL and MySQL.

    @@ -246,15 +268,18 @@ UPDATE t1 SET b=b*2 WHERE a>=9990 AND a<10000;
    COMMIT;
    - - - - + + + +
    PostgreSQL:   1.536
    MySQL:   7.281
    SQLite 2.7.0:   0.817
    SQLite 2.7.0 (nosync):   0.726
    PostgreSQL:   1.740
    MySQL:   8.162
    SQLite 2.7.6:   0.635
    SQLite 2.7.6 (nosync):   0.608

    -Here is a case where MySQL is over 10 times slower than SQLite. -The reason for this is unclear. +For this particular UPDATE test, MySQL is consistently +five or ten times +slower than PostgreSQL and SQLite. I do not know why. MySQL is +normally a very fast engine. Perhaps this problem has been addressed +in later versions of MySQL.

    Test 8: 25000 UPDATEs with an index

    @@ -268,16 +293,16 @@ UPDATE t2 SET b=423958 WHERE a=25000;
    COMMIT;
    - - - - + + + +
    PostgreSQL:   29.318
    MySQL:   7.514
    SQLite 2.7.0:   7.681
    SQLite 2.7.0 (nosync):   7.852
    PostgreSQL:   32.118
    MySQL:   8.132
    SQLite 2.7.6:   4.109
    SQLite 2.7.6 (nosync):   3.712

    -In this case MySQL is slightly faster than SQLite, though not by much. -The difference is believed to have to do with the fact SQLite -handles the integers as strings instead of binary numbers. +As recently as version 2.7.0, SQLite ran at about the same speed as +MySQL on this test. But recent optimizations to SQLite have doubled +speed of UPDATEs.

    Test 9: 25000 text UPDATEs with an index

    @@ -291,72 +316,83 @@ UPDATE t2 SET c='three hundred forty seven thousand three hundred ninety three' COMMIT;
    - - - - + + + +
    PostgreSQL:   50.020
    MySQL:   5.841
    SQLite 2.7.0:   5.346
    SQLite 2.7.0 (nosync):   5.393
    PostgreSQL:   55.309
    MySQL:   6.585
    SQLite 2.7.6:   2.474
    SQLite 2.7.6 (nosync):   1.800

    -When updating a text field instead of an integer field, -SQLite is slightly faster than MySQL. +Here again, version 2.7.0 of SQLite used to run at about the same speed +as MySQL. But now version 2.7.6 is over two times faster than MySQL and +over twenty times faster than PostgreSQL.

    Test 10: INSERTs from a SELECT

    -BEGIN;
    INSERT INTO t1 SELECT * FROM t2;
    INSERT INTO t2 SELECT * FROM t1;
    COMMIT; +BEGIN;
    INSERT INTO t1 SELECT b,a,c FROM t2;
    INSERT INTO t2 SELECT b,a,c FROM t1;
    COMMIT;
    - - - - + + + +
    PostgreSQL:   57.834
    MySQL:   1.335
    SQLite 2.7.0:   5.073
    SQLite 2.7.0 (nosync):   2.085
    PostgreSQL:   58.956
    MySQL:   1.465
    SQLite 2.7.6:   2.926
    SQLite 2.7.6 (nosync):   1.664

    The poor performance of PostgreSQL in this case appears to be due to its synchronous behavior. The CPU was mostly idle the test run. Presumably, PostgreSQL was spending most of its time waiting on disk I/O to complete. -

    - -

    -SQLite is slower than MySQL because it creates a temporary table to store -the result of the query, then does an insert from the temporary table. -A future enhancement that moves data directly from teh query into the -insert table should double the speed of SQLite. +I'm not sure why SQLite performs poorly here. It use to be quicker at this +test, but the same enhancements that sped up the UPDATE logic seem to have +slowed down this test.

    Test 11: DELETE without an index

    DELETE FROM t2 WHERE c LIKE '%fifty%';
    - - - - + + + +
    PostgreSQL:   0.733
    MySQL:   0.768
    SQLite 2.7.0:   5.418
    SQLite 2.7.0 (nosync):   0.668
    PostgreSQL:   1.365
    MySQL:   0.849
    SQLite 2.7.6:   4.005
    SQLite 2.7.6 (nosync):   0.631
    +

    +The synchronous version of SQLite is the slowest of the group in this test, +but the asynchronous version is the fastest. SQLite used about the same +amount of CPU time in both versions; the difference is the extra time needed +to write information to the disk surface. +

    +

    Test 12: DELETE with an index

    DELETE FROM t2 WHERE a>10 AND a<20000;
    - - - - + + + +
    PostgreSQL:   0.867
    MySQL:   2.068
    SQLite 2.7.0:   1.453
    SQLite 2.7.0 (nosync):   0.745
    PostgreSQL:   1.340
    MySQL:   2.167
    SQLite 2.7.6:   2.344
    SQLite 2.7.6 (nosync):   0.858
    +

    +This test is significant because it is one of the few where +PostgreSQL is faster than MySQL. The asynchronous SQLite is, +however, faster then both the other two. +

    + +

    Test 13: A big INSERT after a big DELETE

    INSERT INTO t2 SELECT * FROM t1;
    - - - - + + + +
    PostgreSQL:   66.099
    MySQL:   1.663
    SQLite 2.7.0:   4.029
    SQLite 2.7.0 (nosync):   1.729
    PostgreSQL:   12.672
    MySQL:   1.837
    SQLite 2.7.6:   3.076
    SQLite 2.7.6 (nosync):   1.570

    -Earlier versions of SQLite would show decreasing performance after a +Some older versions of SQLite would show decreasing performance after a sequence DELETEs followed by new INSERTs. As this test shows, the problem has now been resolved.

    @@ -366,32 +402,44 @@ problem has now been resolved. BEGIN;
    DELETE FROM t1;
    INSERT INTO t1 VALUES(1,29676,'twenty nine thousand six hundred seventy six');
    -... 2997 lines omitted
    -INSERT INTO t1 VALUES(2999,37835,'thirty seven thousand eight hundred thirty five');
    -INSERT INTO t1 VALUES(3000,97817,'ninety seven thousand eight hundred seventeen');
    +... 11997 lines omitted
    +INSERT INTO t1 VALUES(11999,71818,'seventy one thousand eight hundred eighteen');
    +INSERT INTO t1 VALUES(12000,58579,'fifty eight thousand five hundred seventy nine');
    COMMIT;
    - - - - + + + +
    PostgreSQL:   1.168
    MySQL:   0.866
    SQLite 2.7.0:   0.288
    SQLite 2.7.0 (nosync):   0.155
    PostgreSQL:   4.165
    MySQL:   1.733
    SQLite 2.7.6:   0.652
    SQLite 2.7.6 (nosync):   0.465
    +

    +SQLite is very good at doing INSERTs within a transaction, which probably +explains why it is so much faster than the other databases at this test. +

    +

    Test 15: DROP TABLE

    DROP TABLE t1;
    DROP TABLE t2;
    - - - - + + + +
    PostgreSQL:   0.100
    MySQL:   0.012
    SQLite 2.7.0:   0.572
    SQLite 2.7.0 (nosync):   0.168
    PostgreSQL:   0.133
    MySQL:   0.014
    SQLite 2.7.6:   0.873
    SQLite 2.7.6 (nosync):   0.224

    SQLite is slower than the other databases when it comes to dropping tables. -This is not seen as a big problem, however, since DROP TABLE is seldom -used in speed-critical situations. +This probably is because when SQLite drops a table, it has to go through and +erase the records in the database file that deal with that table. MySQL and +PostgreSQL, on the other hand, use separate files to represent each table +so they can drop a table simply by deleting a file, which is much faster. +

    + +

    +On the other hand, dropping tables is not a very common operation +so if SQLite takes a little longer, that is not seen as a big problem.

    }