From: Stewart Smith <stewart@linux.ibm.com>
Date: Fri, 10 Aug 2018 08:00:59 +0000 (+1000)
Subject: Add covering index for /list/ query
X-Git-Tag: v2.2.0-rc1~297
X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=102c1571c7026798813b6ce274941fd578f39613;p=thirdparty%2Fpatchwork.git

Add covering index for /list/ query

In constructing the list of patches for a project, there are two
main queries that are executed:
1) get a count() of how many patches there are
2) Get the page of results being displayed

In a test dataset of ~11500 LKML patches and ~4000 others, the existing
code would take around 585ms and 858ms with a cold cache and 28ms and
198ms for a warm cache.

By adding a covering index, we get down to 4ms and 255ms for a cold
cache, and 4ms and 143ms for a warm cache!

Additionally, when there's a lot of archived or accepted patches
(I used ~11000 archived out of the 15000 total in my test set)
the query time goes from 28ms and 72ms down to 2ms and 33-40ms!

Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
[stephenfin: Regenerate migrations per addition of 0027 in earlier
 patch]
Signed-off-by: Stephen Finucane <stephen@that.guru>
---

diff --git a/patchwork/migrations/0029_add_list_covering_index.py b/patchwork/migrations/0029_add_list_covering_index.py
new file mode 100644
index 00000000..8f26b265
--- /dev/null
+++ b/patchwork/migrations/0029_add_list_covering_index.py
@@ -0,0 +1,19 @@
+# -*- coding: utf-8 -*-
+# Generated by Django 1.11.15 on 2018-08-31 23:46
+from __future__ import unicode_literals
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('patchwork', '0028_add_comment_date_index'),
+    ]
+
+    operations = [
+        migrations.AddIndex(
+            model_name='patch',
+            index=models.Index(fields=['archived', 'patch_project', 'state', 'delegate'], name='patch_list_covering_idx'),
+        ),
+    ]
diff --git a/patchwork/models.py b/patchwork/models.py
index fea71c87..0ec5681e 100644
--- a/patchwork/models.py
+++ b/patchwork/models.py
@@ -591,6 +591,13 @@ class Patch(SeriesMixin, Submission):
         verbose_name_plural = 'Patches'
         base_manager_name = 'objects'
 
+        indexes = [
+            # This is a covering index for the /list/ query
+            models.Index(fields=['archived', 'patch_project', 'state',
+                                 'delegate'],
+                         name='patch_list_covering_idx'),
+        ]
+
 
 class Comment(EmailMixin, models.Model):
     # parent