REST: fix patch listing query

The patch listing query is punishingly slow under even very simple
filters.

The new data model in 3.0 will help _a lot_, so this is a simple fix: I
did try indexes but haven't got really deeply into the weeds of what we can do
with them.

Move a number of things from select_related to prefetch_related: we trade off
one big, inefficient query for a slightly larger number of significantly more
efficient queries.

On my laptop with 2 copies of the canonical kernel team list loaded into the
database, and considering only the API view (the JSON view is always faster)
with warm caches and considering the entire set of SQL queries:

 - /api/patches/?project=1
    ~1.4-1.5s -> <100ms, something like 14x better

 - /api/patches/?project=1&since=2010-11-01T00:00:00
    ~1.7-1.8s -> <560ms, something like 3x better (now dominated by the
                         counting query only invoked on the HTML API view,
                         not the pure JSON API view.)

The things I moved:

 * project: this was generating SQL that looked like:

   INNER JOIN `patchwork_project` T5
    ON (`patchwork_submission`.`project_id` = T5.`id`)

   This is correct but we've already had to join the patchwork_submission
   table and perhaps as a result it seems to be inefficient.

 * series__project: Likewise we've already had to join the series table,
   doing another join is possibly why it is inefficient.

 * delegate: I do not know why this was tanking performance. I think it
   might relate to the strategy mysql was using.

Reported-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Signed-off-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Stephen Finucane <stephen@that.guru>
(backported from commit 98a2d051372dcedb889c4cb94ebd8ed7b399b522
 - dropped tests, it depends on a test we don't carry
 - rejigged to suit old M:N series model)
Signed-off-by: Daniel Axtens <dja@axtens.net>

diff --git a/patchwork/api/patch.py b/patchwork/api/patch.py
index c55bf9c919dcd41f06ad45071f3a58f96c0908cf..f593316d98f554e15613da08e41f3ed06877b10b 100644 (file)
--- a/patchwork/api/patch.py
+++ b/patchwork/api/patch.py
@@ -187,9 +187,9 @@ class PatchList(ListAPIView):
 
     def get_queryset(self):
         return Patch.objects.all()\
-            .prefetch_related('series', 'check_set', 'series__project')\
-            .select_related('project', 'state', 'submitter', 'delegate',
-                            )\
+            .prefetch_related('series', 'check_set', 'project',
+                              'delegate', 'series__project')\
+            .select_related('state', 'submitter')\
             .defer('content', 'diff', 'headers')