From: Daniel Axtens Date: Tue, 17 Mar 2020 13:59:14 +0000 (+1100) Subject: REST: fix patch listing query X-Git-Tag: v2.1.6~3 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=4e3100fb3d268b46aebe71dd59c0d8360624cabf;p=thirdparty%2Fpatchwork.git REST: fix patch listing query The patch listing query is punishingly slow under even very simple filters. The new data model in 3.0 will help _a lot_, so this is a simple fix: I did try indexes but haven't got really deeply into the weeds of what we can do with them. Move a number of things from select_related to prefetch_related: we trade off one big, inefficient query for a slightly larger number of significantly more efficient queries. On my laptop with 2 copies of the canonical kernel team list loaded into the database, and considering only the API view (the JSON view is always faster) with warm caches and considering the entire set of SQL queries: - /api/patches/?project=1 ~1.4-1.5s -> <100ms, something like 14x better - /api/patches/?project=1&since=2010-11-01T00:00:00 ~1.7-1.8s -> <560ms, something like 3x better (now dominated by the counting query only invoked on the HTML API view, not the pure JSON API view.) The things I moved: * project: this was generating SQL that looked like: INNER JOIN `patchwork_project` T5 ON (`patchwork_submission`.`project_id` = T5.`id`) This is correct but we've already had to join the patchwork_submission table and perhaps as a result it seems to be inefficient. * series__project: Likewise we've already had to join the series table, doing another join is possibly why it is inefficient. * delegate: I do not know why this was tanking performance. I think it might relate to the strategy mysql was using. Reported-by: Konstantin Ryabitsev Signed-off-by: Daniel Axtens Reviewed-by: Stephen Finucane (backported from commit 98a2d051372dcedb889c4cb94ebd8ed7b399b522 - dropped tests, it depends on a test we don't carry - rejigged to suit old M:N series model) Signed-off-by: Daniel Axtens --- diff --git a/patchwork/api/patch.py b/patchwork/api/patch.py index c55bf9c9..f593316d 100644 --- a/patchwork/api/patch.py +++ b/patchwork/api/patch.py @@ -187,9 +187,9 @@ class PatchList(ListAPIView): def get_queryset(self): return Patch.objects.all()\ - .prefetch_related('series', 'check_set', 'series__project')\ - .select_related('project', 'state', 'submitter', 'delegate', - )\ + .prefetch_related('series', 'check_set', 'project', + 'delegate', 'series__project')\ + .select_related('state', 'submitter')\ .defer('content', 'diff', 'headers')