From: Daniel Axtens <dja@axtens.net>
Date: Thu, 28 Sep 2017 14:12:24 +0000 (+1000)
Subject: xmlrpc/patch_list: only fetch required fields
X-Git-Tag: v2.1.0-rc1~119
X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=76c3dae262590b8fcddc3d24a89b9b3717e3a9dc;p=thirdparty%2Fpatchwork.git

xmlrpc/patch_list: only fetch required fields

OzLabs noticed *massive* slowdowns in queries like this one:

SELECT "patchwork_submission"."id", "patchwork_submission"."msgid",
"patchwork_submission"."date", "patchwork_submission"."headers",
"patchwork_submission"."submitter_id",
"patchwork_submission"."content", "patchwork_submission"."project_id",
"patchwork_submission"."name", "patchwork_patch"."submission_ptr_id",
"patchwork_patch"."diff", "patchwork_patch"."commit_ref",
"patchwork_patch"."pull_url", "patchwork_patch"."delegate_id",
"patchwork_patch"."state_id", "patchwork_patch"."archived",
"patchwork_patch"."hash" FROM "patchwork_patch" INNER JOIN
"patchwork_submission" ON ("patchwork_patch"."submission_ptr_id" =
"patchwork_submission"."id") WHERE
("patchwork_submission"."project_id" = 2 AND
"patchwork_patch"."state_id" = 1) ORDER BY
"patchwork_submission"."date" ASC

These appear to be a result of pwclient list operations. We *do not*
need content/headers/diff in this case - so do not fetch them as it
is incredibly expensive - queries in excess of 50s have been observed.

This should go to stable/2.0.

Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
Reported-by: Michael Ellerman <mpe@ellerman.id.au>
Cc: Jeremy Kerr <jk@ozlabs.org>
Signed-off-by: Daniel Axtens <dja@axtens.net>
Reviewed-by: Stephen Finucane <stephen@that.guru>
---

diff --git a/patchwork/views/xmlrpc.py b/patchwork/views/xmlrpc.py
index 8de9d10e..93f6035e 100644
--- a/patchwork/views/xmlrpc.py
+++ b/patchwork/views/xmlrpc.py
@@ -585,6 +585,11 @@ def patch_list(filt=None):
 
     patches = Patch.objects.filter(**dfilter)
 
+    # Only extract the relevant fields. This saves a big db load as we
+    # no longer fetch content/headers/etc for potentially every patch
+    # in a project.
+    patches = patches.defer('content', 'headers', 'diff')
+
     return _get_objects(patch_to_dict, patches, max_count)