From: Stewart Smith <stewart@linux.ibm.com>
Date: Fri, 10 Aug 2018 08:01:05 +0000 (+1000)
Subject: Be sensible computing project patch counts
X-Git-Tag: v2.2.0-rc1~292
X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=cfcf2f2a80ac0709f1a5fd9aa212c8403daa5a18;p=thirdparty%2Fpatchwork.git

Be sensible computing project patch counts

Django actively fights constructing a query that isn't insane.

So, let's go and just execute a raw one. This is all very standard
SQL so should execute everywhere without a problem.

With the dataset of patchwork.ozlabs.org, looking at the /project/
page for qemu-devel would take 13 queries and 1500ms,
with this patch it's down to 11 queries in ~250ms.
For the dataset of the netdev list, it's down to 440ms from 1500ms.

Signed-off-by: Stewart Smith <stewart@linux.ibm.com>
[stephenfin: Handle projects that have no archived patches and those
 with all patches archived]
Signed-off-by: Stephen Finucane <stephen@that.guru>
---

diff --git a/patchwork/views/project.py b/patchwork/views/project.py
index a2e5b2da..b83dabfa 100644
--- a/patchwork/views/project.py
+++ b/patchwork/views/project.py
@@ -24,9 +24,10 @@ from django.shortcuts import get_object_or_404
 from django.shortcuts import render
 from django.urls import reverse
 
-from patchwork.models import Patch
 from patchwork.models import Project
 
+from django.db import connection
+
 
 def project_list(request):
     projects = Project.objects.all()
@@ -44,14 +45,34 @@ def project_list(request):
 
 def project_detail(request, project_id):
     project = get_object_or_404(Project, linkname=project_id)
-    patches = Patch.objects.filter(project=project)
+
+    # So, we revert to raw sql because if we do what you'd think would
+    # be the correct thing in Django-ese, it ends up doing a *pointless*
+    # join with patchwork_submissions that ends up ruining the query.
+    # So, we do not do this, as this is wrong:
+    #
+    #   patches = Patch.objects.filter(
+    #       patch_project_id=project.id).only('archived')
+    #   patches = patches.annotate(c=Count('archived'))
+    #
+    # and instead do this, because it's simple and fast
+
+    n_patches = {}
+    with connection.cursor() as cursor:
+        cursor.execute('SELECT archived,COUNT(submission_ptr_id) as c '
+                       'FROM patchwork_patch '
+                       'WHERE patch_project_id=%s GROUP BY archived',
+                       [project.id])
+
+        for r in cursor:
+            n_patches[r[0]] = r[1]
 
     context = {
         'project': project,
         'maintainers': User.objects.filter(
             profile__maintainer_projects=project),
-        'n_patches': patches.filter(archived=False).count(),
-        'n_archived_patches': patches.filter(archived=True).count(),
+        'n_patches': n_patches[False] if False in n_patches else 0,
+        'n_archived_patches': n_patches[True] if True in n_patches else 0,
         'enable_xmlrpc': settings.ENABLE_XMLRPC,
     }
     return render(request, 'patchwork/project.html', context)