From: Daniel Axtens <dja@axtens.net>
Date: Mon, 12 Mar 2018 12:08:54 +0000 (+1100)
Subject: api: Only provide JSON version of events list
X-Git-Tag: v2.1.0-rc1~44
X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=90d9ee14e73e8ec9248e89c788d64867c4a4bb74;p=thirdparty%2Fpatchwork.git

api: Only provide JSON version of events list

Something is very, very slow in the d-r-f browsable API events renderer.

In my MySQL test (~33k patches), the CPU time to render the events list
is ~11s, and the time taken by SQL queries is only ~3s. If the JSON
renderer is used, that drops to 0.2s for the entire page (because less
CPU is used, and - for some as yet unknown reason - a *very* expensive
db query is dropped.)

In my PostgreSQL test (~100k patches), the results are even more stark:
30s of CPU time and 0.2s of DB time goes to 0.25s for the entire page.

Something is seriously, seriously wrong with whatever d-r-f is doing.
So, simply render the event list as unlinked JSON for now.

There are a few followups we should do, but this is an important start -
no-one should be able to DoS a patchwork server by just enumerating the
events!

In particular, we should find out:
 - why postgres and mysql behaviour is so different.
 - what on earth d-r-f is doing that makes rendering the pretty-printed
   version so incredibly slow.

Signed-off-by: Daniel Axtens <dja@axtens.net>
---

diff --git a/patchwork/api/event.py b/patchwork/api/event.py
index 7e04b716..9879a9f6 100644
--- a/patchwork/api/event.py
+++ b/patchwork/api/event.py
@@ -18,10 +18,13 @@
 # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
 
 from collections import OrderedDict
+import json
 
 from rest_framework.generics import ListAPIView
 from rest_framework.serializers import ModelSerializer
 from rest_framework.serializers import SerializerMethodField
+from rest_framework.renderers import JSONRenderer
+from rest_framework.renderers import TemplateHTMLRenderer
 
 from patchwork.api.embedded import CheckSerializer
 from patchwork.api.embedded import CoverLetterSerializer
@@ -85,9 +88,21 @@ class EventSerializer(ModelSerializer):
         read_only_fields = fields
 
 
+# The standard template html renderer is broken:
+# https://github.com/encode/django-rest-framework/issues/5236
+class JSONListHTMLRenderer(TemplateHTMLRenderer):
+    def get_template_context(self, data, renderer_context):
+        response = renderer_context['response']
+        if response.exception:
+            data['status_code'] = response.status_code
+        return {'data': json.dumps(data, indent=4)}
+
+
 class EventList(ListAPIView):
     """List events."""
 
+    renderer_classes = (JSONRenderer, JSONListHTMLRenderer)
+    template_name = 'patchwork/event-list.html'
     serializer_class = EventSerializer
     filter_class = EventFilter
     page_size_query_param = None  # fixed page size
diff --git a/patchwork/templates/patchwork/event-list.html b/patchwork/templates/patchwork/event-list.html
new file mode 100644
index 00000000..821c6897
--- /dev/null
+++ b/patchwork/templates/patchwork/event-list.html
@@ -0,0 +1,17 @@
+{% extends "base.html" %}
+
+{% load person %}
+{% load static %}
+
+{% block title %}Event List{% endblock %}
+{% block patch_active %}active{% endblock %}
+
+{% block body %}
+
+<p>Due to a currently undiagnosed issue with django-rest-framework, the browsable API is very CPU intensive and has been disabled. The JSON output is:</p>
+
+<pre>
+{{data}}
+</pre>
+
+{% endblock %}