diff options
author | Daniel Axtens <dja@axtens.net> | 2018-03-12 23:08:54 +1100 |
---|---|---|
committer | Daniel Axtens <dja@axtens.net> | 2018-04-05 19:21:07 +1000 |
commit | 90d9ee14e73e8ec9248e89c788d64867c4a4bb74 (patch) | |
tree | 55c8962166be7bf02b43cf9b48e5232f38e34b4d | |
parent | 5d9b00409fcafc7b8cd09400195c9c9f1b3e4fcc (diff) | |
download | patchwork-90d9ee14e73e8ec9248e89c788d64867c4a4bb74.tar patchwork-90d9ee14e73e8ec9248e89c788d64867c4a4bb74.tar.gz |
api: Only provide JSON version of events list
Something is very, very slow in the d-r-f browsable API events renderer.
In my MySQL test (~33k patches), the CPU time to render the events list
is ~11s, and the time taken by SQL queries is only ~3s. If the JSON
renderer is used, that drops to 0.2s for the entire page (because less
CPU is used, and - for some as yet unknown reason - a *very* expensive
db query is dropped.)
In my PostgreSQL test (~100k patches), the results are even more stark:
30s of CPU time and 0.2s of DB time goes to 0.25s for the entire page.
Something is seriously, seriously wrong with whatever d-r-f is doing.
So, simply render the event list as unlinked JSON for now.
There are a few followups we should do, but this is an important start -
no-one should be able to DoS a patchwork server by just enumerating the
events!
In particular, we should find out:
- why postgres and mysql behaviour is so different.
- what on earth d-r-f is doing that makes rendering the pretty-printed
version so incredibly slow.
Signed-off-by: Daniel Axtens <dja@axtens.net>
-rw-r--r-- | patchwork/api/event.py | 15 | ||||
-rw-r--r-- | patchwork/templates/patchwork/event-list.html | 17 |
2 files changed, 32 insertions, 0 deletions
diff --git a/patchwork/api/event.py b/patchwork/api/event.py index 7e04b71..9879a9f 100644 --- a/patchwork/api/event.py +++ b/patchwork/api/event.py @@ -18,10 +18,13 @@ # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA from collections import OrderedDict +import json from rest_framework.generics import ListAPIView from rest_framework.serializers import ModelSerializer from rest_framework.serializers import SerializerMethodField +from rest_framework.renderers import JSONRenderer +from rest_framework.renderers import TemplateHTMLRenderer from patchwork.api.embedded import CheckSerializer from patchwork.api.embedded import CoverLetterSerializer @@ -85,9 +88,21 @@ class EventSerializer(ModelSerializer): read_only_fields = fields +# The standard template html renderer is broken: +# https://github.com/encode/django-rest-framework/issues/5236 +class JSONListHTMLRenderer(TemplateHTMLRenderer): + def get_template_context(self, data, renderer_context): + response = renderer_context['response'] + if response.exception: + data['status_code'] = response.status_code + return {'data': json.dumps(data, indent=4)} + + class EventList(ListAPIView): """List events.""" + renderer_classes = (JSONRenderer, JSONListHTMLRenderer) + template_name = 'patchwork/event-list.html' serializer_class = EventSerializer filter_class = EventFilter page_size_query_param = None # fixed page size diff --git a/patchwork/templates/patchwork/event-list.html b/patchwork/templates/patchwork/event-list.html new file mode 100644 index 0000000..821c689 --- /dev/null +++ b/patchwork/templates/patchwork/event-list.html @@ -0,0 +1,17 @@ +{% extends "base.html" %} + +{% load person %} +{% load static %} + +{% block title %}Event List{% endblock %} +{% block patch_active %}active{% endblock %} + +{% block body %} + +<p>Due to a currently undiagnosed issue with django-rest-framework, the browsable API is very CPU intensive and has been disabled. The JSON output is:</p> + +<pre> +{{data}} +</pre> + +{% endblock %} |