REST: Embed nested element bodies instead of URLs
In developing a client for the Patchwork REST API, git-pw, it was noted
that it should be possible to embed some information about nested
resources in order to prevent the need for additional requests [1]. It
was seen that this would be particularly beneficial for list operations,
where each element in the N sized list could theoretically require an
additional request for each of the M nested fields, resulting in N * (M
+ 1) total requests.
Upon experimenting with the 2.0 RC1 API, this optimization was found to
be less of a nice-to-have (and possibly something for the 2.1 release)
and more of a must-have, particularly once one took network latency for
each request into account. During testing with 'git-pw', simple list
operations were found to take an average of 31 requests per operation,
of which only one for was the resource endpoint itself ('GET
/api/series'). As each of these requests took ~2 seconds a piece,
listing was essentially broken.
While local caching could be used to offset some of this demand, this
will result in (a) significantly larger, more complex clients or (b)
instances that strain under the load of dumb clients making multiple
requests per operation. Instead, the server should be smarter about
embedding the data that would actually be required by clients.
Resolve the issue by embedding summarized versions of various nested
fields instead of merely linking to them. Nesting is only a single level
deep, to avoid large/complex database queries and with the expectation
that only these basic fields (resource names, dates, etc.) would be
required. These summary serializers are kept in their own module, to
encourage consistent results throughout the API and to prevent circular
import errors.
This will have the side effect of slightly increasing load on the server
due to the additional serialization required. However, this load is
largely mitigated through the avoidance of deeper nesting as noted
above. In addition, any increase in load seen will be a fraction of the
demand that repeat requests will incur. While it would be possible to
make nesting optional (by way of an 'embed' or 'expand' parameter), it
is expected that this would be an atypical request and would result in
far more complicated serialization code.
[1] https://github.com/stephenfin/git-pw/blob/
21e0e593/git_pw/patch.py#L88-L89
Signed-off-by: Stephen Finucane <stephen@that.guru>