api: do not fetch every patch in a patch detail view 404 (v2)

api: do not fetch every patch in a patch detail view 404 (v2)

Commit 08c5856 fixed an issue whereby a 404 on the aforementioned URL
could result in a large DB query due to DRF attempting to populate the
'related' list box with all patches on the instance. That was
accidentally reverted in commit fe07f30. "Unrevert" this change.

Signed-off-by: Stephen Finucane <stephen@that.guru>
Fixes: fe07f30 ("Remove 'PatchRelationSerializer'")
Closes: #397
(cherry picked from commit 79700f321335a2d7c649eb03932797af521942a0)