t9104: kosherly remove remote refs

t9104: kosherly remove remote refs

As there are plans to implement other ref storage systems,
let's use a way to remove remote refs that does not depend
on refs being files.

This makes it clear to readers that this test does not
depend on which ref backend is used.

Suggested-by: Michael Haggerty <mhagger@alum.mit.edu>
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>