From: Michael Tremer <michael.tremer@ipfire.org>
Date: Fri, 2 Dec 2022 11:06:01 +0000 (+0000)
Subject: importer: Fix parsing gzipped content on invalid Content-Type header
X-Git-Tag: 0.9.17~38
X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=c39b3b92b0c557fba49a01ec63879189f3db2da1;p=location%2Flibloc.git

importer: Fix parsing gzipped content on invalid Content-Type header

RIPE seems to have misconfigured their webserver which now sends
application/octet-stream for all gzipped files instead of
application/x-gzip or similar which is what the importer would expect.

Instead of only checking the content type, we will now test whether we
see the gzip magic and try to decompress the file then.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
---

diff --git a/src/python/location/importer.py b/src/python/location/importer.py
index d285162..f391e03 100644
--- a/src/python/location/importer.py
+++ b/src/python/location/importer.py
@@ -148,11 +148,26 @@ class Downloader(object):
 		# Rewind the temporary file
 		t.seek(0)
 
+		gzip_compressed = False
+
 		# Fetch the content type
 		content_type = res.headers.get("Content-Type")
 
 		# Decompress any gzipped response on the fly
 		if content_type in ("application/x-gzip", "application/gzip"):
+			gzip_compressed = True
+
+		# Check for the gzip magic in case web servers send a different MIME type
+		elif t.read(2) == b"\x1f\x8b":
+			gzip_compressed = True
+
+		# Reset again
+		t.seek(0)
+
+		# Decompress the temporary file
+		if gzip_compressed:
+			log.debug("Gzip compression detected")
+
 			t = gzip.GzipFile(fileobj=t, mode="rb")
 
 		# Return the temporary file handle