]> git.ipfire.org Git - location/libloc.git/commit
importer: Change download behaviour
authorMichael Tremer <michael.tremer@ipfire.org>
Fri, 12 Aug 2022 13:51:20 +0000 (13:51 +0000)
committerMichael Tremer <michael.tremer@ipfire.org>
Fri, 12 Aug 2022 13:51:20 +0000 (13:51 +0000)
commit85e44d9a5be09403ba8a432bcd8a13465eec0c0d
tree1f5bb0790d182c83ede6eec14822bbcbb6ca7edc
parentaa23e03d682ce7d262e01f726f0921c5eec49d78
importer: Change download behaviour

The downloader used to open a connection to the web server hosting our
content which would have been decompressed (if necessary) on the fly and
also been parsed on the fly so that it could have been fed into the
database easily.

Some webservers do not seem to be patient enough to keep the connection
open if things take a little bit longer than usual. That caused the
import to fail.

This patch changes the behaviour that we would download all content
first, store it locally, and then start processing it.

Fixes: #12852
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Cc: Peter Müller <peter.mueller@ipfire.org>
src/python/location/importer.py
src/scripts/location-importer.in