location/libloc.git
3 months agolocation.in: do not confuse UTC with local time zones
Peter Müller [Thu, 29 Oct 2020 14:36:46 +0000 (07:36 -0700)] 
location.in: do not confuse UTC with local time zones

Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
3 months agolocation update: Remove double conversion of timestamps
Peter Müller [Thu, 29 Oct 2020 14:25:53 +0000 (07:25 -0700)] 
location update: Remove double conversion of timestamps

This caused that the timestamp in DNS was misinterpreted
as local time and often, databases could not be downloaded.

Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
3 months agolocation: Fix Python syntax error in verify()
Michael Tremer [Wed, 28 Oct 2020 09:52:36 +0000 (09:52 +0000)] 
location: Fix Python syntax error in verify()

The database is now being opened before the requested
method is called and handle_verify() wasn't updated.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
3 months agoexport: Flatten the tree before exporting it
Michael Tremer [Tue, 27 Oct 2020 17:14:30 +0000 (17:14 +0000)] 
export: Flatten the tree before exporting it

This patch removes the possibility that any IP address ranges
might show up in multiple exported files.

If this was content from the database:

  * 10.0.0.0/16 - DE
  * 10.0.1.0/24 - FR

Then the IP address 10.0.1.1 would match for both countries.

The algorithm will now break the larger /16 subnet apart into
smaller subnets so that 10.0.1.0/24 is no longer overlapped.

There was some time spent on this to make this as efficient
as possible.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
3 months agolocation-importer.in: avoid log spam for too small networks
Peter Müller [Wed, 21 Oct 2020 14:47:43 +0000 (14:47 +0000)] 
location-importer.in: avoid log spam for too small networks

Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
3 months agolocation-importer.in: omit historic/orphaned RIR data
Peter Müller [Wed, 21 Oct 2020 14:47:41 +0000 (14:47 +0000)] 
location-importer.in: omit historic/orphaned RIR data

Some RIRs include detailled information regarding networks not managed
by or allocated to themselves, particually APNIC. We need to filter
those networks (they usually have a characteristic network name) in
order to prevent operational quirks or returning wrong country codes.

Fixes: #12501
Partially fixes: #12499

Cc: Michael Tremer <michael.tremer@ipfire.org>
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
3 months agoimporter.py: fetch LACNIC data via HTTPS
Peter Müller [Wed, 21 Oct 2020 14:47:40 +0000 (14:47 +0000)] 
importer.py: fetch LACNIC data via HTTPS

Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
3 months agolocation-importer.in: filter bogus IP networks for both Whois and extended sources
Peter Müller [Wed, 21 Oct 2020 14:47:39 +0000 (14:47 +0000)] 
location-importer.in: filter bogus IP networks for both Whois and extended sources

Sanity checks for parsed networks have been put into a separate function
to avoid boilerplate code for extended sources. This makes the location
database less vulnerable to garbage written into RIR databases on
purpose or by chance.

Fixes: #12500

Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
3 months agoexport.py: fix exporting IP networks for crappy xt_geoip module
Peter Müller [Wed, 21 Oct 2020 14:47:38 +0000 (14:47 +0000)] 
export.py: fix exporting IP networks for crappy xt_geoip module

In contrast to the location database itself, the xt_geoip module
consumes a list of IP networks for each country, and returns after the
first match.

We therefore need to...

(a) sort IP networks by their size, allow as precise matches as possible
(b) export _any_ IP networks - including inverted subnets - to prevent
    undefined overlaps
(c) do the entire thing as fast as possible, consuming as less disk
    space as possible, which is why we can't just iterate over all /24
    chunks

Partially fixes: #12499

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
3 months agoRevert "Revert "location-importer.in: only import relevant data from AFRINIC, APNIC...
Peter Müller [Wed, 21 Oct 2020 14:47:37 +0000 (14:47 +0000)] 
Revert "Revert "location-importer.in: only import relevant data from AFRINIC, APNIC and RIPE""

This reverts commit 13f67f285856e8eabfeff2daf1be3aeaa36a82cc.

Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
3 months agoRevert "Revert "Revert "Revert "importer: Import raw sources for inetnum's again""""
Peter Müller [Wed, 21 Oct 2020 14:47:36 +0000 (14:47 +0000)] 
Revert "Revert "Revert "Revert "importer: Import raw sources for inetnum's again""""

This reverts commit 44341478233115b26bb27fdb24da5b0a1eedb173.

Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
4 months agonetwork: Allow adding single IP addresses and automatically add the prefix
Michael Tremer [Thu, 22 Oct 2020 12:24:34 +0000 (12:24 +0000)] 
network: Allow adding single IP addresses and automatically add the prefix

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
4 months agoimporter: Restructure SQL query to be executed in parallel
Michael Tremer [Wed, 21 Oct 2020 16:01:57 +0000 (16:01 +0000)] 
importer: Restructure SQL query to be executed in parallel

There are no functional changes, this just runs quicker now.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
4 months agoimporter: Add search index to network_overrides table
Michael Tremer [Wed, 21 Oct 2020 13:54:45 +0000 (13:54 +0000)] 
importer: Add search index to network_overrides table

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
4 months agoimporter: Add search index to announcements table
Michael Tremer [Wed, 21 Oct 2020 13:53:36 +0000 (13:53 +0000)] 
importer: Add search index to announcements table

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
4 months agonetworks: Remove accidentially committed debug line
Michael Tremer [Wed, 21 Oct 2020 13:44:50 +0000 (13:44 +0000)] 
networks: Remove accidentially committed debug line

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
4 months agonetworks: Test if we can add localhost (IPv6)
Michael Tremer [Wed, 21 Oct 2020 13:43:21 +0000 (13:43 +0000)] 
networks: Test if we can add localhost (IPv6)

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
4 months agonetworks: Improve parsing IP addresses
Michael Tremer [Wed, 21 Oct 2020 13:36:35 +0000 (13:36 +0000)] 
networks: Improve parsing IP addresses

loc_network_new_from_string() seem to have had some unexpected
behaviour for invalid inputs.

The function has been tidied up slightly and returns as soon as
some invalid input was detected.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
4 months agotests: Try adding an invalid network
Michael Tremer [Wed, 21 Oct 2020 13:19:44 +0000 (13:19 +0000)] 
tests: Try adding an invalid network

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
4 months agowriter: Free countries when the writer is being destroyed
Michael Tremer [Wed, 21 Oct 2020 09:31:29 +0000 (09:31 +0000)] 
writer: Free countries when the writer is being destroyed

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
4 months agowriter: Free array with pointer to ASes, too
Michael Tremer [Wed, 21 Oct 2020 09:28:39 +0000 (09:28 +0000)] 
writer: Free array with pointer to ASes, too

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
4 months agoas: Do not attempt to match name when it wasn't set
Michael Tremer [Wed, 21 Oct 2020 09:18:08 +0000 (09:18 +0000)] 
as: Do not attempt to match name when it wasn't set

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
4 months agoas: Fix dereferencing NULL pointer when setting AS name
Michael Tremer [Tue, 20 Oct 2020 20:44:43 +0000 (20:44 +0000)] 
as: Fix dereferencing NULL pointer when setting AS name

Reported-by: Gisle Vanem <gisle.vanem@gmail.com>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
4 months agoRevert "Revert "Revert "importer: Import raw sources for inetnum's again"""
Michael Tremer [Fri, 16 Oct 2020 12:26:43 +0000 (12:26 +0000)] 
Revert "Revert "Revert "importer: Import raw sources for inetnum's again"""

This reverts commit f532841e9197ce2f40aad8c086d786b2cb783a54.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
4 months agoRevert "location-importer.in: only import relevant data from AFRINIC, APNIC and RIPE"
Michael Tremer [Fri, 16 Oct 2020 12:26:38 +0000 (12:26 +0000)] 
Revert "location-importer.in: only import relevant data from AFRINIC, APNIC and RIPE"

This reverts commit a36bc686865fc87ea386fd90b389338bdcb80cbc.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
4 months agolocation-importer: Include all overridden networks
Michael Tremer [Fri, 16 Oct 2020 12:24:58 +0000 (12:24 +0000)] 
location-importer: Include all overridden networks

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
4 months agolocation-importer.in: only import relevant data from AFRINIC, APNIC and RIPE
Peter Müller [Mon, 12 Oct 2020 20:53:32 +0000 (20:53 +0000)] 
location-importer.in: only import relevant data from AFRINIC, APNIC and RIPE

In contrast to ARIN and LACNIC, we are able to process more detailled
feeds from those RIRs, avoiding storage of obviously unnecessary data.

Thanks to various SQL optimisations, doing so now takes less time than
the first version of this did.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
4 months agoRevert "Revert "importer: Import raw sources for inetnum's again""
Peter Müller [Mon, 12 Oct 2020 20:53:31 +0000 (20:53 +0000)] 
Revert "Revert "importer: Import raw sources for inetnum's again""

This reverts commit 92f6abf4e272672bb0a71cfe991261b95ebe2fef.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
5 months agoRevert "importer: Import raw sources for inetnum's again"
Michael Tremer [Thu, 24 Sep 2020 10:18:58 +0000 (10:18 +0000)] 
Revert "importer: Import raw sources for inetnum's again"

This reverts commit 64e95fa903edec8b4e4e59830b395e2e4a411853.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
5 months agoRevert "importer: Purge any redundant entries"
Michael Tremer [Thu, 24 Sep 2020 10:17:58 +0000 (10:17 +0000)] 
Revert "importer: Purge any redundant entries"

This reverts commit c2cc55d5a6875c3838f060032eaed89dcfb92ef6.

The query stalls the database and therefore the automatic
scripts are no longer able to generate a new version of the
database.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
5 months agoBump version to 0.9.4 0.9.4
Michael Tremer [Mon, 21 Sep 2020 16:21:21 +0000 (16:21 +0000)] 
Bump version to 0.9.4

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
5 months agoimporter: Purge any redundant entries
Peter Müller [Sun, 20 Sep 2020 19:21:03 +0000 (19:21 +0000)] 
importer: Purge any redundant entries

When importing inetnums, we might import various small networks
which are not relevant for us as long as they do not have a
different country code than their parent network.

Therefore we delete all these entries to keep the database
smaller without losing any information. The second version of this
patch introduces a SQL statement parallelised across all CPUs
available, while the DELETE-statement of the first version literally
took ages to complete.

However, cleaning up those data still takes about 26 hours (!) on
our location02 testing machine, making daily updates of the location
database impossible to the current knowledge.

real    1521m30.620s
user    38m45.521s
sys     9m6.027s

Special thanks goes to Michael for spending numerous hours
on this, setting up a testing environment, doing PostgreSQL magic
and providing helpful advice while debugging.

Partially fixes: #12458

Cc: Michael Tremer <michael.tremer@ipfire.org>
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
5 months agoimporter: Import raw sources for inetnum's again
Peter Müller [Sun, 20 Sep 2020 19:20:18 +0000 (19:20 +0000)] 
importer: Import raw sources for inetnum's again

The extended feeds do not have enough detailed information
for us, so that we need to import inetnums from RIRs where
possible. Filtering private networks is necessary as RIR data
may contain 0.0.0.0/0 or similar entries for administrative
purposes or due to misfilings.

Special thanks goes to Michael for spending numerous hours
on this, setting up a testing environment and providing helpful
advice while debugging.

Partially fixes: #12458

Cc: Michael Tremer <michael.tremer@ipfire.org>
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
5 months agolocation-importer.in: avoid violating NOT NULL constraints during JOIN
Peter Müller [Sun, 20 Sep 2020 19:19:36 +0000 (19:19 +0000)] 
location-importer.in: avoid violating NOT NULL constraints during JOIN

Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
5 months agonetwork: Determine family when the network is initialised
Michael Tremer [Fri, 18 Sep 2020 13:50:11 +0000 (13:50 +0000)] 
network: Determine family when the network is initialised

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
5 months agonetwork: Export family, first_address and last_address in Python
Michael Tremer [Fri, 18 Sep 2020 13:46:58 +0000 (13:46 +0000)] 
network: Export family, first_address and last_address in Python

We can use this to avoid formatting the network in a string
representation which is then parsed again later to determine
the first and last addresses.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
5 months agonetwork: Store the first and last address of each network
Michael Tremer [Fri, 18 Sep 2020 13:20:21 +0000 (13:20 +0000)] 
network: Store the first and last address of each network

This avoids recomputing the last address multiple times.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
5 months agonetworks: Rename start_address to first_address
Michael Tremer [Fri, 18 Sep 2020 13:13:32 +0000 (13:13 +0000)] 
networks: Rename start_address to first_address

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
5 months agoexport: Skip writing any subnets
Michael Tremer [Fri, 18 Sep 2020 10:26:15 +0000 (10:26 +0000)] 
export: Skip writing any subnets

It is unnecessary to write any subnets in the output, because
they might already be included in a bigger parent network.

Since the tree is ordered, we only need to check if the network
to be written is a subnet of the previous one. If so, we skip it.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
5 months agolocation: Fix message sting if the database has been recently updated
Stefan Schantl [Mon, 14 Sep 2020 18:36:36 +0000 (20:36 +0200)] 
location: Fix message sting if the database has been recently updated

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
5 months agoperl: Add test for get_continent_code() function.
Stefan Schantl [Thu, 10 Sep 2020 15:57:47 +0000 (17:57 +0200)] 
perl: Add test for get_continent_code() function.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
5 months agoperl: Add test for get_country_name() function.
Stefan Schantl [Thu, 10 Sep 2020 15:57:46 +0000 (17:57 +0200)] 
perl: Add test for get_country_name() function.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
5 months agopython example script: Add country name and continent code.
Stefan Schantl [Thu, 10 Sep 2020 15:57:45 +0000 (17:57 +0200)] 
python example script: Add country name and continent code.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
5 months agoperl: Add get_country_name() function.
Stefan Schantl [Thu, 10 Sep 2020 15:57:44 +0000 (17:57 +0200)] 
perl: Add get_country_name() function.

This function can be used to get the stored name for a given country
code.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
5 months agoperl: Add test for the lookup_network_has_flag() function
Stefan Schantl [Sun, 30 Aug 2020 10:36:02 +0000 (12:36 +0200)] 
perl: Add test for the lookup_network_has_flag() function

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
5 months agoAdd example network flag to the python create-database example script.
Stefan Schantl [Sun, 30 Aug 2020 10:36:01 +0000 (12:36 +0200)] 
Add example network flag to the python create-database example script.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
5 months agoperl: Add lookup_network_has_flag() function.
Stefan Schantl [Sun, 30 Aug 2020 10:36:00 +0000 (12:36 +0200)] 
perl: Add lookup_network_has_flag() function.

This function can be used to check if a given address or network has one
of the following network flags.

* LOC_NETWORK_FLAG_ANONYMOUS_PROXY
* LOC_NETWORK_FLAG_SATELLITE_PROVIDER
* LOC_NETWORK_FLAG_ANYCAST

It will return true if the given flag is set.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
6 months agoperl: Add get_as_name()
Stefan Schantl [Sat, 22 Aug 2020 14:02:31 +0000 (16:02 +0200)] 
perl: Add get_as_name()

This function can be use to give the numer of an Autonomous System and
get back the stored name from the used location database.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
6 months agoperl: Add test for the database_countries() function.
Stefan Schantl [Thu, 20 Aug 2020 17:28:56 +0000 (19:28 +0200)] 
perl: Add test for the database_countries() function.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
6 months agoAdd an example country to the python create-database example script.
Stefan Schantl [Thu, 20 Aug 2020 17:28:55 +0000 (19:28 +0200)] 
Add an example country to the python create-database example script.

The generated database is used by the testsuite of the perl binding.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
6 months agoperl: Add database_countries() function.
Stefan Schantl [Thu, 20 Aug 2020 17:28:54 +0000 (19:28 +0200)] 
perl: Add database_countries() function.

This function is used to the stored countries of a database, which
easily can be assigned to an array.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
6 months agoUpdate translations
Michael Tremer [Tue, 4 Aug 2020 15:44:41 +0000 (15:44 +0000)] 
Update translations

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
6 months agolocation: Provide a return code if the database does not need to be updated.
Stefan Schantl [Wed, 29 Jul 2020 16:51:59 +0000 (18:51 +0200)] 
location: Provide a return code if the database does not need to be updated.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
6 months agolocation-importer: Read all routes and not only those imported through iBGP
Michael Tremer [Tue, 28 Jul 2020 16:00:52 +0000 (16:00 +0000)] 
location-importer: Read all routes and not only those imported through iBGP

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agoimporter: Implement fetch routes from Bird
Michael Tremer [Fri, 24 Jul 2020 16:58:35 +0000 (16:58 +0000)] 
importer: Implement fetch routes from Bird

If the route server is now a socket, the importer will connect
to it and fetch the routing table. This is a lot faster than telnet.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agocountry: Terminate buffer when reading from database
Michael Tremer [Mon, 13 Jul 2020 10:47:30 +0000 (10:47 +0000)] 
country: Terminate buffer when reading from database

Compilers on ARM do not seem to initialise the buffer.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agoconfigure: Bump version to 0.9.3 0.9.3
Michael Tremer [Fri, 10 Jul 2020 14:49:49 +0000 (14:49 +0000)] 
configure: Bump version to 0.9.3

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agolocation(8): Remove any temporary files after unsuccessful downloads
Stefan Schantl [Fri, 10 Jul 2020 14:47:54 +0000 (16:47 +0200)] 
location(8): Remove any temporary files after unsuccessful downloads

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
7 months agolocation(8): Fix pasing timestamp for systems not running on UTC
Stefan Schantl [Fri, 10 Jul 2020 14:40:33 +0000 (16:40 +0200)] 
location(8): Fix pasing timestamp for systems not running on UTC

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
7 months agolocation(8): Allow downloading a database without one on disk
Michael Tremer [Fri, 10 Jul 2020 10:43:46 +0000 (10:43 +0000)] 
location(8): Allow downloading a database without one on disk

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agolocation(8): Allow limiting updates to once a day, week, month
Michael Tremer [Fri, 10 Jul 2020 10:41:23 +0000 (10:41 +0000)] 
location(8): Allow limiting updates to once a day, week, month

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agoimporter: Correctly sort the results
Michael Tremer [Mon, 6 Jul 2020 16:44:20 +0000 (16:44 +0000)] 
importer: Correctly sort the results

The results where incorrectly ordered so that the DISTINCT clause
removed the wrong rows matching an incorrect parent subnet.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agoimporter: Also add networks that are not announced to the database
Michael Tremer [Mon, 29 Jun 2020 15:42:43 +0000 (15:42 +0000)] 
importer: Also add networks that are not announced to the database

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agoimporter: Write NULL into database when bool is not set
Michael Tremer [Mon, 29 Jun 2020 13:34:26 +0000 (13:34 +0000)] 
importer: Write NULL into database when bool is not set

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agoimporter: Walk through all options for flags
Michael Tremer [Mon, 29 Jun 2020 13:28:06 +0000 (13:28 +0000)] 
importer: Walk through all options for flags

It could happen that a network was marked, but the AS was not.

Since the AS is checked first, we won't check the network and
therefore write the wrong information into the database.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
7 months agodatabase: Always require SSL
Michael Tremer [Mon, 29 Jun 2020 13:27:52 +0000 (13:27 +0000)] 
database: Always require SSL

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agolocation.in: strip trailing whitespaces in database description while dumping
Peter Müller [Mon, 22 Jun 2020 17:55:17 +0000 (17:55 +0000)] 
location.in: strip trailing whitespaces in database description while dumping

Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agoconfigure: Bump version to 0.9.2 0.9.2
Michael Tremer [Mon, 15 Jun 2020 17:09:11 +0000 (17:09 +0000)] 
configure: Bump version to 0.9.2

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agoEnable debug logging for all tests
Michael Tremer [Mon, 15 Jun 2020 16:48:04 +0000 (16:48 +0000)] 
Enable debug logging for all tests

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agoconfigure: Update home and contact address
Michael Tremer [Mon, 15 Jun 2020 16:38:07 +0000 (16:38 +0000)] 
configure: Update home and contact address

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agoman: Update documentation on renamed --output-format parameter
Michael Tremer [Mon, 15 Jun 2020 16:33:18 +0000 (16:33 +0000)] 
man: Update documentation on renamed --output-format parameter

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agoman: Document export command
Michael Tremer [Mon, 15 Jun 2020 16:31:19 +0000 (16:31 +0000)] 
man: Document export command

Fixes: #12437
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agoman: Order commands alphabetically on location(8)
Michael Tremer [Mon, 15 Jun 2020 16:23:14 +0000 (16:23 +0000)] 
man: Order commands alphabetically on location(8)

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agolocation: Fix handling of families argument
Michael Tremer [Fri, 12 Jun 2020 14:44:20 +0000 (14:44 +0000)] 
location: Fix handling of families argument

This was ignored, because it was already converted before
and the export function was expecting a string.

Fixes: #12436
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agoFix timezone-dependant interpretation of database timestamp
Michael Tremer [Fri, 12 Jun 2020 14:33:48 +0000 (14:33 +0000)] 
Fix timezone-dependant interpretation of database timestamp

The timestamp that is retrieved from DNS is in UTC and we must
not interpret as local time.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agopython: Correctly set and use __version__
Michael Tremer [Fri, 12 Jun 2020 14:21:55 +0000 (14:21 +0000)] 
python: Correctly set and use __version__

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agodebian: Also build for sid
Stefan Schantl [Wed, 10 Jun 2020 15:28:17 +0000 (17:28 +0200)] 
debian: Also build for sid

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agoPrint ASes in format like: AS204867 - Lightning Wire Labs GmbH
Michael Tremer [Tue, 9 Jun 2020 11:03:03 +0000 (11:03 +0000)] 
Print ASes in format like: AS204867 - Lightning Wire Labs GmbH

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agodebian: Disable perl package.
Stefan Schantl [Mon, 8 Jun 2020 16:35:19 +0000 (18:35 +0200)] 
debian: Disable perl package.

The perl package cannot be cross-compiled correctly for other
architectures different from the hosts one.

So currently disable the perl package until we get this fixed at a later
time.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agoimporter: Return the last block, too
Michael Tremer [Tue, 9 Jun 2020 10:59:04 +0000 (10:59 +0000)] 
importer: Return the last block, too

Fixes: #12427
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agodebian: Create separate package for the importer
Michael Tremer [Fri, 5 Jun 2020 13:08:01 +0000 (13:08 +0000)] 
debian: Create separate package for the importer

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agodebian: build.sh: Stop printing every command that is executed
Michael Tremer [Fri, 5 Jun 2020 11:22:29 +0000 (11:22 +0000)] 
debian: build.sh: Stop printing every command that is executed

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agodebian: Do not destroy the build result
Michael Tremer [Fri, 5 Jun 2020 11:22:09 +0000 (11:22 +0000)] 
debian: Do not destroy the build result

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agodebian: unstable is called bullseye
Michael Tremer [Fri, 5 Jun 2020 11:20:30 +0000 (11:20 +0000)] 
debian: unstable is called bullseye

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agodatabase: Fix broken search for networks with flags
Michael Tremer [Fri, 5 Jun 2020 10:01:47 +0000 (10:01 +0000)] 
database: Fix broken search for networks with flags

The search was ended after the first network. No matter if
it matched, or not.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agoexport flagged networks with their faked country names, too
Michael Tremer [Fri, 5 Jun 2020 09:57:41 +0000 (09:57 +0000)] 
export flagged networks with their faked country names, too

This will lead to some networks showing up twice. Once with
their real country and once with their faked one.

It is likely that the first one will match.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agolocation(8): Export all countries by default
Michael Tremer [Fri, 5 Jun 2020 09:47:36 +0000 (09:47 +0000)] 
location(8): Export all countries by default

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agoAdd option to iterate over all countries and print them to the console
Michael Tremer [Fri, 5 Jun 2020 09:41:28 +0000 (09:41 +0000)] 
Add option to iterate over all countries and print them to the console

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agoMakefile: Remove accidentially committed hacks for Debian
Michael Tremer [Thu, 4 Jun 2020 10:37:50 +0000 (10:37 +0000)] 
Makefile: Remove accidentially committed hacks for Debian

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agodebian: Mark all packages as "Multi-Arch: same"
Michael Tremer [Thu, 4 Jun 2020 10:37:28 +0000 (10:37 +0000)] 
debian: Mark all packages as "Multi-Arch: same"

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agodebian: Correctly define runtime dependencies for all packages
Michael Tremer [Thu, 4 Jun 2020 10:31:31 +0000 (10:31 +0000)] 
debian: Correctly define runtime dependencies for all packages

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agodebian: Rename main package to "location"
Michael Tremer [Thu, 4 Jun 2020 09:54:18 +0000 (09:54 +0000)] 
debian: Rename main package to "location"

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agodebian: Update rootfiles
Michael Tremer [Thu, 4 Jun 2020 08:39:02 +0000 (08:39 +0000)] 
debian: Update rootfiles

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agopo: Import translation strings
Michael Tremer [Wed, 3 Jun 2020 19:08:00 +0000 (19:08 +0000)] 
po: Import translation strings

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agoMerge location-exporter(8) into location(8)
Michael Tremer [Wed, 3 Jun 2020 18:36:28 +0000 (18:36 +0000)] 
Merge location-exporter(8) into location(8)

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agoRename location-query(8) to location(8)
Michael Tremer [Wed, 3 Jun 2020 17:21:31 +0000 (17:21 +0000)] 
Rename location-query(8) to location(8)

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agodownloader: Change user-agent to location
Michael Tremer [Wed, 3 Jun 2020 17:15:27 +0000 (17:15 +0000)] 
downloader: Change user-agent to location

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agosystemd: Rename location-downloader to location-update
Michael Tremer [Wed, 3 Jun 2020 17:14:37 +0000 (17:14 +0000)] 
systemd: Rename location-downloader to location-update

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agolocation-downloader: Merge man page into location-query
Michael Tremer [Wed, 3 Jun 2020 17:10:35 +0000 (17:10 +0000)] 
location-downloader: Merge man page into location-query

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
8 months agoMove location-downloader functionality into location-query
Michael Tremer [Wed, 3 Jun 2020 17:06:13 +0000 (17:06 +0000)] 
Move location-downloader functionality into location-query

The commands are very long and confusion. Hence we merge this
all into one command.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>