location-exporter: Warn, but do not fail on invalid input

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

location-exporter: Do not mistake country AS for an AS number

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

location-query: Require at least one flag

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Makefile: Remove Python path overrides for Debian

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

location-downloader: Do not change content of open database files

The database might be opened by another process. When modified,
it will return random results.

Fixes: #12420
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

test: Generate random temporary files

Debian runs the testsuite in parallel which fails when mutliple
process write to the same file at the same time.

This patch will now create and later delete a temporary file.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Bump version to 0.9.1

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Revert "debian: Drop perl package"

This reverts commit 3d26288681601b273f034de1f2241dc4342791b5.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

debian: Enable location-download.timer after installation

This timer periodly executes the location-downloader and keeps
the location database up to date.

Fixes #12412.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

debian: Ship database signing key.

This key is needed to download and verify the database.

Fixes #12410.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

examples/create-database.py: Add hack to allow loading the module in build environment.

This script is needed to generate the database which is used by the
testsuite of the perl binding of libloc.

This allows us to revert the following commits:

* 482eb94f165507ffcb6503e162535064d6dcecac
* 3d26288681601b273f034de1f2241dc4342791b5

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

debian: Do not add auto-generated files

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

man: Document quiet mode switch

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

location: Print country name if possible

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

location-importer: Implement importing/exporting countries

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

location-query: Fix typo in "aut-num"

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

stringpool: Always add an empty string

This will create an empty string at the first byte of the pool
and is helpful for our database metadata (vendor, license, de-
scription) which can be zero and would therefore return the
first string in the pool.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

format: Reduce the length of the signature header

Since a single signature can only be up to 2048 bytes long,
we do not need 32 bits to tell us how many of those bytes
are being used.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

test: Sign database twice with the same key

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Support passing two signing keys

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

database: Add support for two signatures

This allows us to sign the database with two different keys
in case the first key gets weakened or compromised in any other
way.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

debian: Build a source package, too

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

debian: Remove code that would have built a native package

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

debian: Build each build in their own directory

Otherwise we will have all packages in one large directory

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

debian: Build for unstable

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

debian: Do not build for riscv64

The build requirements could not be satisfied

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

debian: Setup build environment only once per release

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

debian: Drop perl package

This has runtime requirements and won't build when there is
no functional Python module

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

debian: Add script to build package for various arches

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

debian: Fix rootfiles

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

database: Clear signature length before feeding header into the hash function

Fixes: #12399
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

database: Check if we could read the full header

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

hexdump: Show how much data we are dumping

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Revert "Tests: Disable testing for valid signature"

This reverts commit 25deb4d89b7b3f3af6022472392ac26b99af2462.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

location-downloader: Load the right database version

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

test-database: Check opening files with no or random data

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Fix reading database in newer version

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Do not attempt freeing the stringpool when it isn't allocated

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Bump database version to "1"

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

location-importer: Remove unused regular expression

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

location-importer: Abort when we could not get a greeting from route server

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

location-query: Add command to show the database version

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

location-impoter: Allow overriding country by AS override

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

location-importer: Fix typo in "aut-num"

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

location-importer: Skip everything with /0

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

location-query: Include flags in dump output

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

location-query: Show flags in lookup output

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

location-importer: Remove some debugging output

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

location-downloader: Give the database a proper name

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Add quiet mode

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Correctly set log level for root logger

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

.gitignore: Omit *.db and *.db.xz from git status output

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

location-importer: Fix SQL syntax error

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

location-importer: Speed up exporting the database

This optimisation returns the same result, but in about
one fifteenth of the time than the previous version of
the query.

This can also be parallelised by PostgreSQL so that more
CPU cores will decrease the time it takes to generate
the network list.

On my system this went down from 75 to 5 minutes.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

location-import: Abort when we could not parse the prefix

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

resolv: Silence compiler warning about unused variable

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Debian: Require lightwight python development packages

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Makefile.am: Specify CC and LD when compiling perl module.

This fixes the issues when cross compiling libloc and the perl binding.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

location-importer: DELETE multicast and "reserved for future usage" address space

Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

location-importer: DELETE "current network" address space

Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

location-importer: DELETE other unroutable IPv4 networks

Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

location-importer: DELETE local loopback address space

This should never appear in public BGP data, but since people tend to do
really strange things on the interent, it's better to delete 127.0.0.0/8
data, just in case.

Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

location-importer: Some routers omit /24 for IPv4

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

location-query: Print all information we have

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Use override flags when exporting the database

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Use closest match for countries

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Use overridden AS name when possible

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Implement writing database content to file

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Implement importing override files into the database

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Permit passing an empty private key

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

location-query: Add command to dump the whole database

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Fix searching with an empty string

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Implement importing BGP announcements from route servers

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Drop unused list of invalid IP addresses

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Drop importing autnums from extended sources

There are 1:N mappings in there and we lack any additional context to
resolve them.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Parse classic RIR data in one transaction and aggregate asnums

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Import extended WHOIS data

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Import classic RIR importer from database repository

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Add database driver for PostgreSQL

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Import downloader from database repository

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Add scaffolding for an importer tool

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Move common i18n code into python module

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Move common logging code into module

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Expend VERSION at build time

This is now easier than importing __version__ from the
C module.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

python: Move C module to make space for some native python code

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

location-exporter: Fix incorrect logger name

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Tests: Disable testing for valid signature

Testing for a valid database signature currently fails and breaks
the entire build process.

At a later time this very easy can be recovered by reverting this
commit.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Debian: Add missing build dependencies.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

test-database: Remove signature checks

These are performed in an extra test now

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

writer: Erase the padding

We should not leak any data from the stack into the
database and this makes debugging easier, too.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Log blocks that are being fed into the signature

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

writer: Put whole header into the hash instead of only the first 8 bytes

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

test-signature: Fix path to key

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Log the signature in debug mode

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

database: Catch any errors from EVP_DigestVerify_Update()

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

test: Add test to check that invalid signatures do not validate

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

perl: Add get_continent_code()

This function allows to get the continent code by a given country code.

Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

database: Correctly handle error codes from OpenSSL

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

location-downloader: Add command to verify the downloaded database manually

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Install databases to /var/lib/location

This data is being modified and should therefore go to
/var/lib instead of /usr/share.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>