]>
git.ipfire.org Git - people/ms/libloc.git/log
Michael Tremer [Fri, 5 Jun 2020 09:57:41 +0000 (09:57 +0000)]
export flagged networks with their faked country names, too
This will lead to some networks showing up twice. Once with
their real country and once with their faked one.
It is likely that the first one will match.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 5 Jun 2020 09:47:36 +0000 (09:47 +0000)]
location(8): Export all countries by default
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 5 Jun 2020 09:41:28 +0000 (09:41 +0000)]
Add option to iterate over all countries and print them to the console
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 4 Jun 2020 10:37:50 +0000 (10:37 +0000)]
Makefile: Remove accidentially committed hacks for Debian
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 4 Jun 2020 10:37:28 +0000 (10:37 +0000)]
debian: Mark all packages as "Multi-Arch: same"
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 4 Jun 2020 10:31:31 +0000 (10:31 +0000)]
debian: Correctly define runtime dependencies for all packages
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 4 Jun 2020 09:54:18 +0000 (09:54 +0000)]
debian: Rename main package to "location"
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 4 Jun 2020 08:39:02 +0000 (08:39 +0000)]
debian: Update rootfiles
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 3 Jun 2020 19:08:00 +0000 (19:08 +0000)]
po: Import translation strings
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 3 Jun 2020 18:36:28 +0000 (18:36 +0000)]
Merge location-exporter(8) into location(8)
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 3 Jun 2020 17:21:31 +0000 (17:21 +0000)]
Rename location-query(8) to location(8)
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 3 Jun 2020 17:15:27 +0000 (17:15 +0000)]
downloader: Change user-agent to location
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 3 Jun 2020 17:14:37 +0000 (17:14 +0000)]
systemd: Rename location-downloader to location-update
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 3 Jun 2020 17:10:35 +0000 (17:10 +0000)]
location-downloader: Merge man page into location-query
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 3 Jun 2020 17:06:13 +0000 (17:06 +0000)]
Move location-downloader functionality into location-query
The commands are very long and confusion. Hence we merge this
all into one command.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 3 Jun 2020 16:33:44 +0000 (16:33 +0000)]
location-exporter: Warn, but do not fail on invalid input
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 3 Jun 2020 16:31:44 +0000 (16:31 +0000)]
location-exporter: Do not mistake country AS for an AS number
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 3 Jun 2020 16:15:24 +0000 (16:15 +0000)]
location-query: Require at least one flag
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 1 Jun 2020 18:23:50 +0000 (18:23 +0000)]
Makefile: Remove Python path overrides for Debian
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 1 Jun 2020 13:47:44 +0000 (13:47 +0000)]
location-downloader: Do not change content of open database files
The database might be opened by another process. When modified,
it will return random results.
Fixes: #12420
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 28 May 2020 14:17:00 +0000 (14:17 +0000)]
test: Generate random temporary files
Debian runs the testsuite in parallel which fails when mutliple
process write to the same file at the same time.
This patch will now create and later delete a temporary file.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 25 May 2020 08:22:29 +0000 (08:22 +0000)]
Bump version to 0.9.1
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 21 May 2020 18:11:32 +0000 (18:11 +0000)]
Revert "debian: Drop perl package"
This reverts commit
3d26288681601b273f034de1f2241dc4342791b5 .
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Stefan Schantl [Thu, 21 May 2020 18:09:55 +0000 (20:09 +0200)]
debian: Enable location-download.timer after installation
This timer periodly executes the location-downloader and keeps
the location database up to date.
Fixes #12412.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Stefan Schantl [Thu, 21 May 2020 17:38:11 +0000 (19:38 +0200)]
debian: Ship database signing key.
This key is needed to download and verify the database.
Fixes #12410.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Stefan Schantl [Thu, 21 May 2020 17:15:19 +0000 (19:15 +0200)]
examples/create-database.py: Add hack to allow loading the module in build environment.
This script is needed to generate the database which is used by the
testsuite of the perl binding of libloc.
This allows us to revert the following commits:
*
482eb94f165507ffcb6503e162535064d6dcecac
*
3d26288681601b273f034de1f2241dc4342791b5
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 19 May 2020 17:56:53 +0000 (17:56 +0000)]
debian: Do not add auto-generated files
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 19 May 2020 17:54:47 +0000 (17:54 +0000)]
man: Document quiet mode switch
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 19 May 2020 17:50:35 +0000 (17:50 +0000)]
location: Print country name if possible
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 19 May 2020 17:44:59 +0000 (17:44 +0000)]
location-importer: Implement importing/exporting countries
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 19 May 2020 17:38:18 +0000 (17:38 +0000)]
location-query: Fix typo in "aut-num"
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 19 May 2020 16:53:53 +0000 (16:53 +0000)]
stringpool: Always add an empty string
This will create an empty string at the first byte of the pool
and is helpful for our database metadata (vendor, license, de-
scription) which can be zero and would therefore return the
first string in the pool.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 19 May 2020 12:22:50 +0000 (12:22 +0000)]
format: Reduce the length of the signature header
Since a single signature can only be up to 2048 bytes long,
we do not need 32 bits to tell us how many of those bytes
are being used.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 19 May 2020 12:14:11 +0000 (12:14 +0000)]
test: Sign database twice with the same key
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 19 May 2020 12:11:37 +0000 (12:11 +0000)]
python: Support passing two signing keys
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 19 May 2020 12:03:05 +0000 (12:03 +0000)]
database: Add support for two signatures
This allows us to sign the database with two different keys
in case the first key gets weakened or compromised in any other
way.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 19 May 2020 11:48:48 +0000 (11:48 +0000)]
debian: Build a source package, too
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 19 May 2020 11:47:44 +0000 (11:47 +0000)]
debian: Remove code that would have built a native package
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 19 May 2020 11:47:08 +0000 (11:47 +0000)]
debian: Build each build in their own directory
Otherwise we will have all packages in one large directory
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 19 May 2020 11:44:58 +0000 (11:44 +0000)]
debian: Build for unstable
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 19 May 2020 11:33:29 +0000 (11:33 +0000)]
debian: Do not build for riscv64
The build requirements could not be satisfied
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 19 May 2020 11:28:11 +0000 (11:28 +0000)]
debian: Setup build environment only once per release
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 19 May 2020 11:27:46 +0000 (11:27 +0000)]
debian: Drop perl package
This has runtime requirements and won't build when there is
no functional Python module
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 18 May 2020 14:54:24 +0000 (14:54 +0000)]
debian: Add script to build package for various arches
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 18 May 2020 14:53:56 +0000 (14:53 +0000)]
debian: Fix rootfiles
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 18 May 2020 13:47:18 +0000 (13:47 +0000)]
database: Clear signature length before feeding header into the hash function
Fixes: #12399
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 18 May 2020 13:46:53 +0000 (13:46 +0000)]
database: Check if we could read the full header
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 18 May 2020 13:45:52 +0000 (13:45 +0000)]
hexdump: Show how much data we are dumping
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 18 May 2020 13:02:33 +0000 (13:02 +0000)]
Revert "Tests: Disable testing for valid signature"
This reverts commit
25deb4d89b7b3f3af6022472392ac26b99af2462 .
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 16 May 2020 13:57:13 +0000 (13:57 +0000)]
location-downloader: Load the right database version
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 16 May 2020 10:59:26 +0000 (10:59 +0000)]
test-database: Check opening files with no or random data
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 16 May 2020 10:52:18 +0000 (10:52 +0000)]
Fix reading database in newer version
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 16 May 2020 10:33:05 +0000 (10:33 +0000)]
Do not attempt freeing the stringpool when it isn't allocated
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 16 May 2020 10:24:52 +0000 (10:24 +0000)]
Bump database version to "1"
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 16 May 2020 10:05:34 +0000 (10:05 +0000)]
location-importer: Remove unused regular expression
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 16 May 2020 10:05:12 +0000 (10:05 +0000)]
location-importer: Abort when we could not get a greeting from route server
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 15 May 2020 14:00:20 +0000 (14:00 +0000)]
location-query: Add command to show the database version
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 15 May 2020 13:54:46 +0000 (13:54 +0000)]
location-impoter: Allow overriding country by AS override
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 15 May 2020 12:56:33 +0000 (12:56 +0000)]
location-importer: Fix typo in "aut-num"
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 15 May 2020 12:29:07 +0000 (12:29 +0000)]
location-importer: Skip everything with /0
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 15 May 2020 12:08:52 +0000 (12:08 +0000)]
location-query: Include flags in dump output
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 15 May 2020 12:00:53 +0000 (12:00 +0000)]
location-query: Show flags in lookup output
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 15 May 2020 11:48:01 +0000 (11:48 +0000)]
location-importer: Remove some debugging output
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 15 May 2020 11:02:28 +0000 (11:02 +0000)]
location-downloader: Give the database a proper name
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 15 May 2020 11:01:38 +0000 (11:01 +0000)]
Add quiet mode
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 15 May 2020 10:58:25 +0000 (10:58 +0000)]
python: Correctly set log level for root logger
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 15 May 2020 10:50:27 +0000 (10:50 +0000)]
.gitignore: Omit *.db and *.db.xz from git status output
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 15 May 2020 10:49:38 +0000 (10:49 +0000)]
location-importer: Fix SQL syntax error
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 15 May 2020 09:06:22 +0000 (09:06 +0000)]
location-importer: Speed up exporting the database
This optimisation returns the same result, but in about
one fifteenth of the time than the previous version of
the query.
This can also be parallelised by PostgreSQL so that more
CPU cores will decrease the time it takes to generate
the network list.
On my system this went down from 75 to 5 minutes.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 14 May 2020 16:19:41 +0000 (16:19 +0000)]
location-import: Abort when we could not parse the prefix
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 13 May 2020 20:49:57 +0000 (20:49 +0000)]
resolv: Silence compiler warning about unused variable
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Stefan Schantl [Tue, 12 May 2020 11:52:21 +0000 (13:52 +0200)]
Debian: Require lightwight python development packages
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Stefan Schantl [Tue, 12 May 2020 15:07:04 +0000 (17:07 +0200)]
Makefile.am: Specify CC and LD when compiling perl module.
This fixes the issues when cross compiling libloc and the perl binding.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Peter Müller [Wed, 13 May 2020 19:33:54 +0000 (19:33 +0000)]
location-importer: DELETE multicast and "reserved for future usage" address space
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Peter Müller [Wed, 13 May 2020 19:33:32 +0000 (19:33 +0000)]
location-importer: DELETE "current network" address space
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Peter Müller [Wed, 13 May 2020 19:33:13 +0000 (19:33 +0000)]
location-importer: DELETE other unroutable IPv4 networks
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Peter Müller [Wed, 13 May 2020 19:32:51 +0000 (19:32 +0000)]
location-importer: DELETE local loopback address space
This should never appear in public BGP data, but since people tend to do
really strange things on the interent, it's better to delete 127.0.0.0/8
data, just in case.
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 13 May 2020 19:50:22 +0000 (19:50 +0000)]
location-importer: Some routers omit /24 for IPv4
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 13 May 2020 19:50:07 +0000 (19:50 +0000)]
location-query: Print all information we have
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 13 May 2020 17:40:58 +0000 (17:40 +0000)]
python: Use override flags when exporting the database
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 13 May 2020 17:30:05 +0000 (17:30 +0000)]
python: Use closest match for countries
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 13 May 2020 17:27:22 +0000 (17:27 +0000)]
python: Use overridden AS name when possible
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 13 May 2020 17:20:25 +0000 (17:20 +0000)]
python: Implement writing database content to file
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 13 May 2020 17:19:31 +0000 (17:19 +0000)]
python: Implement importing override files into the database
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 13 May 2020 16:38:38 +0000 (16:38 +0000)]
python: Permit passing an empty private key
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 13 May 2020 16:38:09 +0000 (16:38 +0000)]
location-query: Add command to dump the whole database
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 13 May 2020 16:37:15 +0000 (16:37 +0000)]
Fix searching with an empty string
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 13 May 2020 15:06:02 +0000 (15:06 +0000)]
python: Implement importing BGP announcements from route servers
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 13 May 2020 11:21:23 +0000 (11:21 +0000)]
python: Drop unused list of invalid IP addresses
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 12 May 2020 16:44:20 +0000 (16:44 +0000)]
python: Drop importing autnums from extended sources
There are 1:N mappings in there and we lack any additional context to
resolve them.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 12 May 2020 15:58:53 +0000 (15:58 +0000)]
python: Parse classic RIR data in one transaction and aggregate asnums
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 12 May 2020 14:57:51 +0000 (14:57 +0000)]
python: Import extended WHOIS data
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 12 May 2020 13:20:42 +0000 (13:20 +0000)]
python: Import classic RIR importer from database repository
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 12 May 2020 11:42:54 +0000 (11:42 +0000)]
python: Add database driver for PostgreSQL
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 12 May 2020 11:16:14 +0000 (11:16 +0000)]
python: Import downloader from database repository
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 12 May 2020 10:11:34 +0000 (10:11 +0000)]
python: Add scaffolding for an importer tool
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 12 May 2020 10:04:50 +0000 (10:04 +0000)]
python: Move common i18n code into python module
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 12 May 2020 10:01:59 +0000 (10:01 +0000)]
python: Move common logging code into module
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 12 May 2020 10:00:42 +0000 (10:00 +0000)]
python: Expend VERSION at build time
This is now easier than importing __version__ from the
C module.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 12 May 2020 09:49:35 +0000 (09:49 +0000)]
python: Move C module to make space for some native python code
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>