]>
git.ipfire.org Git - location/libloc.git/log
Michael Tremer [Wed, 21 Feb 2024 17:10:49 +0000 (17:10 +0000)]
importer: Tidy up SQL queries
No functional changes. Just better readability.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 17:04:43 +0000 (17:04 +0000)]
importer: Drop previously imported AS names, too
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 17:03:58 +0000 (17:03 +0000)]
importer: Refactor ARIN importer
This patch is taking advantage of Python's CSV parser.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 17:03:27 +0000 (17:03 +0000)]
importer: Fix Python syntax error
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Peter Müller [Sun, 10 Dec 2023 19:37:00 +0000 (19:37 +0000)]
location-importer: Replace ARIN AS names source with one that offers human-readable names
This patch replaces our previous source for AS names in ARIN's realms
with another file provided by ARIN that contains human-readable names
for organizations ASNs have been allocated to.
Please note that a
TRUNCATE autnums;
is necessary on machines previously running the old version of
location-importer, in order to make use of this changed data source.
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 16:28:41 +0000 (16:28 +0000)]
tests: Make the new tests executable
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 16:27:04 +0000 (16:27 +0000)]
tests: Add more dedup/merge tests
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 16:20:44 +0000 (16:20 +0000)]
tests: Show even large diffs in dedup tests
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 16:19:57 +0000 (16:19 +0000)]
tests: Build out more dedup tests
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 16:06:13 +0000 (16:06 +0000)]
python: Implement rich comparison for Network objects
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 16:04:24 +0000 (16:04 +0000)]
python: Fix rich comparison for AS objects
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 16:03:09 +0000 (16:03 +0000)]
country: Return NULL/None for unset attributes
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 16:02:42 +0000 (16:02 +0000)]
tests: Add some tests for the Country object
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 15:54:49 +0000 (15:54 +0000)]
python: Fix rich comparison function for Country
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 15:54:30 +0000 (15:54 +0000)]
python: Make Country hashable
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 15:09:04 +0000 (15:09 +0000)]
tests: Add a simple test for deduplication
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 14:57:02 +0000 (14:57 +0000)]
network: Tidy up some code
No functional changes.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 14:48:08 +0000 (14:48 +0000)]
network-tree: Use the raw prefix to place networks onto the tree
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 14:45:43 +0000 (14:45 +0000)]
network-tree: Run deduplication once for each family
This probably never becomes a real problem, but we did not clear the
stack properly once we switched family when iterating through the
entire tree in one go.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 13:55:06 +0000 (13:55 +0000)]
tools: Import the copy script
This is useful to debug the library by copying one library 1:1 and then
diffing the result.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 13:53:20 +0000 (13:53 +0000)]
network-tree: Split into a separate file
There are no functional changes here, but I find this code easier to
handle if it is split across smaller files.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Peter Müller [Sat, 17 Feb 2024 22:31:00 +0000 (22:31 +0000)]
location-importer: Fix Spamhaus ASN-DROP parsing
The format of this list has changed, from a plain text file with a
customer schema to JSON. Adjust our routines accordingly to make use of
this list again.
The second version of this patch incorporates Michael's feedback on the
first version, and adds AS names to the autnums table in case they are
not there already, which closes some gaps on rogue ASNs in the LACNIC
area.
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Tested-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Peter Müller [Sat, 17 Feb 2024 22:39:00 +0000 (22:39 +0000)]
location-importer: Add missing area code for AWS
It is unclear what exactly "eusc" refers to ("EU Secure Cloud", maybe?),
but according to RIPE database records for IPv6 networks involved, this
area code physically traces back to Germany, likely Berlin instead of
Frankfurt.
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 17 Feb 2024 21:18:22 +0000 (21:18 +0000)]
network: Fix deduplication not looking far enough
When deduplicating the tree, we kept comparing subnets to the largest
supernet only - which is incorrect.
Instead, we need to compare subnets to the next largest supernet which
is now implemented in this patch.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 17 Feb 2024 21:12:32 +0000 (21:12 +0000)]
network: Fix bit length check when merging networks
The check was incorrect as it only expected perfect matches, but we can
of course also merge subnets that don't necessarily require the entire
prefix length for the start address.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sun, 7 Jan 2024 18:08:14 +0000 (18:08 +0000)]
python: Make AS objects hashable
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 31 Jul 2023 16:59:04 +0000 (16:59 +0000)]
configure: Bump version to 0.9.17
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 31 Jul 2023 16:58:26 +0000 (16:58 +0000)]
data: Update database to 2023-07-31
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 31 Jul 2023 16:55:23 +0000 (16:55 +0000)]
network: Decreate log level when deleting networks
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 31 Jul 2023 12:56:24 +0000 (12:56 +0000)]
libloc.so: Bump library revision
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Petr Písař [Mon, 24 Jul 2023 08:57:49 +0000 (10:57 +0200)]
Fix string escaping in location tool
Python 3.12 started to warn on an invalid string escape sequence:
$ location list-networks-by-as --format ipset 1
/usr/bin/location:583: SyntaxWarning: invalid escape sequence '\d'
m = re.match("^AS(\d+)$", object)
IpsetOutputWriter requires family being set
This patch fixes the warning as well as the regular expression by
using a raw string.
<https://docs.python.org/3.12/whatsnew/changelog.html#id77>
Fixes: #13188 - Invalid escape sequence with Python 3.12
Signed-off-by: Petr Písař <ppisar@redhat.com>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 17 Jul 2023 16:13:35 +0000 (16:13 +0000)]
tree: More elegantly prevent deleting the root node
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 17 Jul 2023 16:00:14 +0000 (16:00 +0000)]
networks: Delete networks from the tree on merge
Previously, the tree did not support deleting nodes, but this has now
been added to do this safely.
This saves us running another deduplication check and halves the size of
the database.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 17 Jul 2023 15:36:21 +0000 (15:36 +0000)]
Merge networks before writing the database
This should help us keeping the database more compact where we summarize
two smaller networks to one larger one if they share the same
properties.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 14 Jul 2023 16:37:42 +0000 (16:37 +0000)]
tree: Actually delete any deleted nodes
Usually we only mark nodes as deleted, but finally, we should actually
delete what we can as we keep the pointers for back referencing.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 14 Jul 2023 16:33:49 +0000 (16:33 +0000)]
writer: Cleanup networks before writing
This patch will add a function that will search the tree for duplicate
entries which are not needed and remove them.
Afterwards, any unused nodes will be deleted.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 14 Jul 2023 14:58:19 +0000 (14:58 +0000)]
tree: Add flag to delete nodes
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Valters Jansons [Thu, 13 Jul 2023 17:46:55 +0000 (20:46 +0300)]
debian: Include bash-completion when packaging
dh_missing for a standard `debuild` run on latest `master` commit
complains about the bash-completion file created for location package.
This file should be listed as for inclusion in location package
accordingly, so that auto-built Debian package picks it up properly.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 14 Jul 2023 10:26:55 +0000 (10:26 +0000)]
importer: Decrease the log level if Spamhaus' files are empty
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 14 Jul 2023 10:26:24 +0000 (10:26 +0000)]
importer: Don't write AS without names into the database
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 14 Jul 2023 10:23:31 +0000 (10:23 +0000)]
importer: AWS: Add country code of NZ for ap-southeast-5
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Valters Jansons [Wed, 12 Jul 2023 18:57:00 +0000 (21:57 +0300)]
debian: Housekeep location.install and changelog
In order to run `debuild` for an easy Debian package locally, the
debian/*.install files must list all build results. For version 0.9.16
there is a new pre-built database.db file that belongs with `location`
package, and the `man8` pages have been moved to `man1`.
Additionally, the changelog is added for version 0.9.16 as to maintain
the centralized reference (inside this upstream repository). The
author/timestamp is based on the 0.9.16 tag metadata tracked in Git.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 5 Jul 2023 09:39:35 +0000 (09:39 +0000)]
geofeed: Parse and normalize any URLs
It would be nice if we had an easy way to check if the URL is valid, but
Python does not seem to have a library function for this.
Therefore we might store invalid URLs in the database, but when making a
request to them, urllib with throw an InvalidURL error.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 5 Jul 2023 09:38:43 +0000 (09:38 +0000)]
importer: Improve performance of AS name export query
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 5 Jul 2023 09:38:06 +0000 (09:38 +0000)]
database: Log query execution time in debug mode
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 4 Jul 2023 10:17:19 +0000 (10:17 +0000)]
geofeeds: Catch any invalid URLs
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 4 Jul 2023 10:01:51 +0000 (10:01 +0000)]
geofeeds: Delete any data that did not update within two weeks
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 4 Jul 2023 09:53:41 +0000 (09:53 +0000)]
geofeeds: Delete any data on 404
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 4 Jul 2023 09:52:25 +0000 (09:52 +0000)]
importer: Log any errors to the database
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 4 Jul 2023 09:48:44 +0000 (09:48 +0000)]
importer: Catch TimeoutError when loading Geofeeds
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 4 Jul 2023 09:48:06 +0000 (09:48 +0000)]
importer: Silence invalid country code warning
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 4 Jul 2023 09:47:40 +0000 (09:47 +0000)]
importer: Skip lines we cannot decode
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 4 Jul 2023 09:38:15 +0000 (09:38 +0000)]
importer: Convert country codes to uppercase from Geofeeds
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 4 Jul 2023 09:36:50 +0000 (09:36 +0000)]
importer: Silently ignore any empty country codes in Geofeeds
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 4 Jul 2023 09:32:09 +0000 (09:32 +0000)]
importer: Drop method to import routing information from route servers
telnetlib in Python is deprecated and will be removed in Python 3.13.
Since we don't use this feature any more, this patch removes it
entirely.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 3 Jul 2023 15:23:48 +0000 (15:23 +0000)]
Merge remote-tracking branch 'origin/geofeed'
Michael Tremer [Mon, 3 Jul 2023 15:05:13 +0000 (15:05 +0000)]
export: Raise an error when trying to export ipset for both families
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 3 Jul 2023 14:53:50 +0000 (14:53 +0000)]
location: Fix correct set name when family is selected
This does however not fix the problem if no --family= parameter is being
given because we will then export for multiple families in one file.
This is for example not supported for ipset.
Fixes: #12897
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 4 Mar 2023 10:52:51 +0000 (10:52 +0000)]
network: Drop an unused function to count all networks
Reported-by: Gisle Vanem <gvanem@online.no>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 28 Feb 2023 11:42:50 +0000 (11:42 +0000)]
Makefile: Make sure to ship bash completion files even when disabled
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Stefan Schantl [Fri, 24 Feb 2023 16:22:06 +0000 (17:22 +0100)]
Install bash-completion files.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Hans-Christoph Steiner [Fri, 24 Feb 2023 16:22:05 +0000 (17:22 +0100)]
Add bash-completion file for the location command.
Signed-off-by: Stefan Schantl <stefan.schantl@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Temuri Doghonadze [Wed, 22 Feb 2023 14:53:39 +0000 (14:53 +0000)]
po: Add Georgian translation
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 22 Feb 2023 14:50:03 +0000 (14:50 +0000)]
po: Update translations
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 2 Dec 2022 11:06:01 +0000 (11:06 +0000)]
importer: Fix parsing gzipped content on invalid Content-Type header
RIPE seems to have misconfigured their webserver which now sends
application/octet-stream for all gzipped files instead of
application/x-gzip or similar which is what the importer would expect.
Instead of only checking the content type, we will now test whether we
see the gzip magic and try to decompress the file then.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 29 Oct 2022 13:25:36 +0000 (13:25 +0000)]
configure: Bump version to 0.9.16
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 29 Oct 2022 13:23:08 +0000 (13:23 +0000)]
configure: Drop superfluous bracket
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Petr Písař [Tue, 4 Oct 2022 11:51:21 +0000 (13:51 +0200)]
Install Perl files to Perl vendor directory
On Fedora, the original code placed files into /usr/lib64/perl5/5.36/.
That location is not recognized by Fedora perl.
It also supresses installation and uninstallaion of packlist and perlocal
inventory files. They cannot be reliably uninstalled without
corrupting them.
Signed-off-by: Petr Písař <ppisar@redhat.com>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 29 Oct 2022 13:12:23 +0000 (13:12 +0000)]
database: Fix check if a signature is set
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 29 Oct 2022 13:12:06 +0000 (13:12 +0000)]
writer: Assign correct file descriptor for private keys
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 29 Oct 2022 13:11:48 +0000 (13:11 +0000)]
writer: Fix typo in signature variable
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 21 Oct 2022 10:42:41 +0000 (10:42 +0000)]
database: Correct error code on verification
If the first signature check returned an error, the error code was
overwritten by the second signature check, returning an overall
incorrect result.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 21 Oct 2022 10:26:25 +0000 (10:26 +0000)]
Create the database with read-only permissions for everyone
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Petr Písař [Thu, 6 Oct 2022 11:28:51 +0000 (13:28 +0200)]
Move location manual from section 8 to section 1 in location-update.service
It's a tool a non-superuser can use. It's installed into /usr/bin.
Signed-off-by: Petr Písař <ppisar@redhat.com>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Petr Písař [Tue, 4 Oct 2022 14:16:07 +0000 (16:16 +0200)]
Remove shebangs from Python modules
Shebangs only belong to standalone executables.
Fixes: #12956
Signed-off-by: Petr Písař <ppisar@redhat.com>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Petr Písař [Thu, 6 Oct 2022 11:28:51 +0000 (13:28 +0200)]
Move location manual from section 8 to section 1
It's a tool a non-superuser can use. It's installed into /usr/bin.
Fixes: #12957
Signed-off-by: Petr Písař <ppisar@redhat.com>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 20 Oct 2022 13:35:55 +0000 (13:35 +0000)]
tests: Drop the test database and use the distributed one
Fixes: #12958
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 20 Oct 2022 13:23:14 +0000 (13:23 +0000)]
database: Ship a recent default database
The database can easily be updated using "make update-database" which
will have to be executed before every release.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Petr Písař [Wed, 19 Oct 2022 09:50:23 +0000 (11:50 +0200)]
Declare make dependencies for Perl binding
Running make in parallel (make -j5) randomly failed with:
/bin/sh ./libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -include ./config.h -DSYSCONFDIR=\""/etc"\" -I./src -Wall -Wchar-subscripts -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wpointer-arith -Wshadow -Wsign-compare -Wstrict-prototypes -Wtype-limits -fno-semantic-interposition -ffunction-sections -fdata-sections -DLIBLOC_PRIVATE -fvisibility=hidden -g -O2 -c -o src/libloc_internal_la-database.lo `test -f 'src/database.c' || echo './'`src/database.c
/usr/bin/ld: cannot find -lloc: No such file or directory
collect2: error: ld returned 1 exit status
make[3]: *** [Makefile:482: blib/arch/auto/Location/Location.so] Error 1
make[2]: *** [Makefile:2982: build-perl] Error 2
That's because build-perl did not depend on libloc library.
A similar issue was with running tests and installing files. Moreover,
thess two targets rebuilt Perl binding from scratch.
This patch make the dependencies explicit and turns build-perl into
non-PHONY.
Fixes: #12961
Signed-off-by: Petr Písař <ppisar@redhat.com>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 17 Oct 2022 13:56:49 +0000 (13:56 +0000)]
Revert "perl: Remove RPATH"
This reverts commit
fb574759cbd3a60e64ed2674f514037b9085f078 .
Suggested-by: Petr Pisar <ppisar@redhat.com>
Fixes: #12955
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 4 Oct 2022 08:47:03 +0000 (08:47 +0000)]
export: Fix generating file names for ipset output
The tag that is being used in the filename was set after creating the
filename. The order has now been fixed and the export works fine for me.
Fixes: #12944
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 1 Oct 2022 11:56:31 +0000 (11:56 +0000)]
Makefile: Reset Python path
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 27 Sep 2022 08:49:56 +0000 (08:49 +0000)]
importer: Validate country codes from Geofeeds
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 27 Sep 2022 08:05:11 +0000 (08:05 +0000)]
importer: Ensure that we only use HTTPS URLs for Geofeeds
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 27 Sep 2022 08:03:23 +0000 (08:03 +0000)]
importer: Fix reading Geofeeds from remarks
Only RIPE has a dedicated geofeed field. For others, this data needs to
be read from the remarks section.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 27 Sep 2022 08:01:23 +0000 (08:01 +0000)]
importer: Add a search index match geofeed networks quicker
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 27 Sep 2022 08:00:59 +0000 (08:00 +0000)]
importer: Use a GIST index for networks from geofeeds
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 27 Sep 2022 08:00:37 +0000 (08:00 +0000)]
importer: Use geofeeds for country assignment
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 27 Sep 2022 07:59:37 +0000 (07:59 +0000)]
importer: Sync geofeeds
Geofeeds are kept in a separate table to only fetch them once per URL.
This needs to be kept in sync which is done before we update any feeds.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 27 Sep 2022 07:58:41 +0000 (07:58 +0000)]
importer: Just fetch any exception from the executor
Any exceptions will only be raised in the main thread when they are
fetched from the executor. We do not need to print any other return
values here.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 26 Sep 2022 23:11:16 +0000 (23:11 +0000)]
importer: Add command to import geofeeds into the database
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 26 Sep 2022 22:05:45 +0000 (22:05 +0000)]
importer: Store geofeed URLs from RIR data
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 27 Sep 2022 16:28:44 +0000 (16:28 +0000)]
writer: Add an empty string to the stringpool
This is useful when we have some offset that isn't initialised. It will
point to the start of the string pool which is some random string that
happened to be added first.
An empty string at least signifies better that something is unset.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 26 Sep 2022 23:10:43 +0000 (23:10 +0000)]
python: Export __version__ in location module
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 27 Sep 2022 09:54:45 +0000 (09:54 +0000)]
configure: Fix incorrect database path
Reported-by: Jochen Sprickerhof <jochen@sprickerhof.de>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 27 Sep 2022 09:20:05 +0000 (09:20 +0000)]
importer: Fix potential SQL command injection
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Peter Müller [Mon, 26 Sep 2022 18:33:29 +0000 (18:33 +0000)]
location-importer.in: Add new Amazon region codes
Note that the precise location of "ap-southeast-6" is currently
undocumented. However, all IPv4 networks allocated are not routed at the
time of writing, hence our allocation of Asia/Pacific should be
acceptable for the time being.
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Peter Müller [Mon, 26 Sep 2022 18:26:44 +0000 (18:26 +0000)]
location-importer.in: Conduct sanity checks per DROP list
Previously, the lack of distinction between different DROP lists caused
only the last one to be persisted. The second version of this patch
incorporates suggestions from Michael on the first version.
Tested-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 26 Sep 2022 15:39:58 +0000 (15:39 +0000)]
configure: Bump version to 0.9.15
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Tue, 13 Sep 2022 12:06:24 +0000 (12:06 +0000)]
python: Add new open() interface to easily open a database
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>