]>
git.ipfire.org Git - location/libloc.git/log
Michael Tremer [Mon, 4 Mar 2024 12:06:33 +0000 (12:06 +0000)]
importer: No longer import Geofeeds concurrently
This is kind of a problem since many servers have gone away, respond
very slowly, etc. But since we are moving towards psycopg3, we cannot do
this the old way any more.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 4 Mar 2024 11:47:10 +0000 (11:47 +0000)]
importer: Drop the geofeed sources when updating RIR data
This is a cleaner way to drop any previously imported content and should
also be faster because we can drop a lot of DELETE staments for objects
without Geofeeds.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 4 Mar 2024 11:42:04 +0000 (11:42 +0000)]
importer: Create a better structure to import RIRs
All information about all RIRs will now be imported in one large
database transaction per RIR which should bring us better integrity and
help us to phase out any stale data.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 4 Mar 2024 10:29:38 +0000 (10:29 +0000)]
importer: Add option to only import specific RIRs
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 4 Mar 2024 10:23:49 +0000 (10:23 +0000)]
importer: Also import networks that are smaller than /48 or /24
These cannot appear in the global routing table, but that does not mean
that there isn't any value in importing them into our database.
In case they are just noise, we will filter them out later on.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 4 Mar 2024 10:22:22 +0000 (10:22 +0000)]
importer: Improve checks for unspecified networks
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 4 Mar 2024 10:19:51 +0000 (10:19 +0000)]
importer: Improve check for network objects
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 4 Mar 2024 10:12:42 +0000 (10:12 +0000)]
importer: Change country code logic
If we have imported something without valid country codes, we still need
to import that network object into the database. Further information
could come from a Geofeed. Otherwise we still want to have this network
here for later processing.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 4 Mar 2024 09:56:48 +0000 (09:56 +0000)]
importer: Skip any countries that we don't know
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 4 Mar 2024 09:52:38 +0000 (09:52 +0000)]
importer: When storing country codes, make the code more straight-forward
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 4 Mar 2024 09:50:44 +0000 (09:50 +0000)]
importer: Return known countries as a set()
It should be possible to search through sets quicker.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 4 Mar 2024 09:49:43 +0000 (09:49 +0000)]
importer: Make translating country codes more extensible
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 4 Mar 2024 09:48:19 +0000 (09:48 +0000)]
importer: Ignore certain country codes
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 4 Mar 2024 09:41:14 +0000 (09:41 +0000)]
importer: Fail if no countries have been imported, yet
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 4 Mar 2024 09:34:14 +0000 (09:34 +0000)]
importer: Remove superfluous function call
Normally a function call doesn't have that much overhead in Python, but
since we are calling all these that many times I am sure this is all
adding up.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 2 Mar 2024 19:53:49 +0000 (19:53 +0000)]
importer: Use the downloader to import Geofeeds
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 2 Mar 2024 11:35:37 +0000 (11:35 +0000)]
importer: Drop any data from feeds we no longer support
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 2 Mar 2024 11:26:43 +0000 (11:26 +0000)]
importer: Drop source field from overrides table
Since we now import all feeds into a separate table including their own
source, we can drop this from the *_overrides tables as we only import
our own data here.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 2 Mar 2024 11:20:13 +0000 (11:20 +0000)]
importer: Refactor parsing Spamhaus ASNDROP
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 2 Mar 2024 11:13:53 +0000 (11:13 +0000)]
importer: Refactor parsing Spamhaus DROP feeds
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 2 Mar 2024 11:00:26 +0000 (11:00 +0000)]
importer: Add the option to only update one feed
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 2 Mar 2024 10:54:08 +0000 (10:54 +0000)]
importer: Completely rewrite the AWS parser
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 2 Mar 2024 10:43:42 +0000 (10:43 +0000)]
importer: Reformat AWS dictionary
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 2 Mar 2024 10:36:00 +0000 (10:36 +0000)]
importer: Simplify fetching countries
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 2 Mar 2024 10:28:02 +0000 (10:28 +0000)]
importer: Refactor feed parsing
This adds a bit of common code across all feeds, but is only implemented
for AWS at the moment.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 2 Mar 2024 10:11:09 +0000 (10:11 +0000)]
importer: Create a new subcommand to import feeds
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 2 Mar 2024 09:57:40 +0000 (09:57 +0000)]
importer: Reformat the large SQL query
There are no functional changes here. This is just to be able to extend
the script more easily.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 2 Mar 2024 09:57:25 +0000 (09:57 +0000)]
importer: Fix typo in SQL query
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 2 Mar 2024 09:50:23 +0000 (09:50 +0000)]
importer: Import AWS IP feed into the new feeds table
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 2 Mar 2024 09:45:19 +0000 (09:45 +0000)]
importer: Import Spamhaus DROP feeds into the new feeds table
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 2 Mar 2024 09:43:25 +0000 (09:43 +0000)]
importer: Don't import any AS names from Spamhaus
Those are not very human readable and we should usually have a better
source.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 2 Mar 2024 09:42:29 +0000 (09:42 +0000)]
importer: Create a feeds table for networks, too
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 2 Mar 2024 09:36:05 +0000 (09:36 +0000)]
importer: Import the Spamhaus ASN feed into the new feeds table
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 2 Mar 2024 09:23:30 +0000 (09:23 +0000)]
importer: Create an extra table for feeds
We are currently using the overrides table to import any third-party
feeds. That creates the problem that those tables can only hold one row
for each AS which we cannot easily merge without losing the source of
the information.
Therefore this patch creates a new table which allow us to store this
information per feed and we will use that information if there is no
overrides information.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 23 Feb 2024 15:36:44 +0000 (15:36 +0000)]
importer: Pass downloader to the ARIN import
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 23 Feb 2024 14:08:49 +0000 (14:08 +0000)]
configure: Don't check for Lua when --disable-lua is passed
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 22 Feb 2024 16:25:21 +0000 (16:25 +0000)]
lua: network: Implement checking flags
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 22 Feb 2024 16:20:38 +0000 (16:20 +0000)]
lua: Export network flags
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 22 Feb 2024 16:16:53 +0000 (16:16 +0000)]
lua: Export default database path
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 22 Feb 2024 16:11:56 +0000 (16:11 +0000)]
configure: Check for luaunit
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 22 Feb 2024 15:55:16 +0000 (15:55 +0000)]
lua: database: Implement verify()
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 22 Feb 2024 15:46:57 +0000 (15:46 +0000)]
lua: database: Export description/license/vendor
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 22 Feb 2024 15:43:07 +0000 (15:43 +0000)]
lua: database: Implement fetching countries
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 22 Feb 2024 15:39:25 +0000 (15:39 +0000)]
lua: database: Implementing fetching AS objects
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 22 Feb 2024 15:31:40 +0000 (15:31 +0000)]
lua: Add AS object
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 22 Feb 2024 15:18:21 +0000 (15:18 +0000)]
lua: country: Export remaining properties
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 22 Feb 2024 15:13:41 +0000 (15:13 +0000)]
lua: Don't try to free memory that was allocated by Lua
The userdata that we have allocated cannot be freed as we don't have
control over it.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 22 Feb 2024 15:13:25 +0000 (15:13 +0000)]
lua: Force testing garbage collection for countries
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 22 Feb 2024 15:10:51 +0000 (15:10 +0000)]
lua: Create Country objects
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 22 Feb 2024 14:50:49 +0000 (14:50 +0000)]
lua: Force garbage collection to test those methods
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 22 Feb 2024 14:45:19 +0000 (14:45 +0000)]
lua: Check if we can read the network's properties
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 22 Feb 2024 14:43:20 +0000 (14:43 +0000)]
lua: Fix calling methods that belong to an object
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 22 Feb 2024 14:16:29 +0000 (14:16 +0000)]
lua: network: Add a __close method
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 22 Feb 2024 14:14:41 +0000 (14:14 +0000)]
lua: database: Add __close method
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 22 Feb 2024 14:01:44 +0000 (14:01 +0000)]
lua: database: Rename the __gc method for consistency
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 22 Feb 2024 14:01:08 +0000 (14:01 +0000)]
lua: Actually return a network after lookup()
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 22 Feb 2024 13:43:34 +0000 (13:43 +0000)]
lua: Add a Network object
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 22 Feb 2024 12:32:22 +0000 (12:32 +0000)]
lua: Add a Database object with a dummy lookup function
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 22 Feb 2024 11:27:16 +0000 (11:27 +0000)]
lua: Initialize location context
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 22 Feb 2024 11:21:01 +0000 (11:21 +0000)]
lua: Add version() function
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 22 Feb 2024 11:20:46 +0000 (11:20 +0000)]
lua: Actually load the C module as such
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 22 Feb 2024 11:04:04 +0000 (11:04 +0000)]
.gitignore: Ignore some more temporary files
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 22 Feb 2024 11:02:38 +0000 (11:02 +0000)]
lua: Create scaffolding for a module
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Thu, 22 Feb 2024 10:14:02 +0000 (10:14 +0000)]
configure: Require Lua
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 17:10:49 +0000 (17:10 +0000)]
importer: Tidy up SQL queries
No functional changes. Just better readability.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 17:04:43 +0000 (17:04 +0000)]
importer: Drop previously imported AS names, too
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 17:03:58 +0000 (17:03 +0000)]
importer: Refactor ARIN importer
This patch is taking advantage of Python's CSV parser.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 17:03:27 +0000 (17:03 +0000)]
importer: Fix Python syntax error
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Peter Müller [Sun, 10 Dec 2023 19:37:00 +0000 (19:37 +0000)]
location-importer: Replace ARIN AS names source with one that offers human-readable names
This patch replaces our previous source for AS names in ARIN's realms
with another file provided by ARIN that contains human-readable names
for organizations ASNs have been allocated to.
Please note that a
TRUNCATE autnums;
is necessary on machines previously running the old version of
location-importer, in order to make use of this changed data source.
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 16:28:41 +0000 (16:28 +0000)]
tests: Make the new tests executable
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 16:27:04 +0000 (16:27 +0000)]
tests: Add more dedup/merge tests
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 16:20:44 +0000 (16:20 +0000)]
tests: Show even large diffs in dedup tests
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 16:19:57 +0000 (16:19 +0000)]
tests: Build out more dedup tests
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 16:06:13 +0000 (16:06 +0000)]
python: Implement rich comparison for Network objects
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 16:04:24 +0000 (16:04 +0000)]
python: Fix rich comparison for AS objects
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 16:03:09 +0000 (16:03 +0000)]
country: Return NULL/None for unset attributes
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 16:02:42 +0000 (16:02 +0000)]
tests: Add some tests for the Country object
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 15:54:49 +0000 (15:54 +0000)]
python: Fix rich comparison function for Country
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 15:54:30 +0000 (15:54 +0000)]
python: Make Country hashable
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 15:09:04 +0000 (15:09 +0000)]
tests: Add a simple test for deduplication
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 14:57:02 +0000 (14:57 +0000)]
network: Tidy up some code
No functional changes.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 14:48:08 +0000 (14:48 +0000)]
network-tree: Use the raw prefix to place networks onto the tree
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 14:45:43 +0000 (14:45 +0000)]
network-tree: Run deduplication once for each family
This probably never becomes a real problem, but we did not clear the
stack properly once we switched family when iterating through the
entire tree in one go.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 13:55:06 +0000 (13:55 +0000)]
tools: Import the copy script
This is useful to debug the library by copying one library 1:1 and then
diffing the result.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Wed, 21 Feb 2024 13:53:20 +0000 (13:53 +0000)]
network-tree: Split into a separate file
There are no functional changes here, but I find this code easier to
handle if it is split across smaller files.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Peter Müller [Sat, 17 Feb 2024 22:31:00 +0000 (22:31 +0000)]
location-importer: Fix Spamhaus ASN-DROP parsing
The format of this list has changed, from a plain text file with a
customer schema to JSON. Adjust our routines accordingly to make use of
this list again.
The second version of this patch incorporates Michael's feedback on the
first version, and adds AS names to the autnums table in case they are
not there already, which closes some gaps on rogue ASNs in the LACNIC
area.
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Tested-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Peter Müller [Sat, 17 Feb 2024 22:39:00 +0000 (22:39 +0000)]
location-importer: Add missing area code for AWS
It is unclear what exactly "eusc" refers to ("EU Secure Cloud", maybe?),
but according to RIPE database records for IPv6 networks involved, this
area code physically traces back to Germany, likely Berlin instead of
Frankfurt.
Signed-off-by: Peter Müller <peter.mueller@ipfire.org>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 17 Feb 2024 21:18:22 +0000 (21:18 +0000)]
network: Fix deduplication not looking far enough
When deduplicating the tree, we kept comparing subnets to the largest
supernet only - which is incorrect.
Instead, we need to compare subnets to the next largest supernet which
is now implemented in this patch.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sat, 17 Feb 2024 21:12:32 +0000 (21:12 +0000)]
network: Fix bit length check when merging networks
The check was incorrect as it only expected perfect matches, but we can
of course also merge subnets that don't necessarily require the entire
prefix length for the start address.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Sun, 7 Jan 2024 18:08:14 +0000 (18:08 +0000)]
python: Make AS objects hashable
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 31 Jul 2023 16:59:04 +0000 (16:59 +0000)]
configure: Bump version to 0.9.17
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 31 Jul 2023 16:58:26 +0000 (16:58 +0000)]
data: Update database to 2023-07-31
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 31 Jul 2023 16:55:23 +0000 (16:55 +0000)]
network: Decreate log level when deleting networks
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 31 Jul 2023 12:56:24 +0000 (12:56 +0000)]
libloc.so: Bump library revision
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Petr Písař [Mon, 24 Jul 2023 08:57:49 +0000 (10:57 +0200)]
Fix string escaping in location tool
Python 3.12 started to warn on an invalid string escape sequence:
$ location list-networks-by-as --format ipset 1
/usr/bin/location:583: SyntaxWarning: invalid escape sequence '\d'
m = re.match("^AS(\d+)$", object)
IpsetOutputWriter requires family being set
This patch fixes the warning as well as the regular expression by
using a raw string.
<https://docs.python.org/3.12/whatsnew/changelog.html#id77>
Fixes: #13188 - Invalid escape sequence with Python 3.12
Signed-off-by: Petr Písař <ppisar@redhat.com>
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 17 Jul 2023 16:13:35 +0000 (16:13 +0000)]
tree: More elegantly prevent deleting the root node
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 17 Jul 2023 16:00:14 +0000 (16:00 +0000)]
networks: Delete networks from the tree on merge
Previously, the tree did not support deleting nodes, but this has now
been added to do this safely.
This saves us running another deduplication check and halves the size of
the database.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Mon, 17 Jul 2023 15:36:21 +0000 (15:36 +0000)]
Merge networks before writing the database
This should help us keeping the database more compact where we summarize
two smaller networks to one larger one if they share the same
properties.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 14 Jul 2023 16:37:42 +0000 (16:37 +0000)]
tree: Actually delete any deleted nodes
Usually we only mark nodes as deleted, but finally, we should actually
delete what we can as we keep the pointers for back referencing.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>
Michael Tremer [Fri, 14 Jul 2023 16:33:49 +0000 (16:33 +0000)]
writer: Cleanup networks before writing
This patch will add a function that will search the tree for duplicate
entries which are not needed and remove them.
Afterwards, any unused nodes will be deleted.
Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>