Fall back to registered country if necessary.

When extracting geoip and geoip6 files from MaxMind's GeoLite2 Country
database, we only look at country->iso_code which is the two-character ISO
3166-1 country code of the country where MaxMind believes the end user is
located.

But if MaxMind thinks a range belongs to anonymous proxies, they don't put
anything there.  Hence, we omit those ranges and resolve them all to '??'.
That's not what we want.

What we should do is first try country->iso_code, and if there's no such
key, try registered_country->iso_code which is the country in which the
ISP has registered the IP address.

In short: let's fill all A1 entries with what ARIN et. al think.

diff --git a/src/config/mmdb-convert.py b/src/config/mmdb-convert.py
index 21d170adf68e35445ca5dc4ac4c0763f3684fccd..4245738542e931a20688b4b83a48fdf51b0d24b1 100644 (file)
--- a/src/config/mmdb-convert.py
+++ b/src/config/mmdb-convert.py
@@ -339,11 +339,24 @@ def parse_mm_file(s):
 def format_datum(datum):
     """Given a Datum at a leaf of the tree, return the string that we should
        write as its value.
+
+       We first try country->iso_code which is the two-character ISO 3166-1
+       country code of the country where MaxMind believes the end user is
+       located.  If there's no such key, we try registered_country->iso_code
+       which is the country in which the ISP has registered the IP address.
+       Without falling back to registered_country, we'd leave out all ranges
+       that MaxMind thinks belong to anonymous proxies, because those ranges
+       don't contain country but only registered_country.  In short: let's
+       fill all A1 entries with what ARIN et. al think.
     """
     try:
         return bytesToStr(datum.map['country'].map['iso_code'].data)
     except KeyError:
         pass
+    try:
+        return bytesToStr(datum.map['registered_country'].map['iso_code'].data)
+    except KeyError:
+        pass
     return None
 
 IPV4_PREFIX = "0"*96