Maxmind | How to use 5 different mmdb databases?

[o_calmels]

Hi splunkers,

I have problem about usind maxming geoip datavbses

I get 4 databases from maxmind (GeoIP2-City.mmdb; GeoLite2-ASN.mmdb; GeoIP2-Country.mmdb; GeoIP2-Anonymous-IP.mmdb)

I need to use these 4 databases

Following the html documentation about iplocation (https://docs.splunk.com/Documentation/Splunk/7.0.2/SearchReference/Iplocation), I copy the databases I need to use under a specific directory and configure limits.conf to point to this directory for any of the databases I need to use.
This database was copied over search Head AND Indexers.

Limits.conf :

[root@vlpsospk04-sh databases]# more ../local/limits.conf
[iplocation]
db_path = /data/splunk/etc/apps/cnaf_deploy_maxmind_databases/databases/GeoIP2-City.mmdb
db_path = /data/splunk/etc/apps/cnaf_deploy_maxmind_databases/databases/GeoLite2-ASN.mmdb
db_path = /data/splunk/etc/apps/cnaf_deploy_maxmind_databases/databases/GeoIP2-Country.mmdb
db_path = /data/splunk/etc/apps/cnaf_deploy_maxmind_databases/databases/GeoIP2-Anonymous-IP.mmdb


Then, when I m using this file configuration, Then restart splunkd process, I get data about GeoIP2-City.mmdb, but nothing about GeoIP2-Anonymous-IP.mmdb as an exemple.


In the documentation about iplocation, only one mmdb file is documented, so is this a specific configuration to use multiple mmd files ?

Does someone get results with sevferal databases ?

Thank you !

[reincoder]

You can check out IPinfo as an alternative. We have an app that supports our API and Database both on Splunk.

https://splunkbase.splunk.com/app/4070

Our databases come in MMDB format as well. We offer a free country + ASN database that you can try out with the Splunk app now.: https://ipinfo.io/developers/ip-to-country-asn-database, and we offer a free IP geolocation API.

[jnhth]

Did you deploy from the CM to all your index servers og just copy direct?

[starcher]

Write your own external lookup command in python that uses the maxmind python library per mmdb as each one has different data. You will want to work with your system administrators to out of Splunk sync tge mmbd files to disk and your code point to tge files there. 

[johnhuang]

If you have on-prem Splunk, you can look into this add-on (https://splunkbase.splunk.com/app/6169). For Splunk Cloud, the most straight forward way is to download the Maxmind databases in CSV and create a lookup definition for it.

For example, to configure the Geolite-ASN lookup definition you want to set the match type to CIDR(network) and maximum match to 1.

[NDabhi21]

What about the deployments where Splunk is already using mmdb database. Those can still continue?

And if want to move to the new one, is there any doc yet?

Thanks in Advance..

[richgalloway]

Yes, you can continue to use your existing MMDB file.  It will, of course, become outdated eventually.

If you want to use a new MMDB provider then just install the file as documented at https://docs.splunk.com/Documentation/Splunk/9.0.1/SearchReference/Iplocation#Updating_the_IP_geoloc...

[richgalloway]

Splunk only supports a single iplocation file, usually GeoIP2-City.mmdb.  Furthermore, Splunk recently changed geo-ip providers and no longer ships with a MaxMind database.

Make a case for supporting all four databases at https://ideas.splunk.com

[Dennis]

Hello Rich,

"Furthermore, Splunk recently changed geo-ip providers and no longer ships with a MaxMind database."

What company is now the Splunk geo-ip DB provider, in 2023, since Splunk no longer ships with a MaxMind database as you mentioned?

Also, what is the new DB file name, what directory is it located in, and does the new iplocation DB get updated after the initial SE installation, or not ?

Best regards,

Dennis

[PickleRick]

Don't know about the provider but the database is updated only on Splunk upgrades. You can do manual updates but they will be overwritten when you upgrade your Splunk installation unless you set a custom path to the database file.

[Dennis]

Thanks Rick!

What Rich Galloway stated was that "Splunk recently changed geo-ip providers and no longer ships with a MaxMind database."

If that is the case, I was asking what company is the new geo-ip provider that has taken over from MaxMind ?

Also, what version of SE did the switchover over, and what directory is the new geo-IP DB in, and what is the new mmdb file name?

Best regards,

Dennis

[richgalloway]

Splunk hasn't disclosed the new vendor of geo-ip data, which changed with version 9.0.

The file is $SPLUNK_HOME/share/dbip-city-lite.mmdb.

You can read more about it in the iplocation documentation at https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Iplocation

[Dennis]

Thanks Rich!

That answered all my questions, but brought up 2 new questions.

We are running SE 9.0.5 so we have the new $SPLUNK_HOME/share/dbip-city-lite.mmdb  geo-location DB as you mentioned.

The reason for this new question is I noticed an IP address yesterday whose City seems to be outdated against the results from iplocation.net.

Guessing there is no way to update the new dbip-city-lite.mmdb DB after the initial SE install since Splunk has not divulged the vendor ?

Went to the link you provided, and to the 9.0.5 page for iplocation which does state the new vendor's mmdb file name, but the data after that shows how to update MaxMind DB's, GeoLite2-City.mmdb & GeoIP2-City.mmdb , which as you said were replaced in 9.0.0, and are not shipped with version 9.0.5.  Is this an oversight in the documentation ?

iplocation - Splunk Documentation

"Usage

The iplocation command is a distributable streaming command. See Command types.

The Splunk software ships with a copy of the dbip-city-lite.mmdb IP geolocation database file. This file is located in the $SPLUNK_HOME/share/ directory.

Updating the IP geolocation database file

Through Splunk Web, you can update the .mmdb file that ships with the Splunk software. The file you update it with can be a copy of one of the following two files. Only those two files are supported. To use these two files, you must have a license for the GeoIP2 City database.

File name Description

GeoLite2-City.mmdb	This is a free IP geolocation database that is updated on its download page on a weekly basis.
GeoIP2-City.mmdb	This is a paid version of the GeoLite2-City IP geolocation database that is more accurate than the free version.

Replacing your mmdb file with one of these two files reintroduces the Timezone field that is absent in the default .mmdb file, but does not reintroduce the MetroCode field.

Prerequisites

You must have a role with the upload_mmdb_files capability.

Steps

1. Go online and find a download page for the binary .tar.gz versions of the GeoLite2-City or the GeoIP2-City database files.
2. Download the binary .tar.gz version of the file (GeoLite2-City or GeoIP2-City) that is most appropriate for your needs.
3. Expand the binary .tar.gz version of the file.
   The .tar.gz file expands into a folder which contains the GeoLite2-City.mmdb file, or the GeoIP2-City.mmdb file, depending on the download you selected.
4. In Splunk Web, go to Settings > Lookups > GeoIP lookups file.
5. On the GeoIP lookups file page, click Choose file. Select the .mmdb file.
6. Click Save.

The page displays a success message when the upload completes."

[richgalloway]

You can replace the geo-ip file with an MMDB file from any vendor, including MaxMind.  It does not have to be from the same vendor as the one that shipped with Splunk.

[Dennis]

Great, thanks Rich.

It would be good if Splunk could enable the new geo-location DB that ships with SE 9.0.0 or later, dbip-city-lite.mmdb, to be updated on a regular basis instead of having to replace the new DB with either MaxMind's, or some other vendor's DB.

Splunk could build that update functionality in behind the scenes if divulging the new vendor is top secret for some reason.  😎

Otherwise, the update procedure for the new DB could be added to the iplocation page like for MaxMind's update procedure.

[richgalloway]

Consider putting that into Feedback on the docs page and submitting it at https://ideas.splunk.com