Splunk Enterprise

Maxmind | How to use 5 different mmdb databases?

o_calmels
Communicator

Hi splunkers,

 

I have problem about usind maxming geoip datavbses

I get 4 databases from maxmind (GeoIP2-City.mmdb; GeoLite2-ASN.mmdb; GeoIP2-Country.mmdb; GeoIP2-Anonymous-IP.mmdb)

I need to use these 4 databases


Following the html documentation about iplocation (https://docs.splunk.com/Documentation/Splunk/7.0.2/SearchReference/Iplocation), I copy the databases I need to use under a specific directory and configure limits.conf to point to this directory for any of the databases I need to use.
This database was copied over search Head AND Indexers.

Limits.conf :

[root@vlpsospk04-sh databases]# more ../local/limits.conf
[iplocation]
db_path = /data/splunk/etc/apps/cnaf_deploy_maxmind_databases/databases/GeoIP2-City.mmdb
db_path = /data/splunk/etc/apps/cnaf_deploy_maxmind_databases/databases/GeoLite2-ASN.mmdb
db_path = /data/splunk/etc/apps/cnaf_deploy_maxmind_databases/databases/GeoIP2-Country.mmdb
db_path = /data/splunk/etc/apps/cnaf_deploy_maxmind_databases/databases/GeoIP2-Anonymous-IP.mmdb


Then, when I m using this file configuration, Then restart splunkd process, I get data about GeoIP2-City.mmdb, but nothing about GeoIP2-Anonymous-IP.mmdb as an exemple.


In the documentation about iplocation, only one mmdb file is documented, so is this a specific configuration to use multiple mmd files ?

 

Does someone get results with sevferal databases ?

 

Thank you !

Labels (2)
Tags (2)
0 Karma

reincoder
Engager

You can check out IPinfo as an alternative. We have an app that supports our API and Database both on Splunk.

https://splunkbase.splunk.com/app/4070

Our databases come in MMDB format as well. We offer a free country + ASN database that you can try out with the Splunk app now.: https://ipinfo.io/developers/ip-to-country-asn-database, and we offer a free IP geolocation API.

 

jnhth
Explorer

Did you deploy from the CM to all your index servers og just copy direct?

 

0 Karma

starcher
Influencer

Write your own external lookup command in python that uses the maxmind python library per mmdb as each one has different data. You will want to work with your system administrators to out of Splunk sync tge mmbd files to disk and your code point to tge files there. 

0 Karma

johnhuang
Motivator

If you have on-prem Splunk, you can look into this add-on (https://splunkbase.splunk.com/app/6169). For Splunk Cloud, the most straight forward way is to download the Maxmind databases in CSV and create a lookup definition for it.

For example, to configure the Geolite-ASN lookup definition you want to set the match type to CIDR(network) and maximum match to 1.

johnhuang_0-1666790972461.png

 

0 Karma

NDabhi21
Explorer

What about the deployments where Splunk is already using mmdb database. Those can still continue?

And if want to move to the new one, is there any doc yet?

Thanks in Advance..

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Yes, you can continue to use your existing MMDB file.  It will, of course, become outdated eventually.

If you want to use a new MMDB provider then just install the file as documented at https://docs.splunk.com/Documentation/Splunk/9.0.1/SearchReference/Iplocation#Updating_the_IP_geoloc...

---
If this reply helps you, Karma would be appreciated.

richgalloway
SplunkTrust
SplunkTrust

Splunk only supports a single iplocation file, usually GeoIP2-City.mmdb.  Furthermore, Splunk recently changed geo-ip providers and no longer ships with a MaxMind database.

Make a case for supporting all four databases at https://ideas.splunk.com

---
If this reply helps you, Karma would be appreciated.
0 Karma

Dennis
Explorer

Hello Rich,

"Furthermore, Splunk recently changed geo-ip providers and no longer ships with a MaxMind database."

What company is now the Splunk geo-ip DB provider, in 2023, since Splunk no longer ships with a MaxMind database as you mentioned?

Also, what is the new DB file name, what directory is it located in, and does the new iplocation DB get updated after the initial SE installation, or not ?

Best regards,

Dennis

 

0 Karma

PickleRick
SplunkTrust
SplunkTrust

Don't know about the provider but the database is updated only on Splunk upgrades. You can do manual updates but they will be overwritten when you upgrade your Splunk installation unless you set a custom path to the database file.

0 Karma

Dennis
Explorer

Thanks Rick!

What Rich Galloway stated was that "Splunk recently changed geo-ip providers and no longer ships with a MaxMind database."

If that is the case, I was asking what company is the new geo-ip provider that has taken over from MaxMind ?

Also, what version of SE did the switchover over, and what directory is the new geo-IP DB in, and what is the new mmdb file name?

Best regards,

Dennis

 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Splunk hasn't disclosed the new vendor of geo-ip data, which changed with version 9.0.

The file is $SPLUNK_HOME/share/dbip-city-lite.mmdb.

You can read more about it in the iplocation documentation at https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Iplocation

---
If this reply helps you, Karma would be appreciated.

Dennis
Explorer

Thanks Rich!

That answered all my questions, but brought up 2 new questions.

We are running SE 9.0.5 so we have the new $SPLUNK_HOME/share/dbip-city-lite.mmdb  geo-location DB as you mentioned.

The reason for this new question is I noticed an IP address yesterday whose City seems to be outdated against the results from iplocation.net.

Guessing there is no way to update the new dbip-city-lite.mmdb DB after the initial SE install since Splunk has not divulged the vendor ?

Went to the link you provided, and to the 9.0.5 page for iplocation which does state the new vendor's mmdb file name, but the data after that shows how to update MaxMind DB's, GeoLite2-City.mmdb & GeoIP2-City.mmdb , which as you said were replaced in 9.0.0, and are not shipped with version 9.0.5.  Is this an oversight in the documentation ?

 

iplocation - Splunk Documentation

"Usage

The iplocation command is a distributable streaming command. See Command types.

The Splunk software ships with a copy of the dbip-city-lite.mmdb IP geolocation database file. This file is located in the $SPLUNK_HOME/share/ directory.

Updating the IP geolocation database file

Through Splunk Web, you can update the .mmdb file that ships with the Splunk software. The file you update it with can be a copy of one of the following two files. Only those two files are supported. To use these two files, you must have a license for the GeoIP2 City database.

File name Description

GeoLite2-City.mmdbThis is a free IP geolocation database that is updated on its download page on a weekly basis.
GeoIP2-City.mmdbThis is a paid version of the GeoLite2-City IP geolocation database that is more accurate than the free version.

 

Replacing your mmdb file with one of these two files reintroduces the Timezone field that is absent in the default .mmdb file, but does not reintroduce the MetroCode field.

Prerequisites

You must have a role with the upload_mmdb_files capability.

Steps

  1. Go online and find a download page for the binary .tar.gz versions of the GeoLite2-City or the GeoIP2-City database files.
  2. Download the binary .tar.gz version of the file (GeoLite2-City or GeoIP2-City) that is most appropriate for your needs.
  3. Expand the binary .tar.gz version of the file.
    The .tar.gz file expands into a folder which contains the GeoLite2-City.mmdb file, or the GeoIP2-City.mmdb file, depending on the download you selected.
  4. In Splunk Web, go to Settings > Lookups > GeoIP lookups file.
  5. On the GeoIP lookups file page, click Choose file. Select the .mmdb file.
  6. Click Save.

The page displays a success message when the upload completes."

0 Karma

richgalloway
SplunkTrust
SplunkTrust

You can replace the geo-ip file with an MMDB file from any vendor, including MaxMind.  It does not have to be from the same vendor as the one that shipped with Splunk.

---
If this reply helps you, Karma would be appreciated.

Dennis
Explorer

Great, thanks Rich.

It would be good if Splunk could enable the new geo-location DB that ships with SE 9.0.0 or later, dbip-city-lite.mmdb, to be updated on a regular basis instead of having to replace the new DB with either MaxMind's, or some other vendor's DB.

Splunk could build that update functionality in behind the scenes if divulging the new vendor is top secret for some reason.  😎

Otherwise, the update procedure for the new DB could be added to the iplocation page like for MaxMind's update procedure.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Consider putting that into Feedback on the docs page and submitting it at https://ideas.splunk.com

---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

Federated Search for Amazon S3 | Key Use Cases to Streamline Compliance Workflows

Modern business operations are supported by data compliance. As regulations evolve, organizations must ...

New Dates, New City: Save the Date for .conf25!

Wake up, babe! New .conf25 dates AND location just dropped!! That's right, this year, .conf25 is taking place ...

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud  In today’s fast-paced digital ...