Splunk Search
Highlighted

How to update geoip database for iplocation command?

Builder

Hello,

I use Splunk's iplocation (not Maxmind or other) command extensively in our monitoring dashboards. Since this is Splunk's "built-in" geoip command, does the underlying geoip database get automatically updated or do I have to manually update it? If manual, how do I update it?

Thanks!

++ Edit: I also checked the iplocation documentation but it doesn't describe how to update the geoip database:
http://docs.splunk.com/Documentation/Splunk/6.0.1/SearchReference/Iplocation

+++ Edit 4/4/14: I re-installed Maxmind per a recent suggestion that Splunk's iplocation command uses Maxmind's geoip database. After re-installing Maxmind and restarting Splunk, I ran two identical searches with geoip lookups: one with Splunk's iplocation and the other with Maxmind's geoip command. The results were different, proving that Splunk's iplocation command does not use Maxmind's geoip database. This also explains why the iplocation command will continue to work even if you uninstall the Maxmind app (and delete all of the directories as well). Also, I found that the iplocation command produces much better results (more complete) than Maxmind's "lookup geoip.." command (doesn't have geoip info for many IPs in our logs, which is why I prefer to use Splunk's iplocation command). So this question is still open...

++++ Edit 4/14/14: I searched the /splunk for *geo* (not sure why I didn't do this earlier!) and found this file:

/splunk/share/ipv4.geodb

Does anyone know if this is Splunk's geoip database for the iplocation command? And if so, same question stands... how does it get updated?

Thanks!

Tags (2)
Highlighted

Re: How to update geoip database for iplocation command?

Path Finder

I also would be interested in the answer to this question...

Highlighted

Re: How to update geoip database for iplocation command?

SplunkTrust
SplunkTrust
Highlighted

Re: How to update geoip database for iplocation command?

Builder

Thanks but I don't think so. The link answers how to update the maxmind database, if you are using maxmind for geoip. I've uninstalled maxmind since I decided to only use Splunk's built-in iplocation command... so I don't have the maxmind directory, etc. that this link refers to.

0 Karma
Highlighted

Re: How to update geoip database for iplocation command?

Communicator

echojacques, that IS the file we use. There is also a file for the iso3166 mappings in the same directory - the version of the .db file there does NOT contain them (it is based on the "lite" version of Maxmind) so these are separate. Unfortunately you can't just "update" that db file, but it should be updated with every maintenance release of Splunk - approximately every two months. Also in a future version of Splunk you can swap these files out for "paid" versions of Maxmind data - contact your local Splunk representation for more official detail on this.

View solution in original post

Highlighted

Re: How to update geoip database for iplocation command?

Builder

Thanks for the answer, I suspected that it was updated with every release, and you confirmed this.

Really appreciate the clarification and happy to know that I haven't been slacking by not updating this file... 🙂

Highlighted

Re: How to update geoip database for iplocation command?

Contributor

you're not really "slacking" per se - versus other splunk users - but since the mappings change fairly regularly in some parts of the world.. you'd potentially be using months-old, if not years-old data.

0 Karma
Highlighted

Re: How to update geoip database for iplocation command?

Builder

You can use your own premium MAXMIND data file using [iplocation] stanza in limits.conf

[iplocation]

dbpath = pathtotheMMDB

  • Location of GeoIP database in MMDB format
  • If not set, defaults to database included with splunk
Highlighted

Re: How to update geoip database for iplocation command?

Builder

Simpler way

  1. You can go to this site, http://dev.maxmind.com/geoip/geoip2/geolite2/
  2. Download GeoLite2-City.mmdb
  3. Replace it with $SPLUNK_HOME/share/GeoLite2-City.mmdb
  4. Restart splunk and prove it works as you expect.
Highlighted

Re: How to update geoip database for iplocation command?

Contributor

I recently discovered (maybe even by reading this question) that Splunk does not automatically update their GeoIP data. It lead me down an interesting search into IP location databases and how accurate they are and how much they cost, etc... I also discovered that MaxMind recently released a new "2.0" version of their free Location DB called GeoLite2.

Here's how I solve this problem in our deployment using cron and shell scripts. Note that the cron has a weird trick to catch the GeoLite DB updates on the first Tuesday of every month.

In Splunk user's crontab:

~$ crontab -l
# MaxMind DB Update for Splunk #
################################
SPLUNK_HOME=/opt/splunk    
OUTFILE=/tmp/cron.stdout.log

0 0 23 * * 3 [ $(date +\%d) -le 07 ] && $SPLUNK_HOME/bin/scripts/get_maxmind_db.sh >> $OUTFILE 2>&1

The shell script:

~$ cat $SPLUNK_HOME/bin/scripts/get_maxmind_db.sh
#!/bin/bash

# Author: Andrew Wurster
# Date: 13 Jan 2015

cd /opt/splunk/share

wget -O GeoLite2-City-Latest.mmdb.gz http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.mmdb.gz || { echo 'Could not download MaxMind GeoIP DB, exiting.' ; exit 1; }
gunzip -f GeoLite2-City-Latest.mmdb.gz
chmod 644 GeoLite2-City-Latest.mmdb

In limits.conf

~$ cat etc/system/local/limits.conf 
[iplocation]
db_path = /opt/splunk/share/GeoLite2-City-Latest.mmdb
#db_path = /opt/splunk/share/GeoIP2-City.mmdb

This is valid for MaxMinds new 2.0 format, and so far has not shown me any changes in lookup performance, etc.

FWITW I believe Splunk should make it more obvious where / how they are distributing this file (and the update frequency). A way to update it from the GUI or somewhere else convenient would be nice.