I have an IP (216.3.51.108) that I'm trying to geolocate, but the City and Region fields are returning as Null. When I geolocate using the Maxmind 'GeoIP2 City Database Demo', I get all the values I expect.
In reading the documentation for the iplocation
command I came across the instructions for updating the database. But the instructions aren't quite clear if I need to update the database on the SH cluster or the Index cluster (or both). There is a paragraph stating that you /can/ update on the indexer, but to me this is different than saying you /must/. What are the steps to update the geoip database in a distributed cluster? I actually did it on both but this did not change my results, so I must be doing it wrong.
You should do it on both otherwise you are going to have discrepancies. Iplocation is a streaming command and depending on your search it can either be applied at the indexers level or at the SH level.
You should only need to do so at the search layer. This is how I do it on standalone search heads: http://blog.hortonew.com/splunk-automatically-update-geoip-database-across-environment
In a search cluster, you would probably want to keep the SH deployer file up to date, and schedule a push to the members once a month. I'm not sure the way you're doing it today, but if you're not updating the lookup through the GUI or SH Deployer, it's probably not getting propagated to the other members.
I actually opened a support case on this. I did not get the sense that the engineer was really committed to answering my question, but this is the response I got:
I would suggest you do it on your indexer. Currently, the iplocation command in a distributed environment runs on the peers. When a paid or custom .mmdb file is used, it has to be changed on each peer. You can use localop to force it to the SH, but there's no way to do it automatically. All of the search peers will need it, as unless you have localop, the iplocation will be done on the search peer.