Splunk Search

Can I use a geospatial lookup to add state data to my data model?

pcsegal
Explorer

Using Splunk 6.6, I tried for the first time to create a Data Model.
My Root Event Dataset consists of events which have latitude and longitude fields. I have a geospatial lookup with all the states of Brazil, and I want to use the geospatial lookup to add a State field to my Root Event Dataset.

In the Data Model edit form, I clicked on "Add Field" and saw the option "Lookup". I thought that this would solve the problem. However, I did not find my geospatial lookup listed in the Lookup options. Looking into the Splunk documentation, I found this statement:

The Datasets listing page displays two categories of lookup datasets: lookup table files and lookup definitions. It lists lookup table files for .csv lookups and lookup definitions for .csv lookups and KV Store lookups. Other types of lookups, such as external lookups and geospatial lookups, are not listed as datasets.

So, my question is: how should I go about using the geospatial lookup to add fields to my root event dataset?

Any ideas?

Thank you in advance.

0 Karma

Dohrendorf_Cons
Path Finder

You can not configure the lookup through the interface since a geospatial lookup takes 2 inputs (Lat Lon) and the interface only allows for 1.
I tried and confirmed the possibility to add a geospatial lookup in
<yourApp>/local/data/models/<yourDataModel.json>
by adding a calculated field in the calculated fields section of your datamodel like this:

{
    "outputFields": [
        {
            "fieldName": "<FieldName>",
            "owner": "<DataModelRoot>",
            "type": "string",
            "fieldSearch": "",
            "required": false,
            "multivalue": false,
            "hidden": false,
            "editable": true,
            "displayName": "<FieldName>",
            "comment": "",
            "lookupOutputFieldName": "featureId"
        }
    ],
    "calculationType": "Lookup",
    "lookupName": "<NameOfYourLookupAsDefinedInTransforms>",
    "comment": "",
    "lookupInputs": [
        {
            "inputField": "gps_latitude",
            "lookupField": "latitude"
        },
        {
            "inputField": "gps_longitude",
            "lookupField": "longitude"
        }
    ],
    "owner": "<DatamodelRoot>",
    "calculationID": "<someHash>",
    "editable": true
},

This "works"... At least most of the time. We are curently seeing issues that lead to the looked up field sometimes being Null or even more weirdly, containing the name of the lookup instead of any valid output. Right now we have no explanation how this happens.

There seems to be at least one other user that ran into the same problem, as seen in this question:
https://answers.splunk.com/answers/688678/why-does-data-model-show-name-of-lookup-definition.html
So if you try this out, I would very much like to hear what your experience was.

Some kind of official statement regarding this issue would be great too.

0 Karma

DalJeanis
Legend

Basically, you need to start with a lookup file in KMZ format, (or KML format and zip it to KMZ), set up your lookup stanza in transforms.conf, save the changes and restart splunk. There are a few more optional things you can do, but that's the minimum.

0 Karma

lfedak_splunk
Splunk Employee
Splunk Employee

Hey @pcsegal, These pages in the documentation might help to learn more about geospatial lookups and data models.

0 Karma
Get Updates on the Splunk Community!

Join Us for Splunk University and Get Your Bootcamp Game On!

If you know, you know! Splunk University is the vibe this summer so register today for bootcamps galore ...

.conf24 | Learning Tracks for Security, Observability, Platform, and Developers!

.conf24 is taking place at The Venetian in Las Vegas from June 11 - 14. Continue reading to learn about the ...

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...