Splunk Search

Can I use a geospatial lookup to add state data to my data model?

pcsegal
Explorer

Using Splunk 6.6, I tried for the first time to create a Data Model.
My Root Event Dataset consists of events which have latitude and longitude fields. I have a geospatial lookup with all the states of Brazil, and I want to use the geospatial lookup to add a State field to my Root Event Dataset.

In the Data Model edit form, I clicked on "Add Field" and saw the option "Lookup". I thought that this would solve the problem. However, I did not find my geospatial lookup listed in the Lookup options. Looking into the Splunk documentation, I found this statement:

The Datasets listing page displays two categories of lookup datasets: lookup table files and lookup definitions. It lists lookup table files for .csv lookups and lookup definitions for .csv lookups and KV Store lookups. Other types of lookups, such as external lookups and geospatial lookups, are not listed as datasets.

So, my question is: how should I go about using the geospatial lookup to add fields to my root event dataset?

Any ideas?

Thank you in advance.

0 Karma

Dohrendorf_Cons
Path Finder

You can not configure the lookup through the interface since a geospatial lookup takes 2 inputs (Lat Lon) and the interface only allows for 1.
I tried and confirmed the possibility to add a geospatial lookup in
<yourApp>/local/data/models/<yourDataModel.json>
by adding a calculated field in the calculated fields section of your datamodel like this:

{
    "outputFields": [
        {
            "fieldName": "<FieldName>",
            "owner": "<DataModelRoot>",
            "type": "string",
            "fieldSearch": "",
            "required": false,
            "multivalue": false,
            "hidden": false,
            "editable": true,
            "displayName": "<FieldName>",
            "comment": "",
            "lookupOutputFieldName": "featureId"
        }
    ],
    "calculationType": "Lookup",
    "lookupName": "<NameOfYourLookupAsDefinedInTransforms>",
    "comment": "",
    "lookupInputs": [
        {
            "inputField": "gps_latitude",
            "lookupField": "latitude"
        },
        {
            "inputField": "gps_longitude",
            "lookupField": "longitude"
        }
    ],
    "owner": "<DatamodelRoot>",
    "calculationID": "<someHash>",
    "editable": true
},

This "works"... At least most of the time. We are curently seeing issues that lead to the looked up field sometimes being Null or even more weirdly, containing the name of the lookup instead of any valid output. Right now we have no explanation how this happens.

There seems to be at least one other user that ran into the same problem, as seen in this question:
https://answers.splunk.com/answers/688678/why-does-data-model-show-name-of-lookup-definition.html
So if you try this out, I would very much like to hear what your experience was.

Some kind of official statement regarding this issue would be great too.

0 Karma

DalJeanis
SplunkTrust
SplunkTrust

Basically, you need to start with a lookup file in KMZ format, (or KML format and zip it to KMZ), set up your lookup stanza in transforms.conf, save the changes and restart splunk. There are a few more optional things you can do, but that's the minimum.

0 Karma

lfedak_splunk
Splunk Employee
Splunk Employee

Hey @pcsegal, These pages in the documentation might help to learn more about geospatial lookups and data models.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...