Splunk Search

Can I use a geospatial lookup to add state data to my data model?

pcsegal
Explorer

Using Splunk 6.6, I tried for the first time to create a Data Model.
My Root Event Dataset consists of events which have latitude and longitude fields. I have a geospatial lookup with all the states of Brazil, and I want to use the geospatial lookup to add a State field to my Root Event Dataset.

In the Data Model edit form, I clicked on "Add Field" and saw the option "Lookup". I thought that this would solve the problem. However, I did not find my geospatial lookup listed in the Lookup options. Looking into the Splunk documentation, I found this statement:

The Datasets listing page displays two categories of lookup datasets: lookup table files and lookup definitions. It lists lookup table files for .csv lookups and lookup definitions for .csv lookups and KV Store lookups. Other types of lookups, such as external lookups and geospatial lookups, are not listed as datasets.

So, my question is: how should I go about using the geospatial lookup to add fields to my root event dataset?

Any ideas?

Thank you in advance.

0 Karma

Dohrendorf_Cons
Path Finder

You can not configure the lookup through the interface since a geospatial lookup takes 2 inputs (Lat Lon) and the interface only allows for 1.
I tried and confirmed the possibility to add a geospatial lookup in
<yourApp>/local/data/models/<yourDataModel.json>
by adding a calculated field in the calculated fields section of your datamodel like this:

{
    "outputFields": [
        {
            "fieldName": "<FieldName>",
            "owner": "<DataModelRoot>",
            "type": "string",
            "fieldSearch": "",
            "required": false,
            "multivalue": false,
            "hidden": false,
            "editable": true,
            "displayName": "<FieldName>",
            "comment": "",
            "lookupOutputFieldName": "featureId"
        }
    ],
    "calculationType": "Lookup",
    "lookupName": "<NameOfYourLookupAsDefinedInTransforms>",
    "comment": "",
    "lookupInputs": [
        {
            "inputField": "gps_latitude",
            "lookupField": "latitude"
        },
        {
            "inputField": "gps_longitude",
            "lookupField": "longitude"
        }
    ],
    "owner": "<DatamodelRoot>",
    "calculationID": "<someHash>",
    "editable": true
},

This "works"... At least most of the time. We are curently seeing issues that lead to the looked up field sometimes being Null or even more weirdly, containing the name of the lookup instead of any valid output. Right now we have no explanation how this happens.

There seems to be at least one other user that ran into the same problem, as seen in this question:
https://answers.splunk.com/answers/688678/why-does-data-model-show-name-of-lookup-definition.html
So if you try this out, I would very much like to hear what your experience was.

Some kind of official statement regarding this issue would be great too.

0 Karma

DalJeanis
Legend

Basically, you need to start with a lookup file in KMZ format, (or KML format and zip it to KMZ), set up your lookup stanza in transforms.conf, save the changes and restart splunk. There are a few more optional things you can do, but that's the minimum.

0 Karma

lfedak_splunk
Splunk Employee
Splunk Employee

Hey @pcsegal, These pages in the documentation might help to learn more about geospatial lookups and data models.

0 Karma
Get Updates on the Splunk Community!

Earn a $35 Gift Card for Answering our Splunk Admins & App Developer Survey

Survey for Splunk Admins and App Developers is open now! | Earn a $35 gift card!      Hello there,  Splunk ...

Continuing Innovation & New Integrations Unlock Full Stack Observability For Your ...

You’ve probably heard the latest about AppDynamics joining the Splunk Observability portfolio, deepening our ...

Monitoring Amazon Elastic Kubernetes Service (EKS)

As we’ve seen, integrating Kubernetes environments with Splunk Observability Cloud is a quick and easy way to ...