Using Splunk 6.6, I tried for the first time to create a Data Model.
My Root Event Dataset consists of events which have latitude and longitude fields. I have a geospatial lookup with all the states of Brazil, and I want to use the geospatial lookup to add a State field to my Root Event Dataset.
In the Data Model edit form, I clicked on "Add Field" and saw the option "Lookup". I thought that this would solve the problem. However, I did not find my geospatial lookup listed in the Lookup options. Looking into the Splunk documentation, I found this statement:
The Datasets listing page displays two categories of lookup datasets: lookup table files and lookup definitions. It lists lookup table files for .csv lookups and lookup definitions for .csv lookups and KV Store lookups. Other types of lookups, such as external lookups and geospatial lookups, are not listed as datasets.
So, my question is: how should I go about using the geospatial lookup to add fields to my root event dataset?
Any ideas?
Thank you in advance.
You can not configure the lookup through the interface since a geospatial lookup takes 2 inputs (Lat Lon) and the interface only allows for 1.
I tried and confirmed the possibility to add a geospatial lookup in
<yourApp>/local/data/models/<yourDataModel.json>
by adding a calculated field in the calculated fields section of your datamodel like this:
{
"outputFields": [
{
"fieldName": "<FieldName>",
"owner": "<DataModelRoot>",
"type": "string",
"fieldSearch": "",
"required": false,
"multivalue": false,
"hidden": false,
"editable": true,
"displayName": "<FieldName>",
"comment": "",
"lookupOutputFieldName": "featureId"
}
],
"calculationType": "Lookup",
"lookupName": "<NameOfYourLookupAsDefinedInTransforms>",
"comment": "",
"lookupInputs": [
{
"inputField": "gps_latitude",
"lookupField": "latitude"
},
{
"inputField": "gps_longitude",
"lookupField": "longitude"
}
],
"owner": "<DatamodelRoot>",
"calculationID": "<someHash>",
"editable": true
},
This "works"... At least most of the time. We are curently seeing issues that lead to the looked up field sometimes being Null or even more weirdly, containing the name of the lookup instead of any valid output. Right now we have no explanation how this happens.
There seems to be at least one other user that ran into the same problem, as seen in this question:
https://answers.splunk.com/answers/688678/why-does-data-model-show-name-of-lookup-definition.html
So if you try this out, I would very much like to hear what your experience was.
Some kind of official statement regarding this issue would be great too.
Basically, you need to start with a lookup file in KMZ format, (or KML format and zip it to KMZ), set up your lookup stanza in transforms.conf, save the changes and restart splunk. There are a few more optional things you can do, but that's the minimum.
Hey @pcsegal, These pages in the documentation might help to learn more about geospatial lookups and data models.