Splunk Search

Why is my lookup field from KV store in datamodel not showing correct results?

hettervik
Builder

Hi,

I have an accelerated datamodel. This datamodel have a lookup field based on a KV store lookup, that is, the datamodel takes another calculated numerical field and use the lookup to get a string value for that number. However, after some testing I see that my lookup field the accelerated datamodel does not have the correct value. I cross-checked by using the lookup directly on the raw data, and then it workes fine. I also double checked the expression for the lookup field, making sure it actually uses the correct values. The interesting thing is that the lookup field is indeed calculated, so the  KV store lookup works, and it get a "hit" in the lookup, but the row that is returned from the lookup is wrong. It seems to me that this could be some kind of bug, e.g. related to memory on the search head or something, that causes the lookup field accelerated datamodel to bug out. The KV store lookup is quite big.

Anyone else have experienced this, and knows a way around?

My lookup field in the datamodel:

         { [-]
           calculationID123456
           calculationTypeLookup
           comment:
           editabletrue
           lookupInputs: [ [-]
             { [-]
               inputField: mynumber
               lookupField: mynumber
             }
           ]

           lookupNamenumber_to_string
           outputFields: [ [-]
             { [-]
               comment:
               displayNamemystring
               editabletrue
               fieldName: mystring
               fieldSearch:
               hiddenfalse
               lookupOutputFieldName: mystring
               multivaluefalse
               ownermydatamodel
               requiredfalse
               typestring
             }
           ]

           owner: mydatamodel

         }

 The field "mynumber" is based on an eval expression in the datamodel (which is defined before the lookup field):

         { [-]
           calculationID456789
           calculationTypeEval
           comment:
           editabletrue
           expression'mynumber_raw'
           outputFields: [ [-]
             { [-]
               comment:
               displayName: mynumber
               editabletrue
               fieldName: mynumber
               fieldSearch:
               hiddenfalse
               multivaluefalse
               owner: mydatamodel
               requiredfalse
               typestring
             }
           ]

           owner: mydatamodel

         }

 

Labels (2)
0 Karma
1 Solution

hettervik
Builder

I changed the calculated eval field used in the lookup in the datamodel to an extracted field, and rebuilt the datamodel, and now it works. It very much looks like there is a bug with the calculated eval and lookup fields in datamodels, were the lookup field can get some sort of "mixup" if it is using an calculated eval field. Perhaps some sort of race condition between the calculation of the eval field and the lookup field.

View solution in original post

0 Karma

hettervik
Builder

I changed the calculated eval field used in the lookup in the datamodel to an extracted field, and rebuilt the datamodel, and now it works. It very much looks like there is a bug with the calculated eval and lookup fields in datamodels, were the lookup field can get some sort of "mixup" if it is using an calculated eval field. Perhaps some sort of race condition between the calculation of the eval field and the lookup field.

0 Karma
Get Updates on the Splunk Community!

March Community Office Hours Security Series Uncovered!

Hello Splunk Community! In March, Splunk Community Office Hours spotlighted our fabulous Splunk Threat ...

Stay Connected: Your Guide to April Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars in April. This post ...

Want to Reduce Costs, Mitigate Risk, Improve Performance, or Increase Efficiencies? ...

Splunk Lantern is Splunk’s customer success center that provides advice from Splunk experts on valuable data ...