Why is my lookup field from KV store in datamodel not showing correct results?

hettervik — Tue, 01 Aug 2023 12:20:54 GMT

Hi,

I have an accelerated datamodel. This datamodel have a lookup field based on a KV store lookup, that is, the datamodel takes another calculated numerical field and use the lookup to get a string value for that number. However, after some testing I see that my lookup field the accelerated datamodel does not have the correct value. I cross-checked by using the lookup directly on the raw data, and then it workes fine. I also double checked the expression for the lookup field, making sure it actually uses the correct values. The interesting thing is that the lookup field is indeed calculated, so the KV store lookup works, and it get a "hit" in the lookup, but the row that is returned from the lookup is wrong. It seems to me that this could be some kind of bug, e.g. related to memory on the search head or something, that causes the lookup field accelerated datamodel to bug out. The KV store lookup is quite big.

Anyone else have experienced this, and knows a way around?

My lookup field in the datamodel:

         { [-]
           calculationID: 123456
           calculationType: Lookup
           comment:
           editable: true
           lookupInputs: [ [-]
             { [-]
               inputField: mynumber
               lookupField: mynumber
             }
           ]
           lookupName: number_to_string
           outputFields: [ [-]
             { [-]
               comment:
               displayName: mystring
               editable: true
               fieldName: mystring
               fieldSearch:
               hidden: false
               lookupOutputFieldName: mystring
               multivalue: false
               owner: mydatamodel
               required: false
               type: string
             }
           ]
           owner: mydatamodel
         }

The field "mynumber" is based on an eval expression in the datamodel (which is defined before the lookup field):

         { [-]
           calculationID: 456789
           calculationType: Eval
           comment:
           editable: true
           expression: 'mynumber_raw'
           outputFields: [ [-]
             { [-]
               comment:
               displayName: mynumber
               editable: true
               fieldName: mynumber
               fieldSearch:
               hidden: false
               multivalue: false
               owner: mydatamodel
               required: false
               type: string
             }
           ]
           owner: mydatamodel
         }

Re: Why is my lookup field from KV store in datamodel not showing correct results?

hettervik — Wed, 16 Aug 2023 08:06:08 GMT

I changed the calculated eval field used in the lookup in the datamodel to an extracted field, and rebuilt the datamodel, and now it works. It very much looks like there is a bug with the calculated eval and lookup fields in datamodels, were the lookup field can get some sort of "mixup" if it is using an calculated eval field. Perhaps some sort of race condition between the calculation of the eval field and the lookup field.

topic Why is my lookup field from KV store in datamodel not showing correct results? in Splunk Search

Why is my lookup field from KV store in datamodel not showing correct results?

Re: Why is my lookup field from KV store in datamodel not showing correct results?