Reporting

Data model calculated field dependencies

behudelson
Path Finder

Hi all,

I am trying to use data models to extract a search time value from a lookup. However, the value I am using to join to the lookup table is extracted from the source. I have no other way to get the value other than from the file name.

For example, I have the following regex extraction against source. It is configured in both my data model and as a calculated field:

(?<SourceFolder>[\w-]+).(?<SourceIdentifier>\d+)[\D]+

When I preview or run a search, these fields get populated.

I also have a lookup that has the SourceIdentifier field with matching values. I configured a lookup field in DataModels to add some of the lookup fields to my searches. My lookup field configuration is at the bottom of the list.

When I preview or run a search, these fields do not get populated.

Splunk documentation says: "When Splunk software evaluates calculated fields, it evaluates each expression as if it were independent of all other fields. You cannot chain calculated field expressions, where the evaluation of one calculated field is used in the expression for another calculated field." (https://docs.splunk.com/Documentation/Splunk/7.2.5/Knowledge/definecalcfields)

However, at the bottom of the data models page there is a message that says: "Calculated fields are processed in the order above, so ensure any dependent fields are defined first. Drag to rearrange." (You can find this message by clicking into an unaccelerated data model and scrolling to the bottom of the page.)

I've attempted to work on this issue in both Splunk Enterprise and Splunk Cloud. Both are running v 7.0.8.

Is what I am attempting to do possible?

If not, is there another route to accomplish the same thing?

Thank you!

0 Karma
1 Solution

lakshman239
SplunkTrust
SplunkTrust

Are you having a custom datamodel? If I understand correctly, you want to extract a few fields from 'filename', which you want to use later to map to a 'field' in the datamodel. One way to approach this would be to use search time precedence.

https://docs.splunk.com/Documentation/Splunk/7.2.5/Knowledge/Searchtimeoperationssequence

  • extract the fields which you want using in-line or REPORT-transforms. (e.g SourceIdentifier)
  • use lookups to OUTPUT the field names (e.g. action in email datamodel) that needs to be mapped to datamodel, using SourceIdentifier or any other field extracted.

use btool to check/troubleshoot - https://docs.splunk.com/Documentation/Splunk/latest/Troubleshooting/Usebtooltotroubleshootconfigurat...

View solution in original post

0 Karma

lakshman239
SplunkTrust
SplunkTrust

Are you having a custom datamodel? If I understand correctly, you want to extract a few fields from 'filename', which you want to use later to map to a 'field' in the datamodel. One way to approach this would be to use search time precedence.

https://docs.splunk.com/Documentation/Splunk/7.2.5/Knowledge/Searchtimeoperationssequence

  • extract the fields which you want using in-line or REPORT-transforms. (e.g SourceIdentifier)
  • use lookups to OUTPUT the field names (e.g. action in email datamodel) that needs to be mapped to datamodel, using SourceIdentifier or any other field extracted.

use btool to check/troubleshoot - https://docs.splunk.com/Documentation/Splunk/latest/Troubleshooting/Usebtooltotroubleshootconfigurat...

0 Karma

behudelson
Path Finder

Hi @laksman239, thanks for the assist. I was trying to use a data model field that an autolookup props.cong entry was more suitable for. props.conf entry:

[my_source_type]
LOOKUP-class = mylookup LookupID as SourceIdentifier OUTPUT LookupField

Thanks!

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...