Splunk Search

Extracting field with spath from JSON sourcetype overwrites field in other None-JSON sourcetype

flle
Path Finder

Hi,

looked through documentation and Splunk answers but did not find reason/root cause for the following obervation:

We have an index with 2 sourcetypes. one is JSON, the other plain text.
Event examples:

 

Sourcetype 1, JSON notation:
{ [-]
   context: xyz
   criteria: { [+]
   }
   device: desktop
   results: [ [-]
     50832171
   ]
   searchType: QuickSearch
}

Raw Text notation:
{"context":"xyz","device":"desktop","searchType":"QuickSearch","results":["50832171"],"criteria":{"Item name":"Example"}}

Sourcetype 2, None-JSON event:
2021-01-27 10:27:39.000, timestamp="2021-01-27 10:27:39.0", context="abc", searchType="Advanced Search", device="Mobile", criteria="Item Name", results="93751371"

 

I want to do a lookup of the values in the results field. 
Problem: in the JSON event, this is an array, in the None-JSON a string.
So I tried to use spath to extract results{} into field results and then do a lookup with that common field name for both sourcetypes:

 

<base search>
| spath output=results path=results{}
| lookup myLookup id as results 

 

The problem is, when I do this, the results field in the None-JSON event disappears...
So without the spath I have an auto-extracted results{} field in the JSON event and a results field in the None-JSON. Adding the spath removed the results field from the None-JSON event.
--> Why?

I have found a way to work around this, but I would like to understand the technical reasone behind the behaviour.

My workaround is:

 

| spath output=result path=results{}
| eval results=coalesce(results, result)
| lookup myLookup id as results 

 

Labels (1)
Tags (3)
0 Karma
1 Solution

bowesmana
SplunkTrust
SplunkTrust

@flle 

that's the way spath works, the result of spath on the non-json field will generate a null output, so results will overwritten.

Your workaround is the right solution for this and this is often the way you do things with Splunk when dealing with two or more different data types, e.g. the construct 

| eval myfield=if(data_type=1, type1_field, type2_field)
| eval anotherfield=case(data_type=1, type1_field, data_type=2, type2_field, data_type=3, type3_field, 1==1, "Unknown")

where you do conditional evaluations and field assignment based on your criteria.

 

View solution in original post

bowesmana
SplunkTrust
SplunkTrust

@flle 

that's the way spath works, the result of spath on the non-json field will generate a null output, so results will overwritten.

Your workaround is the right solution for this and this is often the way you do things with Splunk when dealing with two or more different data types, e.g. the construct 

| eval myfield=if(data_type=1, type1_field, type2_field)
| eval anotherfield=case(data_type=1, type1_field, data_type=2, type2_field, data_type=3, type3_field, 1==1, "Unknown")

where you do conditional evaluations and field assignment based on your criteria.

 

Get Updates on the Splunk Community!

Splunk at Cisco Live 2025: Learning, Innovation, and a Little Bit of Mr. Brightside

Pack your bags (and maybe your dancing shoes)—Cisco Live is heading to San Diego, June 8–12, 2025, and Splunk ...

Splunk App Dev Community Updates – What’s New and What’s Next

Welcome to your go-to roundup of everything happening in the Splunk App Dev Community! Whether you're building ...

The Latest Cisco Integrations With Splunk Platform!

Join us for an exciting tech talk where we’ll explore the latest integrations in Cisco &#43; Splunk! We’ve ...