Splunk Search

Extracting field with spath from JSON sourcetype overwrites field in other None-JSON sourcetype

flle
Path Finder

Hi,

looked through documentation and Splunk answers but did not find reason/root cause for the following obervation:

We have an index with 2 sourcetypes. one is JSON, the other plain text.
Event examples:

 

Sourcetype 1, JSON notation:
{ [-]
   context: xyz
   criteria: { [+]
   }
   device: desktop
   results: [ [-]
     50832171
   ]
   searchType: QuickSearch
}

Raw Text notation:
{"context":"xyz","device":"desktop","searchType":"QuickSearch","results":["50832171"],"criteria":{"Item name":"Example"}}

Sourcetype 2, None-JSON event:
2021-01-27 10:27:39.000, timestamp="2021-01-27 10:27:39.0", context="abc", searchType="Advanced Search", device="Mobile", criteria="Item Name", results="93751371"

 

I want to do a lookup of the values in the results field. 
Problem: in the JSON event, this is an array, in the None-JSON a string.
So I tried to use spath to extract results{} into field results and then do a lookup with that common field name for both sourcetypes:

 

<base search>
| spath output=results path=results{}
| lookup myLookup id as results 

 

The problem is, when I do this, the results field in the None-JSON event disappears...
So without the spath I have an auto-extracted results{} field in the JSON event and a results field in the None-JSON. Adding the spath removed the results field from the None-JSON event.
--> Why?

I have found a way to work around this, but I would like to understand the technical reasone behind the behaviour.

My workaround is:

 

| spath output=result path=results{}
| eval results=coalesce(results, result)
| lookup myLookup id as results 

 

Labels (1)
Tags (3)
0 Karma
1 Solution

bowesmana
SplunkTrust
SplunkTrust

@flle 

that's the way spath works, the result of spath on the non-json field will generate a null output, so results will overwritten.

Your workaround is the right solution for this and this is often the way you do things with Splunk when dealing with two or more different data types, e.g. the construct 

| eval myfield=if(data_type=1, type1_field, type2_field)
| eval anotherfield=case(data_type=1, type1_field, data_type=2, type2_field, data_type=3, type3_field, 1==1, "Unknown")

where you do conditional evaluations and field assignment based on your criteria.

 

View solution in original post

bowesmana
SplunkTrust
SplunkTrust

@flle 

that's the way spath works, the result of spath on the non-json field will generate a null output, so results will overwritten.

Your workaround is the right solution for this and this is often the way you do things with Splunk when dealing with two or more different data types, e.g. the construct 

| eval myfield=if(data_type=1, type1_field, type2_field)
| eval anotherfield=case(data_type=1, type1_field, data_type=2, type2_field, data_type=3, type3_field, 1==1, "Unknown")

where you do conditional evaluations and field assignment based on your criteria.

 

Get Updates on the Splunk Community!

Observability | How to Think About Instrumentation Overhead (White Paper)

Novice observability practitioners are often overly obsessed with performance. They might approach ...

Cloud Platform | Get Resiliency in the Cloud Event (Register Now!)

IDC Report: Enterprises Gain Higher Efficiency and Resiliency With Migration to Cloud  Today many enterprises ...

The Great Resilience Quest: 10th Leaderboard Update

The tenth leaderboard update (11.23-12.05) for The Great Resilience Quest is out &gt;&gt; As our brave ...