Splunk Search

Extracting field with spath from JSON sourcetype overwrites field in other None-JSON sourcetype

flle
Path Finder

Hi,

looked through documentation and Splunk answers but did not find reason/root cause for the following obervation:

We have an index with 2 sourcetypes. one is JSON, the other plain text.
Event examples:

 

Sourcetype 1, JSON notation:
{ [-]
   context: xyz
   criteria: { [+]
   }
   device: desktop
   results: [ [-]
     50832171
   ]
   searchType: QuickSearch
}

Raw Text notation:
{"context":"xyz","device":"desktop","searchType":"QuickSearch","results":["50832171"],"criteria":{"Item name":"Example"}}

Sourcetype 2, None-JSON event:
2021-01-27 10:27:39.000, timestamp="2021-01-27 10:27:39.0", context="abc", searchType="Advanced Search", device="Mobile", criteria="Item Name", results="93751371"

 

I want to do a lookup of the values in the results field. 
Problem: in the JSON event, this is an array, in the None-JSON a string.
So I tried to use spath to extract results{} into field results and then do a lookup with that common field name for both sourcetypes:

 

<base search>
| spath output=results path=results{}
| lookup myLookup id as results 

 

The problem is, when I do this, the results field in the None-JSON event disappears...
So without the spath I have an auto-extracted results{} field in the JSON event and a results field in the None-JSON. Adding the spath removed the results field from the None-JSON event.
--> Why?

I have found a way to work around this, but I would like to understand the technical reasone behind the behaviour.

My workaround is:

 

| spath output=result path=results{}
| eval results=coalesce(results, result)
| lookup myLookup id as results 

 

Labels (1)
Tags (3)
0 Karma
1 Solution

bowesmana
SplunkTrust
SplunkTrust

@flle 

that's the way spath works, the result of spath on the non-json field will generate a null output, so results will overwritten.

Your workaround is the right solution for this and this is often the way you do things with Splunk when dealing with two or more different data types, e.g. the construct 

| eval myfield=if(data_type=1, type1_field, type2_field)
| eval anotherfield=case(data_type=1, type1_field, data_type=2, type2_field, data_type=3, type3_field, 1==1, "Unknown")

where you do conditional evaluations and field assignment based on your criteria.

 

View solution in original post

bowesmana
SplunkTrust
SplunkTrust

@flle 

that's the way spath works, the result of spath on the non-json field will generate a null output, so results will overwritten.

Your workaround is the right solution for this and this is often the way you do things with Splunk when dealing with two or more different data types, e.g. the construct 

| eval myfield=if(data_type=1, type1_field, type2_field)
| eval anotherfield=case(data_type=1, type1_field, data_type=2, type2_field, data_type=3, type3_field, 1==1, "Unknown")

where you do conditional evaluations and field assignment based on your criteria.

 

Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...