Hi,
looked through documentation and Splunk answers but did not find reason/root cause for the following obervation:
We have an index with 2 sourcetypes. one is JSON, the other plain text.
Event examples:
Sourcetype 1, JSON notation:
{ [-]
context: xyz
criteria: { [+]
}
device: desktop
results: [ [-]
50832171
]
searchType: QuickSearch
}
Raw Text notation:
{"context":"xyz","device":"desktop","searchType":"QuickSearch","results":["50832171"],"criteria":{"Item name":"Example"}}
Sourcetype 2, None-JSON event:
2021-01-27 10:27:39.000, timestamp="2021-01-27 10:27:39.0", context="abc", searchType="Advanced Search", device="Mobile", criteria="Item Name", results="93751371"
I want to do a lookup of the values in the results field.
Problem: in the JSON event, this is an array, in the None-JSON a string.
So I tried to use spath to extract results{} into field results and then do a lookup with that common field name for both sourcetypes:
<base search>
| spath output=results path=results{}
| lookup myLookup id as results
The problem is, when I do this, the results field in the None-JSON event disappears...
So without the spath I have an auto-extracted results{} field in the JSON event and a results field in the None-JSON. Adding the spath removed the results field from the None-JSON event.
--> Why?
I have found a way to work around this, but I would like to understand the technical reasone behind the behaviour.
My workaround is:
| spath output=result path=results{}
| eval results=coalesce(results, result)
| lookup myLookup id as results
that's the way spath works, the result of spath on the non-json field will generate a null output, so results will overwritten.
Your workaround is the right solution for this and this is often the way you do things with Splunk when dealing with two or more different data types, e.g. the construct
| eval myfield=if(data_type=1, type1_field, type2_field)
| eval anotherfield=case(data_type=1, type1_field, data_type=2, type2_field, data_type=3, type3_field, 1==1, "Unknown")
where you do conditional evaluations and field assignment based on your criteria.
that's the way spath works, the result of spath on the non-json field will generate a null output, so results will overwritten.
Your workaround is the right solution for this and this is often the way you do things with Splunk when dealing with two or more different data types, e.g. the construct
| eval myfield=if(data_type=1, type1_field, type2_field)
| eval anotherfield=case(data_type=1, type1_field, data_type=2, type2_field, data_type=3, type3_field, 1==1, "Unknown")
where you do conditional evaluations and field assignment based on your criteria.