topic Searches returning unexpected results in Splunk Search

Searches returning unexpected results

tchamp — Tue, 11 Mar 2025 19:20:53 GMT

I am fairly new to Splunk. I am testing out different search queries and getting inconsistent results.

In this example I have some pretty simple json/logs with the following format

{ "data": { "tree": { "fruit": { "type": "Pear" } } } }

I'm trying several different searches and seeing some unexpected results.

"data.tree.fruit.type"="Apple" - Returns Apple only results (as expected)
*| spath "data.tree.fruit.type" | search "data.tree.fruit.type"=Apple - Returns Apple only results (as expected)
"data.tree.fruit.type"="Pear" - Returns NO results (unexpected?)
*| spath "data.tree.fruit.type" | search "data.tree.fruit.type"=Pear - Returns Pear only results (as expected)
"data.tree.fruit.type"="*" - Returns Apple only results (unexpected)

Can anyone shed some light on why I'm seeing the varying results?

Re: Searches returning unexpected results

bowesmana — Tue, 11 Mar 2025 21:37:50 GMT

It's likely that your auto extracted JSON fields are not extracting the entire object, i.e. if you search type=* and it does not find some values, then those values do not exist in that field in the auto extracted field.

The fact that they DO give results after the spath, indicates this.

What is the size of your JSON object. By default I believe it will only auto extract the first 5000 (5k?) bytes of a JSON object, so if you show "raw" in your display rather than the syntax highlighted view of the JSON, you can see where your fruit type field exists in the raw.

If this is the case, then you can add some calculated fields using spath eval statement to extract the fields, so they are always present before the search is run.

BTW, I'm not totally sure of the best practice way to manage this 5k limit, but the above will work.

Re: Searches returning unexpected results

tchamp — Tue, 11 Mar 2025 22:08:22 GMT

The JSON objects are very large and way over 5KB in size. I will look into calculated fields unless anyone else has a better suggestion.

Re: Searches returning unexpected results

bowesmana — Tue, 11 Mar 2025 22:25:35 GMT

Here are a couple of links to other posts here

https://community.splunk.com/t5/Getting-Data-In/JSON-field-extraction-beyond-5000-chars/m-p/549963#:~:text=The%20auto%2Dfield%2Dextraction%20stops,result%20list%20after%20a%20search.

https://community.splunk.com/t5/Getting-Data-In/Missing-events-JSON-payload-and-indexed-extractions/m-p/489113

If you start changing limits.conf - which is not simple with Cloud, it will affect general settings, so is not always the best way to go.

If you have a field that is not extracted and it's a simple field - i.e. a single value inside a JSON object with no multivalue component then a simple calculated field can work, i.e.

| eval type=spath(_raw, "data.tree.fruit.type")

In the conf just use the spath... part for the eval definition