topic Re: Search for a path the might not exist in Splunk Search

Search for a path the might not exist

robertlynch2020 — Wed, 21 May 2025 16:30:36 GMT

I have the following data (Below).

I have a situation where I want to search for "*" on a search and have it return all the data. resourceSpans{}.scopeSpans{}.spans{}.attributes{}.value.stringValue = "*"

However, this works for 99.9 % of my data, but the line below.

This path is not there. So when I run the command below, I get no results. However, I am looking for all data with the *. But as it's not there, it is excluding it. Is there any way I can still get the data back?

{"resourceSpans":[{"resource":{"attributes":[{"key":"process.pid","value":{"intValue":"600146"}},{"key":"service.instance.id","value":{"stringValue":"003nhhk3"}},{"key":"service.name","value":{"stringValue":"LAUNCHERMXMARKETRISK_MPC"}},{"key":"service.namespace","value":{"stringValue":"LAUNCHER"}},{"key":"telemetry.sdk.language","value":{"stringValue":"java"}},{"key":"telemetry.sdk.name","value":{"stringValue":"opentelemetry"}},{"key":"telemetry.sdk.version","value":{"stringValue":"1.34.0"}},{"key":"mx.env","value":{"stringValue":"dell945srv:13003"}}]},"scopeSpans":[{"scope":{"name":"mx-traces-api","version":"1.0.0"},"spans":[{"traceId":"10731f4b1d19380ceb33ae33672dbd5f","spanId":"cbf88ed07b403b48","parentSpanId":"3cfc7d85786b676b","name":"createSubmission","kind":1,"startTimeUnixNano":"1747152946314481406","endTimeUnixNano":"1747152946314775297","status":{}},{"traceId":"10731f4b1d19380ceb33ae33672dbd5f","spanId":"8ff7fabcab4b12d0","parentSpanId":"3cfc7d85786b676b","name":"createSubmission","kind":1,"startTimeUnixNano":"1747152946353054099","endTimeUnixNano":"1747152946353187644","status":{}},{"traceId":"10731f4b1d19380ceb33ae33672dbd5f","spanId":"4b14e49df1e1ffd8","parentSpanId":"3cfc7d85786b676b","name":"createSubmission","kind":1,"startTimeUnixNano":"1747152946474942393","endTimeUnixNano":"1747152946475042609","status":{}},{"traceId":"10731f4b1d19380ceb33ae33672dbd5f","spanId":"169b89bf118931d8","parentSpanId":"3cfc7d85786b676b","name":"createSubmission","kind":1,"startTimeUnixNano":"1747152946488875310","endTimeUnixNano":"1747152946488933120","status":{}}]}]}]}

Re: Search for a path the might not exist

livehybrid — Wed, 21 May 2025 16:34:45 GMT

Hi @robertlynch2020

I might be mis-understanding something here, but why are you searching for resourceSpans{}.scopeSpans{}.spans{}.attributes{}.value.stringValue = "*" if you want to include data which does not have it?

If you search for a field with value * then the field must exist.

It might help if we could understand your usecase here if you're able to share a little more info, please?

🌟 Did this answer help you? If so, please consider:

Adding karma to show it was useful
Marking it as the solution if it resolved your issue
Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

Re: Search for a path the might not exist

robertlynch2020 — Wed, 21 May 2025 16:50:00 GMT

Hi @livehybrid

Thanks for looking into this - cheers.

The issue is that I have filters (drop-downs) that I am using to zoom into data.

So when I pick a jobID it works very well for other tables

For example. The table below would have a filter

The code would look like this. So this is perfect for "CONSO_ABAQ | 31/03/2016 | 22".

However, if I put in * from the drop-down. I won't get the original line as it does not have resourceSpans{}.scopeSpans{}.spans{}.attributes{}.value.stringValue - however, I am looking to extract other data from that line to complete the table.

host="MARKET_RISK_PDT_V2" index="murex_logs" sourcetype="Market_Risk_DT" | search "resourceSpans{}.resource.attributes{}.value.stringValue"="*" | search resourceSpans{}.scopeSpans{}.spans{}.attributes{}.value.stringValue = "CONSO_ABAQ | 31/03/2016 | 22" | spath resourceSpans{}.scopeSpans{}.spans{} output=scopeSpans | stats count by scopeSpans | spath input=scopeSpans | rename startTimeUnixNano as start | rename endTimeUnixNano as end | eval _time=start/pow(10,9) | eval duration = end -start | eval duration= duration/1000000 | eval duration = round(duration,0) | eval parentSpanId =if(parentSpanId="" ,"0", $parentSpanId$) | rename name as SPAN_TYPE | search traceId = * | search spanId="*" OR parentSpanId="*" | stats avg(duration) as Average count(duration) AS count, stdev(duration) AS stdev, median(duration) AS median, exactperc75(duration) AS perc75, exactperc95(duration) AS perc95, exactperc99.5(duration) AS perc99.5, min(duration) AS min, max(duration) AS max by SPAN_TYPE | sort - Average

Re: Search for a path the might not exist

livehybrid — Wed, 21 May 2025 23:09:43 GMT

I see, okay - in that case I think the below might work for you? This works by setting the fieldName into the value so you dont need something=$token$ you just do $token$ as it already contains something= within it:

All:

<form version="1.1"> <label>Demo</label> <fieldset submitButton="false"> <input type="dropdown" token="testToken" searchWhenChanged="true"> <label>Test. Token</label> <choice value="*">All</choice> <choice value=""resourceSpans{}.resource.attributes{}.value.stringValue"="CONSO_ABAQ | 31/03/2016 | 23"">CONSO_ABAQ | 31/03/2016 | 23 (Static)</choice> <fieldForLabel>obj</fieldForLabel> <fieldForValue>option</fieldForValue> <search> <query>| makeresults | eval obj="CONSO_ABAQ | 31/03/2016 | 22" | eval option="\"resourceSpans{}.resource.attributes{}.value.stringValue\"=\"".obj."\""</query> <earliest>-24h@h</earliest> <latest>now</latest> </search> </input> </fieldset> <row> <panel> <table> <search> <query>|makeresults | eval _raw=json_set("{}","resourceSpans{}.resource.name.stringValue","Testing") |append [makeresults | eval _raw=json_set("{}","resourceSpans{}.resource.attributes{}.value.stringValue","CONSO_ABAQ | 31/03/2016 | 22")] |spath|search $testToken$</query> <earliest>-24h@h</earliest> <latest>now</latest> <sampleRatio>1</sampleRatio> </search> <option name="count">100</option> <option name="dataOverlayMode">none</option> <option name="drilldown">none</option> <option name="percentagesRow">false</option> <option name="refresh.display">progressbar</option> <option name="rowNumbers">false</option> <option name="totalsRow">false</option> <option name="wrap">true</option> </table> </panel> </row> </form>

🌟 Did this answer help you? If so, please consider:

Adding karma to show it was useful
Marking it as the solution if it resolved your issue
Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

Re: Search for a path the might not exist

myitlab42000 — Wed, 21 May 2025 23:45:57 GMT

hi,

the path you search is :

resourceSpans{}.scopeSpans{}.spans{}.attributes{}.value.stringValue

but in your json data example, the path to stringValue is :

resourceSpans{}.resource{}.attributes{}.value.stringValue

May be it help.

Re: Search for a path the might not exist

robertlynch2020 — Thu, 22 May 2025 15:39:31 GMT

I think we are close, and thanks for your efforts.

A couple of points

This is only a problem for | search resourceSpans{}.scopeSpans{}.spans{}.attributes{}.value.stringValue = "$Token_Mr_jobId$". This is the token that I am looking at.

Token_Mr_jobId - Can be a dynamic list, so more than the 2 I gave in the example.

So the question is, how to pass the dynamic selection?

<input type="dropdown" token="Token_Mr_jobId"> <label>JobId</label> <fieldForLabel>mr_jobId</fieldForLabel> <fieldForValue>mr_jobId</fieldForValue> <search> <query>host="$Host_Token$" index="murex_logs" sourcetype="Market_Risk_DT" "**mr_strategy**" "resourceSpans{}.resource.attributes{}.value.stringValue"="$TOKEN_Service_Namespace$" | fields - resourceSpans{}.* | spath path=resourceSpans{} | mvexpand resourceSpans{} | spath input=resourceSpans{} path=scopeSpans{} | fields - resourceSpans{} | mvexpand scopeSpans{} | spath input=scopeSpans{} path=spans{} | fields - scopeSpans{} | mvexpand spans{} | where match('spans{}', "mr_batchId") | spath input=spans{} path=attributes{} output=attributes | foreach mr_batchId mr_jobId [ eval <<FIELD>> = mvappend(<<FIELD>>, mvmap(attributes, if(spath(attributes, "key") != "<<FIELD>>", null(), spath(attributes, "value")))), <<FIELD>> = coalesce(spath(<<FIELD>>, "doubleValue"), spath(<<FIELD>>, "stringValue"))] | dedup _time mr_batchId ``` the above is key logic. If there is any doubt, you can also use | dedup _time mr_batchId mr_batch_compute_cpu_time mr_batch_compute_time ``` | table _time mr_batchId mr_batch_compute_cpu_time mr_batch_compute_time mr_batch_load_cpu_time mr_batch_load_time mr_strategy mr_jobId | table mr_jobId | dedup mr_jobId</query> <earliest>$time_token.earliest$</earliest> <latest>$time_token.latest$</latest> </search> <change> <condition match="$Token_Mr_jobId$ != "*""> <set token="TOKEN_Strategy">ON</set> <set token="TOKEN_TRACEID">*</set> </condition> <condition match="$Token_Mr_jobId$ = "*""> <unset token="TOKEN_Strategy"></unset> <set token="TOKEN_TRACEID">*</set> </condition> <condition> <set token="TOKEN_TRACEID">*</set> </condition> </change> <choice value="*">*</choice> <default>*</default> </input>

Also, to add - Here are 2 data sets. 1 with resourceSpans{}.scopeSpans{}.spans{}.attributes{}.value.stringValue and one with out.

With | search resourceSpans{}.scopeSpans{}.spans{}.attributes{}.value.stringValue = "CONSO_ABAQ | 31/03/2016 | 21"

With | search resourceSpans{}.scopeSpans{}.spans{}.attributes{}.value.stringValue = * (But I understand your great idea, that we will just put in "|search *")

Re: Search for a path the might not exist

robertlynch2020 — Thu, 22 May 2025 15:48:05 GMT

Yes - this is correct.

resourceSpans{}.scopeSpans{}.spans{}.attributes{}.value.stringValue is the path that the token can change on.

The screenshot below shows me using it and getting back 34 events.

If I run it with =*, I get 225

If I run it without the filter, I get 294. So the issue is when I put in * I want to get 294, as there are other parts to the data that I need to look at.

Re: Search for a path the might not exist

yuanliu — Fri, 23 May 2025 08:05:05 GMT

Let me simplify your problem statement by eliminating JSON path from the equation. The requirements are simply these:

In a dashboard, there is a dropdown input token, say SomeToken.
SomeToken has a fixed, predefined entry with label "All".
The rest of choices for SomeToken are populated by a search. I will call this search <tokenSearch>.
Events in dashboard panel may or may not contain a field of interest named SomeField.

If the user selects "All" (predefined, fixed value), all events should be returned regardless of SomeField.
If the user selects any other value populated by <tokenSearch>, only events with SomeField = SomeToken should be returned.

(In your case, SomeField is resourceSpans{}.scopeSpans{}.spans{}.attributes{}.value.stringValue, and you call SomeToken Token_Mr_jobId.)

@livehybrid already gives the solution: Do not return only SomeFieldValue in <tokenSearch> and use the value to populate both input label and input value. Use a different strategy in <tokenSearch>, i.e., return SomeFieldValue as input label, and "SomeField=SomeFieldValue" as input value.

<fieldForLabel>SomeFieldValue</fieldForLabel> <fieldForValue>SomeField=SomeFieldValue</fieldForValue>

Then, in your panel search, do not use "SomeField = $SomeToken$". Instead, simply insert $SomeToken$ as a search term.

One more suggestion: Do not use a pipe between your index search and the tokenized filter if SomeField is already extracted at search time. This unnecessarily burdens Splunk.

In the following demo dashboard, SomeField is substituted with thread_name from index _internal; thread_name_tok is SomeToken. The key here is <tokenSearch>:

index=_internal component=* | stats values(thread_name) as token_label | mvexpand token_label | eval token_value = "thread_name=" . token_label

This search differs from yours in one critical step: the last eval sets token_value to a search term involving field name thread_name, not a simple value of this field. Then, token_label and token_value are used to populate input label and value, respectively. In this example, I set "All" label to a zero-length character as value, which is equivalent to * in search command but more economical.

Full demo dashboard as follows. Play with it and fit it into your dataset.

<form version="1.1" theme="light"> <label>Search for a path the might not exist</label> <description>https://community.splunk.com/t5/Splunk-Search/Search-for-a-path-the-might-not-exist/m-p/746683#M241692</description> <fieldset submitButton="false"> <input type="dropdown" token="thread_name_tok" searchWhenChanged="true"> <label>Select thread_name</label> <choice value="">All events</choice> <default></default> <fieldForLabel>token_label</fieldForLabel> <fieldForValue>token_value</fieldForValue> <search> <query>index=_internal component=* | stats values(thread_name) as token_label | mvexpand token_label | eval token_value = "thread_name=" . token_label</query> <earliest>-15m</earliest> <latest>now</latest> </search> </input> </fieldset> <row> <panel> <title>Token value of your selection: >$thread_name_tok$<</title> <event> <search> <query>index=_internal component=* $thread_name_tok$</query> <earliest>-15m</earliest> <latest>now</latest> <sampleRatio>1</sampleRatio> </search> </event> </panel> </row> </form>

Hope this helps.

Re: Search for a path the might not exist

robertlynch2020 — Wed, 28 May 2025 08:39:16 GMT

Thanks very much for this great answer. This worked very well.

CHeers