Hi
I have the following data (Below).
I have a situation where I want to search for "*" on a search and have it return all the data. resourceSpans{}.scopeSpans{}.spans{}.attributes{}.value.stringValue = "*"
However, this works for 99.9 % of my data, but the line below.
This path is not there. So when I run the command below, I get no results. However, I am looking for all data with the *. But as it's not there, it is excluding it. Is there any way I can still get the data back?
{"resourceSpans":[{"resource":{"attributes":[{"key":"process.pid","value":{"intValue":"600146"}},{"key":"service.instance.id","value":{"stringValue":"003nhhk3"}},{"key":"service.name","value":{"stringValue":"LAUNCHERMXMARKETRISK_MPC"}},{"key":"service.namespace","value":{"stringValue":"LAUNCHER"}},{"key":"telemetry.sdk.language","value":{"stringValue":"java"}},{"key":"telemetry.sdk.name","value":{"stringValue":"opentelemetry"}},{"key":"telemetry.sdk.version","value":{"stringValue":"1.34.0"}},{"key":"mx.env","value":{"stringValue":"dell945srv:13003"}}]},"scopeSpans":[{"scope":{"name":"mx-traces-api","version":"1.0.0"},"spans":[{"traceId":"10731f4b1d19380ceb33ae33672dbd5f","spanId":"cbf88ed07b403b48","parentSpanId":"3cfc7d85786b676b","name":"createSubmission","kind":1,"startTimeUnixNano":"1747152946314481406","endTimeUnixNano":"1747152946314775297","status":{}},{"traceId":"10731f4b1d19380ceb33ae33672dbd5f","spanId":"8ff7fabcab4b12d0","parentSpanId":"3cfc7d85786b676b","name":"createSubmission","kind":1,"startTimeUnixNano":"1747152946353054099","endTimeUnixNano":"1747152946353187644","status":{}},{"traceId":"10731f4b1d19380ceb33ae33672dbd5f","spanId":"4b14e49df1e1ffd8","parentSpanId":"3cfc7d85786b676b","name":"createSubmission","kind":1,"startTimeUnixNano":"1747152946474942393","endTimeUnixNano":"1747152946475042609","status":{}},{"traceId":"10731f4b1d19380ceb33ae33672dbd5f","spanId":"169b89bf118931d8","parentSpanId":"3cfc7d85786b676b","name":"createSubmission","kind":1,"startTimeUnixNano":"1747152946488875310","endTimeUnixNano":"1747152946488933120","status":{}}]}]}]}
Let me simplify your problem statement by eliminating JSON path from the equation. The requirements are simply these:
(In your case, SomeField is resourceSpans{}.scopeSpans{}.spans{}.attributes{}.value.stringValue, and you call SomeToken Token_Mr_jobId.)
@livehybrid already gives the solution: Do not return only SomeFieldValue in <tokenSearch> and use the value to populate both input label and input value. Use a different strategy in <tokenSearch>, i.e., return SomeFieldValue as input label, and "SomeField=SomeFieldValue" as input value.
<fieldForLabel>SomeFieldValue</fieldForLabel>
<fieldForValue>SomeField=SomeFieldValue</fieldForValue>
Then, in your panel search, do not use "SomeField = $SomeToken$". Instead, simply insert $SomeToken$ as a search term.
One more suggestion: Do not use a pipe between your index search and the tokenized filter if SomeField is already extracted at search time. This unnecessarily burdens Splunk.
In the following demo dashboard, SomeField is substituted with thread_name from index _internal; thread_name_tok is SomeToken. The key here is <tokenSearch>:
index=_internal component=*
| stats values(thread_name) as token_label
| mvexpand token_label
| eval token_value = "thread_name=" . token_label
This search differs from yours in one critical step: the last eval sets token_value to a search term involving field name thread_name, not a simple value of this field. Then, token_label and token_value are used to populate input label and value, respectively. In this example, I set "All" label to a zero-length character as value, which is equivalent to * in search command but more economical.
Full demo dashboard as follows. Play with it and fit it into your dataset.
<form version="1.1" theme="light">
<label>Search for a path the might not exist</label>
<description>https://community.splunk.com/t5/Splunk-Search/Search-for-a-path-the-might-not-exist/m-p/746683#M241692</description>
<fieldset submitButton="false">
<input type="dropdown" token="thread_name_tok" searchWhenChanged="true">
<label>Select thread_name</label>
<choice value="">All events</choice>
<default></default>
<fieldForLabel>token_label</fieldForLabel>
<fieldForValue>token_value</fieldForValue>
<search>
<query>index=_internal component=*
| stats values(thread_name) as token_label
| mvexpand token_label
| eval token_value = "thread_name=" . token_label</query>
<earliest>-15m</earliest>
<latest>now</latest>
</search>
</input>
</fieldset>
<row>
<panel>
<title>Token value of your selection: >$thread_name_tok$<</title>
<event>
<search>
<query>index=_internal component=* $thread_name_tok$</query>
<earliest>-15m</earliest>
<latest>now</latest>
<sampleRatio>1</sampleRatio>
</search>
</event>
</panel>
</row>
</form>
Hope this helps.
Let me simplify your problem statement by eliminating JSON path from the equation. The requirements are simply these:
(In your case, SomeField is resourceSpans{}.scopeSpans{}.spans{}.attributes{}.value.stringValue, and you call SomeToken Token_Mr_jobId.)
@livehybrid already gives the solution: Do not return only SomeFieldValue in <tokenSearch> and use the value to populate both input label and input value. Use a different strategy in <tokenSearch>, i.e., return SomeFieldValue as input label, and "SomeField=SomeFieldValue" as input value.
<fieldForLabel>SomeFieldValue</fieldForLabel>
<fieldForValue>SomeField=SomeFieldValue</fieldForValue>
Then, in your panel search, do not use "SomeField = $SomeToken$". Instead, simply insert $SomeToken$ as a search term.
One more suggestion: Do not use a pipe between your index search and the tokenized filter if SomeField is already extracted at search time. This unnecessarily burdens Splunk.
In the following demo dashboard, SomeField is substituted with thread_name from index _internal; thread_name_tok is SomeToken. The key here is <tokenSearch>:
index=_internal component=*
| stats values(thread_name) as token_label
| mvexpand token_label
| eval token_value = "thread_name=" . token_label
This search differs from yours in one critical step: the last eval sets token_value to a search term involving field name thread_name, not a simple value of this field. Then, token_label and token_value are used to populate input label and value, respectively. In this example, I set "All" label to a zero-length character as value, which is equivalent to * in search command but more economical.
Full demo dashboard as follows. Play with it and fit it into your dataset.
<form version="1.1" theme="light">
<label>Search for a path the might not exist</label>
<description>https://community.splunk.com/t5/Splunk-Search/Search-for-a-path-the-might-not-exist/m-p/746683#M241692</description>
<fieldset submitButton="false">
<input type="dropdown" token="thread_name_tok" searchWhenChanged="true">
<label>Select thread_name</label>
<choice value="">All events</choice>
<default></default>
<fieldForLabel>token_label</fieldForLabel>
<fieldForValue>token_value</fieldForValue>
<search>
<query>index=_internal component=*
| stats values(thread_name) as token_label
| mvexpand token_label
| eval token_value = "thread_name=" . token_label</query>
<earliest>-15m</earliest>
<latest>now</latest>
</search>
</input>
</fieldset>
<row>
<panel>
<title>Token value of your selection: >$thread_name_tok$<</title>
<event>
<search>
<query>index=_internal component=* $thread_name_tok$</query>
<earliest>-15m</earliest>
<latest>now</latest>
<sampleRatio>1</sampleRatio>
</search>
</event>
</panel>
</row>
</form>
Hope this helps.
Hi
Thanks very much for this great answer. This worked very well.
CHeers
hi,
the path you search is :
resourceSpans{}.scopeSpans{}.spans{}.attributes{}.value.stringValue
but in your json data example, the path to stringValue is :
resourceSpans{}.resource{}.attributes{}.value.stringValue
May be it help.
Hi
Yes - this is correct.
resourceSpans{}.scopeSpans{}.spans{}.attributes{}.value.stringValue is the path that the token can change on.
The screenshot below shows me using it and getting back 34 events.
If I run it with =*, I get 225
If I run it without the filter, I get 294. So the issue is when I put in * I want to get 294, as there are other parts to the data that I need to look at.
I might be mis-understanding something here, but why are you searching for resourceSpans{}.scopeSpans{}.spans{}.attributes{}.value.stringValue = "*" if you want to include data which does not have it?
If you search for a field with value * then the field must exist.
It might help if we could understand your usecase here if you're able to share a little more info, please?
🌟 Did this answer help you? If so, please consider:
Your feedback encourages the volunteers in this community to continue contributing
Hi @livehybrid
Thanks for looking into this - cheers.
The issue is that I have filters (drop-downs) that I am using to zoom into data.
So when I pick a jobID it works very well for other tables
For example. The table below would have a filter
The code would look like this. So this is perfect for "CONSO_ABAQ | 31/03/2016 | 22".
However, if I put in * from the drop-down. I won't get the original line as it does not have resourceSpans{}.scopeSpans{}.spans{}.attributes{}.value.stringValue - however, I am looking to extract other data from that line to complete the table.
host="MARKET_RISK_PDT_V2" index="murex_logs" sourcetype="Market_Risk_DT"
| search "resourceSpans{}.resource.attributes{}.value.stringValue"="*"
| search resourceSpans{}.scopeSpans{}.spans{}.attributes{}.value.stringValue = "CONSO_ABAQ | 31/03/2016 | 22"
| spath resourceSpans{}.scopeSpans{}.spans{} output=scopeSpans
| stats count by scopeSpans
| spath input=scopeSpans
| rename startTimeUnixNano as start
| rename endTimeUnixNano as end
| eval _time=start/pow(10,9)
| eval duration = end -start
| eval duration= duration/1000000
| eval duration = round(duration,0)
| eval parentSpanId =if(parentSpanId="" ,"0", $parentSpanId$)
| rename name as SPAN_TYPE
| search traceId = *
| search spanId="*" OR parentSpanId="*"
| stats avg(duration) as Average count(duration) AS count, stdev(duration) AS stdev, median(duration) AS median, exactperc75(duration) AS perc75, exactperc95(duration) AS perc95, exactperc99.5(duration) AS perc99.5, min(duration) AS min, max(duration) AS max by SPAN_TYPE
| sort - Average
I see, okay - in that case I think the below might work for you? This works by setting the fieldName into the value so you dont need something=$token$ you just do $token$ as it already contains something= within it:
All:
<form version="1.1">
<label>Demo</label>
<fieldset submitButton="false">
<input type="dropdown" token="testToken" searchWhenChanged="true">
<label>Test. Token</label>
<choice value="*">All</choice>
<choice value=""resourceSpans{}.resource.attributes{}.value.stringValue"="CONSO_ABAQ | 31/03/2016 | 23"">CONSO_ABAQ | 31/03/2016 | 23 (Static)</choice>
<fieldForLabel>obj</fieldForLabel>
<fieldForValue>option</fieldForValue>
<search>
<query>| makeresults | eval obj="CONSO_ABAQ | 31/03/2016 | 22"
| eval option="\"resourceSpans{}.resource.attributes{}.value.stringValue\"=\"".obj."\""</query>
<earliest>-24h@h</earliest>
<latest>now</latest>
</search>
</input>
</fieldset>
<row>
<panel>
<table>
<search>
<query>|makeresults | eval _raw=json_set("{}","resourceSpans{}.resource.name.stringValue","Testing")
|append [makeresults | eval _raw=json_set("{}","resourceSpans{}.resource.attributes{}.value.stringValue","CONSO_ABAQ | 31/03/2016 | 22")] |spath|search $testToken$</query>
<earliest>-24h@h</earliest>
<latest>now</latest>
<sampleRatio>1</sampleRatio>
</search>
<option name="count">100</option>
<option name="dataOverlayMode">none</option>
<option name="drilldown">none</option>
<option name="percentagesRow">false</option>
<option name="refresh.display">progressbar</option>
<option name="rowNumbers">false</option>
<option name="totalsRow">false</option>
<option name="wrap">true</option>
</table>
</panel>
</row>
</form>
🌟 Did this answer help you? If so, please consider:
Your feedback encourages the volunteers in this community to continue contributing
Hi
I think we are close, and thanks for your efforts.
A couple of points
This is only a problem for | search resourceSpans{}.scopeSpans{}.spans{}.attributes{}.value.stringValue = "$Token_Mr_jobId$". This is the token that I am looking at.
Token_Mr_jobId - Can be a dynamic list, so more than the 2 I gave in the example.
So the question is, how to pass the dynamic selection?
<input type="dropdown" token="Token_Mr_jobId">
<label>JobId</label>
<fieldForLabel>mr_jobId</fieldForLabel>
<fieldForValue>mr_jobId</fieldForValue>
<search>
<query>host="$Host_Token$" index="murex_logs" sourcetype="Market_Risk_DT" "**mr_strategy**"
"resourceSpans{}.resource.attributes{}.value.stringValue"="$TOKEN_Service_Namespace$"
| fields - resourceSpans{}.*
| spath path=resourceSpans{}
| mvexpand resourceSpans{}
| spath input=resourceSpans{} path=scopeSpans{}
| fields - resourceSpans{}
| mvexpand scopeSpans{}
| spath input=scopeSpans{} path=spans{}
| fields - scopeSpans{}
| mvexpand spans{}
| where match('spans{}', "mr_batchId")
| spath input=spans{} path=attributes{} output=attributes
| foreach mr_batchId mr_jobId
[ eval <<FIELD>> = mvappend(<<FIELD>>, mvmap(attributes, if(spath(attributes, "key") != "<<FIELD>>", null(), spath(attributes, "value")))),
<<FIELD>> = coalesce(spath(<<FIELD>>, "doubleValue"), spath(<<FIELD>>, "stringValue"))]
| dedup _time mr_batchId
``` the above is key logic. If there is any doubt, you can also use
| dedup _time mr_batchId mr_batch_compute_cpu_time mr_batch_compute_time
```
| table _time mr_batchId mr_batch_compute_cpu_time mr_batch_compute_time mr_batch_load_cpu_time mr_batch_load_time mr_strategy mr_jobId
| table mr_jobId
| dedup mr_jobId</query>
<earliest>$time_token.earliest$</earliest>
<latest>$time_token.latest$</latest>
</search>
<change>
<condition match="$Token_Mr_jobId$ != "*"">
<set token="TOKEN_Strategy">ON</set>
<set token="TOKEN_TRACEID">*</set>
</condition>
<condition match="$Token_Mr_jobId$ = "*"">
<unset token="TOKEN_Strategy"></unset>
<set token="TOKEN_TRACEID">*</set>
</condition>
<condition>
<set token="TOKEN_TRACEID">*</set>
</condition>
</change>
<choice value="*">*</choice>
<default>*</default>
</input>
Also, to add - Here are 2 data sets. 1 with resourceSpans{}.scopeSpans{}.spans{}.attributes{}.value.stringValue and one with out.
With | search resourceSpans{}.scopeSpans{}.spans{}.attributes{}.value.stringValue = "CONSO_ABAQ | 31/03/2016 | 21"
{"resourceSpans":[{"resource":{"attributes":[{"key":"telemetry.sdk.language","value":{"stringValue":"cpp"}},{"key":"service.name","value":{"stringValue":"MXMARKETRISK.ENGINE.MX"}},{"key":"service.namespace","value":{"stringValue":"MXMARKETRISK.SERVICE"}},{"key":"process.pid","value":{"intValue":"604252"}},{"key":"service.instance.id","value":{"stringValue":"003nhhkz"}},{"key":"telemetry.sdk.name","value":{"stringValue":"opentelemetry"}},{"key":"telemetry.sdk.version","value":{"stringValue":"1.12.0"}},{"key":"mx.env","value":{"stringValue":"dell945srv:13003"}}]},"scopeSpans":[{"scope":{"name":"murex::observability_otel_backend::tracing","version":"v1"},"spans":[{"traceId":"10731f4b1d19380ceb33ae33672dbd5f","spanId":"0392a58e2dfaaa4c","parentSpanId":"ebce3b37999c2ea1","name":"scenario_reaction","kind":1,"startTimeUnixNano":"1747148775503846985","endTimeUnixNano":"1747148782361058175","attributes":[{"key":"market_risk_span","value":{"stringValue":"true"}}],"status":{}},{"traceId":"10731f4b1d19380ceb33ae33672dbd5f","spanId":"ebce3b37999c2ea1","parentSpanId":"825cbaedeb509365","name":"scenario_apply","kind":1,"startTimeUnixNano":"1747148775477950524","endTimeUnixNano":"1747148782362084106","attributes":[{"key":"market_risk_span","value":{"stringValue":"true"}},{"key":"mr_scenario","value":{"stringValue":"10"}}],"status":{}},{"traceId":"10731f4b1d19380ceb33ae33672dbd5f","spanId":"04307bd9c64e20e8","parentSpanId":"825cbaedeb509365","name":"structured_position_evaluation","kind":1,"startTimeUnixNano":"1747148782362177082","endTimeUnixNano":"1747148782379867824","attributes":[{"key":"market_risk_span","value":{"stringValue":"true"}},{"key":"mr_scenario","value":{"stringValue":"10"}}],"status":{}},{"traceId":"10731f4b1d19380ceb33ae33672dbd5f","spanId":"d2abbf63ac87acb4","parentSpanId":"825cbaedeb509365","name":"position_evaluation","kind":1,"startTimeUnixNano":"1747148782380422079","endTimeUnixNano":"1747148782509071609","attributes":[{"key":"market_risk_span","value":{"stringValue":"true"}},{"key":"mr_scenario","value":{"stringValue":"10"}}],"status":{}},{"traceId":"10731f4b1d19380ceb33ae33672dbd5f","spanId":"cc80374715a2e755","parentSpanId":"431a2a6341ac4120","name":"scenario_reaction","kind":1,"startTimeUnixNano":"1747148782510724546","endTimeUnixNano":"1747148782599301641","attributes":[{"key":"market_risk_span","value":{"stringValue":"true"}}],"status":{}},{"traceId":"10731f4b1d19380ceb33ae33672dbd5f","spanId":"431a2a6341ac4120","parentSpanId":"825cbaedeb509365","name":"scenario_restore","kind":1,"startTimeUnixNano":"1747148782509167483","endTimeUnixNano":"1747148782605479850","attributes":[{"key":"market_risk_span","value":{"stringValue":"true"}},{"key":"mr_scenario","value":{"stringValue":"10"}}],"status":{}},{"traceId":"10731f4b1d19380ceb33ae33672dbd5f","spanId":"825cbaedeb509365","parentSpanId":"8e2a92d0a40f203b","name":"scenario_all_apply","kind":1,"startTimeUnixNano":"1747148711449995255","endTimeUnixNano":"1747148782623591981","attributes":[{"key":"market_risk_span","value":{"stringValue":"true"}},{"key":"mr_scenario_nb","value":{"stringValue":"10"}}],"status":{}},{"traceId":"10731f4b1d19380ceb33ae33672dbd5f","spanId":"8e2a92d0a40f203b","parentSpanId":"8e5190bbe86bdaff","name":"fullreval_task","kind":1,"startTimeUnixNano":"1747148638253233246","endTimeUnixNano":"1747148782639890403","attributes":[{"key":"market_risk_span","value":{"stringValue":"true"}},{"key":"mr_batchId","value":{"stringValue":"40"}},{"key":"mr_batchType","value":{"stringValue":"Full Revaluation"}},{"key":"mr_bucketName","value":{"stringValue":""}},{"key":"mr_jobDomain","value":{"stringValue":"Market Risk"}},{"key":"mr_jobId","value":{"stringValue":"CONSO_ABAQ | 31/03/2016 | 21"}},{"key":"mr_strategy","value":{"stringValue":"typo_Callable Bond"}},{"key":"mr_uuid","value":{"stringValue":"7dcdf03d-9dd0-42f6-b0f4-e2508283ff44"}},{"key":"mrb_batch_affinity","value":{"stringValue":"CONSO_ABAQ_run_Batch|CONSO_ABAQ|2016/03/31|21_FullReval0_00040"}},{"key":"mr_batch_compute_cpu_time","value":{"doubleValue":71.000477}},{"key":"mr_batch_compute_time","value":{"doubleValue":71.351}},{"key":"mr_batch_load_cpu_time","value":{"doubleValue":61.569109000000005}},{"key":"mr_batch_load_time","value":{"doubleValue":71.597}},{"key":"mr_batch_status","value":{"stringValue":"WARNING"}},{"key":"mr_batch_total_cpu_time","value":{"doubleValue":133.924506}},{"key":"mr_batch_total_time","value":{"doubleValue":144.375}}],"status":{}}]}]}]}
With | search resourceSpans{}.scopeSpans{}.spans{}.attributes{}.value.stringValue = * (But I understand your great idea, that we will just put in "|search *")
{"resourceSpans":[{"resource":{"attributes":[{"key":"process.pid","value":{"intValue":"600146"}},{"key":"service.instance.id","value":{"stringValue":"003nhhk3"}},{"key":"service.name","value":{"stringValue":"LAUNCHERMXMARKETRISK_MPC"}},{"key":"service.namespace","value":{"stringValue":"LAUNCHER"}},{"key":"telemetry.sdk.language","value":{"stringValue":"java"}},{"key":"telemetry.sdk.name","value":{"stringValue":"opentelemetry"}},{"key":"telemetry.sdk.version","value":{"stringValue":"1.34.0"}},{"key":"mx.env","value":{"stringValue":"dell945srv:13003"}}]},"scopeSpans":[{"scope":{"name":"mx-traces-api","version":"1.0.0"},"spans":[{"traceId":"10731f4b1d19380ceb33ae33672dbd5f","spanId":"cbf88ed07b403b48","parentSpanId":"3cfc7d85786b676b","name":"createSubmission","kind":1,"startTimeUnixNano":"1747152946314481406","endTimeUnixNano":"1747152946314775297","status":{}},{"traceId":"10731f4b1d19380ceb33ae33672dbd5f","spanId":"8ff7fabcab4b12d0","parentSpanId":"3cfc7d85786b676b","name":"createSubmission","kind":1,"startTimeUnixNano":"1747152946353054099","endTimeUnixNano":"1747152946353187644","status":{}},{"traceId":"10731f4b1d19380ceb33ae33672dbd5f","spanId":"4b14e49df1e1ffd8","parentSpanId":"3cfc7d85786b676b","name":"createSubmission","kind":1,"startTimeUnixNano":"1747152946474942393","endTimeUnixNano":"1747152946475042609","status":{}},{"traceId":"10731f4b1d19380ceb33ae33672dbd5f","spanId":"169b89bf118931d8","parentSpanId":"3cfc7d85786b676b","name":"createSubmission","kind":1,"startTimeUnixNano":"1747152946488875310","endTimeUnixNano":"1747152946488933120","status":{}}]}]}]}