Splunk Search

How to include events with non existing fields in a search string ?

AmineTN
Explorer

After fixing filters on some fields that don't exist in all the events, I tried to apply these filters on the graphs and the problem here is that when Splunk reads the search string of a graph, it gets only the events where the fields exist and it excludes the other events. As a result all the statistics and the graphs are wrong !!

Ayone has a solution please ??

Thanks in Advance.

Labels (6)
0 Karma
1 Solution

ITWhisperer
SplunkTrust
SplunkTrust

So, essentially, your ALL option is filtering out all stats events with null SuperForceCouponStatus. Try changing the value of the drop down options to include the condition or an empty string when ALL is selected. Something like this

<input type="dropdown" token="filter_SuperForce">
<label>SuperForce</label>
<choice value="">ALL</choice>
<fieldForLabel>SuperForceCouponStatus</fieldForLabel>
<fieldForValue>SuperForceCouponStatusFilter</fieldForValue>
<search>
<query>|tstats count where index=$filter_index$ $filter_backend$ 
by SuperForceCouponStatus
| eval SuperForceCouponStatusFilter="SuperForceCouponStatus=\"".SuperForceCouponStatus."\""</query>
<earliest></earliest>
<latest>now</latest>
</search>
<default></default>
</input>

View solution in original post

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @AmineTN ,

good for you, see next time!

Ciao and happy splunking

Giuseppe

P.S.: Karma Points are appreciated by all the contributors 😉

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

It is not clear to me what it is you are trying to do - can you give an example of the search you are doing and where the filters are defined and used? If the statistics are incorrect, are the filters being applied at the correct point in the search?

0 Karma

AmineTN
Explorer

I will explain more here with some examples.

For these two JSON events the first contain the field "SuperForceCouponStatus" but the second don't.

event1.pngevent2.png

Then I made a filter for "SuperForceCouponStatus" as following :

<input type="dropdown" token="filter_SuperForce">
<label>SuperForce</label>
<choice value="*">ALL</choice>
<prefix>SuperForceCouponStatus="</prefix>
<suffix>"</suffix>
<fieldForLabel>SuperForceCouponStatus</fieldForLabel>
<fieldForValue>SuperForceCouponStatus</fieldForValue>
<search>
<query>|tstats count where index=$filter_index$ $filter_backend$
by SuperForceCouponStatus</query>
<earliest></earliest>
<latest>now</latest>
</search>
<default>*</default>
</input>

 After that I made a graph to display the number of events depending on the filters qs following :

<panel>
<single>
<title>Total number of Transactions</title>
<search>
<query>| tstats count where index=$filter_index$
$filter_backend$
$filter_status$
$filter_SuperForce$ |</query>
<earliest>$time_range.earliest$</earliest>
<latest>$time_range.latest$</latest>
</search>
<option name="colorBy">value</option>
<option name="colorMode">block</option>
<option name="drilldown">none</option>
<option name="numberPrecision">0</option>
<option name="rangeColors">["0x53a051","0x0877a6","0xf8be34","0xf1813f","0xdc4e41"]</option>
<option name="rangeValues">[100,10000,1000000,10000000]</option>
<option name="refresh.display">progressbar</option>
<option name="showSparkline">1</option>
<option name="showTrendIndicator">1</option>
<option name="trellis.enabled">0</option>
<option name="trellis.scales.shared">1</option>
<option name="trellis.size">medium</option>
<option name="trendColorInterpretation">standard</option>
<option name="trendDisplayMode">absolute</option>
<option name="underLabel">transactions</option>
<option name="unitPosition">after</option>
<option name="useColors">1</option>
<option name="useThousandSeparators">1</option>
</single>
</panel>

And the problem is that this graph takes only the events that have the field "SuperForceCouponStatus" because $filter_SuperForce$ is applied in its search string !!

Hope the problem is more clear now 🙂

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

So, essentially, your ALL option is filtering out all stats events with null SuperForceCouponStatus. Try changing the value of the drop down options to include the condition or an empty string when ALL is selected. Something like this

<input type="dropdown" token="filter_SuperForce">
<label>SuperForce</label>
<choice value="">ALL</choice>
<fieldForLabel>SuperForceCouponStatus</fieldForLabel>
<fieldForValue>SuperForceCouponStatusFilter</fieldForValue>
<search>
<query>|tstats count where index=$filter_index$ $filter_backend$ 
by SuperForceCouponStatus
| eval SuperForceCouponStatusFilter="SuperForceCouponStatus=\"".SuperForceCouponStatus."\""</query>
<earliest></earliest>
<latest>now</latest>
</search>
<default></default>
</input>
0 Karma

AmineTN
Explorer

Thanks a lot @ITWhisperer , It's working perfectly !!!

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @AmineTN,

did yu tried to add a fillnull to give a value (e.g. "N.A. " or "-") to this field when there isn't any value?

| fillnull value="N.A." SuperForceCouponStatus

Ciao.

Giuseppe

0 Karma

AmineTN
Explorer

Yes I tried and the problem the fillnull command is applied after filtering the events, So it's useless !!

Thanks for your try ❤️

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...