Monitoring Splunk

Post-process / Base search is slow

sojanmathew
New Member

I've base search / post process as follows, but it is taking more time than separate in-line query.

<search id="baseSearch">
    <query>
      index=testapp OutgoingCall=google  | stats count by Result
    </query>
    <earliest>-1d@h</earliest>
    <latest>now</latest>
  </search>

<panel>
      <single>
        <title>Total</title>
        <search base="baseSearch">
          <query>
            stats sum(count)
          </query>
        </search>
      </single>
    </panel>

<panel>
<single>
 <search base="baseSearch">
          <query>
             search Result=Success | stats sum(count) AS successCount 
            </query>
        </search>
</single>
</panel>
<panel>
      <single>
        <title>Failed</title>
        <search base="baseSearch">
          <query>search Result=Failed | stats sum(count) as failedCount</query>
        </search>
      </single>
    </panel>

I used following doc as reference:
http://docs.splunk.com/Documentation/Splunk/7.0.3/Viz/Savedsearches

Why this is very slow? Am I doing something wrong ?
Note: splunk enterprise ver 6.6.3

Tags (1)
0 Karma

niketnilay
Legend

@sojanmathew, since you are on Splunk 6.6.3 even if you have multiple rows of Results you can use Trellis Layout to Split the Single Values by Results. Even if you wanted to use two separate Single Value Panels(in case formatting options for both Single Value are different), you can use stats with eval to get Success and Failed count in Single row and then use Search Event Handler <done> or <progress> to pass on the result to Single Value Panels.

alt text

Try the following run anywhere dashboard example based on Splunk's _internal index:
(PS: I have converted log_level as per required field/value i.e. Result="Success" and Result="Failed")

<dashboard>
  <label>Single Value Success And Failed</label>
  <row>
    <panel depends="$alwaysHideCSSPanel$">
      <html>
        <style>
          #singleSuccess h3.dashboard-element-title, #singleFailed h3.dashboard-element-title{
            text-align:center !important;
          }
        </style>
      </html>
    </panel>
  </row>
  <row>
    <panel>
      <title>Stats Generates Single Row One Column for Failed and Another for Success</title>
      <table>
        <search>
          <query>index=_internal sourcetype=splunkd log_level=*
          | eval Result=if(log_level="INFO","Success","Failed")
          | stats count(eval(Result=="Failed")) as Failed count(eval(Result=="Success")) as Success</query>
          <earliest>-1d@h</earliest>
          <latest>now</latest>
          <done>
            <condition match="$job.resultCount$==0">
              <set token="tokSuccess">0</set>
              <set token="tokFailed">0</set>
            </condition>
            <condition>
              <set token="tokSuccess">$result.Success$</set>
              <set token="tokFailed">$result.Failed$</set>
            </condition>
          </done>
        </search>
        <option name="drilldown">none</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
    <panel>
      <title>Two Single Value visualizations in the same Panel</title>
      <single id="singleSuccess">
        <title>Failed</title>
        <search>
          <query>| makeresults 
| fields - _time
| eval Failed=$tokFailed$</query>
          <earliest>-1s</earliest>
          <latest>now</latest>
        </search>
        <option name="refresh.display">progressbar</option>
        <option name="useThousandSeparators">0</option>
      </single>
      <single id="singleFailed">
        <title>Success</title>
        <search>
          <query>| makeresults
| fields - _time
| eval Success=$tokSuccess$</query>
          <earliest>-1s</earliest>
          <latest>now</latest>
        </search>
        <option name="refresh.display">progressbar</option>
        <option name="useThousandSeparators">0</option>
      </single>
    </panel>
  </row>
  <row>
    <panel>
      <title>Stats generates two rows one for Failed and another for Success</title>
      <table>
        <search>
          <query>index=_internal sourcetype=splunkd log_level=*
          | eval Result=if(log_level="INFO","Success","Failed")
          | stats count by Result</query>
          <earliest>-1d@h</earliest>
          <latest>now</latest>
          <progress>
            <condition match="$job.resultCount$==0">
              <set token="tokSuccess">0</set>
              <set token="tokFailed">0</set>
            </condition>
            <condition>
              <set token="tokSuccess">$result.Success$</set>
              <set token="tokFailed">$result.Failed$</set>
            </condition>
          </progress>
        </search>
        <option name="drilldown">none</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
    <panel>
      <title>Single Value Using Trellis</title>
      <single>
        <search>
          <query>index=_internal sourcetype=splunkd log_level=*
          | eval Result=if(log_level="INFO","Success","Failed")
          | stats count by Result
          </query>
          <earliest>-1d@h</earliest>
          <latest>now</latest>
        </search>
        <option name="height">150</option>
        <option name="rangeColors">["0x53a051","0x0877a6","0xf8be34","0xf1813f","0xdc4e41"]</option>
        <option name="trellis.enabled">1</option>
        <option name="trellis.size">medium</option>
        <option name="useThousandSeparators">0</option>
      </single>
    </panel>
  </row>
</dashboard>

PS: CSS Override also has been used in the example to align the Single Value visualization Title to Center.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

Richfez
SplunkTrust
SplunkTrust

What do you mean by "taking more time?" How much more time are we talking about?

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.