How to generate multiple charts from one search result faster?

[hcheang]

Hello,

I'm wondering if there is any faster or more efficient way (either using Pivot or some unique query) to generate multiple charts.

The base query is really simple:

|stats count by src dst port

and on dashboard, I made three panels:

|stats count by src|sort -count|head 10
|stats count by dst|sort -count|head 10
|stats count by port|sort -count|head 10
which takes about 20 minutes to load completely.

I've tried Pivot by putting above query to the Data Model hoping it will load faster but it seems it's loading one by one which is not much different from loading three different searches as shown above.

Is this the only way?

[Raghav2384]

PostProcessSearch should help you    here. Example
       Example XML: 

    1. 

   > <search id="parent> <query>index=x
   >     sourcetype=y earliest=blah
   >     latest=blah|more search | stats
   >     count by src,dst,port,_time</query>
   >     </search>
   >     
   >     /**processed searches **/ <chart1>
   >     <search base="parent"> <query>stats
   >     count by src</query> <search>
   >     </chart1> <chart2> <search
   >     base="parent"> <query>stats count by
   >     dst</query> <search> </chart2>
   >     <chart3> <search base="parent">
   >     <query>stats count by port</query>
   >     <search> </chart3>
       More information on Post process searches :    http://docs.splunk.com/Documentation/Splunk/6.2.1/AdvancedDev/PostProcess
       Hope this help!    Thanks,    Raghav

[hcheang]

Thanks! but can you look at my code and tell me what's wrong with it? I've looked into the document but there wasn't enough example for me to understand your code. What I've tried is like this

<form>
  <label>Testing</label>
  <description/>
  <fieldset submitButton="false">
    <input type="time" token="field1">
      <label/>
      <default>
        <earliestTime>0</earliestTime>
        <latestTime/>
      </default>
    </input>
  </fieldset>
  <row>
    <panel>
        <search id="parent"> 
            <query>index=* |stats count by src dst port </query>
            <earliestTime>$field1.earliest$</earliestTime>
            <latestTime>$field1.latest$</latestTime>
        </search>          
        <chart1>
            <search base="parent"> 
            <query>stats count by src</query> 
            </search>
        </chart1> 
        <chart2> 
            <search base="parent"> 
            <query>stats count by dst</query>
            </search> 
        </chart2>
        <chart3> 
            <search base="parent">
            <query>stats count by port</query>
            </search> 
            </chart3>
    </panel>
  </row>
</form>

[richielynch89]

This works great. Thank you!

[Raghav2384]

IMO parent search shouldn't be in any panel. Try this

  <form>
       <label>Testing</label>

       <fieldset submitButton="false">
          <input type="dropdown" token="earliestTime" searchWhenChanged="true">
                   <label>Select a TimeWindow</label>
                   <choice value="-24h@h">24 Hour Window</choice>
                   <choice value="-48h@h">48 Hour Window</choice>
                   <choice value="-72h@h">72 Hour Window</choice>
                   <choice value="-7d@d">7 Day Window</choice>
                   <default>24 Hour Window</default>
          </input>
       </fieldset>
    <search id="parent"> 
                 <query>index=* |stats count by src,dst,port,_time </query>
                 <earliest>$earliestTime$</earliest>
                 <latest>now</latest>
    </search>    
       <row>
         <panel>

             <chart>
                 <title>FirstChart</title>
                 <search base="parent"> 
                 <query>stats count by src</query> 
                 </search>
             </chart> 
        </panel>
        <panel>
             <chart> 
                 <title>SecondChart</title>
                 <search base="parent"> 
                 <query>stats count by dst</query>
                 </search> 
             </chart>
        </panel>
        <panel>
             <chart> 
                  <title>ThirdChart</title>
                   <search base="parent">
                 <query>stats count by port</query>
                 </search> 
                 </chart>
         </panel>
       </row>
     </form>

I have added custom time ranges 24h,48h,72h and 7D instead of timepicker. Try thiss and let me know if it works.

Also, index=|stats count by src,dst,port is your actual search? If i use index= on our instance, it would die in few minutes 🙂

Thanks,
Raghav

[hcheang]

I get "No search query provided." messages for all three charts.. and yes I use index=some firewall and specify host= something but didn't included in the example sorry XD. I'll try to find fix this but really you've been a great help with this question and the previous one. Thanks!

[Raghav2384]

Couple you please post your xml? please grey out the index and host info. I have used the same xml and get the resuts as i wanted. Interesting.

[hcheang]

Sorry for late reply. Am I doing something wrong? The way I'm creating the dashboard is Dashboards>Create New Dashboard>Edit Source

<form>
        <label>Testing</label>

        <fieldset submitButton="false">
           <input type="dropdown" token="earliestTime" searchWhenChanged="true">
                    <label>Select a TimeWindow</label>
                    <choice value="-24h@h">24 Hour Window</choice>
                    <choice value="-48h@h">48 Hour Window</choice>
                    <choice value="-72h@h">72 Hour Window</choice>
                    <choice value="-7d@d">7 Day Window</choice>
                    <default>24 Hour Window</default>
           </input>
        </fieldset>
     <search id="parent"> 
                  <query>index=someindex action=DENY |stats count by src dst port_dest _time </query>
                  <earliest>$earliestTime$</earliest>
                  <latest>now</latest>
     </search>    
        <row>
          <panel>

              <chart>
                  <title>FirstChart</title>
                  <search base="parent"> 
                  <query>|stats count by src</query> 
                  </search>
              </chart> 
              <chart> 
                  <title>SecondChart</title>
                  <search base="parent"> 
                  <query>|stats count by dst</query>
                  </search> 
              </chart>
              <chart> 
                   <title>ThirdChart</title>
                    <search base="parent">
                  <query>|stats count by port_dest</query>
                  </search> 
                  </chart>
          </panel>
        </row>
      </form>

[Raghav2384]

Add ',' while using stats
stats count by src,dst,port_dest,_time

Also, tricky part here is, if any of the values src, dst,port_dest,_time doesn't exit in your base search results, it's not going to work.
Try running individual searches like index=* earliest=-1h|stats count by src and see if it returns anything. Similarily do it for stats count by dst  and stats count by port_dest.

If one the fields doesn't return anything, exclude that from your parent search and try.

Hope this helps!
Thanks,
Raghav

[Raghav2384]

i meany index=*