Running one of two searches based on time picker selection

[adamblock2]

I am trying to create a dashboard panel which will run one of the following email searches. There are a number of inputs which allow a user to filter exactly what he/she wants to search for.

• One input allows a user to select the search criteria (sender, recipient, source IP, message id, etc.)
• Another input allow the user to input the data being searched for.
• The last input is a time picker.

Each input is a separate token. So, if one wants to search for sender=john.doe@xyz.org, for example, those values (sender, john.doe@xyz.org) would each be passed to the search with tokens.

If the time selected from the time picker is within the last 24h, a search based on raw events (including index=, eventtype=, stats, etc.) should be run. If the time selected is historic (ie. more than 24h ago), I want to run a search based on a summary index (index=summary report=x).

I have been working to figure this out, but each attempt has been unsuccessful. Assistance with this will greatly be appreciated.

Thank you.

[niketn · Accepted solution]

[Update]

Reattaching image as URL, since original image got removed.

---

@adamblock2, questions on similar lines have definitely been answered before. To document the answer I am providing a generic run anywhere example, which should also fit in your use case.

Time Picker does not always provide epoch time values via default earliest and latest tokens (it may have "snap to" time format like "-1h@h" etc. Hence the same can not be used for time based calculations in the dashboard. In order to tackle this situations you have following two options:

PS: For line breaks in the code <br\> needs to be replaced as <br\>, since the same is getting escaped here on Splunk Answers while posting the code.

Option 1- Use search event handler to capture Earliest and Latest String time through $job.earliestTime$ and $job.latestTime$

---

Step 1) Run a dummy search (only use | makeresults to ensure actual index search is not performed) with the time tokens (assuming time picker field name is tokTime) as $tokTime.earliest$ and $tokTime.latest$.

Step 2) Code the Search Event Handler (in the example I have used <done>, <progress> can also be used. These Search Event Handlers can access default search time tokens i.e. $job.earliestTime$ and $job.latestTime$. But in String time format not epoch.

Step 3) Use <eval> tag to convert from String Time to Epoch time using strptime() function and set the tokens. tokEarliestTime1 and tokLatestTime1.

Step 4) Set the tokCurrentTime1 as now() and using eval tag calculate the duration1 token.

Step 5) Use eval tag to set the token for index tokIndex1 either as YourSummaryIndexName or YourRealTimeIndexName based on duration between tokCurrentTime1 and tokEarliestTime1 being greater than 86400 or not.

Option 2- Use addinfo SPL command to generate info_min_time and info_max_time in a dummy search

---

Step 1) Run a dummy search (only use | makeresults | addinfo to ensure actual index search is not performed and search job time tokens info_min_time and info_max_time are generated) with the time tokens (assuming time picker field name is tokTime) as $tokTime.earliest$ and $tokTime.latest$.

Step 2) Code the Search Event Handler (in the example I have used <done>, <progress> can also be used. These Search Event Handlers can access first row of results using $result.<fieldname>$.

Step 3) Use $result.info_min_time$ and $result.info_max_time$ to set Earliest and Latest Search Epoch Time tokens i.e. tokEarliestTime2 and tokLatestTime2

Step 4) Set the tokCurrentTime2 as now() and using eval tag calculate the duration2 token.

Step 5) Use eval tag to set the token for index tokIndex2 either as YourSummaryIndexName or YourRealTimeIndexName based on duration between tokCurrentTime2 and tokEarliestTime2 being greater than 86400 or not.

PS: For this use case we only require $job.earliestTime$ which is used compare with the current time now() and calculate the time difference required to set the index to either summary or real-time based on the duration >24*60*60 i.e. 86400 or not.

Following is the run anywhere dashboard source code for above options. Please try out and confirm:

<form>
  <label>Splunk Answers 578984 - Timepicker Choose Index based on time</label>
  <search>
    <query>| makeresults
    </query>
    <earliest>$tokTime.earliest$</earliest>
    <latest>$tokTime.latest$</latest>
    <done>
      <eval token="tokEarliestTimeString1">strftime(strptime($job.earliestTime$,"%Y/%m/%d %H:%M:%S %p"),"%Y/%m/%d %H:%M:%S %p")</eval>
      <eval token="tokLatestTimeString1">strftime(strptime($job.latestTime$,"%Y/%m/%d %H:%M:%S %p"),"%Y/%m/%d %H:%M:%S %p")</eval>
      <eval token="tokCurrentTimeString1">strftime(now(),"%Y/%m/%d %H:%M:%S %p")</eval>
      <eval token="tokEarliestTime1">strptime($job.earliestTime$,"%Y/%m/%d %H:%M:%S %p")</eval>
      <eval token="tokLatestTime1">strptime($job.latestTime$,"%Y/%m/%d %H:%M:%S %p")</eval>
      <eval token="tokCurrentTime1">now()</eval>
      <eval token="duration1">now()-tokEarliestTime1</eval>      
      <eval token="tokIndex1">if((now()-tokEarliestTime1) &lt;=86400,"yourRealTimeIndex","yourSummaryIndex")</eval>
    </done>
  </search>
  <search>
    <query>| makeresults
| addinfo
    </query>
    <earliest>$tokTime.earliest$</earliest>
    <latest>$tokTime.latest$</latest>
    <done>
      <eval token="tokEarliestTimeString2">strftime($result.info_min_time$,"%Y/%m/%d %H:%M:%S %p")</eval>
      <eval token="tokLatestTimeString2">strftime($result.info_max_time$,"%Y/%m/%d %H:%M:%S %p")</eval>
      <eval token="tokCurrentTimeString2">strftime(now(),"%Y/%m/%d %H:%M:%S %p")</eval>
      <eval token="tokEarliestTime2">$result.info_min_time$</eval>
      <eval token="tokLatestTime2">$result.info_max_time$</eval>
      <eval token="tokCurrentTime2">now()</eval>
      <eval token="duration2">now()-tokEarliestTime2</eval>      
      <eval token="tokIndex2">if((now()-tokEarliestTime2) &lt;=86400,"yourRealTimeIndex","yourSummaryIndex")</eval>
    </done>
  </search>  
  <fieldset submitButton="true">
    <input type="dropdown" token="tokCriteria">
      <label>Search Criteria</label>
      <choice value="sender">Sender</choice>
      <choice value="receiver">Receipient</choice>
      <choice value="source_ip">Source IP</choice>
      <choice value="message_id">Message ID</choice>
      <default>sender</default>
    </input>
    <input type="text" token="tokSearch">
      <label>Search Details</label>
      <default>*</default>
    </input>
    <input type="time" token="tokTime">
      <label>Select Time</label>
      <default>
        <earliest>-24h@h</earliest>
        <latest>now</latest>
      </default>
    </input>
  </fieldset>
  <row>
    <panel>
      <title>Option 1 : job.earliestTime and job.latestTime tokens</title>
      <html>
        <div>
          <code> 
            tokLatestTimeString1: $tokLatestTimeString1$ &lt;br/&gt; 
            tokEarliestTimeString1: $tokEarliestTimeString1$ &lt;br/&gt;
            tokCurrentTimeString1: $tokCurrentTimeString1$ &lt;br/&gt;
            tokLatestTime1: $tokLatestTime1$ &lt;br/&gt; 
            tokEarliestTime1: $tokEarliestTime1$ &lt;br/&gt;
            tokCurrentTime1: $tokCurrentTime1$ &lt;br/&gt;
            duration1 (in secs): $duration1$ &lt;br/&gt;
            tokIndex1: $tokIndex1$
          </code>
        </div>
      </html>
    </panel>
    <panel>
      <title>Option 2 : addinfo command</title>
      <html>
        <div>
         <code>
           tokLatestTimeString2: $tokLatestTimeString2$ &lt;br/&gt; 
           tokEarliestTimeString2: $tokEarliestTimeString2$ &lt;br/&gt;
           tokCurrentTimeString2: $tokCurrentTimeString2$ &lt;br/&gt;
           tokLatestTime2: $tokLatestTime2$ &lt;br/&gt; 
           tokEarliestTime2: $tokEarliestTime2$ &lt;br/&gt;
           tokCurrentTime2: $tokCurrentTime2$ &lt;br/&gt;
           duration2 (in secs): $duration2$ &lt;br/&gt;
           tokIndex2: $tokIndex2$
         </code>
        </div>
      </html>
    </panel>    
  </row>
</form>

[adamblock2]

I apologize for the delayed response, but I have been out of the office for a number of days. I re-tried the example above, and it worked without issue.

My next question is the actual search that I am submitting. Should I be replacing "yourRealTimeIndex","yourSummaryIndex" with the actual searches? If yes, I am concerned that I have done something wrong, as the form does not appear to submit and the query doesn't run.

<eval token="tokIndex2">if((now()-tokEarliestTime2) &lt;=86400,"index=mail eventtype=pps_filter | stats max(_time) AS _time, values(delivery_status) AS delivery_status, values(subject) AS subject, values(from) AS from, values(rcpt) AS to, values(sender_domain) AS sender_domain, values(hops_ip) AS infr_ip, values(Country) AS country, values(file_name) AS file_name, values(message_id) AS message_id by internal_message_id, host | fields - _raw | fields + _time, delivery_status, subject, from, to, sender_domain, infr_ip, country, file_name, message_id, internal_message_id, host| mvexpand file_name | search $i_criteria$=$s_value|s$ AND delivery_status=$d_status$| iplocation infr_ip | rename Country AS country | table _time, delivery_status, subject, from, to, sender_domain, infr_ip, country, file_name, message_id, internal_message_id | sort _time","index=summary report=proofpoint_daily_summary | mvexpand file_name | search $i_criteria$=$s_value|s$ AND delivery_status=$d_status$| iplocation infr_ip | rename Country AS country| eval eventtime=strftime(eventtime, "%x, %X") | table eventtime, delivery_status, subject, from, to, sender_domain, infr_ip, country, file_name, message_id, internal_message_id | sort eventtime") </eval>
.
.
.
.
<row>
    <panel>
      <title>Search Messages</title>
      <table>
        <search>
          <query>$tokIndex2$</query>
          <earliest>$i_time.earliest$</earliest>
          <latest>$i_time.latest$</latest>
          <sampleRatio>1</sampleRatio>
        </search>
        <option name="count">10</option>
        <option name="dataOverlayMode">none</option>
        <option name="drilldown">cell</option>
        <option name="percentagesRow">false</option>
        <option name="rowNumbers">false</option>
        <option name="totalsRow">false</option>
        <option name="wrap">true</option>
    </table>
    </panel>
  </row>

[adamblock2]

The token values in the search are set in dropdown inputs which are submitted.

[DalJeanis]

@niketnilay - Awesome answer. Change < to < and > to > when used as a literal or comparison operator in the XML. (in <= on lines 16 and 33)

@adamblock2 - FYI. The system is thinking that <= is the start of an element tag <SomeWeirdName ThatStartsWith "=" AndGoesOn NearlyForever>

[niketn]

Thanks @DalJeanis. As always 🙂 I had HTML escaped characters in my answer. I think it got converted on Splunk Answers here before converting to code. I have corrected the same.

[DalJeanis]

@niketnilay - the interface is being opinionated. In the <code> sections, it has changed your <br\> to <br\> and won't let it be edited back.

[niketn]

Yes... true... I have seen that before as well. Could not help it. Maybe mentioning in the answer itself is better.

[inventsekar]

@niketnilay, great answer, it deserves upvotes !

[niketn]

Thanks @inventsekar. As stated similar question like these have been asked several times before. So I wanted to document this 🙂

If will be fruitful effort if @adamblock2's problem is solved 🙂

[adamblock2]

I just attempted to copy your code for the first option so to create a test dashboard. When I went to save the dashboard, I received the error "Encountered the following error while trying to save: Error parsing XML on line 16: StartTag: invalid element name"

I believe line 16 corresponds to
if((now()-tokEarliestTime1)<=86400,"yourRealTimeIndex","yourSummaryIndex")

Thoughts? Thank you.

[niketn]

Please use &lt;=86400. While pasting as code here on Splunk Answers it got converted to <=

I have updated in the answer. Please try out and confirm.

[inventsekar]

maybe check this one -
https://answers.splunk.com/answers/488877/how-to-run-two-different-searches-in-a-dashboard-b.html

This is just an idea so far: try to add <condition> and <eval> children under <change> child of the <input type="time"...> tag. Analyze the values and set or unset the tokens according to your needs. Then create a few panels at most one of which will be visible using depends= and rejects= attributes in those panels' tags.

You'll have to read "Dashboards and Visualizations" manual to do everything right. Unfortunately, the documentation does not include ready examples of such complex behavior, but that's the beauty of Splunk development.

Another, a somewhat different idea, is to add id= attribute to your time input and create a JavaScript which will find the time picker by that id and hook a handler to the .change event. In that handler, you can set and unset some tokens to achieve the same result (your panels should still have depends= and rejects= attributes for this to work), but you'll have a much better control of the algorithm.