Splunk Search

Help! I cannot configure the proper search XML for a search feeding a table

Engager

This is my current idea of how a table with latency data should fit together, I am trying to mimic the "Real-Time measured indexing rate and latency" table at the top of the Indexing tab of Splunk On Splunk (SOS). Please let me know your suggestions... asterisks have been removed for security..

<dashboard>
<label>Index Performace</label>
<row>
<table>
<searchString>index=_internal host="*.*.net" | eval latency=round((_indextime - _time),2) | eval seconds_elapsed=(time() - now()) | eval secs=if(seconds_elapsed<0,"1",seconds_elapsed) | eval esize=((len(_raw)/1024)) | eventstats max(secs) AS seconds | eventstats count AS ecount, sum(esize) as sum_esize $type$ | stats last(ecount) AS "event count", last(eval(ecount/seconds)) as eps, last(eval(sum_esize/seconds)) as KBps, min(latency) as "minimum latency (seconds)", avg(latency) as avglat, max(latency) AS "maximum latency (seconds)" first(_time) AS oldestTime last(_time) AS newestTime $type$ | eval avglat=round(avglat,2)| eval eps=round(eps,2) | eval KBps=round(KBps,2) | convert timeformat="%m/%d/%Y %H:%M:%S" ctime(newestTime) | convert timeformat="%m/%d/%Y %H:%M:%S" ctime(oldestTime) | rename newestTime AS "Time stamp of newest event" oldestTime AS "Time stamp of oldest event" avglat AS "average latency (seconds)" eps AS "events per second" KBps AS "indexing rate (KBps)"</searchString>

<title>Indexing Perf new</title>
<earliestTime>-24h@h</earliestTime>
<latestTime>now</latestTime>
</table>
</row>
</dashboard>

Splunk Employee
Splunk Employee

There are two problems in your <searchString> parameter:

  • The "lesser than" sign in the | eval secs=if(seconds_elapsed<0,"1",seconds_elapsed) expression is interpreted as the beginning of an XML tag, which causes the view to break. It needs to be replaced by its HTML-safe encoding of "&lt;".
  • You need to replace the $type$ token references by a valid split-by clause. Typically, this would be a metadata field such as source/sourcetype/host/index, but you can use any field that makes sense to you.

In the example below, I have corrected both problems - $type$ has been replaced by the by sourcetype clause:


index=_internal host="*"
| eval latency=round((_indextime - _time),2)
| eval seconds_elapsed=(time() - now())
| eval secs=if(seconds_elapsed&lt;0,"1",seconds_elapsed)
| eval esize=((len(_raw)/1024))
| eventstats max(secs) AS seconds
| eventstats count AS ecount, sum(esize) as sum_esize by sourcetype
| stats last(ecount) AS "event count", last(eval(ecount/seconds)) as eps, last(eval(sum_esize/seconds)) as KBps, min(latency) as "minimum latency (seconds)", avg(latency) as avglat, max(latency) AS "maximum latency (seconds)" first(_time) AS oldestTime last(_time) AS newestTime by sourcetype
| eval avglat=round(avglat,2)
| eval eps=round(eps,2)
| eval KBps=round(KBps,2)
| convert timeformat="%m/%d/%Y %H:%M:%S" ctime(newestTime)
| convert timeformat="%m/%d/%Y %H:%M:%S" ctime(oldestTime)
| rename newestTime AS "Time stamp of newest event" oldestTime AS "Time stamp of oldest event" avglat AS "average latency (seconds)" eps AS "events per second" KBps AS "indexing rate (KBps)"

Don’t Miss Global Splunk
User Groups Week!

Free LIVE events worldwide 2/8-2/12
Connect, learn, and collect rad prizes
and swag!