Latencies are not stored anywhere. Your approach is correct, however you should be running this report in realtime using the all time (real time) dropdown option. This is to ensure that all incoming events are shown regardless of their extracted time stamp. Note that the latency measured here is affected by how old your events are when you expose them to Splunk. Alternatively, the SOS app includes a distributed indexing performance view that will show you realtime latency and is powered by this search
index=* OR index=_internal
| eval latency=round((_indextime - _time),2)
| eval seconds_elapsed=(time() - now())
| eval secs=if(seconds_elapsed<0,"1",seconds_elapsed)
| eval esize=((len(_raw)/1024))
| eventstats max(secs) AS seconds
| eventstats count AS ecount, sum(esize) AS sum_esize $type$
| stats last(ecount) AS "event count"
last(eval(ecount/seconds)) AS eps
last(eval(sum_esize/seconds)) AS KBps
min(latency) AS "minimum latency (seconds)"
avg(latency) AS avglat
max(latency) AS "maximum latency (seconds)"
min(_time) AS oldestTime
max(_time) AS newestTime $type$
| eval avglat=round(avglat,2)
| eval eps=round(eps,2)
| eval KBps=round(KBps,2)
| convert timeformat="%m/%d/%Y %H:%M:%S" ctime(newestTime)
| convert timeformat="%m/%d/%Y %H:%M:%S" ctime(oldestTime)
| rename newestTime AS "Time stamp of newest event"
oldestTime AS "Time stamp of oldest event"
avglat AS "average latency (seconds)"
eps AS "events per second"
KBps AS "indexing rate (KBps)"
Possible values for $type$: by index | by host | by source | by sourcetype | by splunk_server
You can modify this search to suit your needs.
... View more