topic Re: Indexing Latency Chart in Getting Data In

Indexing Latency Chart

rotten — Fri, 24 Sep 2010 23:52:46 GMT

It would be both useful and interesting to be able to graph the indexing latency for various data sources or hosts over time.

Is there a way to compare "insert time" (for the splunk database) with "event time" (from the source logfile) and build such a set of charts?

Re: Indexing Latency Chart

Stephen_Sorkin — Sat, 25 Sep 2010 01:44:26 GMT

Yes, the "insert time" is the _indextime field and the "event time" is the _time field.

You can search:

... | eval lag = _indextime - _time | timechart median(lag) by ...

Re: Indexing Latency Chart

Lowell — Mon, 27 Sep 2010 23:27:32 GMT

Version note: Keep in mind that the _indextime field was added in version 4.0. With earlier versions you cannot track indexing latency like this.

Re: Indexing Latency Chart

droth333 — Thu, 01 Dec 2011 21:20:32 GMT

This is fascinating stuff, but there's a datum missing that would
help me know everything for an indexing latency problem I have:
_time is timestamp on the event
_indextime is time it was indexed at the indexer.
need _arrivaltime, the time it arrive at the indexer.
Is there such a value?

In general where can I find these hidden groovy _* variables listed?
Thanks,
Dave