How to force Splunk to add additional (local) timestamp to events?

[k1gto]

I'd like to have Splunk add an additional (current) timestamp field to the events that I'm sending so that I can compare what my app thinks the time is, versus what the Splunk server says it is. I understand indexing can take some time and would naturally shift the time slightly, and that is ok. The sort of clock skew (between app server and indexing server) that I'm trying to expose is on the order of years.

My app is currently sending the time in the timestamp field of the events (JSON). Ideally I'd like to have Splunk add a field "timestamp_splunk" or something like that. Alternatively I'm happy to have Splunk set the timestamp field and have my app send the time in a different field.

[richgalloway · Accepted solution]

All events have at least two timestamps, _time and _indextime. _time is usually set from a field in the event whereas _indextime is always set by Splunk to be the time the event was written to disk (roughly). If you are already setting _time from your data then _indextime may be what you are looking for. Note that you can't use _indextime directly in SPL commands - you must first assign it to another field ('eval it=_indextime', for example).