We have a setup where the logs are generated continuously and are being forwarded into Splunk indexers and also into another external application.
Earlier, the application was directly reading from the server with minimal delay.
After we introduced Splunk, we are observing delay of about 13 to 19 secs.
The maximum approved delay factor is 5 secs.
How do I find out where is the delay being induced?
I have _time which is the event occurrence time, _indextime which gives indexed time. Using Splunk App for Stream, I am able to get timestamp factor too.
However, I am struggling to get the logic of where the delay is.
Basically, if I can get the time of arrival of log into Splunk, I can calculate the difference between index time and arrival time.
How do I get the arrival time into Splunk?
Any ideas? Any one faced such situation?