Log latency

[VijaySrrie]

Hi Team,

We could see latency in logs

Log ingestion via - syslog

Network devices --> Syslog server --> splunk 

Using below query, we could see minimum 10 mins to maxminum 60 mins log latency

index="ABC" sourcetype="syslog" source="/syslog*" 
| eval indextime=strftime(_indextime,"%c")
| table _raw _time indextime

What should be our next steps to check where the latency is and how to fix it?

[VijaySrrie · Accepted solution]

@KendallW 
INFO ThruputProcessor [2963 parsing] - Current data throughput (5125 kb/s) has reached maxKBps. As a result, data forwarding may be throttled. Consider increasing the value of maxKBps in limits.conf.

We will try increasing the limits.

[KendallW]

Hi @VijaySrrie assuming you are collecting the logs on syslog server then forwarding to Splunk with a UF?
You can check if the UF is reaching its thruput limit which could cause indexing lag:

index=_internal sourcetype=splunkd component=ThruputProcessor "has reached maxKBps"