topic Log latency in Security

Log latency

VijaySrrie — Fri, 23 Aug 2024 11:00:13 GMT

Hi Team,

We could see latency in logs

Log ingestion via - syslog

Network devices --> Syslog server --> splunk

Using below query, we could see minimum 10 mins to maxminum 60 mins log latency

index="ABC" sourcetype="syslog" source="/syslog*" | eval indextime=strftime(_indextime,"%c") | table _raw _time indextime

What should be our next steps to check where the latency is and how to fix it?

Re: Log latency

KendallW — Mon, 26 Aug 2024 05:35:26 GMT

Hi @VijaySrrie assuming you are collecting the logs on syslog server then forwarding to Splunk with a UF?
You can check if the UF is reaching its thruput limit which could cause indexing lag:

index=_internal sourcetype=splunkd component=ThruputProcessor "has reached maxKBps"

Re: Log latency

VijaySrrie — Wed, 28 Aug 2024 08:53:24 GMT

@KendallW
INFO ThruputProcessor [2963 parsing] - Current data throughput (5125 kb/s) has reached maxKBps. As a result, data forwarding may be throttled. Consider increasing the value of maxKBps in limits.conf.

We will try increasing the limits.