Getting Data In

How can I identity forwarder data rate and index data rate (to identify a lag and prioritize logs)?

chintan_shah
Path Finder

Hi,
Is there any way where we can identify how much data the forwarder is sending and how much data is being indexed in real-time?
The problem is that I have a single forwarder that is sending data to a single indexer and its sending multiple logs i.e. 50 monitored files with different indexes. I am receiving data from a few indexes in real time whereas for some indexes I am having a lag, so I want to remove the lag and if possible give higher preferences to some logs file.

0 Karma

yannK
Splunk Employee
Splunk Employee

For forwarder lag, start to look at the metrics.log on the forwarder, if you see that it is hitting a plateau of kbps speed, it may be that you are hitting the default thuput limit.
see this article
http://docs.splunk.com/Documentation/Splunk/7.0.0/Troubleshooting/Troubleshootingeventsindexingdelay...

Also look at the timestamp, maybe is it a timezone issue.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

A Four-Part Event Series: Full Stack Observability For the AI Era

As AI reshapes applications, infrastructure, and the way teams operate, the traditional boundaries of ...

SOC4Kafka - New Kafka Connector Powered by OpenTelemetry

The new SOC4Kafka connector, built on OpenTelemetry, enables the collection of Kafka messages and forwards ...

Event Series: Level up your SOC: Advancing with Splunk Enterprise Security

AI has fundamentally raised the stakes for security operations, and this three-part series is your guide to ...