Getting Data In

Data flood protection on forwarder or indexers

cwacha
Path Finder

Our splunk universal forwarders may be configured by our customers in terms of logs they want to collect and transfer. An application that is logging very exessively could therefore drastically increase the amount of log data to be indexed.

Is there some sort of flood protection available (on indexer or heavy forwarders) to detect universal forwarders that are transferring lots of data? Actions could be blacklisting the agent for a limited time period (and redirecting its log data to the nullqueue) or generating an alert or logging an error message.

Tags (3)

gkanapathy
Splunk Employee
Splunk Employee

There currently is not, though it would be a very nice enhancement request. What you could do (relatively easily) is set yourself up a realtime alert to alert you if any forwarder send more than some amount of data in the past (say) 5 minutes. You would base this on the Splunk internal metrics log. If you look at the Splunk Deployment Monitor app, you can see examples of searches that do this on a dashboard, and you should be able to create alerts using those as a guide.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Where Innovation Takes Flight: The Splunk4Aviation Flight Sim Lands at .conf26

If you hear someone at .conf26 shouting "gear down, GEAR DOWN" across the show floor, you have found us.  The ...

Turn Cisco Telemetry Into Action with Cisco Data Fabric, powered by the Splunk ...

The surge in machine data is already hitting enterprise budgets, and the agentic era will only intensify it. ...

Persistent Queue at TcpOut — One of Splunk's Most Practical Features

Splunk introduced persistent queueing at the tcpout layer as one of the most practical resilience features in ...