Dashboards & Visualizations

How does Splunk forward events?

nicholashouston
Engager

How does Splunk handle events when forwarded? Does it send them one event at a time or in batches?

I.E.: I am using a heavy forwarder to send a copy of my data off site through a router that collects NetFlow information. My concern is will the recursively created NetFlow records increase uncontrollably if it is monitoring its own output?

Netflow is set to capture session data and then forward it to Splunk.

0 Karma
1 Solution

coccyx
Path Finder

Splunk by default will stick on a given indexer for 30 seconds from a heavy forwarder before connecting to another. This can lead to inconsistent load balancing but there are things you can do to tune it better.

Your concern about Netflow into Splunk also creating a lot of connections via Splunk isn't a valid concern. Splunk sends megabytes or gigabytes of data per TCP connection before establishing another connection. There may be thousands or millions of Netflow records in that stream. Splunk should not meaningfully add to your Netflow record count from HWF to Indexer. On the front side of the HWF, depending on the number of Universal Forwarders though, you could see a significant number of connections. Enough to be a meaningful percentage of your Netflow data.

View solution in original post

coccyx
Path Finder

Splunk by default will stick on a given indexer for 30 seconds from a heavy forwarder before connecting to another. This can lead to inconsistent load balancing but there are things you can do to tune it better.

Your concern about Netflow into Splunk also creating a lot of connections via Splunk isn't a valid concern. Splunk sends megabytes or gigabytes of data per TCP connection before establishing another connection. There may be thousands or millions of Netflow records in that stream. Splunk should not meaningfully add to your Netflow record count from HWF to Indexer. On the front side of the HWF, depending on the number of Universal Forwarders though, you could see a significant number of connections. Enough to be a meaningful percentage of your Netflow data.

richgalloway
SplunkTrust
SplunkTrust

Heavy forwarders forward data in batches. I forget the size of the batch ATM.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...