Getting Data In

Forwarder capacity?

msarro
Builder

I noticed that in the capacity planning guide, there is no mention of the capacity of a forwarder. Right now I am looking at sending a significant amount of data to two different forwarders. How much data can the forwarder handle? These are heavy forwarders, I know the guideline for an indexer is 100GB/day, but I can't find anything similar for forwarders.

Tags (1)
0 Karma

kristian_kolb
Ultra Champion

First: How much is a significant amount?

I think that a lot of this depends on how you set up the forwarding. If you monitor a directory containing thousands of files, with new files being added constantly you may run into problems just because the forwarder will have to keep track of so many files. I've seen forwarders (UF on windows) going up to 35-40% CPU usage for this reason alone (the actual log amount was less than a 100MB daily).

If you have a relatively 'clean' source of logs, i.e. just a few files you could probably send out quite a large amount. The UF is capped at 256KBps, although this can be changed, so in theory this means that a single forwarder can send 21GB/day by default.

hope this helps,

Kristian

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...