Estimating number of Heavy Forwarders based on event volume?

[bandit]

We know that following recommendations that the rule of thumb for indexers is one indexer per 100GB indexed per 24 hours when not performing replication.

Is there a rule of thumb for Heavy Forwarders when performing filtering/routing/time zone functions, however not indexing locally?

i.e. UF (monitor) --> HF (filtering/routing/time zones) --> IDX (indexing)

Could I estimate that I would need a new Heavy Forwarder per 200GB per 24 hours?

http://docs.splunk.com/Documentation/Splunk/6.1.3/Deploy/Summaryofperformancerecommendations

[gkanapathy]

The rule of thumb is actually now 1 indexer per 250 GB/day. Not per 100 GB/day. You can probably estimate a HF as being able to max between 8 and 20 MB/sec, depending on the specifics of the server, the data, and any processing you might be doing.

[bandit]

Is this out of date?
http://docs.splunk.com/Documentation/Splunk/6.1.3/Deploy/Summaryofperformancerecommendations

[gkanapathy]

Yeah, it was, though it was half-updated.

[ltrand · Accepted solution]

This is an awesome question, but I think it's more a function of configuration and environment than anything else. Best thing I can suggest is watch your performance counters (CPU, RAM, I/O, Splunk Queues, Network) for your box and gauge growth from there.

We currently have HF doing some preprocessing for sourcetyping on syslog, but it is largely a bulk forwarder for UF & syslog. We are logging significantly more than 200GB per day per HF and don't seem to have issues (our issues are generally self created or particular to our environment).

[bandit]

Thanks for sharing this helpful info. It's always good if we can avoid going back to management and saying "we underestimated the volume it could handle."