Which is better; HEC on heavyforwarder or clustered indexers for Kafka?

[splunkreal]

Hello,

we plan to try Kafka as data collector and we'd like to know if we should keep our HF to receive HEC inputs for Kafka data or send directly to indexers, about 200-300gb per day?

Looks like HF is better for filtering before indexing.

Thanks.

[richgalloway · Accepted solution]

An HF is just an indexer that doesn't store data so it's no better or worse for filtering than an indexer.

A single HF is a single point of failure for your Kafka data.  It also can lead to poor data distribution which will negatively affect search performance.  The solution, of course, is to have multiple HFs, but if you're going to round-robin among HFs, why not just round-robin among indexers?