Getting Data In

Role of Heavy Forwarder between UF and Indexer

VijaySrrie
Builder

Hi,

Indexer can do Parsing and Indexing then why do we use HF between UF and Indexer?

Labels (1)
Tags (1)
0 Karma
1 Solution

PickleRick
SplunkTrust
SplunkTrust

When you introduce a HF layer in your environment, you offload all the ingestion-phase work, except the actual indexing onto HF's. So your indexers don't have to bother with listening for HEC events, doing time extraction, indexed field extractions, ingest-time evals.This leaves your indexers for the sole purpose of indexing and searching.

Additionally, there are apps which do not run on UF's but need indexer or HF. These are typically apps running some scripted/modular inputs. Running them on indexers would add unnecessary asymmetrical load if you ran them on a single indexer in a cluster.

So as your environment grows introducing a HF layer has its pros.

EDIT: Oh, and one more thing. Two actually.

If you have an intermediate load-ballancing forwarder layer you don't have to update your outputs on UF's when you grow "sideways" adding new indexers. It's annoying especially if you don't manage UF's with deployment server. You're more likely to need another indexer or two than new HF's.

And if if you offload most (if not all) ingest-time apps to HF, you don't have to restart your indexers if you have to change something in them.

View solution in original post

PickleRick
SplunkTrust
SplunkTrust

When you introduce a HF layer in your environment, you offload all the ingestion-phase work, except the actual indexing onto HF's. So your indexers don't have to bother with listening for HEC events, doing time extraction, indexed field extractions, ingest-time evals.This leaves your indexers for the sole purpose of indexing and searching.

Additionally, there are apps which do not run on UF's but need indexer or HF. These are typically apps running some scripted/modular inputs. Running them on indexers would add unnecessary asymmetrical load if you ran them on a single indexer in a cluster.

So as your environment grows introducing a HF layer has its pros.

EDIT: Oh, and one more thing. Two actually.

If you have an intermediate load-ballancing forwarder layer you don't have to update your outputs on UF's when you grow "sideways" adding new indexers. It's annoying especially if you don't manage UF's with deployment server. You're more likely to need another indexer or two than new HF's.

And if if you offload most (if not all) ingest-time apps to HF, you don't have to restart your indexers if you have to change something in them.

Roy_9
Motivator

@VijaySrrie If you want to filter out the logs before even writing those to the indexers, HF will be a great option and it will be saving your license as well on your ingestion amounts.

PickleRick
SplunkTrust
SplunkTrust

License is counted based on data written to _indexes_, not indexers. You can reroute events to nullqueue on indexer if you don't have HF's.

0 Karma

Roy_9
Motivator

Yup indexes, it was a typo.

Thanks for correcting.

0 Karma
Get Updates on the Splunk Community!

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...