Getting Data In

Is it possible to transfer data from a heavy forwarder to two different cloud (cloud 1 and cloud 2) indexers?

vin02ptl
Explorer

I have one heavy forwarder and two different cloud indexers.

  1. heavy forwarder (HF)
  2. indx1(placed in cloud 1)
  3. indx2(placed in cloud 2)

Both are the part for different cluster. Is it possible to send the data from the HF to both the cloud indexers at the same time.

What will be the steps?

0 Karma

woodcock
Esteemed Legend

You can create 2 different output groups in outputs.conf but there are some important considerations/downsides, including:

1: You will take a license hit twice on this data.
2: Test the configurations for back-pressure problems. Depending on the version of Splunk and the settings that are in place, a blockage of one indexer tier can effect transmission of data to the unblocked indexer tier.

0 Karma

dkeck
Influencer

HI,

sure you can send data to both indexer.

Splunk automatically load balance incoming data to indexer you specify in outputs.conf

Like this

[tcpout: my_LB_indexers]
server=10.10.10.1:9997,10.10.10.2:9997

OR

if you want to clone the data like this:

[tcpout]
 defaultGroup=indexer1,indexer2

 [tcpout:indexer1]
 server=10.1.1.197:9997

 [tcpout:indexer2]
 server=10.1.1.200:9997

docs.splunk.com/Documentation/Forwarder/latest/Forwarder/Configureforwardingwithoutputs.conf#Configure_data_cloning_on_a_universal_forwarder_with_outputs.conf

0 Karma

ashishebansal
New Member

It should be work as per the above steps as posted by dKeck. If still not working then please share the output.conf settings.

0 Karma

vin02ptl
Explorer

both the indexers are in different cloud and they are part of totally different cluster and environment.

0 Karma

sarif_splunk
Splunk Employee
Splunk Employee

@vin02ptl - Do you want same data to go to both indexers?
If that's the case I don't think this approach would work.

0 Karma

dkeck
Influencer

Sure just have a look at the docs about cloning.

0 Karma

vin02ptl
Explorer

Thanks for your reply.......yes I want same data to go to both the indexers.

0 Karma

dkeck
Influencer

Just test the config for cloning I posted. Its working

0 Karma

dkeck
Influencer

If you HF can send to both indexer via recieving port 9997 I don´t see why this should not work.

You can also set up your HF to be a gateway forwarder for other UF´s

https://docs.splunk.com/Documentation/SplunkCloud/7.2.3/Data/Forwarddata

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

SOC4Kafka - New Kafka Connector Powered by OpenTelemetry

The new SOC4Kafka connector, built on OpenTelemetry, enables the collection of Kafka messages and forwards ...

Rounding off the Splunk Dashboard Contest

What does a contest-winning Splunk dashboard look like? In this case, it isn't in a browser tab at all. It ...

A Four Part Event Series: AI + Observability: AI Agents, LLMs, Apps, & Infrastructure

AI + Observability: AI Agents, LLMs, Apps, & Infrastructure The rapid evolution of artificial intelligence ...