how to forwarder HEC inputs logs to 2 different splunk instance ( Splunk OnPrem and SplunkCloud )

[raomu]

We have heavy forwarder that accept logs over HEC. 

inputs.conf 
[http://dd-log-token1]

index= ddlogs1
token = XXXXX XXX XXX XXX

[http://dd-log-token2]

index= ddlogs2
token = XXXXX XXX XXX XXX

[http://dd-log-token3]

index= ddlogs3
token = XXXXX XXX XXX XXX


________________________________

I want to forward only below inputs to 2 different splunk Instances -
1- splunkCloud (hosted by Splunk)
2-SplunkOnPrem 

[http://dd-log-token2]

index= ddlogs2
token = XXXXX XXX XXX XXX

________________________________

This is my inputs.conf looks like 

inputs.conf 
[http://dd-log-token1]

index= ddlogs1
token = XXXXX XXX XXX XXX

[http://dd-log-token2]

index= ddlogs2
token = XXXXX XXX XXX XXX
outputgroup = splunkonprem, splunkcloud

[http://dd-log-token3]

index= ddlogs3
token = XXXXX XXX XXX XXX

_____________

outputs.conf 

[tcpout]
defaultgroup = splunkonprem,splunkcloud 
forceTimebasedAutoLB = true 

[tcpout: splunkonprem]
server= zyx.com:9997, abc.com:9997

[tcpout: splunkonprem]
server= mmm.com:9997, bbb.com:9997

But these settings are only sending logs to Onprem indexers not to SplunkCloud indexers.
Please suggest if any idea whats wrong with my configuration.

[livehybrid]

Hi @raomu 

You need to correct your `outputs.conf` configuration as you have a duplicate stanza name "[tcpout: splunkonprem]" and you haven't defined the "splunkcloud" output group.

Additionally, the defaultGroup setting in the [tcpout] stanza determines where data goes if an outputgroup is not specified in inputs.conf.

To send only "dd-log-token2" data to both destinations and all other data only to On-Prem (as implied by your goal), configure outputs.conf

[tcpout]
# Data without a specific outputgroup goes here 
defaultGroup = splunkonprem 

forceTimebasedAutoLB = true

[tcpout:splunkonprem] 
# Your On-Prem indexers 
server = zyx.com:9997, abc.com:9997 

[tcpout:splunkcloud] 
# Your Splunk Cloud forwarder endpoint
server = <your_splunk_cloud_inputs_endpoints>:9997 

Add other relevant settings like compressed=true, useACK=true if needed and  any required Splunk Cloud specific settings (e.g., sslCertPath, sslPassword if using certs)

inputs.conf on Heavy Forwarder

[http://dd-log-token1]
index= ddlogs1
token = XXXXX XXX XXX XXX

[http://dd-log-token2] 
index= ddlogs2 
token = XXXXX XXX XXX XXX 
# This overrides defaultGroup and sends to both 
outputgroup = splunkonprem, splunkcloud 

[http://dd-log-token3] 
index= ddlogs3 
token = XXXXX XXX XXX XXX

 Explanation:

• outputs.conf/[tcpout]/defaultGroup: Sets the default destination(s) for data that doesn't have a specific outputgroup assigned in inputs.conf. In this corrected example, data defaults to "splunkonprem" only.
• outputs.conf/[tcpout:groupname]: Defines named output groups. You need one stanza for each group (`splunkonprem` and `splunkcloud`) with the correct server details. Stanza names must be unique.
• inputs.conf/[stanza]/outputgroup: Assigns data from that specific input stanza to the listed output group(s), overriding the defaultGroup. The setting "outputgroup = splunkonprem, splunkcloud" sends data from [http://dd-log-token2/] to both defined groups.

Further Troubleshooting:

• Can you see your Splunk Forwarder establishing a connection to Splunk Cloud successfully? We need to rule out connection issues to Splunk Cloud which arent related to the outputgroup.
• Check the $SPLUNK_HOME/var/log/splunk/splunkd.log for errors setting up the connection.
• Ensure the Splunk Cloud inputs endpoint (`<your_splunk_cloud_inputs_endpoints>:9997`) is correct for your stack. There are often ~12 input servers listed.
• Verify network connectivity (firewall rules) from the Heavy Forwarder to both your On-Prem indexers and the Splunk Cloud inputs endpoint on port 9997.
• Restart the Splunk forwarder service after applying configuration changes.

Useful Docs:

• outputs.conf: https://docs.splunk.com/Documentation/Splunk/latest/Admin/Outputsconf
• inputs.conf (HTTP Event Collector section): https://docs.splunk.com/Documentation/Splunk/latest/Admin/Inputsconf#http:_.28HTTP_Event_Collector.2...
• Forward data based on source, sourcetype, or host: https://docs.splunk.com/Documentation/Splunk/latest/Forwarding/Routeandfilterdatad#Route_inputs_to_s...

🌟 Did this answer help you? If so, please consider:

• Adding karma to show it was useful
• Marking it as the solution if it resolved your issue
• Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

[PickleRick]

1.

outputgroup = <string>
* The name of the output group to which the event collector forwards data.
* There is no support for using this setting to send data over HTTP with a heavy forwarder.

2. For cloud you don't send to 9997.

3. You can't use http output and normal s2s output at the same time.