scaling HEC on heavy forwarders

[raindrop18]

currently I am using single heavy forwarders as my HEC and the token generated from one heavy forwarders, however to avoid single point of failure I want expand to other 3 heavy forwarders but as the token generated from one single server how to use that one token to the rest for forwarder. any feedback highly appreciate it.

thanks

[fabiocaldas]

Hi raindrop18,

I'm using a cluster of Heavy Forwarders as HEC endpoints and I'm controlling it from master. I set my Heavy Forwarder to be a deployment client and I distribute HEC token from master to all of them !!

[nickhills]

Hi @fabiocaldas,
Do you mean your cluster master is also a deployment server?
(I note that this is not a supported configuration, even though it works)

I hate to be pedantic, but in Splunk terminonology ‘cluster’ has a specific meaning. You can have a team, group, collection or even a pool of heavy forwarders, but you should avoid using the term ‘cluster’ to avoid confusion.

I think what your saying is your ‘pool’ of heavy forwarders are deployment clients of your deployment server.

The fact that your DS may (or may not be) collocated with your “indexer cluster master” is immaterial, but I would caution others from doing so.

[nickhills · Accepted solution]

You can manually create inputs on your HFs (or build an app to deploy them) and set the token across your environment.

In inputs.conf you need:

[http://your input name]
disabled = 0
token =your_token
<any other settings>

[raindrop18]

thanks much for response, do you have a doc link for the setup. I like to see the other settiing like index so on. appreciate it again.

[nickhills]

Side note: Although the Splunk UI will generate you a 'token' you can use any valid 128 bit GUID/UUID which you can generate outside of Splunk, there is no 'magic' in the value Splunk generates for you!
https://www.guidgenerator.com/

[raindrop18]

thanks, last question is that possible to create multiple endpoint and token in one input.conf? I have multiple HEC endpoint pointing different index and source type. one input.conf can handle that or I need multiple input.conf.

[nickhills]

That’s fine, you can load them all up in one file, as long as the input names and tokens are unique.

[raindrop18]

appreciate the quick response. is this works

[http]
index = A
sourcetype = log4j
disabled = 0

[http://collector-A]
disabled = 0
host = splunk-server
index = my-test1
indexes = my-test1
source = test1
token = XXXXXXXXXXXXXXXXXXXXXX
sourcetype = log4j


[http]
index = B
sourcetype = log4j
disabled = 0

[http://collector-B]
disabled = 0
host = splunk-server
index = my-test2
indexes = my-test2
source = test2
token = XXXXXXXXXXXXXXXXXXXXXX
sourcetype = log4j

[nickhills]

Apologies for poor formatting. (I am on a phone) but this should be all you need.

 [http://collector-A]
 disabled = 0
 host = splunk-server
 index = my-test1
 source = test1
 token = XXXXXXXXXXXXXXXXXXXXXX
 sourcetype = log4j

 [http://collector-B]
 disabled = 0
 host = splunk-server
index = my-test2
 source = test2
 token = XXXXXXXXXXXXXXXXXXXXXX

 sourcetype = log4j

[raindrop18]

thank you so very much!!!!

[nickhills]

Sure: http://docs.splunk.com/Documentation/Splunk/7.0.1/admin/Inputsconf#http:_.28HTTP_Event_Collector.29

that covers all input parameters across all input types, but it will take you straight to the HEC specific settings to start.