Archive

scaling HEC on heavy forwarders

Communicator

currently I am using single heavy forwarders as my HEC and the token generated from one heavy forwarders, however to avoid single point of failure I want expand to other 3 heavy forwarders but as the token generated from one single server how to use that one token to the rest for forwarder. any feedback highly appreciate it.

thanks

Tags (1)
0 Karma
1 Solution

Ultra Champion

You can manually create inputs on your HFs (or build an app to deploy them) and set the token across your environment.

In inputs.conf you need:

[http://your input name]
disabled = 0
token =your_token
<any other settings>

View solution in original post

Contributor

Hi raindrop18,

I'm using a cluster of Heavy Forwarders as HEC endpoints and I'm controlling it from master. I set my Heavy Forwarder to be a deployment client and I distribute HEC token from master to all of them !!

0 Karma

Ultra Champion

Hi @fabiocaldas,
Do you mean your cluster master is also a deployment server?
(I note that this is not a supported configuration, even though it works)

I hate to be pedantic, but in Splunk terminonology ‘cluster’ has a specific meaning. You can have a team, group, collection or even a pool of heavy forwarders, but you should avoid using the term ‘cluster’ to avoid confusion.

I think what your saying is your ‘pool’ of heavy forwarders are deployment clients of your deployment server.

The fact that your DS may (or may not be) collocated with your “indexer cluster master” is immaterial, but I would caution others from doing so.

0 Karma

Ultra Champion

You can manually create inputs on your HFs (or build an app to deploy them) and set the token across your environment.

In inputs.conf you need:

[http://your input name]
disabled = 0
token =your_token
<any other settings>

View solution in original post

Communicator

thanks much for response, do you have a doc link for the setup. I like to see the other settiing like index so on. appreciate it again.

0 Karma

Ultra Champion

Side note: Although the Splunk UI will generate you a 'token' you can use any valid 128 bit GUID/UUID which you can generate outside of Splunk, there is no 'magic' in the value Splunk generates for you!
https://www.guidgenerator.com/

0 Karma

Communicator

thanks, last question is that possible to create multiple endpoint and token in one input.conf? I have multiple HEC endpoint pointing different index and source type. one input.conf can handle that or I need multiple input.conf.

0 Karma

Ultra Champion

That’s fine, you can load them all up in one file, as long as the input names and tokens are unique.

0 Karma

Communicator

appreciate the quick response. is this works

[http]
index = A
sourcetype = log4j
disabled = 0

[http://collector-A]
disabled = 0
host = splunk-server
index = my-test1
indexes = my-test1
source = test1
token = XXXXXXXXXXXXXXXXXXXXXX
sourcetype = log4j


[http]
index = B
sourcetype = log4j
disabled = 0

[http://collector-B]
disabled = 0
host = splunk-server
index = my-test2
indexes = my-test2
source = test2
token = XXXXXXXXXXXXXXXXXXXXXX
sourcetype = log4j
0 Karma

Ultra Champion

Apologies for poor formatting. (I am on a phone) but this should be all you need.

 [http://collector-A]
 disabled = 0
 host = splunk-server
 index = my-test1
 source = test1
 token = XXXXXXXXXXXXXXXXXXXXXX
 sourcetype = log4j

 [http://collector-B]
 disabled = 0
 host = splunk-server
index = my-test2
 source = test2
 token = XXXXXXXXXXXXXXXXXXXXXX

 sourcetype = log4j
0 Karma

Communicator

thank you so very much!!!!

0 Karma

Ultra Champion

Sure: http://docs.splunk.com/Documentation/Splunk/7.0.1/admin/Inputsconf#http:_.28HTTP_Event_Collector.29

that covers all input parameters across all input types, but it will take you straight to the HEC specific settings to start.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!