Splunk Connect 4 Syslog (SC4S) with HEC in a Clustered Environment

[avoelk]

Hello,

I'm figuring out the best way to address the above situation. We do have a huge multisite cluster with 10 indexers on each site, a dedicated instance should act as the sc4s instance and send everything to a load balancer whose job will be to forward everything to the cluster. 

now, there are several documentations about the implementation but I still can't wrap my head around the direct approach. 

the SC4S config stanza would currently look something like this :

[http://SC4S]
disabled = 0
source = sc4s
sourcetype = sc4s:fallback
index = main
indexes = main, _metrics, firewall, proxy
persistentQueueSize  = 10MB
queueSize = 5MB
token = XXXXXX

several questions about that tho:
- I'd need to create a hec token first, before configuring SC4S, but in a clustered environment - where do I create the hec token? I've read that I should create it on the CM and then push it to the peers but how exactly? I can't find much info about the specifics. especially since I try to configure it via config files.. so an example of the correct stanza that has to be pushed out would be somehow great - just can't find any. 

- once pushed I need to configure the sc4s on the other side including the generated token (as seen above), does the config here seem correct? theres a lack of example configs so I'm spitballing here a little bit.

Kind regards

[PickleRick]

All "final" inputs that your LB balances traffic to need to have the same configuration so that they behave identically regardless of where your request is redirected - they have to have the same tokens defined, should set the same default metadata fields should they not be set by the sender, have the same queue parameters.

[jawahir007]

Check out this link; it might be helpful for you.

https://community.splunk.com/t5/Getting-Data-In/Setting-up-HEC-HTTP-Event-Collector-in-a-indexer-clu...

[dural_yyz · Accepted solution]

Create an app to be pushed from the CM to the IDX tier and put in an inputs.conf file.

https://docs.splunk.com/Documentation/Splunk/9.3.0/Admin/Inputsconf#HTTP_Event_Collector_.28HEC.29_-...

[http://sc4s]
token = XXXXX
index = target-index-name
### This is the bare minimum I suggest
### SC4S may require a sourcetype, other vendor sources may already come with that value assigned