topic Splunk Connect 4 Syslog (SC4S) with HEC in a Clustered Environment in Getting Data In

Splunk Connect 4 Syslog (SC4S) with HEC in a Clustered Environment

avoelk — Tue, 08 Oct 2024 13:06:38 GMT

Hello,

I'm figuring out the best way to address the above situation. We do have a huge multisite cluster with 10 indexers on each site, a dedicated instance should act as the sc4s instance and send everything to a load balancer whose job will be to forward everything to the cluster.

now, there are several documentations about the implementation but I still can't wrap my head around the direct approach.

the SC4S config stanza would currently look something like this :

[http://SC4S] disabled = 0 source = sc4s sourcetype = sc4s:fallback index = main indexes = main, _metrics, firewall, proxy persistentQueueSize = 10MB queueSize = 5MB token = XXXXXX

several questions about that tho:
- I'd need to create a hec token first, before configuring SC4S, but in a clustered environment - where do I create the hec token? I've read that I should create it on the CM and then push it to the peers but how exactly? I can't find much info about the specifics. especially since I try to configure it via config files.. so an example of the correct stanza that has to be pushed out would be somehow great - just can't find any.

- once pushed I need to configure the sc4s on the other side including the generated token (as seen above), does the config here seem correct? theres a lack of example configs so I'm spitballing here a little bit.

Kind regards

Re: Splunk Connect 4 Syslog (SC4S) with HEC in a Clustered Environment

dural_yyz — Tue, 08 Oct 2024 13:17:18 GMT

Create an app to be pushed from the CM to the IDX tier and put in an inputs.conf file.

https://docs.splunk.com/Documentation/Splunk/9.3.0/Admin/Inputsconf#HTTP_Event_Collector_.28HEC.29_-_Local_stanza_for_each_token

[http://sc4s] token = XXXXX index = target-index-name ### This is the bare minimum I suggest ### SC4S may require a sourcetype, other vendor sources may already come with that value assigned

Re: Splunk Connect 4 Syslog (SC4S) with HEC in a Clustered Environment

Jawahir — Tue, 08 Oct 2024 20:25:36 GMT

Check out this link; it might be helpful for you.

https://community.splunk.com/t5/Getting-Data-In/Setting-up-HEC-HTTP-Event-Collector-in-a-indexer-cluster/m-p/560975/highlight/true#M92683

Re: Splunk Connect 4 Syslog (SC4S) with HEC in a Clustered Environment

PickleRick — Tue, 08 Oct 2024 21:10:26 GMT

All "final" inputs that your LB balances traffic to need to have the same configuration so that they behave identically regardless of where your request is redirected - they have to have the same tokens defined, should set the same default metadata fields should they not be set by the sender, have the same queue parameters.