Getting Data In

Splunk Connect 4 Syslog (SC4S) with HEC in a Clustered Environment

avoelk
Communicator

Hello,

I'm figuring out the best way to address the above situation. We do have a huge multisite cluster with 10 indexers on each site, a dedicated instance should act as the sc4s instance and send everything to a load balancer whose job will be to forward everything to the cluster. 

now, there are several documentations about the implementation but I still can't wrap my head around the direct approach. 

the SC4S config stanza would currently look something like this :

 

[http://SC4S]
disabled = 0
source = sc4s
sourcetype = sc4s:fallback
index = main
indexes = main, _metrics, firewall, proxy
persistentQueueSize  = 10MB
queueSize = 5MB
token = XXXXXX

 

 

several questions about that tho:
- I'd need to create a hec token first, before configuring SC4S, but in a clustered environment - where do I create the hec token? I've read that I should create it on the CM and then push it to the peers but how exactly? I can't find much info about the specifics. especially since I try to configure it via config files.. so an example of the correct stanza that has to be pushed out would be somehow great - just can't find any. 

- once pushed I need to configure the sc4s on the other side including the generated token (as seen above), does the config here seem correct? theres a lack of example configs so I'm spitballing here a little bit.

 

Kind regards

Labels (3)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

All "final" inputs that your LB balances traffic to need to have the same configuration so that they behave identically regardless of where your request is redirected - they have to have the same tokens defined, should set the same default metadata fields should they not be set by the sender, have the same queue parameters.

 

0 Karma

jawahir007
Communicator
0 Karma

dural_yyz
Builder

Create an app to be pushed from the CM to the IDX tier and put in an inputs.conf file.

https://docs.splunk.com/Documentation/Splunk/9.3.0/Admin/Inputsconf#HTTP_Event_Collector_.28HEC.29_-...

[http://sc4s]
token = XXXXX
index = target-index-name
### This is the bare minimum I suggest
### SC4S may require a sourcetype, other vendor sources may already come with that value assigned
0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to November Tech Talks, Office Hours, and Webinars!

🍂 Fall into November with a fresh lineup of Community Office Hours, Tech Talks, and Webinars we’ve ...

Transform your security operations with Splunk Enterprise Security

Hi Splunk Community, Splunk Platform has set a great foundation for your security operations. With the ...

Splunk Admins and App Developers | Earn a $35 gift card!

Splunk, in collaboration with ESG (Enterprise Strategy Group) by TechTarget, is excited to announce a ...