Hello,
I'm figuring out the best way to address the above situation. We do have a huge multisite cluster with 10 indexers on each site, a dedicated instance should act as the sc4s instance and send everything to a load balancer whose job will be to forward everything to the cluster.
now, there are several documentations about the implementation but I still can't wrap my head around the direct approach.
the SC4S config stanza would currently look something like this :
[http://SC4S]
disabled = 0
source = sc4s
sourcetype = sc4s:fallback
index = main
indexes = main, _metrics, firewall, proxy
persistentQueueSize = 10MB
queueSize = 5MB
token = XXXXXX
several questions about that tho:
- I'd need to create a hec token first, before configuring SC4S, but in a clustered environment - where do I create the hec token? I've read that I should create it on the CM and then push it to the peers but how exactly? I can't find much info about the specifics. especially since I try to configure it via config files.. so an example of the correct stanza that has to be pushed out would be somehow great - just can't find any.
- once pushed I need to configure the sc4s on the other side including the generated token (as seen above), does the config here seem correct? theres a lack of example configs so I'm spitballing here a little bit.
Kind regards
All "final" inputs that your LB balances traffic to need to have the same configuration so that they behave identically regardless of where your request is redirected - they have to have the same tokens defined, should set the same default metadata fields should they not be set by the sender, have the same queue parameters.
Check out this link; it might be helpful for you.
Create an app to be pushed from the CM to the IDX tier and put in an inputs.conf file.
[http://sc4s]
token = XXXXX
index = target-index-name
### This is the bare minimum I suggest
### SC4S may require a sourcetype, other vendor sources may already come with that value assigned