Thank you for the repy Adonio.
I was thinking of using syslogNG for "windows" as we are planning to use it for a number of syslog-data sources well. But I agree syslogNG might not be the best solution for a windows events scenario, i.e. security, application, system.
The problem I am facing is that I need to send winevents from UFs (on hosts) to the indexers cooked and to a 3rd party application uncooked/raw.
I have successfully tested sending to both from a HF and it works for a while, but then the HF starts dropping events to the indexers.
11-09-2018 13:06:27.418 +0000 WARN TcpOutputProc - Queue for group INDEXERS_cooked has begun dropping events
11-09-2018 13:07:03.547 +0000 INFO TcpOutputProc - Queue for group INDEXERS_cooked has stopped dropping events
Per Splunk Support, I have tried changing dropClonedEventsOnQueue/dropClonedEventsOnQueueFull, but no luck... I still see dropping events. Ultimately, support says that "cloned events" is not a supported feature.
When I comment out the stanzas to send uncooked to the 3rd party, I notice that there are a number of blocked events to the indexers.
11-12-2018 14:12:20.034 +0000 WARN TcpOutputProc - Forwarding to indexer group lb blocked for 90 seconds.
11-12-2018 14:12:30.054 +0000 WARN TcpOutputProc - Forwarding to indexer group lb blocked for 100 seconds.
It appears when the windows machines (approx 45) send high volume spikes, the HF can handle it but the indexers cannot.
So the primary use case is to get a collection server to aggregate winevents for: 1) to create a buffer for the indexers and 2) has the ability to send raw data to a 3rd party application and or s3....
we plan to ingest >10,000 of endpoint logs...
Any suggestions in this scenario?
Thank you
... View more