I'm think about the best architecture for a huge amount of syslog data.
At first, I used rsyslog in rhel with single Splunk server. But syslog is written very slowly when udp syslog data is about 2GB per day in total and affordable cpu cores and RAM, sometimes Splunk indexed it mistakenly when rsyslog stopped to write the middle of event. I use time_before_close = 300 in an inputs.conf.
It works temporary, but I concerns about happening again when syslog data transferred to this Splunk server increase .
So now I'm thinking about the best architecture for it. We have several options.
Using heavy forwarder instead of rsyslog
Tuning rsyslog parameters
I don't know about the difference of performance between rsyslog and Splunk tcpinput. And in this case, what parameter in rsyslog does increase performance ?
If you know anything about it, please let me know. Thank you very much.
... View more