What is the best log collector solution?

[splunkreal]

Hello,

preferably based on linux (Redhat for instance), which log collector would you use to collect any kind of log (network devices, Checkpoint Log exporter, system logs, application logs, Windows servers...)?

Any newer solution than using syslog-ng or rsyslog?

Thanks.

[gcusello]

Hi @splunkreal,

I suppose that you're speking of log collecting in Splunk. 😉

Anyway, for os logs, if possible I try to always use Universal Forwarder because it gives me important features like local cache, packets compression, packet encryption, bandwidth optimization, etc...

When not possible I use syslogs, installing rsyslog on two Universal Forwarders: I  prefer this solution because using rsyslog I have a greater persistance of data on file system and I don't need to use an Heavy Forwarder that requires higher number of CPUs and RAM than a UF.

For syslogs, always using a Load Balancer to avoid Single Points of Failure.

Ciao.

Giuseppe

[splunkreal]

Hi @gcusello ,

We are rather looking for solutions like syslog-ng or rsyslog as they prefer to have non-commercial solution for basic log collection.

I’ve found interesting article here : https://www.splunk.com/en_us/blog/tips-and-tricks/syslog-ng-and-hec-scalable-aggregated-data-collect...

Does this mean we should use HEC instead of UF?

Thanks for your help.

[isoutamo]

If you are not familiar with syslog (or actually syslog-ng or rsyslog) and haven't easily suitable infra for it, you should look this https://splunkbase.splunk.com/app/4740. Splunk have productised this framework and call it Splunk Connect for Syslog (SC4S).

[splunkreal]

Hi @isoutamo we are rather familiar with syslog-ng or rsyslog, and preferring using non-commercial product so what would you suggest as architecture, also using UF or HEC?

Thanks for your help.

[isoutamo]

SC4S is free solution from Splunk and it has developed to be a gateway between syslog and splunk. Of course if you have already your own architecture, nodes and other needed components (like load balancers) then you could use those and build your solution from scratch. Otherwise I definitely consider SC4S.