Best practices - Syslog-ng to splunk

[gamsecurity]

Hi,

I know this topic isn't the first here, but I have some problem to get a good anwser for this specific problem.

In fact, we have a syslog server who collecting data from devices and we need to forward it to our Splunk server.
In our case syslog server running on syslog-ng and about our splunk server, we have only one server used to indexes and search.

My question is to know what is the best practices to forward data from our syslog-ng server to our splunk instance ?
For now, our syslog forward it directly over udp:514 but we have some problem with that (if splunk restart we loose some data and every-data are indexed in a unique index).

we need to know if its better to install an Universal forwarder on the syslog-ng to forward or install syslog-ng to our splunk instance and then monitor files sent by our syslog server over udp:514 ?

Thks for your help.

[richgalloway]

See https://www.splunk.com/en_us/blog/tips-and-tricks/using-syslog-ng-with-splunk.html

[gamsecurity]

Thanks for the link.

[Richfez · Accepted solution]

Yes, install a Universal Forwarder and have that read the files the syslog server is creating, and send those into your Splunk server. It is exceedingly unlikely that you need anything fancy like HEC, just read files from disk with inputs.conf and if you have your outputs.conf set up (or have it set up via other methods) it'll just work.

Here's the configuration docs for a UF.

You hit on the biggest easy problem this solves - Splunk just takes forever to restart and you drop UDP events during that period. There are lots more problems it solves.

Happy Splunking,
Rich

[arunkumars954]

Hi,

You can install an Universal Forwarder on the Syslog server to forward data to your Splunk instance as a best practice.
Hardware requirements for a Splunk Universal forwarder https://docs.splunk.com/Documentation/Forwarder/8.0.3/Forwarder/Systemrequirements.

As you have also mentioned that you are losing some data while the Splunk server/services are restarted, you can use the UseACK(Indexer Acknowledgement) feature on the Universal Forwarder so that the data sent is acknowledged by the Splunk Instance. Till the ACK is not received, Splunk Universal Forwarder holds the events in queue and will resend again.
Refer this article https://docs.splunk.com/Documentation/Forwarder/8.0.3/Forwarder/Protectagainstthelossofin-flightdata for more information.

Please up vote this answer if it helps you with your query.

[gamsecurity]

Thanks for advices