Getting Data In

Best practices - Syslog-ng to splunk

gamsecurity
Explorer

Hi,

I know this topic isn't the first here, but I have some problem to get a good anwser for this specific problem.

In fact, we have a syslog server who collecting data from devices and we need to forward it to our Splunk server.
In our case syslog server running on syslog-ng and about our splunk server, we have only one server used to indexes and search.

My question is to know what is the best practices to forward data from our syslog-ng server to our splunk instance ?
For now, our syslog forward it directly over udp:514 but we have some problem with that (if splunk restart we loose some data and every-data are indexed in a unique index).

we need to know if its better to install an Universal forwarder on the syslog-ng to forward or install syslog-ng to our splunk instance and then monitor files sent by our syslog server over udp:514 ?

Thks for your help.

0 Karma
1 Solution

Richfez
SplunkTrust
SplunkTrust

Yes, install a Universal Forwarder and have that read the files the syslog server is creating, and send those into your Splunk server. It is exceedingly unlikely that you need anything fancy like HEC, just read files from disk with inputs.conf and if you have your outputs.conf set up (or have it set up via other methods) it'll just work.

Here's the configuration docs for a UF.

You hit on the biggest easy problem this solves - Splunk just takes forever to restart and you drop UDP events during that period. There are lots more problems it solves.

Happy Splunking,
Rich

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

See https://www.splunk.com/en_us/blog/tips-and-tricks/using-syslog-ng-with-splunk.html

---
If this reply helps you, Karma would be appreciated.

gamsecurity
Explorer

Thanks for the link.

0 Karma

Richfez
SplunkTrust
SplunkTrust

Yes, install a Universal Forwarder and have that read the files the syslog server is creating, and send those into your Splunk server. It is exceedingly unlikely that you need anything fancy like HEC, just read files from disk with inputs.conf and if you have your outputs.conf set up (or have it set up via other methods) it'll just work.

Here's the configuration docs for a UF.

You hit on the biggest easy problem this solves - Splunk just takes forever to restart and you drop UDP events during that period. There are lots more problems it solves.

Happy Splunking,
Rich

arunkumars954
Explorer

Hi,

You can install an Universal Forwarder on the Syslog server to forward data to your Splunk instance as a best practice.
Hardware requirements for a Splunk Universal forwarder https://docs.splunk.com/Documentation/Forwarder/8.0.3/Forwarder/Systemrequirements.

As you have also mentioned that you are losing some data while the Splunk server/services are restarted, you can use the UseACK(Indexer Acknowledgement) feature on the Universal Forwarder so that the data sent is acknowledged by the Splunk Instance. Till the ACK is not received, Splunk Universal Forwarder holds the events in queue and will resend again.
Refer this article https://docs.splunk.com/Documentation/Forwarder/8.0.3/Forwarder/Protectagainstthelossofin-flightdata for more information.

Please up vote this answer if it helps you with your query.

arunkumars954
Explorer
 
0 Karma

gamsecurity
Explorer

Thanks for advices

0 Karma
Get Updates on the Splunk Community!

The Splunk Success Framework: Your Guide to Successful Splunk Implementations

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...