Getting Data In

Best practices - Syslog-ng to splunk

gamsecurity
Explorer

Hi,

I know this topic isn't the first here, but I have some problem to get a good anwser for this specific problem.

In fact, we have a syslog server who collecting data from devices and we need to forward it to our Splunk server.
In our case syslog server running on syslog-ng and about our splunk server, we have only one server used to indexes and search.

My question is to know what is the best practices to forward data from our syslog-ng server to our splunk instance ?
For now, our syslog forward it directly over udp:514 but we have some problem with that (if splunk restart we loose some data and every-data are indexed in a unique index).

we need to know if its better to install an Universal forwarder on the syslog-ng to forward or install syslog-ng to our splunk instance and then monitor files sent by our syslog server over udp:514 ?

Thks for your help.

0 Karma
1 Solution

Richfez
SplunkTrust
SplunkTrust

Yes, install a Universal Forwarder and have that read the files the syslog server is creating, and send those into your Splunk server. It is exceedingly unlikely that you need anything fancy like HEC, just read files from disk with inputs.conf and if you have your outputs.conf set up (or have it set up via other methods) it'll just work.

Here's the configuration docs for a UF.

You hit on the biggest easy problem this solves - Splunk just takes forever to restart and you drop UDP events during that period. There are lots more problems it solves.

Happy Splunking,
Rich

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

See https://www.splunk.com/en_us/blog/tips-and-tricks/using-syslog-ng-with-splunk.html

---
If this reply helps you, Karma would be appreciated.

gamsecurity
Explorer

Thanks for the link.

0 Karma

Richfez
SplunkTrust
SplunkTrust

Yes, install a Universal Forwarder and have that read the files the syslog server is creating, and send those into your Splunk server. It is exceedingly unlikely that you need anything fancy like HEC, just read files from disk with inputs.conf and if you have your outputs.conf set up (or have it set up via other methods) it'll just work.

Here's the configuration docs for a UF.

You hit on the biggest easy problem this solves - Splunk just takes forever to restart and you drop UDP events during that period. There are lots more problems it solves.

Happy Splunking,
Rich

arunkumars954
Explorer

Hi,

You can install an Universal Forwarder on the Syslog server to forward data to your Splunk instance as a best practice.
Hardware requirements for a Splunk Universal forwarder https://docs.splunk.com/Documentation/Forwarder/8.0.3/Forwarder/Systemrequirements.

As you have also mentioned that you are losing some data while the Splunk server/services are restarted, you can use the UseACK(Indexer Acknowledgement) feature on the Universal Forwarder so that the data sent is acknowledged by the Splunk Instance. Till the ACK is not received, Splunk Universal Forwarder holds the events in queue and will resend again.
Refer this article https://docs.splunk.com/Documentation/Forwarder/8.0.3/Forwarder/Protectagainstthelossofin-flightdata for more information.

Please up vote this answer if it helps you with your query.

arunkumars954
Explorer
 
0 Karma

gamsecurity
Explorer

Thanks for advices

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...