topic Re: Syslog and Splunk? in Getting Data In

Syslog and Splunk?

splunktrainingu — Thu, 02 Jul 2020 01:09:33 GMT

Hello Everyone,

We are sending VPN data via syslog over UDP to our splunk server. The reason we are using syslog and not a forwarder is because we wanted to filter the VPN logins to only show the ones that are for our environment this is why we used syslog. Anyways my concern is I have been reading online that having the syslog sent to the splunk server can cause issues and splunk would need to operate as root to access the syslogs. Should I have a separate server for the syslogs then figure out a way to send them to the splunk server?

Re: Syslog and Splunk?

gcusello — Thu, 02 Jul 2020 07:10:13 GMT

Hi @splunktrainingu ,

I don't like syslogs because if you have a problem (maintenance, congetione, etc...), you lose them.

If your need is only to filter logs, you can do this following the documentation at https://docs.splunk.com/Documentation/Splunk/8.0.4/Forwarding/Routeandfilterdatad#Filter_event_data_and_send_to_queues

Anyway, there isn't any problem taking syslogs and the the access to the syslogs is the same of the other files: if you set grants for splunk user, you can access the syslogs.

If you want to use syslogs, it's a good practice to use two Heavy Forwarders with a Load Balancer to take them, so you can separate this function from the Indexers, two and a LB for HA needs.

Ciao.

Giuseppe

Re: Syslog and Splunk?

splunktrainingu — Thu, 02 Jul 2020 16:58:18 GMT

@gcusello Sorry I want to clarify something. But do I need two Heavy forwarders if the syslog is already on the same server as my splunk server? If I wanted to use the Heavy forwarders with load balancer then wouldn't I have to setup a separate syslog server that then forwards to my splunk server?

Re: Syslog and Splunk?

richgalloway — Thu, 02 Jul 2020 17:40:54 GMT

See http://www.georgestarcher.com/splunk-success-with-syslog/ for a good description of why one shouldn't send syslog directly into Splunk and how to use a syslog server instead.

You should be able to configure rsyslog or syslog-ng to filter your VPN data for you.

If your syslog server is on the same server as Splunk then there's no need for a forwarder. Just define input(s) to monitor the directories where the syslog data is stored.

Re: Syslog and Splunk?

splunktrainingu — Thu, 02 Jul 2020 18:55:12 GMT

If your syslog server is on the same server as Splunk then there's no need for a forwarder. Just define input(s) to monitor the directories where the syslog data is stored.

@richgalloway when you say inputs.conf file do you mean the inputs.conf file in Splunk/opt/splunk/etc/system/local ?

Re: Syslog and Splunk?

richgalloway — Thu, 02 Jul 2020 19:06:04 GMT

That's one option. You could also use an inputs.conf file in a custom app.

Re: Syslog and Splunk?

splunktrainingu — Thu, 02 Jul 2020 19:38:14 GMT

Which inputs.conf file are you talking about? Would it be easier to make an index that points to the location of the syslog?

Re: Syslog and Splunk?

richgalloway — Fri, 03 Jul 2020 12:54:42 GMT

I'm talking about ANY inputs.conf file. It doesn't make much difference. You can use $SPLUNK_HOME/etc/system/local/inputs.conf if you prefer, but I prefer to put my custom configs in an app I create, like $SPLUNK_HOME/etc/apps/my_inputs/local/inputs.conf.
It's not possible to make an index point to the location of the syslog. You must define an input that reads from the syslog file(s) and writes to an index.