Getting Data In
Highlighted

Using Splunk to collect Syslog and forward to remote syslog

Path Finder

So Splunk can collect syslog by configure data input at TCP/UDP port 514. Can I know:

  • Splunk does not manipulate the syslog data coming in right?
  • How then to forward these syslog data to another remote syslog server? Splunk indexes them as they come in through port 514 so I don't think spunk can forward to a remote syslog server within spunk itself.?
  • I am totally using spunk as syslog collector in this situation. No rsyslog or syslog-ng

thanks

Update: I realize some of my logs could not be converted to syslog format, hence I am still going to try going ahead with Splunk as the syslog collector. I am just using a dedicated splunk instance as the syslog indexer and will not have too much restarting done. Even if there is a restart, my Splunk forwarders can store/buffer events first before sending..I think.

Tags (1)
0 Karma
Highlighted

Re: Using Splunk to collect Syslog and forward to remote syslog

SplunkTrust
SplunkTrust

UPDATE:

YOU CAN FORWARD SYSLOG IN ORIGINAL FORMAT FROM A HEAVY FORWARDER:
http://docs.splunk.com/Documentation/Splunk/6.2.0/Forwarding/Forwarddatatothird-partysystemsd#Syslog...

Old WRONG answer:
Splunk manipulates the syslog data for sure. It changes it into the indexed form of data and compresses the data for storage. If you go look at the index file, it will be binary... not syslog events.

You cannot forward syslog from splunk. You can however pull data out of splunk using ODBC drivers, python, bash scripting, etc.

If you need to forward syslog, you'll need to stick to traditional methods such as syslog-ng, rsyslog, kafka, redis, network load balancing, etc.

View solution in original post

Highlighted

Re: Using Splunk to collect Syslog and forward to remote syslog

SplunkTrust
SplunkTrust

Also if you're listening on port 514 with splunk on a linux machine, then that means you're most likely running splunk as root. That is against best practices. Consider yourself warned.

0 Karma
Highlighted

Re: Using Splunk to collect Syslog and forward to remote syslog

Path Finder

ok then the only solution is to use rsyslog and then use a forwarder and configure output.conf to forward to a remote. thanks.

0 Karma
Highlighted

Re: Using Splunk to collect Syslog and forward to remote syslog

Champion

Pretty sure Splunk can forward syslog (as syslog) to other sources - it's just done at the forwarding layer (and might require a HFW). Not sure that I'd recommend it as a best practice, but it is possible.

Highlighted

Re: Using Splunk to collect Syslog and forward to remote syslog

SplunkTrust
SplunkTrust

A Splunk forwarder forwards "cooked" events by default. Cooked events will not be in syslog format.

I never realized it but you CAN forward traditional syslog. SORRY! EDITED MY ANSWER.

http://docs.splunk.com/Documentation/Splunk/6.2.0/Forwarding/Forwarddatatothird-partysystemsd#Syslog...

0 Karma
Highlighted

Re: Using Splunk to collect Syslog and forward to remote syslog

Splunk Employee
Splunk Employee

I downvoted this post because the solution proposed would be unstable for production use.

0 Karma
Highlighted

Re: Using Splunk to collect Syslog and forward to remote syslog

SplunkTrust
SplunkTrust

If Splunk can't support this as a "production stable" functionality then it shouldn't be in the product IMHO. Regardless the questions asked were answered. Down vote all you want. We both know Splunk isn't designed to be a syslog forwarder.

0 Karma
Highlighted

Re: Using Splunk to collect Syslog and forward to remote syslog

Splunk Employee
Splunk Employee

Splunk CAN forward syslog however this should be avoided in almost all cases. Splunk processes reload or restart for a number of reasons and are not designed to be HA for syslog. There are cases such as small/remote office where this is an appropriate use for Splunk, not the rule however.

Syslog-NG is the most common and preferred aggregation solution in front of Splunk. Generally speaking a NLB (or clustered pair) will be placed in front of two or more syslog servers. Syslog-NG will write a copy of the data to disk for the Universal Forwarder to collect and forward a subset of messages to another system such as the Cisco NAM or UniCenter for It Monitoring.

My guide for syslog configuration would be a good starting point for you
http://www.rfaircloth.com/2016/01/17/building-reliable-syslog-infrastructure-on-centos-7/