topic Re: existing central sysloghost data to new splunk server in Getting Data In

existing central sysloghost data to new splunk server

tnorth — Sat, 20 Aug 2011 13:05:43 GMT

I have:

two existing OpenBSD Centralized Syslog loghosts (one is syslog-ng, one is syslogd)
a new dedicated server for running Splunk

I would like to leave the central loghosts in place and move the data to my new splunk server for analysis.
From what I have read, it seems like my options are:

use syslog-ng on both loghosts and write/forward to a syslog-ng server on the splunk server
use a splunk forwarder on loghosts to send data to splunk server
set up an rsync/cron process from the loghosts to the splunk server
set up a fifo on the loghost filesystems that writes locally and forwards to the splunk server
write logs to shared storage that both loghosts and splunk server have access to

I started on option #1 but stopped when I couldn't get the syslog-ng port to compile with "--enable-spoof-source" under OpenBSD 4.9.

Any recommendations or other options I should be considering?

Thanks,

-Tom

Re: existing central sysloghost data to new splunk server

dwaddle — Sat, 20 Aug 2011 19:26:11 GMT

Assuming you can get the Splunk forwarder to run on OpenBSD -- which you may not be able to -- I would use approach #2. The cleanest approach by far is to use a Splunk forwarder reading the disk files created by syslogd. This gives you added resiliency during maintenance and certain outage situations.

You'll be able to upgrade the splunk installations without losing any of your syslog data (because it will be destaged to disk by syslogd), and if there is an outage either on the indexer or the network between the forwarder and indexer you won't lose any data there either.

A substantial issue, though, is that Splunk does not provide a native OpenBSD binary. And, no one has reported success using OpenBSD's FreeBSD compatibility ABI.

Re: existing central sysloghost data to new splunk server

lguinn2 — Sun, 21 Aug 2011 15:51:50 GMT

Definitely option #2, if there is any way to make it work.

Re: existing central sysloghost data to new splunk server

gkanapathy — Mon, 22 Aug 2011 03:35:57 GMT

If you can't get a forwarder to run on the syslog host, you could run Splunk on Linux (or something else) and have it read from the NFS-exported log directory.