existing central sysloghost data to new splunk server

[tnorth]

I have:

1. two existing OpenBSD Centralized Syslog loghosts (one is syslog-ng, one is syslogd)
2. a new dedicated server for running Splunk

I would like to leave the central loghosts in place and move the data to my new splunk server for analysis.
From what I have read, it seems like my options are:

1. use syslog-ng on both loghosts and write/forward to a syslog-ng server on the splunk server
2. use a splunk forwarder on loghosts to send data to splunk server
3. set up an rsync/cron process from the loghosts to the splunk server
4. set up a fifo on the loghost filesystems that writes locally and forwards to the splunk server
5. write logs to shared storage that both loghosts and splunk server have access to

I started on option #1 but stopped when I couldn't get the syslog-ng port to compile with "--enable-spoof-source" under OpenBSD 4.9.

Any recommendations or other options I should be considering?

Thanks,

-Tom

[dwaddle]

Assuming you can get the Splunk forwarder to run on OpenBSD -- which you may not be able to -- I would use approach #2. The cleanest approach by far is to use a Splunk forwarder reading the disk files created by syslogd. This gives you added resiliency during maintenance and certain outage situations.

You'll be able to upgrade the splunk installations without losing any of your syslog data (because it will be destaged to disk by syslogd), and if there is an outage either on the indexer or the network between the forwarder and indexer you won't lose any data there either.

A substantial issue, though, is that Splunk does not provide a native OpenBSD binary. And, no one has reported success using OpenBSD's FreeBSD compatibility ABI.

[gkanapathy]

If you can't get a forwarder to run on the syslog host, you could run Splunk on Linux (or something else) and have it read from the NFS-exported log directory.

[lguinn2]

Definitely option #2, if there is any way to make it work.