We have two servers. One primary server (Debian Linux) running our software and a secondary server (also Debian Linux) running backups, fail-over and Splunk.
For some time now, I have been running my own software for mirroring the log-files from the primary server to the secondary -- making Splunk pick it up on arrival. My own implementation is working fine, but when the log-files gets rotated I need to spool them from the start to make sure that no log-entries is left behind. It is crucial that every line gets relayed to Splunk.
As an alternative I have decided to instead try to install Splunk Light Forwarder on the primary server. I have read a great deal about it in the manual and around the web, but I can't find any instruction on how to install just the Light Forwarder without the entire Splunk package.
My question is: How do I install the Splunk Light Forwarder on Debian Linux, so it doesn't eat up all my resources?
At the moment there is no standalone package for the Splunk Light Forwarder. You have to install the full package and then tell Splunk to only act as a LightForwarder. You have probably read this ( http://www.splunk.com/base/Documentation/latest/Admin/Enableforwardingandreceiving#Set_up_forwarding ) already.
Those are the important 2 commands:
./splunk enable app SplunkLightForwarder -auth <username>:<password> ./splunk add forward-server <host>:<port> -auth <username>:<password>
Once Splunk is set up as a light forwarder it doesn't use much system resources. We use Splunk on Solaris and limit the resources it is allowed to use at OS level when we start it.
I think that there are plans to ship a forwarder only package with release 4.2 which is due later this year.