I am a newbie on Splunk and I am trying to setup a Splunk server and a Splunk Light forwarder to forward data to it, here are the things that I did during the setup:
1) I installed a full Splunk on a Linux server(Server1). Started the Splunk daemon and installed *nix app.
2) I enabled a script 'df' to test if it is workin. Indeed the monitor is working.
3) I then enabled the receiving by go to Manager->Configure Receiving and Forwarding-> set port 9997 as the listening port.
4) Installed another full Splunk on another Linux server (Server2). Installed the *nix app also and have it working.
I then configured it to be a forwarder via the following commands:
cd /opt/splunk/bin ./splunk start ./splunk enable app SplunkLightForwarder ./splunk restart ./splunk add forward-server server1:9997
My inputs.conf (/opt/splunk/etc/apps/unix/local/inputs) is liked this one:
[script://./bin/cpu.sh] [script://./bin/df.sh] disabled = 0 [script://./bin/hardware.sh] [script://./bin/interfaces.sh] [script://./bin/iostat.sh] [script://./bin/lastlog.sh] [script://./bin/lsof.sh] [script://./bin/netstat.sh] [script://./bin/openPorts.sh] [script://./bin/package.sh] [script://./bin/protocol.sh] [script://./bin/ps.sh] [script://./bin/rlog.sh] [script://./bin/time.sh] [script://./bin/top.sh] [script://./bin/usersWithLoginPrivs.sh] [script://./bin/vmstat.sh] [script://./bin/who.sh]
My outputs.conf (/opt/splunk/etc/apps/unix/local/outputs.conf) is liked this one:
[tcpout] defaultGroup = server1.domain.com_9997 disabled = false [tcpout:server1.domain.com_9997] server = server1.domain.com:9997 [tcpout-server://server1.domain.com:9997]
I restarted both servers Splunk Daemon but still the server1 cannot see the data from server2.
Did I miss any configurations? please advise, thank you very much.
You should add the following line -on the client side-
where /var/log is the directory of logs files.
You may want to check that IPTables is off or allowing TCP 9997 outbound from Server 2 and inbound to server 1.
You may find a useful message in the splunkd.log on your light forwarder. Normally it will indicate if a sucessful network connection has been made. Look here: /opt/splunk/var/log/splunk/splunkd.log
The forwarding of data from the client is now working. It just happened that I have my Splunk server as a client and have my outputs forwarded to another Splunk server. I am not 100% sure this was the cause but this was the only thing I changed before the forwarding worked.
Thanks everyone for the help.