topic Re: Universal Forwarder - stops forwarding after restarting splunk service in Getting Data In

Universal Forwarder - stops forwarding after restarting splunk service

carbonegg — Tue, 20 Dec 2011 17:34:49 GMT

I installed the universal forwarder 4.2.5 on my remote Linux machine and set it to monitor my squid access logs.

After installing it, I run the following commands to have the data sent to my Splunk server:

Start the server

/opt/splunkforwarder/bin/splunk start

Add server address

/opt/splunkforwarder/bin/splunk add forward-server 192.168.2.2:9997

Monitor File

/opt/splunkforwarder/bin/splunk monitor /var/log/squid/access.log -sourcetype squid

Immediately after issuing the last command which monitors the access.log file, the events start flowing into the server and I can view them on the Squid App and Search app. Everything works fine.

The problem starts when I restart Splunk. After issuing the ./splunk restart command, Splunk starts up okay, but the logs are no longer forwarded to the server. I have not seen any errors. I am not sure why it stops sending logs to the server after restarting it.

Any ideas?

Thanks!!!

Re: Universal Forwarder - stops forwarding after restarting splunk service

carbonegg — Tue, 20 Dec 2011 18:32:32 GMT

Hello Gekoner,

After issuing "splunk monitor /var/log/squid/access.log -sourcetype squid" command, the following is appended to the inputs.conf file in the "/opt/splunkforwarder/etc/apps/search/local" directory:

[monitor:///var/log/squid/access.log]
disabled = false
sourcetype = squid

And after the "splunk add forward-server 192.168.2.2:9997" command, the following gets appended to the outputs.conf file in the "/opt/splunkforwarder/etc/system/local" directory:

[tcpout]
defaultGroup = 192.168.2.2_9997
disabled = false

[tcpout:192.168.2.2_9997]
server = 192.168.2.2:9997

[tcpout-server://192.168.2.2:9997]

So I believe that what you are talking about is being done when I give splunk the commands mentioned above. I am not sure what other inputs.conf or outputs.conf are there for me to put the info in.

Even after a restart, if I issue the add monitor command again, it won't let me because it says the file is already being monitored. So there must be a file somewhere that knows I already issued the add monitor command. Even after I delete the info in the inputs.conf file and reissue the add monitor command, it still won't let me because it thinks I am already monitoring it.

Re: Universal Forwarder - stops forwarding after restarting splunk service

gekoner — Tue, 20 Dec 2011 20:54:31 GMT

OK, yes I see that those commands do add the correct stanza to the conf files. Run a splunk list forward-server after you restart splunk on the universal forwarder. And let us know the output.