topic Re: Having issues with universal forwarder in Getting Data In

Having issues with universal forwarder

bhavya_shah — Mon, 15 Jul 2013 20:14:38 GMT

I checked that there are no firewall issues.

On the universal forwarder in splunkd.log:

07-15-2013 13:09:50.264 -0700 INFO TcpOutputProc - Connected to idx=x.x.x.x:9997
07-15-2013 13:09:52.395 -0700 INFO BatchReader - Removed from queue file='/opt/splunkforwarder/var/log/splunk/metrics.log.1'.
07-15-2013 13:09:52.636 -0700 INFO WatchedFile - Will begin reading at offset=4575529 for file=

On splunk server in splunkd.log

07-15-2013 14:36:05.672 -0400 INFO BatchReader - Removed from queue file

I am not sure why I dont see logs in indexer. Not sure what I might be missing?

Here are the files:

/opt/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/inputs.conf

[monitor:///log1/log2/log3]
sourcetype = syslog
index = syslog
disabled = false
crcSalt =
ignoreOlderThan = 1d
host_segment = 4

/opt/splunkforwarder/etc/system/local/outputs.conf

[tcpout]
defaultGroup=syslog_index
disabled = false

Forward the internal indexes as well as the non-internal ones

forwardedindex.0.whitelist = .*
forwardedindex.1.whitelist = _.*

[tcpout:syslog_index]
server=splunkserver:9997

Re: Having issues with universal forwarder

srioux — Mon, 15 Jul 2013 20:41:12 GMT

Do you have anything defined as part of inputs on your forwarder? You can verify by doing:

  $SPLUNK_HOME/bin/splunk list monitor

If you want the internal logs forwarded in, you may have to explicitly allow them in through outputs.conf configuration:

[tcpout]
defaultGroup = GroupName
disabled = false
# Forward the internal indexes as well as the non-internal ones
forwardedindex.0.whitelist = .*
forwardedindex.1.whitelist = _.*

Re: Having issues with universal forwarder

bhavya_shah — Mon, 15 Jul 2013 20:48:27 GMT

Yes I have defined stanza in inputs.conf file. Even after adding your configuration in output.conf file I still dont the logs coming in. Just not sure why?

Re: Having issues with universal forwarder

bmacias84 — Mon, 15 Jul 2013 23:10:58 GMT

If you can provide your outputs.conf and inputs.conf from your UF.

Re: Having issues with universal forwarder

bhavya_shah — Mon, 15 Jul 2013 23:28:18 GMT

I have updated the info.

Re: Having issues with universal forwarder

srioux — Tue, 16 Jul 2013 17:40:20 GMT

Did you set up your indexer to listen for incoming data, on the port you've defined in outputs.conf? Should be able to use the following command to see:

$SPLUNK_HOME/bin/splunk display listen

Re: Having issues with universal forwarder

bmacias84 — Tue, 16 Jul 2013 20:14:21 GMT

So you only have one tcpout configured on your UF? have you defined an index called syslog on your Indexer? On your indexer do you see any within your metrics.log regarding data being sent from your UF? Is this the only input defined on your UF?

Re: Having issues with universal forwarder

bhavya_shah — Tue, 16 Jul 2013 20:59:39 GMT

Yes.

Here is the output:
Receiving is enabled on port 9997.

Re: Having issues with universal forwarder

bhavya_shah — Tue, 16 Jul 2013 21:01:34 GMT

So you only have one tcpout configured on your UF?
Yes.

have you defined an index called syslog on your Indexer?
No.

On your indexer do you see any within your metrics.log regarding data being sent from your UF?

Sometime it shows and then its gone.

Is this the only input defined on your UF?

Yes

Re: Having issues with universal forwarder

bmacias84 — Tue, 16 Jul 2013 21:05:11 GMT

If you are defining index = syslog for your input on your UF you need to have a index called syslog on your indexer.

Re: Having issues with universal forwarder

bhavya_shah — Tue, 16 Jul 2013 21:06:28 GMT

Can you tell me which fine I need to modify on indexer?

Re: Having issues with universal forwarder

bmacias84 — Tue, 16 Jul 2013 21:07:43 GMT

indexes.conf

Re: Having issues with universal forwarder

bhavya_shah — Tue, 16 Jul 2013 21:12:30 GMT

Thank you so much for helping me out. I really appreciate your help.

It worked like a charm.

Re: Having issues with universal forwarder

bhavya_shah — Tue, 16 Jul 2013 21:34:28 GMT

The issue has been resolved.

Thanks bmacias84!

Resolution:

[monitor:///log1/log2/log3]
sourcetype = syslog
index = syslog
disabled = false
crcSalt =
ignoreOlderThan = 1d
host_segment = 4

If you are using index = syslog like in my case then make sure to edit the index.conf on splunk indexer to add it or just use index = default. And everything will work like a charm.

Re: Having issues with universal forwarder

lalit_mohan — Thu, 14 Nov 2013 10:32:31 GMT

Hi Guys,

I have two instances on microsoft azure environment one is splunk-server and other is splunk-forwarder(universalForwarder). Everything is fine with configuration ,then I tried to monitor tomcat logs and I have perform below steps on forwarder.

/usr/share/splunk_setup/splunkforwarder/bin/splunk add monitor /usr/share/apache-tomcat-7.0.42/logs/catalina.out -index default -sourcetype log4j -hostname splunkforwarder

But in search tab of splunk-web I always get No results found. search-query: host=splunkforwarder sourcetype=log4j

I checked inputs.conf ,CLI is not writing anything .So now I decided to write manually in these file.

Please tell me ,what I need to enter in my forwarders's inputs.conf and outputs.conf?

Thanks in advance!!