Getting Data In

Having issues with universal forwarder

bhavya_shah
Path Finder

I checked that there are no firewall issues.

On the universal forwarder in splunkd.log:

07-15-2013 13:09:50.264 -0700 INFO TcpOutputProc - Connected to idx=x.x.x.x:9997
07-15-2013 13:09:52.395 -0700 INFO BatchReader - Removed from queue file='/opt/splunkforwarder/var/log/splunk/metrics.log.1'.
07-15-2013 13:09:52.636 -0700 INFO WatchedFile - Will begin reading at offset=4575529 for file=

On splunk server in splunkd.log

07-15-2013 14:36:05.672 -0400 INFO BatchReader - Removed from queue file

I am not sure why I dont see logs in indexer. Not sure what I might be missing?

Here are the files:

/opt/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/inputs.conf

[monitor:///log1/log2/log3]
sourcetype = syslog
index = syslog
disabled = false
crcSalt =
ignoreOlderThan = 1d
host_segment = 4

/opt/splunkforwarder/etc/system/local/outputs.conf

[tcpout]
defaultGroup=syslog_index
disabled = false

Forward the internal indexes as well as the non-internal ones

forwardedindex.0.whitelist = .*
forwardedindex.1.whitelist = _.*

[tcpout:syslog_index]
server=splunkserver:9997

0 Karma
1 Solution

bhavya_shah
Path Finder

The issue has been resolved.

Thanks bmacias84!

Resolution:

[monitor:///log1/log2/log3]
sourcetype = syslog
index = syslog
disabled = false
crcSalt =
ignoreOlderThan = 1d
host_segment = 4

If you are using index = syslog like in my case then make sure to edit the index.conf on splunk indexer to add it or just use index = default. And everything will work like a charm.

View solution in original post

lalit_mohan
Path Finder

Hi Guys,

I have two instances on microsoft azure environment one is splunk-server and other is splunk-forwarder(universalForwarder). Everything is fine with configuration ,then I tried to monitor tomcat logs and I have perform below steps on forwarder.

/usr/share/splunk_setup/splunkforwarder/bin/splunk add monitor /usr/share/apache-tomcat-7.0.42/logs/catalina.out -index default -sourcetype log4j -hostname splunkforwarder

But in search tab of splunk-web I always get No results found. search-query: host=splunkforwarder sourcetype=log4j

I checked inputs.conf ,CLI is not writing anything .So now I decided to write manually in these file.

Please tell me ,what I need to enter in my forwarders's inputs.conf and outputs.conf?

Thanks in advance!!

0 Karma

bhavya_shah
Path Finder

The issue has been resolved.

Thanks bmacias84!

Resolution:

[monitor:///log1/log2/log3]
sourcetype = syslog
index = syslog
disabled = false
crcSalt =
ignoreOlderThan = 1d
host_segment = 4

If you are using index = syslog like in my case then make sure to edit the index.conf on splunk indexer to add it or just use index = default. And everything will work like a charm.

bhavya_shah
Path Finder

Thank you so much for helping me out. I really appreciate your help.

It worked like a charm.

0 Karma

bmacias84
Champion

indexes.conf

bhavya_shah
Path Finder

Can you tell me which fine I need to modify on indexer?

0 Karma

bmacias84
Champion

If you are defining index = syslog for your input on your UF you need to have a index called syslog on your indexer.

bhavya_shah
Path Finder

So you only have one tcpout configured on your UF?
Yes.

have you defined an index called syslog on your Indexer?
No.

On your indexer do you see any within your metrics.log regarding data being sent from your UF?

Sometime it shows and then its gone.

Is this the only input defined on your UF?

Yes

0 Karma

bmacias84
Champion

So you only have one tcpout configured on your UF? have you defined an index called syslog on your Indexer? On your indexer do you see any within your metrics.log regarding data being sent from your UF? Is this the only input defined on your UF?

bhavya_shah
Path Finder

I have updated the info.

0 Karma

bmacias84
Champion

If you can provide your outputs.conf and inputs.conf from your UF.

0 Karma

srioux
Communicator

Do you have anything defined as part of inputs on your forwarder? You can verify by doing:

  $SPLUNK_HOME/bin/splunk list monitor

If you want the internal logs forwarded in, you may have to explicitly allow them in through outputs.conf configuration:

[tcpout]
defaultGroup = GroupName
disabled = false
# Forward the internal indexes as well as the non-internal ones
forwardedindex.0.whitelist = .*
forwardedindex.1.whitelist = _.*
0 Karma

bhavya_shah
Path Finder

Yes.

Here is the output:
Receiving is enabled on port 9997.

0 Karma

srioux
Communicator

Did you set up your indexer to listen for incoming data, on the port you've defined in outputs.conf? Should be able to use the following command to see:

$SPLUNK_HOME/bin/splunk display listen

0 Karma

bhavya_shah
Path Finder

Yes I have defined stanza in inputs.conf file. Even after adding your configuration in output.conf file I still dont the logs coming in. Just not sure why?

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...