Getting Data In

Why is syslog data not getting indexed with my current inputs.conf on the forwarder?

athorat
Communicator

We are trying to Index data from syslog and have the following configuration in the inputs.conf on the forwarder.

[tcp://IP:10514]
index = NWK
disabled = 0
#followTail = 0
sourcetype = NW:PROD:SYSLOG

From the search head, when we use index=NWK, it does not show any data.
What can be checked to see where the connection is breaking?

Thanks,
Anil.

0 Karma

lguinn2
Legend

On the forwarder, take a look at

SPLUNK_HOME/var/log/splunk/splunkd.log

This will show you if there are any error messages regarding the inputs.conf, outputs.conf etc.
Connection failures usually appear here.

Also, is there an index named NWK on the indexer?

0 Karma

athorat
Communicator

Ok,I see this error in the log files

and thats the source ip address.
Which is up and running and have the source files.
Able to resolve the IP.
Also tried using the fQDN in the inputs.conf instead of the ip no luck.

08-14-2015 11:32:59.636 -0700 INFO TcpInputConfig - No matching config for 172.26.95.7
08-14-2015 11:32:59.636 -0700 WARN TcpInputProc - Could not find matching host.
08-14-2015 11:33:11.126 -0700 INFO TcpInputConfig - No matching config for 172.26.95.7
08-14-2015 11:33:11.126 -0700 WARN TcpInputProc - Could not find matching host.

How do I identify if Splunk is not able to get connect and fetch the data or is it something else.

0 Karma

diogofgm
SplunkTrust
SplunkTrust

Is the source sending data to port 10514?
I had some issues before on network devices because some devices didn't allow me the change the port for syslog from514 and if I was are running splunk with other user than root I couldnt create inputs for port 514

------------
Hope I was able to help you. If so, some karma would be appreciated.
0 Karma

athorat
Communicator

Yes there is a index named NWK on the indexer
And also it shows up in inex=NWK which is coming from some remote_searches.log file.
I will check from the log file in the mean time.
Thanks.

0 Karma

cramasta
Builder

Is the syslog data being generated from the same host thats running the forwarder?
If it is I would suggest setting it up like this and testing.

[tcp://:10514]
connection_host = none
index=NWK
sourcetype = NW:PROD:SYSLOG

0 Karma

athorat
Communicator

Its coming from a different server not from the heavy forwarder.

0 Karma

diogofgm
SplunkTrust
SplunkTrust

Is that the only data source in the forwarder?
If not, are the other working ok?
If it is what's your outputs.conf like?

------------
Hope I was able to help you. If so, some karma would be appreciated.
0 Karma

athorat
Communicator

No there are multiple data sources which are working using this forwarder.
outputs.conf contains server = Servernanme:9997 and some other config (is there anything which I should be looking here)

0 Karma

diogofgm
SplunkTrust
SplunkTrust

If the others sources are ok I would look at the origin of the data. Or any connection between the forwarder and the IP:10514

------------
Hope I was able to help you. If so, some karma would be appreciated.
0 Karma

athorat
Communicator

Yes makes sense, though what do we look at on the origin of Data.
Can you shed some light on it.

0 Karma
Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Get the T-shirt to Prove You Survived Splunk University Bootcamp

As if Splunk University, in Las Vegas, in-person, with three days of bootcamps and labs weren’t enough, now ...

Wondering How to Build Resiliency in the Cloud?

IT leaders are choosing Splunk Cloud as an ideal cloud transformation platform to drive business resilience,  ...