Syslog data inputs not creating new sources

[lmarcel]

I used to have 2 UDP syslog data inputs: UDP://514 going to the default index, UDP://515 going to a new index. They were successfully collecting syslog messages from network equipment. I noticed yesterday that they haven't indexed anything for a while. Disabling and re-enabling the data inputs briefly allowed some messages to be indexed but then it stopped again.

Today I deleted both data inputs and entered new ones.
Splunk\etc\apps\search\local\inputs.conf shows this:

[udp://514]
connection_host = ip
sourcetype = syslog
disabled = 0
index = network
source = Network

[udp://49200]
connection_host = ip
index = ecb
sourcetype = syslog
source = EmergencyCallBox

In the Search app, when I click on the Data Summary button, the Network and EmergencyCallBox sources I'm expecting are not listed, the syslog sourcetype is not updated. I still see udp:514 listed as a source.
I'm using Splunk 6.0 on Windows 2008 R2. Rebooting the OS and Splunk didn't help.

If I run index=_internal source=*metrics.log* per_source_thruput series=EmergencyCallBox I see results (series=Network also gives me results).

Do I have to manually create the source and sourcetype? If so, where?

Any suggestions are welcome.

[lguinn2 · Accepted solution]

You do not need to create the source or sourcetype - if you use them in inputs.conf on any indexer or forwarder, that's all you need to do.

I don't see a UPD://515 in your inputs.conf.

Also, you probably have more than one copy of inputs.conf in the Splunk\etc\ directory tree. There is a precedence of configuration files, so your settings in one inputs.conf may be overridden in another. You can read about it at Configuration File Precedence.

Next, changes to your configuration files are not retroactive - they only apply to new data as it is indexed. So older data may have a different source or sourcetype, even if you don't have a configuration precedence issue.

Finally, I would look carefully at your networking characteristics. Are the ports open, is Splunk able to access them, etc. Examining your splunkd.log may be helpful. Splunk actually indexes this log and many other internal logs, so you could search index=_internal source=*splunkd.log error OR warn*. And you could install the Splunk SOS app, which is free and analyzes the Splunk internal logs for you - it's pretty nice.

HTH

[lguinn2]

The Data Summary only shows the data in the indexes that you search by "default". You have permissions to access the ecb and network indexes but they are not listed as a "default" index for you. You can change this by editing the Access Controls in the Splunk Manager.

[lmarcel]

Thanks for your reply. It turns out to be a GUI problem (in my view) because if I just search for index=ecb or index=network, I can see all the data, properly indexed!

I am just used to go to the Search app, and click on the Data Summary button to select the source or host I want to look at. Apparently the Data Summary window is not updating for me. But the data is being received and indexed, that's all I wanted.