Getting Data In

How to troubleshoot why Splunk isn't reading and indexing incoming syslog messages?

ozirus
Path Finder

Hi,

I'm trying to read and index messages that come from a Juniper Pulse device using syslog protocol. I used the "Data Input" menu and add 10520/UDP as input port and bind it to a new index.

When I listen to port using tcpdump, I can see the messages from console, however, Splunk can't see and index the incoming data. I tried different sourcetypes like syslog, __singleline etc...

When I run netstat -tunalp | grep 10520, I could see that Splunk is listening on udp port 10520.

How can I debug this situation? What's your advice?

0 Karma

diogofgm
SplunkTrust
SplunkTrust

have you tried to use TCP input instead?

------------
Hope I was able to help you. If so, some karma would be appreciated.
0 Karma

ozirus
Path Finder

It also doesn't work

0 Karma

masonmorales
Influencer

Does the data show up in the index if you search All Time?

0 Karma

ozirus
Path Finder

No. There is no data in any way.

0 Karma

jtrucks
Splunk Employee
Splunk Employee

Start with http://docs.splunk.com/Documentation/Splunk/6.4.1/Troubleshooting/Cantfinddata and https://answers.splunk.com/answers/221885/how-to-troubleshoot-why-i-can-see-network-traffic.html just in case something there helps.

I would try a combination of the splunkd logs and using strace on the Splunk process. Also, enable debug and sifting through the results may be useful.

--
Jesse Trucks
Minister of Magic
0 Karma

craigv_splunk
Splunk Employee
Splunk Employee

My recommendation is to take a sample of the data and put it into a file on your local machine. Then go to add data in the Splunk GUI and upload from your local machine. You will then be brought to a screen where it tries to determine a sourcetype. You can play around with different sourcetype settings. When you try one like syslog for example make sure that linebreaking is happening as you'd expect and the a timestamp is extracted from the data.

The other thing to check would be to look at the splunkd.log in index=_internal to check for errors. That could give you a more specific idea of what might be wrong.

0 Karma

ozirus
Path Finder

How can I achive this data-import for syslog? tcpdump gives messy ASCII data when I listen syslog port. Any suggestion?

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...