Getting Data In

UDP syslog not showing up on windows splunk server

digihax
New Member

I've set up Snare on remote servers to forward syslog events on port 6161 to my Splunk server.

I've run wireshark on my Splunk server to ensure the events are arriving -- they are.

I've added a UDP data source on port 6161 to accept the syslog messages.

After 24 hours, there's nothing showing up, even though I've confirmed they are still arriving via wireshark.

Restarting the service has not resolved either. Ideas?

Tags (3)
0 Karma

southeringtonp
Motivator

A few more things to try:

  • Double-check your inputs configuration, and see if you are overriding `source` or `sourcetype` -- if you are, modify the searches suggested by Simeon accordingly.

  • Broaden your searches and try to match on raw text instead of source or sourcetype to see if those fields have something different than expected. Pick a string of text that you know appears in your Windows logs, and search on that: `index=* SomeRawText`

  • Run `netstat -p udp -b` to verify that the Splunk daemon is actually the process bound to that port.

  • If you suspect a firewall issue, try stopping Splunk and installing Kiwi Syslog or similar on that port, just long enough to verify that it can receive syslog messages. That will help narrow down the problem to either network/host configuration or Splunk configuration.

0 Karma

dwaddle
SplunkTrust
SplunkTrust

Check your iptables rules. Wireshark is great, but its capture modules plug into the network stack below iptables -- so wireshark will see packets arrive at the interface that iptables could deny/drop after the fact.

0 Karma

digihax
New Member

Sorry, it was supposed to say "the local firewall is off". There is no firewall service running on the system.

Just for fun, I enabled the FW, added a UDP 6161 allow rule, and re-disabled it.

0 Karma

dwaddle
SplunkTrust
SplunkTrust

The same advice for iptables rules applies to the Windows firewall as well.

If it's really UDP you can't telnet to it. Telnet does not work with UDP, it uses TCP.

digihax
New Member

It's a Windows system, so no iptables. The local is off. I can telnet to the port from a remote system.

0 Karma

Simeon
Splunk Employee
Splunk Employee

It sounds like the events are not being identified with the right search, or the input is not working correctly. I recommend:

1 - verify that you can locally send udp traffic to port 6161. On unix systems, using netcat or similar is a good test

2 - Run a search for

"index=* source=udp:6161" over all time

3 - Run a metrics search in the internal index to see if you have data incoming from that port. For example:

"index=_internal source=*metrics.log* per_source_thruput series=udp:6161"   
0 Karma

digihax
New Member

The port is open/listening/accepting traffic.

Neither 2, nor 3, show data.

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...