Getting Data In

Local udp:514 input not forwarded

hethu
Path Finder

Hi,

I have 2 heavy forwarders set up; F1 is forwarding to F2, and F2 forwards to splunk cloud.

On F1 i have set up a local input to listening on UDP:514 for events, this works great and forwards to cloud.
On F2 i have set up a local input for UDP:514 exactly like i did on F1, but no events are forwarded, does anyone here have a clue to what could be wrong?

The events are of the same type, so as long as this works on F1 it should not be an issue with interpreting/reading the events.

I have checked the FW and the events are beeing received, and also after setting UDP processor log level to debug i get this in my splunkd.log on F2:

 

02-01-2021 12:54:00.520 +0100 DEBUG UDPInputProcessor - callback()
02-01-2021 12:54:10.512 +0100 DEBUG UDPInputProcessor - callback()
02-01-2021 12:54:18.502 +0100 INFO  TcpOutputProc - Found currently active indexer. Connected to idx=ForwarderIP:30132, reuse=1.
02-01-2021 12:54:20.467 +0100 DEBUG UDPInputProcessor - Generating UDP metrics
02-01-2021 12:54:20.467 +0100 DEBUG UDPInputProcessor - callback()
02-01-2021 12:54:30.514 +0100 DEBUG UDPInputProcessor - callback()
02-01-2021 12:54:34.790 +0100 DEBUG UDPInputProcessor - event=data from="PC100.Local (new)" status=accepted
02-01-2021 12:54:34.790 +0100 DEBUG UDPInputProcessor - UDPInputProcessor::when_events called
02-01-2021 12:54:34.801 +0100 DEBUG UDPInputProcessor - event=data from=PC100.Local status=accepted
02-01-2021 12:54:34.801 +0100 DEBUG UDPInputProcessor - UDPInputProcessor::when_events called
02-01-2021 12:54:34.812 +0100 DEBUG UDPInputProcessor - event=data from=PC100.Local status=accepted
02-01-2021 12:54:34.812 +0100 DEBUG UDPInputProcessor - UDPInputProcessor::when_events called
02-01-2021 12:54:34.830 +0100 DEBUG UDPInputProcessor - event=data from=PC100.Local status=accepted
02-01-2021 12:54:34.831 +0100 DEBUG UDPInputProcessor - UDPInputProcessor::when_events called
02-01-2021 12:54:44.829 +0100 DEBUG UDPInputProcessor - callback()
02-01-2021 12:54:44.829 +0100 DEBUG UDPInputProcessor - event=sendDoneKey source=PC100.Local localport=514
02-01-2021 12:54:44.829 +0100 DEBUG UDPInputProcessor - event=deleteSource source=PC100.Local localport=514
02-01-2021 12:54:48.413 +0100 INFO  TcpOutputProc - Found currently active indexer. Connected to idx=ForwarderIP:30132, reuse=1.
02-01-2021 12:54:50.471 +0100 DEBUG UDPInputProcessor - Generating UDP metrics
02-01-2021 12:54:50.471 +0100 DEBUG UDPInputProcessor - callback()

 

 I have had to replace some hostnames as you probably can see. Hopefully someone here can help me figure this out.

Labels (2)
Tags (2)
0 Karma
1 Solution

hethu
Path Finder

It seems the input i set up through the web interface, did not change the active inputs.conf.... after i manually altered this config file, the forwarder correctly received and forwarded my events.

View solution in original post

0 Karma

hethu
Path Finder

It seems the input i set up through the web interface, did not change the active inputs.conf.... after i manually altered this config file, the forwarder correctly received and forwarded my events.

0 Karma

richgalloway
SplunkTrust
SplunkTrust
To help future readers, please describe the manual changes you had to make.
---
If this reply helps you, Karma would be appreciated.
0 Karma

richgalloway
SplunkTrust
SplunkTrust

Heavy forwarder F2 should be listening on port 9997 for the data from F1.

The use of intermediate forwarders like F2 is discouraged.  Forwarders should send data directly to indexers.  Having another forwarder in the path can lead to unbalanced data on the indexers, can be a bottleneck, and is an extra layer to manage and troubleshoot.

---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

Splunk Forwarders and Forced Time Based Load Balancing

Splunk customers use universal forwarders to collect and send data to Splunk. A universal forwarder can send ...

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...