Local udp:514 input not forwarded

hethu — Mon, 01 Feb 2021 12:19:42 GMT

Hi,

I have 2 heavy forwarders set up; F1 is forwarding to F2, and F2 forwards to splunk cloud.

On F1 i have set up a local input to listening on UDP:514 for events, this works great and forwards to cloud.
On F2 i have set up a local input for UDP:514 exactly like i did on F1, but no events are forwarded, does anyone here have a clue to what could be wrong?

The events are of the same type, so as long as this works on F1 it should not be an issue with interpreting/reading the events.

I have checked the FW and the events are beeing received, and also after setting UDP processor log level to debug i get this in my splunkd.log on F2:

02-01-2021 12:54:00.520 +0100 DEBUG UDPInputProcessor - callback() 02-01-2021 12:54:10.512 +0100 DEBUG UDPInputProcessor - callback() 02-01-2021 12:54:18.502 +0100 INFO TcpOutputProc - Found currently active indexer. Connected to idx=ForwarderIP:30132, reuse=1. 02-01-2021 12:54:20.467 +0100 DEBUG UDPInputProcessor - Generating UDP metrics 02-01-2021 12:54:20.467 +0100 DEBUG UDPInputProcessor - callback() 02-01-2021 12:54:30.514 +0100 DEBUG UDPInputProcessor - callback() 02-01-2021 12:54:34.790 +0100 DEBUG UDPInputProcessor - event=data from="PC100.Local (new)" status=accepted 02-01-2021 12:54:34.790 +0100 DEBUG UDPInputProcessor - UDPInputProcessor::when_events called 02-01-2021 12:54:34.801 +0100 DEBUG UDPInputProcessor - event=data from=PC100.Local status=accepted 02-01-2021 12:54:34.801 +0100 DEBUG UDPInputProcessor - UDPInputProcessor::when_events called 02-01-2021 12:54:34.812 +0100 DEBUG UDPInputProcessor - event=data from=PC100.Local status=accepted 02-01-2021 12:54:34.812 +0100 DEBUG UDPInputProcessor - UDPInputProcessor::when_events called 02-01-2021 12:54:34.830 +0100 DEBUG UDPInputProcessor - event=data from=PC100.Local status=accepted 02-01-2021 12:54:34.831 +0100 DEBUG UDPInputProcessor - UDPInputProcessor::when_events called 02-01-2021 12:54:44.829 +0100 DEBUG UDPInputProcessor - callback() 02-01-2021 12:54:44.829 +0100 DEBUG UDPInputProcessor - event=sendDoneKey source=PC100.Local localport=514 02-01-2021 12:54:44.829 +0100 DEBUG UDPInputProcessor - event=deleteSource source=PC100.Local localport=514 02-01-2021 12:54:48.413 +0100 INFO TcpOutputProc - Found currently active indexer. Connected to idx=ForwarderIP:30132, reuse=1. 02-01-2021 12:54:50.471 +0100 DEBUG UDPInputProcessor - Generating UDP metrics 02-01-2021 12:54:50.471 +0100 DEBUG UDPInputProcessor - callback()

I have had to replace some hostnames as you probably can see. Hopefully someone here can help me figure this out.

Re: Local udp:514 input not forwarded

richgalloway — Mon, 01 Feb 2021 14:15:52 GMT

Heavy forwarder F2 should be listening on port 9997 for the data from F1.

The use of intermediate forwarders like F2 is discouraged. Forwarders should send data directly to indexers. Having another forwarder in the path can lead to unbalanced data on the indexers, can be a bottleneck, and is an extra layer to manage and troubleshoot.

Re: Local udp:514 input not forwarded

hethu — Fri, 05 Feb 2021 09:37:43 GMT

It seems the input i set up through the web interface, did not change the active inputs.conf.... after i manually altered this config file, the forwarder correctly received and forwarded my events.

Re: Local udp:514 input not forwarded

richgalloway — Fri, 05 Feb 2021 13:58:10 GMT

To help future readers, please describe the manual changes you had to make.

topic Re: Local udp:514 input not forwarded in Getting Data In

Local udp:514 input not forwarded

Re: Local udp:514 input not forwarded

Re: Local udp:514 input not forwarded

Re: Local udp:514 input not forwarded