syslog data on udp port is showing as zero bytes in Splunk

[vikasverma1985]

Hi Splunk experts,

I ran into a strange situation in Splunk wherein udp connections but data size is zero.

Just to give you a background of what i am trying to do and how.

I am using Splunk in our organisation and currently working on creating a solution to integrate syslogs/event logs from network appliances (Cisco ASA, Fe big IP), Checkpoint etc) in Splunk.
Our setup includes, heavy forwarders which receives syslogs from network appliances and then HF sends that data to indexers. On HFs, we have redirected data coming on UDP 514 port to 1514 using iptables. Also, I have created inputs.conf file for all network appliances on the HF side. Example inputs.conf file is:

[udp://XX.YY.ZZ.AA:1514]
index=test
source=udp:514
sourcetype=qos_syslog
connection_host=ip
disabled=false

IP tables on HF side looks like below:

Table: filter
Chain INPUT (policy ACCEPT)
num target prot opt source destination
1 ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpt:1514
2 ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpt:2514
3 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:1514
4 ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpt:514
5 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:9997
6 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:8000
7 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:22
8 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:514
9 ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpt:2514

Chain FORWARD (policy ACCEPT)
num target prot opt source destination

Chain OUTPUT (policy ACCEPT)
num target prot opt source destination

Table: nat
Chain PREROUTING (policy ACCEPT)
num target prot opt source destination
1 REDIRECT tcp -- 0.0.0.0/0 0.0.0.0/0 tcp dpt:514 redir ports 1514
2 REDIRECT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpt:514 redir ports 1514

Chain POSTROUTING (policy ACCEPT)
num target prot opt source destination

Chain OUTPUT (policy ACCEPT)
num target prot opt source destination

When i am running an search on Search head (or checking the HF logs in HF), I can see that in "metrics.log", there are entries wherein it says that "group=udpin_connections, XX.YY.ZZ.AA:1514, sourcePort=1514, _udp_bps=0.00, _udp_kbps=0.00, _udp_avg_thruput=0.00, _udp_kprocessed=0.00, _udp_eps=0.00" where XX.YY.ZZ.AA is the IP address if the network appliance.

Now, my questions are:

1. Why I am getting all udp data as zero as per metrics logs? I have checked the index as well and I cannot see anything in it.
2. Does that mean that connection with the network appliance is established (because i can see entry) in the metrics log? Does it rule out the possibility of any firewall blocks in between HF and network appliance?

Any help, guidance, suggestions on this issue is very much appreciated. I am running out of options now and the problem is still there. 😞

Regards
Vikas

[Priyankakumari1]

Did you find the answer for this?