I have cluster of indexers i1, i2 and i3 and not seeing any data coming from universal forwarder f1 to custom index network. I can see index=_internal host="f1" on search head sh but nothing in network index.
I am filling up file random.log on f1
[ec2-user@f1 log]$ sudo /opt/splunkforwarder/bin/splunk btool inputs list monitor:///var/log/*.log [monitor:///var/log/*.log] _rcvbuf = 1572864 disabled = 0 host = $decideOnStartup index = network
[ec2-user@f1 log]$ cat /var/log/random.log Success 655 Error 78
Forwarder seems connected to Indexers
[ec2-user@f1 log]$ sudo tail -f /opt/splunkforwarder/var/log/splunk/splunkd.log 09-14-2022 12:59:15.389 +0000 INFO AutoLoadBalancedConnectionStrategy [2938 TcpOutEloop] - Connected to idx=10.0.7.4:9997, pset=0, reuse=0. using ACK. 09-14-2022 12:59:45.300 +0000 INFO AutoLoadBalancedConnectionStrategy [2938 TcpOutEloop] - Connected to idx=10.0.7.2:9997, pset=0, reuse=0. using ACK. ^C
[ec2-user@f1 log]$ sudo /opt/splunkforwarder/bin/splunk list forward-server Active forwards: 10.0.7.2:9997 10.0.7.4:9997 Configured but inactive forwards: 10.0.7.3:9997
This is how it looks on one of indexers
[ec2-user@i1 ~]$ sudo /opt/splunk/bin/splunk list index | grep network network /opt/splunk/etc/network/db /opt/splunk/etc/network/colddb /opt/splunk/etc/network/thaweddb [ec2-user@i1 ~]$ sudo ls -l /opt/splunk/etc/network/db total 4 -rw------- 1 splunk splunk 10 Sep 14 11:45 CreationTime drwx--x--- 2 splunk splunk 6 Sep 14 11:45 GlobalMetaData
... View more