How do I check to see if a couple of Hosts are sending data to Splunk Enterprise. They both are VMs.
Hi
here is collection of links where you could find some ways to check this.
There are a lot of options for finding hosts or sources that stop submitting events:
Meta Woot! https://splunkbase.splunk.com/app/2949/
TrackMe https://splunkbase.splunk.com/app/4621/
Broken Hosts App for Splunk https://splunkbase.splunk.com/app/3247/
Alerts for Splunk Admins ("ForwarderLevel" alerts) https://splunkbase.splunk.com/app/3796/
Monitoring Console https://docs.splunk.com/Documentation/Splunk/latest/DMC/Configureforwardermonitoring
Deployment Server https://docs.splunk.com/Documentation/DepMon/latest/DeployDepMon/Troubleshootyourdeployment#Forwarde...Some helpful posts:
https://lantern.splunk.com/hc/en-us/articles/360048503294-Hosts-logging-data-in-a-certain-timeframe
https://www.duanewaddle.com/proving-a-negative/
Some ideas:
1) Search Splunk for data from those hosts
2) Search Splunk's _internal index for events from forwarders on those hosts, especially TcpInputProc.
3) Use netstat or a similar tool on the hosts to see if connections are established with Splunk.
4) Make sure the forwarder on each host is running.
Rich, I have a a few Forwarders that are missing. I don't know what data comes from them due to them being out of state. How do I know if they are alive? Also is there a way to ping them from Splunk to see if they talk back? Thank u again
If you don't see anything from those hosts in _internal then chances are they are not alive, but it's possible they just can't connect to the indexers.
Communication between Splunk and its forwarders is initiated by the forwarders. There is no built-in way for Splunk to ping a server (Enterprise Security provides a way, if you have that installed). Even then, ping only tells you the server is running and not if the forwarder is up or not.