Getting Data In

Splunk monitor shows Missing forwarders

vdamiangf
Engager

Splunk monitor shows Missing forwarders:
universal forwarder 4.3.2
deployed on linux 64
over redhat-release-5Server-5.9.0.2.0.1
splunk Indexer version

stopped being active or sending logs to indexer suddenly. So far I have not idea why. Any one experienced this issue before?

What can I check to verify everything is working correctly?

lukejadamec
Super Champion

There are a number of things you can do. Here are some of them:

1) run a search for that host, and start with the last 15 minutes. Increase the time to see how long it has been off line.

If it populates with current data, then it automatically came back online. Sometimes forwarders will go offline when the system reboots, if it takes to long, but they come back by themselves. The missing forwarder message will go away in about 15 minutes.

2) Log onto the server with the down forwarder and check the status of the forwarder: service splunk status.

If the forwarder status is stopped, then restart it with service splunk restart.

If the forwarder fails to start, post the error message here.

0 Karma

valameti
Explorer

Hi
i have an issue in Deployment monitor app, even after removing the UFs forwarding to that particular environment it is still showing the UFs under missing forwarder Warnings.why it is showing so and can you please suggest e with the solution

Thanks in advance

0 Karma

dehtallyutedeh
Explorer

Good day,

Any solution for this issue?

0 Karma

akocak
Contributor

I am researching task, below search is nice from other Splunk answer:

index=_internal sourcetype=splunkd group=tcpin_connections NOT eventType=* 
| stats max(_time) as last_connected, sum(kb) as sum_kb by guid, hostname 
| addinfo 
| eval status = if(isnull(sum_kb) or (sum_kb <= 0) or (last_connected < (info_max_time - 900)), "missing", "active") 
| where status="missing" 
| convert ctime(info_max_time) ctime(info_min_time) ctime(info_search_time) ctime(last_connected)

However, missing part for me still, how would i determine the state switches ? like passive to active vs active to passive, then to back to active.
Currently, thinking of feeding a lookup table every 15 minutes and run my alert search against this new table.

0 Karma
Get Updates on the Splunk Community!

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

(view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...

Adoption of Infrastructure Monitoring at Splunk

  Splunk's Growth Engineering team showcases one of their first Splunk product adoption-Splunk Infrastructure ...