Is there a way to get an alert when (at the time of) a UF is considered missing?

[X_Kinkead]

Hello,

Is there a way to get an alert when (at the time of)  a UF is considered missing?  I don't mean a report of all missing UFs over all time, but when one of them goes offline recently?

In the Cloud Monitoring Console app, I see there is a screen for Forwarders:Deployment, so I copied the query for the Status & Configuration table with the hopes that might be a good jumping off point - here is my query:

| inputlookup sim_forwarder_assets
            | makemv delim=" " avg_tcp_kbps_sparkline
            | `sim_rename_forwarder_type(forwarder_type)` | search NOT [| inputlookup sim_assets | dedup serverName | rename serverName as hostname | fields hostname]
                        | `sim_time_format(last_connected)`
                        | fields hostname, forwarder_type, version, os, arch, status, last_connected
                        | search hostname="***"
                        | search status="*"
						| search last_connected < -20m@s
                        | rename hostname as host, forwarder_type as Type, version as Version, os as OS, arch as Architecture, status as Status, last_connected as "Last Connected to Indexers"

As I understand it the UF status is set to 'missing' after 15 minutes of inactivity. The above search is run in a short window of say the last 30 minutes.  

Is there perhaps a more direct way to get what I need?  Else is there a way to get the above to work?

Thanks for any advice!  

[X_Kinkead · Accepted solution]

Thanks - I ended up using this query:

| metadata type=hosts index=* 
| eval latestEventSeen=strftime(lastTime,"%x %X") 
| eval age_in_minutes=round((now()-lastTime)/60,0)
| search age_in_minutes > 5 
| search host="*"
| fields host latestEventSeen age_in_minutes

It seems to work as expected, after the throttling was configured.  

[sekhar463]

hai all, 

is any solution or how to create an alert for missing UF 

[richgalloway]

The solution to the OP's problem is within the green box.  If that doesn't solve your problem then submit a new question with details about your particular problem.

[richgalloway]

What doesn't work with the query you have?

Have you considered monitoring _internal for log messages from your UFs and triggering an alert when the count falls below a certain level?

Finding something that is not there is not Splunk's strong suit.  See this blog entry for a good write-up on it.

https://www.duanewaddle.com/proving-a-negative/