DMC Alert - Why is search peer not responding?

[inventsekar]

Hi, 

The DMC got an alert "DMC Alert - Search Peer Not Responding".. it works fine when a search peer goes down, but then when it come back, we should get an alert/notification saying "Search Peer up now", right, but, somehow the DMC/Splunk developers missed to consider this situation/condition. 

i can check the "DMC Alert - Search Peer Not Responding" alert's search query and modify it to create the opposite.. like "DMC Alert - Search Peer Responding Fine"

Now the question is...the "DMC Alert - Search Peer Responding Fine" alert should work only after the first SH down alert. hope you got this issue. Please suggest how we can achieve this, thanks. 

[gcusello]

Hi @inventsekar,

if you're not using ES you enable an additional action to your Search Peer Down of writing in a Summary or in  a lookup and use the content of (e.g. of the last hour) this Summary index to filter the Peer Up search, something like this:

| rest splunk_server=local /services/search/distributed/peers/ 
| search status="Up" disabled=0 [ search
    index=summary_triggered_alerts earliest=-1h@h latest=now 
    | fields peerName ]  
| fields peerName, status 
| rename peerName as Instance, status as Status

 Ciao.

Giuseppe