Splunk Enterprise

Triggered Alerts search producing a count of zero,but i see the results when I run the search manually.Why is it?

Ashwini008
Builder

I have simple alert setup and it is supposed to trigger when there are no events. But the alert is getting triggered even when i can see the results when run the search manually in search App.

But when i inspect the job and run the search from under Activity>Triggered Alerts i do not see any events.

How do i rectify this issue? 

I am searching the query for earliest=-11m in the alert query and my cron schedule is 4,14,24,34,44,54 * * * *

Alert is created by other USER ,who has same privilege's as mine i.e. Admin

Tags (2)
0 Karma

aasabatini
Motivator

Hi,

to have an accurate analysis  we need to check the search behind the alert.

I can suggest you to check if the permissions fields are correctly set or if you use an lookup on the search please check the permissions also.

I appreciate if you can share more details to help😀

Alessandro

“The answer is out there, Neo, and it’s looking for you, and it will find you if you want it to.”

Ashwini008
Builder

@aasabatini  This is my alert query.

index=os host="p05666z" earliest=-11m source=ps | search *router* | dedup COMMAND ARGS host | table _time app pid.

 

When i inspected one of the triggered job i found the following warning message.

  • Unable to distribute to peer named ***** at uri https://***.**.****.**:***** because replication was unsuccessful. replicationStatus Failed failure info: failed_because_HTTP_REPLY_READ_FAILURE Please verify connectivity to the search peer, that the search peer is up, and an adequate level of system resources are available. See the Troubleshooting Manual for more information.

 

0 Karma

aasabatini
Motivator

Hi 

 

What is the action associated at this alert?what do you expect after this alert?

Example: mail or summaries the event on other index, other actions...

 

, also try to put a condition on your alert
example:
| where (conditionexample source=ps)

 

 

“The answer is out there, Neo, and it’s looking for you, and it will find you if you want it to.”
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...