Alerting

Search help please

csatech245
Engager

Hello!  I'm trying to set an alert that let's me know if tasks in a specific queue pass a specific duration.  The search has been giving me issues.  I tried a transaction line, but I don't have a endswith.  Does anyone know how to run a search like this?

I'm trying something like:

earliest=-30d@d index=[DATA] sourcetype=incident_history incident_type=[SPECIFIC QUEUE] event_type=[SPECIFIC ACTION (LIKE A TASK ON HOLD)]
| transaction incident_id when startswith=[SPECIFIC ACTION (LIKE A TASK ON HOLD)] endswith= > 72h
| table incident_id, duration
| sort - duration

It's not a transaction, but the only thing I could thing of.  What would be a search command forwhen an incident_id has been in a specific queue past a specific duration?

Any help would be appreciated.

Labels (1)
0 Karma
1 Solution

venkatasri
SplunkTrust
SplunkTrust

Hi @csatech245 

This first query gives you the duration_in_seconds 

 

earliest=-30d@d latest=now index=[DATA] sourcetype=incident_history incident_type=[SPECIFIC QUEUE] event_type=[SPECIFIC ACTION (LIKE A TASK ON HOLD)]
| stats range(_time) as duration_in_seconds by incident_id

 

second query which provides all queues and respective time for each incident_id. Assumed your incident_type is having your queue details. 

 

earliest=-30d@d latest=now index=[DATA] sourcetype=incident_history  event_type=[SPECIFIC ACTION (LIKE A TASK ON HOLD)]
| stats range(_time) as duration_in_seconds by incident_id, incident_type

 

---

An upvote would be appreciated and Accept solution if this reply helps!

 

View solution in original post

0 Karma

venkatasri
SplunkTrust
SplunkTrust

Hi @csatech245 

This first query gives you the duration_in_seconds 

 

earliest=-30d@d latest=now index=[DATA] sourcetype=incident_history incident_type=[SPECIFIC QUEUE] event_type=[SPECIFIC ACTION (LIKE A TASK ON HOLD)]
| stats range(_time) as duration_in_seconds by incident_id

 

second query which provides all queues and respective time for each incident_id. Assumed your incident_type is having your queue details. 

 

earliest=-30d@d latest=now index=[DATA] sourcetype=incident_history  event_type=[SPECIFIC ACTION (LIKE A TASK ON HOLD)]
| stats range(_time) as duration_in_seconds by incident_id, incident_type

 

---

An upvote would be appreciated and Accept solution if this reply helps!

 

0 Karma

csatech245
Engager

Thank you very much!  Your second example works perfect for every incident_id of all queues... this is even better.  I appreciate your help!

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Can you share some sample anonymised events? Also, detailing which fields you already have extracted?

0 Karma

csatech245
Engager

I apologize, I'm not sure what you're asking for.  Thank you for reaching out to a Splunk noob and offering assistance.

0 Karma
Get Updates on the Splunk Community!

Updated Team Landing Page in Splunk Observability

We’re making some changes to the team landing page in Splunk Observability, based on your feedback. The ...

New! Splunk Observability Search Enhancements for Splunk APM Services/Traces and ...

Regardless of where you are in Splunk Observability, you can search for relevant APM targets including service ...

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...