- Splunk Answers
- :
- Using Splunk
- :
- Alerting
- :
- Alert condition with dynamic variables
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi, I am having trouble in create a condition for an alert that I would like. I have just started using Splunk and I do not know all the fancy search conditions.
So I have indexed results for all our projects that contain information like the number of unit tests failed. Sample events being :
| Project: A | FailedTests: 0| Date: 12062010 | ...
| Project: B | FailedTests: 3| Date: 11042010 | ...
I would like an alert whenever the number of tests failed for a certain project is greater then that of the last value for the number of tests failed. Is there any custom condition that I could use to do this? Thanks in advance.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I'll assume that you've already extracted the relevant fields and have good time extraction in place. You can achieve this by either using streamstats to compute the differences between subsequent runs for each project or by dedup and stats to find the most recent two runs for each project. I'll provide the dedup solution since it seems a bit simpler to me.
... | dedup 2 Project | stats first(FailedTests) as current_failed last(FailedTests) as previous_failed by Project | where current_failed > previous_failed
Provided that the search is run over a long enough time range, it will find all projects where the most recent number of failures is more than the previous recorded number of failures. You can then configure your alert to trigger when the number of events is greater than zero.
The biggest problem here is that the alert will keep on firing until we have the same or fewer failures for all projects. Perhaps that's desirable. If not, you could use a lookup table to store the previous number of failures per Project. Then you'll have exactly one line in an alert per increase. A search like this will work:
... | stats first(FailedTests) as current_failed by Project
| lookup failure_count.csv Project OUTPUT current_failed as past_failed
| outputlookup failure_count.csv
| where current_failed > past_failed
Here we compute the most recent failure count by project, then we look up the previous failure count (called current_failed in the lookup), then we save our revised table and finally filter out those projects that have a past failure and a higher FailedTests than before.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I'll assume that you've already extracted the relevant fields and have good time extraction in place. You can achieve this by either using streamstats to compute the differences between subsequent runs for each project or by dedup and stats to find the most recent two runs for each project. I'll provide the dedup solution since it seems a bit simpler to me.
... | dedup 2 Project | stats first(FailedTests) as current_failed last(FailedTests) as previous_failed by Project | where current_failed > previous_failed
Provided that the search is run over a long enough time range, it will find all projects where the most recent number of failures is more than the previous recorded number of failures. You can then configure your alert to trigger when the number of events is greater than zero.
The biggest problem here is that the alert will keep on firing until we have the same or fewer failures for all projects. Perhaps that's desirable. If not, you could use a lookup table to store the previous number of failures per Project. Then you'll have exactly one line in an alert per increase. A search like this will work:
... | stats first(FailedTests) as current_failed by Project
| lookup failure_count.csv Project OUTPUT current_failed as past_failed
| outputlookup failure_count.csv
| where current_failed > past_failed
Here we compute the most recent failure count by project, then we look up the previous failure count (called current_failed in the lookup), then we save our revised table and finally filter out those projects that have a past failure and a higher FailedTests than before.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Our build server has been down for a while so I haven't had the time to fully test this until now. And it worked perfectly, exactly what we needed, thank you so much.
Introducing Splunk Enterprise 9.2
Adoption of RUM and APM at Splunk
Routing logs with Splunk OTel Collector for Kubernetes
