I'm referencing this post - https://answers.splunk.com/answers/321226/how-to-create-an-alert-to-notify-me-via-email-when.html
I want to alert on pool violations, and gives me the correct info, but I'm not sure where to put the threshold check. I've put the "where" clause in a number of places, and each time, it returns nothing.
Also, is there a way to trigger a search when this happens and then send an email from the results?
@mmodestino_splunk, I've compared your search with the one in the DMC for the total license. This one seems to take into account you can have multiple license masters (splunk_server in the search). It also uses a join which is a bit more explicit than a "|search " in my opinion. So I've rewritten partly your search and the result is below.
However, in both cases, you consider a license group can only contain 1 single stack (by renaming stack_ids to stack_id), is it really the case ?
Here's my search :
| rest splunk_server=local /services/licenser/pools | rename title AS pool | join type=outer splunk_server stack_id [ rest splunk_server=local /services/licenser/groups | eval stack_id=stack_ids | fields stack_id splunk_server is_active] | search is_active=1 | eval quota=if(isnull(effective_quota),quota,effective_quota) | eval "% used"=round(used_bytes/quota*100,2) | fields pool "% used" | where '% used' >= 0
The where clause goes at the end, but you gotta be careful with fields with spaces...i used single quotes to get it to work.
| rest splunk_server=local /services/licenser/pools | rename title AS Pool | search [ rest splunk_server=local /services/licenser/groups | search is_active=1 | eval stack_id=stack_ids | fields stack_id] | eval quota=if(isnull(effective_quota),quota,effective_quota) | eval "% used"=round(used_bytes/quota*100,2) | fields Pool "% used" | where '% used' > .2
To send an email, hit Save AS > Alert and configure it run on a schedule and to trigger an email action when number of results is greater than 0 (and list in triggered alerts for verification/troubleshooting)
Also note @hexx answer that an alert for overall usage exists in the management console and can be triggered as an email alert as well.
Yes. This alert will indicate a problem, and the customer is going to want some data to do research, so I want to then run a report giving him the needed data. Actually, something missing from the product - should be able to request an action that just points to another saved search and run it as an action. (At least, IMHO...)
A couple of options here: