Alerting

Setting alerts on exceptions

usmanchaudhri
Engager

Hi,

I potentially want to set a scheduled search - where i specify the list of exceptions in the search - and if there is any new exception outside of those listed exceptions, Splunk should send an email alert.

For example: Consider, here is my list of exceptions: "error: null pointer exception (login.class:1494)" "error: database down exception (database.class:1594)" "error: read PFD (readPDF.class:1694)"

Now, whenever there is a new exception generated (outside of those listed above), Splunk sends me alert.

Thanks for looking into this. Usman Chaudhri

Tags (1)
1 Solution

David
Splunk Employee
Splunk Employee

The way I did this was to set up eventtypes.conf so that each event has an eventtype. You can then run a query:

YourSearch NOT eventtype=*

And that will show all the events that aren't on your pre-defined list. You can toss that in a scheduled search no problem.

http://www.splunk.com/base/Documentation/latest/Admin/eventtypesconf

View solution in original post

0 Karma

David
Splunk Employee
Splunk Employee

The way I did this was to set up eventtypes.conf so that each event has an eventtype. You can then run a query:

YourSearch NOT eventtype=*

And that will show all the events that aren't on your pre-defined list. You can toss that in a scheduled search no problem.

http://www.splunk.com/base/Documentation/latest/Admin/eventtypesconf

0 Karma

David
Splunk Employee
Splunk Employee

That sounds like a good solution. For my use case, we had a relatively small number of events (45 or so) with fairly unique descriptors. Having a specific event type to each error message also allows me to toss up a dashboard with the daily average over the last month for each event type, compared with the last 24 hours. That way I can catch changes in known errors. Food for thought 😉

0 Karma

usmanchaudhri
Engager

Yeah, that's what i ended up doing. I saved a search as an eventtype, the search had the pre-defined list of events. Than i went ahead and scheduled another search and just specified eventtype!=< predefine list >. This gave me list of new events.

Thank you
Usman Chaudhri

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Meet Splunk Observability Studio: AI-Assisted OpenTelemetry Instrumentation Without ...

Instrumentation is usually the last step or even an afterthought when building out a project. The feature ...

Federated Search for Cisco Security and Analytics Logging (SAL) is now GA on Splunk ...

Federated Search for Cisco  Security Analytics and Logging (SAL) is now generally available as part of the ...

Your Path to AgenticOps: AI Experiences for Every Splunk Practitioner

Your Path to AgenticOps: AI Experiences for Every Splunk Practitioner   Join us for a demo-driven look at how ...