Alerting

How to write a query to find when certain log pattern are not present in the raw logs?

Hemnaath
Motivator

Hi All, I have requirement to create an alert. 

Condition:

In the raw data when certain log Pattern are not found then it should trigger an alert.

Log Pattern details

INFO construct_email body done successfully 

INFO Email notification sent successfully 

Query built like this 

index=test sourcetype=test1 (source= cloudwatch/aws/lambda/BLK OR source=cloudwatch/aws/lambda/PRD OR source=cloudwatch/aws/lambda/add)

| eval alert=case(match(_raw,"INFO construct_email body done successfully") AND match(_raw,"INFO Email notification sent successfully"),"No Email sent")

| stats count by alert | where count =0 

But not getting any output, so can any one guide me on how built the query to fetch the desired results.

thanks in advance. 

 

 

 

 

 

 

 

Labels (3)
Tags (1)
0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Not sure what you are trying to achieve here

| eval alert=case(match(_raw,"INFO construct_email body done successfully") AND match(_raw,"INFO Email notification sent successfully"),"No Email sent")

Can _raw match both strings at the same time i.e. for the same event? If not, alert is never set to "No Email sent" and will always be null. If alert is null

| stats count by alert

will have no events left in the pipeline and count won't exist for the where clause to check.

Having said that, splunk is not smart enough to know what is not in your events, you have to tell it what specifically to look for.

0 Karma

Hemnaath
Motivator

thanks for you quick response on this. 

Hey I am trying to capture the event logs which does not contain the two event pattern from the raw events.

Log Pattern

INFO construct_email body done successfully

INFO Email notification sent successfully

currently I don't see any events  which are not having these two pattern in the raw data in our test environment.  All the events contains the above two pattern of logs. 

As per the client there is a chance of failure of not sending the email, so told to capture the events which are not having this pattern. I am not sure how to capture those event using the SPL query when there is no failure events found in the raw log.

So can you guide me in building some logic to achieve the end result. 

 

 

Tags (1)
0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...