Alerting

How to write a query to find when certain log pattern are not present in the raw logs?

Hemnaath
Motivator

Hi All, I have requirement to create an alert. 

Condition:

In the raw data when certain log Pattern are not found then it should trigger an alert.

Log Pattern details

INFO construct_email body done successfully 

INFO Email notification sent successfully 

Query built like this 

index=test sourcetype=test1 (source= cloudwatch/aws/lambda/BLK OR source=cloudwatch/aws/lambda/PRD OR source=cloudwatch/aws/lambda/add)

| eval alert=case(match(_raw,"INFO construct_email body done successfully") AND match(_raw,"INFO Email notification sent successfully"),"No Email sent")

| stats count by alert | where count =0 

But not getting any output, so can any one guide me on how built the query to fetch the desired results.

thanks in advance. 

 

 

 

 

 

 

 

Labels (3)
Tags (1)
0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Not sure what you are trying to achieve here

| eval alert=case(match(_raw,"INFO construct_email body done successfully") AND match(_raw,"INFO Email notification sent successfully"),"No Email sent")

Can _raw match both strings at the same time i.e. for the same event? If not, alert is never set to "No Email sent" and will always be null. If alert is null

| stats count by alert

will have no events left in the pipeline and count won't exist for the where clause to check.

Having said that, splunk is not smart enough to know what is not in your events, you have to tell it what specifically to look for.

0 Karma

Hemnaath
Motivator

thanks for you quick response on this. 

Hey I am trying to capture the event logs which does not contain the two event pattern from the raw events.

Log Pattern

INFO construct_email body done successfully

INFO Email notification sent successfully

currently I don't see any events  which are not having these two pattern in the raw data in our test environment.  All the events contains the above two pattern of logs. 

As per the client there is a chance of failure of not sending the email, so told to capture the events which are not having this pattern. I am not sure how to capture those event using the SPL query when there is no failure events found in the raw log.

So can you guide me in building some logic to achieve the end result. 

 

 

Tags (1)
0 Karma
Get Updates on the Splunk Community!

Enter the Agentic Era with Splunk AI Assistant for SPL 1.4

  🚀 Your data just got a serious AI upgrade — are you ready? Say hello to the Agentic Era with the ...

Stronger Security with Federated Search for S3, GCP SQL & Australian Threat ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Accelerating Observability as Code with the Splunk AI Assistant

We’ve seen in previous posts what Observability as Code (OaC) is and how it’s now essential for managing ...