topic How to write a query to find when certain log pattern are not present in the raw logs? in Alerting

How to write a query to find when certain log pattern are not present in the raw logs?

Hemnaath — Thu, 10 Jun 2021 11:01:18 GMT

Hi All, I have requirement to create an alert.

Condition:

In the raw data when certain log Pattern are not found then it should trigger an alert.

Log Pattern details

INFO construct_email body done successfully

INFO Email notification sent successfully

Query built like this

index=test sourcetype=test1 (source= cloudwatch/aws/lambda/BLK OR source=cloudwatch/aws/lambda/PRD OR source=cloudwatch/aws/lambda/add)

| eval alert=case(match(_raw,"INFO construct_email body done successfully") AND match(_raw,"INFO Email notification sent successfully"),"No Email sent")

| stats count by alert | where count =0

But not getting any output, so can any one guide me on how built the query to fetch the desired results.

thanks in advance.

Re: How to write a query to find when certain log pattern are not present in the raw logs?

ITWhisperer — Thu, 10 Jun 2021 11:12:55 GMT

Not sure what you are trying to achieve here

| eval alert=case(match(_raw,"INFO construct_email body done successfully") AND match(_raw,"INFO Email notification sent successfully"),"No Email sent")

Can _raw match both strings at the same time i.e. for the same event? If not, alert is never set to "No Email sent" and will always be null. If alert is null

| stats count by alert

will have no events left in the pipeline and count won't exist for the where clause to check.

Having said that, splunk is not smart enough to know what is not in your events, you have to tell it what specifically to look for.

Re: How to write a query to find when certain log pattern are not present in the raw logs?

Hemnaath — Thu, 10 Jun 2021 14:10:11 GMT

thanks for you quick response on this.

Hey I am trying to capture the event logs which does not contain the two event pattern from the raw events.

Log Pattern

INFO construct_email body done successfully

INFO Email notification sent successfully

currently I don't see any events which are not having these two pattern in the raw data in our test environment. All the events contains the above two pattern of logs.

As per the client there is a chance of failure of not sending the email, so told to capture the events which are not having this pattern. I am not sure how to capture those event using the SPL query when there is no failure events found in the raw log.

So can you guide me in building some logic to achieve the end result.