Splunk Search

How to group events and extract a field when grouped events contain a specific value?

kevinkuszyk
Engager

We have some overnight jobs that run and log out to Splunk. On top of this, we have a dashboard which groups by the job id and extracts information like start time, end time, duration etc.

The query looks a bit like this:

index=foo | stats range(_time) as duration by job-id | table job-id duration

We now want to add a status column to tell us if the job completed or had an error. If any of the events in a grouping have a log level of ERROR it should show Error, otherwise it should show Ok.
I've tried this snippet:

eval status=if(in(level, "ERROR"), "Error", "Ok")

Which is fine for evaluating on each event, but I want the grouping to show either Error or Ok depending on values in the the level field for each group.

Is this possible in Splunk, and how should I write the query?

0 Karma
1 Solution

renjith_nair
Legend

@kevinkuszyk ,

Try this ,

|stats min(eval(if(level=="ERROR","Error","Ok"))) as status by group

OR

|stats count(eval(if(level=="ERROR",1,null()))) as status by group|eval status=if(status>0,"Error","Ok")
---
What goes around comes around. If it helps, hit it with Karma 🙂

View solution in original post

woodcock
Esteemed Legend

Maybe this:

index=foo
| stats range(_time) AS duration count(eval(level="ERROR")) AS Errors BY job-id
0 Karma

woodcock
Esteemed Legend

Like this:

Your base search here
| stats count(eval(level == "ERROR")) AS errors count AS total BY group
| eval non_errors = total - errors
0 Karma

renjith_nair
Legend

@kevinkuszyk ,

Try this ,

|stats min(eval(if(level=="ERROR","Error","Ok"))) as status by group

OR

|stats count(eval(if(level=="ERROR",1,null()))) as status by group|eval status=if(status>0,"Error","Ok")
---
What goes around comes around. If it helps, hit it with Karma 🙂
Get Updates on the Splunk Community!

.conf25 technical session recap of Observability for Gen AI: Monitoring LLM ...

If you’re unfamiliar, .conf is Splunk’s premier event where the Splunk community, customers, partners, and ...

A Season of Skills: New Splunk Courses to Light Up Your Learning Journey

There’s something special about this time of year—maybe it’s the glow of the holidays, maybe it’s the ...

Announcing the Migration of the Splunk Add-on for Microsoft Azure Inputs to ...

Announcing the Migration of the Splunk Add-on for Microsoft Azure Inputs to Officially Supported Splunk ...