Splunk Search

How to write query to get count of total volume and system errors in a single query

GaneshAryan
New Member

Hi All,
we have a scenario to throw an alert if  System error rate exceeds 5%  i.e. (#system errors / #total volume)*100% .
How to get the count of total events and system errors then calculate the percentage based on count as per above formula.

Ex : Query for total volume : sourcetype="sfdc:transaction_log__c" | eval message = "b2cforce-liveperson" | where like(_raw,"%".message."%")

Query for System errors  :  sourcetype="sfdc:transaction_log__c" | eval message = "userId Retrieval Failure"
| where like(_raw,"%".message."%")

0 Karma

bowesmana
SplunkTrust
SplunkTrust

Something like this

sourcetype="sfdc:transaction_log__c" (message = "*b2cforce-liveperson*" OR message = "*userId Retrieval Failure*")
| eval isEvent=if(match(message, "b2cforce-liveperson"), 1, 0)
| stats sum(isEvent) as Events count(eval(isEvent=0)) as Errors
| eval FailureRate=Errors/Events*100

It's better if you can use something else other than matching message to differentiate between errors and events, but as long as events contain the b2c... text and errors do not, this should work.

 

0 Karma

rnowitzki
Builder

Hi @GaneshAryan ,

Based on what you provided, I came up with this:

index=? sourcetype="sfdc:transaction_log__c" 
| eval totalmessage="b2cforce-liveperson", errormessage="userId Retrieval Failure"
| eval total=if(like(_raw,"%".totalmessage."%"),1,total), errors=if(like(_raw,"%".errormessage."%"),1,errors)
| stats sum(total) as total, sum(errors) as errors

After that you can do your math with "total" and "errors".

Not sure I'd use the like() function to search something in _raw in general. But it works it seems.

Maybe you can setup a field that indicates if the event is an error, instead of searching for a string in _raw all the time.

Cheers
Ralph

--
Karma and/or Solution tagging appreciated.
0 Karma
Get Updates on the Splunk Community!

What's New in Splunk Observability Cloud and Splunk AppDynamics - May 2025

This month, we’re delivering several new innovations in Splunk Observability Cloud and Splunk AppDynamics ...

Getting Started with Splunk Artificial Intelligence, Insights for Nonprofits, and ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Splunk Observability Cloud’s AI Assistant in Action Series: Identifying Unknown ...

Agentic AI powers the Splunk AI Assistant within the Splunk Observability Cloud interface to help you quickly ...