Splunk Search

Calculation based on comparison of previous event's field to current event's field

actionabledata
Path Finder


My objective is to increment ReplicaCount if the previous event's field value matches the current event's field value. 

 

 

 

| makeresults count=10 
| eval num = random() % 100 
| eval error = case( num < 10, 404, num >= 10 AND num < 13, 500, num >= 13 AND num < 40, 200, num >41, 333 ) 
| eval error_msg = case(error == 404, "Not found", error == 333, "Made Up Error Code", error == 500, "Internal Server Error", error == 200, "OK")

| sort 0 - _time
| streamstats current=f last(error) AS prev_error BY error
| eval ReplicaCount=1
| eval ReplicaCount =if(like(error,prev_error), ReplicaCount+1,ReplicaCount)
| table _time error prev_error ReplicaCount error_msg num

 

 

actionabledata_0-1623786960099.png

Sorry, cannot figure this out. What am I missing?
I don't think last() is doing what I think it's doing which is ... obtain the field value from the previous event.

[1] Why are some of my prev_error values blank?

[2] why does ReplicaCount max out at 2?

Thank you in advance.

 

Labels (1)
Tags (2)
0 Karma
1 Solution

actionabledata
Path Finder

Turns out that you don't really need the sameFlag and prev_error fields and associated calculations.
I just change the BY clause on the streamstats

| streamstats count as repCount BY error reset_on_change=true 



| makeresults count=10 
| rename COMMENT AS "------------------------------------------------------------"
| rename COMMENT AS "Generate fake data"
| eval group = "GROUP"
| eval r = random() % 10 + 1
| eval num = case( r=1,18, r=2,10, r=3,99, r=4,82, r=5,95, r=6,54, r=7,4, r=8,40, r=9,83, r=10,33)
| rename COMMENT AS " eval num = random() % 100" 
| eval error = case( num < 10, 404, num >= 10 AND num < 13, 500, num >= 13 AND num < 40, 200, num >=40, 333) 
| eval error_msg = case(error == 404, "Not found", error == 333, "Made Up Error Code", error == 500, "Internal Server Error", error == 200, "OK")

| rename COMMENT AS "------------------------------------------------------------"
| rename COMMENT AS "Calculate repCount"
| streamstats count as repCount BY error reset_on_change=true 

| table group repCount error

 

View solution in original post

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust
| makeresults count=10 
| eval num = random() % 100 
| eval error = case( num < 10, 404, num >= 10 AND num < 13, 500, num >= 13 AND num < 40, 200, num >41, 333 ) 
| eval error_msg = case(error == 404, "Not found", error == 333, "Made Up Error Code", error == 500, "Internal Server Error", error == 200, "OK")

| sort 0 - _time
| streamstats window=1 current=f last(error) AS prev_error
| eval ReplicaCount =if(like(error,prev_error), 1,0)
| streamstats sum(ReplicaCount) as ReplicaCount
| table _time error prev_error ReplicaCount error_msg num
0 Karma

actionabledata
Path Finder

Thank you ITWhisperer !

However, I need to clarify my requirements a bit. Sorry for not clarifying previously.

I need to count the number of sequential replications of a field. Eventually, I will aggregate the fields into a smaller list where only the unique fields are included with a suffix of the max(replication factor).

The replication factor (repCount) looks like this:
Note: The first 3 values or "error" are 333 and the repCount increments to 3, which is correct.

Note: The next 2 values also have a repeat of 222 and repCount increments to 2 which is correct.

Note: The repCount value resets every time the error value changes state.

Note: The subsequent 333's are an independent set of repeats, so the repCount begins fresh and is not related to the previous count of 333's.

actionabledata_0-1623865482693.png

 

Here is my code that produced the above results but there has to be a more elegant way to do this.
I experimented with the streamstats "reset_on_change" option but had to add an eval command afterward to force the results.

| eval repCount = if(sameFlag=="0", 1, repCount+1)

 

 

| makeresults count=10 
| rename COMMENT AS "------------------------------------------------------------"
| rename COMMENT AS "Generate fake data"
| eval group = "GROUP"
| eval r = random() % 10 + 1
| eval num = case( r=1,18, r=2,10, r=3,99, r=4,82, r=5,95, r=6,54, r=7,4, r=8,40, r=9,83, r=10,33)
| rename COMMENT AS " eval num = random() % 100" 
| eval error = case( num < 10, 404, num >= 10 AND num < 13, 500, num >= 13 AND num < 40, 200, num >=40, 333) 
| eval error_msg = case(error == 404, "Not found", error == 333, "Made Up Error Code", error == 500, "Internal Server Error", error == 200, "OK")


| rename COMMENT AS "------------------------------------------------------------"
| rename COMMENT AS "When sameFlag=1, increment repCount; When sameFlag=1, repCount=1"
| rename COMMENT AS "------------------------------------------------------------"

| rename COMMENT AS "------------------------------------------------------------"
| rename COMMENT AS "Calculate sameFlag"
| streamstats window=1 current=f last(error) AS prev_error BY group
| eval sameFlag = if(like(error,prev_error), 1, 0)

| rename COMMENT AS "------------------------------------------------------------"
| rename COMMENT AS "Calculate repCount"
| streamstats window=1 current=f last(sameFlag) AS prev_sameFlag BY group
| streamstats count as repCount BY sameFlag reset_on_change=true 
| eval repCount = if(sameFlag=="0", 1, repCount+1)

| table group repCount sameFlag error prev_error

 

 

Thank you in advance for any help on optimizing this code.

 

0 Karma

actionabledata
Path Finder

Turns out that you don't really need the sameFlag and prev_error fields and associated calculations.
I just change the BY clause on the streamstats

| streamstats count as repCount BY error reset_on_change=true 



| makeresults count=10 
| rename COMMENT AS "------------------------------------------------------------"
| rename COMMENT AS "Generate fake data"
| eval group = "GROUP"
| eval r = random() % 10 + 1
| eval num = case( r=1,18, r=2,10, r=3,99, r=4,82, r=5,95, r=6,54, r=7,4, r=8,40, r=9,83, r=10,33)
| rename COMMENT AS " eval num = random() % 100" 
| eval error = case( num < 10, 404, num >= 10 AND num < 13, 500, num >= 13 AND num < 40, 200, num >=40, 333) 
| eval error_msg = case(error == 404, "Not found", error == 333, "Made Up Error Code", error == 500, "Internal Server Error", error == 200, "OK")

| rename COMMENT AS "------------------------------------------------------------"
| rename COMMENT AS "Calculate repCount"
| streamstats count as repCount BY error reset_on_change=true 

| table group repCount error

 

0 Karma
Get Updates on the Splunk Community!

Introducing Splunk Enterprise 9.2

WATCH HERE! Watch this Tech Talk to learn about the latest features and enhancements shipped in the new Splunk ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...