My objective is to increment ReplicaCount if the previous event's field value matches the current event's field value.
| makeresults count=10
| eval num = random() % 100
| eval error = case( num < 10, 404, num >= 10 AND num < 13, 500, num >= 13 AND num < 40, 200, num >41, 333 )
| eval error_msg = case(error == 404, "Not found", error == 333, "Made Up Error Code", error == 500, "Internal Server Error", error == 200, "OK")
| sort 0 - _time
| streamstats current=f last(error) AS prev_error BY error
| eval ReplicaCount=1
| eval ReplicaCount =if(like(error,prev_error), ReplicaCount+1,ReplicaCount)
| table _time error prev_error ReplicaCount error_msg num
Sorry, cannot figure this out. What am I missing?
I don't think last() is doing what I think it's doing which is ... obtain the field value from the previous event.
[1] Why are some of my prev_error values blank?
[2] why does ReplicaCount max out at 2?
Thank you in advance.
Turns out that you don't really need the sameFlag and prev_error fields and associated calculations.
I just change the BY clause on the streamstats
| streamstats count as repCount BY error reset_on_change=true
| makeresults count=10
| rename COMMENT AS "------------------------------------------------------------"
| rename COMMENT AS "Generate fake data"
| eval group = "GROUP"
| eval r = random() % 10 + 1
| eval num = case( r=1,18, r=2,10, r=3,99, r=4,82, r=5,95, r=6,54, r=7,4, r=8,40, r=9,83, r=10,33)
| rename COMMENT AS " eval num = random() % 100"
| eval error = case( num < 10, 404, num >= 10 AND num < 13, 500, num >= 13 AND num < 40, 200, num >=40, 333)
| eval error_msg = case(error == 404, "Not found", error == 333, "Made Up Error Code", error == 500, "Internal Server Error", error == 200, "OK")
| rename COMMENT AS "------------------------------------------------------------"
| rename COMMENT AS "Calculate repCount"
| streamstats count as repCount BY error reset_on_change=true
| table group repCount error
| makeresults count=10
| eval num = random() % 100
| eval error = case( num < 10, 404, num >= 10 AND num < 13, 500, num >= 13 AND num < 40, 200, num >41, 333 )
| eval error_msg = case(error == 404, "Not found", error == 333, "Made Up Error Code", error == 500, "Internal Server Error", error == 200, "OK")
| sort 0 - _time
| streamstats window=1 current=f last(error) AS prev_error
| eval ReplicaCount =if(like(error,prev_error), 1,0)
| streamstats sum(ReplicaCount) as ReplicaCount
| table _time error prev_error ReplicaCount error_msg num
Thank you ITWhisperer !
However, I need to clarify my requirements a bit. Sorry for not clarifying previously.
I need to count the number of sequential replications of a field. Eventually, I will aggregate the fields into a smaller list where only the unique fields are included with a suffix of the max(replication factor).
The replication factor (repCount) looks like this:
Note: The first 3 values or "error" are 333 and the repCount increments to 3, which is correct.
Note: The next 2 values also have a repeat of 222 and repCount increments to 2 which is correct.
Note: The repCount value resets every time the error value changes state.
Note: The subsequent 333's are an independent set of repeats, so the repCount begins fresh and is not related to the previous count of 333's.
Here is my code that produced the above results but there has to be a more elegant way to do this.
I experimented with the streamstats "reset_on_change" option but had to add an eval command afterward to force the results.
| eval repCount = if(sameFlag=="0", 1, repCount+1)
| makeresults count=10
| rename COMMENT AS "------------------------------------------------------------"
| rename COMMENT AS "Generate fake data"
| eval group = "GROUP"
| eval r = random() % 10 + 1
| eval num = case( r=1,18, r=2,10, r=3,99, r=4,82, r=5,95, r=6,54, r=7,4, r=8,40, r=9,83, r=10,33)
| rename COMMENT AS " eval num = random() % 100"
| eval error = case( num < 10, 404, num >= 10 AND num < 13, 500, num >= 13 AND num < 40, 200, num >=40, 333)
| eval error_msg = case(error == 404, "Not found", error == 333, "Made Up Error Code", error == 500, "Internal Server Error", error == 200, "OK")
| rename COMMENT AS "------------------------------------------------------------"
| rename COMMENT AS "When sameFlag=1, increment repCount; When sameFlag=1, repCount=1"
| rename COMMENT AS "------------------------------------------------------------"
| rename COMMENT AS "------------------------------------------------------------"
| rename COMMENT AS "Calculate sameFlag"
| streamstats window=1 current=f last(error) AS prev_error BY group
| eval sameFlag = if(like(error,prev_error), 1, 0)
| rename COMMENT AS "------------------------------------------------------------"
| rename COMMENT AS "Calculate repCount"
| streamstats window=1 current=f last(sameFlag) AS prev_sameFlag BY group
| streamstats count as repCount BY sameFlag reset_on_change=true
| eval repCount = if(sameFlag=="0", 1, repCount+1)
| table group repCount sameFlag error prev_error
Thank you in advance for any help on optimizing this code.
Turns out that you don't really need the sameFlag and prev_error fields and associated calculations.
I just change the BY clause on the streamstats
| streamstats count as repCount BY error reset_on_change=true
| makeresults count=10
| rename COMMENT AS "------------------------------------------------------------"
| rename COMMENT AS "Generate fake data"
| eval group = "GROUP"
| eval r = random() % 10 + 1
| eval num = case( r=1,18, r=2,10, r=3,99, r=4,82, r=5,95, r=6,54, r=7,4, r=8,40, r=9,83, r=10,33)
| rename COMMENT AS " eval num = random() % 100"
| eval error = case( num < 10, 404, num >= 10 AND num < 13, 500, num >= 13 AND num < 40, 200, num >=40, 333)
| eval error_msg = case(error == 404, "Not found", error == 333, "Made Up Error Code", error == 500, "Internal Server Error", error == 200, "OK")
| rename COMMENT AS "------------------------------------------------------------"
| rename COMMENT AS "Calculate repCount"
| streamstats count as repCount BY error reset_on_change=true
| table group repCount error