How to count how many times a field value has changed for a stream of events over a period of time?

[jedatt01]

I want to count the number of times the value of a field called "Node_Group" has changed for a stream of events over a period of time and group it by a field called NetworkDeviceName.

I believe the streamstats command should accomplish this but I'm not confident in how to know when the change occurs. So far I just have | streamstats count(Node_Group) by NetworkDeviceName.

Any ideas?

[Runals · Accepted solution]

I'd try some variation of

... | sort _time NetworkDeviceName | streamstats dc(Node_Group) by NetworkDeviceName

[jedatt01]

With a slight variation this worked......add window=2. thanks

[musskopf]

Yes, I believe streamstats is the way to go. This command basically calculates stats for every event, based on the order the events are being returned in a search on Splunk... here an example, how you could use it:

index=bla "your search" fieldA=* | sort +_time
| streamstats window=1 current=false last(fieldA) AS previous_fieldA
| where fieldA!=last_fieldA | table *

Note that I've used sort to guarantee the events are processed on the right order, from the oldest to the newest. After that I use streamstats with the options window=1 current=false to grab the previous value from fieldA and "copy" to the current event as previous_fieldA. After that just make a search to see the ones which are different, so it'll indicate a change!

Does that any sense for your use case? Let me know if we're getting close to what you're after.

Cheers,

[ramdaspr]

++
And you could replace the ending | table * by |stats count as CountofChanges if you are interested only in the count.

However, I am curious about the

and group it by a field called NetworkDeviceName.

@jedatt01 Are you interested in finding how many times it has changed from/to each Node_Group?

[jedatt01]

Yes, trying to count the number of changes in field Node_Group because this implies a failover. Here's what did and it seems to be working correctly.

streamstats window=2 dc(Node_Group) As NG_Count by NetworkDeviceName

The window=2 is what solved it for me