How to remove consecutive duplicate values by host from custom perfmon counters?

[dayananda7449]

Hi There,

I am trying to figure out how to remove duplicates in a custom perfmon counters data that is exported to Splunk.

We have a custom perfmon counter implemented for a service, which reports response time of a service when ever it is called. Otherwise, it will be returning the same old data from the last service response until the new request is served. The same service is hosted on multiple hosts and all the host report perfmon data.

I am trying to find the average by removing all the consecutive duplicates by host and get an average.

index=perfmon collection=ServiceBus counter="Response Time" instance="Service" | dedup Value host consecutive=true | chart avg(Value)

This search is not accurate.

Example Data :

Host 1: 
Time              Servicebus/ResponseTime    
00:00:01                   460
00:00:02                   460
00:00:03                   520
00:00:04                   520
00:00:05                   630

Host 2:

Time              Servicebus/ResponseTime    
00:00:01                   480
00:00:02                   480
00:00:03                   590
00:00:04                   590
00:00:05                   610

Since we are sending data to Splunk every 1s from all the hosts, this is how the data in the Splunk is represented (as per my knowledge). See below. In this case, the above search is not removing the duplicates.

Time              Servicebus/ResponseTime     Host
00:00:01                   460                                Host 1
00:00:01                   480                                Host 2
00:00:02                   460                                Host 1
00:00:02                  480                                Host 2
00:00:03                   520                                Host 1
00:00:03                   590                                Host 2
00:00:04                   520                                Host 1
00:00:04                   590                                Host 2
00:00:05                   630                                Host 1
00:00:05                   610                                 Host 2

Can someone help me on how to remove consecutive duplicates by host?

Thanks,
Dayananda

[gokadroid]

Assuming you have field host (which has host values) and field Value (which has Servicebus/ResponseTime) , please try this:

index=perfmon collection=ServiceBus counter="Response Time" instance="Service"
| sort host
| fields host, Value
| dedup host, Value consecutive=true
| stats avg(Value) by host