Calculate the difference between two values by time division

[quixand]

Hi all,

I need to get some stats from an Apache ActiveMQ message broker. The broker has a web dashboard of broker queue depths.
- current queue depth
- messages dequeued (since last restart)

As far as I'm aware, there are no other stats available.

I want to know the number of messages that have dequeued from the broker by time division (say 1h but could be by 3h or day or week). I simply want to know the total number of messages that have been consumed per queue so I can visualize queue performance over time.

We have a bash script that harvests the counter and records to a log, something like

Tue Apr 12 15:01:02 BST 2016 AppQueueA_depth="10" AppQueueA_dequeue="500"

This is recorded every 5 minutes, but because this is a total since application restart, I need to subtract the first occurrence of AppQueueA_dequeue from the first occurrence from the previous hour, and so on and so forth.

I think i need to bucket the events by hour and extract the first event per bucket, then calculate the difference between each one.

Any help appreciated.

[twinspop]

EDIT: removed streamstats hackery. somesoni2's answer shows off delta which would be better than streamstats in this situation.

[somesoni2 · Accepted solution]

Give this a try

your base search | bucket span=1h _time | stats min(AppQueueA_dequeue) as AppQueueA_dequeue by _time | delta AppQueueA_dequeue as msgProcessed | table _time msgProcessed

[quixand]

Thanks, this is exactly what I needed.