Deployment Architecture

Calculate the difference between two values by time division

quixand
Path Finder

Hi all,

I need to get some stats from an Apache ActiveMQ message broker. The broker has a web dashboard of broker queue depths.
- current queue depth
- messages dequeued (since last restart)

As far as I'm aware, there are no other stats available.

I want to know the number of messages that have dequeued from the broker by time division (say 1h but could be by 3h or day or week). I simply want to know the total number of messages that have been consumed per queue so I can visualize queue performance over time.

We have a bash script that harvests the counter and records to a log, something like

Tue Apr 12 15:01:02 BST 2016 AppQueueA_depth="10" AppQueueA_dequeue="500"

This is recorded every 5 minutes, but because this is a total since application restart, I need to subtract the first occurrence of AppQueueA_dequeue from the first occurrence from the previous hour, and so on and so forth.

I think i need to bucket the events by hour and extract the first event per bucket, then calculate the difference between each one.

Any help appreciated.

0 Karma
1 Solution

somesoni2
Revered Legend

Give this a try

your base search | bucket span=1h _time | stats min(AppQueueA_dequeue) as AppQueueA_dequeue by _time | delta AppQueueA_dequeue as msgProcessed | table _time msgProcessed

View solution in original post

twinspop
Influencer

EDIT: removed streamstats hackery. somesoni2's answer shows off delta which would be better than streamstats in this situation.

0 Karma

somesoni2
Revered Legend

Give this a try

your base search | bucket span=1h _time | stats min(AppQueueA_dequeue) as AppQueueA_dequeue by _time | delta AppQueueA_dequeue as msgProcessed | table _time msgProcessed

quixand
Path Finder

Thanks, this is exactly what I needed.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...