Splunk Search

delta counts by keyname

malex
Engager

How can I get a delta count by a key name when there are multiple keys for plotting the delta in a report?

I have a collection that outputs like this via syslog:

TimeStampMsec="1390586680463" QueueName="ad.input" ConsumerCount="1" MessagePendingCount="0" EnqueueCount="9" DequeueCount="9"
TimeStampMsec="1390586680463" QueueName="ldap.input" ConsumerCount="0" MessagePendingCount="0" EnqueueCount="0" DequeueCount="0"
TimeStampMsec="1390586680463" QueueName="foo.bar" ConsumerCount="0" MessagePendingCount="4" EnqueueCount="0" DequeueCount="0"

The DequeueCount could increment for the next log entry for any of these records as identified by the QueueName key. I would like to setup a report that provides a linear graph by time for by QueueName of the delta on DequeueCount. I cannot figure this out with delta since I can't seem to get it to take the delta by the QueueName, it can only take the delta of the record previously.

I have done this with mvlist, but we could add/subtract the QueueNames and mvlist feels like it's accessing points via an array and I can't guarantee the order.

Tags (2)
1 Solution

kristian_kolb
Ultra Champion

Maybe not perfect, but can perhaps serve as some inspiration

your_base_search 
| sort QueueName, _time 
| streamstats window=1 current=f first(DequeueCount) AS prevDQ by QueueName 
| table _time QueueName DequeueCount prevDQ  
| eval delta = DequeueCount - prevDQ 
| fields - prevDQ

/K

View solution in original post

alacercogitatus
SplunkTrust
SplunkTrust

This is EXTREMELY EXPERIMENTAL - so your feedback is appreciated. It has been submitted to SplunkBase - hasn't been approved yet - feel free to try it out. But Seriously - I only tested this on a 6.0 Linux install. Use At YOUR OWN RISK 😄 (But send me feedback if it craps.)

http://data.kyleasmith.info/TA-deltaby.spl

The command for your data would look like this:

your_base_search | deltaby by=QueueName value=DequeueCount output=DeQueueCountDelta | timechart span=10m sum(DeQueueCountDelta) by QueueName

output is optional but useful. I think you can even stack the commands (I haven't tested that yet fully)

your_base_search | deltaby by=QueueName value=DequeueCount output=DequeueDelta | deltaby=QueueName value=EnqueueCount output=EnqueueDelta | timechart span=10m sum(DequeueDelta) sum(EnqueueDelta) by QueueName

Again: Experimental (v0.1). Prone to failure (depending on your inputs). Send me feedback: splunkapps@kyleasmith.info.

kristian_kolb
Ultra Champion

Maybe not perfect, but can perhaps serve as some inspiration

your_base_search 
| sort QueueName, _time 
| streamstats window=1 current=f first(DequeueCount) AS prevDQ by QueueName 
| table _time QueueName DequeueCount prevDQ  
| eval delta = DequeueCount - prevDQ 
| fields - prevDQ

/K

malex
Engager

This is getting there, thanks a lot.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

To compute a grouped-by delta you can use streamstats:

base search | streamstats window=1 current=f global=f last(value) as last by key | eval delta = value-last | charting stuff

(precise syntax may vary)

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Sure, but I assume that's done by the base search 😛 else the delta without grouping wouldn't have worked properly either.

0 Karma

kristian_kolb
Ultra Champion

but you'll need to sort before the streamstats

0 Karma

kristian_kolb
Ultra Champion

dammit, spent too long editing 🙂

0 Karma
Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Get the T-shirt to Prove You Survived Splunk University Bootcamp

As if Splunk University, in Las Vegas, in-person, with three days of bootcamps and labs weren’t enough, now ...

Wondering How to Build Resiliency in the Cloud?

IT leaders are choosing Splunk Cloud as an ideal cloud transformation platform to drive business resilience,  ...