Splunk Search

How to create a search to find the largest contributor to increasing values?


I have a log with statistics from mq containing some key values (time, MQmanager, Queuename) and some variables (number of messages added, number of messages removed and largest queue size).

If I do a search over time

sourcetype=MQResetStat (mqmanager=MQP) queue=* | dedup _raw | timechart sum(deqcount) 

I see a line where over time, more and more messages are pushed thru the system.

How do I create a search that tells me which queues are responsible for the increased number of messages?

I could of course just add by queue to the above search, but either the rise is hidden in "other" or there are to many queues to make the graf readable.

Ideally I would want just the 10 queues with the largest difference between the start and end period of the search, but still graphed over time with the sum of deqcount as the variable.

I could also do a subsearch, where I find the 10 queues, and then do the original search on top of that, but that might not be feasable due to time limit on subsearches.

I have tried looking for something similar, but apparently is not able to frame my search, so anything useable surfaces.

Any help is much appreciated.

kind regards


Re: How to create a search to find the largest contributor to increasing values?


@somesoni2 had a great solution to a similar question last week here https://answers.splunk.com/answers/402954/how-to-use-top-in-timechart.html#answer-402996. Using that as the basis, I believe this should work in your case.

sourcetype=MQResetStat (mqmanager=MQP) queue=* | bin span=15m _time as time | stats sum(deqcount) as deqcount by time queue | eventstats first(deqcount) as start last(deqcount) as end by queue | eval total = end-start | sort - total | streamstats current=f window=1 max(total) as prevval by queue | eval toplist=case(isnull(prevval ),1,prevval =total,0,1=1,1) |accum toplist  | where toplist<10 | table time queue deqcount | xyseries time queue deqcount 

View solution in original post