Splunk Search

How to create a search to find the largest contributor to increasing values?

las
Contributor

I have a log with statistics from mq containing some key values (time, MQmanager, Queuename) and some variables (number of messages added, number of messages removed and largest queue size).

If I do a search over time

sourcetype=MQResetStat (mqmanager=MQP) queue=* | dedup _raw | timechart sum(deqcount) 

I see a line where over time, more and more messages are pushed thru the system.

How do I create a search that tells me which queues are responsible for the increased number of messages?

I could of course just add by queue to the above search, but either the rise is hidden in "other" or there are to many queues to make the graf readable.

Ideally I would want just the 10 queues with the largest difference between the start and end period of the search, but still graphed over time with the sum of deqcount as the variable.

I could also do a subsearch, where I find the 10 queues, and then do the original search on top of that, but that might not be feasable due to time limit on subsearches.

I have tried looking for something similar, but apparently is not able to frame my search, so anything useable surfaces.

Any help is much appreciated.

kind regards

1 Solution

sundareshr
Legend

@somesoni2 had a great solution to a similar question last week here https://answers.splunk.com/answers/402954/how-to-use-top-in-timechart.html#answer-402996. Using that as the basis, I believe this should work in your case.

sourcetype=MQResetStat (mqmanager=MQP) queue=* | bin span=15m _time as time | stats sum(deqcount) as deqcount by time queue | eventstats first(deqcount) as start last(deqcount) as end by queue | eval total = end-start | sort - total | streamstats current=f window=1 max(total) as prevval by queue | eval toplist=case(isnull(prevval ),1,prevval =total,0,1=1,1) |accum toplist  | where toplist<10 | table time queue deqcount | xyseries time queue deqcount 

View solution in original post

sundareshr
Legend

@somesoni2 had a great solution to a similar question last week here https://answers.splunk.com/answers/402954/how-to-use-top-in-timechart.html#answer-402996. Using that as the basis, I believe this should work in your case.

sourcetype=MQResetStat (mqmanager=MQP) queue=* | bin span=15m _time as time | stats sum(deqcount) as deqcount by time queue | eventstats first(deqcount) as start last(deqcount) as end by queue | eval total = end-start | sort - total | streamstats current=f window=1 max(total) as prevval by queue | eval toplist=case(isnull(prevval ),1,prevval =total,0,1=1,1) |accum toplist  | where toplist<10 | table time queue deqcount | xyseries time queue deqcount 
Get Updates on the Splunk Community!

Shape the Future of Splunk: Join the Product Research Lab!

Join the Splunk Product Research Lab and connect with us in the Slack channel #product-research-lab to get ...

Auto-Injector for Everything Else: Making OpenTelemetry Truly Universal

You might have seen Splunk’s recent announcement about donating the OpenTelemetry Injector to the ...

[Puzzles] Solve, Learn, Repeat: Character substitutions with Regular Expressions

This challenge was first posted on Slack #puzzles channelFor BORE at .conf23, we had a puzzle question which ...