Alerting

How to create an alert/dashboard for queueing fill ratio in Master, Monitoring console in search head

phamxuantung
Communicator

In Monitoring Console in our Master, we have

Capture11.PNG

And

inci1.PNG

I open it in search and got this query

| rest splunk_server=test43 /services/server/introspection/queues
      | eval current_fill_perc = round(current_size_bytes / max_size_bytes * 100, 0)
      | fields title, current_fill_perc | search title="parsingQueue.*" OR title="aggQueue.*" OR title="typingQueue.*" OR title="indexQueue.*"
            | rex field=title "(?<queue_name>^\w+)\.(?<pipeline_number>\d+)"
            | chart values(current_fill_perc) over pipeline_number by queue_name
            | fields pipeline_number, parsingQueue, aggQueue, typingQueue, indexQueue
            | rename pipeline_number as "Pipeline Number", parsingQueue as "Parsing Queue Fill Ratio (%)", aggQueue as "Aggregator Queue Fill Ratio (%)", typingQueue as "Typing Queue Fill Ratio (%)", indexQueue as "Index Queue Fill Ratio (%)"

I want to create a dashboard for some members that don't have access in our Master server to see, but when I run that search in our Search head, it produce no result.

So there is other way to show it in our SH?

0 Karma

isoutamo
SplunkTrust
SplunkTrust

Hi

this seems to work

| rest splunk_server=<your indexer group name> /services/server/introspection/queues 
| eval current_fill_perc = round(current_size_bytes / max_size_bytes * 100, 0) 
| fields title, current_fill_perc 
| search title="parsingQueue*" OR title="aggQueue*" OR title="typingQueue*" OR title="indexQueue*" 
| rex field=title "(?<queue_name>^\w+)\.?(?<pipeline_number>\d+)?"
| fillnull VALUE=0 pipeline_number 
| chart values(current_fill_perc) over pipeline_number by queue_name 
| fields pipeline_number, parsingQueue, aggQueue, typingQueue, indexQueue 
| rename pipeline_number as "Pipeline Number", parsingQueue as "Parsing Queue Fill Ratio (%)", aggQueue as "Aggregator Queue Fill Ratio (%)", typingQueue as "Typing Queue Fill Ratio (%)", indexQueue as "Index Queue Fill Ratio (%)"

You should add correct wildcard like splunk-idx-* to splunk_server to target this query towards correct server.

Also you should create this as a report which are owned by some admin user (which have capability to run REST query) and run it "as owner" to allow other people to see results.

r. Ismo

0 Karma
Get Updates on the Splunk Community!

Index This | When is October more than just the tenth month?

October 2025 Edition  Hayyy Splunk Education Enthusiasts and the Eternally Curious!   We’re back with this ...

Observe and Secure All Apps with Splunk

  Join Us for Our Next Tech Talk: Observe and Secure All Apps with SplunkAs organizations continue to innovate ...

What’s New & Next in Splunk SOAR

 Security teams today are dealing with more alerts, more tools, and more pressure than ever.  Join us for an ...