Alerting

How to create an alert/dashboard for queueing fill ratio in Master, Monitoring console in search head

phamxuantung
Communicator

In Monitoring Console in our Master, we have

Capture11.PNG

And

inci1.PNG

I open it in search and got this query

| rest splunk_server=test43 /services/server/introspection/queues
      | eval current_fill_perc = round(current_size_bytes / max_size_bytes * 100, 0)
      | fields title, current_fill_perc | search title="parsingQueue.*" OR title="aggQueue.*" OR title="typingQueue.*" OR title="indexQueue.*"
            | rex field=title "(?<queue_name>^\w+)\.(?<pipeline_number>\d+)"
            | chart values(current_fill_perc) over pipeline_number by queue_name
            | fields pipeline_number, parsingQueue, aggQueue, typingQueue, indexQueue
            | rename pipeline_number as "Pipeline Number", parsingQueue as "Parsing Queue Fill Ratio (%)", aggQueue as "Aggregator Queue Fill Ratio (%)", typingQueue as "Typing Queue Fill Ratio (%)", indexQueue as "Index Queue Fill Ratio (%)"

I want to create a dashboard for some members that don't have access in our Master server to see, but when I run that search in our Search head, it produce no result.

So there is other way to show it in our SH?

0 Karma

isoutamo
SplunkTrust
SplunkTrust

Hi

this seems to work

| rest splunk_server=<your indexer group name> /services/server/introspection/queues 
| eval current_fill_perc = round(current_size_bytes / max_size_bytes * 100, 0) 
| fields title, current_fill_perc 
| search title="parsingQueue*" OR title="aggQueue*" OR title="typingQueue*" OR title="indexQueue*" 
| rex field=title "(?<queue_name>^\w+)\.?(?<pipeline_number>\d+)?"
| fillnull VALUE=0 pipeline_number 
| chart values(current_fill_perc) over pipeline_number by queue_name 
| fields pipeline_number, parsingQueue, aggQueue, typingQueue, indexQueue 
| rename pipeline_number as "Pipeline Number", parsingQueue as "Parsing Queue Fill Ratio (%)", aggQueue as "Aggregator Queue Fill Ratio (%)", typingQueue as "Typing Queue Fill Ratio (%)", indexQueue as "Index Queue Fill Ratio (%)"

You should add correct wildcard like splunk-idx-* to splunk_server to target this query towards correct server.

Also you should create this as a report which are owned by some admin user (which have capability to run REST query) and run it "as owner" to allow other people to see results.

r. Ismo

0 Karma
Get Updates on the Splunk Community!

Splunk Enterprise Security 8.x: The Essential Upgrade for Threat Detection, ...

 Prepare to elevate your security operations with the powerful upgrade to Splunk Enterprise Security 8.x! This ...

Get Early Access to AI Playbook Authoring: Apply for the Alpha Private Preview ...

Passionate about security automation? Apply now to our AI Playbook Authoring Alpha private preview ...

Reduce and Transform Your Firewall Data with Splunk Data Management

Managing high-volume firewall data has always been a challenge. Noisy events and verbose traffic logs often ...