Dashboards & Visualizations

HiddenPostProcess spawns a new realtime search for each dashboard item?

vectorsc
Explorer

When using a realtime HiddenPostProcess search to power a dashboard, I find that splunk still spawns one RT search for each dashboard item.

I was given to understand this would not occur?
Dashboard works fine, just spawns 60 searches when run.

Tags (1)
1 Solution

sideview
SplunkTrust
SplunkTrust

This happens once in a while when people use the postprocess modules before really understanding when and where searches get kicked off.

Short answer -- put a single JobProgressIndicator module at the same level of the XML, or above, where all those HiddenPostProcess modules are. That will solve your problem. The JobProgressIndicator is one of the modules that "requires a search to be running", and because of putting that one module at or above the place where the XML branches, a single search will be then dispatched by the framework to serve that one JobProgressIndicator. Then as the data is passed to the various forks below each with their own HiddenPostProcess module, that one dispatched job will be distributed among all the modules, and no further searches will be dispatched.

When you dont have any dispatching module up at the level where it branches, what happens is the data all branches first. Then when that data gets deeper in and the framework realizes it all needs searches kicked off, it has to kick off one search for each branch. Bad news.

For further reading Sideview Utils has a good docs page that gives you an intro to this stuff.

You can install the latest Sideview Utils app (2.1.1) http://sideviewapps.com/apps/sideview-utils/ and then once it's installed you navigate to "Key Techniques > Overview of the Advanced XML"

View solution in original post

sideview
SplunkTrust
SplunkTrust

This happens once in a while when people use the postprocess modules before really understanding when and where searches get kicked off.

Short answer -- put a single JobProgressIndicator module at the same level of the XML, or above, where all those HiddenPostProcess modules are. That will solve your problem. The JobProgressIndicator is one of the modules that "requires a search to be running", and because of putting that one module at or above the place where the XML branches, a single search will be then dispatched by the framework to serve that one JobProgressIndicator. Then as the data is passed to the various forks below each with their own HiddenPostProcess module, that one dispatched job will be distributed among all the modules, and no further searches will be dispatched.

When you dont have any dispatching module up at the level where it branches, what happens is the data all branches first. Then when that data gets deeper in and the framework realizes it all needs searches kicked off, it has to kick off one search for each branch. Bad news.

For further reading Sideview Utils has a good docs page that gives you an intro to this stuff.

You can install the latest Sideview Utils app (2.1.1) http://sideviewapps.com/apps/sideview-utils/ and then once it's installed you navigate to "Key Techniques > Overview of the Advanced XML"

abhiram
Explorer

hi vectorsc,
I have exactly the same issue of panels spawning multiple searches. can u let me know where exactly to place JobProgressIndicator...I tried placing at couple of places above all the hiddenPostProcess modules...its not working.

0 Karma

vectorsc
Explorer

This was 100% accurate and the answer of moving the JobProgressIndicator to the top level worked perfectly.

Additionally, leaving it out of the XML altogether was not functional for other people looking at this question.

0 Karma
Get Updates on the Splunk Community!

Fall Into Learning with New Splunk Education Courses

Every month, Splunk Education releases new courses to help you branch out, strengthen your data science roots, ...

Super Optimize your Splunk Stats Searches: Unlocking the Power of tstats, TERM, and ...

By Martin Hettervik, Senior Consultant and Team Leader at Accelerate at Iver, Splunk MVPThe stats command is ...

How Splunk Observability Cloud Prevented a Major Payment Crisis in Minutes

Your bank's payment processing system is humming along during a busy afternoon, handling millions in hourly ...