Dashboards & Visualizations

Why are my dashboard panels giving an error when being created using report reference as base search?

mbasharat
Contributor

Hi,

I have a simple scheduled report.

This report is scheduled and runs over night every day and gets me data for 1 year. It takes about an hour for this report to run over night so I can use it in the morning. This report has about 22 fields populated. The number of statistics results it populates is about 3 million rows. This report is accelerated for 1 year.

The next morning, when I come and look at completed jobs, it is always successful and it opens up perfectly fine.

Now, the trouble I am having is I am using this report to create dashboard panels using a report as a reference in a base search. When I try to get for example, simple stats(dc), it throws me the following error: "Error while fetching data" after few seconds and fails to populate my dashboard panels. However, when I run the same report for a smaller time span, e.g. 2 months or 3 months, then the dashboard panels load fine. They do take some time though but they load as I expect them to.

Why am I getting an error for a 1 year report when populating dashboard panel vs 2/3 months version of report? Maybe dashboard panel has some kind of rendering restrictions etc...??

It is a simple report, and when I simply pull the report the next morning, it literally takes about 3 seconds for it to show me all 3 million stats it pulled last night for the whole year. But, the problem is when I use it to create dashboard panels and also, even for smaller report for 2/3 months, it takes some time but it loads.

Larger span like a 1 year, it fails as stated above.

I am just trying to create several panels based off of one report for optimization and performance perspective, and I have been successful in pretty much all my work using this strategy. It is just this one that is giving me a hard time — possibly because it is a larger report? How do I fix this issue? And also, how do I get the panels to populate quick along with it?

Thanks in advance for the guidance.

0 Karma
1 Solution

martin_mueller
SplunkTrust
SplunkTrust

By default a base search can only be 500k results: http://docs.splunk.com/Documentation/Splunk/7.2.0/Viz/Savedsearches#Post-process_searches_2

You can either try increasing the setting - no idea if 10x the default will work well - or try to reduce the results count of your base search. How to do the latter best depends on your use case.

View solution in original post

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

By default a base search can only be 500k results: http://docs.splunk.com/Documentation/Splunk/7.2.0/Viz/Savedsearches#Post-process_searches_2

You can either try increasing the setting - no idea if 10x the default will work well - or try to reduce the results count of your base search. How to do the latter best depends on your use case.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Any app you use for keeping search head configuration in, local or - given that it's your app - default, limits.conf.

0 Karma

mbasharat
Contributor

Actually, we have designated apps for everything. This particular report is also in its specific App named ABC. What is the location of the file for this setting in any app?

Thanks

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

It's a searchhead setting. Ideally you'll create an app that contains this setting, deploy it to your dev searchhead, see if it works, and then deploy it to your SHC.

In almost all use cases it's possible to define smaller datacubes that power your panels... but again there's no way to help you there without knowing more details about your searches and the use cases you're trying to power with them.

0 Karma

mbasharat
Contributor

My development searchhead is not in a cluster, so to change the max_count settings in limits.conf @ search head? And where exactly? The one in /server or /local?

What about the clustered area? Does it need to be changed on each searchhead in cluster if it works?

Reducing the results count is not an option for me because I need all results as the report is for vulnerabilities for each asset.

Again, I am trying to reference report as base search for a dashboard panel where I am running stats to get results FROM the saved report.

I think increasing limit should work since dashboard panels populate when I use a smaller reprt e.g. 1month-3months.

Thanks and awaiting response.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...