Knowledge Management

Report acceleration, summary updating

mblauw
Path Finder

I've got a dashboard in which the panels depend on accelerated reports. When building these reports, I've let them run once on 00:00 using scheduling. The next couple of days, these dashboards are extremely fast. However, I have to turn of scheduling because I want users to be able to use a time picker in my dashboard. After a couple of days, my dashboard panels have to be loaded all over again when an user wants to see the dashboard. Because the underlaying data consists of approx 10M events, this takes about half an hour.

From what I read on Splunk Answers and Splunk Docs, accelerated reports are filled every 10 minutes. My report acceleration summary page, however, shows for all of my reports that the summary is not updated for the last 8 days, when I ran the complete search for the last time.

What I want, is users to be able to use a time-picker on my dashboards and have results displayed immediately after they make their selection, just like the dashboard would behave just after running the report manually. Does anybody know how to achieve this?

Thank you very much in advance!

0 Karma

cmerriman
Super Champion

I would either suggest creating a summary index of your data or creating a data model. Data models will hold data for so long (like caching it) so the user could search for, say, a month and if the data model is holding data for the last 7 days, it'll grab that data fast and search the remaining time. A summary index will just keep adding data into itself when ran, so you could have data going back to the beginning of time, and searches run much faster from them since it's aggregated data.

http://docs.splunk.com/Documentation/Splunk/6.6.0/Knowledge/Usesummaryindexing
http://docs.splunk.com/Documentation/SplunkCloud/6.6.0/PivotTutorial/Buildtutorialdatamodel

0 Karma

mblauw
Path Finder

I've already tried both, but they take longer than one minute to come to any results. The dashboard panels based on the accelerated reports only took less than 1 second. Is this also possible with the options you say?

0 Karma

cmerriman
Super Champion

when you created the data model, you accelerated it?

0 Karma

mblauw
Path Finder

Yes, I did indeed!

0 Karma

mblauw
Path Finder

I ran the same search over my data with two different types of searches. These were the results:

Report type / event count / time elapsed
Acc. report / 93,680,712 / 3746 sec.
Pivot / 93,680,712 / 1910 sec.

Far from ideal..

My reports only have to be ran once every month. But the time picker has to be available. On a report with schedule, this is not possible. But when I ran the report using schedule and afterwards remove this schedule, the results are saved and can be manipulated with the time picker.

What would be the best solution?

0 Karma

mblauw
Path Finder

When I run my accelerated report again, it shows me this:

Event count: 30,820,596
Time elapsed: 327 secs.

But with completely different results (almost 15% off). How come it only scans about one third of all events ?

0 Karma

cmerriman
Super Champion

the accelerated report might only be adding onto the existing data it has cached. Also, report acceleration only works properly if you saved the report while running it in smart or fast mode, not verbose.

http://docs.splunk.com/Documentation/Splunk/6.6.0/Report/Acceleratereports

when you created the summary index, did that work better or worse?

0 Karma

mblauw
Path Finder

The accelerated report is in smart mode.

I will create an summary index right away and see how well that works!

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to November Tech Talks, Office Hours, and Webinars!

🍂 Fall into November with a fresh lineup of Community Office Hours, Tech Talks, and Webinars we’ve ...

Transform your security operations with Splunk Enterprise Security

Hi Splunk Community, Splunk Platform has set a great foundation for your security operations. With the ...

Splunk Admins and App Developers | Earn a $35 gift card!

Splunk, in collaboration with ESG (Enterprise Strategy Group) by TechTarget, is excited to announce a ...