Knowledge Management

Search Acceleration or Summary indexing?

hartfoml
Motivator

My Firewall guys run this report every day to get data out of the firewall

index="firewall" source_zone_name="*" destination_zone_name="*" policy_name="*" | stats count by "source_zone_name", "destination_zone_name", "policy_name" | sort "source_zone_name", "destination_zone_name", "policy_name"

I ran this report for 1 minute of time in the recent past in "Fast Mode" and it took more than 1 minute to get the return.

The 24 hour report takes almost 7 hours to run and returned 25M records from 195M records and shows 1330 resultant rows

I don't know what would be the best way to set this up to have the lowest impact on the system so that they could login and get the report whenever they like without taking too much time.

Also they manually combine the daily reports into one large report for monthly activity.

0 Karma

woodcock
Esteemed Legend

I would definitely go with a Summary Index because you are doing multiple roll-ups (daily/monthly). You will find incredible speedups using Summary Index.

0 Karma

gfuente
Motivator

Hello

I will go with the Report Acceleration feature, you could just accelerate the last day or last week to save disk if you want. With that feature on that report will load much faster.

If you don't want to spend resources continuously, you also could schedule the query at midnight so they get the report loaded when they arrive at work

Regards

0 Karma

ChrisG
Splunk Employee
Splunk Employee

FWIW, there is a topic in the Knowledge Manager Manual that describes the differences between the three ways to accelerate searching using data summaries, with some guidance about when to use which one: http://docs.splunk.com/Documentation/Splunk/6.2.2/Knowledge/Aboutsummaryindexing.

Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...