Graphing scheduled saved search results

[rereeser]

Hello fellow splunkers,

I have a large dataset that I am searching through, and I want to create a historical timechart which goes back for several months. Because of the size of the dataset, having a search which goes that far back is impracticable (or at least impractical).

My solution was to schedule a daily search which would save the results from the last 24 hours. After 3 months, for example, I would have 90 saved results which each only contain a simple count of the number of events, and my chart could therefore simply graph the counts from each saved result, with each one being a datapoint. I'd just set the TTL for the saved results to be 90 days.

I'm fairly new to Splunk, but this seems like it would be a pretty basic feature, so I feel like I'm missing something. The closest I've gotten is using something like | append loadjob savedsearch=foo, but that will only add a single saved result, unless foo is somehow a "living" result which always has the results from the past 90 days.

I've heard of summary indexes for dealing with large datasets, and I'll research them to see if it's what I need, but I was hoping for a relatively simple solution which could be carried out within the Splunk web interface.

Thanks in advance, and sorry if this has been answered before.

[emiller42 · Accepted solution]

What you're trying to do is essentially summary indexing. Basically, you take your daily scheduled search and instead of sending the output to display, you send it to a separate index. Then in three months, you run your output search against the summary index so it only has to deal with 90 datapoints.

This is all configurable in the UI.
Details are here

[rereeser]

Great, thanks. I guess I initially misunderstood how summary indexing worked.