How do I keep my saved search artifacts available longer?

[chustar]

I have some splunk dashboards and websites that are powered by a lot of aggregated data.

The data are presented in the form:

| Field1 | Field2 | ... | AvgValue1 | AvgValue2 | AllUniqueIds |

While a search like this works:

stats avg(Value1), avg(Value2), ..., list(UniqueId) by Field1, Field2, ...

In order to get our searches running as fast as possible, we process our data by first splitting this into two accelerated searches.

Accelerated_Search_A combines just the data:

| UniqueId | Value1 | Value2 | ... | AvgValue1 | AvgValue2 | ... | SumValue1 | SumValue2 | ... |

And Accelerated_Search_B just the metadata:

| UniqueId | Field1 | Field2 | ... |

These searches are then aggregated in a scheduled saved search Scheduled_Search:

 | savedsearch "Accelerated_Search_A" 
    | append [ savedsearch "Accelerated_Search_B"] 
    | stats list(UniqueId), avg(Value1), avg(Value2),... by Field1, Field2, ...

This works for now, however changing the Scheduled_Search from 14 days to two weeks changed how long it takes to run from is only limited to two weeks and increasing that to a month causes the search to take too long (from 30 minutes to about 4 hours).

Is there another way to increase how much data is available in our saved search?

[somesoni2]

You should consider using summary indexing (with summary index having appropriate retention to cater all your time ranges) to store the aggregated search results to an index. More info on summary indexing here: http://docs.splunk.com/Documentation/Splunk/6.6.2/Knowledge/Usesummaryindexing