Splunk Search

How do Accelerated Searches work with retention policies?

Ricapar
Communicator

I'm trying to plan out retention policies, and I'm unsure about how they play alongside searches that I've marked as accelerated.

For example, if I have simple saved search like this, marked as accelerated:

index=mydata | timechart span=1d count by host

Search runs over the past year's data.

If the retention policy sets something like this in indexes.conf:

# Freeze after 90 days
frozenTimePeriodInSecs = 7776000

What happens there? Will the search acceleration keep the summarized data, and (eventually) let me see the "| timechart count by host" chart, even after the data has been frozen (deleted)?

If not... How would I go about doing something like that?

bandit
Motivator

hmm... possibly we can mix tsidx reduction introduced in 6.4.0 with report acceleration?
http://docs.splunk.com/Documentation/Splunk/latest/Indexer/Reducetsidxdiskusage

I'm thinking of something like the following:
- increase index retention from 2 months to 6 months. (i'm expecting this to increase disk utilization)
- use tsidx reduction (hoping this will reduce my disk utilization to somewhat offset the increase in retention time)
- enable report acceleration

Wondering if I would get 6 months of acceleration???

0 Karma

mattness
Splunk Employee
Splunk Employee

Unfortunately, frozen data is frozen--Splunk cannot include it in report acceleration summaries, because report acceleration summaries are tied to the indexed data at the index bucket level (they live in your primary index, in other words).

So you have two choices. If you want to use report acceleration over a year's span, you'll need to extend your retention policy from 90 days to a year. But if you can't do that, you might try summary indexing instead. This process summarizes your data in a separate summary index that can have a different retention policy than your primary index, or no retention policy at all.

http://docs.splunk.com/Documentation/Splunk/latest/Knowledge/Usesummaryindexing

bandit
Motivator

Would be really cool if we could get independent retention for acceleration summaries in a future version of Splunk. Who wants to go back to summary indexes?

0 Karma
Get Updates on the Splunk Community!

Check out this month’s brand new Splunk Lantern articles

Splunk Lantern is a customer success center providing advice from Splunk experts on valuable data insights, ...

Happy CX Day to our Community Superheroes!

Happy 10th Birthday CX Day!What is CX Day? It’s a global celebration recognizing innovation and success in the ...

Routing Data to Different Splunk Indexes in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...