How do I set retention for an index to 30 days?

[melonman]

Hi,

I am trying to configure retention period for an index.

I want to archive (compress) the indexed data after 3 months.
I want to delete the indexed data after 1 year.

for test, I set frozenTimePeriodInSecs to 600, but there is noting even in warmDB, coldDB, and frozenDB.

I have been reading documentations, but I still can not get exactly how to set retantion period in indexes.conf.

Could anyone please point me to this information?

Thank you,

[melonman]

I read this answer, but this does not solve my question...
Is there NO way to specify the exact retantion period by Splunk???

http://splunk-base.splunk.com/answers/2392/how-can-i-rotate-all-hot-and-warm-buckets-older-than-30-d...

[rgonzale6 · Accepted solution]

Does your indexes.conf clause for your index resemble something like this?

[whatever]
coldPath = *path*
homePath = *path*
thawedPath = *path*
coldToFrozenDir = *path*
frozenTimePeriodInSecs = 600

If you do not set a coldToFrozenDir in the indexes.conf clause once the frozenTimePeriodInSecs value is reached splunk simply deletes the data.

[melonman]

Now I have this, and see how this configuration works.
Some answers mentioned rebooting splunk will affect the number of warm buckets, so interested in the result.

maxDataSize = 1024 (bucket size is to be 1GB, or gib enough to hold 1 day indexing volume)
maxHotIdleSecs = 86400 (1 day, for hot to warm roll or call the holl-hot-bucket script)
maxWarmDBCount = 30 (30 buckets = 30days, for warm to cold)
frozenTimePeriodInSecs = 7776000 (90 days in sec, cold to frozen)
coldToFrozenDir = /archive/myindex ( after 90 days, index goes here)

Thank you all for helping me out!

[adamw]

You can manually roll hot buckets to warm if you'd like

$SPLUNK_HOME/bin/splunk _internal call /data/indexes/main/roll-hot-buckets -auth admin:password

[melonman]

Yes, I rebooted, but no luck. I think hotDB is not controlled by fronTmePeriodInSecs. Also noticed that Splunk's bucket mover is triggered when:
- backetsize is exceeded (hot to warm)
- backet number is excceeded (warm to cold)
- backet age is exceeded (cold to frozen/delete)

Seems very hard to set exactly 3 month retention.

[rgonzale6]

Did you restart splunk after making your indexes.conf changes? Do you see anything in your splunkd.log about 'attempting to freeze'?

[melonman]

Yes, I actually set the values same as your example with correct paths. However, there is still nothing in coldToFrozenDir after 600+ sec. Data is continuously generated and indexed. Any idea?