topic Re: Use _indextime for data retention policy? in Monitoring Splunk

Use _indextime for data retention policy?

jinxkamada — Thu, 04 Jul 2024 16:56:59 GMT

Is it possible to reconfigure Splunk to use _indextime instead of _time for data retention policy?

Re: Use _indextime for data retention policy?

richgalloway — Thu, 04 Jul 2024 18:44:19 GMT

That is not possible.

What problem are you trying to solve?

Re: Use _indextime for data retention policy?

jinxkamada — Fri, 05 Jul 2024 08:39:25 GMT

I am regularly uploading a file over Splunk which I use for my reporting (might include events with various timestamp).

I don't want data retention policy to be bounded by _time since it might affect the report I am creating.
I definitely want Splunk to delete older uploads based on indexed time to manage my server disk usage.

Any suggestion will be appreciated!

Re: Use _indextime for data retention policy?

PickleRick — Sat, 06 Jul 2024 11:28:31 GMT

Can't you use time of ingestion as _time (which would influence retention) and use another field for storing your event's time (in this case it could make sense to make it an indexed field).

Buckets are rolled based on either age of data within the bucket (in terms of _time) or index size. That's it.

Re: Use _indextime for data retention policy?

isoutamo — Sat, 06 Jul 2024 14:35:09 GMT

Or is there a possibility to use separate index for those events and afterwards even swipe out that content?

Anyhow as @PickleRick said a bucket is removed after all events inside it has expired. Using old and new data (timestamp/_time point of view) usually make this to take quite long time.

Re: Use _indextime for data retention policy?

jinxkamada — Mon, 08 Jul 2024 08:20:14 GMT

Most of the indexes we have right now are used for the same purpose of creating a report.

It would have been ideal for me if they can use the Search menu (out-of-the-box UI) so it would save me the hassle of creating a custom UI to point datetime filtering to the correct field.

However, I would say your recommendations do make sense.
Thanks for the inputs!