Index searchable retentions questions

[cbiraris]

Hi team,

i have a index with 4 sourcetype.  index has searchable retention of 4 months.

is there any way we can keep same retention for 3 sourcetype and 1sourcetype can be increased to 8 months ?

For example:

Index=xyz	sourcetype = 1	searchable retention 4 Months
	sourcetype = 2	searchable retention 4 Months
	sourcetype = 3	searchable retention 4 Months
	sourcetype = 4	searchable retention 8 Months

[kiran_panchavat]

@cbiraris 

In Splunk, retention policies are set at the index level, not at the sourcetype level. This means that all sourcetypes within a single index (like your xyz index) will inherit the same retention period 4 months in your case. Unfortunately, there’s no native way to assign different retention periods to individual sourcetypes within the same index.

[PickleRick]

As others already pointed out - no. So you've just hit one of the main reasons for splitting data into indexes. There are two main factors when deciding whether you want the data in single index or multiple ones:

1) Data retention settings (and that's your case)

2) Access control

Both of those work at index level.

There are some other things which might come into play in some border cases (like not mixing high-volume and low-volume data in a single index) but you get much less often that deeply into  data architecture.

[livehybrid]

Hi @cbiraris 

Check out https://lantern.splunk.com/Splunk_Platform/Product_Tips/Data_Management/Setting_data_retention_rules.... for some guidance on best practices for retention.

When you consider which index should collect a data source, remember that you set retention policies by index. If you have two data sources, one that you need to keep for 3 years and one that you can discard after 30 days, send them to separate indexes. Otherwise, you will be paying to store 35 months of data you don’t really want, or discarding data 35 months too early.

Essentially you should split your data into different indexes when you have different retention, permissions or usecase/category. Its a good idea to use a naming convention to achieve this so you can easily distinguish between different types. Such as adding a _nonprod or _prod suffix for non-production/production data - which might have different RBAC / Users.

Please let me know how you get on and consider adding karma to this or any other answer if it has helped.
Regards

Will

[gcusello]

Hi @cbiraris ,

in Splunk retention is only defined at index level, so the only way is to store the longer sourcetype in a different index.

Ciao.

Giuseppe

[vsommer]

Hi @cbiraris,

unfortunately retention time can only be applied to indexes.