Data Retention for different log types

[dbcottam]

According to the splunk documentation splunk can retain then delete data for a specified interval of time. I have multiple different log files with different retention requirements.

For example our Business Information Management System (BIM) designates Retention Policies for various types of data.  E-mail is 4 months other data might be weeks or even years.  How does Splunk handle this?

Thank you.

[sdaniels · Accepted solution]

You need to create different indexes in order to meet your retention requirements for various data sources. For each index you will be able to control how long you want the data.

http://docs.splunk.com/Documentation/Splunk/latest/admin/HowSplunkstoresindexes

Create a new index and then you can assign that index to one of your data sources in inputs.conf.
index=something. When the data comes in, it will be stored in that index. You can then use the indexes.conf file to change the per index options.

Look at indexes.conf details here - http://docs.splunk.com/Documentation/Splunk/latest/admin/indexesconf

[sdaniels]

Follow this:

http://docs.splunk.com/Documentation/Splunk/latest/admin/Setupmultipleindexes

In the UI whatever context you are in (whichever app) when you click on the Manager link, that is where it will create or modify the indexes.conf file. Look in the 'local' directory under the app. So if you were in the Search app and created an index you'd find the changes in /etc/apps/search/local/. Then use the documentation and/or splunkbase to see how you want to change settings for the retention.

[dbcottam]

another thing: I searched my splunk parent directory and there are many indexes.conf files. Which one do I use to alter override the defaults?

[dbcottam]

Thank you, I have been exploring the files, and found where to create the indexes in inputs.conf. However, I cannot find where it links in the indexes.conf file. What I think is the index is this "[default]" and if I wanted to change the index it would similar to this "[email]"

Am I on the right path?
Thanks