topic Re: need basic directions for cleaning out indexed data in Dashboards & Visualizations

need basic directions for cleaning out indexed data

jchilovich — Tue, 16 Apr 2013 19:11:23 GMT

I'm new to the tool and just attemtping to build a POC. I have limited disk space and have reached tis limit. tried command "./splunk clean eventdata -index <???>" but
1) not positive as to how to determine what index files I should look for to finish the argument. Where do I identify each index?
2) I get folloiwng message "In order to clean, Splunked must not be running" - How do I 'turn it off'?
3) when running the above clean command, do I run this from the /bin or do I run from a different library?

Re: need basic directions for cleaning out indexed data

kristian_kolb — Tue, 16 Apr 2013 19:22:33 GMT

1) when logged in, you can find the indexes, in Manager (top right) -> Indexes (middle left). You have probably all your events in the index called 'main', which is the default index.

To run the clean command, you do not need to know the directory or filenames of where the data is stored. Only the name of the index.

2) /opt/splunk/bin/splunk stop

3) see 2)

Re: need basic directions for cleaning out indexed data

jchilovich — Thu, 18 Apr 2013 17:38:38 GMT

Thank you. This cleared out what I needed. Can I follow up with yet another question?

What kind of script do I need (or whatever the process would be) to clean out all data after say 7 days? A weeks worth of data should be more than enough for any RCA needed.

Re: need basic directions for cleaning out indexed data

stefandagerman — Thu, 18 Apr 2013 18:39:10 GMT

You don't need a script.
May I suggest reading up on how to configure your index(es) to define data retention?
This may be helpful:
http://docs.splunk.com/Documentation/Splunk/5.0.2/Indexer/HowSplunkstoresindexes
http://wiki.splunk.com/Deploy:BucketRotationAndRetention

Re: need basic directions for cleaning out indexed data

kristian_kolb — Thu, 18 Apr 2013 18:47:26 GMT

Well, the easiest would be to set the limit on the amount of data, if you are not yet up to speed on config files.

Go into Manager -> Indexes and select the index you're putting data into (main most likely). There you can set the total amount of data the index can hold (in MB).

If you want to set the retention on age, you'll have to create/edit a config file called indexes.conf in /opt/splunk/etc/system/local, roughly like this;

[main] frozenTimePeriodInSecs = xxx

Be aware though that messing with this file and getting it wrong, can result in an unusable system or loss of data.

Re: need basic directions for cleaning out indexed data

kristian_kolb — Thu, 18 Apr 2013 18:49:01 GMT

where xxx would be seconds; 60 * 60 * 24 * 7 = 604800