Remove Raw data from splunk server

npandith — Mon, 09 Apr 2012 01:46:06 GMT

We are running splunk 4.2.3 on a RHEL 5.7 server and nearly 250 universal forwarders forwarding data to this splunk server. Right now we have 2 mounts created, 1 for hot/warm db's and 1 for colddbs. We are indexing appr. 80G of data everyday and space is filling up very fast and we have appr.1TB of data. Out of this raw data is consuming more space. I am planning to remove rawdata from the colddb. IS IT OK TO REMOVE THE RAWDATA FROM COLDDB? i guess splunk will not touch rawdata's.

output from one of the cold db-

ls -ltr

total 657436
-rw------- 1 root root 72262461 Apr 7 23:22 1331548047-1331389182-7634922573347700672.tsidx
-rw------- 1 root root 1695441 Apr 7 23:22 1331515778-1331389730-3545913347331342493.tsidx
-rw------- 1 root root 69248060 Apr 7 23:22 Strings.data
drwx------ 2 root root 4096 Apr 7 23:22 rawdata
-rw------- 1 root root 11557 Apr 7 23:22 Hosts.data
-rw------- 1 root root 14083668 Apr 7 23:22 1331515766-1331389660-4513270691130261649.tsidx
-rw------- 1 root root 0 Apr 7 23:22 splunk-need-optimize.dat
-rw------- 1 root root 71 Apr 7 23:22 splunk-autogen-params.dat
-rw------- 1 root root 4646 Apr 7 23:22 SourceTypes.data
-rw------- 1 root root 23812 Apr 7 23:22 Sources.data
-rw------- 1 root root 49 Apr 7 23:22 optimize.result
-rw------- 1 root root 72468285 Apr 7 23:22 merged_lexicon.lex
-rw------- 1 root root 442641753 Apr 7 23:22 1331547238-1331386067-4874605572483200482.tsidx

Re: Remove Raw data from splunk server

yannK — Mon, 09 Apr 2012 06:06:12 GMT

Hum, not the best idea, removing raw data means that you will not be able to access the data after, therefore those cold buckets will be useless.

if you really want to delete cold buckets, then setup a retention policy (on total index size or on time retention.)

topic Remove Raw data from splunk server in Getting Data In

Remove Raw data from splunk server

ls -ltr

Re: Remove Raw data from splunk server

Re: Remove Raw data from splunk server