Getting Data In

Remove Raw data from splunk server

npandith
Explorer

We are running splunk 4.2.3 on a RHEL 5.7 server and nearly 250 universal forwarders forwarding data to this splunk server. Right now we have 2 mounts created, 1 for hot/warm db's and 1 for colddbs. We are indexing appr. 80G of data everyday and space is filling up very fast and we have appr.1TB of data. Out of this raw data is consuming more space. I am planning to remove rawdata from the colddb. IS IT OK TO REMOVE THE RAWDATA FROM COLDDB? i guess splunk will not touch rawdata's.

output from one of the cold db-

ls -ltr

total 657436
-rw------- 1 root root 72262461 Apr 7 23:22 1331548047-1331389182-7634922573347700672.tsidx
-rw------- 1 root root 1695441 Apr 7 23:22 1331515778-1331389730-3545913347331342493.tsidx
-rw------- 1 root root 69248060 Apr 7 23:22 Strings.data
drwx------ 2 root root 4096 Apr 7 23:22 rawdata
-rw------- 1 root root 11557 Apr 7 23:22 Hosts.data
-rw------- 1 root root 14083668 Apr 7 23:22 1331515766-1331389660-4513270691130261649.tsidx
-rw------- 1 root root 0 Apr 7 23:22 splunk-need-optimize.dat
-rw------- 1 root root 71 Apr 7 23:22 splunk-autogen-params.dat
-rw------- 1 root root 4646 Apr 7 23:22 SourceTypes.data
-rw------- 1 root root 23812 Apr 7 23:22 Sources.data
-rw------- 1 root root 49 Apr 7 23:22 optimize.result
-rw------- 1 root root 72468285 Apr 7 23:22 merged_lexicon.lex
-rw------- 1 root root 442641753 Apr 7 23:22 1331547238-1331386067-4874605572483200482.tsidx

Tags (1)
0 Karma

yannK
Splunk Employee
Splunk Employee

Hum, not the best idea, removing raw data means that you will not be able to access the data after, therefore those cold buckets will be useless.

if you really want to delete cold buckets, then setup a retention policy (on total index size or on time retention.)

0 Karma

gkanapathy
Splunk Employee
Splunk Employee
0 Karma
Get Updates on the Splunk Community!

Splunk Community Platform Survey

Hey Splunk Community, Starting today, the community platform may prompt you to participate in a survey. The ...

Observability Highlights | November 2022 Newsletter

 November 2022Observability CloudEnd Of Support Extension for SignalFx Smart AgentSplunk is extending the End ...

Avoid Certificate Expiry Issues in Splunk Enterprise with Certificate Assist

This blog post is part 2 of 4 of a series on Splunk Assist. Click the links below to see the other ...