Getting Data In

Remove Raw data from splunk server

npandith
Explorer

We are running splunk 4.2.3 on a RHEL 5.7 server and nearly 250 universal forwarders forwarding data to this splunk server. Right now we have 2 mounts created, 1 for hot/warm db's and 1 for colddbs. We are indexing appr. 80G of data everyday and space is filling up very fast and we have appr.1TB of data. Out of this raw data is consuming more space. I am planning to remove rawdata from the colddb. IS IT OK TO REMOVE THE RAWDATA FROM COLDDB? i guess splunk will not touch rawdata's.

output from one of the cold db-

ls -ltr

total 657436
-rw------- 1 root root 72262461 Apr 7 23:22 1331548047-1331389182-7634922573347700672.tsidx
-rw------- 1 root root 1695441 Apr 7 23:22 1331515778-1331389730-3545913347331342493.tsidx
-rw------- 1 root root 69248060 Apr 7 23:22 Strings.data
drwx------ 2 root root 4096 Apr 7 23:22 rawdata
-rw------- 1 root root 11557 Apr 7 23:22 Hosts.data
-rw------- 1 root root 14083668 Apr 7 23:22 1331515766-1331389660-4513270691130261649.tsidx
-rw------- 1 root root 0 Apr 7 23:22 splunk-need-optimize.dat
-rw------- 1 root root 71 Apr 7 23:22 splunk-autogen-params.dat
-rw------- 1 root root 4646 Apr 7 23:22 SourceTypes.data
-rw------- 1 root root 23812 Apr 7 23:22 Sources.data
-rw------- 1 root root 49 Apr 7 23:22 optimize.result
-rw------- 1 root root 72468285 Apr 7 23:22 merged_lexicon.lex
-rw------- 1 root root 442641753 Apr 7 23:22 1331547238-1331386067-4874605572483200482.tsidx

Tags (1)
0 Karma

yannK
Splunk Employee
Splunk Employee

Hum, not the best idea, removing raw data means that you will not be able to access the data after, therefore those cold buckets will be useless.

if you really want to delete cold buckets, then setup a retention policy (on total index size or on time retention.)

0 Karma

gkanapathy
Splunk Employee
Splunk Employee
0 Karma
Get Updates on the Splunk Community!

Synthetic Monitoring: Not your Grandma’s Polyester! Tech Talk: DevOps Edition

Register today and join TekStream on Tuesday, February 28 at 11am PT/2pm ET for a demonstration of Splunk ...

Instrumenting Java Websocket Messaging

Instrumenting Java Websocket MessagingThis article is a code-based discussion of passing OpenTelemetry trace ...

Announcing General Availability of Splunk Incident Intelligence!

Digital transformation is real! Across industries, companies big and small are going through rapid digital ...