Getting Data In

Remove Raw data from splunk server

npandith
Explorer

We are running splunk 4.2.3 on a RHEL 5.7 server and nearly 250 universal forwarders forwarding data to this splunk server. Right now we have 2 mounts created, 1 for hot/warm db's and 1 for colddbs. We are indexing appr. 80G of data everyday and space is filling up very fast and we have appr.1TB of data. Out of this raw data is consuming more space. I am planning to remove rawdata from the colddb. IS IT OK TO REMOVE THE RAWDATA FROM COLDDB? i guess splunk will not touch rawdata's.

output from one of the cold db-

ls -ltr

total 657436
-rw------- 1 root root 72262461 Apr 7 23:22 1331548047-1331389182-7634922573347700672.tsidx
-rw------- 1 root root 1695441 Apr 7 23:22 1331515778-1331389730-3545913347331342493.tsidx
-rw------- 1 root root 69248060 Apr 7 23:22 Strings.data
drwx------ 2 root root 4096 Apr 7 23:22 rawdata
-rw------- 1 root root 11557 Apr 7 23:22 Hosts.data
-rw------- 1 root root 14083668 Apr 7 23:22 1331515766-1331389660-4513270691130261649.tsidx
-rw------- 1 root root 0 Apr 7 23:22 splunk-need-optimize.dat
-rw------- 1 root root 71 Apr 7 23:22 splunk-autogen-params.dat
-rw------- 1 root root 4646 Apr 7 23:22 SourceTypes.data
-rw------- 1 root root 23812 Apr 7 23:22 Sources.data
-rw------- 1 root root 49 Apr 7 23:22 optimize.result
-rw------- 1 root root 72468285 Apr 7 23:22 merged_lexicon.lex
-rw------- 1 root root 442641753 Apr 7 23:22 1331547238-1331386067-4874605572483200482.tsidx

Tags (1)
0 Karma

yannK
Splunk Employee
Splunk Employee

Hum, not the best idea, removing raw data means that you will not be able to access the data after, therefore those cold buckets will be useless.

if you really want to delete cold buckets, then setup a retention policy (on total index size or on time retention.)

0 Karma

gkanapathy
Splunk Employee
Splunk Employee
0 Karma
Get Updates on the Splunk Community!

Register to Attend BSides SPL 2022 - It's all Happening October 18!

Join like-minded individuals for technical sessions on everything Splunk!  This is a community-led and run ...

What's New in Splunk Cloud Platform 9.0.2208?!

Howdy!  We are happy to share the newest updates in Splunk Cloud Platform 9.0.2208! Analysts can benefit ...

Admin Console: A Single, Unified Interface for All Your Cloud Admin Needs

WATCH NOWJoin us to learn how the admin console can save you time and give you more control over the Splunk® ...