Getting Data In

Remove Raw data from splunk server

npandith
Explorer

We are running splunk 4.2.3 on a RHEL 5.7 server and nearly 250 universal forwarders forwarding data to this splunk server. Right now we have 2 mounts created, 1 for hot/warm db's and 1 for colddbs. We are indexing appr. 80G of data everyday and space is filling up very fast and we have appr.1TB of data. Out of this raw data is consuming more space. I am planning to remove rawdata from the colddb. IS IT OK TO REMOVE THE RAWDATA FROM COLDDB? i guess splunk will not touch rawdata's.

output from one of the cold db-

ls -ltr

total 657436
-rw------- 1 root root 72262461 Apr 7 23:22 1331548047-1331389182-7634922573347700672.tsidx
-rw------- 1 root root 1695441 Apr 7 23:22 1331515778-1331389730-3545913347331342493.tsidx
-rw------- 1 root root 69248060 Apr 7 23:22 Strings.data
drwx------ 2 root root 4096 Apr 7 23:22 rawdata
-rw------- 1 root root 11557 Apr 7 23:22 Hosts.data
-rw------- 1 root root 14083668 Apr 7 23:22 1331515766-1331389660-4513270691130261649.tsidx
-rw------- 1 root root 0 Apr 7 23:22 splunk-need-optimize.dat
-rw------- 1 root root 71 Apr 7 23:22 splunk-autogen-params.dat
-rw------- 1 root root 4646 Apr 7 23:22 SourceTypes.data
-rw------- 1 root root 23812 Apr 7 23:22 Sources.data
-rw------- 1 root root 49 Apr 7 23:22 optimize.result
-rw------- 1 root root 72468285 Apr 7 23:22 merged_lexicon.lex
-rw------- 1 root root 442641753 Apr 7 23:22 1331547238-1331386067-4874605572483200482.tsidx

Tags (1)
0 Karma

yannK
Splunk Employee
Splunk Employee

Hum, not the best idea, removing raw data means that you will not be able to access the data after, therefore those cold buckets will be useless.

if you really want to delete cold buckets, then setup a retention policy (on total index size or on time retention.)

0 Karma

gkanapathy
Splunk Employee
Splunk Employee
0 Karma
Get Updates on the Splunk Community!

Why You Can't Miss .conf25: Unleashing the Power of Agentic AI with Splunk & Cisco

The Defining Technology Movement of Our Lifetime The advent of agentic AI is arguably the defining technology ...

Deep Dive into Federated Analytics: Unlocking the Full Power of Your Security Data

In today’s complex digital landscape, security teams face increasing pressure to protect sprawling data across ...

Your summer travels continue with new course releases

Summer in the Northern hemisphere is in full swing, and is often a time to travel and explore. If your summer ...