Is there a way to delete old data from wide bucket spans

[sonicZ]

Looking to expire some really old 2005-2006 data in some of our older buckets with wide bucket spans, some stretch from earliest date-2011 to oldest 2006.
This index is taking up a few terrabytes of cold storage and i would like to reduce it

In the past i recall we cannot expire buckets if the earliest events are within the frozenTimePeriodInSecs value, is there a way to just delete the old events?

here's the bucket examples

drwx--x--x 4 root root 2048 Oct  5  2011 db_1316171253_1151333680_1591
drwx--x--x 4 root root 2048 Oct  5  2011 db_1316186570_1148764640_1593
drwx--x--x 4 root root 1024 Oct  5  2011 db_1316187080_1163161371_1585
drwx--x--x 4 root root 1024 Oct  5  2011 db_1316215973_1146088931_1595
drwx--x--x 4 root root 1024 Oct  5  2011 db_1316217599_1149873438_1587
drwx--x--x 4 root root 1024 Oct  5  2011 db_1316261550_1143637536_1597
drwx--x--x 4 root root 1024 Oct  6  2011 db_1316262900_1147121232_1596
drwx--x--x 4 root root 1024 Oct  6  2011 db_1316275823_1144749944_1594
drwx--x--x 4 root root 1024 Oct  6  2011 db_1316283607_1143766728_1598
drwx--x--x 4 root root 1024 Oct  6  2011 db_1316303999_1143409080_1592
drwx--x--x 4 root root 1024 Oct  6  2011 db_1316317510_1143947712_1599
drwx--x--x 4 root root 1024 Oct  6  2011 db_1316351098_1143646976_1600
drwx--x--x 4 root root 1024 Oct  6  2011 db_1316370771_1144292776_1601
drwx--x--x 4 root root 1024 Oct  6  2011 db_1316423496_1143689504_1603
drwx--x--x 4 root root 1024 Oct  6  2011 db_1316437356_1144262888_1604
drwx--x--x 4 root root 1024 Oct  7  2011 db_1316512974_1146488376_1609
drwx--x--x 3 root root 1024 Oct  7  2011 db_1314489599_1272192960_1719
drwx--x--x 4 root root 1024 Oct  7  2011 db_1316523360_1144554816_1608
drwx--x--x 4 root root 1024 Oct  7  2011 db_1316563199_1143628240_1602
drwx--x--x 4 root root 1024 Oct  7  2011 db_1316563199_1144056216_1605
drwx--x--x 4 root root 1024 Oct  7  2011 db_1316563199_1146463960_1606
drwx--x--x 4 root root 1024 Oct  7  2011 db_1316563199_1145669992_1607
drwx--x--x 4 root root 1024 Oct  7  2011 db_1316581963_1144563144_1613
drwx--x--x 4 root root 1024 Oct  7  2011 db_1316600608_1144622992_1614
drwx--x--x 4 root root 2048 Oct  7  2011 db_1316614176_1146094296_1615
drwx--x--x 4 root root 1024 Oct  7  2011 db_1316627896_1145345824_1616
drwx--x--x 4 root root 1024 Oct  7  2011 db_1316649599_1144204064_1610
drwx--x--x 4 root root 1024 Oct  7  2011 db_1316649599_1150239440_1611
drwx--x--x 4 root root 1024 Oct  8  2011 db_1316649599_1143870472_1612
drwx--x--x 4 root root 1024 Oct  8  2011 db_1316676544_1143990200_1618
drwx--x--x 4 root root 1024 Oct  8  2011 db_1316696944_1144613920_1619
drwx--x--x 4 root root 1024 Oct  8  2011 db_1316707239_1146104336_1620

[Drainy]

Sadly you can't delete data from Splunk once its been indexed. Using the delete command will just flag it for removal from your search results.
Best bet is to make sure your retention policy matches what you want to achive, if you are keeping data for years when you only need it for one or two then you need to reconfigure your indexes, have a look at;

http://docs.splunk.com/Documentation/Splunk/latest/admin/Setaretirementandarchivingpolicy

[dwaddle]

It may be worth looking at a new project one of the Splunkers is working on, where you could archive off some of these buckets to reclaim space but still have them available as needed -- http://blogs.splunk.com/2012/07/02/shuttl-for-big-data-archiving/

[sonicZ]

Hey Drainy, yeah i was hoping something might have changed with the | delete command to reclaim space, that's my main goal is getting back precious disk blocks 🙂

Our current retention policy is good, (manager wants to keep 1-2 yrs worth of cold storage data) it's that these older buckets were around before my time. Splunks retention policy cant delete a bucket if it's newest event does not fall into the retirement policy date range so some of those buckets i show above span db_Sep2011_Jun2006 so this is a pain.

[rturk]

Hi sonicZ,

The delete (LINK) function may be of interest to you. It may be worthwhile re-visiting your bucket sizing strategy as well.

More details found here: http://splunk-base.splunk.com/answers/46401/how-to-delete-old-date-from-splunk

Hope this helps 🙂