Getting Data In

Solution for: How to find and delete OLD Frozen data by older than X days

Mehran_Safari
Explorer

this bash script will search frozen path you give + oldest needed time then will show older logs and asks you to remove them.

it shows you path + size + start and end time of logs each bucket contains

this will find logs forexample older than 30 days and will ask you to remove them if you agree

this script detects logs with wrong time ( logtime > current time) too

 

 

 

 

 

#!/bin/bash
clear
echo  "############################"
echo  "##created.by mehran.safari##"
echo  "##        2022            ##"
echo  "############################"
##############
echo -n " Enter index name to lookup:"
read INAME
####
FROZENPATH="/frozendata"
echo " Default Splunk Frozen Indexes Path is "$FROZENPATH". is it ok? (y to continue or n to give new path):"
read  ANSWER1;
case "$ANSWER1" in
"y")
echo -e "OK Deafult Frozen Index Path Selected.";;
"n")
echo -e "Enter NEW Frozen Index Path:";
read FROZENPATH;;
esac
####
find "$FROZENPATH/$INAME" -type d -iname "db_*" -print > "./frozendb.txt"
ODATE=30
echo " oldest Frozen Bucket Should be "$ODATE" days old. is it ok?(press "y" to continue & "n" to change it):"
read ANSWER3
case $ANSWER3 in
y )
echo -e "OK Default Frozen Age Kept.";
break;;
n )
echo -e "Enter NEW Frozen AGE You Want:";
read ODATE; 
break;;
esac
BODATE=$(date --date="`date`-"$ODATE"days" +%s)
BCDATE=`date +%s`
#############
FILE1='./frozendb.txt'
 while read line; do
          LOGSTART=`echo $line | cut -d "_" -f3`;
          LOGEND=`echo $line | cut -d "_" -f2`;
if [[ $LOGEND -gt $BCDATE || $LOGSTART -lt $BODATE ]]; then
echo -e "******************************"
echo -e "Frozen Log Path You want: $line"
HLOGSTART=`date -d @"$LOGSTART"`
HLOGEND=`date -d @"$LOGEND"`
LOGSIZE=`du -hs "$line" | cut -d "/" -f1`
echo -e "*** this Bucket contains logs from: $HLOGSTART"
echo -e "*** this Bucket contains logs to: $HLOGEND "
echo -e "**** The Size Of This Log Is: $LOGSIZE"
echo -e "$line" >> "./frozenmatched.txt"
echo -e "******************************"
fi
done<$FILE1
############
sudo rm -rf "./frozendb.txt"
echo "Do you Want to DELETE this Logs?(y to DELETE): "
read  ANSWER3
FILE2='./frozenmatched.txt'
if [[ "$ANSWER3" == "y" ]]; then
while read line2; do
        sudo rm -rf "$line2"
        echo -e "DELETING of $line2 DONE."
done<$FILE2
fi
sudo rm -rf "./frozenmatched.txt"
##########
echo     "################################"
echo  -e "## GOOD LUCk WITH BEST REGARDS##"
echo     "################################"
#########

 

 

 

 

 

this is github link if you want https://github.com/mehransafari/Splunk_Frozen_Cleanup

Labels (1)
Tags (2)
Get Updates on the Splunk Community!

A Guide To Cloud Migration Success

As enterprises’ rapid expansion to the cloud continues, IT leaders are continuously looking for ways to focus ...

Join Us for Splunk University and Get Your Bootcamp Game On!

If you know, you know! Splunk University is the vibe this summer so register today for bootcamps galore ...

.conf24 | Learning Tracks for Security, Observability, Platform, and Developers!

.conf24 is taking place at The Venetian in Las Vegas from June 11 - 14. Continue reading to learn about the ...