Getting Data In

Solution for: How to find and delete OLD Frozen data by older than X days

Mehran_Safari
Explorer

this bash script will search frozen path you give + oldest needed time then will show older logs and asks you to remove them.

it shows you path + size + start and end time of logs each bucket contains

this will find logs forexample older than 30 days and will ask you to remove them if you agree

this script detects logs with wrong time ( logtime > current time) too

 

 

 

 

 

#!/bin/bash
clear
echo  "############################"
echo  "##created.by mehran.safari##"
echo  "##        2022            ##"
echo  "############################"
##############
echo -n " Enter index name to lookup:"
read INAME
####
FROZENPATH="/frozendata"
echo " Default Splunk Frozen Indexes Path is "$FROZENPATH". is it ok? (y to continue or n to give new path):"
read  ANSWER1;
case "$ANSWER1" in
"y")
echo -e "OK Deafult Frozen Index Path Selected.";;
"n")
echo -e "Enter NEW Frozen Index Path:";
read FROZENPATH;;
esac
####
find "$FROZENPATH/$INAME" -type d -iname "db_*" -print > "./frozendb.txt"
ODATE=30
echo " oldest Frozen Bucket Should be "$ODATE" days old. is it ok?(press "y" to continue & "n" to change it):"
read ANSWER3
case $ANSWER3 in
y )
echo -e "OK Default Frozen Age Kept.";
break;;
n )
echo -e "Enter NEW Frozen AGE You Want:";
read ODATE; 
break;;
esac
BODATE=$(date --date="`date`-"$ODATE"days" +%s)
BCDATE=`date +%s`
#############
FILE1='./frozendb.txt'
 while read line; do
          LOGSTART=`echo $line | cut -d "_" -f3`;
          LOGEND=`echo $line | cut -d "_" -f2`;
if [[ $LOGEND -gt $BCDATE || $LOGSTART -lt $BODATE ]]; then
echo -e "******************************"
echo -e "Frozen Log Path You want: $line"
HLOGSTART=`date -d @"$LOGSTART"`
HLOGEND=`date -d @"$LOGEND"`
LOGSIZE=`du -hs "$line" | cut -d "/" -f1`
echo -e "*** this Bucket contains logs from: $HLOGSTART"
echo -e "*** this Bucket contains logs to: $HLOGEND "
echo -e "**** The Size Of This Log Is: $LOGSIZE"
echo -e "$line" >> "./frozenmatched.txt"
echo -e "******************************"
fi
done<$FILE1
############
sudo rm -rf "./frozendb.txt"
echo "Do you Want to DELETE this Logs?(y to DELETE): "
read  ANSWER3
FILE2='./frozenmatched.txt'
if [[ "$ANSWER3" == "y" ]]; then
while read line2; do
        sudo rm -rf "$line2"
        echo -e "DELETING of $line2 DONE."
done<$FILE2
fi
sudo rm -rf "./frozenmatched.txt"
##########
echo     "################################"
echo  -e "## GOOD LUCk WITH BEST REGARDS##"
echo     "################################"
#########

 

 

 

 

 

this is github link if you want https://github.com/mehransafari/Splunk_Frozen_Cleanup

Labels (1)
Tags (2)
Get Updates on the Splunk Community!

Community Content Calendar, November Edition

Welcome to the November edition of our Community Spotlight! Each month, we dive into the Splunk Community to ...

October Community Champions: A Shoutout to Our Contributors!

As October comes to a close, we want to take a moment to celebrate the people who make the Splunk Community ...

Stay Connected: Your Guide to November Tech Talks, Office Hours, and Webinars!

What are Community Office Hours? Community Office Hours is an interactive 60-minute Zoom series where ...