Deployment Architecture

script to convert the bucket name to the time range of the data

Splunk Employee
Splunk Employee

I have plenty of frozen buckets, and I want to thawed some of them.
How do figure the timerange of each bucket from the folder name with a script.

example on linux :


ls -l /splunk/myindex/frozendb/
drwx--x--x 13 yannk staff 442 Dec 25 12:04 db_1356465674_1356465287_0
drwx--x--x 13 yannk staff 442 Jan 7 00:48 db_1356465863_1356465863_1

PS: for the hot/warm/cold I use the splunk command "|dbinspect index=myindex"

Tags (2)
1 Solution

SplunkTrust
SplunkTrust

🙂

$ ls -l | fgrep db_ | sed -r s/.*\(db_\(.+\)_\(.+\)_.*\)/\\1\\n\\2\\n\\3/ | gawk '/^db/ {print $0} /^1/ {print "   " strftime("%c", $0)}'

Produces this for your example:

db_1356465674_1356465287_0
   Tue Dec 25 21:01:14 2012
   Tue Dec 25 20:54:47 2012
db_1356465863_1356465863_1
   Tue Dec 25 21:04:23 2012
   Tue Dec 25 21:04:23 2012

Will break once we roll over 2000000000...

View solution in original post

SplunkTrust
SplunkTrust

This one is a little faster when dealing with lots of buckets, and it's also more accurate when working with clustered data.

Also, I switched the fields so the start time is before the end time.

ls -d1 db_*  | gawk -F'_' '{print $0} {print "   " strftime("%c", $3)} {print "   " strftime("%c", $2)}'

Path Finder

Hi,

great stuff. Had some problems to run this on a Mac PB using OSX 10.7 🙂

Here is my version:

lf=$'\n' ; ls -ld db_* rb_* | sed -E s/.*\(db_\(.+\)_\(.+\)_.*\)/\\1"\\$lf"\\2"\\$lf"\\3/ | awk '/^db/ {print " "$0" "} /^1/ {cmd="date -r " $0; printf "     "; system(cmd) }'

Explanation: sed on MacOS X does not support "\n" and awk does not support strftime()...

Feel free to add some optimizations.

0 Karma

SplunkTrust
SplunkTrust

🙂

$ ls -l | fgrep db_ | sed -r s/.*\(db_\(.+\)_\(.+\)_.*\)/\\1\\n\\2\\n\\3/ | gawk '/^db/ {print $0} /^1/ {print "   " strftime("%c", $0)}'

Produces this for your example:

db_1356465674_1356465287_0
   Tue Dec 25 21:01:14 2012
   Tue Dec 25 20:54:47 2012
db_1356465863_1356465863_1
   Tue Dec 25 21:04:23 2012
   Tue Dec 25 21:04:23 2012

Will break once we roll over 2000000000...

View solution in original post

SplunkTrust
SplunkTrust

If you want to swap the dates you can just swap the 2 and 3 in the sed command.

Splunk Employee
Splunk Employee

reminder the format of the buckets is
db_(recentevent)(oldestevent)(id)

or for the hot buckets
hot_v1_(id)

0 Karma

Splunk Employee
Splunk Employee

thank you very much, I was trying to do the same , but a oneliner is much better.

0 Karma