Archive

How to generate a report on multiple indexes?

Builder

There's a limitation in the dbinspect command where you cannot specify multiple indexes to report on, therefore reporting on an installation where multiple indexes are used can be a time consuming thing.

This answers article is a good start, but unfortunately you can only do one index at a time. How do I get around this?

Tags (2)
1 Solution

Builder

This PERL script will generate a csv report and email it (assuming all required packages are installed) to specified email address.

Only thing required is the Shared utilities package (in my case it was sharutils-4.6.1-2.x86_64) for the uuencode portion.

You will need to modify some of the variables (mainly the $username and $password) if you want it to automatically log you in.

!/usr/bin/perl

### Set variables
$splunk_bin_dir="/opt/splunk/bin";
$mail_to="$ARGV[0]";
$header="Index Name,DB Type,earliest event time,latest event time, size (mb)";
$output_dir="/tmp";
$report_name="index_report.csv";
$username="admin";
$password="passwordhere!";

### Delete previous versions of the report

$output_name="> ${output_dir}/${report_name}";

open(OUTPUT,$output_name);


### Get list of indexes
@index_config_raw=`cat /opt/splunk/etc/system/local/indexes.conf`;

foreach $line (@index_config_raw) {
        chomp $line;

        if ($line=~m/\[/) {
                $line=~m/\[(.*)\]/g;
                $raw_index=$1;

                push(@indexes,$raw_index);

        }
}

print OUTPUT "$header \n";

### Processes indexes
foreach $index (@indexes) {
        chomp $index;

        $splunk_command="${splunk_bin_dir}/splunk search \"| dbinspect index=\"${index}\" timeformat=\"\%s\" | rename state as category | stats min(earliestTime) as earliestTime max(latestTime) as latestTime sum(sizeOnDiskMB) as MB by category | convert timeformat=\"\%m/\%d/\%Y\" ctime(earliestTime) as earliestTime ctime(latestTime) as latestTime\" -auth ${username}:${password}| grep -v \"category\" | grep -v \"-\" ";


        @result=`${splunk_command}`;

        if ($#result ne "-1") {
                foreach $return (@result) {
                        chomp $return;

                        $return=~m/(hot|warm|cold|frozen)\s+([\d]+\/[\d]+\/[\d]+)\s+([\d]+\/[\d]+\/[\d]+)\s+([\d]+\.[\d]+)/gi;

                        $db_type=$1;
                        $earliest_event=$2;
                        $latest_event=$3;
                        $size=$4;

                        print OUTPUT "$index,$db_type,$earliest_event,$latest_event,$size \n";


                }
        }

        if ($#result eq "-1") {
                print OUTPUT "$index,Empty Index,Empty Index,Empty Index \n";
        }


}

if ($mail_to ne "") {
        `uuencode ${output_dir}/${report_name} ${output_dir}/${report_name} | mailx -s \"Splunk Index Report\" $mail_to`;
}

View solution in original post

Splunk Employee
Splunk Employee

I will also note that dbinspect does not work in distributed search mode, it only does the local server. If you have a few indexers, this can also be tedious. That would be a nice enhancement too. If someone were interested.

Builder

This PERL script will generate a csv report and email it (assuming all required packages are installed) to specified email address.

Only thing required is the Shared utilities package (in my case it was sharutils-4.6.1-2.x86_64) for the uuencode portion.

You will need to modify some of the variables (mainly the $username and $password) if you want it to automatically log you in.

!/usr/bin/perl

### Set variables
$splunk_bin_dir="/opt/splunk/bin";
$mail_to="$ARGV[0]";
$header="Index Name,DB Type,earliest event time,latest event time, size (mb)";
$output_dir="/tmp";
$report_name="index_report.csv";
$username="admin";
$password="passwordhere!";

### Delete previous versions of the report

$output_name="> ${output_dir}/${report_name}";

open(OUTPUT,$output_name);


### Get list of indexes
@index_config_raw=`cat /opt/splunk/etc/system/local/indexes.conf`;

foreach $line (@index_config_raw) {
        chomp $line;

        if ($line=~m/\[/) {
                $line=~m/\[(.*)\]/g;
                $raw_index=$1;

                push(@indexes,$raw_index);

        }
}

print OUTPUT "$header \n";

### Processes indexes
foreach $index (@indexes) {
        chomp $index;

        $splunk_command="${splunk_bin_dir}/splunk search \"| dbinspect index=\"${index}\" timeformat=\"\%s\" | rename state as category | stats min(earliestTime) as earliestTime max(latestTime) as latestTime sum(sizeOnDiskMB) as MB by category | convert timeformat=\"\%m/\%d/\%Y\" ctime(earliestTime) as earliestTime ctime(latestTime) as latestTime\" -auth ${username}:${password}| grep -v \"category\" | grep -v \"-\" ";


        @result=`${splunk_command}`;

        if ($#result ne "-1") {
                foreach $return (@result) {
                        chomp $return;

                        $return=~m/(hot|warm|cold|frozen)\s+([\d]+\/[\d]+\/[\d]+)\s+([\d]+\/[\d]+\/[\d]+)\s+([\d]+\.[\d]+)/gi;

                        $db_type=$1;
                        $earliest_event=$2;
                        $latest_event=$3;
                        $size=$4;

                        print OUTPUT "$index,$db_type,$earliest_event,$latest_event,$size \n";


                }
        }

        if ($#result eq "-1") {
                print OUTPUT "$index,Empty Index,Empty Index,Empty Index \n";
        }


}

if ($mail_to ne "") {
        `uuencode ${output_dir}/${report_name} ${output_dir}/${report_name} | mailx -s \"Splunk Index Report\" $mail_to`;
}

View solution in original post

State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!