Splunk Enterprise

How to generate a report on multiple indexes?

Brian_Osburn
Builder

There's a limitation in the dbinspect command where you cannot specify multiple indexes to report on, therefore reporting on an installation where multiple indexes are used can be a time consuming thing.

This answers article is a good start, but unfortunately you can only do one index at a time. How do I get around this?

Tags (2)
1 Solution

Brian_Osburn
Builder

This PERL script will generate a csv report and email it (assuming all required packages are installed) to specified email address.

Only thing required is the Shared utilities package (in my case it was sharutils-4.6.1-2.x86_64) for the uuencode portion.

You will need to modify some of the variables (mainly the $username and $password) if you want it to automatically log you in.

!/usr/bin/perl

### Set variables
$splunk_bin_dir="/opt/splunk/bin";
$mail_to="$ARGV[0]";
$header="Index Name,DB Type,earliest event time,latest event time, size (mb)";
$output_dir="/tmp";
$report_name="index_report.csv";
$username="admin";
$password="passwordhere!";

### Delete previous versions of the report

$output_name="> ${output_dir}/${report_name}";

open(OUTPUT,$output_name);


### Get list of indexes
@index_config_raw=`cat /opt/splunk/etc/system/local/indexes.conf`;

foreach $line (@index_config_raw) {
        chomp $line;

        if ($line=~m/\[/) {
                $line=~m/\[(.*)\]/g;
                $raw_index=$1;

                push(@indexes,$raw_index);

        }
}

print OUTPUT "$header \n";

### Processes indexes
foreach $index (@indexes) {
        chomp $index;

        $splunk_command="${splunk_bin_dir}/splunk search \"| dbinspect index=\"${index}\" timeformat=\"\%s\" | rename state as category | stats min(earliestTime) as earliestTime max(latestTime) as latestTime sum(sizeOnDiskMB) as MB by category | convert timeformat=\"\%m/\%d/\%Y\" ctime(earliestTime) as earliestTime ctime(latestTime) as latestTime\" -auth ${username}:${password}| grep -v \"category\" | grep -v \"-\" ";


        @result=`${splunk_command}`;

        if ($#result ne "-1") {
                foreach $return (@result) {
                        chomp $return;

                        $return=~m/(hot|warm|cold|frozen)\s+([\d]+\/[\d]+\/[\d]+)\s+([\d]+\/[\d]+\/[\d]+)\s+([\d]+\.[\d]+)/gi;

                        $db_type=$1;
                        $earliest_event=$2;
                        $latest_event=$3;
                        $size=$4;

                        print OUTPUT "$index,$db_type,$earliest_event,$latest_event,$size \n";


                }
        }

        if ($#result eq "-1") {
                print OUTPUT "$index,Empty Index,Empty Index,Empty Index \n";
        }


}

if ($mail_to ne "") {
        `uuencode ${output_dir}/${report_name} ${output_dir}/${report_name} | mailx -s \"Splunk Index Report\" $mail_to`;
}

View solution in original post

gkanapathy
Splunk Employee
Splunk Employee

I will also note that dbinspect does not work in distributed search mode, it only does the local server. If you have a few indexers, this can also be tedious. That would be a nice enhancement too. If someone were interested.

Brian_Osburn
Builder

This PERL script will generate a csv report and email it (assuming all required packages are installed) to specified email address.

Only thing required is the Shared utilities package (in my case it was sharutils-4.6.1-2.x86_64) for the uuencode portion.

You will need to modify some of the variables (mainly the $username and $password) if you want it to automatically log you in.

!/usr/bin/perl

### Set variables
$splunk_bin_dir="/opt/splunk/bin";
$mail_to="$ARGV[0]";
$header="Index Name,DB Type,earliest event time,latest event time, size (mb)";
$output_dir="/tmp";
$report_name="index_report.csv";
$username="admin";
$password="passwordhere!";

### Delete previous versions of the report

$output_name="> ${output_dir}/${report_name}";

open(OUTPUT,$output_name);


### Get list of indexes
@index_config_raw=`cat /opt/splunk/etc/system/local/indexes.conf`;

foreach $line (@index_config_raw) {
        chomp $line;

        if ($line=~m/\[/) {
                $line=~m/\[(.*)\]/g;
                $raw_index=$1;

                push(@indexes,$raw_index);

        }
}

print OUTPUT "$header \n";

### Processes indexes
foreach $index (@indexes) {
        chomp $index;

        $splunk_command="${splunk_bin_dir}/splunk search \"| dbinspect index=\"${index}\" timeformat=\"\%s\" | rename state as category | stats min(earliestTime) as earliestTime max(latestTime) as latestTime sum(sizeOnDiskMB) as MB by category | convert timeformat=\"\%m/\%d/\%Y\" ctime(earliestTime) as earliestTime ctime(latestTime) as latestTime\" -auth ${username}:${password}| grep -v \"category\" | grep -v \"-\" ";


        @result=`${splunk_command}`;

        if ($#result ne "-1") {
                foreach $return (@result) {
                        chomp $return;

                        $return=~m/(hot|warm|cold|frozen)\s+([\d]+\/[\d]+\/[\d]+)\s+([\d]+\/[\d]+\/[\d]+)\s+([\d]+\.[\d]+)/gi;

                        $db_type=$1;
                        $earliest_event=$2;
                        $latest_event=$3;
                        $size=$4;

                        print OUTPUT "$index,$db_type,$earliest_event,$latest_event,$size \n";


                }
        }

        if ($#result eq "-1") {
                print OUTPUT "$index,Empty Index,Empty Index,Empty Index \n";
        }


}

if ($mail_to ne "") {
        `uuencode ${output_dir}/${report_name} ${output_dir}/${report_name} | mailx -s \"Splunk Index Report\" $mail_to`;
}
Get Updates on the Splunk Community!

New Case Study Shows the Value of Partnering with Splunk Academic Alliance

The University of Nevada, Las Vegas (UNLV) is another premier research institution helping to shape the next ...

How to Monitor Google Kubernetes Engine (GKE)

We’ve looked at how to integrate Kubernetes environments with Splunk Observability Cloud, but what about ...

Index This | How can you make 45 using only 4?

October 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...