Deployment Architecture

How do you create 4000+ index without creating it manually?

ejmin
Path Finder

I have a client that consists of 4000+ branches, and I want to create an index using a file consisting different names of an index.

Is there any way to create an index of 4000+ by just using a file?

Tags (2)
1 Solution

woodcock
Esteemed Legend

So you are not asking about the deployment mechanism, you are asking about a script to create 4000 stanzas inside the indexes.conf file. That is pretty trivial. Something like this:

for i in {1..4000}
do
   index_name = "index${i}"
   echo "[$index_name]" >> indexes.conf
   echo "homePath   = volume:primary/${index_name}/db" >> indexes.conf
   echo "coldPath   = volume:primary/${index_name}/colddb" >> indexes.conf
   echo "thawedPath = \$SPLUNK_DB/${index_name}/thaweddb" >> indexes.conf
done

View solution in original post

woodcock
Esteemed Legend

So you are not asking about the deployment mechanism, you are asking about a script to create 4000 stanzas inside the indexes.conf file. That is pretty trivial. Something like this:

for i in {1..4000}
do
   index_name = "index${i}"
   echo "[$index_name]" >> indexes.conf
   echo "homePath   = volume:primary/${index_name}/db" >> indexes.conf
   echo "coldPath   = volume:primary/${index_name}/colddb" >> indexes.conf
   echo "thawedPath = \$SPLUNK_DB/${index_name}/thaweddb" >> indexes.conf
done

ejmin
Path Finder

yes very much like that. Sorry for the first question because my boss said it to me right now that he wants a script rather than a file and thats what I working on.

0 Karma

woodcock
Esteemed Legend

See updated answer.

0 Karma

ejmin
Path Finder

Ok.. Thanks It works but I made some changes in your script by the way Thankyou it helps a lot to me.

0 Karma

sobrien
Splunk Employee
Splunk Employee

Hi ejmin,

The file to modify is indexes.conf, outlined here:

https://docs.splunk.com/Documentation/Splunk/latest/admin/indexesconf

If you have a naming convention, you can then script the creation of all the required stanzas.

I would caution you to consider why you want that many indexes though, it will lead to difficulties in management and maintenance.

0 Karma

ejmin
Path Finder

yeahh I know that it will be difficult to manage but for some instances like we are currently in right now is if for certain branch has a problem with the data it will be easy to troubleshoot or delete an index without affecting the whole indexes.

0 Karma

woodcock
Esteemed Legend

Use a Deployment Server and create an all_index app which has an indexes.conf file containing all 4000+ values. Add this all_index app to a all_indexer serverclass in the $SPLUNK_HOME/etc/system/local/serverclass.conf file with restartSplunkd=true. Restart Splunk on your Deployment Server to enable this new configuration.

0 Karma

ejmin
Path Finder

hmmm.. I get your idea but it seems it is not applicable for my architecture because I have this one cluster master pushing to the 3 search peers which are clustered. but anyway thank you for your response.

0 Karma

woodcock
Esteemed Legend

The Cluster Master performs the same function as the DS except that it uses $SPLUNK_HOME/etc/master-apps/. Create this app and then do a cluster bundle push to the slaves.

0 Karma

ejmin
Path Finder

Ok I get it... You mean to say that I need to put all the config files in indexes.conf.. Hmmm that's was my first option but my boss said that he wants me to have create a script rather than created a 4000+ index stanzas in indexes.conf which I am looking for right now... I know that your instruction will work but my boss wants it to create automatically..

0 Karma
Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

WATCH NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If exploited, ...

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...