topic Re: How do you create 4000+ index without creating it manually? in Deployment Architecture

How do you create 4000+ index without creating it manually?

ejmin — Tue, 12 Feb 2019 05:34:53 GMT

I have a client that consists of 4000+ branches, and I want to create an index using a file consisting different names of an index.

Is there any way to create an index of 4000+ by just using a file?

Re: How do you create 4000+ index without creating it manually?

woodcock — Tue, 12 Feb 2019 05:44:19 GMT

Use a Deployment Server and create an all_index app which has an indexes.conf file containing all 4000+ values. Add this all_index app to a all_indexer serverclass in the $SPLUNK_HOME/etc/system/local/serverclass.conf file with restartSplunkd=true. Restart Splunk on your Deployment Server to enable this new configuration.

Re: How do you create 4000+ index without creating it manually?

sobrien — Tue, 12 Feb 2019 05:45:17 GMT

Hi ejmin,

The file to modify is indexes.conf, outlined here:

https://docs.splunk.com/Documentation/Splunk/latest/admin/indexesconf

If you have a naming convention, you can then script the creation of all the required stanzas.

I would caution you to consider why you want that many indexes though, it will lead to difficulties in management and maintenance.

Re: How do you create 4000+ index without creating it manually?

ejmin — Tue, 12 Feb 2019 06:19:40 GMT

hmmm.. I get your idea but it seems it is not applicable for my architecture because I have this one cluster master pushing to the 3 search peers which are clustered. but anyway thank you for your response.

Re: How do you create 4000+ index without creating it manually?

woodcock — Tue, 12 Feb 2019 06:25:23 GMT

The Cluster Master performs the same function as the DS except that it uses $SPLUNK_HOME/etc/master-apps/. Create this app and then do a cluster bundle push to the slaves.

Re: How do you create 4000+ index without creating it manually?

ejmin — Tue, 12 Feb 2019 06:54:59 GMT

Ok I get it... You mean to say that I need to put all the config files in indexes.conf.. Hmmm that's was my first option but my boss said that he wants me to have create a script rather than created a 4000+ index stanzas in indexes.conf which I am looking for right now... I know that your instruction will work but my boss wants it to create automatically..

Re: How do you create 4000+ index without creating it manually?

ejmin — Tue, 12 Feb 2019 06:57:21 GMT

yeahh I know that it will be difficult to manage but for some instances like we are currently in right now is if for certain branch has a problem with the data it will be easy to troubleshoot or delete an index without affecting the whole indexes.

Re: How do you create 4000+ index without creating it manually?

woodcock — Tue, 12 Feb 2019 07:00:09 GMT

So you are not asking about the deployment mechanism, you are asking about a script to create 4000 stanzas inside the indexes.conf file. That is pretty trivial. Something like this:

for i in {1..4000}
do
   index_name = "index${i}"
   echo "[$index_name]" >> indexes.conf
   echo "homePath   = volume:primary/${index_name}/db" >> indexes.conf
   echo "coldPath   = volume:primary/${index_name}/colddb" >> indexes.conf
   echo "thawedPath = \$SPLUNK_DB/${index_name}/thaweddb" >> indexes.conf
done

Re: How do you create 4000+ index without creating it manually?

ejmin — Tue, 12 Feb 2019 07:06:35 GMT

yes very much like that. Sorry for the first question because my boss said it to me right now that he wants a script rather than a file and thats what I working on.

Re: How do you create 4000+ index without creating it manually?

woodcock — Tue, 12 Feb 2019 07:07:41 GMT

See updated answer.

Re: How do you create 4000+ index without creating it manually?

ejmin — Tue, 12 Feb 2019 09:51:45 GMT

Ok.. Thanks It works but I made some changes in your script by the way Thankyou it helps a lot to me.