topic Re: How to create index using REST API in a clustered environment? in Getting Data In

How to create index using REST API in a clustered environment?

dhavamanis — Tue, 29 Mar 2016 21:02:34 GMT

Can you please tell us, How to create index using REST API in a clustered environment?

Re: How to create index using REST API in a clustered environment?

MuS — Wed, 30 Mar 2016 01:25:19 GMT

Hi dhavamanis,

according to the docs http://docs.splunk.com/Documentation/Splunk/6.3.3/RESTREF/RESTcluster#cluster.2Fmaster.2Findexes a POST is not supported, therefore you will not be able to create an index in the cluster directly.
But, you can create it on the Cluster Master itself using POST on this REST endpoint http://docs.splunk.com/Documentation/Splunk/6.3.3/RESTREF/RESTconf#configs.2Fconf-.7Bfile.7D using indexes.conf. This will create your indexes.conf on the Cluster Master in $SPLUNK_HOME/etc/system/local/ which can be copied over to $SPLUNK_HOME/etc/master-apps/_cluster/local on the Cluster Master using a cron job. After the copy the cluster bundle can be applied using this undocumented REST end point:

 curl -k -u admin:changme https://localhost:8089/services/cluster/master/control/default/apply

This is a work around but it will get you there in the end.

Hope this helps ...

cheers, MuS

Re: How to create index using REST API in a clustered environment?

thisissplunk — Wed, 06 Jun 2018 22:29:19 GMT

Is this still the only way to create an index in a cluster with the api? Very disappointed.

Re: How to create index using REST API in a clustered environment?

thisissplunk — Wed, 25 Jul 2018 21:05:13 GMT

Could you symlink system/local to master-apps/_cluster/local instead of copying from one to the other?

Re: How to create index using REST API in a clustered environment?

MuS — Wed, 25 Jul 2018 21:07:38 GMT

The question would be here should you instead of could you 😉 While technically possible this will for sure not be support you will be on your own when things start to get crazy and you need help from support ....

cheers, MuS

Re: How to create index using REST API in a clustered environment?

thisissplunk — Wed, 25 Jul 2018 22:54:00 GMT

Well until Splunk gets off their lazy butts and make the API cover all basic use cases, we'll need a way to not require OS level access for users running automation scripts and this sounded like a solution. The API is pretty weak.

Re: How to create index using REST API in a clustered environment?

MuS — Wed, 25 Jul 2018 23:08:28 GMT

So, what prevents you from using your automation script to change indexes.conf in $SPLUNK_HOME/etc/master_apps/<whatevername>/local/indexes.conf and call the un-documented REST endpoint /services/cluster/master/control/default/apply to apply the cluster bundle ?

here is an example to call the endpoint:

curl -k -u admin:changeme https://localhost:8089/services/cluster/master/control/control/apply  -X POST

cheers, MuS

Re: How to create index using REST API in a clustered environment?

thisissplunk — Thu, 26 Jul 2018 17:44:07 GMT

Well, the API does not write to master-apps folder at all. You can see that in this answer: https://answers.splunk.com/answers/387133/how-to-create-index-using-rest-api-in-a-clustered.html or try using the /servicesNS/nobody/ endpoint. I'm already using the undocumented bundle push endpoint like you mentioned though.

We have to use the API because of US vs UK data sensitivity, and well, best practice. We cannot allow certain employees to have OS access at all, hence trying to figure out how to prevent the need for OS accounts doing anything on the master or forwarder, hence symlinks.

Re: How to create index using REST API in a clustered environment?

MuS — Thu, 26 Jul 2018 21:32:28 GMT

Look, in my almost 8 years of providing Splunk PS I came along a lot of things that are not supported by Splunk, but doable 😉
For example: one customer had its cluster master setup as deployment client to work around this problem; they created the indexes.conf on the deployment server using REST calls, the indexes.conf was symlinked into a deployment app, which was deployed to the cluster master into $SPLUNK_HOME/etc/master-apps/ ... it worked. Is it supported, probably not.

But you can always file an enhancement request with Splunk if you reckon this is a needed feature, use the Support Portal here https://www.splunk.com/en_us/support-and-services.html

Re: How to create index using REST API in a clustered environment?

thisissplunk — Thu, 26 Jul 2018 21:49:05 GMT

Yes, that is effectively what we are trying to do here, just on the same box. Not sure we want to spin up another box for it. Don't get me wrong, I love working with Splunk, but I am very disappointed at how such an integral part of Splunk administration is not possible with the API.

Thanks for the help so far and I will look into filing a request.

Re: How to create index using REST API in a clustered environment?

Flesz — Mon, 07 Feb 2022 22:14:58 GMT

4 years later....

Have there been any improvements with that?