Getting Data In

How to do the equivalent of a debug/refresh on cluster master master-apps?

thisissplunk
Builder

I'm updating /master-apps/_cluster/local/indexes.conf and then pushing the bundle. I check the cluster's search head and my new index is there and ready to go. HOWEVER, the following API endpoint has yet to see my new index:

/services/cluster/master/indexes

I solved a similar issue for my /system/local/inputs.conf changes on my forwarder by using this to force a reloadvia the api:

/services/data/inputs/monitor/_reload

There seems to be no such api endpoint for the /services/cluster/master though. Even doing the following on the master CLI doesn't update the index list:

.../splunk reload index

What gives? How do I reload the indexes.conf after I make a change so that I can confirm my index was created? Not even a splunk restart on the master is picking up the master-apps changes.

Tags (4)
0 Karma
1 Solution

dxu_splunk
Splunk Employee
Splunk Employee

You need data in the newly created index, and then the Cluster Master will create an entry at cluster/master/indexes.
(The cluster master just sits there and listens for all the buckets the indexers has, it actually has no idea of what indexes its supposed to have - it never peeks into etc/master-apps/)

View solution in original post

dxu_splunk
Splunk Employee
Splunk Employee

You need data in the newly created index, and then the Cluster Master will create an entry at cluster/master/indexes.
(The cluster master just sits there and listens for all the buckets the indexers has, it actually has no idea of what indexes its supposed to have - it never peeks into etc/master-apps/)

View solution in original post

MuS
SplunkTrust
SplunkTrust

That's what I tried to explain, but much better wording here 😉 Thanks !

thisissplunk
Builder

Thanks, this was it. Sounds like I might want to check the search head or an indexer for the indexes then, as I need to know if an index exists whether it has data or not. Thoughts?

0 Karma

dxu_splunk
Splunk Employee
Splunk Employee

i'd check the indexers themselves for list of indexes. some indexes might be not replicated, and wont show up on the master/indexes endpoint.

| rest services/data/indexes | fields title repFactor

searching this will give u a list of indexes and whether they're replicated

thisissplunk
Builder

Ok thanks, I guess that will have to do. Was trying to avoid touching additional servers. It's already slightly annoying there is no API way to get changes into /master-apps/_cluster config files without also touching the master's filesystem.

Edit: Actually, if I go this route (the only route) for updating the master configs, then I can just view the local config on the master and assume that is what is on the indexers: https://answers.splunk.com/answers/387133/how-to-create-index-using-rest-api-in-a-clustered.html

Not ideal, but /shrug.

0 Karma

MuS
SplunkTrust
SplunkTrust

Do I get you right here, you created an index on the master in master-apps that will be pushed out to the idx cluster peers only? In this case the idx will never show up anywhere on the cluster master, because you did not create it there.

Hope to understand the question correct ¯\_(ツ)_/¯

cheers, MuS

0 Karma

thisissplunk
Builder

That is correct, but why do I see all of my other indexes in the master-apps/_cluster/local/indexes.conf when using that API endpoint? I feel like there is some kind of delay, or the new indexes aren't showing up because there isn't data in them yet. Testing that now.

0 Karma

MuS
SplunkTrust
SplunkTrust

Well, on the cluster master you will never have any data in the indexes shown using this API endpoint /services/cluster/master/indexes. Maybe the cluster-bundle apply or verify commands update the endpoint?

0 Karma

thisissplunk
Builder

Once I sent data to the index, this API endpoint on the master worked. Not sure what this endpoint is pulling from, but its not just a indexes.conf file like the inputs endpoint does.

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!