Deployment Architecture

Peer Update Process with Clustering

Builder

I'm developing a plan to update my Splunk environment to 5.0 and take advantage of the clustering feature. I've read that it stated all peers should have the same config files and to use the peer update process to do this.

My question is what if you have different index.conf files on some of your peers based on the where the index was created? Will that index somehow get created on all peer nodes as a result:

For example: I have 1 indexer with a standard indexes.conf file, but I have another indexer that has some additional indexes listed in the file due to the fact specific indexes were created on that indexer alone.

Should I create a master list of all the indexes for indexes.conf and have the peer update process have a master copy of all index specifications, regardless of where the index is located? I assume with clustering it will somehow keep a copy of this data in the index on each peer, even if it doesn't reside on that peer alone?

Tags (1)
1 Solution

Splunk Employee
Splunk Employee

If you want an index to be replicated, that index must be defined in an indexes.conf file on each peer. The easiest way to do this is to create a common indexes.conf file and then use the master node to distribute that file to all peers at once. For more info, see:

http://docs.splunk.com/Documentation/Splunk/5.0/Indexer/Configurethepeerindexes

If, however, you want the index to be local to a single peer and not replicated, it doesn't need to appear in the other peers' indexes.conf files.

And don't forgot - you also need to set repfactor=auto in each index's definition, in order for the data in that index to be replicated.

Also, it appears from your question that you're planning to migrate existing non-clustered indexers to a cluster. If so, please be sure to read this topic first:

http://docs.splunk.com/Documentation/Splunk/5.0/Indexer/Migratenon-clusteredindexerstoaclusteredenvi...

View solution in original post

Splunk Employee
Splunk Employee

If you want an index to be replicated, that index must be defined in an indexes.conf file on each peer. The easiest way to do this is to create a common indexes.conf file and then use the master node to distribute that file to all peers at once. For more info, see:

http://docs.splunk.com/Documentation/Splunk/5.0/Indexer/Configurethepeerindexes

If, however, you want the index to be local to a single peer and not replicated, it doesn't need to appear in the other peers' indexes.conf files.

And don't forgot - you also need to set repfactor=auto in each index's definition, in order for the data in that index to be replicated.

Also, it appears from your question that you're planning to migrate existing non-clustered indexers to a cluster. If so, please be sure to read this topic first:

http://docs.splunk.com/Documentation/Splunk/5.0/Indexer/Migratenon-clusteredindexerstoaclusteredenvi...

View solution in original post

Splunk Employee
Splunk Employee

a section of the documentation seems to apply to this situation:

"Note: Under limited circumstances (for example, to perform local testing or monitoring), you might want to add an index (or an app with a new index) to one peer but not the others. You can do this by creating a peer-specific indexes.conf, so long as you're careful about how you configure the index and are clear about the ramifications. The data in such an index will not get replicated. The peer-specific indexes.conf supplements, but does not replace, the common version of the file that all peers get. ..."

more information later in the same topic:
http://docs.splunk.com/Documentation/Splunk/5.0/Indexer/Configurethepeers#Add_an_index_to_a_single_p...

in particular, your question about replication of the data is answered:
"If you need to add an index to a single peer, you can do so by creating a separate indexes.conf file on the peer. However, the data in the new index will remain only on that peer and will not get replicated"

edit: i don't think i really answer your question, which i think is: can i just have all the indexes exist on all the peers even if they're only in use on one of the peers? is that what you're asking?

further edit: ok so--yes, if you want the content of a particular index replicated, you can/should create a master indexes.conf and distribute it. you must do this explicitly, it doesn't happen automatically. also please note that you must set repfactor=auto on those indexes. that will cause the content of the index to be replicated to the other nodes as defined.

Splunk Employee
Splunk Employee

ok, i updated my answer-- if you want them replicated, you can go ahead and define all the indexes you need in one copy of indexes.conf that you distribute to all the nodes. if you don't want them replicated, you can define them in a separate copy, as explained in the docs.

Builder

it's sounding like indexes that are local to a specific peer will have to stay local to that peer and won't be replicated as part of the cluster, there making them excluded from being fault tolerant. Does that sound correct? In a simpler form, sounds like splunk is only good for replicating the system and default indexes then...?

0 Karma

Builder

And yes, you are right. That is what I am asking. Can i have a master indexes.conf file on all peers? If one index was made on Indexer A located at datacenter A and another was made on Indexer B at datacenter B, do i actually have to physically make sure each peer has these indexes created on them before deploying a master indexes.conf file and before doing clustering? OR would splunk know, hey, this index doesn't exist here, I'm going to make it because they want us to replicate it. I did see the input you posted from the documentation, but to me it didn't make sense to me at the time...

0 Karma

Builder

This is actually disappointing because I have some indexers that have specific indexes based on what is being indexed from a particular data center. Having the same indexes.conf on each indexer at first didn't make sense to me. But the point of this upgrade is replication.

0 Karma