We are not able to see all indexes (not even 3%) in the Available search indexes and Available indexes drop downs to select while creating a new role from the search heads. The indexes are created by pushing indexes.conf from master since it's a clustered environment.
We are able to create role successfully by adding the parameter srchIndexesAllowed from the command line on the search head and is being replicated across SHC and users are able to use it. However, it's still not showing in UI in the Settings » Access controls » Roles .
The indexes are visible in indexer clustering page on master and even in DMC but not also on master **Settings » Access controls » Roles **
There was a bug in the earlier versions of splunk but it was fixed in the latest versions.
Are there any limit on number of indexes displayed because we have more than 2K indexes?
The search head doesn't actually get a list of the indexes from the master, at least not one that it uses for populating this list (yeah, kinda lame). As such, we usually create "dummy" indexes on search heads, just to populate the list. As long as your search head is forwarding its events back to the index cluster (best practice), then the indexes really only get used for populating GUI stuff.
Sorry but it's hard to believe, because
I understand that Splunk says that this bug is fixed, however, in my experience, that isn't true. I would open a case with Splunk support if the list in the roles view isn't the same as those available in the indexes view, as that sounds like a bug.