We recently deployed a dedicated search head (as it is not indexing any data) in our environment with a single index (for now). Users used to search on the indexing node itself and could utilize the sources, sourcetypes, and hosts fields on the search summary page: https:///en-US/app/search/dashboard but now with our search head in place, all the fields that were populated on the indexer (sources, sourcetypes, hosts) are completely lacking from the search head. The search head does have some data there but it appears to be displaying only local data.
Is there any way to replicate the data provided on the indexer's summary page to the search-head's?
It should do so if it is a search head. Either the default indexes on the search head do not include the indexes that contain the data (on the indexer) or the search head is not in fact connected to the indexers at all.
Configuration of the search head looks good and connect to the indexing node ok:
(Manager >> Distributed search >> Search peers)
Replication Status: Successful
The only indexes on the dedicated search head that include any data are the _audit and _internal indexes. Will I need to create a placeholder index on the search head for unique indexes found on the search peer?
Not necessarily, but whatever indexes are specified as default on the search head will be the ones queried by default. Role settings on the indexers are not considered.
I've created a local search head and attached the original search peer to this. I suspect the issue lies within the search peer/indexing node as the search head is displaying the same results as the dedicated search head.
What specifically should I check on the search peer/indexing node that could be causing this inconsistency? The data that IS displayed on the "All indexed data" portion on the summary page reflects the latest LightForwarder that we added last week - but no data from any of the other forwarding hosts is displayed.
Rather strange, eh?
Last Thursday 5/16 we were given a license that resets our LicenseViolation count and on Friday 5/17 we upgraded to a new license level. Could this of had any effect?
Note: this dedicated search head is configured in exactly the same way as another dedicated search head that we're using with a separate search peer/indexer.
$SPLUNK_HOME/etc/system/local/authorize.conf was configured on the search peer/indexing node but it was missing on the search-head. Created $SPLUNK_HOME/etc/system/local/authorize.conf with the same settings found on the indexer's copy, restarted the search-head and voila I'm good to go.