We are in a process of setting up new splunk env on CentOS 7. As part of it we have configured 1 search head and 1 indexer server.
We added the indexer to SH in the distributed search section(distsearch.conf), the status of that indexer is "up", Replication status "Successful", healthy status "healthy and No health check failures.
We are using splunkforwarders on our servers to push data to indexers, which is also working fine, i can see data coming from the selected servers in the metrics.log on indexer.
However when i search anything from web UI it gives me "no results found", also in data summary on home page it says "Waiting for results".
Even for index="_internal" there are "no results found".
Can anyone please point me in the right direction if i'm missing anything in the configs here.