Hi Team,
Environment
1 - Search Head, 2-Indexers, 1 - Deployment Server, 1 - Heavy Forwarder, 1 -Cluster Master
Problem Statement
1)I am unable to retrieve events when searching with index=*
2) When checked with connectives all were connected (SH --> Indexers --> CM --> HF --> DS)
When checked with internal index showing 401 client is not authenticated.
When checked from backend there is no error showing in splunkd.log
Hi @Atchyuth_P,
wher do you runned the search with results?
if you see data in HF, there something wrong in your configuration because there are two choices:
in both cases it isn't correct.
As I said: where do you runned the search with 0 results?
If in Indexer, it's correct because you cannot use Indexers for searching only SH.
If in SH you have to debug: are other searches running on SH (e.g. index=_internal)?
Configurations seems to be ok.
Ciao.
Giuseppe
Hi @Atchyuth_P,
are you speaking of searches on SH or on IDXs?
if you have an IDXs Cluster, you cannot use them for searching only SH.
The other systems cannot be used for searching, only SH.
for using other systems for searching, you have to configurate them as SH.
Ciao.
Giuseppe
Hi @gcusello
Ok, i found the mistake that i have done but from HF the data is not pushing to indexers.
I am sharing the screenshots for reference
Heavy Forwarder :
inputs.conf
outputs.conf
Indexer 1
inputs.conf
Indexer 2
When i check with connectivity all were connected
The index is showing "0" Events
In HF i can see the data
Please suggest
Hi @Atchyuth_P,
wher do you runned the search with results?
if you see data in HF, there something wrong in your configuration because there are two choices:
in both cases it isn't correct.
As I said: where do you runned the search with 0 results?
If in Indexer, it's correct because you cannot use Indexers for searching only SH.
If in SH you have to debug: are other searches running on SH (e.g. index=_internal)?
Configurations seems to be ok.
Ciao.
Giuseppe
Hi @gcusello
I can see for HF to Indexer 2 the connection is in TIME_WAIT and for indexer 1 it is established
Yes there is a local copy but when i tried to check previously it worked the events got shown in indexer 2 but not in indexer 1
Now the data is not showing in two indexers
Hi @Atchyuth_P,
outputs.conf seems to be corrects, did you checked the connection between HF and IDX2 (if not try using telnet not ping)?
About local copy you shouldn't have it also because you have in your outputs.conf "indexAndForward = false"
I repeat the question: where are you running searches: on SH or on another system?
How do you configured SH to search on IDXs?
Ciao.
Giuseppe
Hi @gcusello
I am trying to check the search in both the indexers because the events is showing zero
I tried both telnet and ping HF---> IDX2,IDX2 ---> HF all the connection established
Hi @Atchyuth_P,
ping isn't relevant to check connections, uso only telnet on port 9997.
About searches: you cannot use Indexers (when clustered) for searching only Search Heads.
If search runs on a IDX means that there's a misconfiguration in the cluster.
What does it happen running a search a different index (obviously on SH)?
Ciao.
Giuseppe
Hi @gcusello
Thanks for the info i miss the catch i have done the configuration in SH as well. Almost, forgot IDX will not acts as SH.
Sorry for the trouble.
Hi @Atchyuth_P,
good for you, see next time!
Ciao and happy splunking
Giuseppe
P.S.: Karma Points are appreciated 😉
Hi @Atchyuth_P,
no problem, tell me if I can help you more on this issue, otherwise, if one answer solves your need, please accept one answer for the other people of Community.
Ciao and happy splunking
Giuseppe
P.S.: Karma Points are appreciated by all the Contributors;-)