Splunk Search

After adding two new indexers as search peers to my existing search head, why is my search not returning results from an index on these instances?

Path Finder

I have an environment that has two indexers. I recently added an additional two indexers and added them as search peers to my existing search head. All 4 indexers have an index called "pcoip" that stores data related to virtual desktops.

When I run this search index=pcoip, I only get results from the original two indexers, even though all four have data in that index during the specified time frame.

When I run the search and add the splunkserver field`index=pcoip splunkserver=*`, I get results back for all four indexers.

Is there some setting or configuration that I am missing that prevents these searches from returning the same data?

0 Karma
1 Solution

Champion

What version of Splunk? Is DMC configured? There's a known issue we've run into related to DMC where some of our indexers don't get searched. The workaround is to just click apply on the set up page of DMC

SPL-99116
After enabling the Distributed Management Console (DMC) in distributed mode in an indexing cluster, the search head may not be able to search all the peers. The error will mention splunkservergroup : "Search filters specified using splunkserver/splunkserver_group do not match any search peer". To work around the issue, go to the DMC setup page and click Apply. To avoid the issue, run the DMC in standalone mode.

View solution in original post

Champion

What version of Splunk? Is DMC configured? There's a known issue we've run into related to DMC where some of our indexers don't get searched. The workaround is to just click apply on the set up page of DMC

SPL-99116
After enabling the Distributed Management Console (DMC) in distributed mode in an indexing cluster, the search head may not be able to search all the peers. The error will mention splunkservergroup : "Search filters specified using splunkserver/splunkserver_group do not match any search peer". To work around the issue, go to the DMC setup page and click Apply. To avoid the issue, run the DMC in standalone mode.

View solution in original post

SplunkTrust
SplunkTrust

Yesterday I ran into this on a Splunk 6.3.0 instance....looks like this feature is still available 🙂

0 Karma

Path Finder

Not sure how to give maciep the credit but that hit the nail on the head.

I am using DMC and by going in to the Setup screen and hitting Apply, my search is now able to correctly pull results from all indexers.

0 Karma

Community Manager
Community Manager

Hi @stevepraz

Just converted @maciep's comment under your question to an answer and accepted it 🙂 To give maciep even more credit, you can always upvote their answer so they get a boost of 15 karma points. Cheers!

Patrick

0 Karma

Path Finder

Currently running 6.2.1 on the search head and original indexers and 6.2.4 on the new indexers. I do have DMC configured.

When I went into DMC, I saw the two new indexers listed as State of "New". I hit apply changes. After that I ran the search again and it worked.

I never actually saw the error mentioned above but that fix appears to have worked.

Splunk Employee
Splunk Employee

Did you edit distsearch.conf on your searchead to add the two servers in?

Check out http://docs.splunk.com/Documentation/Splunk/6.2.4/DistSearch/Configuredistributedsearch

0 Karma

Path Finder

Yes. I configured the indexers as Search Peers using Splunk web.

0 Karma

Esteemed Legend

I would open a support case (be sure to let us know what you find out).

0 Karma