Deployment Architecture

distributed search: Application does not exist: search

krugger
Communicator

Hi,

There are 2 splunk servers( A and B) that have differente data and indexes. I have setup distributed search from A to B and B to A.

searches done from A to B: everything is working as expected. I search for an index that only exists in B and I get results.

searches done from B to A: searches for indexes in A fail, and searches for local indexes give out warnings about searches in B failing.

Looking through the search.log I see messages like
ERROR dispatchRunner - RunDispatch::runDispatchThread threw error: Application does not exist: search

However the application exists in both splunk server. And why is every single search being replicated between both splunk servers. If I am looking for something in an index that is in B while I am connected to B is shouldn't replicate the search to A.

I think I should be using clustering instead of distributed search as I plan on having different indexes and data on each of the splunk servers.

Tags (1)
0 Karma
1 Solution

kristian_kolb
Ultra Champion

One of the points of a distributed setup is the performance benefit of spreading the workload of searching over several indexers (indexers do a major part of work in searches).

Also, this will let you use load balancing from the forwarders, which will give a higher resilience, since you can take one indexer down for maintenance, and events will still be forwarded to the other indexer.

If you really want to have separate indexes on separate servers, and to limit where the searches are sent, I guess you could use the splunk_server=serverB search restriction, since Splunk does not inherently know on which indexer a certain index exists.

Or... hmm.. perhaps that field (splunk_server) just filters incoming results from all indexers... in that case have a look at localop

http://docs.splunk.com/Documentation/Splunk/5.0.2/SearchReference/Localop

/K

View solution in original post

0 Karma

kristian_kolb
Ultra Champion

One of the points of a distributed setup is the performance benefit of spreading the workload of searching over several indexers (indexers do a major part of work in searches).

Also, this will let you use load balancing from the forwarders, which will give a higher resilience, since you can take one indexer down for maintenance, and events will still be forwarded to the other indexer.

If you really want to have separate indexes on separate servers, and to limit where the searches are sent, I guess you could use the splunk_server=serverB search restriction, since Splunk does not inherently know on which indexer a certain index exists.

Or... hmm.. perhaps that field (splunk_server) just filters incoming results from all indexers... in that case have a look at localop

http://docs.splunk.com/Documentation/Splunk/5.0.2/SearchReference/Localop

/K

View solution in original post

0 Karma

krugger
Communicator

I would have to edit all the searches on both server. They are like 300 of them, I will do if there is no other way

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!