Deployment Architecture

distributed search: Application does not exist: search

krugger
Communicator

Hi,

There are 2 splunk servers( A and B) that have differente data and indexes. I have setup distributed search from A to B and B to A.

searches done from A to B: everything is working as expected. I search for an index that only exists in B and I get results.

searches done from B to A: searches for indexes in A fail, and searches for local indexes give out warnings about searches in B failing.

Looking through the search.log I see messages like
ERROR dispatchRunner - RunDispatch::runDispatchThread threw error: Application does not exist: search

However the application exists in both splunk server. And why is every single search being replicated between both splunk servers. If I am looking for something in an index that is in B while I am connected to B is shouldn't replicate the search to A.

I think I should be using clustering instead of distributed search as I plan on having different indexes and data on each of the splunk servers.

Tags (1)
0 Karma
1 Solution

kristian_kolb
Ultra Champion

One of the points of a distributed setup is the performance benefit of spreading the workload of searching over several indexers (indexers do a major part of work in searches).

Also, this will let you use load balancing from the forwarders, which will give a higher resilience, since you can take one indexer down for maintenance, and events will still be forwarded to the other indexer.

If you really want to have separate indexes on separate servers, and to limit where the searches are sent, I guess you could use the splunk_server=serverB search restriction, since Splunk does not inherently know on which indexer a certain index exists.

Or... hmm.. perhaps that field (splunk_server) just filters incoming results from all indexers... in that case have a look at localop

http://docs.splunk.com/Documentation/Splunk/5.0.2/SearchReference/Localop

/K

View solution in original post

0 Karma

kristian_kolb
Ultra Champion

One of the points of a distributed setup is the performance benefit of spreading the workload of searching over several indexers (indexers do a major part of work in searches).

Also, this will let you use load balancing from the forwarders, which will give a higher resilience, since you can take one indexer down for maintenance, and events will still be forwarded to the other indexer.

If you really want to have separate indexes on separate servers, and to limit where the searches are sent, I guess you could use the splunk_server=serverB search restriction, since Splunk does not inherently know on which indexer a certain index exists.

Or... hmm.. perhaps that field (splunk_server) just filters incoming results from all indexers... in that case have a look at localop

http://docs.splunk.com/Documentation/Splunk/5.0.2/SearchReference/Localop

/K

0 Karma

krugger
Communicator

I would have to edit all the searches on both server. They are like 300 of them, I will do if there is no other way

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In December, the Splunk Threat Research Team had 1 release of new security content via the Enterprise Security ...

Why am I not seeing the finding in Splunk Enterprise Security Analyst Queue?

(This is the first of a series of 2 blogs). Splunk Enterprise Security is a fantastic tool that offers robust ...

Index This | What are the 12 Days of Splunk-mas?

December 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...