All Apps and Add-ons

Can the DB Connect Database Lookups use an RPC pool as well?

Path Finder

With the new version 2.0 I saw the documentation for Outputs and Inputs using the RPC pool to offload the request, can the same be done for the lookups?

This would reduce the need for open database connections from the search head.

0 Karma
1 Solution

Splunk Employee
Splunk Employee

sorry, not yet. It's something that we're very interested in figuring out, though.

View solution in original post

0 Karma

Splunk Employee
Splunk Employee

The RPC protocol is always used, whether the operations are local or remote. Efficiency isn't my main concern, although I'm sure there is room for improvement in that regard.

An early version of the resource pool architecture had search peers as auto-discovered resource pools, but due to the complexity of dblookups, specifically securely replicating cache from search heads to indexers, we don't have dblookups running on indexers yet.

All lookups, whether scripted or external, only run on indexers when the search is non-transforming (i.e. does not use table, stats, etc), so for transforming searches, you always have to run the lookup on the search head and they will never run on the peer.

As an alternative, you can always dbinput data and make it in to a conventional CSV lookup, which will then be replicated to search peers via normal replication.

0 Karma

Splunk Employee
Splunk Employee

This would be tough since you lookups are performed at the search head - we'd have to shuttle search results off to the resource pool, decorate them, then shuttle them back, which would likely make for terrible performance. We'll still give a good think as to how to improve this feature from a performance perspective.

0 Karma

Path Finder

Depending on how efficient the rpc protocol is, how is this much different than passing work off to the indexers or using external lookups.

0 Karma

Splunk Employee
Splunk Employee

sorry, not yet. It's something that we're very interested in figuring out, though.

View solution in original post

0 Karma