topic Re: Use DBConnect/dbxquery to Populate a Lookup Table having more than a Million Rows in All Apps and Add-ons

Use DBConnect/dbxquery to Populate a Lookup Table having more than a Million Rows

mnm1987 — Wed, 15 Mar 2017 13:19:20 GMT

Hello Splunkers,
What is the best possible approach to use inorder to build a Lookup table in Splunk which would have more than a Million Rows, the source of which is a Database Table.

I have tried using DbConnect to fetch the data and Index it into Splunk, but to make it usable we end up needing to run a subsearch for all time and also dedup the results. The issue with the above is that 1) Its Slow 2) Sub-Search Needs to run on all time.

Thanks.

Re: Use DBConnect/dbxquery to Populate a Lookup Table having more than a Million Rows

s2_splunk — Wed, 15 Mar 2017 16:36:14 GMT

Is this maybe what you are looking for?

Re: Use DBConnect/dbxquery to Populate a Lookup Table having more than a Million Rows

woodcock — Wed, 15 Mar 2017 22:35:02 GMT

You could build another system that access the DB and publishes the list on a webserver and use this app:

https://splunkbase.splunk.com/app/635/