topic Re: Splunk DB Connect 1: Do Splunk dbquery jobs count against search limits? in Splunk Search

Splunk DB Connect 1: Do Splunk dbquery jobs count against search limits?

a212830 — Thu, 28 Jan 2016 20:11:59 GMT

Hi,

I have customers using dbquery to augment Splunk dashboards (not joining the data, but presenting the data in another panel). Some of these dashboards have a lot of dbqueries, and I don't want it affecting "real" Splunk queries.

Re: Splunk DB Connect 1: Do Splunk dbquery jobs count against search limits?

sloshburch — Thu, 28 Jan 2016 22:01:22 GMT

Any search that runs a dbquery obviously counts as a search and would count against that account's role limits (and overall system limits). Even if its the first command (a "generating command" http://docs.splunk.com/Splexicon:Generatingcommand) like metadata still counts just like any other search.

Re: Splunk DB Connect 1: Do Splunk dbquery jobs count against search limits?

a212830 — Thu, 28 Jan 2016 22:25:39 GMT

Thanks. So... next question. Is there any way to limit the number of queries/ contained in a dashboard? I have people going nuts...

Re: Splunk DB Connect 1: Do Splunk dbquery jobs count against search limits?

sloshburch — Thu, 28 Jan 2016 22:39:55 GMT

Oh, now this is a 2fer. 😉

Limit no. BUT one approach I use when there's too many things going on on a dashboard is to use the post processing feature.

This got especially strong in 6.2+.

Check out this page which walks through how to run a common search once, then let the panels inherit from that. So you get one search to pull the raw data, then other searches that represent it in different ways. If a dashboards used to have 8 searches that all looked for the same data, you could reduce that down to 1 that pulls the data (the heavy work) and the rest just manipulate it.
http://docs.splunk.com/Documentation/Splunk/6.3.2/Viz/Savedsearches#Post-process_searches

Re: Splunk DB Connect 1: Do Splunk dbquery jobs count against search limits?

pgreer_splunk — Thu, 28 Jan 2016 23:53:18 GMT

I recently attended a Splunk .conf 2015 replay on using lookup tables:

http://conf.splunk.com/session/2015/recordings/2015-splunk-38.mp4

Although it is more geared to really large or long running searches and summarizing the data into a table (at periodic time periods - scheduled searches that create/update lookup tables) it could also be an option for your dashboards. Like @SloshBurch states, if there is common data that you're obtaining from the remote database, possibly pull it at regular intervals that makes sense and store it into a lookup table, then have your dashboards pull from that instead of creating a session into your remote database(s).

Just a thought...