All Apps and Add-ons

splunk db connect 3: dbxquery very slow

damucka
Builder

Hello,

We would like to move our dashboards from another tool into Splunk, so this question is quite important for us. The issue is that the dbxquery seems to be very slow and our dashboards base on the online DB queries.

Example:

Following query takes 49 ms when executed directly on the database:

| noop search_optimization=false| dbxquery query="select timestamp, errorid, sum(error_cnt) as error_cnt from sapiop.ZKPID_AAPPDUMP where errorid <> '%TECHNICAL ITEM%' and sysid = 'ISP' and timestamp between cast('2018-11-28 08:04:00' as timestamp) and cast('2018-11-28 12:04:00' as timestamp) and errorid = 'TIME_OUT' group by timestamp, errorid" connection="HANA_MLBSO"

On Splunk, 2.745 seconds:

   1.59  command.dbxquery   1   -   16
0.00     dispatch.check_disk_usage  1   -   -
0.00     dispatch.createdSearchResultInfrastructure 1   -   -
1.14     dispatch.evaluate  1   -   -
1.14     dispatch.evaluate.dbxquery 1   -   -
0.00     dispatch.evaluate.noop 2   -   -
0.01     dispatch.writeStatus   7   -   -
0.11     startup.configuration  1   -   -
0.00     startup.handoff    1   -   -

This is only one of several examples.
Now, ... we really need to get it running faster, otherwise we will not decide to use Splunk for it.
Could you please advice how to optimize it?
I have read in another question that this would be the java based DB Conn 3 causing issues and the DB Conn 1 would be way faster. Is it possible to install the DB Conn 1 still? Where would we download it?
If not, is it possible to modify the DB Conn App in the way that the normal ODBC driver is used and not the jDBC?

Kind Regards,
Kamil

Labels (1)
Tags (1)
0 Karma
1 Solution

damucka
Builder

Hello,

The DBConnect v.3.3 solved the issue, performance is back to normal now.

Kind Regards,
Kamil

View solution in original post

0 Karma

damucka
Builder

Hello,

The DBConnect v.3.3 solved the issue, performance is back to normal now.

Kind Regards,
Kamil

0 Karma

Richfez
SplunkTrust
SplunkTrust

The fine docs say that dispatch.evaluate is "The time spent parsing the search and setting up the data structures needed to run the search." I'm not sure why that would be as high as it is in this case.

Have you tried converting the above dbxquery into a view inside your DBMS, so you then only have to have a dbxquery of something like

select timestamp, errorid, error_cnt from sapiop.view_ZKPID_for_Splunk

I'd be very interested in seeing the difference in run time.

As to going back to DBX1 ... no, don't do that if there's any other option. But I guess if you have to, it might do what you need. The thing is, DBX 1 is pretty terrible.

If I were you, I'd be raising a support ticket with Splunk. They may or may not have an answer, but even if they have no good answer it'll be one more support ticket for a performance problem out of DBX that isn't related (really) to the actual query or DB itself, but seems just a problem with the java interface they're using.

0 Karma
Get Updates on the Splunk Community!

Happy CX Day to our Community Superheroes!

Happy 10th Birthday CX Day!What is CX Day? It’s a global celebration recognizing innovation and success in the ...

Check out This Month’s Brand new Splunk Lantern Articles

Splunk Lantern is a customer success center providing advice from Splunk experts on valuable data insights, ...

Routing Data to Different Splunk Indexes in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...