All Apps and Add-ons

SQL DBX Query is taking 500 seconds to execute the query

Ashwini008
Builder

Hi,

I am using dbxquery to fetch around 800000 data from database into splunk

| dbxquery connection=x query="select * from table002 " shortnames=t maxrows=800000

The above query is taking around 500 seconds.The default maxrow is 1000 in dbxquery.py script.How do reduce the time taken without impaction performance of my server?

P.S | noop search_optimization=false This doesnt improve my search time

 

Ashwini008_0-1631817021695.png

 

 

0 Karma

PickleRick
SplunkTrust
SplunkTrust

Firstly - do I understand correctly from your log that you're returning several hundred thousand records from db query and then do a search filtering it down to 27 records? If so, that's kinda... unwise.

Secondly - what's your memory status? (free, used, swap and so on. Are you not swapping out?

Thirdly - if you do a "normal" query using a bare cli client - does it also take that long? In short - are you sure it's splunk's fault?

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...