Hi,
I am using dbxquery to fetch around 800000 data from database into splunk
| dbxquery connection=x query="select * from table002 " shortnames=t maxrows=800000
The above query is taking around 500 seconds.The default maxrow is 1000 in dbxquery.py script.How do reduce the time taken without impaction performance of my server?
P.S | noop search_optimization=false This doesnt improve my search time
Firstly - do I understand correctly from your log that you're returning several hundred thousand records from db query and then do a search filtering it down to 27 records? If so, that's kinda... unwise.
Secondly - what's your memory status? (free, used, swap and so on. Are you not swapping out?
Thirdly - if you do a "normal" query using a bare cli client - does it also take that long? In short - are you sure it's splunk's fault?