Knowledge Management

outputlookup takes 2 hours to update KVStore table with 2m rows

markgrin
Engager

I'm adding ~2k rows to a KVStore table with 14 fields and ~2 million rows. The outputlookup command takes nearly 2 hours.

The query is like this:

index=myindex earliest=-1d 
| fields id,a,b,c,d,e,f,g,h,i,j,k,l,m
| inputlookup append=true kvtable | dedup id
| outputlookup kvtable

The initial part of the query typically returns a couple thousand rows.

From the job inspector:

 duration components           invocations inputcount outputcount
 0.00     command.addinfo      6    2,138      2,138
 2.50     command.dedup        49   2,216,938  2,214,898
 0.00     command.fields       10   4,276      4,276
 17.30    command.inputlookup  1    150,538    2,450,000
 6,338.78 command.outputlookup 1    2,414,800  2,414,800

Is this normal? If not, can you suggestion some troubleshooting steps?

Labels (1)
0 Karma
1 Solution

martin_mueller
SplunkTrust
SplunkTrust

If you're just adding to the table, have you considered setting an appropriate _key value and using | outputlookup append=true? Then you wouldn't need to overwrite the entire collection on every update.

View solution in original post

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Additionally, what does your collections.conf look like, especially accelerated fields for this collection?

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

If you're just adding to the table, have you considered setting an appropriate _key value and using | outputlookup append=true? Then you wouldn't need to overwrite the entire collection on every update.

0 Karma

markgrin
Engager

Thanks for your help.  Finally was able to implement this and it's a big improvement. 

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...