topic Re: Avoiding CSV lookup replication errors in Splunk Search

Avoiding CSV lookup replication errors

Kenshiro70 — Sat, 23 Jun 2018 20:37:31 GMT

I've got a medium-sized (50MB) CSV lookup file with two columns (email address and server name) that I want to use. I tried a straight upload and managed to put down our Splunk instance because replication failed and blocked all searches. Can I dribble the file in 100K lines at a time using outputlookup append=t? Or does the replication just take the whole lookup bundle and try and replicate everything?

Please note: I do not have access to the file system; whatever the solution is, I have to be able to do it from Splunk Web.

Thanks!

Re: Avoiding CSV lookup replication errors

paulbannister — Tue, 26 Jun 2018 11:01:19 GMT

Hi There,

That seems a tad bit more than a medium size csv you have there, how many records have you got within it? Have you looked into utilising a KV store instead?

Re: Avoiding CSV lookup replication errors

jkat54 — Tue, 26 Jun 2018 12:54:58 GMT

The problem isn’t going to be fixed by “dribbling in” the csv one piece at a time.

Depending on the version of splunk you have limits.conf on the search heads and indexers will have a default setting of 800MB or 2GB for search bundle replication (I think it’s 2GB since 6.6).

You’re going over that limit by x MEgabytes when you upload the csv... and causing the issue.

There are several solutions documented for this.

Increase the limits (just know it affects network bandwidth. See search bundle replication settings in limits.conf - you can’t do this via UI as far as I know.
Reduce the size and or number of existing lookups
you can probably do this
Index the data via the web UI and use join, append, etc (using an “OR” condition is better than joins, google how to join without join in splunk)
you can probably do this