At what point do very large lookup files (csv) get indexed?

[rroberts]

After creating a very large lookup csv file splunk creates *.csv.index directories under my lookup directory. What is the default csv file size for this to happen?

[Genti · Accepted solution]

See this previous answer:

http://answers.splunk.com/questions/8326/are-lookup-tables-indexed

From limits.conf:

 [lookup]

 max_memtable_bytes = <integer> 
 * maximum size of static lookup file to use a in-memory index for
 * Defaults to 10000000

10MB.

[supersleepwalke]

Turns out this only works for .csv that are not also gzipped. So, if you write out to .csv.gz, this won't work. If you write out to .csv, it will.