Getting Data In

At what point do very large lookup files (csv) get indexed?

Splunk Employee
Splunk Employee

After creating a very large lookup csv file splunk creates *.csv.index directories under my lookup directory. What is the default csv file size for this to happen?

Tags (2)
1 Solution

Splunk Employee
Splunk Employee

See this previous answer:

http://answers.splunk.com/questions/8326/are-lookup-tables-indexed

From limits.conf:

 [lookup]

 max_memtable_bytes = <integer> 
 * maximum size of static lookup file to use a in-memory index for
 * Defaults to 10000000

10MB.

View solution in original post

Splunk Employee
Splunk Employee

See this previous answer:

http://answers.splunk.com/questions/8326/are-lookup-tables-indexed

From limits.conf:

 [lookup]

 max_memtable_bytes = <integer> 
 * maximum size of static lookup file to use a in-memory index for
 * Defaults to 10000000

10MB.

View solution in original post

Communicator

Turns out this only works for .csv that are not also gzipped. So, if you write out to .csv.gz, this won't work. If you write out to .csv, it will.