Splunk Search

Lookup table vs Add Data

mistydennis
Communicator

Basic question: when using a static csv as a data source, what are the pros and cons of creating a new lookup table vs. adding data through the Settings section (Settings -> Add Data)?

I don't understand the difference between the two, can someone help me out?

0 Karma
1 Solution

somesoni2
Revered Legend

When you add the static CSV with Add Data, the CSV data will be stored in specified index and would be tied to the timestamp of when it was indexed OR whatever timestamp field you've chosed (or Splunk found it). So, in order to search that data, you'll need to specify appropriate time range. Since it's indexed data, it'll cost Splunk license when ingested and its retention period will be bound to the retention period of the index.

When you add the CSV as lookup, it's still is a static csv data in Splunk and doesn't get indexed (so not license usage OR retention), or is timerange bound. It's a literal csv file available on all Search Head servers (assuming you use SH clustering OR use deployment server to push that lookup, else it'll only be available on the node you uploaded it). Please note that all lookups are part of knowledge bundle which SH sends to it's search peers (indexers), so if your static file is very large, you may see replication failures and errors during search.

Also, the way to you access the data will be different too (regular search vs lookup specific commands).

View solution in original post

somesoni2
Revered Legend

When you add the static CSV with Add Data, the CSV data will be stored in specified index and would be tied to the timestamp of when it was indexed OR whatever timestamp field you've chosed (or Splunk found it). So, in order to search that data, you'll need to specify appropriate time range. Since it's indexed data, it'll cost Splunk license when ingested and its retention period will be bound to the retention period of the index.

When you add the CSV as lookup, it's still is a static csv data in Splunk and doesn't get indexed (so not license usage OR retention), or is timerange bound. It's a literal csv file available on all Search Head servers (assuming you use SH clustering OR use deployment server to push that lookup, else it'll only be available on the node you uploaded it). Please note that all lookups are part of knowledge bundle which SH sends to it's search peers (indexers), so if your static file is very large, you may see replication failures and errors during search.

Also, the way to you access the data will be different too (regular search vs lookup specific commands).

mistydennis
Communicator

I can always depend on you, @somesoni2 ! Thanks 🙂

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Vibe-coding, AI, and Splunkcraft: Highlights from the .conf26 Builder Bar

If you stopped by the Builder Bar at .conf26, thank you! This year, we brought ...

Thanks for the Memories: .conf26 Took Learning to New Heights

Thank you, Splunk Community, for making .conf26 in Denver one for the books. From packed Splunk University ...

Best Practices: Splunk auto adjust pipeline queue

When you enable autoAdjustQueue in Splunk, maxSize should be understood as the queue size Splunk starts with ...