Splunk Cloud Platform

Lookup File Limits

kathhuynh
Explorer

I had some questions about the limits of a lookup file that I wasn't able to find when referencing documentation (below) or anywhere else in Splunk Cloud.

https://docs.splunk.com/Documentation/SplunkCloud/latest/Knowledge/DefineaKVStorelookupinSplunkWeb

  1. What is the lookup file limit/is there a file limit when uploading directly to browser? 
  2. How long will data in lookup files be stored (do they ever get deleted after a time period?)
  3. Does joining with large lookups with OUTPUT/OUTPUTNEW have a limit to how much data is joined between the two lookups OR an index/sourcetype and a lookup?
  4. Is there a max limit for the amount of records that can be overwritten into the lookup when you run |outputlookup?

 

Business Use Case Example:

We are ingesting logs and putting them into an index/sourcetype. We've created a search to append the sourcetype with a lookup file by an ID. This search will get updated everyday by the hour and output a new lookup. The amount of new data that gets added into the sourcetype varies in the 10s up to the 100s daily. If we keep doing it this way, the data size for the lookup on the browser will increase exponentially so I'm worried if there is a limit. Also open to recommendations on a better way of doing this.

Labels (2)
0 Karma

Roy_9
Motivator

Hello,

On our cloud stack, splunk support defined a limit of 50 MB for the lookup files.

I guess this might be variable.

 

Thanks

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Splunk does not store lookup files on a browser.

Data in a lookup file stays there until you change it or remove the lookup file.

Any limits between join and lookup are within the join command, not the lookup.

As far as I know, there is no limit on the number of results outputlookup can process.

---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

Updated Team Landing Page in Splunk Observability

We’re making some changes to the team landing page in Splunk Observability, based on your feedback. The ...

New! Splunk Observability Search Enhancements for Splunk APM Services/Traces and ...

Regardless of where you are in Splunk Observability, you can search for relevant APM targets including service ...

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...