Splunk Enterprise Security

Maximum Asset & Identity Lookup Size

Path Finder

What is the maximum recommended size for asset/identity lookups?

https://dev.splunk.com/enterprise/docs/developapps/enterprisesecurity/assetandidentityframework/ 

I've had issues with Splunk handling large numbers of assets and/or identities.  I increased the maximum bundle size to 4GB, but still had to distribute the entire huge bundle every time an identity changed.

Is there an option to use a KV store for assets & identities? Or a way to update them with a diff, rather than pushing the entire lookup?

Is there a memory requirement for a certain number of assets & identities? Or any related performance impact for having a large number of assets & identities?

Labels (2)
0 Karma
1 Solution

SplunkTrust
SplunkTrust
The guidance is to keep bundle sizes below 1GB.
---
If this reply helps you, an upvote would be appreciated.

View solution in original post

0 Karma

SplunkTrust
SplunkTrust
Try to keep bundle size below 1GB. Beyond that you'll have problems.
Blacklist the A&I lookups from the bundle and push them to the indexers using a different method (scp via a cron job, for example).
---
If this reply helps you, an upvote would be appreciated.

Path Finder

Thanks for the help.

With large A&I lookups, does Splunk provide memory recommendation for acceptable performance?

For example, if my asset list only contains ip, dns, priority, bunit,  andcategory, how many Class A networks can I put in the lookup if the networks are 50% allocated?

0 Karma

SplunkTrust
SplunkTrust
The guidance is to keep bundle sizes below 1GB.
---
If this reply helps you, an upvote would be appreciated.

View solution in original post

0 Karma

Path Finder

I don't know if I can stay under that size, or even under 4GB, but I understand that is the recommended limit.

0 Karma

SplunkTrust
SplunkTrust
Like I said in my first reply, if you keep large lookup files out of the bundle it will help keep the bundle size down.
---
If this reply helps you, an upvote would be appreciated.
0 Karma

Path Finder

Thanks for clarifying that. Can KV lookups be distributed through different channels, like scp?

0 Karma

SplunkTrust
SplunkTrust
KVStore collections are not included in the search bundle.
---
If this reply helps you, an upvote would be appreciated.
0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!