Splunk Enterprise Security

Maximum Asset & Identity Lookup Size

Path Finder

What is the maximum recommended size for asset/identity lookups?

https://dev.splunk.com/enterprise/docs/developapps/enterprisesecurity/assetandidentityframework/ 

I've had issues with Splunk handling large numbers of assets and/or identities.  I increased the maximum bundle size to 4GB, but still had to distribute the entire huge bundle every time an identity changed.

Is there an option to use a KV store for assets & identities? Or a way to update them with a diff, rather than pushing the entire lookup?

Is there a memory requirement for a certain number of assets & identities? Or any related performance impact for having a large number of assets & identities?

Labels (2)
0 Karma
1 Solution

SplunkTrust
SplunkTrust
The guidance is to keep bundle sizes below 1GB.
---
If this reply helps you, an upvote would be appreciated.

View solution in original post

0 Karma

SplunkTrust
SplunkTrust
Try to keep bundle size below 1GB. Beyond that you'll have problems.
Blacklist the A&I lookups from the bundle and push them to the indexers using a different method (scp via a cron job, for example).
---
If this reply helps you, an upvote would be appreciated.

Path Finder

Thanks for the help.

With large A&I lookups, does Splunk provide memory recommendation for acceptable performance?

For example, if my asset list only contains ip, dns, priority, bunit,  andcategory, how many Class A networks can I put in the lookup if the networks are 50% allocated?

0 Karma

SplunkTrust
SplunkTrust
The guidance is to keep bundle sizes below 1GB.
---
If this reply helps you, an upvote would be appreciated.

View solution in original post

0 Karma

Path Finder

I don't know if I can stay under that size, or even under 4GB, but I understand that is the recommended limit.

0 Karma

SplunkTrust
SplunkTrust
Like I said in my first reply, if you keep large lookup files out of the bundle it will help keep the bundle size down.
---
If this reply helps you, an upvote would be appreciated.
0 Karma

Path Finder

Thanks for clarifying that. Can KV lookups be distributed through different channels, like scp?

0 Karma

SplunkTrust
SplunkTrust
KVStore collections are not included in the search bundle.
---
If this reply helps you, an upvote would be appreciated.
0 Karma