I keep getting this error on my 3 Enterprise Security search heads:
msg="A lookup table used in a CIDR or WILDCARD definition exceeds the maximum allowable value" file="asn_by_cidr.csv" size="16360595" param="max_memtable_bytes" limit="10000000".
I am aware of the fix: https://answers.splunk.com/answers/152483/splunk-app-for-enterprise-security-where-to-change-the-set...,
but after I made the suggested change to all 3 search heads, the error keeps popping up.
I have verified with btool that the max_memtable_bytes limit has been set to 20000000:
./bin/splunk cmd btool --debug limits list |grep mem /opt/splunk/etc/apps/tsp_esh_limits/default/limits.conf max_memtable_bytes = 20000000
Limits.conf is one file that is not passed to peers/indexers with the search bundle. You must put limits.conf on your peers/indexers too.
# limits.conf settings and DISTRIBUTED SEARCH # Unlike most settings which affect searches, limits.conf settings are not # provided by the search head to be used by the search peers. This means that if # you need to alter search-affecting limits in a distributed environment, typically # you will need to modify these settings on the relevant peers and search head for # consistent results.
A few things:
The max_memtable_bytes parameter is set under the [lookup] stanza in your limits.conf?
Does asn_by_cidr.csv live in the tsp_esh_limits app, if not, is the app configured to share its configuration globally, as limits.conf is evaluated at the app/user level. If your lookup table lives in another app, and tsp_esh_limits does not share its configuration globally, then max_memtable_bytes = 20000000 will not apply.