Splunk Enterprise Security

Assets Exceeding Field Limits, Source: [merge]

pizzor
Explorer

Been getting messages saying that some identities are exceeding the field limits. I've increased the limit on some of them, but I'm having difficulty finding the exact field that is causing this issue. Is there a way to find the exact instance where this limit is being exceeded?

  • Identity: 25 assets are currently exceeding the field limits set in the Asset and Identity Management page. Data truncation will occur unless the field limits are increased. Sources: [merge].
Labels (2)
0 Karma
1 Solution

pizzor
Explorer

Sorry for the late reply, I have actually, and completely forgot to post it. Someone referred me to a search that I pasted below. You may need to adjust the ..mv_limit it looks for according to what you have configured in ES. For us it turned out the cause was from load balances responding to our vulnerability scanner with the same MAC address, causing merges. I nulled out the MACs from my search that populated the corresponding lookup with the vuln scanner info and haven't had any merge issues since.

 

| inputlookup asset_lookup_by_str | eval es_lookup_type="asset_lookup_by_str" | inputlookup append=t asset_lookup_by_cidr | eval es_lookup_type=coalesce(es_lookup_type, "asset_lookup_by_cidr") | inputlookup append=t identity_lookup_expanded | eval es_lookup_type=coalesce(es_lookup_type, "identity_lookup_expanded") 
| rename _* AS es_lookup_*

| eval es_lookup_mv_limit=25

| foreach asset ip mac nt_host dns identity [eval count_<<FIELD>>=coalesce(mvcount(<<FIELD>>), 0), es_lookup_is_problem=mvappend(es_lookup_is_problem, if(count_<<FIELD>> >= es_lookup_mv_limit, "yes - field <<FIELD>> has over ". es_lookup_mv_limit . " entries", null()))]
| where isnotnull(es_lookup_is_problem)
| table es_lookup_*, count_*, *

 

 

View solution in original post

hettervi
Builder

I'm getting the same error messages. Can't figure out what exactly is causing them. I've tried this search (and variations of it).

| inputlookup asset_lookup_by_str | stats values(dns) dc(dns) as dc by ip | sort limit=0 -dc


Also, I think that DHCP can cause troubles with the asset lists i Splunk ES. 

Check out this thread as well: https://community.splunk.com/t5/Splunk-Enterprise-Security/Assets-with-overlapping-DHCP-Addresses-Me...

Have you found any better solution than my search above?

0 Karma

pizzor
Explorer

Sorry for the late reply, I have actually, and completely forgot to post it. Someone referred me to a search that I pasted below. You may need to adjust the ..mv_limit it looks for according to what you have configured in ES. For us it turned out the cause was from load balances responding to our vulnerability scanner with the same MAC address, causing merges. I nulled out the MACs from my search that populated the corresponding lookup with the vuln scanner info and haven't had any merge issues since.

 

| inputlookup asset_lookup_by_str | eval es_lookup_type="asset_lookup_by_str" | inputlookup append=t asset_lookup_by_cidr | eval es_lookup_type=coalesce(es_lookup_type, "asset_lookup_by_cidr") | inputlookup append=t identity_lookup_expanded | eval es_lookup_type=coalesce(es_lookup_type, "identity_lookup_expanded") 
| rename _* AS es_lookup_*

| eval es_lookup_mv_limit=25

| foreach asset ip mac nt_host dns identity [eval count_<<FIELD>>=coalesce(mvcount(<<FIELD>>), 0), es_lookup_is_problem=mvappend(es_lookup_is_problem, if(count_<<FIELD>> >= es_lookup_mv_limit, "yes - field <<FIELD>> has over ". es_lookup_mv_limit . " entries", null()))]
| where isnotnull(es_lookup_is_problem)
| table es_lookup_*, count_*, *

 

 

View solution in original post