Splunk Enterprise Security

Enterprise Security (ES) asset lookups failing with SRC and dest fields reporting “unknown-internal” and “unknown-external”

ekost
Splunk Employee
Splunk Employee

Running ES 5.1 on Splunk 7.1. The asset lookups have been working fine. This morning the SRC and dest fields display “unknown-internal” and “unknown-external” in place of the usual asset information. The lookups are populating normally, and nothing’s changed on the ES Search Head recently.

Digging deeper, the behavior makes it appear that the CIDR lookup asset_by_cidr.csv is happening before the string lookup assets_by_str.csv. But why would that be, and what would change the normal order-of-operations behavior for those lookups?

0 Karma

ekost
Splunk Employee
Splunk Employee

The size of the assets_by_str.csv lookup is exceeding the limits.conf setting max_memtable_bytes, and is being treated as a batched lookup. Splunk will manage lookups in-memory for CSVs less than 10MB (by default,) and index the rest as external batch based. The in-memory lookups get executed immediately, and order is consistent. For batched lookups, Splunk waits until a certain batch size is reached before performing a lookup operation. Due to this, there are higher chances of running into out-of-order conflicts when utilizing indexed lookups.

Raise the limits.conf setting max_memtable_bytes to a value larger than your assets_by_string.csv lookup on the SH and Indexers to eliminate the issue. Note: this will use more RAM on the hosts.

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...