The size of the
assets_by_str.csv lookup is exceeding the limits.conf setting
max_memtable_bytes, and is being treated as a batched lookup. Splunk will manage lookups in-memory for CSVs less than 10MB (by default,) and index the rest as external batch based. The in-memory lookups get executed immediately, and order is consistent. For batched lookups, Splunk waits until a certain batch size is reached before performing a lookup operation. Due to this, there are higher chances of running into out-of-order conflicts when utilizing indexed lookups.
Raise the limits.conf setting
max_memtable_bytes to a value larger than your
assets_by_string.csv lookup on the SH and Indexers to eliminate the issue. Note: this will use more RAM on the hosts.