Splunk Enterprise Security

Why ES threat list lookup fails on pivot data, but works with direct search on indexed data?

dragoslungu
Explorer

Is there anything different when running a lookup on data returned by a pivot compared to the same lookup running on data returned from a direct search on indexed data ?

The following lookup returns 1400 matches :

tag=ids tag=attack | lookup threatlist_lookup_by_str key AS dest
OUTPUTNEW name AS dest_threatlist_name| search dest_threatlist_name=*

If the same lookup runs agains a pivot, there are no matches:

| tstats `summariesonly` count from datamodel=Intrusion_Detection where
*  by _time,IDS_Attacks.src,IDS_Attacks.dest span=10m | `drop_dm_object_name("IDS_Attacks")` | lookup threatlist_lookup_by_str key AS dest OUTPUTNEW name AS dest_threatlist_name| search dest_threatlist_name=*

The root element definition of the Intrusion_Detection datamodel is the same as the direct search

tag=ids tag=attack

The purpose of this is to integrate custom threat lists in the default ES content like the "Threat List Activity" dashboard. Most of the ES default panels use datamodel / pivot searches which causes the lookups to fail.

Thanks.

sjohnson_splunk
Splunk Employee
Splunk Employee

Since your tstats is using summariesonly (which is true), it's possible that your data model accelerations are not keeping up and there are no results in the time window.

0 Karma
Get Updates on the Splunk Community!

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...