Add lookup based source for ES

[proylea]

Looking over the clients configuration for adding a lookup based source for Enterprise Security Threat Intelligence, it appears to be configured correctly.
However I still see zero events in the dashboard even though a search returns the test values for threats that have been ingested.
The source lookup for IP's containing the Crowdstrike IOC's has global permissions and contains 3 fields only
description, ip, weight
The document followed for this configuration is here
http://docs.splunk.com/Documentation/ES/3.3.0/Install/Configureblocklists#Add_a_file_based_threat_so...

Continuing to look for the source of the problem but would appreciate any input from our awesome Splunk crew.

[mdessus_splunk]

Hi,

first, what version of ES are you using: 3.3 is an old release and you should look at the relevant doc version like this one: http://docs.splunk.com/Documentation/ES/5.0.0/Admin/UploadCSVthreatfile . That said, the file format should be the same.
You should check if your threat list is correctly uploaded, either by looking in the Threat artifacts dashboards, either using this command: | inputlookup threatintel_by_cidr

Next, validate that your src field from your event is correctly mapped to the CIM, and is used by ES. What kind of data do you want to match to ?

[proylea]

The lookup is local_ip_intel and it contains IP addresses and descriptions
In the Threat intelligence audit dashboard the download status is blank.

The source field is correctly mapped to the CIM

The threat list "local_ip_intel" does not appear when I execute
| inputlookup threatintel_by_cidr

So I assume I need the content from the new lookup "local_ip_intel" to end up in the "threatintel_by_cdr" lookup. and if so how is it supposed to get there?

I notice when you upload a new threat list it places it in the local/data/threat_intel dir in the app.
Are the threat lists supposed to live there? these ones are currently in the lookup dir
What is the standard for these threat lists, the documentation is not that clear.

[mdessus_splunk]

Just to be sure, you configured the local input in ES here: Data inputs » Threat Intelligence Management » local_lookups ?

[proylea]

Under threat intelligence management it looks like this

Name Directory
da_ess_threat_default $SPLUNK_HOME/etc/apps/DA-ESS-ThreatIntelligence/default/data/threat_intel
da_ess_threat_local $SPLUNK_HOME/etc/apps/DA-ESS-ThreatIntelligence/local/data/threat_intel
local_lookups ignored
sa_threat_local $SPLUNK_HOME/etc/apps/SA-ThreatIntelligence/local/data/threat_intel

The lookups in question are currently in the app/lookups dir

The events that I am trying to get picked up into the "threat_activity" index are watchguard logs like this:
Apr 10 13:32:11 -FB-02 *1016F5DC (2018-04-10T03:32:11) http-proxy[2256]: msg_id="1AFF-0024" Allow 2-Inside 5-Uecomm10 tcp 172...* 212...* 51153 80 msg="HTTP request" proxy_act="HTTP-Client.3" op="GET" dstname="api.wipmania.com" arg="/jsonp?callback=jQuery191009073215578267857_1523331069485&_=1523331069486" sent_bytes="442" rcvd_bytes="602" elapsed_time="0.657724 sec(s)" app_id="128" app_cat_id="13" app_name="Microsoft Edge" app_cat_name="Web services" reputation="1" reason="262189" action="allow" (HTTP-proxy-00)

[rom1btn]

Hi proylea,

Looking at your dashboard, have you checked that your tokens are well configured in your search?
- For the four filters
- And especially for the 'Threat match value', do you have '*' value by default?

Sometimes things are simple, I hope this would help