topic Re: Add lookup based source for ES in Splunk Enterprise Security

Add lookup based source for ES

proylea — Fri, 20 Apr 2018 05:23:43 GMT

Looking over the clients configuration for adding a lookup based source for Enterprise Security Threat Intelligence, it appears to be configured correctly.
However I still see zero events in the dashboard even though a search returns the test values for threats that have been ingested.
The source lookup for IP's containing the Crowdstrike IOC's has global permissions and contains 3 fields only
description, ip, weight
The document followed for this configuration is here
http://docs.splunk.com/Documentation/ES/3.3.0/Install/Configureblocklists#Add_a_file_based_threat_source

Continuing to look for the source of the problem but would appreciate any input from our awesome Splunk crew.

Re: Add lookup based source for ES

rom1btn — Fri, 20 Apr 2018 11:48:37 GMT

Hi proylea,

Looking at your dashboard, have you checked that your tokens are well configured in your search?
- For the four filters
- And especially for the 'Threat match value', do you have '*' value by default?

Sometimes things are simple, I hope this would help

Re: Add lookup based source for ES

mdessus_splunk — Fri, 20 Apr 2018 12:03:54 GMT

Hi,

first, what version of ES are you using: 3.3 is an old release and you should look at the relevant doc version like this one: http://docs.splunk.com/Documentation/ES/5.0.0/Admin/UploadCSVthreatfile . That said, the file format should be the same.
You should check if your threat list is correctly uploaded, either by looking in the Threat artifacts dashboards, either using this command: | inputlookup threatintel_by_cidr

Next, validate that your src field from your event is correctly mapped to the CIM, and is used by ES. What kind of data do you want to match to ?

Re: Add lookup based source for ES

proylea — Tue, 29 Sep 2020 19:11:34 GMT

The lookup is local_ip_intel and it contains IP addresses and descriptions
In the Threat intelligence audit dashboard the download status is blank.

The source field is correctly mapped to the CIM

The threat list "local_ip_intel" does not appear when I execute
| inputlookup threatintel_by_cidr

So I assume I need the content from the new lookup "local_ip_intel" to end up in the "threatintel_by_cdr" lookup. and if so how is it supposed to get there?

I notice when you upload a new threat list it places it in the local/data/threat_intel dir in the app.
Are the threat lists supposed to live there? these ones are currently in the lookup dir
What is the standard for these threat lists, the documentation is not that clear.

Re: Add lookup based source for ES

mdessus_splunk — Mon, 23 Apr 2018 07:00:04 GMT

Just to be sure, you configured the local input in ES here: Data inputs » Threat Intelligence Management » local_lookups ?

Re: Add lookup based source for ES

proylea — Tue, 29 Sep 2020 19:12:21 GMT

Under threat intelligence management it looks like this

Name Directory
da_ess_threat_default $SPLUNK_HOME/etc/apps/DA-ESS-ThreatIntelligence/default/data/threat_intel
da_ess_threat_local $SPLUNK_HOME/etc/apps/DA-ESS-ThreatIntelligence/local/data/threat_intel
local_lookups ignored
sa_threat_local $SPLUNK_HOME/etc/apps/SA-ThreatIntelligence/local/data/threat_intel

The lookups in question are currently in the app/lookups dir

The events that I am trying to get picked up into the "threat_activity" index are watchguard logs like this:
Apr 10 13:32:11 -FB-02 *1016F5DC (2018-04-10T03:32:11) http-proxy[2256]: msg_id="1AFF-0024" Allow 2-Inside 5-Uecomm10 tcp 172...* 212...* 51153 80 msg="HTTP request" proxy_act="HTTP-Client.3" op="GET" dstname="api.wipmania.com" arg="/jsonp?callback=jQuery191009073215578267857_1523331069485&_=1523331069486" sent_bytes="442" rcvd_bytes="602" elapsed_time="0.657724 sec(s)" app_id="128" app_cat_id="13" app_name="Microsoft Edge" app_cat_name="Web services" reputation="1" reason="262189" action="allow" (HTTP-proxy-00)