Splunk Enterprise Security
Highlighted

How is the index=threat_activity filled up with data in splunk Enterprise Security (ES)? How can I add an additional field to it?

Path Finder

We have got squid proxy logs that are compared with the threat lists in splunk ES.
It works fine, but on the list on splunk ES Advanced Threat - Threatlist Activity - Threat Activity Details we only see ip addresses in the dest field.

alt text

In the log events of squid I also have the URL, which is much more human readable.
What I want is to add the field urihost also to my data in the index=threatactivity.

It looks like the index is filled by the a saved search: Threat - Source And Destination Matches - Threat Gen
The data looks like:

11/27/2015 14:15:00 +0100, searchname="Threat - Source And Destination Matches - Threat Gen", searchnow=1448630100.000, infomintime=1448622000.000, infomaxtime=1448630100.000, infosearchtime=1448630114.038, dest="xxx.xxx.xx.xxx", origsourcetype="cisco:asa", src="yyy.yyy.yyy.yyy", threatcollection=ipintel, threatcollectionkey="emergingthreatsipblocklist|43.229.52.0/22", threatkey=emergingthreatsipblocklist, threatmatchfield=src, threatmatchvalue="43.229.53.53"

The search looks like this:

| src_dest_tstats("allowed") | truncate_domain_dedup(src) | truncate_domain_dedup(dest) | threatintel_multilookup(src) | threatintel_multilookup(dest) | search threatcollectionkey=* | fields - count | zipexpand_threat_matches | fields sourcetype,src,dest,threat*

I tried to add just | fields sourcetype,src,dest, uri_host, threat* but this is not working.

Does anybody have a description of this macros? Or where can I find them to adjust them?

Highlighted

Re: How is the index=threat_activity filled up with data in splunk Enterprise Security (ES)? How can I add an additional field to it?

Splunk Employee
Splunk Employee

You can find the macro in the GUI under Settings -> Advanced Search -> Search Macros. You can dissect them from there.

0 Karma
Highlighted

Re: How is the index=threat_activity filled up with data in splunk Enterprise Security (ES)? How can I add an additional field to it?

Path Finder

If you want to capture the domain field values in the threat activity dashboard, you need to write a search driven lookup ( say for example "Threat - URL squid Matches - Threat Gen".

It would be good if you have a datamodel for squid or you can go with normal index command. Please find the query below.

The query will look for the squid domains, followed by comparing with the lookup to see if hit.

| tstats prestats=true local=false values(sourcetype) as sourcetype,values(squid.src),values(squid.dest) from datamodel=squid by squid.domain | eval url='squid.domain' | eval threatmatchfield="squid.domain" | eval url=if(isnull(url),'squid.domain',url) | eval threatmatchfield=if(isnull(threatmatchfield),"url",threatmatchfield) | stats values(sourcetype) as sourcetype,values(squid.src) as src,values(squid.dest) as dest by url,threatmatchfield | lookup update=true ip_intel domain as url OUTPUTNEW

Hope this works

0 Karma
Highlighted

Re: How is the index=threat_activity filled up with data in splunk Enterprise Security (ES)? How can I add an additional field to it?

Explorer

Hello,

You need to understand how this search works. This correlation rule is running against all the data source and matching the fields against the Source types. In squid proxy sourcetype, extract the field called dest for the URL. then you threat list activity will match the dest filed to URL. Hope this should work.

View solution in original post

0 Karma
Highlighted

Re: How is the index=threat_activity filled up with data in splunk Enterprise Security (ES)? How can I add an additional field to it?

Path Finder

Yes, that's it. I already fixed it last year.

0 Karma
Highlighted

Re: How is the index=threat_activity filled up with data in splunk Enterprise Security (ES)? How can I add an additional field to it?

Path Finder

If you want to capture the domain field values in the threat activity dashboard, you need toa create a saved search ( say for example "Threat - URL squid Matches - Threat Gen".

It would be good if you have a datamodel for squid or you can go with normal index command. Please find the query below.

The query will look for the squid domains, followed by comparing with the lookup to see if there's a hit.

| tstats prestats=true local=false values(sourcetype) as sourcetype,values(squid.src),values(squid.dest) from datamodel=squid by squid.domain | eval url='squid.domain' | eval threatmatchfield="squid.domain" | eval url=if(isnull(url),'squid.domain',url) | eval threatmatchfield=if(isnull(threatmatchfield),"url",threatmatchfield) | stats values(sourcetype) as sourcetype,values(squid.src) as src,values(squid.dest) as dest by url,threatmatchfield | lookup update=true ip_intel domain as url OUTPUTNEW

Hope this works

0 Karma
Highlighted

Re: How is the index=threat_activity filled up with data in splunk Enterprise Security (ES)? How can I add an additional field to it?

Path Finder

Paste the query in search box and press Ctrl+shift+E. You will get to know the full query, modify the query based on your requirement.
"Threat - Source And Destination Matches - Threat Gen" check logs from datamodels of Network_Traffic, Web & IDS.

0 Karma