topic Re: Need help modifying this correlation search in Splunk Enterprise Security

Need help modifying this correlation search

echojacques — Fri, 23 Aug 2013 15:43:42 GMT

This correlation search detects a "substantial increase in port activity" and it works well. How can I tune/modify it so that it is a little less sensitive so that it doesn't "trigger" as often? Basically, increase the threshold/limits. I'm pretty new with Splunk searches in general so I'm a little hesitant to modify this myself. Thanks!

| `tstats` sum(count) from sa_port_proto groupby _time,transport,dest_port span=30m | stats sum(count) as count by _time,transport,dest_port | `timeDiff` | appendpipe [search timeDiff<=86400 | stats max(_time) as _time,sum(count) as count by transport,dest_port | eval group="Last 24 hours"] | eval group=if(_time<relative_time(time(),"@d") AND timeDiff<=5184000,"Last 60 days",group) | bin _time span=1d | stats sum(count) as count by _time,group,transport,dest_port | eval temp=if(group="Last 60 days",transport.dest_port,null()) | eventstats stdev(count) as stdev,avg(count) as avg by temp | eventstats max(stdev) as stdev,max(avg) as avg by transport,dest_port | dedup transport,dest_port sortby -_time | eval limit=(3*stdev)+avg | eval diff=count-limit | search diff>0

Re: Need help modifying this correlation search

sowings — Fri, 23 Aug 2013 16:08:32 GMT

The "limit" field near the end is the magic. It gets set to 3 standard deviations (3 sigma) from the average. According to this wiki page, that should account for 99% of the values in a standard distribution curve. You could change it to 3.5 or 4 stdev, but that would probably never fire. It's a balancing act between crying wolf and not hearing about a potential problem.

Re: Need help modifying this correlation search

echojacques — Fri, 23 Aug 2013 16:11:41 GMT

Thank very much for the info! I will tweak it (starting at 3.1) and find a healthy balance.

Any idea what the: 'search diff>0' at the end means?

Thanks

Re: Need help modifying this correlation search

sowings — Fri, 23 Aug 2013 16:15:03 GMT

"Filter results to those where the value of the 'diff' field is greater than zero."

We first set the limit with the stdev term I identified earlier. Next, we set a new field called diff which is the difference between the count of events and our "limit" or threshold. Finally, we look for cases where this is greater than zero, indicating "more events than our threshold".

Re: Need help modifying this correlation search

echojacques — Fri, 23 Aug 2013 16:18:24 GMT

Great explanation, now I understand, thanks again 🙂

Re: Need help modifying this correlation search

dirkmeeuwsen — Wed, 18 Nov 2015 21:05:03 GMT

It looks like the new version of the Enterprise Security App is using extreme search and looks like this:

| tstats allow_old_summaries=true count from datamodel=Network_Traffic by All_Traffic.dest_port | `drop_dm_object_name("All_Traffic")` | localop | xswhere count from count_by_dest_port_1d in network_traffic by dest_port is extreme

Any idea how to tweak the threshold on this?

Re: Need help modifying this correlation search

kamal_jagga — Thu, 12 Oct 2017 18:19:32 GMT

I am also facing the same issue with the new version of this query using extreme search.
Our ES setup is not very old so the xswhere is not able to establish a base line for each destination port. I read that we can check the current threshold level for this using extreme search. But I am unable to do that.
Kindly advise.