Need help modifying this correlation search

[echojacques]

This correlation search detects a "substantial increase in port activity" and it works well. How can I tune/modify it so that it is a little less sensitive so that it doesn't "trigger" as often? Basically, increase the threshold/limits. I'm pretty new with Splunk searches in general so I'm a little hesitant to modify this myself. Thanks!

| `tstats` sum(count) from sa_port_proto groupby _time,transport,dest_port span=30m | stats sum(count) as count by _time,transport,dest_port | `timeDiff` | appendpipe [search timeDiff<=86400 | stats max(_time) as _time,sum(count) as count by transport,dest_port | eval group="Last 24 hours"] | eval group=if(_time<relative_time(time(),"@d") AND timeDiff<=5184000,"Last 60 days",group) | bin _time span=1d | stats sum(count) as count by _time,group,transport,dest_port | eval temp=if(group="Last 60 days",transport.dest_port,null()) | eventstats stdev(count) as stdev,avg(count) as avg by temp | eventstats max(stdev) as stdev,max(avg) as avg by transport,dest_port | dedup transport,dest_port sortby -_time | eval limit=(3*stdev)+avg | eval diff=count-limit | search diff>0

[sowings · Accepted solution]

The "limit" field near the end is the magic. It gets set to 3 standard deviations (3 sigma) from the average. According to this wiki page, that should account for 99% of the values in a standard distribution curve. You could change it to 3.5 or 4 stdev, but that would probably never fire. It's a balancing act between crying wolf and not hearing about a potential problem.

[dirkmeeuwsen]

It looks like the new version of the Enterprise Security App is using extreme search and looks like this:

| tstats allow_old_summaries=true count from datamodel=Network_Traffic by All_Traffic.dest_port | `drop_dm_object_name("All_Traffic")` | localop | xswhere count from count_by_dest_port_1d in network_traffic by dest_port is extreme

Any idea how to tweak the threshold on this?

[kamal_jagga]

I am also facing the same issue with the new version of this query using extreme search.
Our ES setup is not very old so the xswhere is not able to establish a base line for each destination port. I read that we can check the current threshold level for this using extreme search. But I am unable to do that.
Kindly advise.

[echojacques]

Great explanation, now I understand, thanks again 🙂

[sowings]

"Filter results to those where the value of the 'diff' field is greater than zero."

We first set the limit with the stdev term I identified earlier. Next, we set a new field called diff which is the difference between the count of events and our "limit" or threshold. Finally, we look for cases where this is greater than zero, indicating "more events than our threshold".

[echojacques]

Thank very much for the info! I will tweak it (starting at 3.1) and find a healthy balance.

Any idea what the: 'search diff>0' at the end means?

Thanks