Splunk Enterprise Security

Need help modifying this correlation search

Builder

This correlation search detects a "substantial increase in port activity" and it works well. How can I tune/modify it so that it is a little less sensitive so that it doesn't "trigger" as often? Basically, increase the threshold/limits. I'm pretty new with Splunk searches in general so I'm a little hesitant to modify this myself. Thanks!

| `tstats` sum(count) from sa_port_proto groupby _time,transport,dest_port span=30m | stats sum(count) as count by _time,transport,dest_port | `timeDiff` | appendpipe [search timeDiff<=86400 | stats max(_time) as _time,sum(count) as count by transport,dest_port | eval group="Last 24 hours"] | eval group=if(_time<relative_time(time(),"@d") AND timeDiff<=5184000,"Last 60 days",group) | bin _time span=1d | stats sum(count) as count by _time,group,transport,dest_port | eval temp=if(group="Last 60 days",transport.dest_port,null()) | eventstats stdev(count) as stdev,avg(count) as avg by temp | eventstats max(stdev) as stdev,max(avg) as avg by transport,dest_port | dedup transport,dest_port sortby -_time | eval limit=(3*stdev)+avg | eval diff=count-limit | search diff>0
0 Karma
1 Solution

Splunk Employee
Splunk Employee

The "limit" field near the end is the magic. It gets set to 3 standard deviations (3 sigma) from the average. According to this wiki page, that should account for 99% of the values in a standard distribution curve. You could change it to 3.5 or 4 stdev, but that would probably never fire. It's a balancing act between crying wolf and not hearing about a potential problem.

View solution in original post

Splunk Employee
Splunk Employee

The "limit" field near the end is the magic. It gets set to 3 standard deviations (3 sigma) from the average. According to this wiki page, that should account for 99% of the values in a standard distribution curve. You could change it to 3.5 or 4 stdev, but that would probably never fire. It's a balancing act between crying wolf and not hearing about a potential problem.

View solution in original post

Explorer

It looks like the new version of the Enterprise Security App is using extreme search and looks like this:

| tstats allow_old_summaries=true count from datamodel=Network_Traffic by All_Traffic.dest_port | `drop_dm_object_name("All_Traffic")` | localop | xswhere count from count_by_dest_port_1d in network_traffic by dest_port is extreme

Any idea how to tweak the threshold on this?

Contributor

I am also facing the same issue with the new version of this query using extreme search.
Our ES setup is not very old so the xswhere is not able to establish a base line for each destination port. I read that we can check the current threshold level for this using extreme search. But I am unable to do that.
Kindly advise.

0 Karma

Builder

Great explanation, now I understand, thanks again 🙂

0 Karma

Splunk Employee
Splunk Employee

"Filter results to those where the value of the 'diff' field is greater than zero."

We first set the limit with the stdev term I identified earlier. Next, we set a new field called diff which is the difference between the count of events and our "limit" or threshold. Finally, we look for cases where this is greater than zero, indicating "more events than our threshold".

0 Karma

Builder

Thank very much for the info! I will tweak it (starting at 3.1) and find a healthy balance.

Any idea what the: 'search diff>0' at the end means?

Thanks

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!