Splunk Search
Highlighted

Detecting outliers query

Hi everyone
I was reading through "endpoint security analyst with Splunk (online experience)" which you can find here:
http://si_usecase_02.splunkoxygen.com/en-US/app/OLE_Security_Endpoint/sec_search_01?tour=gs_main_int...

This is a four exercises tutorial that will show you how to detect and prevent advanced malware,
anyway I was moving along with the tutorial step by step and this statement caught my attention:

Any process activities with a command line command length that is more than four times the average and standard deviation command line command lengths for each host is an outlier

My question is why?
is this a standard formula?
an axiom?
where did this come from?

Here is the query that was used in this tutorial:

sourcetype=xmlwineventlog:microsoft-windows-sysmon/operational EventCode=1
| eval cmdlen=len(CommandLine)
| eventstats avg(cmdlen) as avg, stdev(cmdlen) as stdev by host
| stats max(cmdlen) as maxlen, values(avg) as avgperhost, values(stdev) as stdevperhost by host, CommandLine
| eval threshold = 4 * ( stdevperhost + avgperhost )
| where maxlen > threshold 

Here you can see it in the second eval command (in the line before the last).

Thanks in advance.

0 Karma
Highlighted

Re: Detecting outliers query

Ultra Champion

My question is why?: These are the techniques and best practices how to detect and prevent ransomware infections.
is this a standard formula?: no
an axiom?: no
where did this come from? best practices

https://www.splunk.com/en_us/form/security-investigation-online-experience-endpoint.html

Once ransomware hits, you have a very short window of time to detect it and respond, before critical business data is encrypted and therefore no longer accessible to you, your business, or your customers. This sandbox environment guides you, in a safe environment with "live threats," through techniques and best practices on how to detect and prevent ransomware infections.

How about this as an example of a long string of commands?

http://unit42.elegance.work/unit42-pulling-back-the-curtains-on-encodedcommand-powershell-attacks/

Highlighted

Re: Detecting outliers query

Communicator

This doesn't make much sense, and the threshold is too relaxed for anomaly detection. Statistically speaking, if it is 3-sigma or 4-sigma away from the mean/average, it is considered an outlier/anomaly. I think it was a typo in the tutorial, and it should be like this:

 sourcetype=xmlwineventlog:microsoft-windows-sysmon/operational EventCode=1
 | eval cmdlen=len(CommandLine)
 | eventstats avg(cmdlen) as avg, stdev(cmdlen) as stdev by host
 | stats max(cmdlen) as maxlen, values(avg) as avgperhost, values(stdev) as stdevperhost by host, CommandLine
 | eval threshold = avgperhost + 4 *stdevperhost 
 | where maxlen > threshold 

Note I changed the the 5th line of the search to | eval threshold = avgperhost + 4 *stdevperhost .

View solution in original post

Highlighted

Re: Detecting outliers query

Communicator

OK. I found something published on Splunk website, pay attention to 5m40s mark of the video tutorial.

0 Karma
Highlighted

Re: Detecting outliers query

Ultra Champion

4 *stdevperhost3 * stdevperhost (99.7%)
I think this is good enough for me.

Highlighted

Re: Detecting outliers query

Thank you, both of you @tauliang @to4kawa

0 Karma
Highlighted

Re: Detecting outliers query

Communicator

@muralikoppula you are most welcome. If it answers your question, could you please accept it as an answer? Thanks.

0 Karma