In this Splunk's video how do the values of "avg_gap" and "count" been specified?

[rosho]

Hi

In this video of Splunk: https://www.splunk.com/en_us/resources/videos/splunk-for-security-investigation-command-and-control-...

It can be seen how to detect C&C traffic.
It looks for patterns in communication: an average gap smaller than 50 and a number of connections to a website higher than 500.

But how have those numbers been specified? I mean, why 50 and why 500? Where can I find more information?

index=example_C&C_Splunk   
| streamstats current=f last(_time) as next_time by dest
| eval gap = next_time - _time
| stats count, avg(gap) as avg_gap, var(gap) as var_gap by site dest src
| search avg_gap<50 count>500
| sort avg_gap

[youngc_splunk]

Hi @rosho ! I am Young Cho, author of "Getting Started with Splunk Security" app. Saw this question and also saw @woodcock 's good answer. This is to show you the approach (Technique) to engineer features from security activities for either machine learning and applying statistics. So, the real application should really consider the right threshold or it also can be a machine learning, to point out anomalies based on the type of traffic in the network.

Also, streamstats should do "| streamstats avg(gap) by src dest" where I feel that it should be by "src dest" instead of just "dest". Let me know what you think and love to know if some of these techniques work for your environment.

[rosho]

Hi @youngc_splunk !

Thanks for taking the time to answer my question!

I used your code SPL as it is. The index is a firewall. What I found is the following:

• I am not sure if the avg_gap and the var_gap are in seconds or ms
• The results that I am obtaining includes the communication between switches and routers
• I think it can be used to not only detect C&C but also if someone is gaming or video streaming instead of working.

Can you give me some other clues on how to find the threshold?
And for the Machine Learning, I thought of applying a clustering algorithm over the features I obtained thanks to your SPL. Is there other way?

Thank you very much

[woodcock · Accepted solution]

Did you mean to tag this with the Splunk Security Essentials app (https://splunkbase.splunk.com/app/3435/) because that is not the same thing as the Getting Started with Splunk Security app (https://splunkbase.splunk.com/app/3358/). In any case, the latter app's name implies and the description text (included below) cleary indicates that this is a tutorial with functional examples. It is up to you to adjust to your taste. In any case, if you really need to know why, ask the author who is listed as Young Cho who is probably @youngc_splunk (hopefully he will followup).

This is the description of the app (copied from splunkbase):

Get started with security investigation and analysis in Splunk Enterprise or Splunk Cloud with the Getting Started with Splunk Security app. This guided experience walks you through using simple searches to identify, validate, and analyze security events in your environment. Rapidly gain insight and situational awareness to remediate threats quickly.

This app includes sample data and includes step-by-step tutorials for constructing security-relevant searches for specific use cases.

[woodcock]

You should treat just about every app that you didn't write as though it is only partly done. You should never expect that anything that you download from splunkbase to work as you need it to without modification. The Splunk_TA_nix is definitely in that boat. That is just the way that it is.

[David]

I downvoted this post because it's off topic, and not helpful. ❤️ @woodcock

[jawaharas]

Interesting question!

It's merely a logic to find network anomaly.

In this example, If a particular internal host makes more number of connections (example threshold: 500) to external hosts with minimal intervals (example threshold: less than 50 seconds) , certainly that doesn't sound right.