topic Re: How to assign a custom scoring scale based on device type? (eval related) in Splunk Search

How to assign a custom scoring scale based on device type? (eval related)

UMDTERPS — Mon, 16 Dec 2019 17:15:27 GMT

Hello,

Currently we have a scoring for our systems that counts each server, router, switch, firewall, workstation, etc on an equal playing field. We count norton as 40% (.40), tanium 25% (.25), nessus 10% (.10), openvas 5% (.05), and nexpose 10% (.10).
The scoring system SPL that works is something similar to below:

`comment("1 means that the device was found by a scanner in | inputlookup scanner_visbility.csv")`
| inputlookup scanner_visbility.csv
| eventstats count(eval(ip)) as total 
             count(eval(norton="1")) as norton_count
             count(eval(tanium="1")) as tanium_count
             count(eval(nessus="1")) as nessus_count
             count(eval(metasploit="1")) as metasploit_count
             count(eval(openvas="1")) as oprnvas_count
             count(eval(nexpose="1")) as nexpose_count
| eval norton_result = round((((norton_count / total) * 100) * 0.40),2)
| eval tanium_result = round((((tanium_count / total) * 100) * 0.25),2)
| eval nessus_result = round((((nessus_count / total) * 100) * 0.10),2) 
| eval metaspoit_result = round((((metasploit_count / total) * 100) * 0.10),2)
| eval openvas_result = round((((openvas_count / total) * 100) * 0.05),2)
| eval nexpose_result = round((((nexpose_count / total) * 100) * 0.10),2)
| eval visibility =  norton_result + tanium_result + nessus_result + metasploit_result + openvas_result + nexpose_result
| fields visibility

What we want to do with the above code is to count each network device on a different scoring scale. For Example, for routers we want ONLY count Nessus, Metasploit, and Nexpose. We want to assign Nessus as 40%, Metasploit as 40%, and Nexpose as 10%.
How would I go about assigning a different scoring scale for certain devices?

Re: How to assign a custom scoring scale based on device type? (eval related)

woodcock — Mon, 16 Dec 2019 17:50:40 GMT

First, create a lookup file called servertypes_scanner_weights.csv1 with these fields:

servertype,nessus_weight,metasploit_weight,nexpose_weight,norton_weight,tanium_weight,oprnvas_weight,Other_weight,Stuff_weight,Here_weight
routers,.4,.4,.1,0,0,0,0,0,0
another_type,0,0,0,.5,.75,0,0,0,0

Now, assuming that you have a field called servertype (if not, create one, probably with RegEx on host), you do something like this:

| inputlookup scanner_visbility.csv
| eventstats count(eval(ip)) as total 
             count(eval(norton="1")) as norton_count
             count(eval(tanium="1")) as tanium_count
             count(eval(nessus="1")) as nessus_count
             count(eval(metasploit="1")) as metasploit_count
             count(eval(openvas="1")) as oprnvas_count
             count(eval(nexpose="1")) as nexpose_count
| lookup servertypes_scanner_weights.csv servertype
| eval visibility = 0
| foreach *_count [ eval  visibility = visibility + (<<MATCHSTR>>_weight * 100 * <<FIELD>> / total) | fields - <<MATCHSTR>>_* ]
| eval visibility = round(visibility, 2)
| fields visibility

This also fixes rounding errors the way that you were doing it.

Re: How to assign a custom scoring scale based on device type? (eval related)

to4kawa — Mon, 16 Dec 2019 18:16:08 GMT

| makeresults count=1000 
| eval count=random() % 6 
| eval vuln_soft=mvindex(split("norton#tanium#nessus#metasploit#openvas#nexpose","#"),count) 
| eval equipment=mvindex(split("server,router,switch,firewall,workstation",","),(random() % 4))
| stats count by vuln_soft equipment
| eval args1=case(vuln_soft=="norton", .40
    ,vuln_soft=="tanium",  .25
    ,vuln_soft="nessus",  .10
    ,vuln_soft="metasploit",  .10
    ,vuln_soft="openvas",  .05
    ,vuln_soft="nexpose",  .10
    ,true(), 1)
| eval args2=case(vuln_soft=="nessus", .4,vuln_soft=="metasploit", .4, vuln=="nexpose", .1, true(),1)
| eval flag=case(equipment=="router",1, true(),NULL)
| eval count=if(flag==1,count * args2,count * args1)
| stats sum(count) as count by vuln_soft
| sort 0 - count

If the weight doesn't change, CSV and lookup are better.

Re: How to assign a custom scoring scale based on device type? (eval related)

UMDTERPS — Mon, 16 Dec 2019 20:46:43 GMT

1.  | inputlookup scanner_visibility.csv
2.  | lookup visibility_blue.csv Acronym AS application local=t OUTPUTNEW "Risk Score"
3.  | lookup server_dump.csv Acronym AS application local=t OUTPUTNEW "Authorization Removal Date"
4.  | rename norton_assets as norton 
5.  | eval norton = if(like(norton, "%2019") AND relative_time(now(), "-30d@d") < strptime(norton,"%m/%d/%Y"), 1, 0)
6.  | eval tanium = if(like(tanium, "%2019") AND relative_time(now(), "-30d@d") < strptime(tanium,"%m/%d/%Y"), 1, 0)
7.  | eval nessus = if(like(nessus, "%2019") AND relative_time(now(), "-30d@d") < strptime(nessus,"%m/%d/%Y"), 1, 0)
8.  | eval metasploit = if(like(metasploit, "%2019") AND relative_time(now(), "-30d@d") < strptime(metasploit,"%m/%d/%Y"), 1, 0)
9.  | eval openas = if(like(openvas, "%2019") AND relative_time(now(), "-30d@d") < strptime(openvas,"%m/%d/%Y"), 1, 0)
10. | eval nexpose = if(like(nexpose, "%2019") AND relative_time(now(), "-30d@d") < strptime(nexpose,"%m/%d/%Y"), 1, 0)
11.               count(eval(norton="1")) as norton_count
12.               count(eval(tanium="1")) as tanium_count
13.               count(eval(nessus="1")) as nessus_count
14.               count(eval(metasploit="1")) as metasploit_count
15.               count(eval(openvas="1")) as oprnvas_count
16.               count(eval(nexpose="1")) as nexpose_count
17.  | lookup servertypes_scanner_weights.csv servertype
18.  | eval visibility = 0
19.  | foreach *_count [ eval  visibility = visibility + (<<MATCHSEG>>_weight * 100 * <<FIELD>> / total) | fields - <<MATCHSEG>>_* ]
20.  | eval visibility = round(visibility, 2)
21.  | fields visibility

The code with the time I added looks at fields less than 30 days old ( it shouldn't affect the scoring). I am getting the following error:

Error in 'eval' command: The expression is malformed. An unexpected character is reached at '<<MATCHSEG>>_weight * 100 * field / total)'.
The search job has failed due to an error. You may be able view the job in the Job Inspector.

Re: How to assign a custom scoring scale based on device type? (eval related)

woodcock — Mon, 16 Dec 2019 22:55:42 GMT

Had a typo: <<MATCHSEG>> should have been <<MATCHSTR>>. I fixed the original answer text.

Re: How to assign a custom scoring scale based on device type? (eval related)

UMDTERPS — Wed, 30 Sep 2020 03:23:14 GMT

The search runs with no errors, however the search does not return a visibility percentage, it's just blank.

Example:

| fields application servertype ip visibility

HHS_System workstation 192.168.1.50
HHS_System server 192.168.1.55
PPH_System workstation 192.168.2.50
PPH_System server 192.168.2.51
PPH_System router 192.168.1.1

Any ideas? =0(

Re: How to assign a custom scoring scale based on device type? (eval related)

UMDTERPS — Wed, 30 Sep 2020 03:25:33 GMT

| inputlookup scanner_visibility.csv
| lookup visibility_blue.csv Acronym AS application local=t OUTPUTNEW "Risk Score"
| lookup server_dump.csv Acronym AS application local=t OUTPUTNEW "Authorization Removal Date"
| rename norton_assets as norton 
| lookup servertypes_scanner_weights.csv servertype OUTPUTNEW norton_weight tanium_weight nessus_weight metasploit_weight 
        openvas_weight nexpose_weight
| eval norton = if(like(norton, "%2019") AND relative_time(now(), "-30d@d") < strptime(norton,"%m/%d/%Y"), norton_weight, 0)
| eval tanium = if(like(tanium, "%2019") AND relative_time(now(), "-30d@d") < strptime(tanium,"%m/%d/%Y"), tanium_weight, 0)
| eval nessus = if(like(nessus, "%2019") AND relative_time(now(), "-30d@d") < strptime(nessus,"%m/%d/%Y"), nessus_weight, 0)
| eval metasploit = if(like(metasploit, "%2019") AND relative_time(now(), "-30d@d") < strptime(metasploit,"%m/%d/%Y"), metasploit_weight,0)
| eval openas = if(like(openvas, "%2019") AND relative_time(now(), "-30d@d") < strptime(openvas,"%m/%d/%Y"), openvas_weight, 0)
| eval nexpose = if(like(nexpose, "%2019") AND relative_time(now(), "-30d@d") < strptime(nexpose,"%m/%d/%Y"), nexpose_weight, 0)
|eventstats count(ip) as total 
      sum(norton) as norton_count
      sum(tanium) as tanium_count
      sum(nessus) as nessus_count
      sum(meteasploit) as metasploit_count
      sum(openvas) as openvas_count  
      sum(nexpose) as nexpose_count
      count(eval(found="Yes")) as found_yes by system
| eval norton_score = round (((norton_count / total)*100), 2)
| eval tanium_score = round (((tanium_count / total)*100), 2)
| eval nessus_score = round (((nessus_count / total)*100), 2)
| eval metasploit_score = round (((metasploit_count / total)*100), 2)
| eval openvas_score = round (((openvas_count / total)*100), 2)
| eval nexpose_score = round (((nexpose_count / total)*100), 2)
| dedup system
| eval final_result = norton_score + tanium_score + nessus_score + metasploit_score + openvas_score + nexpose_score
| rename final_result as visbility
| fields system total visibility norton_count norton_score

I got with a programmer at work and he suggested to use an "| lookup" to loop over the weights in the servertypes_scanner_weights.csv and OUTPUT them as new fields. With that we reworked the "count"s as "sum"s and brought back the "eval" statements without the scoring. It worked!

| fields system total visibility norton_count norton_score

HHS_System 3 100% 3 100%
PPH_System 6 77% 3 100%

I know the above is true because I verified it manually. Thanks for the the help! =0)

Re: How to assign a custom scoring scale based on device type? (eval related)

woodcock — Fri, 20 Dec 2019 20:02:25 GMT

I had the same typo twice; I fixed it again in my original answer. Try it now. It is vastly more efficient than your other one.

Re: How to assign a custom scoring scale based on device type? (eval related)

UMDTERPS — Wed, 30 Sep 2020 03:30:04 GMT

Hi! Sorry for the late reply, I was out. I tried the following code:

| inputlookup scanner_visbility.csv
| eventstats count(eval(ip)) as total
count(eval(norton="1")) as norton_count
count(eval(tanium="1")) as tanium_count
count(eval(nessus="1")) as nessus_count
count(eval(metasploit="1")) as metasploit_count
count(eval(openvas="1")) as oprnvas_count
count(eval(nexpose="1")) as nexpose_count
| lookup servertypes_scanner_weights.csv servertype
| eval visibility = 0
| foreach count [ eval visibility = visibility + (<>_weight * 100 * <> / total) | fields - <> ]
| eval visibility = round(visibility, 2)
| fields visibility

It only shows the application name, not the visibility score.

application visibility

HHS_System
PPH_System

The solution I provided a few replies above does work, but I'm always looking for more efficient SPL.