Calculating data throughput for cloud

[taijusoup64]

I'm trying to calculate the data throughput for a cloud computing solution that will be charging based on outgoing data throughput.

We're collecting on the link using security onion and forwarding those zeek logs to our splunk instance. 

index="zeek" source="conn.log"
((id.orig_h IN `front end`) AND NOT (id.resp_h IN `backend`)) OR 
((id.resp_h IN `front end`) AND NOT (id.orig_h IN `backend`))
| fields orig_bytes, resp_bytes
| eval terabytes=(((((resp_bytes+orig_bytes)/1024)/1024)/1024)/1024)
| stats sum (terabytes)

This gives me traffic throughput in and out of the network for external connections however what I need is to calculate orig_bytes only when the id.orig_h is my `frontend` and resp_bytes when id.resp_h is `frontend`.
I can get them separately by just doing two different searches and then adding the results up by hand. But I'm sure theres a way to do what I want to in one search using some sort of conditional. I've tried using where and eval if but I'm just not skilled enough it seems. 

[gcusello]

Hi @taijusoup64 ,

let me understand: you want to calculate bytes only when:  id.orig_h="frontend" AND id.resp_h="frontend", is this correct?

in this case add the condition to the eval statement:

index="zeek" source="conn.log"
((id.orig_h IN `front end`) AND NOT (id.resp_h IN `backend`)) OR 
((id.resp_h IN `front end`) AND NOT (id.orig_h IN `backend`))
| fields orig_bytes, resp_bytes
| eval terabytes=((if(id.resp_h="front end",resp_bytes,0))+(if(id.orig_h="front end",orig_bytes,0)))/1024/1024/1024/1024
| stats sum (terabytes)

Ciao.

Giuseppe

why did you used all that parenthesis?

Ciao.

Giuseppe

[taijusoup64]

apologies for all the parenthesis, I was just trying to keep things straight in my head. There's definitely a better way to frame the query. 

I tried what you suggested with:

if(id.resp_h="front end",resp_bytes,0)

even simplifying the expression to filter on one ip address at a time gives an error. trying to use it like this 

index="zeek" source="conn.log"
((id.orig_h IN `front end`) AND NOT (id.resp_h IN `backend`)) OR 
((id.resp_h IN `front end`) AND NOT (id.orig_h IN `backend`))
| fields orig_bytes, resp_bytes
| eval terabytes=((if(id.resp_h=192.168.0.1,resp_bytes,0))+(if(id.orig_h=192.168.0.1,orig_bytes,0)))/1024/1024/1024/1024
| stats sum (terabytes)

I just get an error back from splunk. 
Error in EvalCommand: the number 192.168.0.1 is invalid

[gcusello]

Hi @taijusoup64,

use always quotes in the eval condition:

index="zeek" source="conn.log"
((id.orig_h IN `front end`) AND NOT (id.resp_h IN `backend`)) OR 
((id.resp_h IN `front end`) AND NOT (id.orig_h IN `backend`))
| fields orig_bytes, resp_bytes
| eval terabytes=((if(id.resp_h="192.168.0.1",resp_bytes,0))+(if(id.orig_h="192.168.0.1",orig_bytes,0)))/1024/1024/1024/1024
| stats sum (terabytes)

Ciao.

Giuseppe