topic Re: Calculating data throughput for cloud in Splunk Search

Calculating data throughput for cloud

taijusoup64 — Thu, 28 Mar 2024 20:16:15 GMT

I'm trying to calculate the data throughput for a cloud computing solution that will be charging based on outgoing data throughput.

We're collecting on the link using security onion and forwarding those zeek logs to our splunk instance.

index="zeek" source="conn.log" ((id.orig_h IN `front end`) AND NOT (id.resp_h IN `backend`)) OR ((id.resp_h IN `front end`) AND NOT (id.orig_h IN `backend`)) | fields orig_bytes, resp_bytes | eval terabytes=(((((resp_bytes+orig_bytes)/1024)/1024)/1024)/1024) | stats sum (terabytes)

This gives me traffic throughput in and out of the network for external connections however what I need is to calculate orig_bytes only when the id.orig_h is my `frontend` and resp_bytes when id.resp_h is `frontend`.
I can get them separately by just doing two different searches and then adding the results up by hand. But I'm sure theres a way to do what I want to in one search using some sort of conditional. I've tried using where and eval if but I'm just not skilled enough it seems.

Re: Calculating data throughput for cloud

gcusello — Fri, 29 Mar 2024 06:51:08 GMT

Hi @taijusoup64 ,

let me understand: you want to calculate bytes only when: id.orig_h="frontend" AND id.resp_h="frontend", is this correct?

in this case add the condition to the eval statement:

index="zeek" source="conn.log" ((id.orig_h IN `front end`) AND NOT (id.resp_h IN `backend`)) OR ((id.resp_h IN `front end`) AND NOT (id.orig_h IN `backend`)) | fields orig_bytes, resp_bytes | eval terabytes=((if(id.resp_h="front end",resp_bytes,0))+(if(id.orig_h="front end",orig_bytes,0)))/1024/1024/1024/1024 | stats sum (terabytes)

Ciao.

Giuseppe

why did you used all that parenthesis?

Ciao.

Giuseppe

Re: Calculating data throughput for cloud

taijusoup64 — Fri, 29 Mar 2024 13:34:12 GMT

apologies for all the parenthesis, I was just trying to keep things straight in my head. There's definitely a better way to frame the query.

I tried what you suggested with:

if(id.resp_h="front end",resp_bytes,0)

even simplifying the expression to filter on one ip address at a time gives an error. trying to use it like this

index="zeek" source="conn.log" ((id.orig_h IN `front end`) AND NOT (id.resp_h IN `backend`)) OR ((id.resp_h IN `front end`) AND NOT (id.orig_h IN `backend`)) | fields orig_bytes, resp_bytes | eval terabytes=((if(id.resp_h=192.168.0.1,resp_bytes,0))+(if(id.orig_h=192.168.0.1,orig_bytes,0)))/1024/1024/1024/1024 | stats sum (terabytes)

I just get an error back from splunk.
Error in EvalCommand: the number 192.168.0.1 is invalid

Re: Calculating data throughput for cloud

gcusello — Sat, 30 Mar 2024 07:27:25 GMT

Hi @taijusoup64,

use always quotes in the eval condition:

index="zeek" source="conn.log" ((id.orig_h IN `front end`) AND NOT (id.resp_h IN `backend`)) OR ((id.resp_h IN `front end`) AND NOT (id.orig_h IN `backend`)) | fields orig_bytes, resp_bytes | eval terabytes=((if(id.resp_h="192.168.0.1",resp_bytes,0))+(if(id.orig_h="192.168.0.1",orig_bytes,0)))/1024/1024/1024/1024 | stats sum (terabytes)

Ciao.

Giuseppe