topic Re: How do I better understand the noise/static? in Splunk Search

How do I better understand the noise/static?

talbot7 — Mon, 04 May 2015 16:52:36 GMT

Having fun with temperature sensors inside of bee hives. In the attached picture, BaitHive2, yellow, has more noise/static then the rest of the hives. I have replaced the sensor and confirmed that the problem is the bee's, not hardware. 🙂

I am not even sure how to ask this one. How would I extract the noise/static as a value? Something like signal to noise ratio, or standard deviation?

Here is my current search code:

index=beemonitor Temp AND HWID=* HWID=FE:FE:C5:0E:84:DC:AD:48 
| regex _raw="HWID=\w\w:\w\w:\w\w:\w\w:\w\w:\w\w:\w\w:\w\w\sUUID=\w\w:\w\w:\w\w:\w\w:\w\w:\w\w:\w\w:\w\w\sTemp=\w*.\w*"  
| where Temp &gt; -55 | eval TempF = ((Temp*9)/5)+32 
| timechart span=30m limit=32 eval(round(avg(TempF),2)) by UUID 
| rename 28:E3:32:E8:03:00:00:D8 as BaitHive1 
| rename 28:4B:5B:E8:03:00:00:C7 as HexAHive 
| rename 28:30:4F:E8:03:00:00:C4 as BaitHive2 
| rename 28:4D:43:E8:03:00:00:37 as BaitHive3 
| rename 28:B4:4C:E8:03:00:00:BC as BaitHive4 
| rename 28:43:97:CD:02:00:00:3F as Outside 
| fields - NULL

Re: How do I better understand the noise/static?

talbot7 — Wed, 06 May 2015 15:59:19 GMT

Any one???

Re: How do I better understand the noise/static?

ErikaE — Mon, 06 Jul 2015 15:23:36 GMT

Found this while searching around for sensor data examples.

Looks like you could calculate a signal to noise ratio over time and compare that for your various hives. I'm new to splunk and not sure how you'd implement it, but here's a quick mathematical primer on noise:

http://terpconnect.umd.edu/~toh/spectrum/SignalsAndNoise.html#Measuring

Re: How do I better understand the noise/static?

talbot7 — Mon, 06 Jul 2015 16:35:17 GMT

Thanks for the link. I understand the math, I just dont know how to make the Splunk statement. 😞

Re: How do I better understand the noise/static?

aljohnson_splun — Mon, 06 Jul 2015 17:15:42 GMT

@talbot7 there are lots of ways to do this !

Do you want a SNR ratio at a point in time ? Overall ? Just for the BaitHive2 ?

Re: How do I better understand the noise/static?

talbot7 — Mon, 06 Jul 2015 17:17:20 GMT

Overall at a per hive (UUID) level

Re: How do I better understand the noise/static?

aljohnson_splun — Mon, 06 Jul 2015 17:52:21 GMT

As per your response int he comments, you're looking for an overal signal-to-noise ratio across the time you searched.

If you define the signal-to-noise ratio as mean / standard deviation, then you can just pack it all into that eval function in the timechart command.

| timechart eval(round(avg(TempF), 2)) as hive, eval(round(avg(TempF), 2)/stdev(TempF)) as snr by UUID

However, this will mess up the rename thing you got going on. I'd suggest using eval case to deal with the renames earlier.

| eval UUID = case(
UUID=="28:E3:32:E8:03:00:00:D8", "BaitHive1", 
UUID=="28:4B:5B:E8:03:00:00:C7", "HexAHive" ,
UUID=="foo", "bar")

Where you'll need to still add the other values from your renames that you were using earlier.

After that, you can then use the timechart I put up above.

 index=beemonitor Temp AND HWID=* HWID=FE:FE:C5:0E:84:DC:AD:48 
 | regex _raw="HWID=\w\w:\w\w:\w\w:\w\w:\w\w:\w\w:\w\w:\w\w\sUUID=\w\w:\w\w:\w\w:\w\w:\w\w:\w\w:\w\w:\w\w\sTemp=\w*.\w*"  
 | where Temp > -55 | eval TempF = ((Temp*9)/5)+32 
 | eval UUID = case(
 UUID=="28:E3:32:E8:03:00:00:D8", "BaitHive1", 
 UUID=="28:4B:5B:E8:03:00:00:C7", "HexAHive",
 UUID=="foo", "bar")
 | timechart eval(round(avg(TempF), 2)) as hive, eval(round(avg(TempF), 2)/stdev(TempF)) as snr by UUID
 | fields - NULL

Re: How do I better understand the noise/static?

aljohnson_splun — Mon, 06 Jul 2015 17:52:48 GMT

Alrighty. I submitted an answer but it has to get approved because I put a link to wikipedia in it haha 😠 but the gist is

... timechart eval(avg(round(TempF, 2)) as hive, eval(avg(round(TempF,2)/stdev(TempF))) as snr by UUID

plus a little extra for the renames.