Thanks for clarifying - then you still use streamstats - see this modified example, most of the first part is getting your data to a form where you have _time, src_ip, dest_ip, gb_in, gb_out for each of the two comparison weeks for the IP pairings. You'll need to run it a few times to get include=1 in the results, as it randomises byte counts. But the include field calculation looks for the 5% variance or whether the pairing is only seen in ONE of the weeks, in which case it also includes it. If you want to change the 5% variance, I suggest rather than editing the search each time, just create a macro with the value in there, so your test would look like if(abs('variance_%_<<MATCHSTR>>')>`get_variance_threshold`... where your macro get_variance_threshold simply has the value you want to change. Hope this gets you closer to your target! | makeresults count=28
| eval bytes_in=random() % 200000000000 + 10000000000, bytes_out=random() % 200000000000 + 10000000000
| streamstats c
| eval src_ip=mvindex(split("1.1.1.1,2.2.2.2", ","), c % 2), dest_ip=mvindex(split("3.3.3.3,4.4.4.4",","), c % 2)
| eval _time=if(c<=14, now(), now() - (86400*7))
| bin _time span=7d
| stats sum(bytes_*) as total_bytes_* by _time src_ip dest_ip
| eval gb_out = round(total_bytes_out/1024/1024/1024,2)
| eval gb_in = round(total_bytes_in/1024/1024/1024,2)
| fields _time, src_ip, dest_ip, gb_in, gb_out
``` Above creates a dataset for comparison - in your case do tstats with a time span of the 1w and a time range of the full period you are comparing ```
``` This sorts the row of data for this week and last week into two rows by _time ```
| sort src_ip dest_ip _time
``` Now streamstats pulls forward the last week value to the current week row ```
| streamstats window=2 global=f first(gb_*) as last_week_gb_* by src_ip dest_ip
``` Look how many times we have seen the pairing so we can detect count=1 ```
| eventstats count as seen_count by src_ip dest_ip
``` Calculate all the variances for in/out between the current and last week ```
| foreach gb_* [ eval diff_<<MATCHSTR>>='<<FIELD>>'-'last_week_<<FIELD>>', ``` Calculates difference in values from last week to this week ```
variance_%_<<MATCHSTR>> = round(diff_<<MATCHSTR>> / '<<FIELD>>' * 100, 2), ``` Calculate the variance % between the two weeks ```
include=if(abs('variance_%_<<MATCHSTR>>')>5 OR seen_count=1, 1, coalesce(include, 0)) ] ``` Looks at the 5% threshold and if it has only been seen one of the weeks ```
| fields - last_week_*
``` Here if include is set then it has exceeded the 5% variance or if it was only seen this week or last week ```
```| where include=1```
... View more
