How to compare today hourly stats and previous week same day hourly stats?

[mikeyty07]

is there a way to alert an email if today's hourly stats are 25% higher than the previous week same day hourly stats?

[yeahnah]

Hi @mikeyty07 

Something like this run anywhere example would meet your requirements...

index=_internal source=*metrics.log sourcetype=splunkd earliest=-1h@h latest=@h  | head 7777
| append [ search index=_internal source=*metrics.log sourcetype=splunkd earliest=-7d-1h@h latest=-7d@h | head 5555 ]
| timechart count
| timewrap w align=end
| bin _time span=1h
| stats sum(*) AS * BY _time
| eval diff_pct=((('latest_week' - '1week_before')/latest_week)*100)
| where diff_pct>=25
| table _time latest* *before diff*

 Save it as an Alert and when it triggers notify with an email.

Hope that helps

[mikeyty07]

I have this working query which needs some additional detailing.

index=_internal earliest=-1h@h latest=@h
| lookup api uri OUTPUT operation service
| rex "duration=(?<response_time>[^,]+)" | multikv | eval ReportKey="Today"
| append
[ search index=_internal earliest=-7d-1h@h latest=-7d@h
| lookup api uri OUTPUT operation service
| rex "duration=(?<response_time>[^,]+)" | multikv | eval ReportKey="lastweek" | eval _time=_time+604800]
| stats first(uri) as apiName avg(response_time) as avgresponse_time count by operationName ReportKey

Is there a way to get like this where it will compare all average response time and then give the percentile differences.

operation	uri	today_avg (response_time)	lastweek_avg (response_time)	% differrences avg(response_time)	today count	last week count
abc	/api/abc	222	333		12312	42343
xyz	/api/xyz	867	4234		87978	67867

[yeahnah]

Hi @mikeyty07 

I think I know what you mean and eventstats is what you can use to avg all response times then make a %diff comparison

 .. ... your search ...
| stats first(uri) as apiName avg(response_time) as avgresponse_time count by operationName ReportKey
| eventstats avg(avgresponse_time) AS avg_resp_time_all
| ... then eval you % diffs against the result of avg_resp_time_all field ...

If this answers your follow up question then please mark this with solution provided

[mikeyty07]

can you help me how to write those eval % differences, i'm not quite sure if i got it correctly.

[yeahnah]

I did provide an example in my original post.  You are simply doing a mathematical percentage diff of the values in two columns.

It's not overly clear what the avg response differences you want to compare against is (today or lastweek?), but something like

... current results ...
| eval today_avg_diff=('today_avg' - 'avgresponse_time')
      ,today_avg_diff_pct=round(('today_avg_diff'/'today_avg')*100, 1)
      ,lastweek_avg_diff=('lastweek_avg' - 'avgresponse_time')
      ,lastweek_avg_diff_pct=round(('lastweek_avg_diff'/'lastweek_avg')*100, 1)
| table ... the fields you want ...

You could get the diff percentage as one eval statement but broke it down for some clarity.   It's untested so hopefully no typos above.

Basically start simply and iterate until you get what you want.

Hope that keeps you going