Splunk Search

how to compare avg server response time with the one from the current day?

zooky92
New Member

Hello,

here is what I try to do: I want to know if my todays average duration is slower or faster than my average duration from the last 7 days. If it is higher I want to receive a notification. Assuming that right now we have 2pm I want todays average from 00am until 2pm compared to the average of the same timespan but for the last 7 days. I came up with a solution in 2 separate graphs which looks like this:

1: todays average

index=myindex host=myhost
source="*access_log" duration!="" NOT "status" (date_hour > 00 AND date_hour < now)|eval apacheDuration=apacheDuration/1000 |stats avg(duration) by host

2: avarage of the last 7 days for the same timespan

index=myindex* host="my host
source="*access_log" duration!="" NOT "status" earliest=-7d latest=now (date_hour > 00 AND date_hour < now) | stats avg(duration) by host |eval apacheDuration=apacheDuration/1000

Is there a way to combine those to in 1 graph and make it able to send me notifications if the average time of today is higher than the average? It would be even better if I could plot the deviation.

I thought about something like this which obviously doesn't work:

  1. dataset:

    index=myindex host=myhost
    source="*access_log" duration!="" NOT "status" (date_hour > 00 AND date_hour < now)|eval apacheDuration=apacheDuration/1000 |stats avg(duration) by host as avaragethoday

  2. dataset

    index=myindex* host="my host
    source="*access_log" duration!="" NOT "status" earliest=-7d latest=now (date_hour > 00 AND date_hour < now) | stats avg(duration) by host as avarageWeek |eval apacheDuration=apacheDuration/1000

eval spike=if(avarageToday >1.2 * avarageWeek, avarageToday, 0) + show em the difference between avarageToday and avarageWeek

Tags (2)
0 Karma

soumyasaha25
Contributor

Hi @zooky92 try something like the below query.

index=myindex host=myhost source="*access_log" duration!="" NOT "status" earliest=-24h latest=-1h (date_hour > 00 AND date_hour < 14)
|eval apacheDuration_today=apacheDuration/1000 
|stats avg(apacheDuration_today) as avg_apacheDuration_today by host
| appendcols
    [search index=myindex host=myhost source="*access_log" duration!="" NOT "status" earliest=-8d latest=-1d (date_hour > 00 AND date_hour < 14) 
    |eval apacheDuration_week=apacheDuration/1000
    | stats avg(apacheDuration_week) avg_apacheDuration_week by host]
| eval is_spike=if(avg_apacheDuration_today >1.2 * avg_apacheDuration_week, avarageToday, 0)
| stats values(spike_value) as spike_value values(apacheDuration_today) as apacheDuration_today values(avg_apacheDuration_week) as avg_apacheDuration_week values(is_spike) by _time

you can try to run it as a scheduled search at 15:00.

0 Karma

zooky92
New Member

anyone? :S

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.