Splunk Search

how to compare avg server response time with the one from the current day?

zooky92
New Member

Hello,

here is what I try to do: I want to know if my todays average duration is slower or faster than my average duration from the last 7 days. If it is higher I want to receive a notification. Assuming that right now we have 2pm I want todays average from 00am until 2pm compared to the average of the same timespan but for the last 7 days. I came up with a solution in 2 separate graphs which looks like this:

1: todays average

index=myindex host=myhost
source="*access_log" duration!="" NOT "status" (date_hour > 00 AND date_hour < now)|eval apacheDuration=apacheDuration/1000 |stats avg(duration) by host

2: avarage of the last 7 days for the same timespan

index=myindex* host="my host
source="*access_log" duration!="" NOT "status" earliest=-7d latest=now (date_hour > 00 AND date_hour < now) | stats avg(duration) by host |eval apacheDuration=apacheDuration/1000

Is there a way to combine those to in 1 graph and make it able to send me notifications if the average time of today is higher than the average? It would be even better if I could plot the deviation.

I thought about something like this which obviously doesn't work:

  1. dataset:

    index=myindex host=myhost
    source="*access_log" duration!="" NOT "status" (date_hour > 00 AND date_hour < now)|eval apacheDuration=apacheDuration/1000 |stats avg(duration) by host as avaragethoday

  2. dataset

    index=myindex* host="my host
    source="*access_log" duration!="" NOT "status" earliest=-7d latest=now (date_hour > 00 AND date_hour < now) | stats avg(duration) by host as avarageWeek |eval apacheDuration=apacheDuration/1000

eval spike=if(avarageToday >1.2 * avarageWeek, avarageToday, 0) + show em the difference between avarageToday and avarageWeek

Tags (2)
0 Karma

soumyasaha25
Contributor

Hi @zooky92 try something like the below query.

index=myindex host=myhost source="*access_log" duration!="" NOT "status" earliest=-24h latest=-1h (date_hour > 00 AND date_hour < 14)
|eval apacheDuration_today=apacheDuration/1000 
|stats avg(apacheDuration_today) as avg_apacheDuration_today by host
| appendcols
    [search index=myindex host=myhost source="*access_log" duration!="" NOT "status" earliest=-8d latest=-1d (date_hour > 00 AND date_hour < 14) 
    |eval apacheDuration_week=apacheDuration/1000
    | stats avg(apacheDuration_week) avg_apacheDuration_week by host]
| eval is_spike=if(avg_apacheDuration_today >1.2 * avg_apacheDuration_week, avarageToday, 0)
| stats values(spike_value) as spike_value values(apacheDuration_today) as apacheDuration_today values(avg_apacheDuration_week) as avg_apacheDuration_week values(is_spike) by _time

you can try to run it as a scheduled search at 15:00.

0 Karma

zooky92
New Member

anyone? :S

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Take Action Automatically on Splunk Alerts with Red Hat Ansible Automation Platform

 Are you ready to revolutionize your IT operations? As digital transformation accelerates, the demand for ...

Calling All Security Pros: Ready to Race Through Boston?

Hey Splunkers, .conf25 is heading to Boston and we’re kicking things off with something bold, competitive, and ...

Beyond Detection: How Splunk and Cisco Integrated Security Platforms Transform ...

Financial services organizations face an impossible equation: maintain 99.9% uptime for mission-critical ...