Splunk Search

how to compare avg server response time with the one from the current day?

zooky92
New Member

Hello,

here is what I try to do: I want to know if my todays average duration is slower or faster than my average duration from the last 7 days. If it is higher I want to receive a notification. Assuming that right now we have 2pm I want todays average from 00am until 2pm compared to the average of the same timespan but for the last 7 days. I came up with a solution in 2 separate graphs which looks like this:

1: todays average

index=myindex host=myhost
source="*access_log" duration!="" NOT "status" (date_hour > 00 AND date_hour < now)|eval apacheDuration=apacheDuration/1000 |stats avg(duration) by host

2: avarage of the last 7 days for the same timespan

index=myindex* host="my host
source="*access_log" duration!="" NOT "status" earliest=-7d latest=now (date_hour > 00 AND date_hour < now) | stats avg(duration) by host |eval apacheDuration=apacheDuration/1000

Is there a way to combine those to in 1 graph and make it able to send me notifications if the average time of today is higher than the average? It would be even better if I could plot the deviation.

I thought about something like this which obviously doesn't work:

  1. dataset:

    index=myindex host=myhost
    source="*access_log" duration!="" NOT "status" (date_hour > 00 AND date_hour < now)|eval apacheDuration=apacheDuration/1000 |stats avg(duration) by host as avaragethoday

  2. dataset

    index=myindex* host="my host
    source="*access_log" duration!="" NOT "status" earliest=-7d latest=now (date_hour > 00 AND date_hour < now) | stats avg(duration) by host as avarageWeek |eval apacheDuration=apacheDuration/1000

eval spike=if(avarageToday >1.2 * avarageWeek, avarageToday, 0) + show em the difference between avarageToday and avarageWeek

Tags (2)
0 Karma

soumyasaha25
Contributor

Hi @zooky92 try something like the below query.

index=myindex host=myhost source="*access_log" duration!="" NOT "status" earliest=-24h latest=-1h (date_hour > 00 AND date_hour < 14)
|eval apacheDuration_today=apacheDuration/1000 
|stats avg(apacheDuration_today) as avg_apacheDuration_today by host
| appendcols
    [search index=myindex host=myhost source="*access_log" duration!="" NOT "status" earliest=-8d latest=-1d (date_hour > 00 AND date_hour < 14) 
    |eval apacheDuration_week=apacheDuration/1000
    | stats avg(apacheDuration_week) avg_apacheDuration_week by host]
| eval is_spike=if(avg_apacheDuration_today >1.2 * avg_apacheDuration_week, avarageToday, 0)
| stats values(spike_value) as spike_value values(apacheDuration_today) as apacheDuration_today values(avg_apacheDuration_week) as avg_apacheDuration_week values(is_spike) by _time

you can try to run it as a scheduled search at 15:00.

0 Karma

zooky92
New Member

anyone? :S

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.
Get Updates on the Splunk Community!

Splunk AI Assistant for SPL vs. ChatGPT: Which One is Better?

In the age of AI, every tool promises to make our lives easier. From summarizing content to writing code, ...

Data Persistence in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. What happens if the OpenTelemetry collector ...

Thanks for the Memories! Splunk University, .conf25, and our Community

Thank you to everyone in the Splunk Community who joined us for .conf25, which kicked off with our iconic ...