Splunk Search

Why is the timechart avg function showing the value from last event instead of average for the selected time range from the time picker?

sidekix24
Path Finder

Hello all,

I have another issue with timechart, stats, and timepicker. I have the search below that needs to pull up a value of the average "response time" over the time range picked in the timepicker input, as well as show trending and with color thresholds. I have the trending and thresholds all set, but the issue I'm having is that the value appearing is just the "response time" avg in the last event instead of changing when I change the range in the timepicker.

| stats count by _time,source,Login_Status,Login_Response,QuickSearch_Status,QuickSearch_Response,Recruiter_Status,Recruiter_Response | eval Response_Time=(Login_Response+QuickSearch_Response+Recruiter_Response)  | table _time,Response_Time | chart avg(Response_Time) by _time

Anyone have any ideas on how to tweak the search to show the value of the avg of the range selected in the timepicker?

0 Karma
1 Solution

masonmorales
Influencer

It's hard to tell without sample data, but why wouldn't you just do this?

...base search... | eval Response_Time=(Login_Response+QuickSearch_Response+Recruiter_Response)  | timechart avg(Response_Time) 

I don't think there is really a need to do a stats and table before your timechart in this case. You can also play with the minspan and maxspan options. See: http://docs.splunk.com/Documentation/Splunk/6.3.3/SearchReference/Timechart

Edit: Try searching a longer time range as well, or see if the events from your script are being timestamped correctly in Splunk.

View solution in original post

0 Karma

sidekix24
Path Finder

alt text

0 Karma

sidekix24
Path Finder

alt text

0 Karma

cramasta
Builder

That confirms you are only getting one event per 15 minutes. Your last Response_Time value will always be the same as the average if your timechart span=15m.

0 Karma

sidekix24
Path Finder

any suggestions on how to change that? how to get the search to display the true avg?

0 Karma

sidekix24
Path Finder

only option is to probably run the script more frequently, right?

0 Karma

cramasta
Builder

Correct, or you change the span to say span=1h so that you will have up to 4 values to factor into the average each hour.

About timechart - If you dont set a span= timechart will automatically pick the best span for the total amount of time you are searching. So if you search the past 15 minutes it will set the span to 10 seconds. Searching past 24 hours will set the span to 30 minutes. If you want a fixed span you must specify it in the query.

0 Karma

sidekix24
Path Finder

alt text

0 Karma

sidekix24
Path Finder

yup...in the UI the single value that it returns every single time is the most recent response time. So it matches the value of the last time the script ran no matter what the time picker is set to

0 Karma

sidekix24
Path Finder

I ran

| eval Response_Time=(Login_Response+QuickSearch_Response+Recruiter_Response) | timechart span=15m avg(Response_Time) AS avg

and

| eval Response_Time=(Login_Response+QuickSearch_Response+Recruiter_Response) | timechart span=15m last(Response_Time) AS avg

and I get the same single value for both searches when I match the timepicker range

0 Karma

cramasta
Builder

If you are only generating one event every 15 minutes then that would make sense. There would be no average to calculate as theres only one Response_Time Value in each 15 minute period. The last 15 minute Response_Time "value" would be the same as the 15 minute "average"

Run this over the past hour and provide the results.
| timechart span=15m count

and also run this over 1 hour to see what values are being used in the average calculation.
| eval Response_Time=(Login_Response+QuickSearch_Response+Recruiter_Response) | timechart span=15m list(Response_Time) AS values_used_in _average

0 Karma

cramasta
Builder

Also upload your screen shot to somewhere like http://imgur.com/ and post the link to share

0 Karma

sidekix24
Path Finder

if we can get it chart over time then everything else will fall into place 🙂

0 Karma

cramasta
Builder

Well that brings us back to my answer from the first post

your base search | eval Response_Time=(Login_Response+QuickSearch_Response+Recruiter_Response) | timechart span=15m avg(Response_Time) AS avg

Run the query in the search UI and look at the results to understand what the single value would be displaying. If the results table shows the most recent timespan value as 0 then its because the script did not export the data yet for that 15 minute interval.

Might also be worth putting partial=false at the end so that only complete 15 minute spans are provided in the results.
your base search | eval Response_Time=(Login_Response+QuickSearch_Response+Recruiter_Response) | timechart span=15m avg(Response_Time) AS avg partial=false

0 Karma

sidekix24
Path Finder

Ok...I'm with you so far. It's working, now the important part of this whole thng. How do you timechart this or do a chart over time?

When I do a | timechart values(avg) or | chart values(avg) over _time...i don't get back any results.

That's what I've been trying to figure out this whole time :). the ability to post screen shots would have been helpful in this whole process 🙂

0 Karma

sidekix24
Path Finder

thanks cramasta

I still don't think this is going to do what I need it to do. Let's forget about the compare. Do you have any suggestions on just a search that gives me a timechart or chart over time of the average total response time based on the entire timeframe selected in the timerange picker?

Thanks again

0 Karma

cramasta
Builder

Thats as simple as

your base search | eval Response_Time=(Login_Response+QuickSearch_Response+Recruiter_Response) | stats avg(Response_Time) AS avg

Dont do this in the dashboard, try it in the search UI. If it works there then try it in the dashboard. If the results dont change when you change the time then its because you did not hook up the search query to the timerange picker.

0 Karma

cramasta
Builder

sorry made a few edits, think i got it right now.

0 Karma

sidekix24
Path Finder

yes, that's correct...the trend indicators are set to compare the same time frame from the timepicker from 7 days ago. The trend indicators, compare to, colors and thresholds are all set up in the single value chart settings in the dashboard panel so we don't really need to worry about it in the search or the chart command in the search. I just need to be able to build a search that charts the avg total response over time (using the time frame in the timepicker)

Sorry about all the confusion 🙂

0 Karma

cramasta
Builder

Heres one possible way. You will hardcode the timeframe in the base search to earliest=-7d to get the past 7 day average. Then in the subsearch (the search in brackets), it will use the timerange selected from the timerange picker.

earliest=-7d your base search | eval Response_Time=(Login_Response+QuickSearch_Response+Recruiter_Response) | stats avg(Response_Time) AS avg | eval _time=NOW()-604800 | append [search your base search | eval Response_Time=(Login_Response+QuickSearch_Response+Recruiter_Response) | stats avg(Response_Time) AS avg | eval _time=NOW()]

Now depending on how long it takes for the 7 day search to run you may want to setup a scheduled search that stores the 7 day result in a lookup file.

the search that would export the lookup file

earliest=-7d your base search | eval Response_Time=(Login_Response+QuickSearch_Response+Recruiter_Response) | stats avg(Response_Time) AS avg | eval _time=NOW()-604800| fields avg _time | outputlookup my7dayavg.csv

The search for the dashboard that uses the lookup

|inputlookup my7dayavg.csv | fields _time avg | append [search your base search | eval Response_Time=(Login_Response+QuickSearch_Response+Recruiter_Response) | stats avg(Response_Time) AS avg | eval _time=NOW()]

0 Karma

sidekix24
Path Finder

not really...

I have the trend compare to the last 7 days and they want to do an average of the chosen time frame in the timepicker input

0 Karma

sidekix24
Path Finder

bascially...the script runs every 15 mins so they want to be able to choose any timeframe in the timepicker and have the value dynamically change to the average total response time of that timeframe as they pick the timerange in the timepicker

0 Karma
Get Updates on the Splunk Community!

Accelerating Observability as Code with the Splunk AI Assistant

We’ve seen in previous posts what Observability as Code (OaC) is and how it’s now essential for managing ...

Integrating Splunk Search API and Quarto to Create Reproducible Investigation ...

 Splunk is More Than Just the Web Console For Digital Forensics and Incident Response (DFIR) practitioners, ...

Congratulations to the 2025-2026 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...