topic Re: how to delay results by a few minutes in Splunk Search

how to delay results by a few minutes

dang — Wed, 16 May 2012 21:53:02 GMT

I'm writing a search that is comparing the count of an event versus what happned one and two weeks ago. My search looks like this:

    index="monitoring" Metric=attemptSuccess earliest=@d 
  | timechart span="15m" sum(attemptSuccess) as today 
  | appendcols [ search index="monitoring" Metric=attemptSuccess earliest=-7d@d latest=-6d@d | timechart span="15m" sum(attemptSuccess) as lastweek | fields lastweek ]
  | appendcols [ search index="monitoring" Metric=attemptSuccess earliest=-14d@d latest=-13d@d | timechart span="15m" sum(attemptSuccess) as twoweeksago | fields twoweeksago ]
  | timechart span="15m" sum(today) as Today, sum(lastweek) as LastWeek, sum(twoweeksago) as TwoWeeksAgo

I'm having problems with the most current data point for "Today" - as I'm collecting the data for the last 15 minutes, the most recent results when visualized in a chart give an incorrect appearance of being low, when they're really a work in progress. I know I'd want to add a "latest" clause in the search somehow, but I can't seem to get it working correctly. Any suggestions how to best do this? I've got data being collected every 5 minutes, so ideally this would offset my reading somewhere between 5 and 15 minutes.

Re: how to delay results by a few minutes

lguinn2 — Thu, 17 May 2012 01:12:27 GMT

Does it work if you change the first line to

index="monitoring" Metric=attemptSuccess earliest=@d latest=-15m@m

Re: how to delay results by a few minutes

dwaddle — Thu, 17 May 2012 01:28:23 GMT

It's tough using the relative time operations for earliest and latest to round to something larger than a minute, but smaller than an hour. If I understand your request correctly, you'd like to be able to do something like latest=h/15m@m or latest=h/5m@m to say "Give me the current hour, rounded to the most recent 5/15 minute interval in said hour, rounded to the minute."

I may have entirely misunderstood what you were looking for. But, if I didn't here is an UGLY way of getting there using a subsearch:

[ search earliest=@h latest=@h+1m 
   | head 1 | addinfo 
   | eval td=(floor((now()-info_min_time)/900)*900)+info_min_time 
   | rename td as latest | fields latest 
]

Put that in your original search and it will, using the subsearch, evaluate out latest to the time_t (seconds since 1/1/1970 00:00:00 GMT) of the end of the most recent 15-minute interval in the current hour.

I'm not sure whether to be proud of this or not 🙂