Splunk Search

how to delay results by a few minutes

dang
Path Finder

I'm writing a search that is comparing the count of an event versus what happned one and two weeks ago. My search looks like this:

    index="monitoring" Metric=attemptSuccess earliest=@d 
  | timechart span="15m" sum(attemptSuccess) as today 
  | appendcols [ search index="monitoring" Metric=attemptSuccess earliest=-7d@d latest=-6d@d | timechart span="15m" sum(attemptSuccess) as lastweek | fields lastweek ]
  | appendcols [ search index="monitoring" Metric=attemptSuccess earliest=-14d@d latest=-13d@d | timechart span="15m" sum(attemptSuccess) as twoweeksago | fields twoweeksago ]
  | timechart span="15m" sum(today) as Today, sum(lastweek) as LastWeek, sum(twoweeksago) as TwoWeeksAgo

I'm having problems with the most current data point for "Today" - as I'm collecting the data for the last 15 minutes, the most recent results when visualized in a chart give an incorrect appearance of being low, when they're really a work in progress. I know I'd want to add a "latest" clause in the search somehow, but I can't seem to get it working correctly. Any suggestions how to best do this? I've got data being collected every 5 minutes, so ideally this would offset my reading somewhere between 5 and 15 minutes.

Tags (2)

dwaddle
SplunkTrust
SplunkTrust

It's tough using the relative time operations for earliest and latest to round to something larger than a minute, but smaller than an hour. If I understand your request correctly, you'd like to be able to do something like latest=h/15m@m or latest=h/5m@m to say "Give me the current hour, rounded to the most recent 5/15 minute interval in said hour, rounded to the minute."

I may have entirely misunderstood what you were looking for. But, if I didn't here is an UGLY way of getting there using a subsearch:

[ search earliest=@h latest=@h+1m 
   | head 1 | addinfo 
   | eval td=(floor((now()-info_min_time)/900)*900)+info_min_time 
   | rename td as latest | fields latest 
] 

Put that in your original search and it will, using the subsearch, evaluate out latest to the time_t (seconds since 1/1/1970 00:00:00 GMT) of the end of the most recent 15-minute interval in the current hour.

I'm not sure whether to be proud of this or not 🙂

lguinn2
Legend

Does it work if you change the first line to

index="monitoring" Metric=attemptSuccess earliest=@d latest=-15m@m
Get Updates on the Splunk Community!

Introducing Splunk Enterprise 9.2

WATCH HERE! Watch this Tech Talk to learn about the latest features and enhancements shipped in the new Splunk ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...