Splunk Search

piping events from a custom generating command into timechart results in irregular buckets when not using all-time

ruman
Splunk Employee
Splunk Employee

I have 26 days of events (Monday 9/15 through Friday 10/10) piped to a timechart span=7d.

I'd like to have 3 buckets of 7 days each, and one bucket with the 5 remainder days. It doesn't matter to me if the remainder bucket is at the beginning or end but it would be nice if Splunk was consistent about it.

Instead, my buckets begin at 9/15, 9/22, 9/29 (Mondays), 10/3, 10/10 (Fridays) which means I have 5 buckets, instead of 4,
with 7, 7, 4, 7, and 1 day in them.

How can I make timechart behave reasonably?

I'm running Splunk 6.1.2

Tags (3)
1 Solution

acharlieh
Influencer

So the crux of this issue here is that every search in Splunk has a timerange with it. Also timechart uses the earliest time of the search to figure out what times fit into what bucket. When you generate results (with an external command or creative uses of other commands) that have results with _time fields outside of the range of the search, oddity occurs. As jrodman says in the comments below, often times you should set the search timeframe to all time, or at least manipulate earliest.

Alternatively you can use eval with the relative_time function to manipulate _time into desired buckets, and then use chart.


Original Comment:

Ah! So the issue there is that every search in Splunk has an earliest and latest time. When you generate events from an external command that fall outside the time range of your search, it throws off the bucketing done by the bin / timechart command. My gist is generating 40 days of events so I needed to make sure my search window was 40 days otherwise you wind up with funniness at the search boundary. But the second form using eval and relative_time should work better

View solution in original post

acharlieh
Influencer

So the crux of this issue here is that every search in Splunk has a timerange with it. Also timechart uses the earliest time of the search to figure out what times fit into what bucket. When you generate results (with an external command or creative uses of other commands) that have results with _time fields outside of the range of the search, oddity occurs. As jrodman says in the comments below, often times you should set the search timeframe to all time, or at least manipulate earliest.

Alternatively you can use eval with the relative_time function to manipulate _time into desired buckets, and then use chart.


Original Comment:

Ah! So the issue there is that every search in Splunk has an earliest and latest time. When you generate events from an external command that fall outside the time range of your search, it throws off the bucketing done by the bin / timechart command. My gist is generating 40 days of events so I needed to make sure my search window was 40 days otherwise you wind up with funniness at the search boundary. But the second form using eval and relative_time should work better

jrodman
Splunk Employee
Splunk Employee

I moved this to an answer because it was the crucial insight. If you're willing to edit it a bit to be clearer for future readers feel free, or I'm happy to do so.

0 Karma

ruman
Splunk Employee
Splunk Employee

oh, that's really interesting. so maybe i need to match the time picker to range of the events that hit timechart in order to make the buckets happy? i've got lots to experiment with now, thank you!

0 Karma

ruman
Splunk Employee
Splunk Employee

yup, ding. setting the time picker to all time makes very normal buckets. thanks everyone.

0 Karma

ruman
Splunk Employee
Splunk Employee

and even better, setting the time picker to the exact day the events start, makes the buckets start on that day, e.g. i can make sure the buckets start on mondays by having the timepicker set earliest to -3w@w1.

0 Karma

jrodman
Splunk Employee
Splunk Employee

I assumed your search was all-time. Is it?

0 Karma

ruman
Splunk Employee
Splunk Employee

no, the time picker was definitely not set to 'all time', and the external command i'm running completely ignores the time picke.r

0 Karma

jrodman
Splunk Employee
Splunk Employee

It should probably be set to all time if the input is going to ignore it.

0 Karma

acharlieh
Influencer

Next year at conf, I need to not give away so much karma that I can no longer comment. But what are you using for your earliest search time? I've noticed that timechart and bucket/bin use this for the base of the buckets: https://gist.github.com/acharlieh/777d6767378e82427d00

0 Karma

acharlieh
Influencer

Indeed! Or instead of manipulating the search range, use eval to manipulate the _time field into the correct buckets and then chart (instead of timechart) 🙂 have fun!

ruman
Splunk Employee
Splunk Employee

in this case the events are coming from an external command, then i'm removing older events with a where clause. i don't know what that means "earliest" is. And I don't quite get what's going on with your gist, but I'll experiment with it and write back...if span=w, then why does earliest need be -40d or earlier?

| foo bar bas
| eval earliest_ts=relative_time(now(), "-3w@w0")
| where _time > earliest_ts
| timechart span=1w count by bat
0 Karma

jrodman
Splunk Employee
Splunk Employee

I'd suggest editing this to describe your specific goal. Documenting the fail seems more like a thing for jira.

0 Karma

ruman
Splunk Employee
Splunk Employee

okay, i tried!

0 Karma
Get Updates on the Splunk Community!

Happy CX Day to our Community Superheroes!

Happy 10th Birthday CX Day!What is CX Day? It’s a global celebration recognizing innovation and success in the ...

Check out This Month’s Brand new Splunk Lantern Articles

Splunk Lantern is a customer success center providing advice from Splunk experts on valuable data insights, ...

Routing Data to Different Splunk Indexes in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...