Deployment Architecture

Why is bin command creating too many bins? Issue with 5-minute segments

FeatureCreeep
Path Finder

I have the same problem that is unanswered here

I'm trying to do stats on the last 10 minutes of data by two separate 5 minute buckets.

My command looks like this:

host=servers* index=iis | bin _time bins=2 | stats count as Request by _time, host

When I run this for the last 10 minutes I expect 2 separate 5 minute buckets. If my query starts on an even 5 minute interval like 1:05-1:15, it works fine but the buckets expect even 5 minute increments and if I run 1:02-1:12, it will create a "1:00" bucket for the requests between 1:00 and 1:05, a "1:05" bucket for the 1:05-1:10 requests, and a "1:10" bucket for the 1:10-1:15 requests. That is 3 buckets of different sizes.

Using the span=5 minutes doesn't help either.

I tried just using the epoch number like:

eval Time=_time | bin Time bins=2 

I was trying to stop Splunk from treating the _time field specially but it put everything into a single bin.

How can I get Splunk to just create 2 evenly sized/spanned buckets?

0 Karma
1 Solution

somesoni2
SplunkTrust
SplunkTrust

Splunk's bucketing, on time, is done considering starting point as 01/01/1970 00:00, hence the bucketing of 5 mins always takes minutes with multiplicating factor of 5 (0,5,10,15...).

If you just want to two buckets with last 5 mins to now and last 10 mins to last 5 mins, you need to use other workarounds, like this

host=server* index=iis | addinfo| eval _time=if(_time<relative_time(info_min_time,"-5m@m"),relative_time(info_max_time,"@m"),relative_time(info_min_time,"-5m@m"))| stats count by _time, host

View solution in original post

somesoni2
SplunkTrust
SplunkTrust

Splunk's bucketing, on time, is done considering starting point as 01/01/1970 00:00, hence the bucketing of 5 mins always takes minutes with multiplicating factor of 5 (0,5,10,15...).

If you just want to two buckets with last 5 mins to now and last 10 mins to last 5 mins, you need to use other workarounds, like this

host=server* index=iis | addinfo| eval _time=if(_time<relative_time(info_min_time,"-5m@m"),relative_time(info_max_time,"@m"),relative_time(info_min_time,"-5m@m"))| stats count by _time, host

FeatureCreeep
Path Finder

Perfect! This information should be in the "bin" documentation. Your workaround strategy worked great though a few min and maxes were backwards that I corrected. Thanks!

Here is the updated workaround:

host=server* index=iis | addinfo| eval _time=if(_time<relative_time(info_max_time,"-5m@m"),relative_time(info_min_time,"@m"),relative_time(info_max_time,"-5m@m"))  | stats count by _time, host
0 Karma

jgbricker
Contributor

can you clarify what isn't working about |bin span=5min _time You may also find |convert ctime(_time) as time useful because sometimes it comes out as epoch format. convert has a 'timeformat=' property also to help output in preferred syntax

btw.. bins doesn't force the number of bins it sets a limit

binsSyntax: bins=Description: Sets the maximum number of bins to discretize into.
https://docs.splunk.com/Documentation/SplunkCloud/6.6.3/SearchReference/Bin

0 Karma

FeatureCreeep
Path Finder

It won't create just 2 bins. Notice that the query goes back 10 minutes and the span is 5 minutes, but I still get 3 buckets. It's the same if I use "bins=2"

This query: host=server* index=iis | bin span=5min _time | stats count by _time, host

Run for this time: (11/3/17 1:37:17.000 PM to 11/3/17 1:47:17.000 PM)

Returns these results:

_time host count
2017-11-03 13:35:00 server01 7339
2017-11-03 13:40:00 server01 12910
2017-11-03 13:45:00 server01 6432
2017-11-03 13:35:00 server02 7402
2017-11-03 13:40:00 server02 14509
2017-11-03 13:45:00 server02 6167
2017-11-03 13:35:00 server03 7034
2017-11-03 13:40:00 server03 13665
2017-11-03 13:45:00 server03 6273

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...