topic Re: Why does a real-time search with a small time range not return any results in Splunk 6.1.3? in Splunk Search

Why does a real-time search with a small time range not return any results in Splunk 6.1.3?

nk-1 — Thu, 16 Apr 2015 20:29:38 GMT

Sample Splunk Web search in Splunk 6.1.3 (Windows Server 2012):

host=MyHost level=INFO | stats count

always returns zero if I use Real Time 1-minute window.
If I change to Real Time 5-minute window, I get numbers that change every couple of seconds.

Why won't the 1-minute real-time window return results?

Re: Why does a real-time search with a small time range not return any results in Splunk 6.1.3?

stephane_cyrill — Thu, 16 Apr 2015 20:47:53 GMT

Hi, When you simply do a ....|stats count ,splunk is doing statistics over all fields and that may take time so 1 minute window may be not be sufficient for that.

Re: Why does a real-time search with a small time range not return any results in Splunk 6.1.3?

nk-1 — Fri, 17 Apr 2015 18:23:33 GMT

Yes, this seems to make sense now.
I had radial gauges in my real-time dashboards that showed the count of incoming events in a 1-minute window.
It stopped working (always reporting zero) after I turned on DEBUG logging level on some application servers which increased incoming events from 1.5GB/day to about 36GB/day.

I might have to look at clustering Splunk to process things faster if I want the 1-min real-time reporting?

Re: Why does a real-time search with a small time range not return any results in Splunk 6.1.3?

stephane_cyrill — Fri, 17 Apr 2015 23:46:43 GMT

Hi nk-1, feel free to vote and accept the answer. thanks

Re: Why does a real-time search with a small time range not return any results in Splunk 6.1.3?

nk-1 — Tue, 21 Apr 2015 15:25:18 GMT

I'd just like to add a note that a reason why my 1-minute real-time window was not producing results when I went from indexing 1.5GB/day to 36GB/day was because the forwarders sending events to my indexers were, by default, configured to throttle after 256KB/second.
I changed maxKBps in limits.conf to zero in the forwarders, and the 1-minute real-time window displays updating counts now, without the need for clustering.