Splunk Search

Why does a real-time search with a small time range not return any results in Splunk 6.1.3?

nk-1
Path Finder

Sample Splunk Web search in Splunk 6.1.3 (Windows Server 2012):

host=MyHost level=INFO | stats count

always returns zero if I use Real Time 1-minute window.
If I change to Real Time 5-minute window, I get numbers that change every couple of seconds.

Why won't the 1-minute real-time window return results?

0 Karma
1 Solution

stephane_cyrill
Builder

Hi, When you simply do a ....|stats count ,splunk is doing statistics over all fields and that may take time so 1 minute window may be not be sufficient for that.

View solution in original post

nk-1
Path Finder

I'd just like to add a note that a reason why my 1-minute real-time window was not producing results when I went from indexing 1.5GB/day to 36GB/day was because the forwarders sending events to my indexers were, by default, configured to throttle after 256KB/second.
I changed maxKBps in limits.conf to zero in the forwarders, and the 1-minute real-time window displays updating counts now, without the need for clustering.

0 Karma

stephane_cyrill
Builder

Hi, When you simply do a ....|stats count ,splunk is doing statistics over all fields and that may take time so 1 minute window may be not be sufficient for that.

stephane_cyrill
Builder

Hi nk-1, feel free to vote and accept the answer. thanks

0 Karma

nk-1
Path Finder

Yes, this seems to make sense now.
I had radial gauges in my real-time dashboards that showed the count of incoming events in a 1-minute window.
It stopped working (always reporting zero) after I turned on DEBUG logging level on some application servers which increased incoming events from 1.5GB/day to about 36GB/day.

I might have to look at clustering Splunk to process things faster if I want the 1-min real-time reporting?

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...