Splunk Search

Why does a real-time search with a small time range not return any results in Splunk 6.1.3?

nk-1
Path Finder

Sample Splunk Web search in Splunk 6.1.3 (Windows Server 2012):

host=MyHost level=INFO | stats count

always returns zero if I use Real Time 1-minute window.
If I change to Real Time 5-minute window, I get numbers that change every couple of seconds.

Why won't the 1-minute real-time window return results?

0 Karma
1 Solution

stephane_cyrill
Builder

Hi, When you simply do a ....|stats count ,splunk is doing statistics over all fields and that may take time so 1 minute window may be not be sufficient for that.

View solution in original post

nk-1
Path Finder

I'd just like to add a note that a reason why my 1-minute real-time window was not producing results when I went from indexing 1.5GB/day to 36GB/day was because the forwarders sending events to my indexers were, by default, configured to throttle after 256KB/second.
I changed maxKBps in limits.conf to zero in the forwarders, and the 1-minute real-time window displays updating counts now, without the need for clustering.

0 Karma

stephane_cyrill
Builder

Hi, When you simply do a ....|stats count ,splunk is doing statistics over all fields and that may take time so 1 minute window may be not be sufficient for that.

stephane_cyrill
Builder

Hi nk-1, feel free to vote and accept the answer. thanks

0 Karma

nk-1
Path Finder

Yes, this seems to make sense now.
I had radial gauges in my real-time dashboards that showed the count of incoming events in a 1-minute window.
It stopped working (always reporting zero) after I turned on DEBUG logging level on some application servers which increased incoming events from 1.5GB/day to about 36GB/day.

I might have to look at clustering Splunk to process things faster if I want the 1-min real-time reporting?

0 Karma
Get Updates on the Splunk Community!

Database Performance Sidebar Panel Now on APM Database Query Performance & Service ...

We’ve streamlined the troubleshooting experience for database-related service issues by adding a database ...

IM Landing Page Filter - Now Available

We’ve added the capability for you to filter across the summary details on the main Infrastructure Monitoring ...

Dynamic Links from Alerts to IM Navigators - New in Observability Cloud

Splunk continues to improve the troubleshooting experience in Observability Cloud with this latest enhancement ...