Splunk Search

Why does a real-time search with a small time range not return any results in Splunk 6.1.3?

nk-1
Path Finder

Sample Splunk Web search in Splunk 6.1.3 (Windows Server 2012):

host=MyHost level=INFO | stats count

always returns zero if I use Real Time 1-minute window.
If I change to Real Time 5-minute window, I get numbers that change every couple of seconds.

Why won't the 1-minute real-time window return results?

0 Karma
1 Solution

stephane_cyrill
Builder

Hi, When you simply do a ....|stats count ,splunk is doing statistics over all fields and that may take time so 1 minute window may be not be sufficient for that.

View solution in original post

nk-1
Path Finder

I'd just like to add a note that a reason why my 1-minute real-time window was not producing results when I went from indexing 1.5GB/day to 36GB/day was because the forwarders sending events to my indexers were, by default, configured to throttle after 256KB/second.
I changed maxKBps in limits.conf to zero in the forwarders, and the 1-minute real-time window displays updating counts now, without the need for clustering.

0 Karma

stephane_cyrill
Builder

Hi, When you simply do a ....|stats count ,splunk is doing statistics over all fields and that may take time so 1 minute window may be not be sufficient for that.

stephane_cyrill
Builder

Hi nk-1, feel free to vote and accept the answer. thanks

0 Karma

nk-1
Path Finder

Yes, this seems to make sense now.
I had radial gauges in my real-time dashboards that showed the count of incoming events in a 1-minute window.
It stopped working (always reporting zero) after I turned on DEBUG logging level on some application servers which increased incoming events from 1.5GB/day to about 36GB/day.

I might have to look at clustering Splunk to process things faster if I want the 1-min real-time reporting?

0 Karma
Get Updates on the Splunk Community!

Monitoring MariaDB and MySQL

In a previous post, we explored monitoring PostgreSQL and general best practices around which metrics to ...

Financial Services Industry Use Cases, ITSI Best Practices, and More New Articles ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Splunk Federated Analytics for Amazon Security Lake

Thursday, November 21, 2024  |  11AM PT / 2PM ET Register Now Join our session to see the technical ...