Splunk Search

Why enabling real time search does not display data immediately?

asingla
Communicator

I am receiving events every 15 seconds. But when I enable real time search in default splunk search app for query sourcetype="mysource" with 1 minute window, It does not display the events arriving after enabling this search. If I see the timeline, it shows the current time is proceeding but no event appears. I see the the count against the scanned events but matching events remains 0. It takes a while (almost 3-4 minutes) after which the events starts appearing as expected.

If I enable real-time for 'All Time' then everything works as expected.

Is there anything which I am missing about the real-time search.

Tags (2)
0 Karma
1 Solution

jflomenberg
Splunk Employee
Splunk Employee

It sounds like 1 of 2 things may be happening.

  1. Events could be arriving late. If events take more than 60s to reach the indexer then they wont meet the time bounds of rt-1m to rt (1 minute window) and won't be displayed. If you are receiving events every 15s then this seems less likely.
  2. You have some clock skew going on. If events have older time stamps they effectively reduce the 60s window. If all events are time stamped 1 minute in the past then the window is effectively zero.

Based on the comment about waiting 3-4 minutes for an event, it sounds like your indexer may be ahead 4 minutes relative to the data source.

View solution in original post

jflomenberg
Splunk Employee
Splunk Employee

It sounds like 1 of 2 things may be happening.

  1. Events could be arriving late. If events take more than 60s to reach the indexer then they wont meet the time bounds of rt-1m to rt (1 minute window) and won't be displayed. If you are receiving events every 15s then this seems less likely.
  2. You have some clock skew going on. If events have older time stamps they effectively reduce the 60s window. If all events are time stamped 1 minute in the past then the window is effectively zero.

Based on the comment about waiting 3-4 minutes for an event, it sounds like your indexer may be ahead 4 minutes relative to the data source.

asingla
Communicator

Thanks. It was the second reason.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...