Deployment Architecture

Search Head not Getting latest events from Indexer

TLAZO
Explorer

Good morning,

We have an splunk architecture with 2 Search Heads and 2 Indexers.
This morning when our user tried to look for today's logs from the SearchHead, he could not retrieve any data. Concerned about that, I ran the same query on both SearchHeads and Indexers, same as the user I could not find any data from today on the SearchHead but I found that on the Indexer. Last event was from 2 days ago.
That case only happened with one index. I tried the same for another couple of indexes and could not see the same behavior.
This is concerning me because users create their alerts on the SearchHead (They don't have access to the Indexers UI) and if they cannot see realtime information neither will the alerts.
After a 40 minutes waiting we could retrieve todays' information. Please, we need this to be addressed as soon as possible. We need real time responses.

Tags (1)
0 Karma

jplumsdaine22
Influencer

As @somesoni2 mentioned, check the user timezone settings. If there are no timezone issues have a look at http://docs.splunk.com/Documentation/Splunk/latest/Troubleshooting/Troubleshootingeventsindexingdela...

0 Karma

renjith_nair
Legend

Your splunk infra is clustered or distributed? Are the two search heads connecting to both indexers? Ideally you shouldn't be seeing any difference in search between indexer UI and search head unless your search head is also indexing some data. Have you seen any errors in splunkd logs on search head or indexers?

---
What goes around comes around. If it helps, hit it with Karma 🙂
0 Karma

TLAZO
Explorer

Yes, both indexers are visible from both search heads.

0 Karma

somesoni2
Revered Legend

Check if the timezone is same on all SH and Indexers.

0 Karma
Get Updates on the Splunk Community!

Thanks for the Memories! Splunk University, .conf24, and Community Connections

Thank you to everyone in the Splunk Community who joined us for .conf24 – starting with Splunk University and ...

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...