Deployment Architecture

Search Head not Getting latest events from Indexer

TLAZO
Explorer

Good morning,

We have an splunk architecture with 2 Search Heads and 2 Indexers.
This morning when our user tried to look for today's logs from the SearchHead, he could not retrieve any data. Concerned about that, I ran the same query on both SearchHeads and Indexers, same as the user I could not find any data from today on the SearchHead but I found that on the Indexer. Last event was from 2 days ago.
That case only happened with one index. I tried the same for another couple of indexes and could not see the same behavior.
This is concerning me because users create their alerts on the SearchHead (They don't have access to the Indexers UI) and if they cannot see realtime information neither will the alerts.
After a 40 minutes waiting we could retrieve todays' information. Please, we need this to be addressed as soon as possible. We need real time responses.

Tags (1)
0 Karma

jplumsdaine22
Influencer

As @somesoni2 mentioned, check the user timezone settings. If there are no timezone issues have a look at http://docs.splunk.com/Documentation/Splunk/latest/Troubleshooting/Troubleshootingeventsindexingdela...

0 Karma

renjith_nair
Legend

Your splunk infra is clustered or distributed? Are the two search heads connecting to both indexers? Ideally you shouldn't be seeing any difference in search between indexer UI and search head unless your search head is also indexing some data. Have you seen any errors in splunkd logs on search head or indexers?

---
What goes around comes around. If it helps, hit it with Karma 🙂
0 Karma

TLAZO
Explorer

Yes, both indexers are visible from both search heads.

0 Karma

somesoni2
Revered Legend

Check if the timezone is same on all SH and Indexers.

0 Karma
Get Updates on the Splunk Community!

Monitoring MariaDB and MySQL

In a previous post, we explored monitoring PostgreSQL and general best practices around which metrics to ...

Financial Services Industry Use Cases, ITSI Best Practices, and More New Articles ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Splunk Federated Analytics for Amazon Security Lake

Thursday, November 21, 2024  |  11AM PT / 2PM ET Register Now Join our session to see the technical ...