Splunk Search

Scheduled search for populating summary index ignores the last 30 seconds of events

Sharzi
Explorer

Hello,

I recently faced an issue when populating a summary index. I scheduled a saved search to run every hour (with the last 60 minutes time range) and populate a summary index. The search takes around 5 minutes every time to be completed. My problem is that every time this scheduled search runs to populate the index, events in the last 30 seconds of the time range will be discarded from the results by Splunk. For example, for a one-hour time range like 9:00:00 to 10:00:00, the index is only populated with the events from 9:00:00 to 9:59:30. This issue caused some gaps and discrepancies in our index data.  Is there any way to solve this?

I searched a lot but couldn't find any answer 😞

Thanks.

Labels (1)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

There's often (usually) a delay from when an event is generated and when it is searchable within Splunk.  That delay can exceed 30 seconds and is why many Enterprise Security searches use latest=-1m rather than latest=now.

There may be other explanations, like a transaction command discarding events because they aren't (yet) part of a complete transaction.

---
If this reply helps you, Karma would be appreciated.

View solution in original post

ITWhisperer
SplunkTrust
SplunkTrust

I usually schedule summary index updates at least 5 minutes past the hour for the previous hour, just to give the indexers time to do their work.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

There's often (usually) a delay from when an event is generated and when it is searchable within Splunk.  That delay can exceed 30 seconds and is why many Enterprise Security searches use latest=-1m rather than latest=now.

There may be other explanations, like a transaction command discarding events because they aren't (yet) part of a complete transaction.

---
If this reply helps you, Karma would be appreciated.

Sharzi
Explorer

Hi @richgalloway ,

We don't have any transaction command in our search query, but as you said, the problem was "latest".

I changed the "latest" and now it is working fine! Thanks.

 

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...