Reporting

Durable Search does not refill the data

blablabla
Path Finder

Hello, 

I have a scheduled report, which is used to fill a summary index. 

  • The report for the summary index is scheduled hourly (based on indextime)
  • The event data is usually indexed on daily basis at once (sometimes though there are multiple data ingestions of the server per day)

Now the thing is, that the search is inefficient. In normal cases, this is no issue, but when a day was skipped for ingestion and the next day therefore indexes two days of event data at once, the report for the summary index will crash and there will be a gap in the summary index.

To avoid data gaps, I configured the search to be durable

This is the configuration regarding the time constraints of the search:

blablabla_0-1648106475925.pngblablabla_0-1648106475925.png

This is the configuration for the durable search (Backfill method is multiple, as it is recommended for searches with transforming commands)

blablabla_2-1648106629673.pngblablabla_2-1648106629673.png

To test, if data gaps of the summary index are recovered automatically, I stopped the dataflow of for the event index for 5 days and then indexed all the data at once, knowing, that this will lead the scheduled report to crash.

Unfortunately, the durable search did not recover the data gap of the summary index. From the 5 days only a few hours were indexed into the summary index. Does someone have an idea why this is so?

Thanks and best regards

 

Labels (2)
0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

How much can you really learn in 3 minutes?

Observability can certainly be hard to understand – there's a lot of jargon and buzzwords and it seems to ...

Event Series: The Agentic SOC: Trust Before Autonomy

AI is fundamentally changing security operations, but true progress requires more than just automation—it ...

Free Professional Services for .conf26 Attendees

This year at .conf26, we are doing something a little different. We are bringing the best minds from ...