Knowledge Management

Saved search deleting older records from Summary Index

Dark_Ichigo
Builder

A very strange behaviour has occurred, we have defined a saved search that gets stored into its own Summary Index, below is the saved index stanza we have defined,

[Unique GroupId Type]
action.email.inline = 1
action.summary_index = 1
action.summary_index._name = summary_index
alert.severity = 2
alert.suppress = 1
alert.suppress.period = 1h
alert.track = 1
# Search, run this daily at 5:00 am
cron_schedule = 0 5 * * *
description = <description>
dispatch.earliest_time = -1d@d
dispatch.latest_time = now
enableSched = 1
realtime_schedule = 0
search = index="index_data" GroupId!="null" | fields GroupId | stats count by GroupId

The data we have is 140 days back in time, so by changing the dispatch.earliest_time = -140d@d, and restart Splunk, ther summary index gets populated with the data successfully.

Now here's the strange part, after setting the configuration parameter back to dispatch.earliest_time = -1d@d and restart Splunk, all the data disappears, the data that was stored there originally is gone, keeping in mind that we have no current data or any data from yesterday, so we do not expect any new data, but at least the saved search would have kept the old data that was there originally, all my other saved searches are pretty much the same but are working, I cant find the issue here to why its removing the data.

Any ideas?

0 Karma
1 Solution

Dark_Ichigo
Builder

Apparently the issue was with the Search itself, you need to include the _time field in the search, in my case I was omitting it as part of the raw search which is why the data was being removed from the summary index.

All good now

View solution in original post

0 Karma

Dark_Ichigo
Builder

Apparently the issue was with the Search itself, you need to include the _time field in the search, in my case I was omitting it as part of the raw search which is why the data was being removed from the summary index.

All good now

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

(view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...