Knowledge Management

Saved search deleting older records from Summary Index

Dark_Ichigo
Builder

A very strange behaviour has occurred, we have defined a saved search that gets stored into its own Summary Index, below is the saved index stanza we have defined,

[Unique GroupId Type]
action.email.inline = 1
action.summary_index = 1
action.summary_index._name = summary_index
alert.severity = 2
alert.suppress = 1
alert.suppress.period = 1h
alert.track = 1
# Search, run this daily at 5:00 am
cron_schedule = 0 5 * * *
description = <description>
dispatch.earliest_time = -1d@d
dispatch.latest_time = now
enableSched = 1
realtime_schedule = 0
search = index="index_data" GroupId!="null" | fields GroupId | stats count by GroupId

The data we have is 140 days back in time, so by changing the dispatch.earliest_time = -140d@d, and restart Splunk, ther summary index gets populated with the data successfully.

Now here's the strange part, after setting the configuration parameter back to dispatch.earliest_time = -1d@d and restart Splunk, all the data disappears, the data that was stored there originally is gone, keeping in mind that we have no current data or any data from yesterday, so we do not expect any new data, but at least the saved search would have kept the old data that was there originally, all my other saved searches are pretty much the same but are working, I cant find the issue here to why its removing the data.

Any ideas?

0 Karma
1 Solution

Dark_Ichigo
Builder

Apparently the issue was with the Search itself, you need to include the _time field in the search, in my case I was omitting it as part of the raw search which is why the data was being removed from the summary index.

All good now

View solution in original post

0 Karma

Dark_Ichigo
Builder

Apparently the issue was with the Search itself, you need to include the _time field in the search, in my case I was omitting it as part of the raw search which is why the data was being removed from the summary index.

All good now

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...