Monitoring Splunk

Abruptly missing old data

bloizides
Observer

We have been collecting syslog data on our hosts for the past 5 years or so. Syslog is in our 'main' index, along with other events.  We performed a query today and noticed that all of our data in 'main', and a few other indexes, prior to late-2018 are gone. *poof*   Looking at our monitoring graphs, it looks like our disk space usage plummeted back in March, which is probably when this happened. 

I have no idea where the data went. The time of the drop in disk usage does not correspond to any upgrade or maintenance during that period. 

Can anyone offer suggestions on how to troubleshoot this? Is it possible that, for some reason, Splunk rolled it over to frozen? I'm grasping. Any suggestions would be appreciated. 

Thanks!

Labels (2)
0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Character substitutions with Regular Expressions

This challenge was first posted on Slack #puzzles channelFor BORE at .conf23, we had a puzzle question which ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...

[Puzzles] Solve, Learn, Repeat: Matching cron expressions

This puzzle (first published here) is based on matching timestamps to cron expressions.All the timestamps ...