Monitoring Splunk

Abruptly missing old data

bloizides
Observer

We have been collecting syslog data on our hosts for the past 5 years or so. Syslog is in our 'main' index, along with other events.  We performed a query today and noticed that all of our data in 'main', and a few other indexes, prior to late-2018 are gone. *poof*   Looking at our monitoring graphs, it looks like our disk space usage plummeted back in March, which is probably when this happened. 

I have no idea where the data went. The time of the drop in disk usage does not correspond to any upgrade or maintenance during that period. 

Can anyone offer suggestions on how to troubleshoot this? Is it possible that, for some reason, Splunk rolled it over to frozen? I'm grasping. Any suggestions would be appreciated. 

Thanks!

Labels (2)
0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Splunk Auto Ingestion Parallel Pipeline Scaling

Why this feature matters Many Splunk environments experience ingestion pressure long before the host is fully ...

Best Practices: Splunk auto adjust pipeline queue

When you enable autoAdjustQueue in Splunk, maxSize should be understood as the queue size Splunk starts with ...

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...