Monitoring Splunk

Abruptly missing old data

bloizides
Observer

We have been collecting syslog data on our hosts for the past 5 years or so. Syslog is in our 'main' index, along with other events.  We performed a query today and noticed that all of our data in 'main', and a few other indexes, prior to late-2018 are gone. *poof*   Looking at our monitoring graphs, it looks like our disk space usage plummeted back in March, which is probably when this happened. 

I have no idea where the data went. The time of the drop in disk usage does not correspond to any upgrade or maintenance during that period. 

Can anyone offer suggestions on how to troubleshoot this? Is it possible that, for some reason, Splunk rolled it over to frozen? I'm grasping. Any suggestions would be appreciated. 

Thanks!

Labels (2)
0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Quick connection discovery mode for forwarders

When a Splunk forwarder loses connectivity to its indexers, it does not always reconnect immediately. In many ...

Build and Launch AI Agents from Your Splunk Workflows

  Register We’ve all been there: juggling alerts, runbooks, and endless manual searches. What if you could ...

Splunk Cloud Application Management in Terraform

Register   On Tuesday, August 4 at 11AM PDT / 2PM EDT, we’re diving into how you can bring Infrastructure as ...