How to troubleshoot why I'm missing log data in Splunk for one day?

[Navanitha]

Hi,

I have logs coming into Splunk from our Mainframe server for a long time. I noticed that Splunk is suddenly not showing any logs on 25/04/2016 and there were partial results on 24/04. Although it is working fine now, I still don't see logs for only 25/04. What might be the possibilities for such discrepancies and is there something I need to check on my end?

Thank you..

[jkat54]

Define "suddenly" please.

Does this mean that yesterday you had data for 25/04 and 24/04 but today "suddenly" the data no longer appears?

Or does it mean, you have a gap in your data on 25/04 and 24/04 that you didnt notice until today?

Possible issues for the 1st scenario:
-Bad data retirement/retention policy
-Someone used the |delete command
-Someone manually erased buckets from the filesystem
-Filesystem corruption

Possible issues for the 2nd scenario:
-Network was down
-Forwarders were down
-Splunk was down
-Maintenance to mainframe
-Maintenance to anything between mainframe and splunk indexers
-etc

[Navanitha]

it is the second scenario, I have a gap in data for those two dates and till now, I don't see the data coming in for those two days until now.

so assuming the forwarder was down/network was down, how can I get the data for those days into Splunk now?