I have noticed that the latest event in the my index in the PROD instance of Splunk has no events after 9:01: AM this morning, 3/23. The same index in the Sandbox is indexing properly and events are showing almost real-time.
Could you please let know how to check the error and what is causing this issue? Also are there any systems in place to alert us if data is not indexing correctly, or do we need to set up Splunk alerts for each index to get notified of these types of issues?
This is a very open ended question, you could start with Troubleshoot the input process and the monitoring console overview . Either custom alerts or applications like Meta Woot! could assist with tracking which index received data and which point in time...