I would ideally like to catch the chances of developers breaking splunk dashboards/ reports at an early stage in the development life cycle (may at unit testing/integration testing stage) . Is there a way Splunk exposes what fields it used in a report or dashboard through some api so that I can write some unit tests to make sure we are still sending the fields etc.?
If all you care about is specific functionality of certain reports or searches, you could try coming up with a simpler mechanism using the REST API and a known sample data set to see if the returned results are changing. This would only be testing the searches and data, not dashboards and interactivity.