Recently, my Splunk environment decided to re-index ALL of my IIS logs (which crushed my daily license quota). I have been tasked with finding the root cause of why that happened.
Is there anyway to find in the Splunk logs why it decided to re-index all these logs?
A place to start would be to look at timestamps on your fishbucket.. Fishbucket is responsible for keeping pointers of what's been indexed, so this would be a reasonable assumption to check
Looks like a new deployed was created that monitored the IIS log location and the old deployed app was removed.
Would that cause Splunk to re-index? I thought that data was separate from the app.
It would. Once the old app was removed, it will clear Splunk's monitoring list/_fishbucket which tracks the files being monitored (and till what point it has monitored the log file). When the new app was deployed, Splunk will treat that a new data monitoring and will read the file from start and can cause duplicates.