We are on the Splunk Free license, which has a daily indexing limit of 500Mb. This has never before been a problem because we've had a pretty consistently stable +2Mb/day log volume. The total size of ALL of our logs, 150Mb, is far less than the daily limit. Yet somehow Splunk has complained and shut down our license.
Does anyone have familiarity with this kind of error? Why would it trigger on such a small log database and low flow rate?
Check your licensing report and split the usage report by index, host or source to narrow down what consumes your license.
Most common causes:
1) adding new source which "backtracks" logs from the past. But that's - as you say - not your case since all logs available for indexing are 150 MB in size.
2) ingesting same files multiple times (usually due to wrongly configured CRC length/CRC salt settings.
It could be a couple of things. Randomly guessing it could have been from a host that was offline for a while and then brought back online thus the Splunk Forwarder played catch-up and sent a huge amount of data to the indexers.
You would have to investigate what caused the sudden unexpected increase.
Is this the first time you've had the violation? If you're using the free license as long as you hadn't had three or more warnings you should still be able to search and investigate what index/sourcetype/host sent the most data on the day that you received the violation.