I am looking for a little clarity on this...
Like many folks here, I have carved out a small part of our total license volume for QA. For simplicity's sake, lets say, I have a 20GB/day license and I carve off 5GB/day for QA. But...one of our QA servers goes nuts and starts spewing crazy amounts of log data.
I know that if the QA_Pool license volume is violated 5 times in a 30-day period, then the search functionality for that pool stops working until one of those violations rolls off. That's fine...I get that.
What I don't quite get is this: Even though the QA box that is spewing logs has blasted through the QA_Pool license volume, it will continue to spew and continue to get indexed...right? And that indexing goes against our TOTAL license volume...so, even though we have the QA pool capped at 5GB, it doesn't prevent a runaway QA machine from blowing through our Full license volume all by itself.
Right? If so...is there an automated way to fix this? We are alerting when the license volumes hit 75%, but it's still a manual process to: 1) Figure out which QA box is spewing the data, 2) Log into that box and shut down the splunk forwarder.
Or am I missing something here?
Thanks in advance!
... View more