OK. Summing up what's been already said and then adding some. The amount of data you're receiving affects several things: 1) Licensing. While it is indeed true what @gcusello pointed at - you can e...
See more...
OK. Summing up what's been already said and then adding some. The amount of data you're receiving affects several things: 1) Licensing. While it is indeed true what @gcusello pointed at - you can exceed your license for some time but this is meant for some unforeseen unusual situations. You should not rely on constatntly exceeding your license. Even if it does technically work (and judging by your license size your license is most probably a non-enforcing one which means it will only generate a warning), that's not what you bought. And any contact with Splunk (be it support case, be it a question for license extension) might end up with uncomfortable questions about your license size and real usage. Of course if this is something that happens just once in a while, that's OK. And BTW, if you exceed your ingestion limit it's the searching which gets blocked with an enforcing license, not indexing - you will not (contrary to some competitors' solutions) lose your data. 2) Storage - this is kinda obvious. The more data you're ingesting, the more storage you need to hold it given constant retention period. Since Splunk rolls buckets from cold to frozen (by default that means deleting the data) based on size limit or age limit, whichever is hit first that means that if you don't have enough space allocated and configured for your indexes, even if you are able to ingest that additional amount of data, it will not be held for long enough because it will get deleted due to lack of space. So instead of holding data for - let's say - last two weeks, you'll have only two days of data because the rest will have been pushed out of the index. 3) Processing power. There are some guidelines to sizing Splunk environments. Of course the real life performance may differ compared to the rule of thumb for generalized cases but still your cluster seems relatively small even for the amount of data you're receiving now (depending on how evenly spread the ingestion is across your sites it might be already hugely undersized), not to mention additional data you'll be receiving normally and definitely not adding the DDOS data. If you overstress the indexers you will clog your pipelines. That will create a pushback beause the forwarders won't be able to forward their data to indexers. So they might stop getting/receiving data from their sources. It's only half-bad if the sources can be "paused" and queried later for the missing data so you'll only cause lag. But if you have "pushing" sources (like syslog), you'll end up losing data. So licensing is the least of your problems.