topic Re: How to "correct before midnight"? in Getting Data In

How to "correct before midnight"?

seriea — Tue, 28 Jun 2011 12:50:46 GMT

My Splunk installation has indexed some files that weren't supposed to be indexed (dot files created by rsync), and now I'm seeing a Pool quota overage alert in Manager > Licensing > Licensing alerts.

The message states "please correct before midnight", but doesn't tell me how.

I can search for the unwanted events by filtering the source filed, and I could pipe the result to the Delete operator - but AFAIK, that has zero effect on the licensing.

So what exactly is Splunk encouraging me to do before midnight?

Re: How to "correct before midnight"?

jbsplunk — Tue, 28 Jun 2011 15:03:58 GMT

Splunk is warning you that if your situation isn't corrected, you may run into a license violation. If you don't correct the situation, it may carry over into the next day as well. As such, the proper course of action in order to correct would be to determine where the additional sources of data are coming from and to either disable the inputs or set up some type of null queue routing to keep the data from being indexed. Instructions for routing unwanted/unecessary data to the null queue can be found here:

http://docs.splunk.com/Documentation/Splunk/latest/Deploy/Routeandfilterdatad

You can find some useful searches here:

http://www.splunk.com/wiki/Community:TroubleshootingIndexedDataVolume

Removing the indexed data in a surgical way isn't possible. Data is aged out via retention policies that are based on age and size. Even if you did age the data out, splunk would still know the volume of data recorded for the day and you'd run into a violation.

Re: How to "correct before midnight"?

joemac — Fri, 18 Nov 2011 20:51:53 GMT

This response doesn't provide an answer to the actual question of how to "correct before midnight".

I understand that Splunk is warning me because I have exceeded by index quota, but I would like to understand if there is a way for me to remove some of the indexed data so that I can stay below my daily quota. I have already identified where the additional data came from and I have stopped the flow of additional data.

How do I correct before midnight?

Re: How to "correct before midnight"?

jbsplunk — Fri, 18 Nov 2011 21:10:08 GMT

If you've identified and stopped the flow of additional data, you've corrected the situation. That doesn't change the fact that you went over what was allowed for a particular queue.

Re: How to "correct before midnight"?

joemac — Fri, 18 Nov 2011 21:22:34 GMT

Thanks.
So, just to confirm, if my data has already been indexed (and I have already exceeded my indexing quota), there is nothing that I can do to "un-index" that data?

Wouldn't the nullQueue routing just avoid indexing future data?

Re: How to "correct before midnight"?

jbsplunk — Fri, 18 Nov 2011 21:26:38 GMT

Yes, but often you find that you were indexing data you didn't really care about anyway. There is no way to 'un-index' data.

Re: How to "correct before midnight"?

omartorres — Fri, 16 Dec 2011 18:07:51 GMT

So I just noticed that we incorrectly added the security log which I don't need in Splunk and that was millions of entries that I don't want to be indexed. This means that even though I removed that data input, I can't remove it from the index? We just installed it yesterday and I really don't want the security event logs in Splunk.

Re: How to "correct before midnight"?

FunPolice — Fri, 10 Feb 2012 00:11:11 GMT

I always put a new data source to a "development" index first - that way if I make a mistake I can wipe the index without losing all of my other data.

Re: How to "correct before midnight"?

Masa — Mon, 04 Feb 2013 21:15:59 GMT

You can "correct" and avoid a license violation only if you have other pools to move extra volume to the pool you have the warning. Or, if you have additional license you haven't used, you can add it to the pool. If the total of the license volume is larger than the day's indexed volume, you can avoid a license violation in the mid-night.

Many users use only one pool for all the licenses. In such case, unfortunately there is no way to do any action for "correct-before-midnight" warning.

Re: How to "correct before midnight"?

wsw70 — Tue, 12 Feb 2013 13:25:02 GMT

Thanks for the details on how actually "correct" the issue (= there is nothing you can do if you do not have additional licenses)

Re: How to "correct before midnight"?

ben363 — Thu, 19 Jun 2014 04:03:18 GMT

You can delete records you don't want, but it doesn't recover your quota, and it doesn't recover disk space.

Re: How to "correct before midnight"?

driver8man — Thu, 09 Feb 2017 20:36:50 GMT

I downvoted this post because this doesn't actually answer the question. the correct answer is posted below.

Re: How to "correct before midnight"?

afarren — Thu, 14 Sep 2017 21:00:59 GMT

I downvoted this post because doesn't answer question