Splunk Enterprise

index size keeps adding on top of previous indexes

afatehi
Explorer

I'm using the splunk light free. i know it has a limitation of 500 MB of daily indexing which for now it has reached 360 MB.
I'm quite sure that this 360 MB, is the index since I've installed splunk (almost a month), The question is, ain't i supposed to have 500 MB daily? meaning that each day i can index up to 500 MB? if so, why do my indexes keep adding up to my previous indexes files?

P.S. i've cleaned up other (unnecessary)indexes to free some space, and also disabled data input for _audit and _internal.

i appreciate your answers in advance.

Tags (1)
0 Karma

adonio
SplunkTrust
SplunkTrust

Hello,

your license is to index up to 500MB per day with no charge (money) from splunk. and also without your revoking your search ability if you violate that limit.
the indexed data, will remain in an index (file system) until retention period is met or index size caps it
splunk default indexes size is 500GB
you can easily change it by doing to settings -> indexes -> edit the index you want
hope it helps

0 Karma

afatehi
Explorer

Hello,

Thanks for the answer adonio. to be precise, my problem is with this error " Your Splunk Light license expired or you have exceeded your license limit too many times" which to my understanding is due to my index size going above 500 MB. i've cleaned up some indexes to bring the overall index size back to below 500 MB which should solve the problem in a month.
the question is, let's say my _main index was 350 MB yesterday, and 365 MB today, so shouldn't it mean that my splunk has been indexing 15 MB out of my 500 MB free license in a day?
but the way it is, the whole 365 MB counts as my free license not just the 15 MB of what splunk indexed during the last 24 hours.

0 Karma

rbittner_splunk
Splunk Employee
Splunk Employee

Afetehi,

Hi I can probably help but will need you to ell me what version you are running, and when you first installed Splunk Light?

I am pretty sure that this has nothing to do with the index size as Splunk Light licensing does not limit the total size of the index (as mentioned that is configuration on data retention).

There are two possibilities: either you have a version that had a 1 year free license or you exceeded your indexing volume too many times at some point. If you have a 1 year license you can either upgrade to the most current license and switch to the free version or if you have for some reason exceeded your license we will need to provide you with a reset license.

Looking forward to hearing from you.

Robb - Splunk Light Product manager

0 Karma

afatehi
Explorer

Hi Robb,

thanks for your comment.

i believe it's splunk light 6.6.1. we've been having it installed for two months. the problem is surely indexing volume, based on the message i receive, and the version. what confuses me is the info on setting -> licensing, "today's used volume" is 4 MB! i'm receiving traffic from 8-10 different switches only which do not really generate much logs.

since the index size shown on data -> indexes has nothing to do with that daily 500 MB limitation, i'd like to know how exactly can i monitor the incoming licensed volume.

0 Karma

rbittner_splunk
Splunk Employee
Splunk Employee

Afetehi,

I sent you an email since we are now getting into specifics about your particular situation.

Robb

0 Karma

adonio
SplunkTrust
SplunkTrust

splunk counts the data for every 24 hours from 00:00:00 till 23:59:59
if you indexed more than your license, here free so 500MB, you will get a violation.
if you get number of violations per month (not sure about the number for splunk light) your search ability is blocked

afatehi
Explorer

so if i keep the entire size of my indexes in settings -> indexes below 500 MB i should be good?

0 Karma

adonio
SplunkTrust
SplunkTrust

no,
these two metrics has nothing to do with each other.
splunk licenses counts on data coming in each day, it zeros out at 00:00:00 every 24 hours
size of indexes is the amount of the data that already in splunk and keeps growing until meeting the cap size of the index (directory)

afatehi
Explorer

alright, i got it. thanks.

one last question, how can i measure or control (i believe controlling it is a total different subject of course) the data coming in? just to make sure i'm not going beyond the limit.

0 Karma

adonio
SplunkTrust
SplunkTrust

you have a license meter in your splunk
not sure exactly how it is in splunk light but you can try and go to settings -> licensing and look at the dashboards and reports for daily and monthly ingestion
if that answers your question, kindly mark the answer that suits and up vote any comments you found helpful
cheers