How to generate a search to determine why our license usage doubled?

[avalle]

Over the holiday weekend our license usage doubled and I need to figure out what is causing this. I have S.o.S - Splunk on Splunk installed but I am not sure if this is where I find this information. anyone have any searches to determine what devices started sending data or volume change?

[woodcock · Accepted solution]

Try running these searches for the last 10 days:

| tstats count GroupBy index _time span=1d 
| timechart span=1d sum(count) BY index

Get more granular with this:

| tstats count GroupBy sourcetype index _time span=1d 
| eval entity = sourcetype . "/" . index 
| timechart span=1d sum(count) BY entity

Get the most granularity with this:

| tstats count GroupBy host sourcetype index _time span=1d 
| eval entity = host . "/" . sourcetype . "/" . index 
| timechart span=1d sum(count) BY entity

These will chart changes in # of events, but if the problem is that this metric is more or less the same and the problem is that the average size of a particular event has increased, then it will not be evident.

[avalle]

Thank you! this helped determine what index doubled in size. How do i run these searches for that particular index? I tried adding index=XXX but the search did not like it at all

[woodcock]

Also, check out the Meta Woot! app:

https://splunkbase.splunk.com/app/2949/

[avalle]

Oh this looks like a great app!!! do i have to install it in all HF and indexers? I installed in the search head and no data came up.

[woodcock]

After the tstats line add this line:

| search index="XXX"

[avalle]

NICE! that worked!!