How do I find the source of indexed data volume?

[rjlohan]

I am investigating Splunk for use with a new operational solution, but I seem to keep hitting the trial index limits and getting warned. The only data I've imported is the tutorial data, some small (<1MB) log files and some perfmon results using the 'Splunk for Microsoft Windows Infrastructure' app.

When I look at Settings->Indexes, the 'Current size' of all indexes is less than 200MB, but my license usage says I'm close to hitting 1GB.

How do I find where this licenses limit is calculated from? Perhaps some of my index samples are excessive. Since this is only a new (hours old) trial instance on a single machine, I'm concerned how the cost will actually pan out for a full live instance. These numbers aren't making sense to me right now...

[bhawkins1]

In Splunk 6, in the license manager /en-US/manager/search/licenseusage, the "Previous 30 Days" tab gives you the option to "Split By", for example, Source Type, which was sufficient for my usage. No need for additional apps.

[jtrucks · Accepted solution]

To see your license usage, go to Settings menu, then Licensing. To further analyze the details, install the Splunk on Splunk (S.O.S.) app and use it to analyze your data sources and indexing/license use.

Also, the on-disk size of indexes is not the same size as your total ingested data volume due to compression. Generally it works out to roughly 50% disk usage compared to data ingested. However, this is an average that works out over time across multiple types of data sources. If you've ingested data that compresses poorly, you will have more disk used than 50%, or if you've ingested data that compresses well, you will have less disk used than 50%.

[rjlohan]

Thanks for this suggestion; the S.O.S indexing detail clearly shows a breakdown which is exactly what I needed. winnetmon looks to be the main culprit. Probably misconfigured that one for what I really needed.

[Richfez]

May I suggest the Fire Brigade app?

If you are on 6.x (I expect so given your description above), you'll need
Technology Add-on for Fire Brigade version 2
and
Fire Brigade version 2

It will take a day for it to generate information, but you should be able to find out more information then.