All Apps and Add-ons
Highlighted

Splunk App for Stream: How to find the total amount of data being indexed from forwarders for a 24 hour period?

Path Finder

Hola,

To avoid filling up our license, I'd like to find the total amount of data being indexed from the forwarders for a 24 hour period. I'm new to my environment, splunk, and was not sure where to start. Anyone got any suggestions?

0 Karma
Highlighted

Re: Splunk App for Stream: How to find the total amount of data being indexed from forwarders for a 24 hour period?

Splunk Employee
Splunk Employee

This will depend on what you've actually configured in terms of enabled Streams, enabled fields, etc.

Have you tried looking at the License Usage Report? The "Previous 30 days" tab plus "Split By Source/Source type" may offer the insight you need.

Highlighted

Re: Splunk App for Stream: How to find the total amount of data being indexed from forwarders for a 24 hour period?

Path Finder

Hmm, odd. Our Monitor App doesn't have anything.

Screenshot

Manually running the search string (Inspect) doesn't work either. I wonder if we've configured this properly...

0 Karma
Highlighted

Re: Splunk App for Stream: How to find the total amount of data being indexed from forwarders for a 24 hour period?

Path Finder

I'm now seeing "[subsearch]: No matching fields exist" at the top of splunk when I go to the License Usage page.

0 Karma
Highlighted

Re: Splunk App for Stream: How to find the total amount of data being indexed from forwarders for a 24 hour period?

Splunk Employee
Splunk Employee

I was using the License Usage report here: http://yoursplunkserver:8000/en-US/manager/search/licenseusage

You can access via Settings -> System -> Licensing.

alt text

And then the green Usage Report button.

alt text

0 Karma
Highlighted

Re: Splunk App for Stream: How to find the total amount of data being indexed from forwarders for a 24 hour period?

Splunk Employee
Splunk Employee

Search adapted from http://answers.splunk.com/answers/168412/how-to-determine-index-volume-by-sourcetype.html:

 earliest=-1d@d latest=@d  index=_internal source=*license_usage.log* type=Usage st=stream*
 | stats sum(b) AS Bytes by st 
 | sort -Bytes

I just added "st=stream*" to only include data generated by stream.

0 Karma
Highlighted

Re: Splunk App for Stream: How to find the total amount of data being indexed from forwarders for a 24 hour period?

Path Finder

Thanks. However, there is no type=Usage in any of the license_usage.logs in my index on any of the indexers or even the search head.

0 Karma
Highlighted

Re: Splunk App for Stream: How to find the total amount of data being indexed from forwarders for a 24 hour period?

Splunk Employee
Splunk Employee

This may not be as accurate as the other methods, but as an alternative you can also just run a search on the raw events to get a feel for index usage:

source=stream* | eval l=len(_raw) | stats sum(l) as BytesSum | eval TotalMB=round(BytesSum/(1024*1024), 0) | fields - BytesSum

(for 1 day, use something like "Yesterday" or "Last 24 hours" for the search time range)

View solution in original post

Highlighted

Re: Splunk App for Stream: How to find the total amount of data being indexed from forwarders for a 24 hour period?

Path Finder

Sweet, exactly what I was looking for. I'll study the string and see what you did. Appreciate it!

0 Karma