All Apps and Add-ons

Splunk App for Stream: How to find the total amount of data being indexed from forwarders for a 24 hour period?

helius
Path Finder

Hola,

To avoid filling up our license, I'd like to find the total amount of data being indexed from the forwarders for a 24 hour period. I'm new to my environment, splunk, and was not sure where to start. Anyone got any suggestions?

0 Karma
1 Solution

mdickey_splunk
Splunk Employee
Splunk Employee

This may not be as accurate as the other methods, but as an alternative you can also just run a search on the raw events to get a feel for index usage:

source=stream* | eval l=len(_raw) | stats sum(l) as BytesSum | eval TotalMB=round(BytesSum/(1024*1024), 0) | fields - BytesSum

(for 1 day, use something like "Yesterday" or "Last 24 hours" for the search time range)

View solution in original post

mdickey_splunk
Splunk Employee
Splunk Employee

This may not be as accurate as the other methods, but as an alternative you can also just run a search on the raw events to get a feel for index usage:

source=stream* | eval l=len(_raw) | stats sum(l) as BytesSum | eval TotalMB=round(BytesSum/(1024*1024), 0) | fields - BytesSum

(for 1 day, use something like "Yesterday" or "Last 24 hours" for the search time range)

helius
Path Finder

Sweet, exactly what I was looking for. I'll study the string and see what you did. Appreciate it!

0 Karma

mdickey_splunk
Splunk Employee
Splunk Employee

Search adapted from http://answers.splunk.com/answers/168412/how-to-determine-index-volume-by-sourcetype.html:

 earliest=-1d@d latest=@d  index=_internal source=*license_usage.log* type=Usage st=stream*
 | stats sum(b) AS Bytes by st 
 | sort -Bytes

I just added "st=stream*" to only include data generated by stream.

0 Karma

helius
Path Finder

Thanks. However, there is no type=Usage in any of the license_usage.logs in my index on any of the indexers or even the search head.

0 Karma

jsie_splunk
Splunk Employee
Splunk Employee

This will depend on what you've actually configured in terms of enabled Streams, enabled fields, etc.

Have you tried looking at the License Usage Report? The "Previous 30 days" tab plus "Split By Source/Source type" may offer the insight you need.

helius
Path Finder

Hmm, odd. Our Monitor App doesn't have anything.

Screenshot

Manually running the search string (Inspect) doesn't work either. I wonder if we've configured this properly...

0 Karma

jsie_splunk
Splunk Employee
Splunk Employee

I was using the License Usage report here: http://yoursplunkserver:8000/en-US/manager/search/licenseusage

You can access via Settings -> System -> Licensing.

alt text

And then the green Usage Report button.

alt text

0 Karma

helius
Path Finder

I'm now seeing "[subsearch]: No matching fields exist" at the top of splunk when I go to the License Usage page.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...