All Apps and Add-ons

Calculating storage requirements: how do I identify daily data rate without going beyond license capacity?

hemendralodhi
Contributor

Storage Estimation : Daily data rate

Hello Folks,

I am trying to identify daily data ingestion for indexes. Based on this I want to calculate storage requirement taking retention/RF/SF into account.

I am using below query to identify daily data rate, but it seems it is not the correct way to identify as results are showing too much data beyond license capacity.

index=_internal source=*metrics.log group=per_index_thruput | eval GB=kb/(1024*1024) | timechart span=1d sum(GB) by series | addtotals fieldname=TotalDailyVolume(GB) | sort - _time

When I checked from Monitoring Console - License usage for last 30 days split by indexer - results are quite different and much less from above query.

I was under the impression that, from the query above, we could get the daily data ingestion rate. But it looks like i am missing something here.

Can you please advise and help me understand this?

Thanks

0 Karma
1 Solution

deepashri_123
Motivator

Hey@hemendralodhi,

You can use the query below:

index=_internal source=license_usage.log type=Usage | stats sum(eval(b/1024/1024/1024)) AS volume_b by idx date_mday date_month date_year | stats max(volume_b) by idx date_month date_year

Also for storage estimation :
http://splunk-sizing.appspot.com/

Let me know if this helps!!!

View solution in original post

0 Karma

deepashri_123
Motivator

Hey@hemendralodhi,

You can use the query below:

index=_internal source=license_usage.log type=Usage | stats sum(eval(b/1024/1024/1024)) AS volume_b by idx date_mday date_month date_year | stats max(volume_b) by idx date_month date_year

Also for storage estimation :
http://splunk-sizing.appspot.com/

Let me know if this helps!!!

0 Karma
Get Updates on the Splunk Community!

BSides Splunk 2022 - The Call for Papers is now Open!

TLDR; Main Site: https://bsidessplunk.com CFP Site: https://bsidessplunk.com/cfp CFP Opens: December 15th, ...

Sending Metrics to Splunk Enterprise With the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...

What's New in Splunk Cloud Platform 9.0.2208?!

Howdy!  We are happy to share the newest updates in Splunk Cloud Platform 9.0.2208! Analysts can benefit ...