Calculating storage requirements: how do I identify daily data rate without going beyond license capacity?

[hemendralodhi]

Storage Estimation : Daily data rate

Hello Folks,

I am trying to identify daily data ingestion for indexes. Based on this I want to calculate storage requirement taking retention/RF/SF into account.

I am using below query to identify daily data rate, but it seems it is not the correct way to identify as results are showing too much data beyond license capacity.

index=_internal source=*metrics.log group=per_index_thruput | eval GB=kb/(1024*1024) | timechart span=1d sum(GB) by series | addtotals fieldname=TotalDailyVolume(GB) | sort - _time

When I checked from Monitoring Console - License usage for last 30 days split by indexer - results are quite different and much less from above query.

I was under the impression that, from the query above, we could get the daily data ingestion rate. But it looks like i am missing something here.

Can you please advise and help me understand this?

Thanks

[deepashri_123 · Accepted solution]

Hey@hemendralodhi,

You can use the query below:

index=_internal source=license_usage.log type=Usage | stats sum(eval(b/1024/1024/1024)) AS volume_b by idx date_mday date_month date_year | stats max(volume_b) by idx date_month date_year

Also for storage estimation :
http://splunk-sizing.appspot.com/

Let me know if this helps!!!