Splunk search command for daily indexed volume?

[remy06]

Hi,

I am currently testing out two searches to report and alert on the daily indexed volume.

The first search is as follows.:
index=_internal sourcetype=splunkd LicenseManager-Audit todaysBytesIndexed licenseSize | eval todayMb=(todaysBytesIndexed/1024/1024) | eval percentUsed=round(100*todayMb/licenseSize,2)

The second search will have:
index=_internal source=*metrics.log splunk_server="*" group="per_host_thruput" | eval MB=kb/1024 | chart sum(MB) by series | sort sum(MB) | addcoltotals col=t | fillnull value="[ Total Indexed Volume ] last 24 hours" Host(s)

Both are scheduled to run at midnight daily and should report on the indexed volume for that day.

However,I've noticed the reports for today,these 2 searches show different indexed volume which are misleading.

Which command is more accurate?Any suggestion or advice?

[yannK]

Since Splunk 4.2, a new log record the license usage on the license-master.

you can use new searches for the detail per : sourcetype/host/source per pool
see http://wiki.splunk.com/Community:TroubleshootingIndexedDataVolume

[sideview · Accepted solution]

The LicenseManager search will not count things like index=_internal and index=_audit data, because that volume doesnt count against your license. And the per_host search does.

However you can use the per_index_thruput numbers and then filter out the indexes that have leading underscores.

index=_internal source=*metrics.log splunk_server="*" group="per_index_thruput" | eval MB=kb/1024 | stats sum(MB) by series | rename series as index | search index!=_* | sort sum(MB) | addcoltotals | fillnull value="[ Total Indexed Volume ] last 24 hours" index

If I run this search against the data from yesterday and compare it to the licenseManager's search from today (necessary because the licenseManager runs just after midnight and its talking about yesterday), then the numbers seem very close to eachother but oddly they are not equal. Im not sure why.

[jlaigo2]

This is what I have been looking for thanks for your answer:)