Getting Data In

Amount of data sent by forwarder Vs Amount of data indexed Vs License usage Vol. Vs Size of Indexed data on Disk

splunker12er
Motivator

Amount of data sent by forwarder Vs Amount of data indexed Vs License usage Vol. Vs Size of Indexed data on Disk

ideally,
Amount of data sent by forwarder = Amount of data indexed (Considering no logs are directed to nullqueue)
E.g. 60MB data / min (Splunk forwarder ---> Splunk Indexer)

Here, are my assumptions,
Amount of data sent = 60mb
Amount of data indexed = 60mb
License usage = 60mb
Size of indexed data on disk = ? (Is there any metrics to identify this field ?)

I do run various search against metrics.log to analyze:

  1. speed of indexing
  2. amount of data sent by hosts from forwarder to indexer
  3. License usage by hosts

Is there any way we can correlate the above fields and derive the size of indexed data on disk? Any help is much appreciated. I would like to create a dashboard comparing these fields / on a daily-basis

help on this will be much useful

rphillips_splk
Splunk Employee
Splunk Employee

running this search will give you disk consumption by index & splunk_server:

| rest /services/data/indexes
| eval indexSizeGB = if(currentDBSizeMB > 1, round(currentDBSizeMB / 1024, 2), null())
| rename title AS index
| stats first(indexSizeGB) AS "Disk Usage (GB)" by index, splunk_server

NOTE: This Information is exposed in the "Indexes & Volumes" views of the Distributed Management Console as of 6.3.

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...