Getting Data In

Amount of data sent by forwarder Vs Amount of data indexed Vs License usage Vol. Vs Size of Indexed data on Disk

splunker12er
Motivator

Amount of data sent by forwarder Vs Amount of data indexed Vs License usage Vol. Vs Size of Indexed data on Disk

ideally,
Amount of data sent by forwarder = Amount of data indexed (Considering no logs are directed to nullqueue)
E.g. 60MB data / min (Splunk forwarder ---> Splunk Indexer)

Here, are my assumptions,
Amount of data sent = 60mb
Amount of data indexed = 60mb
License usage = 60mb
Size of indexed data on disk = ? (Is there any metrics to identify this field ?)

I do run various search against metrics.log to analyze:

  1. speed of indexing
  2. amount of data sent by hosts from forwarder to indexer
  3. License usage by hosts

Is there any way we can correlate the above fields and derive the size of indexed data on disk? Any help is much appreciated. I would like to create a dashboard comparing these fields / on a daily-basis

help on this will be much useful

rphillips_splk
Splunk Employee
Splunk Employee

running this search will give you disk consumption by index & splunk_server:

| rest /services/data/indexes
| eval indexSizeGB = if(currentDBSizeMB > 1, round(currentDBSizeMB / 1024, 2), null())
| rename title AS index
| stats first(indexSizeGB) AS "Disk Usage (GB)" by index, splunk_server

NOTE: This Information is exposed in the "Indexes & Volumes" views of the Distributed Management Console as of 6.3.

0 Karma
Get Updates on the Splunk Community!

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...