Splunk Search

How to write a search to identify the total amount of data is being indexed by MB per minute?

danielsofoulis
Path Finder

I need to identify the total amount of data is being indexed by my indexer cluster, by MB per minute.
I think the best way would be to search against the license.log file but I am having trouble with writing the search.
Any assistance is greatly appreciated.

1 Solution

twinspop
Influencer

Try this, adding indexers as required:

(host=your_indexer OR host=your_indexer2 OR host=your_indexer3) index=_internal metrics name=index_thruput sourcetype=splunkd | 
    timechart span=1m sum(eval(kb/1024)) as "MB/min"

If you run for more than a few hours, forcing a different span in the timechart, you'll need to do some extra math. This one has 5 minute buckets and therefore requires dividing the value by 5.

(host=your_indexer OR host=your_indexer2 OR host=your_indexer3) index=_internal metrics name=index_thruput sourcetype=splunkd | 
    timechart span=5m sum(eval(kb/1024/5)) as "MB/min"

This one by hour

(host=your_indexer OR host=your_indexer2 OR host=your_indexer3) index=_internal metrics name=index_thruput sourcetype=splunkd | 
    timechart span=1h sum(eval(kb/1024/60)) as "MB/min"

View solution in original post

Richfez
SplunkTrust
SplunkTrust

You could start from the DMC's searches. (Assuming you are on a reasonably recent Splunk version).

Settings, Distributed Management Console. Then in the DMC's menus, drop down Indexing, then look at the various "Indexing Performance" pages.

If any of those is close, you can hover your mouse over the chart and underneath it will be a magnifying glass icon to "Open in Search".

While it may or may not be exactly what you need, it should get you in the ballpark. If you need more help after finding those and looking at them, ask away!

Happy Splunking!
Rich

0 Karma

twinspop
Influencer

Try this, adding indexers as required:

(host=your_indexer OR host=your_indexer2 OR host=your_indexer3) index=_internal metrics name=index_thruput sourcetype=splunkd | 
    timechart span=1m sum(eval(kb/1024)) as "MB/min"

If you run for more than a few hours, forcing a different span in the timechart, you'll need to do some extra math. This one has 5 minute buckets and therefore requires dividing the value by 5.

(host=your_indexer OR host=your_indexer2 OR host=your_indexer3) index=_internal metrics name=index_thruput sourcetype=splunkd | 
    timechart span=5m sum(eval(kb/1024/5)) as "MB/min"

This one by hour

(host=your_indexer OR host=your_indexer2 OR host=your_indexer3) index=_internal metrics name=index_thruput sourcetype=splunkd | 
    timechart span=1h sum(eval(kb/1024/60)) as "MB/min"

danielsofoulis
Path Finder

Thank up for your help.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...