Monitoring Splunk

Splunk index roll-out information?

rsathish47
Contributor

Hi All,

I have to build dashboard for Splunk index roll-out information From hot to Warm Warm to Cold and Cold to Frozen ? Please let me which internal index and source-type I need to use to get this detail?

Tags (2)
0 Karma
1 Solution

inventsekar
SplunkTrust
SplunkTrust

you can use this query and get details...you can fine-tune your requirement and create the dashboard -

index=_internal sourcetype=splunkd bucketmover

10/6/16 7:44:06.942 AM 10-06-2016 07:44:06.942 -0400 INFO BucketMover - idx=windows Moving bucket='db_1475034240_1474948377_1671' because maximum number of warm databases exceeded, starting warm_to_cold: from='/opt/splunk_hot/window

10/6/16 8:00:39.724 AM 10-06-2016 08:00:39.724 -0400 INFO BucketMover - AsyncFreezer freeze succeeded for bkt='/opt/splunk_cold/_internaldb/colddb/db_1473163226_1473156927_1613'

10/6/16 8:02:00.852 AM 10-06-2016 08:02:00.852 -0400 INFO BucketMover - will attempt to freeze: candidate='/opt/splunk_cold/_internaldb/colddb/db_1473163314_1446206601_2368' because frozenTimePeriodInSecs=2592000 is exceeded by the difference between now=1475755320 and latest=1473163314

 |dbinspect index=main|convert timeformat=""%m/%d/%Y:%H:%M:%S"" mktime(earliestTime) as earliestTime|convert timeformat=""%m/%d/%Y:%H:%M:%S"" mktime(latestTime) as latestTime|stats min(earliestTime) as earliestTime max(latestTime) as latestTime sum(sizeOnDiskMB) as sizeOnDiskMB dc(path) as NumberOfBuckets by state|eval diff_seconds=(latestTime-earliestTime)/3600|eval earliestTime=strftime(earliestTime,"%m/%d/%Y:%H:%M:%S")|eval latestTime=strftime(latestTime,"%m/%d/%Y:%H:%M:%S")

The result set for this query is something like this:

 state    earliestTime        latestTime        sizeOnDiskMB    NumberOfBuckets
 hot    09/15/2006:18:47:20    03/24/2012:12:00:00    10043.741711    10
 warm    02/19/2012:00:00:01    03/21/2012:03:59:00    92646.695278    20
 cold    01/03/2006:16:35:20    02/27/2012:17:00:00    204971.245710    586
thanks and best regards,
Sekar

PS - If this or any post helped you in any way, pls consider upvoting, thanks for reading !

View solution in original post

inventsekar
SplunkTrust
SplunkTrust

you can use this query and get details...you can fine-tune your requirement and create the dashboard -

index=_internal sourcetype=splunkd bucketmover

10/6/16 7:44:06.942 AM 10-06-2016 07:44:06.942 -0400 INFO BucketMover - idx=windows Moving bucket='db_1475034240_1474948377_1671' because maximum number of warm databases exceeded, starting warm_to_cold: from='/opt/splunk_hot/window

10/6/16 8:00:39.724 AM 10-06-2016 08:00:39.724 -0400 INFO BucketMover - AsyncFreezer freeze succeeded for bkt='/opt/splunk_cold/_internaldb/colddb/db_1473163226_1473156927_1613'

10/6/16 8:02:00.852 AM 10-06-2016 08:02:00.852 -0400 INFO BucketMover - will attempt to freeze: candidate='/opt/splunk_cold/_internaldb/colddb/db_1473163314_1446206601_2368' because frozenTimePeriodInSecs=2592000 is exceeded by the difference between now=1475755320 and latest=1473163314

 |dbinspect index=main|convert timeformat=""%m/%d/%Y:%H:%M:%S"" mktime(earliestTime) as earliestTime|convert timeformat=""%m/%d/%Y:%H:%M:%S"" mktime(latestTime) as latestTime|stats min(earliestTime) as earliestTime max(latestTime) as latestTime sum(sizeOnDiskMB) as sizeOnDiskMB dc(path) as NumberOfBuckets by state|eval diff_seconds=(latestTime-earliestTime)/3600|eval earliestTime=strftime(earliestTime,"%m/%d/%Y:%H:%M:%S")|eval latestTime=strftime(latestTime,"%m/%d/%Y:%H:%M:%S")

The result set for this query is something like this:

 state    earliestTime        latestTime        sizeOnDiskMB    NumberOfBuckets
 hot    09/15/2006:18:47:20    03/24/2012:12:00:00    10043.741711    10
 warm    02/19/2012:00:00:01    03/21/2012:03:59:00    92646.695278    20
 cold    01/03/2006:16:35:20    02/27/2012:17:00:00    204971.245710    586
thanks and best regards,
Sekar

PS - If this or any post helped you in any way, pls consider upvoting, thanks for reading !

rsathish47
Contributor

Thank You

0 Karma

richgalloway
SplunkTrust
SplunkTrust

What exactly do you want to display on the dashboard?

---
If this reply helps you, Karma would be appreciated.
0 Karma

rsathish47
Contributor

I have to show the no.of bytes and time it got transferred

0 Karma
Get Updates on the Splunk Community!

Splunk + ThousandEyes: Correlate frontend, app, and network data to troubleshoot ...

 Are you tired of troubleshooting delays caused by siloed frontend, application, and network data? We've got a ...

Splunk Observability for AI

Don’t miss out on an exciting Tech Talk on Splunk Observability for AI!Discover how Splunk’s agentic AI ...

🔐 Trust at Every Hop: How mTLS in Splunk Enterprise 10.0 Makes Security Simpler

From Idea to Implementation: Why Splunk Built mTLS into Splunk Enterprise 10.0  mTLS wasn’t just a checkbox ...