Monitoring Splunk

Splunk index roll-out information?

rsathish47
Contributor

Hi All,

I have to build dashboard for Splunk index roll-out information From hot to Warm Warm to Cold and Cold to Frozen ? Please let me which internal index and source-type I need to use to get this detail?

Tags (2)
0 Karma
1 Solution

inventsekar
SplunkTrust
SplunkTrust

you can use this query and get details...you can fine-tune your requirement and create the dashboard -

index=_internal sourcetype=splunkd bucketmover

10/6/16 7:44:06.942 AM 10-06-2016 07:44:06.942 -0400 INFO BucketMover - idx=windows Moving bucket='db_1475034240_1474948377_1671' because maximum number of warm databases exceeded, starting warm_to_cold: from='/opt/splunk_hot/window

10/6/16 8:00:39.724 AM 10-06-2016 08:00:39.724 -0400 INFO BucketMover - AsyncFreezer freeze succeeded for bkt='/opt/splunk_cold/_internaldb/colddb/db_1473163226_1473156927_1613'

10/6/16 8:02:00.852 AM 10-06-2016 08:02:00.852 -0400 INFO BucketMover - will attempt to freeze: candidate='/opt/splunk_cold/_internaldb/colddb/db_1473163314_1446206601_2368' because frozenTimePeriodInSecs=2592000 is exceeded by the difference between now=1475755320 and latest=1473163314

 |dbinspect index=main|convert timeformat=""%m/%d/%Y:%H:%M:%S"" mktime(earliestTime) as earliestTime|convert timeformat=""%m/%d/%Y:%H:%M:%S"" mktime(latestTime) as latestTime|stats min(earliestTime) as earliestTime max(latestTime) as latestTime sum(sizeOnDiskMB) as sizeOnDiskMB dc(path) as NumberOfBuckets by state|eval diff_seconds=(latestTime-earliestTime)/3600|eval earliestTime=strftime(earliestTime,"%m/%d/%Y:%H:%M:%S")|eval latestTime=strftime(latestTime,"%m/%d/%Y:%H:%M:%S")

The result set for this query is something like this:

 state    earliestTime        latestTime        sizeOnDiskMB    NumberOfBuckets
 hot    09/15/2006:18:47:20    03/24/2012:12:00:00    10043.741711    10
 warm    02/19/2012:00:00:01    03/21/2012:03:59:00    92646.695278    20
 cold    01/03/2006:16:35:20    02/27/2012:17:00:00    204971.245710    586

View solution in original post

inventsekar
SplunkTrust
SplunkTrust

you can use this query and get details...you can fine-tune your requirement and create the dashboard -

index=_internal sourcetype=splunkd bucketmover

10/6/16 7:44:06.942 AM 10-06-2016 07:44:06.942 -0400 INFO BucketMover - idx=windows Moving bucket='db_1475034240_1474948377_1671' because maximum number of warm databases exceeded, starting warm_to_cold: from='/opt/splunk_hot/window

10/6/16 8:00:39.724 AM 10-06-2016 08:00:39.724 -0400 INFO BucketMover - AsyncFreezer freeze succeeded for bkt='/opt/splunk_cold/_internaldb/colddb/db_1473163226_1473156927_1613'

10/6/16 8:02:00.852 AM 10-06-2016 08:02:00.852 -0400 INFO BucketMover - will attempt to freeze: candidate='/opt/splunk_cold/_internaldb/colddb/db_1473163314_1446206601_2368' because frozenTimePeriodInSecs=2592000 is exceeded by the difference between now=1475755320 and latest=1473163314

 |dbinspect index=main|convert timeformat=""%m/%d/%Y:%H:%M:%S"" mktime(earliestTime) as earliestTime|convert timeformat=""%m/%d/%Y:%H:%M:%S"" mktime(latestTime) as latestTime|stats min(earliestTime) as earliestTime max(latestTime) as latestTime sum(sizeOnDiskMB) as sizeOnDiskMB dc(path) as NumberOfBuckets by state|eval diff_seconds=(latestTime-earliestTime)/3600|eval earliestTime=strftime(earliestTime,"%m/%d/%Y:%H:%M:%S")|eval latestTime=strftime(latestTime,"%m/%d/%Y:%H:%M:%S")

The result set for this query is something like this:

 state    earliestTime        latestTime        sizeOnDiskMB    NumberOfBuckets
 hot    09/15/2006:18:47:20    03/24/2012:12:00:00    10043.741711    10
 warm    02/19/2012:00:00:01    03/21/2012:03:59:00    92646.695278    20
 cold    01/03/2006:16:35:20    02/27/2012:17:00:00    204971.245710    586

rsathish47
Contributor

Thank You

0 Karma

richgalloway
SplunkTrust
SplunkTrust

What exactly do you want to display on the dashboard?

---
If this reply helps you, Karma would be appreciated.
0 Karma

rsathish47
Contributor

I have to show the no.of bytes and time it got transferred

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In September, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...