Where does data model store the accelerated data?

[_pravin]

Hi Community,

I have a use case where the client needs data to be stored over an extended period of time.

That data powers the dashboard that uses datamodels to generate the panels. Since the client wants data to be available for at least 6 months, the idea was to create an index that has hot/warm buckets in SSD and cold buckets in slower storage.

I have two different issues here:

1. I have implemented this setup in our test environment with mixed storage for hot and cold buckets. Is there a way for me to check where my data is being stored?
2. Since my dashboards are all powered by datamodels, I have a question regarding the storage location and method of accelerated data. If the data is accelerated, does the data model summary folder store the complete accelerated data or will it have some pointers that point to the location where the data is actually present?

The main problem here is that if we have mixed storage of SSD and HDD, and since all the dashboards are powered by datamodels how much will this affect the performance of Splunk? Will the time to load the dashboard be affected by such a storage model?

Regards,

Pravin

[richgalloway]

1. Examine the _bkt field of an event to find out which bucket it's in then correlate that to results from the dbinspect command.  The state field will say if the bucket is hot, warm, or cold.

| dbinspect index=foo [ search index=foo | eval bucketId=_bkt | dedup bucketId | fields bucketId | format ] | fields bucketId state

2. Data model data is stored with the index from which it was extracted.  The location can be specified with the tstatsHomePath setting in indexes.conf.

[_pravin · Accepted solution]

Hi @richgalloway ,

Thanks for your response.

• The below query works, but the problem is that the time range picker doesn't work for the search.

• I understand that we can use tstatsHomePath to specify where the data model summary is stored but what I am trying to understand is that does this have the complete accelerated data. Will this accelerated data have some pointer that references the original data or is this acting independently of the original data?

Regards,

Pravin

[richgalloway]

Accelerated data is complete.  There are no references to the raw data.