Splunk Enterprise

Where does data model store the accelerated data?

_pravin
Communicator

Hi Community,

 

I have a use case where the client needs data to be stored over an extended period of time.

That data powers the dashboard that uses datamodels to generate the panels. Since the client wants data to be available for at least 6 months, the idea was to create an index that has hot/warm buckets in SSD and cold buckets in slower storage.

I have two different issues here:

  1. I have implemented this setup in our test environment with mixed storage for hot and cold buckets. Is there a way for me to check where my data is being stored?
  2. Since my dashboards are all powered by datamodels, I have a question regarding the storage location and method of accelerated data. If the data is accelerated, does the data model summary folder store the complete accelerated data or will it have some pointers that point to the location where the data is actually present?

The main problem here is that if we have mixed storage of SSD and HDD, and since all the dashboards are powered by datamodels how much will this affect the performance of Splunk? Will the time to load the dashboard be affected by such a storage model?

 

Regards,

Pravin

0 Karma
1 Solution

_pravin
Communicator

Hi @richgalloway ,

 

Thanks for your response.

  • The below query works, but the problem is that the time range picker doesn't work for the search.

 

 

  • I understand that we can use tstatsHomePath to specify where the data model summary is stored but what I am trying to understand is that does this have the complete accelerated data. Will this accelerated data have some pointer that references the original data or is this acting independently of the original data?

Regards,

Pravin

View solution in original post

0 Karma

richgalloway
SplunkTrust
SplunkTrust

1. Examine the _bkt field of an event to find out which bucket it's in then correlate that to results from the dbinspect command.  The state field will say if the bucket is hot, warm, or cold.

 

| dbinspect index=foo [ search index=foo | eval bucketId=_bkt | dedup bucketId | fields bucketId | format ] | fields bucketId state

 

2. Data model data is stored with the index from which it was extracted.  The location can be specified with the tstatsHomePath setting in indexes.conf.

---
If this reply helps you, Karma would be appreciated.

_pravin
Communicator

Hi @richgalloway ,

 

Thanks for your response.

  • The below query works, but the problem is that the time range picker doesn't work for the search.

 

 

  • I understand that we can use tstatsHomePath to specify where the data model summary is stored but what I am trying to understand is that does this have the complete accelerated data. Will this accelerated data have some pointer that references the original data or is this acting independently of the original data?

Regards,

Pravin

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Accelerated data is complete.  There are no references to the raw data.

---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...