Knowledge Management
Highlighted

Query about datamodel acceleration and how data is stored

Super Champion

I was going through the documents on Datamodel Acceleration. Can you please help me in confirming if my understanding below is correct?

  1. An Addon (TA) does the Data interpretation, classification, enrichment and normalisation. Most key value pairs are extracted during search-time.
  2. A data model then abstracts/maps multiple such datasets (and brings hierarchy) during search-time . So datamodel as such does not speed-up searches, but just abstracts to make it easy for end-user.
  3. During acceleration of datamodel, key value pairs (which were search-time in Step2) are extracted and stored as indexed key-value pairs ?
  4. The fields stored in .tsidx files, are independent of TA's and any search on it, should NOT require effort on re-extracting data from _raw?
0 Karma
Highlighted

Re: Query about datamodel acceleration and how data is stored

SplunkTrust
SplunkTrust
  1. Yup.
  2. Yeah.
  3. Fields in an accelerated data model still are search-time for most purposes. For example, index-time fields cannot be added retroactively while you can add a field to a data model and use that without re-indexing... though there will be an acceleration rebuild. From the performance point of view they behave like indexed fields, and are available through tstats.
  4. I'm not quite sure where you're going for here, data model accelerations are dependent on TAs - see question one. For already-accelerated data there's no need to descend into _raw, yes. For very recent data before acceleration happened or for very old data beyond the acceleration window the datamodel-backed search will by default descend into _raw to complete the results.

View solution in original post

Highlighted

Re: Query about datamodel acceleration and how data is stored

Super Champion

@martin_mueller. Thank you

Regarding (4) : I meant to ask once the data is in high perfomance store, the extraction is independent of TAs? or during a search, will it still have to go through all TA regex etc?

0 Karma
Highlighted

Re: Query about datamodel acceleration and how data is stored

SplunkTrust
SplunkTrust

It's not going to apply the regexes to the raw data, the values will already have been extracted while the acceleration searches run in the background.

0 Karma
Highlighted

Re: Query about datamodel acceleration and how data is stored

Super Champion

thanks Martin for your help. Accepted your answer.

0 Karma