Knowledge Management

Adding index to accelerated CIM datamodel

PickleRick
Ultra Champion

I have an accelerated CIM data model.

The indexes used to populate the datamodel (and accelerated summaries) are defined by a macro (a typical CIM approach - cim_Email_indexes, cim_Network_Traffic_indexes and so on).

What will happen if I change this macro to include additional index?

Will splunk:

a) Just add data from new index to next summary rebuild starting from the last summarized timestamp?

b) Add data from new index looking back up to Summary Range  the during next rebuild?

c) Rebuild whole summaries back up to Summary Range?

 

Labels (2)
0 Karma
1 Solution

gcusello
Legend

Hi @PickleRick,

if you modify the macro containing the indexes for an accelerated Data Model, there are two different choices:

  • if you don't rebuild the DataModel, Splunk will start to add logs from that index when you  save the macro and old events aren't added to the Datamodel, only the new ones,
  • if you rebuild the DataModel, Splunk will add to the DataModel all the events in all indexes contained in the macro until the retention period (e.g. Network Traffic 1month, Authentication 1 year, and so on).

Ciao.

Giuseppe

View solution in original post

gcusello
Legend

Hi @PickleRick,

if you modify the macro containing the indexes for an accelerated Data Model, there are two different choices:

  • if you don't rebuild the DataModel, Splunk will start to add logs from that index when you  save the macro and old events aren't added to the Datamodel, only the new ones,
  • if you rebuild the DataModel, Splunk will add to the DataModel all the events in all indexes contained in the macro until the retention period (e.g. Network Traffic 1month, Authentication 1 year, and so on).

Ciao.

Giuseppe

PickleRick
Ultra Champion

I know that if I wanted to edit the datamodel itself, I'd of course have to disable acceleration first so that re-enabling acceleration would trigger complete rebuild of the summaries.

So I understand that if I simply change the macro, I do not trigger a rebuild. That's good news 🙂

I do _not_ want to rebuild the datamodel "backwards" (I have way too many terabytes of network data and don't want to kill my indexers XD). So I just add the index to the macro and the summaries will be built on new index set from now on, right?

0 Karma

gcusello
Legend

Hi @PickleRick,

no, you have to disable acceleration to modify a DataModel, but when you restart acceleration, the updates will be applied only on new data, otherwise, to apply on all data,you have to rebuild the DataModel.

Anyway, modifying the macro you don't need to stop acceleration.

Ciao.

Giuseppe

PickleRick
Ultra Champion

Yes, I know that I don't need to disable acceleration to edit macro. That's why it's a clever little trick 🙂

I just thought that disabling acceleration and re-enabling it causes the whole summary to be rebuilt.

Anyway. Long story short, I assume that I can safely add the index to the macro and it will not cause a huge rebuild of a whole backlog of a month or so.. That's most important for me. 🙂

Thanks for help.

0 Karma
Get Updates on the Splunk Community!

Welcome to the Future of Data Search & Exploration

You have more data coming at you than ever before. Over the next five years, the total amount of digital data ...

What’s new on Splunk Lantern in August

This month’s Splunk Lantern update gives you the low-down on all of the articles we’ve published over the past ...

This Week's Community Digest - Splunk Community Happenings [8.3.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...