Adding index to accelerated CIM datamodel

[PickleRick]

I have an accelerated CIM data model.

The indexes used to populate the datamodel (and accelerated summaries) are defined by a macro (a typical CIM approach - cim_Email_indexes, cim_Network_Traffic_indexes and so on).

What will happen if I change this macro to include additional index?

Will splunk:

a) Just add data from new index to next summary rebuild starting from the last summarized timestamp?

b) Add data from new index looking back up to Summary Range  the during next rebuild?

c) Rebuild whole summaries back up to Summary Range?

[gcusello · Accepted solution]

Hi @PickleRick,

if you modify the macro containing the indexes for an accelerated Data Model, there are two different choices:

• if you don't rebuild the DataModel, Splunk will start to add logs from that index when you  save the macro and old events aren't added to the Datamodel, only the new ones,
• if you rebuild the DataModel, Splunk will add to the DataModel all the events in all indexes contained in the macro until the retention period (e.g. Network Traffic 1month, Authentication 1 year, and so on).

Ciao.

Giuseppe

[PickleRick]

I know that if I wanted to edit the datamodel itself, I'd of course have to disable acceleration first so that re-enabling acceleration would trigger complete rebuild of the summaries.

So I understand that if I simply change the macro, I do not trigger a rebuild. That's good news 🙂

I do _not_ want to rebuild the datamodel "backwards" (I have way too many terabytes of network data and don't want to kill my indexers XD). So I just add the index to the macro and the summaries will be built on new index set from now on, right?

[gcusello]

Hi @PickleRick,

no, you have to disable acceleration to modify a DataModel, but when you restart acceleration, the updates will be applied only on new data, otherwise, to apply on all data,you have to rebuild the DataModel.

Anyway, modifying the macro you don't need to stop acceleration.

Ciao.

Giuseppe

[PickleRick]

Yes, I know that I don't need to disable acceleration to edit macro. That's why it's a clever little trick 🙂

I just thought that disabling acceleration and re-enabling it causes the whole summary to be rebuilt.

Anyway. Long story short, I assume that I can safely add the index to the macro and it will not cause a huge rebuild of a whole backlog of a month or so.. That's most important for me. 🙂

Thanks for help.