Knowledge Management

Adding index to accelerated CIM datamodel

PickleRick
SplunkTrust
SplunkTrust

I have an accelerated CIM data model.

The indexes used to populate the datamodel (and accelerated summaries) are defined by a macro (a typical CIM approach - cim_Email_indexes, cim_Network_Traffic_indexes and so on).

What will happen if I change this macro to include additional index?

Will splunk:

a) Just add data from new index to next summary rebuild starting from the last summarized timestamp?

b) Add data from new index looking back up to Summary Range  the during next rebuild?

c) Rebuild whole summaries back up to Summary Range?

 

Labels (2)
0 Karma
1 Solution

gcusello
SplunkTrust
SplunkTrust

Hi @PickleRick,

if you modify the macro containing the indexes for an accelerated Data Model, there are two different choices:

  • if you don't rebuild the DataModel, Splunk will start to add logs from that index when you  save the macro and old events aren't added to the Datamodel, only the new ones,
  • if you rebuild the DataModel, Splunk will add to the DataModel all the events in all indexes contained in the macro until the retention period (e.g. Network Traffic 1month, Authentication 1 year, and so on).

Ciao.

Giuseppe

View solution in original post

gcusello
SplunkTrust
SplunkTrust

Hi @PickleRick,

if you modify the macro containing the indexes for an accelerated Data Model, there are two different choices:

  • if you don't rebuild the DataModel, Splunk will start to add logs from that index when you  save the macro and old events aren't added to the Datamodel, only the new ones,
  • if you rebuild the DataModel, Splunk will add to the DataModel all the events in all indexes contained in the macro until the retention period (e.g. Network Traffic 1month, Authentication 1 year, and so on).

Ciao.

Giuseppe

PickleRick
SplunkTrust
SplunkTrust

I know that if I wanted to edit the datamodel itself, I'd of course have to disable acceleration first so that re-enabling acceleration would trigger complete rebuild of the summaries.

So I understand that if I simply change the macro, I do not trigger a rebuild. That's good news 🙂

I do _not_ want to rebuild the datamodel "backwards" (I have way too many terabytes of network data and don't want to kill my indexers XD). So I just add the index to the macro and the summaries will be built on new index set from now on, right?

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @PickleRick,

no, you have to disable acceleration to modify a DataModel, but when you restart acceleration, the updates will be applied only on new data, otherwise, to apply on all data,you have to rebuild the DataModel.

Anyway, modifying the macro you don't need to stop acceleration.

Ciao.

Giuseppe

PickleRick
SplunkTrust
SplunkTrust

Yes, I know that I don't need to disable acceleration to edit macro. That's why it's a clever little trick 🙂

I just thought that disabling acceleration and re-enabling it causes the whole summary to be rebuilt.

Anyway. Long story short, I assume that I can safely add the index to the macro and it will not cause a huge rebuild of a whole backlog of a month or so.. That's most important for me. 🙂

Thanks for help.

0 Karma
Get Updates on the Splunk Community!

March Community Office Hours Security Series Uncovered!

Hello Splunk Community! In March, Splunk Community Office Hours spotlighted our fabulous Splunk Threat ...

Stay Connected: Your Guide to April Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars in April. This post ...

Want to Reduce Costs, Mitigate Risk, Improve Performance, or Increase Efficiencies? ...

Splunk Lantern is Splunk’s customer success center that provides advice from Splunk experts on valuable data ...