topic Re: Splunk ES Datamodel Constantly Rebuilding in Reporting

Splunk ES Datamodel Constantly Rebuilding

domenico_perre — Fri, 25 Sep 2015 11:38:56 GMT

Hi All,

I have a problem that has been giving me a bit of grief with ES. Essentially my larger datamodels (Authentication / Network) never seem to get over 10% before restarting at 0%. I am running 2 clustered indexers and a dedicated SH for ES.

I have confirmed that there are no lookup table errors, uninstalled all apps that are not being used, disabled correlation searches that I don't have data for and disabled data models that I don't have data for.

I must admit that I am running VM's with 8 CPUs each idx and ES. When ES is not running, the idxs run at around 3-10% cpu then spike up to 100% when it is started. I know I am technically not supported because the lack of cpu, but would like to know if there is anything else I can look for?

Thx in advance.

Re: Splunk ES Datamodel Constantly Rebuilding

domenico_perre — Fri, 25 Sep 2015 21:05:48 GMT

Even if someone could,tell me where data model operations log to I could look into that.

Re: Splunk ES Datamodel Constantly Rebuilding

domenico_perre — Sun, 27 Sep 2015 19:50:32 GMT

So I will answer my own question.

The issue I had was related to the total time it took to build a Datamodel. As it was attempting to build for over an hour it would rerun and wipe the data model. I edited the datamodels.conf relating to the DMs that were taking the longest with the following

acceleration.max_time = 86400

This is probably due to my underspecced indexers but was an adequate solution imo.

Most DMs are 100% completed and CPU has dropped 🙂