Reporting

Splunk ES Datamodel Constantly Rebuilding

domenico_perre
Path Finder

Hi All,

I have a problem that has been giving me a bit of grief with ES. Essentially my larger datamodels (Authentication / Network) never seem to get over 10% before restarting at 0%. I am running 2 clustered indexers and a dedicated SH for ES.

I have confirmed that there are no lookup table errors, uninstalled all apps that are not being used, disabled correlation searches that I don't have data for and disabled data models that I don't have data for.

I must admit that I am running VM's with 8 CPUs each idx and ES. When ES is not running, the idxs run at around 3-10% cpu then spike up to 100% when it is started. I know I am technically not supported because the lack of cpu, but would like to know if there is anything else I can look for?

Thx in advance.

1 Solution

domenico_perre
Path Finder

So I will answer my own question.

The issue I had was related to the total time it took to build a Datamodel. As it was attempting to build for over an hour it would rerun and wipe the data model. I edited the datamodels.conf relating to the DMs that were taking the longest with the following

acceleration.max_time = 86400

This is probably due to my underspecced indexers but was an adequate solution imo.

Most DMs are 100% completed and CPU has dropped 🙂

View solution in original post

domenico_perre
Path Finder

So I will answer my own question.

The issue I had was related to the total time it took to build a Datamodel. As it was attempting to build for over an hour it would rerun and wipe the data model. I edited the datamodels.conf relating to the DMs that were taking the longest with the following

acceleration.max_time = 86400

This is probably due to my underspecced indexers but was an adequate solution imo.

Most DMs are 100% completed and CPU has dropped 🙂

domenico_perre
Path Finder

Even if someone could,tell me where data model operations log to I could look into that.

0 Karma
Get Updates on the Splunk Community!

How to Monitor Google Kubernetes Engine (GKE)

We’ve looked at how to integrate Kubernetes environments with Splunk Observability Cloud, but what about ...

Index This | How can you make 45 using only 4?

October 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...

Splunk Education Goes to Washington | Splunk GovSummit 2024

If you’re in the Washington, D.C. area, this is your opportunity to take your career and Splunk skills to the ...