Splunk ITSI

ITSI Service Template too big to sync!

ewan000
Path Finder

I created a KPI base search with 197 metrics

I made a service template with each of these metrics as a KPI

When I try to sync the service template, or create a new service using the template I get the error:

"Error while updating services linked to service_template="241ed73e-000c-4635-a9b0-3ea04577e5dd". Exception="'Object you are trying to save is too large (55209132 bytes). KV store only supports documents within 16MB sizes.'""

Checking the documentation on base searches I find:

"Most of the KPI base searches delivered with ITSI are configured to run every minute. Based on testing on a system with 32 cores and 16 GB of memory, a single KPI base search can support up to 5,000 KPIs with 15 entities matched by service entity rules reasonably well."

I'm well below the 5000 KPIs with 15 entities.

If I retrieve the Service Template via the API it is rather large at 6Mb mainly due to the long base search being repeated in each KPI. But still nothing like the 55Mb reported in the error

What limit am i running into here and how can i get around it?

Labels (1)
0 Karma

ewan000
Path Finder

I moved the base search's search to a saved search and made the KPI base search run the saved search, massively reducing the length of the search string in the base KPI.

After that the Service Template had no trouble syncing, so it does seem to be purely related to the length of the search string.

0 Karma

ewan000
Path Finder

deleting a few KPIs drastically starts to reduce the size. I got rid of 10 or so and it went down to 44Mb

0 Karma
Get Updates on the Splunk Community!

Earn a $35 Gift Card for Answering our Splunk Admins & App Developer Survey

Survey for Splunk Admins and App Developers is open now! | Earn a $35 gift card!      Hello there,  Splunk ...

Continuing Innovation & New Integrations Unlock Full Stack Observability For Your ...

You’ve probably heard the latest about AppDynamics joining the Splunk Observability portfolio, deepening our ...

Monitoring Amazon Elastic Kubernetes Service (EKS)

As we’ve seen, integrating Kubernetes environments with Splunk Observability Cloud is a quick and easy way to ...