Splunk IT Service Intelligence
Highlighted

ITSI Service Template too big to sync!

Path Finder

I created a KPI base search with 197 metrics

I made a service template with each of these metrics as a KPI

When I try to sync the service template, or create a new service using the template I get the error:

"Error while updating services linked to service_template="241ed73e-000c-4635-a9b0-3ea04577e5dd". Exception="'Object you are trying to save is too large (55209132 bytes). KV store only supports documents within 16MB sizes.'""

Checking the documentation on base searches I find:

"Most of the KPI base searches delivered with ITSI are configured to run every minute. Based on testing on a system with 32 cores and 16 GB of memory, a single KPI base search can support up to 5,000 KPIs with 15 entities matched by service entity rules reasonably well."

I'm well below the 5000 KPIs with 15 entities.

If I retrieve the Service Template via the API it is rather large at 6Mb mainly due to the long base search being repeated in each KPI. But still nothing like the 55Mb reported in the error

What limit am i running into here and how can i get around it?

Labels (1)
0 Karma
Highlighted

Re: ITSI Service Template too big to sync!

Path Finder

deleting a few KPIs drastically starts to reduce the size. I got rid of 10 or so and it went down to 44Mb

0 Karma
Highlighted

Re: ITSI Service Template too big to sync!

Path Finder

I moved the base search's search to a saved search and made the KPI base search run the saved search, massively reducing the length of the search string in the base KPI.

After that the Service Template had no trouble syncing, so it does seem to be purely related to the length of the search string.

0 Karma