Splunk IT Service Intelligence

ITSI Import Objects - itsi_entity_name_normalizer: Why does it fail after entity volume gets too high?

travisakins
Engager

Documenting a solution built while working with ITSI 4.13 and CP for Monitoring and Alerting 1.5 on a Splunk 8.2.4 platform. 

The content pack for monitoring and alerting creates the itsi_entity_name_normalizer import job to ensure that every entity will get an alias called entity_name.  They use this alias for other searches in the logic created from the content pack so it is important that it runs as intended.  

When the number of entities grows the likelihood of this job failing increases.  Attempts to run the out-of-the-box spl on a larger entity pool will highlight a 414 error that the URI being too long since it looks at the entire pool every run.

To fix this issue we need to change the logic in the 'ITSI Import Objects - itsi_entity_name_normalizer' job so it will not run against entities which already have the entity_name alias.

Original SPL:

| inputlookup itsi_entities | eval entity_name=title

Updated SPL:

| inputlookup itsi_entities where NOT _itsi_identifier_lookups=entity_name*
| search retirable!=1
| eval entity_name=title
| eval entity_title=title
| head 5000

While the eval for entity_title is redundant it is useful when using the search for ad-hoc entity import cases as the UI will restrict us from mapping title to title.  Doesn't hurt to have it.  If others disagree please update as needed.  

Additionally, since we need to control the volume in each batch the head function gives us that flexibility.  We are protected in case we get a large influx of new entities.

This assumes you do not need the entity_name field continually overwritten every cycle.  I could not find a reason why it matters to be updated after reviewing the other knowledge items the content pack creates. 

Lastly, with the introduction of Entity Management Policies in ITIS 4.x we added an extra filter for entities without the retirable flag set.  If an entity is flagged to be retired we concluded it should be excluded from this job.  Likelihood an entity would qualify would be rare as that function represents the end of an entity lifecycle but no harm in having the extra check. 

Labels (2)
Tags (1)

yannK
Splunk Employee
Splunk Employee

Great wokaround for large number of entities.

0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...