Splunk IT Service Intelligence

ITSI Entity Filtering includes entities from other services

Path Finder

I have a KPI base search which is:

  • split by entity,
  • has an entity split field which is an eval of the host + the service name,
  • is set to filter by entities in service
  • and has the entity filter field set as the same entity split field
  • the entity alias filtering field is blank.

I have 2 or 3 services with KPIs based on this base search. each has Matched entities which are a subset of the entities returned by the base search

Service A correctly filters to its Matched Entities
Service B filters to its matched entities, PLUS the matched entities from Service A!

if I delete the KPI from Service A and delete and recreate the KPI on Service B, Service B will now correctly show only its matched entities

If I re-add the KPI to Service A, Service B will again show both its Matched Entities PLUS those from Service A

Does anyone have any idea what might be going wrong and how to fix it?

PS. The PerEntity Threshold screen on the Configure Service page for each service correctly shows only the matched entities for that service in its preview window

EDIT - Partial solution

So it seems the problem is caused by Service A's entity match rules.

All services had a simple Entity Title matches *servicename*

Service A has an extra AND does not match *_excludethis

When I change the matching rules on Service A so that the extra rule is removed, the other services Dont pick up on Service A's Matched Entities

Note, the other services were NOT picking up on entities that simply matched the extra rule, they picked up Service A's entities

This behaviour seems like a bug to me, surely one service's setup shouldn't affect other services? I still want to know the underlying cause.

0 Karma

Splunk Employee
Splunk Employee

This issue was fixed in version 4.0.4 (https://docs.splunk.com/Documentation/ITSI/4.0.4/ReleaseNotes/Fixedissues#Service_Definition).

Note that once you upgrade to a version with the fix, you need to run the kvstore_to_json mode 4 option. See the version-specific upgrade notes for 4.0.4 here: https://docs.splunk.com/Documentation/ITSI/4.4.0/Install/VersionNotes

As a workaround for pre-4.0.4 versions, you can open the service and go to the Entities tab. If you see an entity with a "does not match" rule, remove the rule and find a way to match to an entity differently that does not use the “does not match” rule.

Path Finder

is this issue ITSI-1868? I think we are on 4.0.2

0 Karma
Get Updates on the Splunk Community!

Synthetic Monitoring: Not your Grandma’s Polyester! Tech Talk: DevOps Edition

Register today and join TekStream on Tuesday, February 28 at 11am PT/2pm ET for a demonstration of Splunk ...

Instrumenting Java Websocket Messaging

Instrumenting Java Websocket MessagingThis article is a code-based discussion of passing OpenTelemetry trace ...

Announcing General Availability of Splunk Incident Intelligence!

Digital transformation is real! Across industries, companies big and small are going through rapid digital ...