Splunk IT Service Intelligence
Highlighted

ITSI Entity Filtering includes entities from other services

Path Finder

I have a KPI base search which is:

  • split by entity,
  • has an entity split field which is an eval of the host + the service name,
  • is set to filter by entities in service
  • and has the entity filter field set as the same entity split field
  • the entity alias filtering field is blank.

I have 2 or 3 services with KPIs based on this base search. each has Matched entities which are a subset of the entities returned by the base search

Service A correctly filters to its Matched Entities
Service B filters to its matched entities, PLUS the matched entities from Service A!

if I delete the KPI from Service A and delete and recreate the KPI on Service B, Service B will now correctly show only its matched entities

If I re-add the KPI to Service A, Service B will again show both its Matched Entities PLUS those from Service A

Does anyone have any idea what might be going wrong and how to fix it?

PS. The PerEntity Threshold screen on the Configure Service page for each service correctly shows only the matched entities for that service in its preview window

EDIT - Partial solution

So it seems the problem is caused by Service A's entity match rules.

All services had a simple Entity Title matches *servicename*

Service A has an extra AND does not match *_excludethis

When I change the matching rules on Service A so that the extra rule is removed, the other services Dont pick up on Service A's Matched Entities

Note, the other services were NOT picking up on entities that simply matched the extra rule, they picked up Service A's entities

This behaviour seems like a bug to me, surely one service's setup shouldn't affect other services? I still want to know the underlying cause.

0 Karma
Highlighted

Re: ITSI Entity Filtering includes entities from other services

Path Finder

is this issue ITSI-1868? I think we are on 4.0.2

0 Karma
Highlighted

Re: ITSI Entity Filtering includes entities from other services

Splunk Employee
Splunk Employee

This issue was fixed in version 4.0.4 (https://docs.splunk.com/Documentation/ITSI/4.0.4/ReleaseNotes/Fixedissues#Service_Definition).

Note that once you upgrade to a version with the fix, you need to run the kvstoretojson mode 4 option. See the version-specific upgrade notes for 4.0.4 here: https://docs.splunk.com/Documentation/ITSI/4.4.0/Install/VersionNotes

As a workaround for pre-4.0.4 versions, you can open the service and go to the Entities tab. If you see an entity with a "does not match" rule, remove the rule and find a way to match to an entity differently that does not use the “does not match” rule.