I am not sure whether it helps anyone. I was trying to create a KPI using an adhoc search. The host as per this KPI was not entered as Entity rather it can be seen only in the Analyzer view for status. When I checked in the itsi_entities kvstore, I found that these search based KPI result(host) was entered in it. It should not be happening. I remember trying to update some entities, one of its info field had this new KPI result host. I am not sure what caused that to happen. For now I deleted all those entities and created them again, which removed and added those entities from the itsi_entities kvstore. Also I had to recreate the KPI as it was showing the past results. Now everything works fine. Looks like some issue with 4.1.0 version of ITSI or I did something wrong while updating the Entity list.
Interesting. That is the only place it would be set that I know of. I am not sure how you can have an entity linked to a service without with being included in the entities rules. I suspected there was some rule in the service grabbing it.