All Apps and Add-ons

Splunk App for Infrastructure (SAI) missing entity

New Member

We have deployed SAI and the corresponding UF ps scripts to over 30 Windows systems without issue.... However we have one Windows server that is missing from the SAI Entities dashboard view. I confirmed we are collecting metrics from the system via | mstats and | mcollect on our SH and see the hostname within the content of the metric index and metrics counts. I also confirmed we are getting log/windows data from the system. I have attempted to reinstall UF and corresponding SAI app to no avail. I attempted modifying the KV store for entities and add this system in manually but that also did not work. I confirmed the instances.cfg on the client windows server differs from all other systems. Im out of ideas has anyone else run into this or can recommend something to try??

0 Karma
1 Solution

Splunk Employee
Splunk Employee

First try this search:
| mcatalog values("entitytype") as "entitytype" values("os") as "os" WHERE metricname=processor.* AND index=emmetrics BY "host"

Do you see your missing server/entity ? If no, try this:
| mstats avg(value) WHERE index=emmetrics AND metricname=* AND host="missingservername" by metricname

Replace missingservername in this search.

View solution in original post

0 Karma

Splunk Employee
Splunk Employee

First try this search:
| mcatalog values("entitytype") as "entitytype" values("os") as "os" WHERE metricname=processor.* AND index=emmetrics BY "host"

Do you see your missing server/entity ? If no, try this:
| mstats avg(value) WHERE index=emmetrics AND metricname=* AND host="missingservername" by metricname

Replace missingservername in this search.

View solution in original post

0 Karma

New Member

|mcatalog search contained the missing server - reported entity_type and OS are identical to other servers reporting that do show up in the dashboard

| mstats search for just the missing server returned 25 metric_name results base on 300K events over a 24 hour span

0 Karma

Splunk Employee
Splunk Employee

Can you check splunk log files here for errors:
$SPLUNKHOME/var/log/splunk/ementity_manager.log (check other log files as well if you see any errors)

ALso try to see if the missing host is present in the kv store:
| inputlookup em_entities

To run this search, go to splunkweb:8000/app/splunkapp_infrastructure/search
{i.e run search in context of SAI}

0 Karma

New Member

The host in question does not show up in the em_entities kvstore

Looking on the client and splunk system for errors associated with SAI and host found nothing

The only place where the host shows up was mentioned above in the metrics indexes.

0 Karma

New Member

Resolved ! the problem wasnt splunk but there server itself the time was not sync'd 3 min difference between it and Splunk once it was updated the machine appears in the dashboard 🙂

0 Karma

Splunk Employee
Splunk Employee

Great !!!!

0 Karma