We have configured the "Splunk App for Infrastructure" per the documentation. We see some data in our metrics index but the host field is the name of my Splunk HEC receiver? Is this expected?
The "Splunk App for Infrastructure" doesn't see any entities at this point, I think it's because of the host field maybe?
Our infra looks like this.. Linux UF with collectd/write_splunk plugin ---> HEC HF ---> Index Cluster
Here is the query that I am using to try to find my data.
| mstats avg(_value) prestats=t WHERE index=mymetrics AND metric_name="*" span=1m by metric_name,host
I can see metrics with the following command.
| mcatalog values(metric_name) WHERE index=mymetrics
What am I missing, I am very new to the metrics indexes.
This is normally an indicator that you did not install the "Splunk Add-on for Infrastructure". You need to install both in order to get your Host Field to populate correctly.
Check out this link to see exactly what needs to be installed where based on your environment. https://docs.splunk.com/Documentation/InfraApp/1.2.2/Install/DistributedDeployment
Try adding "connection_host = dns" or "connection_host = ip" on the HEC input.
This is the HEC config that I have in inputs.conf
disabled = 0
index = em_metrics
indexes = em_metrics
token = 0000
connection_host = dns