topic Re: Why are metrics producing the wrong value for host field? in Getting Data In

Why are metrics producing the wrong value for host field?

diogoribeirosky — Wed, 29 May 2019 16:25:30 GMT

Hello,

I deployed a Splunk APP that listens to TCP in port 7001, protocol starts with the following code:

inputs.conf:

tcp:7001]
index = metrics
sourcetype = statsd

And I have in \splunk-forwarder\etc\system\local\inputs.conf the following definition:

[default]
host = myhostname
index = prod

The index value is correctly overwritten. 🙂

The problem happens when I am searching for mstats, the value for the host is 127.0.0.1,
which is the IP where I am sending my metrics with powershell.
I can't understand why the value host = myhostname is overwritten, I have no definition of this field in the Splunk APP deployed.

Can someone help me?

Re: Why are metrics producing the wrong value for host field?

richgalloway — Wed, 29 May 2019 19:47:17 GMT

Btool may help. Run splunk btool --debug inputs list | Select-string -Pattern "host" <filepattern> -Simplematch in powershell to see where the host is getting set.

Re: Why are metrics producing the wrong value for host field?

diogoribeirosky — Wed, 29 May 2019 20:40:57 GMT

After running, I receive this message
\splunk-forwarder\etc\system\local\inputs.conf host = myhostname
and in search app splunk host is still 127.0.0.1

Re: Why are metrics producing the wrong value for host field?

diogoribeirosky — Wed, 29 May 2019 20:54:12 GMT

[tcp]
_rcvbuf = 1572864
acceptFrom = *
connection_host = dns
evt_dc_name =
evt_dns_name =
evt_resolve_ad_obj = 0
host = myhostname
index = prod
[tcp:7051]
_rcvbuf = 1572864
acceptFrom = 127.0.0.1
evt_dc_name =
evt_dns_name =
evt_resolve_ad_obj = 0
host = myhostname
index = metrics
listenOnIPv6 = no
sourcetype = statsd

When running, ./splunk cmd btool inputs list command

Re: Why are metrics producing the wrong value for host field?

aromanauskas — Thu, 30 May 2019 03:48:12 GMT

Since this is an input to a metric index the host is the Splunk Server doing the conversion to metrics. There instead will be a different recalculated field for the original host. This is similar to the host of a summary index. Simplest option is to use Hostname instead as the dimension.

Re: Why are metrics producing the wrong value for host field?

diogoribeirosky — Thu, 30 May 2019 08:23:10 GMT

Thank you aromanauskas for your answer
But if the host is the Splunk Server doing the conversion to metrics, How can the host be 127.0.0.1? which is the value I set $server in my TCPConnection.

Re: Why are metrics producing the wrong value for host field?

aromanauskas — Thu, 30 May 2019 13:55:58 GMT

That is because the splunk server is using the rest API to communicate to itself on 127.0.0.1 . If you were using mcollect to write to the metric index, then it would replace the host with the name of the search head running the report. To keep confusion to a minimum I find it best to stay away from using host AND sourcetype for anything in metric indexes.

Re: Why are metrics producing the wrong value for host field?

evania — Thu, 06 Jun 2019 18:04:41 GMT

Hi @diogoribeirosky ,

Did you have a chance to check out any answers? If any work, please resolve this post by approving it! If your problem is still not solved, keep us updated so that someone else can help you.

Thanks for posting!

Re: Why are metrics producing the wrong value for host field?

diogoribeirosky — Fri, 07 Jun 2019 08:39:42 GMT

Hi @evzhang_splunk ,
Yes. I checked all the answers but unfortunately the problem is not solved. I think this might be a Splunk bug or malfunctioning....I used the statsd dimensions to pass the hostname info, which is redundant if the host is set correctly.