Why are metrics producing the wrong value for host field?

[diogoribeirosky]

Hello,

I deployed a Splunk APP that listens to TCP in port 7001, protocol starts with the following code:

inputs.conf:

tcp:7001]
index = metrics
sourcetype = statsd

And I have in \splunk-forwarder\etc\system\local\inputs.conf the following definition:

[default]
host = myhostname
index = prod

The index value is correctly overwritten. 🙂

The problem happens when I am searching for mstats, the value for the host is 127.0.0.1,
which is the IP where I am sending my metrics with powershell.
I can't understand why the value host = myhostname is overwritten, I have no definition of this field in the Splunk APP deployed.

Can someone help me?

[evania]

Hi @diogoribeirosky ,

Did you have a chance to check out any answers? If any work, please resolve this post by approving it! If your problem is still not solved, keep us updated so that someone else can help you.

Thanks for posting!

[diogoribeirosky]

Hi @evzhang_splunk ,
Yes. I checked all the answers but unfortunately the problem is not solved. I think this might be a Splunk bug or malfunctioning....I used the statsd dimensions to pass the hostname info, which is redundant if the host is set correctly.

[aromanauskas]

Since this is an input to a metric index the host is the Splunk Server doing the conversion to metrics. There instead will be a different recalculated field for the original host. This is similar to the host of a summary index. Simplest option is to use Hostname instead as the dimension.

[diogoribeirosky]

Thank you aromanauskas for your answer
But if the host is the Splunk Server doing the conversion to metrics, How can the host be 127.0.0.1? which is the value I set $server in my TCPConnection.

[aromanauskas]

That is because the splunk server is using the rest API to communicate to itself on 127.0.0.1 . If you were using mcollect to write to the metric index, then it would replace the host with the name of the search head running the report. To keep confusion to a minimum I find it best to stay away from using host AND sourcetype for anything in metric indexes.

[richgalloway]

Btool may help. Run splunk btool --debug inputs list | Select-string -Pattern "host" <filepattern> -Simplematch in powershell to see where the host is getting set.

[diogoribeirosky]

[tcp]
_rcvbuf = 1572864
acceptFrom = *
connection_host = dns
evt_dc_name =
evt_dns_name =
evt_resolve_ad_obj = 0
host = myhostname
index = prod
[tcp:7051]
_rcvbuf = 1572864
acceptFrom = 127.0.0.1
evt_dc_name =
evt_dns_name =
evt_resolve_ad_obj = 0
host = myhostname
index = metrics
listenOnIPv6 = no
sourcetype = statsd

When running, ./splunk cmd btool inputs list command

[diogoribeirosky]

After running, I receive this message

\splunk-forwarder\etc\system\local\inputs.conf host = myhostname

and in search app splunk host is still 127.0.0.1