How to override host metadata?

[jmonreal]

Good day everyone!

I have my Splunk cluster separated in Forwarders (inside each application server), Indexer (a set of servers running only Splunk as indexer) and Search Heads (a set of servers running only Splunk as search head).

I have on each application server a rsyslog that is sending logs to a certain $PORT which is also configured in the Splunk Forwarder as follows:

Inside the directory $SPLUNK_HOME/etc/apps/search/default/

inputs.conf

[udp://44444]
disabled        = false
_TCP_ROUTING    = test
index   = main
source  = docker
sourcetype      = docker
_meta   = host::"registrydocker.localdomain.com" site::"sao" environment::"_default" pool::"registrydocker" domain::"localdomain.com" cloud::"true"

outputs.conf

[tcpout:test]
server = 192.168.1.101:9997,192.168.1.100:9997
disabled = false

Now the indexer has no special configuration.

When I look at the logs in Splunk, the host shows 127.0.0.1 instead of registrydocker.localdomain.com
How should I be overriding the host and other metadata? Is it possible to override host and sourcetype for the same log lines?

[dshpritz · Accepted solution]

Hi jmonreal,

There are a few things here:
Please do not make your configuration changes in /default. There is a good chance that if you upgrade, your configurations will be overwritten.

Have you looked at using the "host" directive rather than using _meta?

[udp://44444]
disabled        = false
index   = main
source  = docker
sourcetype   = docker
host = registrydocker.localdomain.com
_TCP_ROUTING    = test
_meta   =  site::"sao" environment::"_default" pool::"registrydocker" domain::"localdomain.com" cloud::"true"

You can also add the "host" directive to your inputs.conf:

[tcpout:test]
 server = 192.168.1.101:9997,192.168.1.100:9997
 disabled = false
 host = registrydocker.localdomain.com

If you are not seeing any of your _meta fields, you may also need to check your fields.conf.

[dshpritz]

Worked over IRC, turns out the issue was that the inputs.conf needed:

connection_host = none

On the [udp://] stanza.

[jmonreal]

This one worked! Thanks 🙂