Splunk Forwarder monitor hostname key is not working.
Amazon Linux AMI release 2015.03 3.14.48-33.39.amzn1.x86_64
Splunk Universal Forwarder 6.2.5 (build 272645)
Forwards:
input-prd-p-t865bwklrqxn.cloud.splunk.com:9997 (ssl)
I want to monitor log files from one host with Splunk Light.
Used command:
/opt/splunkforwarder/bin/splunk add monitor /var/log/messages -hostname <node hostname>
/opt/splunkforwarder/bin/splunk add monitor /var/log/audit/audit.log -hostname <node hostname>
/opt/splunkforwarder/bin/splunk add monitor /var/log/docker -hostname <node hostname>
But there are no hostnames when I check:
/opt/splunkforwarder/bin/splunk list monitor
Monitored Files:
/var/log/audit/audit.log
/var/log/docker
/var/log/messages
And as result i got 3(!) different Hosts on my cloud.splunk.com
ip-10-1-0-82 201 9/17/15 9:36:08.000 AM
_node hostname_ 14,843 9/17/15 9:44:03.000 AM
_local hostname_ 39 9/14/15 1:42:59.000 PM
What is the solution to this problem?
Couple of questions:
Are you running any applications in the cloud?
Do you have any field extractions that may be overwriting the hostname?
While you are setting the hostname at input time, you can overwrite it when it gets to the indexer..
No custom apps, jobs, filters or field extractions.
Field extractions and Field transformations in cloud.splunk.com is stored by defaults.
I just install splunkforwarder, add you cluster:
/opt/splunkforwarder/bin/splunk install app /opt/splunkforwarder/etc/splunkclouduf.spl -auth admin:changeme
/opt/splunkforwarder/bin/splunk restart
splunkclouduf.spl from https://.cloud.splunk.com/en-US/app/search/splunkclouduf
add 3 logs (see topic):
/var/log/audit/audit.log
/var/log/docker
/var/log/messages
And change some hosts in a disorderly heap of configs for tests.
All what i want - have a same host field or something other field
You shouldn't have to specify hostname in the command. Splunk's default is to use the system name. You can use btool to debug your configs. This command below will also show which apps each setting is coming from.
$SPLUNK_HOME/bin/splunk cmd btool --debug inputs list
< hostname > - for example, in config used real hostname.
Ok, output:
...
/opt/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/inputs.conf [monitor:///var/log/cron]
/opt/splunkforwarder/etc/system/default/inputs.conf _rcvbuf = 1572864
/opt/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/inputs.conf disabled = false
/opt/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/inputs.conf host = vastest3.<hostname>
/opt/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/inputs.conf ignoreOlderThan = 14d
/opt/splunkforwarder/etc/system/default/inputs.conf index = default
...
/opt/splunkforwarder/etc/system/local/inputs.conf host = vastest.<hostname>
opt/splunkforwarder/etc/apps/search/local/inputs.conf host = <hostname>
...
In logs
host = ip-10-1-0-41 source = /var/log/cron sourcetype = syslog
And i think it is worst way to use something like this for every new log record.
http://answers.splunk.com/answers/23507/why-is-host-localhost-when-inputs-conf-set-up-to-use-custom-...
Ok... let see props.conf
[linux_messages_syslog]
TRANSFORMS = syslog-host
...
[syslog]
TRANSFORMS = syslog-host
...
Created transforms.conf:
[syslog-host]
REGEX = vastest4.<hostname>
DEST_KEY = MetaData:Host
FORMAT = host::$1
not work too:
REGEX = \s(\w*)$
DEFAULT_VALUE = vastest5.qdoba-mera.seed1)
Result:
host = ip-10-1-0-41 source = /var/log/cron sourcetype = syslog
That next ?
p. s. And how can I delete "bad" Hosts events from *.cloud.splunk.com ?
Probably need to contact Support.
I submitted CASE [271586] 17.09.15, but don't got any answers and can't find any link of case for track result.
@vgolof: i looked up your case--you do not appear to have an active Support entitlement, which is why your case was not responded to. you must have a paid Support plan to receive a response from our Support team. i'll see if anyone has a minute to look at this.
Are you think it's not a bug ?