Duplicate Host Field from JSON Event

max_weber — Wed, 10 Jul 2019 08:52:37 GMT

Hey there,

we are pumping millions of Zabbix events in to our splunk environment over a Heavy Forwarder. The events are JSON string like this:

{"host":"myHostname","groups":["OS_RHEL","OS_RHEL_ES"],"applications":["FS RHELBASIC","Filesystems"],"itemid":1234,"name":"/var/log - used space (total)","clock":1562748008,"ns":583690877,"value":194605056}

the props for this sourcetype looks like this:

props.conf
TIME_PREFIX=\"clock\"\:
CHARSET=UTF-8
INDEXED_EXTRACTIONS=json
SHOULD_LINEMERGE=false

At first my problem was that the host field was filled with the hostname of the Heavy Forwarder. For easier use for our users, i want the host from the JSON event in my splunk "host" field. I tried to do that with following transforms:

[set_hostname_zabbix]
REGEX = "host":"(?P<host>[^"]+)
FORMAT = host::$1
DEST_KEY = MetaData:Host

This kind of worked, but now my problem is that i got two host fields both filled with the same data.
Any ideas how I can fix this, so I just get one host field filled with the hostname from the JSON event?

Regards,

Max

Re: Duplicate Host Field from JSON Event

woodcock — Wed, 10 Jul 2019 18:34:14 GMT

You have 2 options. You could set KV_MODE = none and because you are using indexed_extractions anyway, this should work fine. You could also set a calculated field like this: eval-host=mvdedup(host).

Re: Duplicate Host Field from JSON Event

max_weber — Fri, 19 Jul 2019 09:40:22 GMT

thanks for you advice. i have to try this.

but I think the first method won't help, cause the "host" field is a field which is always extracted not depending on the KV_MODE setting.

topic Duplicate Host Field from JSON Event in Splunk Search

Duplicate Host Field from JSON Event

Re: Duplicate Host Field from JSON Event

Re: Duplicate Host Field from JSON Event