topic Re: Duplicate host field after indexing JSON event in Getting Data In

Duplicate host field after indexing JSON event

ColinCH — Thu, 17 Aug 2017 09:01:19 GMT

We are indexing a Logfile that has the following JSON format:

{"_check_command":"hostalive",_execution_time":4.0079541206359863281,**,"host":"myHostname",**"short_message":"PING OK","timestamp":1502959941.4374480247}

The Splunkforwarder is on the same server with the same "host" entry in the

inputs.conf 
[default] 
host = myHostname

When i'm searching in Splunk after the events from the Logfile, i have the problem that every Events has 2 Host fields, that one that is extracted from the JSON and that from the Forwarder. The counter is also doubled)

My first opinion was i cut out the host field in the JSON but it's not my prefered one.

Any ideas?

EDIT:
I added following line to the props.conf:
SEDCMD = s/\"host\"/\"hostname\"/

Re: Duplicate host field after indexing JSON event

cpetterborg — Thu, 17 Aug 2017 15:25:18 GMT

I'd modify the source at index time to contain hostname instead of host so that there would be no conflict. That will happen so that any search time field extractions will keep the host names separate. It is hard to deal with the host having two values if it does automatic field extractions.

The other thing that you could do at search time it use a rex to create a hostname field, then eliminate the extra host value that matches the hostname in the event, something like this:

| makeresults 
| eval host="orighost,myHostname" 
| makemv host delim="," 
| eval _raw="{\"_check_command\":\"hostalive\",_execution_time\":4.0079541206359863281,**,\"host\":\"myHostname\",**\"short_message\":\"PING OK\",\"timestamp\":1502959941.4374480247}" 
| rex "\"host\":\"(?P<hostname>[^\"]*)\"" 
| mvexpand host 
| where host!=hostname

The first four lines are just setting up the data, the last three are doing the work of setting the host and hostname properly.

Re: Duplicate host field after indexing JSON event

ColinCH — Fri, 18 Aug 2017 06:28:11 GMT

ahh, thank you. I did not think of that. I will SED the host field to hostname, so i don't need do delete it!

Re: Duplicate host field after indexing JSON event

cpetterborg — Fri, 18 Aug 2017 17:41:33 GMT

If this works for you, please accept the answer so that future viewers will know that it is a valid answer to your question. 🙂 Thanks!