Getting Data In

Duplicate host field after indexing JSON event

ColinCH
Path Finder

We are indexing a Logfile that has the following JSON format:

{"_check_command":"hostalive",_execution_time":4.0079541206359863281,**,"host":"myHostname",**"short_message":"PING OK","timestamp":1502959941.4374480247}

The Splunkforwarder is on the same server with the same "host" entry in the

inputs.conf 
[default] 
host = myHostname

When i'm searching in Splunk after the events from the Logfile, i have the problem that every Events has 2 Host fields, that one that is extracted from the JSON and that from the Forwarder. The counter is also doubled)

My first opinion was i cut out the host field in the JSON but it's not my prefered one.

Any ideas?

EDIT:
I added following line to the props.conf:
SEDCMD = s/\"host\"/\"hostname\"/

Tags (2)
0 Karma
1 Solution

cpetterborg
SplunkTrust
SplunkTrust

I'd modify the source at index time to contain hostname instead of host so that there would be no conflict. That will happen so that any search time field extractions will keep the host names separate. It is hard to deal with the host having two values if it does automatic field extractions.

The other thing that you could do at search time it use a rex to create a hostname field, then eliminate the extra host value that matches the hostname in the event, something like this:

| makeresults 
| eval host="orighost,myHostname" 
| makemv host delim="," 
| eval _raw="{\"_check_command\":\"hostalive\",_execution_time\":4.0079541206359863281,**,\"host\":\"myHostname\",**\"short_message\":\"PING OK\",\"timestamp\":1502959941.4374480247}" 
| rex "\"host\":\"(?P<hostname>[^\"]*)\"" 
| mvexpand host 
| where host!=hostname

The first four lines are just setting up the data, the last three are doing the work of setting the host and hostname properly.

View solution in original post

0 Karma

cpetterborg
SplunkTrust
SplunkTrust

I'd modify the source at index time to contain hostname instead of host so that there would be no conflict. That will happen so that any search time field extractions will keep the host names separate. It is hard to deal with the host having two values if it does automatic field extractions.

The other thing that you could do at search time it use a rex to create a hostname field, then eliminate the extra host value that matches the hostname in the event, something like this:

| makeresults 
| eval host="orighost,myHostname" 
| makemv host delim="," 
| eval _raw="{\"_check_command\":\"hostalive\",_execution_time\":4.0079541206359863281,**,\"host\":\"myHostname\",**\"short_message\":\"PING OK\",\"timestamp\":1502959941.4374480247}" 
| rex "\"host\":\"(?P<hostname>[^\"]*)\"" 
| mvexpand host 
| where host!=hostname

The first four lines are just setting up the data, the last three are doing the work of setting the host and hostname properly.

0 Karma

ColinCH
Path Finder

ahh, thank you. I did not think of that. I will SED the host field to hostname, so i don't need do delete it!

0 Karma

cpetterborg
SplunkTrust
SplunkTrust

If this works for you, please accept the answer so that future viewers will know that it is a valid answer to your question. 🙂 Thanks!

0 Karma
Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Get the T-shirt to Prove You Survived Splunk University Bootcamp

As if Splunk University, in Las Vegas, in-person, with three days of bootcamps and labs weren’t enough, now ...

Wondering How to Build Resiliency in the Cloud?

IT leaders are choosing Splunk Cloud as an ideal cloud transformation platform to drive business resilience,  ...