Getting Data In

Duplicate host field after indexing JSON event

ColinCH
Path Finder

We are indexing a Logfile that has the following JSON format:

{"_check_command":"hostalive",_execution_time":4.0079541206359863281,**,"host":"myHostname",**"short_message":"PING OK","timestamp":1502959941.4374480247}

The Splunkforwarder is on the same server with the same "host" entry in the

inputs.conf 
[default] 
host = myHostname

When i'm searching in Splunk after the events from the Logfile, i have the problem that every Events has 2 Host fields, that one that is extracted from the JSON and that from the Forwarder. The counter is also doubled)

My first opinion was i cut out the host field in the JSON but it's not my prefered one.

Any ideas?

EDIT:
I added following line to the props.conf:
SEDCMD = s/\"host\"/\"hostname\"/

Tags (2)
0 Karma
1 Solution

cpetterborg
SplunkTrust
SplunkTrust

I'd modify the source at index time to contain hostname instead of host so that there would be no conflict. That will happen so that any search time field extractions will keep the host names separate. It is hard to deal with the host having two values if it does automatic field extractions.

The other thing that you could do at search time it use a rex to create a hostname field, then eliminate the extra host value that matches the hostname in the event, something like this:

| makeresults 
| eval host="orighost,myHostname" 
| makemv host delim="," 
| eval _raw="{\"_check_command\":\"hostalive\",_execution_time\":4.0079541206359863281,**,\"host\":\"myHostname\",**\"short_message\":\"PING OK\",\"timestamp\":1502959941.4374480247}" 
| rex "\"host\":\"(?P<hostname>[^\"]*)\"" 
| mvexpand host 
| where host!=hostname

The first four lines are just setting up the data, the last three are doing the work of setting the host and hostname properly.

View solution in original post

0 Karma

cpetterborg
SplunkTrust
SplunkTrust

I'd modify the source at index time to contain hostname instead of host so that there would be no conflict. That will happen so that any search time field extractions will keep the host names separate. It is hard to deal with the host having two values if it does automatic field extractions.

The other thing that you could do at search time it use a rex to create a hostname field, then eliminate the extra host value that matches the hostname in the event, something like this:

| makeresults 
| eval host="orighost,myHostname" 
| makemv host delim="," 
| eval _raw="{\"_check_command\":\"hostalive\",_execution_time\":4.0079541206359863281,**,\"host\":\"myHostname\",**\"short_message\":\"PING OK\",\"timestamp\":1502959941.4374480247}" 
| rex "\"host\":\"(?P<hostname>[^\"]*)\"" 
| mvexpand host 
| where host!=hostname

The first four lines are just setting up the data, the last three are doing the work of setting the host and hostname properly.

0 Karma

ColinCH
Path Finder

ahh, thank you. I did not think of that. I will SED the host field to hostname, so i don't need do delete it!

0 Karma

cpetterborg
SplunkTrust
SplunkTrust

If this works for you, please accept the answer so that future viewers will know that it is a valid answer to your question. 🙂 Thanks!

0 Karma
Get Updates on the Splunk Community!

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...