So my main search page in the bottom right hosts summary has the following error message listed:
[SimpleResultsTable module] Input is not proper UTF-8, indicate encoding ! Bytes: 0xD8 0xCE 0x89 0xB9, line 12, column 142
If I click on the page numbers it brings up the other pages as per normal and I can sort by host name and look around. But go back to page 1 and this error message is still there(assuming I didn't change the sort).
How can I find the undoubtedly garbled host name that is causing this without being able to see it?
Splunk support helped me with this issue.
Their solution:
putty into server, navigate to where the splunk data is and run
find . -name Hosts.data |xargs grep --color='auto' -P -n "[\x80-\xFF]"
this will return some of the junk host names you have e.g. in my case "host::#000#000#000#000#000#000#000#000#000#000#000#000#000#000#000#000#000#000#000#000#000#000#000#000!#000#000#000#000#000#000#000imudp#000#000#000?#013z#001#000#000#000#000 "
I could search for this in the splunk GUI (host=#000*) but the data would never show, but the interesting fields would.
Then try adding |delete to get rid of this data and if your lucky you fix your issues.
I would recommend working with support, this was a nightmare that took a few weeks to get to the bottom of.
Splunk support helped me with this issue.
Their solution:
putty into server, navigate to where the splunk data is and run
find . -name Hosts.data |xargs grep --color='auto' -P -n "[\x80-\xFF]"
this will return some of the junk host names you have e.g. in my case "host::#000#000#000#000#000#000#000#000#000#000#000#000#000#000#000#000#000#000#000#000#000#000#000#000!#000#000#000#000#000#000#000imudp#000#000#000?#013z#001#000#000#000#000 "
I could search for this in the splunk GUI (host=#000*) but the data would never show, but the interesting fields would.
Then try adding |delete to get rid of this data and if your lucky you fix your issues.
I would recommend working with support, this was a nightmare that took a few weeks to get to the bottom of.