I have a serious problem with logs.. some events (below 0.01%) have strange characters.
- such strange characters seems to not repeat (all are unique)
- there are no regional characters in events - OS has english language set
- it affects mainly Splunk processes, but not only
- for one selected host - there are "good" and "bad" process names. For example host ending with "PAL01" has hundreds of events with "splunk-powershell.exe" process name, and one with chinese artifacts.
My example search to find such hosts:
index="wineventlog" host=* (EventID=4688 OR EventID=861) NewProcessName=* SubjectUserName=* *splunk-pow* NOT *splunk-power*
What is going on? It is a serious problem, because we don't know how many artifacts are there and how to find them all.
I suggest that you examine the actual event logs on one of the servers with the event viewer and see if it originates there. If the logs are OK I would re-install the UF on those servers.
CHARSET = <string> in
props.conf for this sourcetype on the universal forwarder. Maybe you need to adjust this in your case http://docs.splunk.com/Documentation/Splunk/6.4.0/Admin/Propsconf#GLOBAL_SETTINGS
I changed it to CP1252, it didn't help - strange characters appear.