How to stop getting garbage HEXA ASCII logs from log source?

[pm2012]

Hi SMEs,

I am getting some garbage/hexa format/ASCII format logs from one of the log source integrated with Splunk, it is customized linux platform and been integrated using TCP input. Sharing the sample log below. Seeking suggestions to find and fix it. thanks in advance 

[isoutamo]

Hi

I suppose that your source file contains UTF16-LE characters? If so you must add encoding information to props.conf on HF side.

Here is couple of old answers which clarify this quite well

• https://community.splunk.com/t5/Getting-Data-In/Seing-null-x00-bytes-in-indexed-data-from-log-file-i...
• https://community.splunk.com/t5/Splunk-Search/Splunk-indexes-text-file-in-binary-format/m-p/61677

r. Ismo

[pm2012]

thanks @isoutamo 

Just last doubt, how to check which format is it and which is supposed to be like UTF-18 or something else 

[isoutamo]

You can use command file in linux see: https://www.shellhacks.com/linux-check-change-file-encoding/

Usually Splunk wants to use UTF-8 encoding.

[ITWhisperer]

How have you configured the input?

[pm2012]

@ITWhisperer  Input are configured using TCP at HF and logs are being sent using rsyslog.conf input parameters having needed filename. Logs are being sent to customized TCP port 615xx. Created multiple inputs for each filepath defined in rsyslog.conf

[ITWhisperer]

Please can you share the config?

[pm2012]

Here is the rsyslog.conf file appended config, where 10.10.10.10 is Splunk HF IP and defined ports being used for log collection against TCP inputs.

##############Splunk#############

$InputFileName /var/SharedStorage/dl_logs/hostname01.com/dl_security/*
$InputFileTag dl-dl_security-log
$InputFileStateFile dl-dl_security-log
$InputFileSeverity error
$InputFileFacility local9
$InputRunFileMonitor

$InputFileName /var/SharedStorage/dl_logs/hostname01.com/pacemaker/*
$InputFileTag dl-pacemaker-log
$InputFileStateFile dl-pacemaker-log
$InputFileSeverity error
$InputFileFacility local9
$InputRunFileMonitor

$InputFilePollInterval 10


$template cmdlogsTemplate,"<dl> %timereported:::date-year%-%timereported:::date-month%-%timereported:::date-day% %timereported:::date-hour%:%timereported:::date-minute%:%timereported:::date-second% %HOSTNAME% %syslogtag% %msg% \n "

if $programname == 'dl-dl_security-log' then @@10.10.10.10:61515;cmdlogsTemplate
& ~
if $programname == 'dl-pacemaker-log' then @@10.10.10.10:61516;cmdlogsTemplate

*.* @@10.10.10.10:61500
#########################