topic How to stop getting garbage HEXA ASCII logs from log source? in Getting Data In

How to stop getting garbage HEXA ASCII logs from log source?

pm2012 — Wed, 07 Jun 2023 19:25:51 GMT

Hi SMEs,

I am getting some garbage/hexa format/ASCII format logs from one of the log source integrated with Splunk, it is customized linux platform and been integrated using TCP input. Sharing the sample log below. Seeking suggestions to find and fix it. thanks in advance

Re: Getting garbage HEXA ASCII logs from log source

ITWhisperer — Wed, 07 Jun 2023 06:35:43 GMT

How have you configured the input?

Re: Getting garbage HEXA ASCII logs from log source

pm2012 — Wed, 07 Jun 2023 06:44:37 GMT

@ITWhisperer Input are configured using TCP at HF and logs are being sent using rsyslog.conf input parameters having needed filename. Logs are being sent to customized TCP port 615xx. Created multiple inputs for each filepath defined in rsyslog.conf

Re: Getting garbage HEXA ASCII logs from log source

ITWhisperer — Wed, 07 Jun 2023 07:20:57 GMT

Please can you share the config?

Re: Getting garbage HEXA ASCII logs from log source

pm2012 — Wed, 07 Jun 2023 07:51:46 GMT

Here is the rsyslog.conf file appended config, where 10.10.10.10 is Splunk HF IP and defined ports being used for log collection against TCP inputs.

##############Splunk#############

$InputFileName /var/SharedStorage/dl_logs/hostname01.com/dl_security/*
$InputFileTag dl-dl_security-log
$InputFileStateFile dl-dl_security-log
$InputFileSeverity error
$InputFileFacility local9
$InputRunFileMonitor

$InputFileName /var/SharedStorage/dl_logs/hostname01.com/pacemaker/*
$InputFileTag dl-pacemaker-log
$InputFileStateFile dl-pacemaker-log
$InputFileSeverity error
$InputFileFacility local9
$InputRunFileMonitor

$InputFilePollInterval 10

$template cmdlogsTemplate,"<dl> %timereported:::date-year%-%timereported:::date-month%-%timereported:::date-day% %timereported:::date-hour%:%timereported:::date-minute%:%timereported:::date-second% %HOSTNAME% %syslogtag% %msg% \n "

if $programname == 'dl-dl_security-log' then @@10.10.10.10:61515;cmdlogsTemplate
& ~
if $programname == 'dl-pacemaker-log' then @@10.10.10.10:61516;cmdlogsTemplate

*.* @@10.10.10.10:61500
#########################

Re: Getting garbage HEXA ASCII logs from log source

isoutamo — Wed, 07 Jun 2023 09:05:46 GMT

I suppose that your source file contains UTF16-LE characters? If so you must add encoding information to props.conf on HF side.

Here is couple of old answers which clarify this quite well

r. Ismo

Re: Getting garbage HEXA ASCII logs from log source

pm2012 — Wed, 07 Jun 2023 11:32:56 GMT

thanks @isoutamo

Just last doubt, how to check which format is it and which is supposed to be like UTF-18 or something else

Re: Getting garbage HEXA ASCII logs from log source

isoutamo — Wed, 07 Jun 2023 12:34:41 GMT

You can use command file in linux see: https://www.shellhacks.com/linux-check-change-file-encoding/

Usually Splunk wants to use UTF-8 encoding.