Getting Data In

How to stop getting garbage HEXA ASCII logs from log source?

pm2012
Explorer

Hi SMEs,

I am getting some garbage/hexa format/ASCII format logs from one of the log source integrated with Splunk, it is customized linux platform and been integrated using TCP input. Sharing the sample log below. Seeking suggestions to find and fix it. thanks in advance 

pm2012_0-1686118951686.png

 

 

Labels (1)
Tags (1)
0 Karma

isoutamo
SplunkTrust
SplunkTrust

Hi

I suppose that your source file contains UTF16-LE characters? If so you must add encoding information to props.conf on HF side.

Here is couple of old answers which clarify this quite well

r. Ismo

0 Karma

pm2012
Explorer

thanks @isoutamo 

Just last doubt, how to check which format is it and which is supposed to be like UTF-18 or something else 

pm2012_0-1686137568328.png

 

0 Karma

isoutamo
SplunkTrust
SplunkTrust

You can use command file in linux see: https://www.shellhacks.com/linux-check-change-file-encoding/

Usually Splunk wants to use UTF-8 encoding.

 

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

How have you configured the input?

0 Karma

pm2012
Explorer

@ITWhisperer  Input are configured using TCP at HF and logs are being sent using rsyslog.conf input parameters having needed filename. Logs are being sent to customized TCP port 615xx. Created multiple inputs for each filepath defined in rsyslog.conf

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Please can you share the config?

0 Karma

pm2012
Explorer

Here is the rsyslog.conf file appended config, where 10.10.10.10 is Splunk HF IP and defined ports being used for log collection against TCP inputs.

 

##############Splunk#############

$InputFileName /var/SharedStorage/dl_logs/hostname01.com/dl_security/*
$InputFileTag dl-dl_security-log
$InputFileStateFile dl-dl_security-log
$InputFileSeverity error
$InputFileFacility local9
$InputRunFileMonitor

$InputFileName /var/SharedStorage/dl_logs/hostname01.com/pacemaker/*
$InputFileTag dl-pacemaker-log
$InputFileStateFile dl-pacemaker-log
$InputFileSeverity error
$InputFileFacility local9
$InputRunFileMonitor



$InputFilePollInterval 10


$template cmdlogsTemplate,"<dl> %timereported:::date-year%-%timereported:::date-month%-%timereported:::date-day% %timereported:::date-hour%:%timereported:::date-minute%:%timereported:::date-second% %HOSTNAME% %syslogtag% %msg% \n "

if $programname == 'dl-dl_security-log' then @@10.10.10.10:61515;cmdlogsTemplate
& ~
if $programname == 'dl-pacemaker-log' then @@10.10.10.10:61516;cmdlogsTemplate

*.* @@10.10.10.10:61500
#########################

0 Karma
Get Updates on the Splunk Community!

SOC4Kafka - New Kafka Connector Powered by OpenTelemetry

The new SOC4Kafka connector, built on OpenTelemetry, enables the collection of Kafka messages and forwards ...

Your Voice Matters! Help Us Shape the New Splunk Lantern Experience

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Building Momentum: Splunk Developer Program at .conf25

At Splunk, developers are at the heart of innovation. That’s why this year at .conf25, we officially launched ...