Getting odd data uploaded to my Splunk

[alanxu]

Hello,
I added to the .config file so whatever gets added to a folder will automatically be added to Splunk, however looking through my events I am getting weird characters that are not in the file that look like...

Last Name, First name\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00X\x00u\x00,\x00 \x00A\x00l\x00a\x00n\x00\x00\x00\x00\x00\x00\x00\xB0~r\x00\x00\x00\x8A \x00V\x00i\x00s\x00a\x00 \x00I\x00n\x00c\x00.\x00\x00\x00\x00\x00\xB0~r\x00\x00\x00\x8C\x00X\x00u\x00,\x00 \x00A\x00l\x00a\x00n\x00\x00\x00\x00\x00\x00\x00\xB0~r\x00\x00\x00\x88oMj\xF0\D6Lj\xD7j

[martin_mueller · Accepted solution]

It's a mix of how your file is written and how Splunk is trying to understand it. Here's how you would tell Splunk to try to understand the file as UTF-16:

props.conf on the Splunk instance reading the file (usually a Universal Forwarder)
[your_sourcetype]
CHARSET = UTF-16

[alanxu]

So in my \SplunkLightForwarder\default\props.conf

I added the lines
[Log]
CHARSET = UT-16

[martin_mueller]

My bet is on two-byte UTF-16, I can spot a Xu, Alan in between all the zero-bytes.

[alanxu]

Is there a reason why that comes up? Is it the type of file im updloading

[acharlieh]

A few initial questions, are these binary files (excel files or whatnot) of some sort as opposed to "plain text"? Secondly, are you specifying the correct CHARSET for these files in props.conf (Are they ASCII files or are they a different encoding)?

[alanxu]

They are .rtf files.

[alanxu]

Hmm change props.config? So whats happening is that my file can be in a different format that is specified in the props.conf?

[MuS]

Or do you forward data using a universal forwarder and send it to a Splunk TCP port http://docs.splunk.com/Documentation/Splunk/6.2.3/Data/Monitornetworkports instead the default Splunk receiving port http://docs.splunk.com/Documentation/Splunk/6.2.3/Forwarding/Enableareceiver#Set_up_receiving

[alanxu]

All I did was change the .config file

[alanxu]

The inputs.config