Why is "�" character found in field values after uploading CSV file and indexing?

[dojiepreji]

Hi,

After uploading csv file and indexing, I found out that most, if not all of my special characters becomes "�" when viewed in splunk search.

Any help to fix this would be appreciated.
Thank you.

[DavidHourani · Accepted solution]

Hi @dojiepreji,
As explained in the documentation for CSV lookups :

The characters in the CSV file must be plain ASCII text and valid UTF-8 characters. Non-UTF-8 characters are not supported.

More info here :
https://docs.splunk.com/Documentation/Splunk/7.3.0/Knowledge/Usefieldlookupstoaddinformationtoyourev...

You should try to replace these characters with character from the UTF-8 set.

Cheers,
David

[denzelchung]

How are you creating and saving these csv files?

[dojiepreji]

I create them from microsoft excel and use export to csv within the app to save it as a csv file. The csv is fine. It is only in splunk that the special characters gets replaced with �.