How can i get Splunk 8.0.5 to accept accented and special characters in a csv field name?

[andrewtrobec]

Hello,

I recently upgraded from Splunk 7.3.2 to Splunk 8.0.5 and I noticed that the newer version does not manage special characters in csv field names.  As a simple exercise I created the following csv saved with Notepad++ wtih UTF-8 encoding:

timestamp;field_no_spaces;Field With Spaces;field_with_accented_a_à;field_with_degree_°
2020-01-01 10:00:00;value;value;value;value

The field names in the header contain characters that are in my source csv: spaces, accented letters and a special character (in this case °).  I followed the import wizard and left default settings in the "Set Source Type" section.  Without making any changes, here is the difference between the versions:

7.3.2

8.0.5

As you can see, 8.0.5 doesn't seem to handle those characters as standard.  I suspect that it might have to do with Python2 in 7.3.2 vs. Python3 in 8.0.5, but I'm not sure what to do.  I browsed the corresponding version of props.conf, but I wasn't able to find anything useful.

Would appreciate if someone could shed some light!  Hopefully someone has faced this issue before me...

Thank you and best regards,

Andrew

[arowsell_splunk · Accepted solution]

Hi,

Did you try setting "HEADER_FIELD_ACCEPTABLE_SPECIAL_CHARACTERS" attribute in props.conf in the [csv] stanza, as per:

https://docs.splunk.com/Documentation/Splunk/8.0.5/Data/Extractfieldsfromfileswithstructureddata

[andrewtrobec]

@arowsell_splunk This is exactly what I needed!  I set HEADER_FIELD_ACCEPTABLE_SPECIAL_CHARACTERS in props.conf to "à°" to solve.  Thank you so much!

@manuelostertag I tried the suggestion with the german characters in the field headers and it worked:

updated csv:

timestamp;field_no_spaces;Field With Spaces;field_with_accented_a_à;field_with_degree_°;german_characters_ßäü
2020-01-01 10:00:00;value;value;value;value;value

HEADER_FIELD_ACCEPTABLE_SPECIAL_CHARACTERS = à°ßäü in props.conf

does it not work for you?

[manuelostertag]

Hello @andrewtrobec ,

I've tested HEADER_FIELD_ACCEPTABLE_SPECIAL_CHARACTERS = . and it doesn't work (I think this has something to do with ACSI above 128).

But if I use HEADER_FIELD_ACCEPTABLE_SPECIAL_CHARACTERS = ä,Ä,ü,Ü,ö,Ö,ß it worked 👍

@arowsell_splunkare there any disadvantage of using this?

Regards

Manuel

[manuelostertag]

Hello @arowsell_splunk ,

Thanks for the hint, but in my case this solves not the problem., cause (https://docs.splunk.com/Documentation/Splunk/8.0.8/Admin/Propsconf😞

* This setting only supports characters with ASCII codes below 128.

The German character like (ä, ü, ß, etc.) are above 128.

Regards

Manuel

[manuelostertag]

Hello @andrewtrobec ,

after the upgrade from Splunk 7.2.10 to 8.0.8 I run into the same problem. On the day I switch from 7 to 8, the problem starts.

Were you able to solve the problem in the meantime?

Regards

Manuel

[andrewtrobec]

Ciao @manuelostertag!

Unfortunately I did not find a solution and no responses came from the community.  If you manage to find a solution then feel free to update here 🙂

Best regards,

Andrew