Getting Data In

How can i get Splunk 8.0.5 to accept accented and special characters in a csv field name?

andrewtrobec
Motivator

Hello,

I recently upgraded from Splunk 7.3.2 to Splunk 8.0.5 and I noticed that the newer version does not manage special characters in csv field names.  As a simple exercise I created the following csv saved with Notepad++ wtih UTF-8 encoding:

timestamp;field_no_spaces;Field With Spaces;field_with_accented_a_à;field_with_degree_°
2020-01-01 10:00:00;value;value;value;value

The field names in the header contain characters that are in my source csv: spaces, accented letters and a special character (in this case °).  I followed the import wizard and left default settings in the "Set Source Type" section.  Without making any changes, here is the difference between the versions:

7.3.2

screenshot_732.png

8.0.5

screenshot_805.png

As you can see, 8.0.5 doesn't seem to handle those characters as standard.  I suspect that it might have to do with Python2 in 7.3.2 vs. Python3 in 8.0.5, but I'm not sure what to do.  I browsed the corresponding version of props.conf, but I wasn't able to find anything useful.

Would appreciate if someone could shed some light!  Hopefully someone has faced this issue before me...

Thank you and best regards,

Andrew

Labels (3)
0 Karma
1 Solution

arowsell_splunk
Splunk Employee
Splunk Employee

Hi,

Did you try setting "HEADER_FIELD_ACCEPTABLE_SPECIAL_CHARACTERS" attribute in props.conf in the [csv] stanza, as per:

https://docs.splunk.com/Documentation/Splunk/8.0.5/Data/Extractfieldsfromfileswithstructureddata

 

View solution in original post

arowsell_splunk
Splunk Employee
Splunk Employee

Hi,

Did you try setting "HEADER_FIELD_ACCEPTABLE_SPECIAL_CHARACTERS" attribute in props.conf in the [csv] stanza, as per:

https://docs.splunk.com/Documentation/Splunk/8.0.5/Data/Extractfieldsfromfileswithstructureddata

 

andrewtrobec
Motivator

@arowsell_splunk This is exactly what I needed!  I set HEADER_FIELD_ACCEPTABLE_SPECIAL_CHARACTERS in props.conf to "à°" to solve.  Thank you so much!

@manuelostertag I tried the suggestion with the german characters in the field headers and it worked:

updated csv:

timestamp;field_no_spaces;Field With Spaces;field_with_accented_a_à;field_with_degree_°;german_characters_ßäü
2020-01-01 10:00:00;value;value;value;value;value

HEADER_FIELD_ACCEPTABLE_SPECIAL_CHARACTERS = à°ßäü in props.conf

works.png

does it not work for you?

0 Karma

manuelostertag
Path Finder

Hello @andrewtrobec ,

I've tested HEADER_FIELD_ACCEPTABLE_SPECIAL_CHARACTERS = . and it doesn't work (I think this has something to do with ACSI above 128).

But if I use HEADER_FIELD_ACCEPTABLE_SPECIAL_CHARACTERS = ä,Ä,ü,Ü,ö,Ö,ß it worked 👍

@arowsell_splunkare there any disadvantage of using this?

Regards

Manuel

0 Karma

manuelostertag
Path Finder

Hello @arowsell_splunk ,

Thanks for the hint, but in my case this solves not the problem., cause (https://docs.splunk.com/Documentation/Splunk/8.0.8/Admin/Propsconf😞

* This setting only supports characters with ASCII codes below 128.

The German character like (ä, ü, ß, etc.) are above 128.

Regards

Manuel

0 Karma

manuelostertag
Path Finder

Hello @andrewtrobec ,

after the upgrade from Splunk 7.2.10 to 8.0.8 I run into the same problem. On the day I switch from 7 to 8, the problem starts.

Were you able to solve the problem in the meantime?

 

Regards

Manuel

0 Karma

andrewtrobec
Motivator

Ciao @manuelostertag!

Unfortunately I did not find a solution and no responses came from the community.  If you manage to find a solution then feel free to update here 🙂

Best regards,

Andrew

0 Karma
Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

WATCH NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If exploited, ...

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...