topic Searching strings with accented characters in Splunk Search

Searching strings with accented characters

LordLeet — Mon, 20 Aug 2018 11:42:44 GMT

Hello,

I'm having an issue when trying to filter events based on accented characters.

For instance if I look at the ingested events, index=my_index sourcetype=my_source , I will be able to see the events that have the field value I'm looking for:

... asset_name ...
... D. João ...

If I try to filter the events at search time index=my_index sourcetype=my_source asset_name="D. João" , the "No results found." message is displayed, the same applies if I select the desired field value from the field's list on the left.

How can I get this to work?

I've looked for similar questions here on the Splunk Answers forum but it mainly points for the sourcetype encoding, which I think might not be the issue, since the events seem to be properly encoded.

Thanks in advance!

Re: Searching strings with accented characters

CarsonZa — Mon, 20 Aug 2018 14:20:43 GMT

try something to the affect of |regex ã

Re: Searching strings with accented characters

LordLeet — Tue, 29 Sep 2020 21:00:19 GMT

Hello CarzonZa,

It works if I use the ´| regex ´ command, is this the only way to retrieve the filtered data?

I'm worried about this, since I tried to use the ´like()´ function and it didn't calculate properly.

´index="index_name" source="source_name"
| regex field_name="Carvão"
| eval x=if(like(field_name,"%ã%"),1,0)
| stats count by x´

It only counts x=0

Thanks

Re: Searching strings with accented characters

ddrillic — Tue, 21 Aug 2018 13:36:05 GMT

-- since the events seem to be properly encoded...

What is the encoding?

A similar thread at - How do I search for accented characters?

Re: Searching strings with accented characters

CarsonZa — Tue, 21 Aug 2018 13:58:34 GMT

create a capture group like so

| rex field=_raw "\s(?<name>.*)" 
| eval x=if(like(name,"%ã%"),1,0) 
| stats count by name

Re: Searching strings with accented characters

LordLeet — Tue, 21 Aug 2018 14:00:57 GMT

Hello ddrillic,

It's set to default.

From my previous research on other threads here on the forum, my understanding was that if the characters are show properly the encoding selected by Splunk would be correct.

If this is not the case I might need to manually select the correct one. Would changing the current encoding affect already ingested data or only the data ingested from now on?

Thanks

Re: Searching strings with accented characters

LordLeet — Wed, 22 Aug 2018 15:17:52 GMT

Hello CarsonZa,

That works, though using regex would not be the final solution I would like to implement since the majority of the users that perform searches do not possess the required knowledge.

I might have to assess the encoding to prevent this from happening.

Thanks