Dashboards & Visualizations

auto-decode UTF-8 encoded-string

mushkevych
Explorer

In our system we have 2 pipelines: one via Kafka->Connector->HEC->Splunk, the other DB Connect->Splunk.
Both pipelines are transporting data with the same sourcetype, which is marked with UTF-8 in props.conf: CHARSET=UTF-8.

UTF-8 strings that flow thru DB Connect are shown in Search app in their decoded format:
alt text

While the data flowing via Connector->HEC is still encoded:
alt text

Later on, the encoded values (such as "\u30a2...") are shown in the drop-down filters and so on.

Advise is very appreciated.

0 Karma

woodcock
Esteemed Legend

Create a macro and put a giant sed command in that and then create a Calculated Field for your sourcetype. For example, here is one that I wrote to do URL decoding:

... | rex field=fieldURLencoded mode=sed "s:%25:%:g s:%20: :g s:%3C:<:g s:%3E:>:g s:%23:#:g s:%7B:{:g s:%7D:}:g s:%7C:\|:g s:%5C:\\\:g s:%5E:\^:g s:%7E:~:g s:%5B:\[:g s:%5D:\]:g s:%60:\`:g s:%3B:;:g s:%2F:/:g s:%3F:\?:g s/%3A/:/g s:%40:@:g s:%3D:=:g s:%26:&:g s:%24:\$:g s:%21:\!:g s:%2A:\*:g s:%22:\":g s:%28:\(:g s:%29:\):g s:%2B:\+:g"

mushkevych
Explorer

Thank you @woodcock. it seems as manual UTF-8 decoding function. I am wondering if Splunk has anything in place for this.

0 Karma

woodcock
Esteemed Legend

My friend @Anonymous just wrote one and should be posting to splunkbase soon!

0 Karma

woodcock
Esteemed Legend

Check SplunkBase.

0 Karma

DanielP1
Engager

Hello,

Splunk Cloud does not support "MIME Decoder Add-on for Cisco ESA". Did your colleague publish a decoder? I haven't found anything on Splunkbase.

0 Karma

PickleRick
SplunkTrust
SplunkTrust

Well, this is a rather old thread and Greg hasn't been much online lately. You might get bigger chance of getting a reply if you post your question about an app in a new thread. (possibly linking to this one for reference).

0 Karma
Get Updates on the Splunk Community!

Prove Your Splunk Prowess at .conf25—No Prereqs Required!

Your Next Big Security Credential: No Prerequisites Needed We know you’ve got the skills, and now, earning the ...

Splunk Observability Cloud's AI Assistant in Action Series: Observability as Code

This is the sixth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how to ...

Splunk Answers Content Calendar, July Edition I

Hello Community! Welcome to another month of Community Content Calendar series! For the month of July, we will ...