Is there any way to decode an encoded html values saved in a log file?

[Boopalan]

I want decode all the encoded html values present in an log file while indexing itself.
Is there any way to do it ?

[sampathramtvnr]

urldecode works for decoding the values of URL addresses and strings
Try the below examples:

with url :
| makeresults

| eval field1= "f%23has%2Bofh%20a"
| eval field1 = urldecode(field1)

with string:
For example you already have field value then,
| rex mode=sed field=field1 "s/ / /g"
| eval a=urldecode(field1)

[pkeenan87]

You could try and decode it at search time with the urldecode eval function:

https://docs.splunk.com/Documentation/Splunk/7.2.5/SearchReference/TextFunctions

[Boopalan]

@pkeenan87, urldecode function is not working as expected. I tried doing that but that is working only for decoding values of url addresses not for an string containing ASCII encoded values in html.

[dmarling]

Urldecode decodes with the url encoding that starts with a precentage sign. You can manipulate the data a bit to force it to work. Using the most recent example by @Boopalan I got it to work, but I had to manually account for the HTML Encoded Line Feed character

Run anywhere example:

| makeresults count=1
| eval data="- 
--  Start of getter Meods
retiremevice.io.ReturnCd: 0
retiremevice.io.RtSCd:"
| eval test=data
| rex mode=sed field=test "s/
/\n/g"
| rex mode=sed field=test "s/&#x?([^\;]+);/%\1/g"
| eval test2=urldecode(test)

It makes it look like this when it runs:

- 
--  Start of getter Meods
retiremevice.io.ReturnCd: 0
retiremevice.io.RtSCd:

[Boopalan]

Hey Splunk folks,
Is there any possible way/ideas to do that?

[niketn]

@Boopalan any example from the log where html values are encoded? Which kind of encoding is in place? Also are these specific technology logs or are they custom logs? Please mock/anonymize any sensitive data.

[Boopalan]

@niketnilay, PFB sample of encoded html values in the log file.

%[datetime] [Default: 0] [] [INFO ] [*] - 
--  Start of getter Meods
retiremevice.io.ReturnCd: 0
retiremevice.io.RtSCd: 

[niketn]

@Boopalan, for the above text what is the expected decoded characters? I am not sure if this looks like either encoding or escaping of special characters. Would need to wait for others to comment. Or more details would help us assist you better!