topic Re: Convert HEX data to Text via search query in Dashboards & Visualizations

Convert HEX data to Text via search query

sarahnazzar — Wed, 29 Apr 2020 12:17:13 GMT

Hello Splunkers,

I've a event with hexadecimal data which I extracted as a field named X and I need to convert this X into text value which will be in human readable format i.e decoding hexadecimal value to text via splunk query..

Is there any way to do so via splunk search commands?

Thanks in Advance!
Sarah

Re: Convert HEX data to Text via search query

richgalloway — Wed, 29 Apr 2020 12:49:29 GMT

Hexadecimal numbers can be expressed as text (use table), but it sounds like you want to convert them to decimal. Do that with tonumber. ... | eval D = tonumber(X, 16) | ...

Re: Convert HEX data to Text via search query

sarahnazzar — Thu, 30 Apr 2020 07:59:32 GMT

Nope.. I need to convert the hexadecimal values to text format not to any numerical conversions..

I used table but that just displays the field value right? but I need that to be converted to text..

For Example: I have X=48 69 20 and I need this X to be converted to Hi (human readable format) using splunk search commands.

Many thanks!

Re: Convert HEX data to Text via search query

sarahnazzar — Thu, 30 Apr 2020 08:00:39 GMT

For Example: I have X=48 69 20 and I need this X to be converted to Hi (human readable format) using splunk search commands.

Re: Convert HEX data to Text via search query

to4kawa — Thu, 30 Apr 2020 08:30:06 GMT

| makeresults
| eval _raw="46 69 6c 65 53 69 7a 65 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 36 39 36 33 32 0a 0a 49 73 4c 6f 67 46 75 6c 6c 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 46 61 6c 73 65 0a 0a 4c 61 73 74 41 63 63 65 73 73 54 69 6d 65 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 32 2f 31 34 2f 32 30 30 38 20 31 32 3a 35 35 3a 31 32 20 41 4d 0a 0a 4c 61 73 74 57 72 69 74 65 54 69 6d 65 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 37 2f 39 2f 32 30 30 38 20 33 3a 31 32 3a 30 35 20 41 4d 0a 0a 4f 6c 64 65 73 74 52 65 63 6f 72 64 4e 75 6d 62 65 72 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 31 0a 0a 52 65 63 6f 72 64 43 6f 75 6e 74 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 33 0a 0a 4c 6f 67 4e 61 6d 65 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 53 65 74 75 70 0a 0a 4c 6f 67 54 79 70 65 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 4f 70 65 72 61 74 69 6f 6e 61 6c 0a 0a 4c 6f 67 49 73 6f 6c 61 74 69 6f 6e 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 41 70 70 6c 69 63 61 74 69 6f 6e 0a 0a 49 73 45 6e 61 62 6c 65 64 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 54 72 75 65 0a 0a 49 73 43 6c 61 73 73 69 63 4c 6f 67 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 46 61 6c 73 65 0a 0a 53 65 63 75 72 69 74 79 44 65 73 63 72 69 70 74 6f 72 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 4f 3a 42 41 47 3a 53 59 44 3a 28 41 3b 3b 30 78 66 30 30 30 37 3b 3b 3b 53 59 29 28 41 3b 0a 0a 28 41 3b 3b 30 78 31 3b 3b 3b 53 2d 31 2d 35 2d 33 32 2d 35 37 33 29 0a 0a 4c 6f 67 46 69 6c 65 50 61 74 68 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 25 53 79 73 74 65 6d 52 6f 6f 74 25 5c 53 79 73 74 65 6d 33 32 5c 57 69 6e 65 76 74 5c 4c 0a 0a 4d 61 78 69 6d 75 6d 53 69 7a 65 49 6e 42 79 74 65 73 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 31 30 35 32 36 37 32 0a 0a 4c 6f 67 4d 6f 64 65 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 43 69 72 63 75 6c 61 72 0a 0a 4f 77 6e 69 6e 67 50 72 6f 76 69 64 65 72 4e 61 6d 65 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 4d 69 63 72 6f 73 6f 66 74 2d 57 69 6e 64 6f 77 73 2d 45 76 65 6e 74 6c 6f 67 0a 0a 50 72 6f 76 69 64 65 72 4e 61 6d 65 73 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 7b 4d 69 63 72 6f 73 6f 66 74 2d 57 69 6e 64 6f 77 73 2d 57 55 53 41 2c 20 4d 69 63 72 6f 0a 0a 50 72 6f 76 69 64 65 72 4c 65 76 65 6c 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 0a 0a 50 72 6f 76 69 64 65 72 4b 65 79 77 6f 72 64 73 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 0a 0a 50 72 6f 76 69 64 65 72 42 75 66 66 65 72 53 69 7a 65 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 36 34 0a 0a 50 72 6f 76 69 64 65 72 4d 69 6e 69 6d 75 6d 4e 75 6d 62 65 72 4f 66 42 75 66 66 65 72 73 20 3a 20 30 0a 0a 50 72 6f 76 69 64 65 72 4d 61 78 69 6d 75 6d 4e 75 6d 62 65 72 4f 66 42 75 66 66 65 72 73 20 3a 20 36 34 0a 0a 50 72 6f 76 69 64 65 72 4c 61 74 65 6e 63 79 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 31 30 30 30 0a 0a 50 72 6f 76 69 64 65 72 43 6f 6e 74 72 6f 6c 47 75 69 64 20 20 20 20 20 20 20 20 20 20 20 20 3a"
| rex mode=sed "s/(\w\w) ?/%\1/g"
| eval text=urldecode(_raw)
| table text

| makeresults
| eval X="48 69 20"
| rex field=X mode=sed "s/(\d+)/%\1/g"
| eval decode=urldecode(X)

I see. try urldecode

Re: Convert HEX data to Text via search query

sarahnazzar — Thu, 30 Apr 2020 09:14:19 GMT

Thanks for the response!

urldecode is working for small strings but I'm not able to pass the field, my hex value field is having value around 20+ lines in that.. and its not working when tried using the same.

Re: Convert HEX data to Text via search query

to4kawa — Thu, 30 Apr 2020 09:17:15 GMT

provide logs.

Re: Convert HEX data to Text via search query

MuS — Thu, 30 Apr 2020 09:23:30 GMT

Hi sarahnazzar,

Take a look at this answer https://answers.splunk.com/answers/151846/how-to-convert-hex-to-ascii-in-splunk.html#comment-220057 which provides a nice way to convert HEX to ASCII.

Hope this helps ...

cheers, MuS

PS: If it does not, please provide sample events and expected result so the community is able to help you 😉

Re: Convert HEX data to Text via search query

sarahnazzar — Thu, 30 Apr 2020 11:43:43 GMT

Thanks for the response @MuS !

I even tried with that but was not getting the expected result..

I have the winevt logs are encoded.. so I need to decode..

Sample data:
Hexadecimal input - which I have extracted as one field using regular expression for instance let's keep it as X. X is my fieldname the below is my value,

46 69 6c 65 53 69 7a 65 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 36 39 36 33 32 0a 0a 49 73 4c 6f 67 46 75 6c 6c 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 46 61 6c 73 65 0a 0a 4c 61 73 74 41 63 63 65 73 73 54 69 6d 65 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 32 2f 31 34 2f 32 30 30 38 20 31 32 3a 35 35 3a 31 32 20 41 4d 0a 0a 4c 61 73 74 57 72 69 74 65 54 69 6d 65 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 37 2f 39 2f 32 30 30 38 20 33 3a 31 32 3a 30 35 20 41 4d 0a 0a 4f 6c 64 65 73 74 52 65 63 6f 72 64 4e 75 6d 62 65 72 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 31 0a 0a 52 65 63 6f 72 64 43 6f 75 6e 74 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 33 0a 0a 4c 6f 67 4e 61 6d 65 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 53 65 74 75 70 0a 0a 4c 6f 67 54 79 70 65 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 4f 70 65 72 61 74 69 6f 6e 61 6c 0a 0a 4c 6f 67 49 73 6f 6c 61 74 69 6f 6e 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 41 70 70 6c 69 63 61 74 69 6f 6e 0a 0a 49 73 45 6e 61 62 6c 65 64 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 54 72 75 65 0a 0a 49 73 43 6c 61 73 73 69 63 4c 6f 67 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 46 61 6c 73 65 0a 0a 53 65 63 75 72 69 74 79 44 65 73 63 72 69 70 74 6f 72 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 4f 3a 42 41 47 3a 53 59 44 3a 28 41 3b 3b 30 78 66 30 30 30 37 3b 3b 3b 53 59 29 28 41 3b 0a 0a 28 41 3b 3b 30 78 31 3b 3b 3b 53 2d 31 2d 35 2d 33 32 2d 35 37 33 29 0a 0a 4c 6f 67 46 69 6c 65 50 61 74 68 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 25 53 79 73 74 65 6d 52 6f 6f 74 25 5c 53 79 73 74 65 6d 33 32 5c 57 69 6e 65 76 74 5c 4c 0a 0a 4d 61 78 69 6d 75 6d 53 69 7a 65 49 6e 42 79 74 65 73 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 31 30 35 32 36 37 32 0a 0a 4c 6f 67 4d 6f 64 65 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 43 69 72 63 75 6c 61 72 0a 0a 4f 77 6e 69 6e 67 50 72 6f 76 69 64 65 72 4e 61 6d 65 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 4d 69 63 72 6f 73 6f 66 74 2d 57 69 6e 64 6f 77 73 2d 45 76 65 6e 74 6c 6f 67 0a 0a 50 72 6f 76 69 64 65 72 4e 61 6d 65 73 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 7b 4d 69 63 72 6f 73 6f 66 74 2d 57 69 6e 64 6f 77 73 2d 57 55 53 41 2c 20 4d 69 63 72 6f 0a 0a 50 72 6f 76 69 64 65 72 4c 65 76 65 6c 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 0a 0a 50 72 6f 76 69 64 65 72 4b 65 79 77 6f 72 64 73 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 0a 0a 50 72 6f 76 69 64 65 72 42 75 66 66 65 72 53 69 7a 65 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 36 34 0a 0a 50 72 6f 76 69 64 65 72 4d 69 6e 69 6d 75 6d 4e 75 6d 62 65 72 4f 66 42 75 66 66 65 72 73 20 3a 20 30 0a 0a 50 72 6f 76 69 64 65 72 4d 61 78 69 6d 75 6d 4e 75 6d 62 65 72 4f 66 42 75 66 66 65 72 73 20 3a 20 36 34 0a 0a 50 72 6f 76 69 64 65 72 4c 61 74 65 6e 63 79 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 31 30 30 30 0a 0a 50 72 6f 76 69 64 65 72 43 6f 6e 74 72 6f 6c 47 75 69 64 20 20 20 20 20 20 20 20 20 20 20 20 3a

Result must be like below,

FileSize : 69632

IsLogFull : False

LastAccessTime : 2/14/2008 12:55:12 AM

LastWriteTime : 7/9/2008 3:12:05 AM

OldestRecordNumber : 1

RecordCount : 3

LogName : Setup

LogType : Operational

LogIsolation : Application

IsEnabled : True

IsClassicLog : False

SecurityDescriptor : O:BAG:SYD:(A;;0xf0007;;;SY)(A;

(A;;0x1;;;S-1-5-32-573)

LogFilePath : %SystemRoot%\System32\Winevt\L

MaximumSizeInBytes : 1052672

LogMode : Circular

OwningProviderName : Microsoft-Windows-Eventlog

ProviderNames : {Microsoft-Windows-WUSA, Micro

ProviderLevel :

ProviderKeywords :

ProviderBufferSize : 64

ProviderMinimumNumberOfBuffers : 0

ProviderMaximumNumberOfBuffers : 64

ProviderLatency : 1000

ProviderControlGuid :

Re: Convert HEX data to Text via search query

to4kawa — Fri, 01 May 2020 11:14:48 GMT

I see, please confirm my ans.

Re: Convert HEX data to Text via search query

sarahnazzar — Mon, 04 May 2020 07:00:41 GMT

Thanks @to4kawa !

Its working fine when I pass the hex value like you have mentioned.. but when I tried passing a field instead of that.. its not working as expected for some of the field values and for some values its absolutely working fine..do we need to do any modifications in the expression?

Could you please help on that.!

HEX is my field extracted via search using rex command
| eval HexaValue=HEX
| rex field=HexaValue mode=sed "s/(\w\w) ?/%\1/g"
| eval Text=urldecode(HexaValue) |table HexaValue Text

Re: Convert HEX data to Text via search query

to4kawa — Mon, 04 May 2020 07:25:57 GMT

please provide logs.
your HEX is key. but I don't know.

Re: Convert HEX data to Text via search query

sarahnazzar — Mon, 04 May 2020 08:08:29 GMT

yes.. it is the key..!

When I tried modifying the expression like below.. its working only for first line of hex value,

| rex mode=sed "s/(^[0-9A-Fa-f]+) ?/%\1/g"

Re: Convert HEX data to Text via search query

to4kawa — Mon, 04 May 2020 08:14:39 GMT

| rex mode=sed "s/(?m)(\w\w)\s?/%\1/g"
your HEX is multivalue or contains \n