Convert HEX data to Text via search query

[sarahnazzar]

Hello Splunkers,

I've a event with hexadecimal data which I extracted as a field named X and I need to convert this X into text value which will be in human readable format i.e decoding hexadecimal value to text via splunk query..

Is there any way to do so via splunk search commands?

Thanks in Advance!
Sarah

[MuS]

Hi sarahnazzar,

Take a look at this answer https://answers.splunk.com/answers/151846/how-to-convert-hex-to-ascii-in-splunk.html#comment-220057 which provides a nice way to convert HEX to ASCII.

Hope this helps ...

cheers, MuS

PS: If it does not, please provide sample events and expected result so the community is able to help you 😉

[sarahnazzar]

Thanks for the response @MuS !

I even tried with that but was not getting the expected result..

I have the winevt logs are encoded.. so I need to decode..

Sample data:
Hexadecimal input - which I have extracted as one field using regular expression for instance let's keep it as X. X is my fieldname the below is my value,

46 69 6c 65 53 69 7a 65 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 36 39 36 33 32 0a 0a 49 73 4c 6f 67 46 75 6c 6c 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 46 61 6c 73 65 0a 0a 4c 61 73 74 41 63 63 65 73 73 54 69 6d 65 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 32 2f 31 34 2f 32 30 30 38 20 31 32 3a 35 35 3a 31 32 20 41 4d 0a 0a 4c 61 73 74 57 72 69 74 65 54 69 6d 65 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 37 2f 39 2f 32 30 30 38 20 33 3a 31 32 3a 30 35 20 41 4d 0a 0a 4f 6c 64 65 73 74 52 65 63 6f 72 64 4e 75 6d 62 65 72 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 31 0a 0a 52 65 63 6f 72 64 43 6f 75 6e 74 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 33 0a 0a 4c 6f 67 4e 61 6d 65 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 53 65 74 75 70 0a 0a 4c 6f 67 54 79 70 65 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 4f 70 65 72 61 74 69 6f 6e 61 6c 0a 0a 4c 6f 67 49 73 6f 6c 61 74 69 6f 6e 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 41 70 70 6c 69 63 61 74 69 6f 6e 0a 0a 49 73 45 6e 61 62 6c 65 64 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 54 72 75 65 0a 0a 49 73 43 6c 61 73 73 69 63 4c 6f 67 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 46 61 6c 73 65 0a 0a 53 65 63 75 72 69 74 79 44 65 73 63 72 69 70 74 6f 72 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 4f 3a 42 41 47 3a 53 59 44 3a 28 41 3b 3b 30 78 66 30 30 30 37 3b 3b 3b 53 59 29 28 41 3b 0a 0a 28 41 3b 3b 30 78 31 3b 3b 3b 53 2d 31 2d 35 2d 33 32 2d 35 37 33 29 0a 0a 4c 6f 67 46 69 6c 65 50 61 74 68 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 25 53 79 73 74 65 6d 52 6f 6f 74 25 5c 53 79 73 74 65 6d 33 32 5c 57 69 6e 65 76 74 5c 4c 0a 0a 4d 61 78 69 6d 75 6d 53 69 7a 65 49 6e 42 79 74 65 73 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 31 30 35 32 36 37 32 0a 0a 4c 6f 67 4d 6f 64 65 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 43 69 72 63 75 6c 61 72 0a 0a 4f 77 6e 69 6e 67 50 72 6f 76 69 64 65 72 4e 61 6d 65 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 4d 69 63 72 6f 73 6f 66 74 2d 57 69 6e 64 6f 77 73 2d 45 76 65 6e 74 6c 6f 67 0a 0a 50 72 6f 76 69 64 65 72 4e 61 6d 65 73 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 7b 4d 69 63 72 6f 73 6f 66 74 2d 57 69 6e 64 6f 77 73 2d 57 55 53 41 2c 20 4d 69 63 72 6f 0a 0a 50 72 6f 76 69 64 65 72 4c 65 76 65 6c 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 0a 0a 50 72 6f 76 69 64 65 72 4b 65 79 77 6f 72 64 73 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 0a 0a 50 72 6f 76 69 64 65 72 42 75 66 66 65 72 53 69 7a 65 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 36 34 0a 0a 50 72 6f 76 69 64 65 72 4d 69 6e 69 6d 75 6d 4e 75 6d 62 65 72 4f 66 42 75 66 66 65 72 73 20 3a 20 30 0a 0a 50 72 6f 76 69 64 65 72 4d 61 78 69 6d 75 6d 4e 75 6d 62 65 72 4f 66 42 75 66 66 65 72 73 20 3a 20 36 34 0a 0a 50 72 6f 76 69 64 65 72 4c 61 74 65 6e 63 79 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 31 30 30 30 0a 0a 50 72 6f 76 69 64 65 72 43 6f 6e 74 72 6f 6c 47 75 69 64 20 20 20 20 20 20 20 20 20 20 20 20 3a

Result must be like below,

FileSize : 69632

IsLogFull : False

LastAccessTime : 2/14/2008 12:55:12 AM

LastWriteTime : 7/9/2008 3:12:05 AM

OldestRecordNumber : 1

RecordCount : 3

LogName : Setup

LogType : Operational

LogIsolation : Application

IsEnabled : True

IsClassicLog : False

SecurityDescriptor : O:BAG:SYD:(A;;0xf0007;;;SY)(A;

(A;;0x1;;;S-1-5-32-573)

LogFilePath : %SystemRoot%\System32\Winevt\L

MaximumSizeInBytes : 1052672

LogMode : Circular

OwningProviderName : Microsoft-Windows-Eventlog

ProviderNames : {Microsoft-Windows-WUSA, Micro

ProviderLevel :

ProviderKeywords :

ProviderBufferSize : 64

ProviderMinimumNumberOfBuffers : 0

ProviderMaximumNumberOfBuffers : 64

ProviderLatency : 1000

ProviderControlGuid :

[to4kawa]

I see, please confirm my ans.

[sarahnazzar]

Thanks @to4kawa !

Its working fine when I pass the hex value like you have mentioned.. but when I tried passing a field instead of that.. its not working as expected for some of the field values and for some values its absolutely working fine..do we need to do any modifications in the expression?

Could you please help on that.!

HEX is my field extracted via search using rex command
| eval HexaValue=HEX
| rex field=HexaValue mode=sed "s/(\w\w) ?/%\1/g"
| eval Text=urldecode(HexaValue) |table HexaValue Text

[to4kawa]

please provide logs.
your HEX is key. but I don't know.

[sarahnazzar]

yes.. it is the key..!

When I tried modifying the expression like below.. its working only for first line of hex value,

| rex mode=sed "s/(^[0-9A-Fa-f]+) ?/%\1/g"

[to4kawa]

| rex mode=sed "s/(?m)(\w\w)\s?/%\1/g"
your HEX is multivalue or contains \n

[to4kawa]

| makeresults
| eval _raw="46 69 6c 65 53 69 7a 65 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 36 39 36 33 32 0a 0a 49 73 4c 6f 67 46 75 6c 6c 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 46 61 6c 73 65 0a 0a 4c 61 73 74 41 63 63 65 73 73 54 69 6d 65 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 32 2f 31 34 2f 32 30 30 38 20 31 32 3a 35 35 3a 31 32 20 41 4d 0a 0a 4c 61 73 74 57 72 69 74 65 54 69 6d 65 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 37 2f 39 2f 32 30 30 38 20 33 3a 31 32 3a 30 35 20 41 4d 0a 0a 4f 6c 64 65 73 74 52 65 63 6f 72 64 4e 75 6d 62 65 72 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 31 0a 0a 52 65 63 6f 72 64 43 6f 75 6e 74 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 33 0a 0a 4c 6f 67 4e 61 6d 65 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 53 65 74 75 70 0a 0a 4c 6f 67 54 79 70 65 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 4f 70 65 72 61 74 69 6f 6e 61 6c 0a 0a 4c 6f 67 49 73 6f 6c 61 74 69 6f 6e 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 41 70 70 6c 69 63 61 74 69 6f 6e 0a 0a 49 73 45 6e 61 62 6c 65 64 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 54 72 75 65 0a 0a 49 73 43 6c 61 73 73 69 63 4c 6f 67 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 46 61 6c 73 65 0a 0a 53 65 63 75 72 69 74 79 44 65 73 63 72 69 70 74 6f 72 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 4f 3a 42 41 47 3a 53 59 44 3a 28 41 3b 3b 30 78 66 30 30 30 37 3b 3b 3b 53 59 29 28 41 3b 0a 0a 28 41 3b 3b 30 78 31 3b 3b 3b 53 2d 31 2d 35 2d 33 32 2d 35 37 33 29 0a 0a 4c 6f 67 46 69 6c 65 50 61 74 68 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 25 53 79 73 74 65 6d 52 6f 6f 74 25 5c 53 79 73 74 65 6d 33 32 5c 57 69 6e 65 76 74 5c 4c 0a 0a 4d 61 78 69 6d 75 6d 53 69 7a 65 49 6e 42 79 74 65 73 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 31 30 35 32 36 37 32 0a 0a 4c 6f 67 4d 6f 64 65 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 43 69 72 63 75 6c 61 72 0a 0a 4f 77 6e 69 6e 67 50 72 6f 76 69 64 65 72 4e 61 6d 65 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 4d 69 63 72 6f 73 6f 66 74 2d 57 69 6e 64 6f 77 73 2d 45 76 65 6e 74 6c 6f 67 0a 0a 50 72 6f 76 69 64 65 72 4e 61 6d 65 73 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 7b 4d 69 63 72 6f 73 6f 66 74 2d 57 69 6e 64 6f 77 73 2d 57 55 53 41 2c 20 4d 69 63 72 6f 0a 0a 50 72 6f 76 69 64 65 72 4c 65 76 65 6c 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 0a 0a 50 72 6f 76 69 64 65 72 4b 65 79 77 6f 72 64 73 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 0a 0a 50 72 6f 76 69 64 65 72 42 75 66 66 65 72 53 69 7a 65 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 36 34 0a 0a 50 72 6f 76 69 64 65 72 4d 69 6e 69 6d 75 6d 4e 75 6d 62 65 72 4f 66 42 75 66 66 65 72 73 20 3a 20 30 0a 0a 50 72 6f 76 69 64 65 72 4d 61 78 69 6d 75 6d 4e 75 6d 62 65 72 4f 66 42 75 66 66 65 72 73 20 3a 20 36 34 0a 0a 50 72 6f 76 69 64 65 72 4c 61 74 65 6e 63 79 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 3a 20 31 30 30 30 0a 0a 50 72 6f 76 69 64 65 72 43 6f 6e 74 72 6f 6c 47 75 69 64 20 20 20 20 20 20 20 20 20 20 20 20 3a"
| rex mode=sed "s/(\w\w) ?/%\1/g"
| eval text=urldecode(_raw)
| table text

---

| makeresults
| eval X="48 69 20"
| rex field=X mode=sed "s/(\d+)/%\1/g"
| eval decode=urldecode(X)

I see. try urldecode

[sarahnazzar]

Thanks for the response!

urldecode is working for small strings but I'm not able to pass the field, my hex value field is having value around 20+ lines in that.. and its not working when tried using the same.

[to4kawa]

provide logs.

[sarahnazzar]

For Example: I have X=48 69 20 and I need this X to be converted to Hi (human readable format) using splunk search commands.

[richgalloway]

Hexadecimal numbers can be expressed as text (use table), but it sounds like you want to convert them to decimal. Do that with tonumber. ... | eval D = tonumber(X, 16) | ...

[sarahnazzar]

Nope.. I need to convert the hexadecimal values to text format not to any numerical conversions..

I used table but that just displays the field value right? but I need that to be converted to text..

For Example: I have X=48 69 20 and I need this X to be converted to Hi (human readable format) using splunk search commands.

Many thanks!