topic Re: Converting a field value from Hexadecimal to Decimal... in Splunk Search

Converting a field value from Hexadecimal to Decimal...

Steve_Litras — Sat, 05 Jun 2010 03:27:30 GMT

This may end up being a dumb question, but my regex/sed mojo is not strong today...

I have 2 log files monitored that each refer to a "common" error code, but of course, one of them reports that code as a hex number, and the other reports it as the decimal representation of the same number (nothing like coding consistency!). I pull them each in as a separate sourcetype, and I want to be able to correlate across both sourcetypes, so I want to convert one of the two to match the other (so they're both hex or both dec - don't really care which). I'm planning on doing this as a SEDCMD- entry in transforms.conf, but can't figure out if I can do that - I know I can do it at search time with eval and tostring(), but I really want the field extracted and converted @ index time.

Any thoughts?

Re: Converting a field value from Hexadecimal to Decimal...

Stephen_Sorkin — Sat, 05 Jun 2010 03:39:23 GMT

There's no good way to convert from hex to decimal at index time. I'd suggest sticking to search time evaluation.

Re: Converting a field value from Hexadecimal to Decimal...

Lowell — Sat, 05 Jun 2010 04:19:15 GMT

You can't really use regexes to do hex/dec conversion, so I don't think the SEDCMD option will work. (Maybe you could pull it off with multiple SEDCMD steps if you had just a 2 digit hex string, but even so that's still lots of overhead.)

There is a hexdec app out there written for 3.x. But upgrading it to 4.x is just a matter of getting the metdata setup. It converts both to and from hexadecimal and decimal.

I really do wish splunk would allow some way of handling this out of the box.

Re: Converting a field value from Hexadecimal to Decimal...

gkanapathy — Sat, 05 Jun 2010 10:24:15 GMT

At index time, you can only run regexes. I suppose if your life depended on it you could just enumerate the finite number of codes and SEDCMD each one, but seems much better to just do it at search time. If it's an issue of searching for the error codes, I'd suggest a search macro that would perform an eval on the macro argument (say, in decimal), figures out the hex equivalent and replaces the macro with an OR clause on both values.

Re: Converting a field value from Hexadecimal to Decimal...

Lowell — Thu, 07 Oct 2010 20:50:56 GMT

In Splunk 4.1.5 there is a new eval function called tonumber() which can be used to convert a hex value back to a decimal (based-10) value.

| eval dec_field=tostring(hex_field, 16)

Prior to 4.1.5 you have to do hex->dec conversions using and add-on search command; there was no way to do it out of the box.