Turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

Splunk Search

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

- Find Answers
- :
- Using Splunk
- :
- Splunk Search
- :
- Re: Conver to Hexadecimal code to bit value

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page

- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

smanojkumar

Contributor

2 weeks ago

I would like to convert my hexadecimal code to a bit value based on this calculation.

Hex code - 0002

Seperate 2 bytes each 00/02

2 Byte bitmask

Byte 0: HEX = 00 - 0000 0000

Byte 1: HEX = 02 - 0000 0010

Byte 1 Byte0 - 0000 0010 0000 0000

calculate the non zero th position values from right side

Byte combination - 0000 0010 0000 0000

Position - 9 8765 4321

At position 10, we got 1 while counting from right side. so the bit value is 9.

I need to calculate this in splunk, where the HEX_Code is the value from the lookup.

Thanks in Advance!

Happy Splunking!

1 Solution

- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

yuanliu

SplunkTrust

2 weeks ago

@yuanliu 01100011 was _not_ hex. It was binary for 0x63. That's why I'm completely confused by

@PickleRick Lol you wouldn't believe how much time I spent trying to decipher the OP's intent from the various posted replies to everybody's attempt to help. After hours of scrolling up and down, back and forth, I distilled the instructions into the following algorithm given an even numbered HEX string, e.g., aabbcc

- Break the string into 2-HEX chunks. (OP used the term 2-bytes - I realize that is actually 4-bytes)
- Convert each chunk into binary.
- Reverse the order of the binary chunks.
- Count the positions of nonzero bits of the full reversed binary string from the right.

(As I said, I can't think of a practical purpose of this exercise. By the way, to anyone who is going to ask a question here, even though I strongly encourage describing problem without SPL first, please make the description as algorithmic as possible.) As a weird game, this applies to any even-length HEX string. Here's a sequence of up to 16 HEX characters.

hex | padded_binary | nonzero_bits |

01 | 00000001 | 0 |

0002 | 00000010 00000000 | 9 |

000003 | 00000011 00000000 00000000 | 16 17 |

00000004 | 00000100 00000000 00000000 00000000 | 26 |

0000000005 | 00000101 00000000 00000000 00000000 00000000 | 32 34 |

000000000006 | 00000110 00000000 00000000 00000000 00000000 00000000 | 41 42 |

00000000000007 | 00000111 00000000 00000000 00000000 00000000 00000000 00000000 | 48 49 50 |

0000000000000008 | 00001000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 | 59 |

90000000000000 | 00000000 00000000 00000000 00000000 00000000 00000000 10010000 | 4 7 |

a00000000000 | 00000000 00000000 00000000 00000000 00000000 10100000 | 5 7 |

b000000000 | 00000000 00000000 00000000 00000000 10110000 | 4 5 7 |

c0000000 | 00000000 00000000 00000000 11000000 | 6 7 |

d00000 | 00000000 00000000 11010000 | 4 6 7 |

e000 | 00000000 11100000 | 5 6 7 |

f0 | 11110000 | 4 5 6 7 |

Another thing I realize is that I must handle 2-HEX (single-chunk) specially. Here is the emulation code

```
| makeresults format=csv data="hex
01
0002
000003
00000004
0000000005
000000000006
00000000000007
0000000000000008
90000000000000
a00000000000
b000000000
c0000000
d00000
e000
f0"
``` data emulation above ```
| eval idx = mvrange(0, len(hex) / 2)
| eval reverse2hex = mvreverse(mvmap(idx, substr(hex, idx*2 + 1, 2)))
| eval ASbinary=if(idx < 1, tostring(tonumber(reverse2hex,16),"binary"), mvmap(reverse2hex, tostring(tonumber(reverse2hex,16),"binary")))
| eval padded_binary = if(idx < 1, printf("%08d", ASbinary), mvmap(ASbinary, printf("%08d", ASbinary)))
| eval reverse_bits = mvreverse(mvmap(padded_binary, split(padded_binary, ""))), position = -1
| foreach reverse_bits mode=multivalue
[eval position = position + 1, nonzero_bits = if(<<ITEM>> == 0, nonzero_bits, mvappend(nonzero_bits, position))]
| fields hex padded_binary nonzero_bits
```

(Technically this works for odd number of HEX characters, too, if OP can define where to split.)

- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

tscroggins

Influencer

2 weeks ago

Hi @smanojkumar,

This is a 16-bit adaptation of Gaudet's algorithm from *Hacker's Delight Second Edition* (Warren, 2013):

```
| makeresults
| eval HEX_Code="0002"
``` convert to number ```
| eval x=tonumber(HEX_Code, 16)
``` swap bytes ```
| eval x=bit_shift_right(x, 8)+bit_and(bit_shift_left(x, 8), 65280)
``` calculate number of trailing zeros (ntz) ```
| eval y=bit_and(x, 65535-x+1)
| eval bz=if(y>0, 0, 1), b3=if(bit_and(y, 255)>0, 0, 8), b2=if(bit_and(y, 3855)>0, 0, 4), b1=if(bit_and(y, 13107)>0, 0, 2), b0=if(bit_and(y, 21845)>0, 0, 1)
| eval ntz=bz+b3+b2+b1+b0
``` ntz=9 ```
```

- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

smanojkumar

Contributor

2 weeks ago

Thanks for your reply!

I'm having this error - "Error in 'EvalCommand': The 'bit_shift_right' function is unsupported or undefined."

Can you help in resolving this error.

Thanks in Advance!

- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

tscroggins

Influencer

2 weeks ago

Here's an alternative that uses a few helper macros to replace the bitwise eval functions. Bit rotate functions would be a nice addition to Splunk, as would a parameter on all bitwise functions to specify width.

```
| makeresults
| eval HEX_Code="0002"
``` convert to number ```
| eval x=tonumber(HEX_Code, 16)
``` swap bytes ```
| eval t=`bitshl(x, 8)`, x=`bitshr(x, 8)`+`bitand_16(t, 65280)`
``` calculate number of trailing zeros (ntz) ```
| eval t=65535-x+1, y=`bitand_16(x, t)`
| eval bz=if(y>0, 0, 1), b3=if(`bitand_16(y, 255)`>0, 0, 8), b2=if(`bitand_16(y, 3855)`>0, 0, 4), b1=if(`bitand_16(y, 13107)`>0, 0, 2), b0=if(`bitand_16(y, 21845)`>0, 0, 1)
| eval ntz=bz+b3+b2+b1+b0
``` ntz=9 ```
```

```
# macros.conf
[bitand_16(2)]
args = x, y
definition = sum(1 * (floor($x$ / 1) % 2) * (floor($y$ / 1) % 2), 2 * (floor($x$ / 2) % 2) * (floor($y$ / 2) % 2), 4 * (floor($x$ / 4) % 2) * (floor($y$ / 4) % 2), 8 * (floor($x$ / 😎 % 2) * (floor($y$ / 😎 % 2), 16 * (floor($x$ / 16) % 2) * (floor($y$ / 16) % 2), 32 * (floor($x$ / 32) % 2) * (floor($y$ / 32) % 2), 64 * (floor($x$ / 64) % 2) * (floor($y$ / 64) % 2), 128 * (floor($x$ / 128) % 2) * (floor($y$ / 128) % 2), 256 * (floor($x$ / 256) % 2) * (floor($y$ / 256) % 2), 512 * (floor($x$ / 512) % 2) * (floor($y$ / 512) % 2), 1024 * (floor($x$ / 1024) % 2) * (floor($y$ / 1024) % 2), 2048 * (floor($x$ / 2048) % 2) * (floor($y$ / 2048) % 2), 4096 * (floor($x$ / 4096) % 2) * (floor($y$ / 4096) % 2), 8192 * (floor($x$ / 8192) % 2) * (floor($y$ / 8192) % 2), 16384 * (floor($x$ / 16384) % 2) * (floor($y$ / 16384) % 2), 32768 * (floor($x$ / 32768) % 2) * (floor($y$ / 32768) % 2))
iseval = 0
[bitshl(2)]
args = x, k
definition = floor(pow(2, $k$) * $x$)
iseval = 0
[bitshr(2)]
args = x, k
definition = floor(pow(2, -$k$) * $x$)
iseval = 0
```

- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

ITWhisperer

SplunkTrust

2 weeks ago

- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

yuanliu

SplunkTrust

2 weeks ago

@smanojkumar Can you confirm that results you are looking for are like the following?

hex | padded_binary | nonzero_bits |

0002 | 00000010 00000000 | 9 |

00200100 | 00000000 00000001 00100000 00000000 | 13 16 |

01100011 | 00010001 00000000 00010000 00000001 | 0 12 24 28 |

This sounds like some data compression game. I can't think of a practical reason to do this in SPL. Is this some sort of homework?

Anyway, here is a more or less literal way to interpret your instructions:

```
| eval idx = mvrange(0, len(hex) / 2)
| eval reverse2hex = mvreverse(mvmap(idx, substr(hex, idx*2 + 1, 2)))
| eval ASbinary=if(idx < 1, tostring(tonumber(reverse2hex,16),"binary"), mvmap(reverse2hex, tostring(tonumber(reverse2hex,16),"binary")))
| eval padded_binary = if(idx < 1, printf("%08d", ASbinary), mvmap(ASbinary, printf("%08d", ASbinary)))
| eval reverse_bits = mvreverse(mvmap(padded_binary, split(padded_binary, ""))), position = -1
| foreach reverse_bits mode=multivalue
[eval position = position + 1, nonzero_bits = if(<<ITEM>> == 0, nonzero_bits, mvappend(nonzero_bits, position))]
| fields hex padded_binary nonzero_bits
```

Note mvreverse on padded binary is sort of expensive and can be avoided by arithmetics if there are lots of data.

Here is the emulation of the three examples you give:

```
| makeresults format=csv data="hex
0002
00200100
01100011"
``` data emulation above ```
```

Apply the algorithm to this emulation gives the results tabulated at the top.

- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

PickleRick

SplunkTrust

2 weeks ago

- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

yuanliu

SplunkTrust

2 weeks ago

@yuanliu 01100011 was _not_ hex. It was binary for 0x63. That's why I'm completely confused by

@PickleRick Lol you wouldn't believe how much time I spent trying to decipher the OP's intent from the various posted replies to everybody's attempt to help. After hours of scrolling up and down, back and forth, I distilled the instructions into the following algorithm given an even numbered HEX string, e.g., aabbcc

- Break the string into 2-HEX chunks. (OP used the term 2-bytes - I realize that is actually 4-bytes)
- Convert each chunk into binary.
- Reverse the order of the binary chunks.
- Count the positions of nonzero bits of the full reversed binary string from the right.

(As I said, I can't think of a practical purpose of this exercise. By the way, to anyone who is going to ask a question here, even though I strongly encourage describing problem without SPL first, please make the description as algorithmic as possible.) As a weird game, this applies to any even-length HEX string. Here's a sequence of up to 16 HEX characters.

hex | padded_binary | nonzero_bits |

01 | 00000001 | 0 |

0002 | 00000010 00000000 | 9 |

000003 | 00000011 00000000 00000000 | 16 17 |

00000004 | 00000100 00000000 00000000 00000000 | 26 |

0000000005 | 00000101 00000000 00000000 00000000 00000000 | 32 34 |

000000000006 | 00000110 00000000 00000000 00000000 00000000 00000000 | 41 42 |

00000000000007 | 00000111 00000000 00000000 00000000 00000000 00000000 00000000 | 48 49 50 |

0000000000000008 | 00001000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 | 59 |

90000000000000 | 00000000 00000000 00000000 00000000 00000000 00000000 10010000 | 4 7 |

a00000000000 | 00000000 00000000 00000000 00000000 00000000 10100000 | 5 7 |

b000000000 | 00000000 00000000 00000000 00000000 10110000 | 4 5 7 |

c0000000 | 00000000 00000000 00000000 11000000 | 6 7 |

d00000 | 00000000 00000000 11010000 | 4 6 7 |

e000 | 00000000 11100000 | 5 6 7 |

f0 | 11110000 | 4 5 6 7 |

Another thing I realize is that I must handle 2-HEX (single-chunk) specially. Here is the emulation code

```
| makeresults format=csv data="hex
01
0002
000003
00000004
0000000005
000000000006
00000000000007
0000000000000008
90000000000000
a00000000000
b000000000
c0000000
d00000
e000
f0"
``` data emulation above ```
| eval idx = mvrange(0, len(hex) / 2)
| eval reverse2hex = mvreverse(mvmap(idx, substr(hex, idx*2 + 1, 2)))
| eval ASbinary=if(idx < 1, tostring(tonumber(reverse2hex,16),"binary"), mvmap(reverse2hex, tostring(tonumber(reverse2hex,16),"binary")))
| eval padded_binary = if(idx < 1, printf("%08d", ASbinary), mvmap(ASbinary, printf("%08d", ASbinary)))
| eval reverse_bits = mvreverse(mvmap(padded_binary, split(padded_binary, ""))), position = -1
| foreach reverse_bits mode=multivalue
[eval position = position + 1, nonzero_bits = if(<<ITEM>> == 0, nonzero_bits, mvappend(nonzero_bits, position))]
| fields hex padded_binary nonzero_bits
```

(Technically this works for odd number of HEX characters, too, if OP can define where to split.)

- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

smanojkumar

Contributor

2 weeks ago

Hi @yuanliu & @ITWhisperer & @tscroggins & @PickleRick & @dural_yyz ,

Thanks everyone for your time, it works for me.

Thanks in Advance!

- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

smanojkumar

Contributor

2 weeks ago

Hello @yuanliu ,

Thanks for your response!

I'm having this error "Error in 'EvalCommand': The arguments to the 'tostring' function are invalid.", can you please help me in this.

Thanks in advance!

- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

ITWhisperer

SplunkTrust

2 weeks ago

- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

smanojkumar

Contributor

2 weeks ago

Thanks for that.

I'm currebtly using Splunk Enterprise with Version 9.1.1, So thats the reason.

Any alternative way to work on this with this version?

Thanks for pointing it out.

regards,

Manoj Kumar S

- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

ITWhisperer

SplunkTrust

2 weeks ago

- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

smanojkumar

Contributor

2 weeks ago

Hello @ITWhisperer ,

Thanks for your resposne!

If you don't mind changing the code as well.

Thansk a lot for your resposne!

- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

ITWhisperer

SplunkTrust

2 weeks ago

Here are the first four, I am sure you can workout from this how to do the other 12 hex digits

`| eval ASbinary=if(idx < 1, replace(replace(replace(replace(reverse2hex,"0","0000"),"1","0001"),"2","0010"),"3","0011"), mvmap(reverse2hex, replace(replace(replace(replace(reverse2hex,"0","0000"),"1","0001"),"2","0010"),"3","0011")))`

- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

smanojkumar

Contributor

2 weeks ago

Hi @ITWhisperer ,

Thanks, It works on that place.

- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

PickleRick

SplunkTrust

2 weeks ago

Nice job. I'm stil not convinced it's how it was supposed to work 😉

- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

smanojkumar

Contributor

2 weeks ago

Hex code - 00200100

Seperate 2 bytes each 00/20/01/00

4 Byte bitmask

Byte 0: HEX = 00 - 0000 0000

Byte 1: HEX = 20 - 0010 0000

Byte 2: HEX = 01 - 0000 0001

Byte 3: HEX = 00 - 0000 0000

Byte 3 Byte 2 Byte1 Byte 0 - 0000 0000 0000 0001 0010 0000 0000 0000

calculate the non zero th position values from right side

Byte combination - 0000 0000 0000 0001 0010 0000 0000 0000

Position - 16 13 ...9 8765 4321

At position 14 and 17, we got 1 while counting from right side. so the bit value is 13 and 16.

- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

PickleRick

SplunkTrust

2 weeks ago

OK. This is confusing. You have four hexadecimal digits but they're little-endian so the resulting bit order (not byte, mind you; you're happily using the same word for both bits and bytes). But what the calculation should be based on? What do you want to achieve?

You showed only one example which is a power of 2 so it gives you just one set bit in your whole 16-bit sequence. But what if you had 0x63 0x3A?

0x63 is 01100011, 0x3A is 00111010

As this is little-endian, the resulting bit-stream would be

00111010 01100011

And what now?

You want the position of first non-zero bit from the right?

And what does it have to do with "lookup"?

- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

smanojkumar

Contributor

2 weeks ago

Hello @PickleRick ,

Sorry for the mistake!

I have edited in the post,

Considering 0*63 is 01100011

Seperating the byte as with 2positions

byte 0 - 01 -> 0000 0001

byte 1 - 10 -> 0001 0000

byte 2 - 00 -> 0000 0000

byte 3 - 11 -> 0001 0001

Combining it byte 3 byte 2 byte 1 byte 0 - 0001 0001 0000 0000 0001 0000 0000 0001

Counting the non zero postions - 0, 12, 24, 28

These were the answers.

please let me know if there are anything.

Thanks in advance!

- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content

PickleRick

SplunkTrust

2 weeks ago

OK. Fully honestly, I don't see the point in doing so. You're adding some synthetic zeros and then count something which will easily be divisible by 4 (because when you're preparing your bitstring he ones must be separated by multiplies of 4).

You're doing some strange bit swapping (and it's completely inconsistent - your original example had something that resembled a 16-bits little-endian integer which you "converted" to 32-bit value and from my single bit 0x63 value you also built a 32-bit value but obviously in a different way).

So, to be absolutely frank, it doesn't make much sense.

A completely separate thing is why would you want to do it in splunk? (actually the easiest solution could be to implement an external lookup using a python script).

Get Updates on the Splunk Community!

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Register Now
This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...

Register
Join this Tech Talk to learn how unique features like Service Centric Views, Tag Spotlight, and ...