Splunk Search

Is it possible to perform bitwise operations on values in the eval function?

whisperstream
Explorer

In the following query I'm trying to get the logical AND of two numbers:

 * | head 1 | eval x=2 | eval y=2 | eval z = and(x, y)) | table x y z

Ideally the result would be:

x y z
2 2 2

because function "and" above would be similar to python's:
LOGICAL AND: x & y

So it's easy in pseudo code/query, but is there a way to actually do this, or a plugin that provides these operations? I wasn't able to find it in the command reference.

0 Karma
1 Solution

javiergn
Super Champion

You could write your command in Python. See this and this

Alternatively, try the following query I've written as I was really curious on how to achieve this from Splunk. The logic should work fine if you have more than 2 numbers, simply use another appendcols and play with the bitwise variables. This works with numbers up to 2^20 - 1, simply increase all the mvrange parameters if you want it to be able to cope with bigger numbers.

EDIT1: to use your naming convention.
EDIT2: also calculating Z back at the bottom
EDIT3: quick performance improvements and now you don't have to specify X and Y twice

| stats count
| fields - count
| eval x = 63

| eval power = mvrange(0,20)
| mvexpand power
| eval base2 = pow(2, power)
| where x >= base2
| eval mydiv = floor(x / base2)
| eval mybin = mydiv % 2
| eval binx = mybin

| appendcols [
   | stats count
   | fields - count
   | eval y = 2

   | eval power = mvrange(0,20)
   | mvexpand power
   | eval base2 = pow(2, power)
   | where y >= base2
   | eval mydiv = floor(y / base2)
   | eval mybin = mydiv % 2
   | eval biny = mybin
 ]
| fillnull
| eval bitwiseOr = round((binx + biny) / 2)
| eval bitwiseAnd = (binx * biny)
| eval zAnd = bitwiseAnd * base2
| eval zOr = bitwiseOr * base2
| stats max(x) as x, max(y) as y, list(binx) as binx, list(biny) as biny, list(bitwiseOr) as bitwiseOr, list(bitwiseAnd) as bitwiseAnd, sum(zAnd) as zAnd, sum(zOr) as zOr

And this is the result (see picture below):

alt text

View solution in original post

jaypatel
Splunk Employee
Splunk Employee

Now you can use the eval command's bitwise operators. 
https://docs.splunk.com/Documentation/SplunkCloud/9.1.2308/SearchReference/BitFunctions
Your query will look like: 

* | head 1 | eval x=2 | eval y=2 | eval z = bit_and(x, y) | table x y z

 

0 Karma

javiergn
Super Champion

You could write your command in Python. See this and this

Alternatively, try the following query I've written as I was really curious on how to achieve this from Splunk. The logic should work fine if you have more than 2 numbers, simply use another appendcols and play with the bitwise variables. This works with numbers up to 2^20 - 1, simply increase all the mvrange parameters if you want it to be able to cope with bigger numbers.

EDIT1: to use your naming convention.
EDIT2: also calculating Z back at the bottom
EDIT3: quick performance improvements and now you don't have to specify X and Y twice

| stats count
| fields - count
| eval x = 63

| eval power = mvrange(0,20)
| mvexpand power
| eval base2 = pow(2, power)
| where x >= base2
| eval mydiv = floor(x / base2)
| eval mybin = mydiv % 2
| eval binx = mybin

| appendcols [
   | stats count
   | fields - count
   | eval y = 2

   | eval power = mvrange(0,20)
   | mvexpand power
   | eval base2 = pow(2, power)
   | where y >= base2
   | eval mydiv = floor(y / base2)
   | eval mybin = mydiv % 2
   | eval biny = mybin
 ]
| fillnull
| eval bitwiseOr = round((binx + biny) / 2)
| eval bitwiseAnd = (binx * biny)
| eval zAnd = bitwiseAnd * base2
| eval zOr = bitwiseOr * base2
| stats max(x) as x, max(y) as y, list(binx) as binx, list(biny) as biny, list(bitwiseOr) as bitwiseOr, list(bitwiseAnd) as bitwiseAnd, sum(zAnd) as zAnd, sum(zOr) as zOr

And this is the result (see picture below):

alt text

javiergn
Super Champion

Example with 3 numbers (x, y, w):

| stats count
| fields - count
| eval x = 19

| eval power = mvrange(0,21)
| mvexpand power
| eval base2 = pow(2, power)
| where x >= base2
| eval mydiv = floor(x / base2)
| eval mybin = mydiv % 2
| eval binx = mybin

| appendcols [
   | stats count
   | fields - count
   | eval y = 3

   | eval power = mvrange(0,20)
   | mvexpand power
   | eval base2 = pow(2, power)
   | where y >= base2
   | eval mydiv = floor(y / base2)
   | eval mybin = mydiv % 2
   | eval biny = mybin
]

| appendcols [
   | stats count
   | fields - count
   | eval w = 1

   | eval power = mvrange(0,20)
   | mvexpand power
   | eval base2 = pow(2, power)
   | where w >= base2
   | eval mydiv = floor(w / base2)
   | eval mybin = mydiv % 2
   | eval binw = mybin
 ]

| fillnull
| eval bitwiseOr = round((binx + biny) / 2)
| eval bitwiseOr = round((bitwiseOr + binw) / 2)
| eval bitwiseAnd = (binx * biny * binw)
| eval zAnd = bitwiseAnd * base2
| eval zOr = bitwiseOr * base2
| stats max(x) as x, max(y) as y, max(w) as w, list(binx) as binx, list(biny) as biny, list(binw) as binw, list(bitwiseOr) as bitwiseOr, list(bitwiseAnd) as bitwiseAnd, sum(zAnd) as zAnd, sum(zOr) as zOr
0 Karma

javiergn
Super Champion

I have edited the above a couple of times:

EDIT1: to use your naming convention.
EDIT2: also calculating Z back at the bottom
EDIT3: quick performance improvements and now you don't have to specify X and Y twice

0 Karma

whisperstream
Explorer

Wow, thanks for this, amazing how much work it is for such a basic operation, hopefully someone from Splunk will take notice and add the features to the both eval and stats functions.

Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Wednesday, May 29, 2024  |  11AM PST / 2PM ESTRegister now and join us to learn more about how you can ...

Get Your Exclusive Splunk Certified Cybersecurity Defense Engineer Certification at ...

We’re excited to announce a new Splunk certification exam being released at .conf24! If you’re headed to Vegas ...