topic Re: Is it possible to perform bitwise operations on values in the eval function? in Splunk Search

Is it possible to perform bitwise operations on values in the eval function?

whisperstream — Thu, 14 Jan 2016 00:01:30 GMT

In the following query I'm trying to get the logical AND of two numbers:

 * | head 1 | eval x=2 | eval y=2 | eval z = and(x, y)) | table x y z

Ideally the result would be:

x y z
2 2 2

because function "and" above would be similar to python's:
LOGICAL AND: x & y

So it's easy in pseudo code/query, but is there a way to actually do this, or a plugin that provides these operations? I wasn't able to find it in the command reference.

Re: Is it possible to perform bitwise operations on values in the eval function?

javiergn — Thu, 14 Jan 2016 11:40:44 GMT

You could write your command in Python. See this and this

Alternatively, try the following query I've written as I was really curious on how to achieve this from Splunk. The logic should work fine if you have more than 2 numbers, simply use another appendcols and play with the bitwise variables. This works with numbers up to 2^20 - 1, simply increase all the mvrange parameters if you want it to be able to cope with bigger numbers.

EDIT1: to use your naming convention.
EDIT2: also calculating Z back at the bottom
EDIT3: quick performance improvements and now you don't have to specify X and Y twice

| stats count
| fields - count
| eval x = 63

| eval power = mvrange(0,20)
| mvexpand power
| eval base2 = pow(2, power)
| where x >= base2
| eval mydiv = floor(x / base2)
| eval mybin = mydiv % 2
| eval binx = mybin

| appendcols [
   | stats count
   | fields - count
   | eval y = 2

   | eval power = mvrange(0,20)
   | mvexpand power
   | eval base2 = pow(2, power)
   | where y >= base2
   | eval mydiv = floor(y / base2)
   | eval mybin = mydiv % 2
   | eval biny = mybin
 ]
| fillnull
| eval bitwiseOr = round((binx + biny) / 2)
| eval bitwiseAnd = (binx * biny)
| eval zAnd = bitwiseAnd * base2
| eval zOr = bitwiseOr * base2
| stats max(x) as x, max(y) as y, list(binx) as binx, list(biny) as biny, list(bitwiseOr) as bitwiseOr, list(bitwiseAnd) as bitwiseAnd, sum(zAnd) as zAnd, sum(zOr) as zOr

And this is the result (see picture below):

Re: Is it possible to perform bitwise operations on values in the eval function?

javiergn — Thu, 14 Jan 2016 15:15:01 GMT

I have edited the above a couple of times:

EDIT1: to use your naming convention.
EDIT2: also calculating Z back at the bottom
EDIT3: quick performance improvements and now you don't have to specify X and Y twice

Re: Is it possible to perform bitwise operations on values in the eval function?

javiergn — Thu, 14 Jan 2016 15:49:20 GMT

Example with 3 numbers (x, y, w):

| stats count
| fields - count
| eval x = 19

| eval power = mvrange(0,21)
| mvexpand power
| eval base2 = pow(2, power)
| where x >= base2
| eval mydiv = floor(x / base2)
| eval mybin = mydiv % 2
| eval binx = mybin

| appendcols [
   | stats count
   | fields - count
   | eval y = 3

   | eval power = mvrange(0,20)
   | mvexpand power
   | eval base2 = pow(2, power)
   | where y >= base2
   | eval mydiv = floor(y / base2)
   | eval mybin = mydiv % 2
   | eval biny = mybin
]

| appendcols [
   | stats count
   | fields - count
   | eval w = 1

   | eval power = mvrange(0,20)
   | mvexpand power
   | eval base2 = pow(2, power)
   | where w >= base2
   | eval mydiv = floor(w / base2)
   | eval mybin = mydiv % 2
   | eval binw = mybin
 ]

| fillnull
| eval bitwiseOr = round((binx + biny) / 2)
| eval bitwiseOr = round((bitwiseOr + binw) / 2)
| eval bitwiseAnd = (binx * biny * binw)
| eval zAnd = bitwiseAnd * base2
| eval zOr = bitwiseOr * base2
| stats max(x) as x, max(y) as y, max(w) as w, list(binx) as binx, list(biny) as biny, list(binw) as binw, list(bitwiseOr) as bitwiseOr, list(bitwiseAnd) as bitwiseAnd, sum(zAnd) as zAnd, sum(zOr) as zOr

Re: Is it possible to perform bitwise operations on values in the eval function?

whisperstream — Thu, 14 Jan 2016 16:04:38 GMT

Wow, thanks for this, amazing how much work it is for such a basic operation, hopefully someone from Splunk will take notice and add the features to the both eval and stats functions.

Re: Is it possible to perform bitwise operations on values in the eval function?

jaypatel — Thu, 18 Jan 2024 02:54:48 GMT

Now you can use the eval command's bitwise operators.
https://docs.splunk.com/Documentation/SplunkCloud/9.1.2308/SearchReference/BitFunctions
Your query will look like:

* | head 1 | eval x=2 | eval y=2 | eval z = bit_and(x, y) | table x y z