Splunk Search

Is it possible to perform bitwise operations on values in the eval function?

whisperstream
Explorer

In the following query I'm trying to get the logical AND of two numbers:

 * | head 1 | eval x=2 | eval y=2 | eval z = and(x, y)) | table x y z

Ideally the result would be:

x y z
2 2 2

because function "and" above would be similar to python's:
LOGICAL AND: x & y

So it's easy in pseudo code/query, but is there a way to actually do this, or a plugin that provides these operations? I wasn't able to find it in the command reference.

0 Karma
1 Solution

javiergn
Super Champion

You could write your command in Python. See this and this

Alternatively, try the following query I've written as I was really curious on how to achieve this from Splunk. The logic should work fine if you have more than 2 numbers, simply use another appendcols and play with the bitwise variables. This works with numbers up to 2^20 - 1, simply increase all the mvrange parameters if you want it to be able to cope with bigger numbers.

EDIT1: to use your naming convention.
EDIT2: also calculating Z back at the bottom
EDIT3: quick performance improvements and now you don't have to specify X and Y twice

| stats count
| fields - count
| eval x = 63

| eval power = mvrange(0,20)
| mvexpand power
| eval base2 = pow(2, power)
| where x >= base2
| eval mydiv = floor(x / base2)
| eval mybin = mydiv % 2
| eval binx = mybin

| appendcols [
   | stats count
   | fields - count
   | eval y = 2

   | eval power = mvrange(0,20)
   | mvexpand power
   | eval base2 = pow(2, power)
   | where y >= base2
   | eval mydiv = floor(y / base2)
   | eval mybin = mydiv % 2
   | eval biny = mybin
 ]
| fillnull
| eval bitwiseOr = round((binx + biny) / 2)
| eval bitwiseAnd = (binx * biny)
| eval zAnd = bitwiseAnd * base2
| eval zOr = bitwiseOr * base2
| stats max(x) as x, max(y) as y, list(binx) as binx, list(biny) as biny, list(bitwiseOr) as bitwiseOr, list(bitwiseAnd) as bitwiseAnd, sum(zAnd) as zAnd, sum(zOr) as zOr

And this is the result (see picture below):

alt text

View solution in original post

jaypatel
Splunk Employee
Splunk Employee

Now you can use the eval command's bitwise operators. 
https://docs.splunk.com/Documentation/SplunkCloud/9.1.2308/SearchReference/BitFunctions
Your query will look like: 

* | head 1 | eval x=2 | eval y=2 | eval z = bit_and(x, y) | table x y z

 

0 Karma

javiergn
Super Champion

You could write your command in Python. See this and this

Alternatively, try the following query I've written as I was really curious on how to achieve this from Splunk. The logic should work fine if you have more than 2 numbers, simply use another appendcols and play with the bitwise variables. This works with numbers up to 2^20 - 1, simply increase all the mvrange parameters if you want it to be able to cope with bigger numbers.

EDIT1: to use your naming convention.
EDIT2: also calculating Z back at the bottom
EDIT3: quick performance improvements and now you don't have to specify X and Y twice

| stats count
| fields - count
| eval x = 63

| eval power = mvrange(0,20)
| mvexpand power
| eval base2 = pow(2, power)
| where x >= base2
| eval mydiv = floor(x / base2)
| eval mybin = mydiv % 2
| eval binx = mybin

| appendcols [
   | stats count
   | fields - count
   | eval y = 2

   | eval power = mvrange(0,20)
   | mvexpand power
   | eval base2 = pow(2, power)
   | where y >= base2
   | eval mydiv = floor(y / base2)
   | eval mybin = mydiv % 2
   | eval biny = mybin
 ]
| fillnull
| eval bitwiseOr = round((binx + biny) / 2)
| eval bitwiseAnd = (binx * biny)
| eval zAnd = bitwiseAnd * base2
| eval zOr = bitwiseOr * base2
| stats max(x) as x, max(y) as y, list(binx) as binx, list(biny) as biny, list(bitwiseOr) as bitwiseOr, list(bitwiseAnd) as bitwiseAnd, sum(zAnd) as zAnd, sum(zOr) as zOr

And this is the result (see picture below):

alt text

javiergn
Super Champion

Example with 3 numbers (x, y, w):

| stats count
| fields - count
| eval x = 19

| eval power = mvrange(0,21)
| mvexpand power
| eval base2 = pow(2, power)
| where x >= base2
| eval mydiv = floor(x / base2)
| eval mybin = mydiv % 2
| eval binx = mybin

| appendcols [
   | stats count
   | fields - count
   | eval y = 3

   | eval power = mvrange(0,20)
   | mvexpand power
   | eval base2 = pow(2, power)
   | where y >= base2
   | eval mydiv = floor(y / base2)
   | eval mybin = mydiv % 2
   | eval biny = mybin
]

| appendcols [
   | stats count
   | fields - count
   | eval w = 1

   | eval power = mvrange(0,20)
   | mvexpand power
   | eval base2 = pow(2, power)
   | where w >= base2
   | eval mydiv = floor(w / base2)
   | eval mybin = mydiv % 2
   | eval binw = mybin
 ]

| fillnull
| eval bitwiseOr = round((binx + biny) / 2)
| eval bitwiseOr = round((bitwiseOr + binw) / 2)
| eval bitwiseAnd = (binx * biny * binw)
| eval zAnd = bitwiseAnd * base2
| eval zOr = bitwiseOr * base2
| stats max(x) as x, max(y) as y, max(w) as w, list(binx) as binx, list(biny) as biny, list(binw) as binw, list(bitwiseOr) as bitwiseOr, list(bitwiseAnd) as bitwiseAnd, sum(zAnd) as zAnd, sum(zOr) as zOr
0 Karma

javiergn
Super Champion

I have edited the above a couple of times:

EDIT1: to use your naming convention.
EDIT2: also calculating Z back at the bottom
EDIT3: quick performance improvements and now you don't have to specify X and Y twice

0 Karma

whisperstream
Explorer

Wow, thanks for this, amazing how much work it is for such a basic operation, hopefully someone from Splunk will take notice and add the features to the both eval and stats functions.

Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...