Splunk Search

Is it possible to perform bitwise operations on values in the eval function?

whisperstream
Explorer

In the following query I'm trying to get the logical AND of two numbers:

 * | head 1 | eval x=2 | eval y=2 | eval z = and(x, y)) | table x y z

Ideally the result would be:

x y z
2 2 2

because function "and" above would be similar to python's:
LOGICAL AND: x & y

So it's easy in pseudo code/query, but is there a way to actually do this, or a plugin that provides these operations? I wasn't able to find it in the command reference.

0 Karma
1 Solution

javiergn
Super Champion

You could write your command in Python. See this and this

Alternatively, try the following query I've written as I was really curious on how to achieve this from Splunk. The logic should work fine if you have more than 2 numbers, simply use another appendcols and play with the bitwise variables. This works with numbers up to 2^20 - 1, simply increase all the mvrange parameters if you want it to be able to cope with bigger numbers.

EDIT1: to use your naming convention.
EDIT2: also calculating Z back at the bottom
EDIT3: quick performance improvements and now you don't have to specify X and Y twice

| stats count
| fields - count
| eval x = 63

| eval power = mvrange(0,20)
| mvexpand power
| eval base2 = pow(2, power)
| where x >= base2
| eval mydiv = floor(x / base2)
| eval mybin = mydiv % 2
| eval binx = mybin

| appendcols [
   | stats count
   | fields - count
   | eval y = 2

   | eval power = mvrange(0,20)
   | mvexpand power
   | eval base2 = pow(2, power)
   | where y >= base2
   | eval mydiv = floor(y / base2)
   | eval mybin = mydiv % 2
   | eval biny = mybin
 ]
| fillnull
| eval bitwiseOr = round((binx + biny) / 2)
| eval bitwiseAnd = (binx * biny)
| eval zAnd = bitwiseAnd * base2
| eval zOr = bitwiseOr * base2
| stats max(x) as x, max(y) as y, list(binx) as binx, list(biny) as biny, list(bitwiseOr) as bitwiseOr, list(bitwiseAnd) as bitwiseAnd, sum(zAnd) as zAnd, sum(zOr) as zOr

And this is the result (see picture below):

alt text

View solution in original post

jaypatel
Splunk Employee
Splunk Employee

Now you can use the eval command's bitwise operators. 
https://docs.splunk.com/Documentation/SplunkCloud/9.1.2308/SearchReference/BitFunctions
Your query will look like: 

* | head 1 | eval x=2 | eval y=2 | eval z = bit_and(x, y) | table x y z

 

0 Karma

javiergn
Super Champion

You could write your command in Python. See this and this

Alternatively, try the following query I've written as I was really curious on how to achieve this from Splunk. The logic should work fine if you have more than 2 numbers, simply use another appendcols and play with the bitwise variables. This works with numbers up to 2^20 - 1, simply increase all the mvrange parameters if you want it to be able to cope with bigger numbers.

EDIT1: to use your naming convention.
EDIT2: also calculating Z back at the bottom
EDIT3: quick performance improvements and now you don't have to specify X and Y twice

| stats count
| fields - count
| eval x = 63

| eval power = mvrange(0,20)
| mvexpand power
| eval base2 = pow(2, power)
| where x >= base2
| eval mydiv = floor(x / base2)
| eval mybin = mydiv % 2
| eval binx = mybin

| appendcols [
   | stats count
   | fields - count
   | eval y = 2

   | eval power = mvrange(0,20)
   | mvexpand power
   | eval base2 = pow(2, power)
   | where y >= base2
   | eval mydiv = floor(y / base2)
   | eval mybin = mydiv % 2
   | eval biny = mybin
 ]
| fillnull
| eval bitwiseOr = round((binx + biny) / 2)
| eval bitwiseAnd = (binx * biny)
| eval zAnd = bitwiseAnd * base2
| eval zOr = bitwiseOr * base2
| stats max(x) as x, max(y) as y, list(binx) as binx, list(biny) as biny, list(bitwiseOr) as bitwiseOr, list(bitwiseAnd) as bitwiseAnd, sum(zAnd) as zAnd, sum(zOr) as zOr

And this is the result (see picture below):

alt text

javiergn
Super Champion

Example with 3 numbers (x, y, w):

| stats count
| fields - count
| eval x = 19

| eval power = mvrange(0,21)
| mvexpand power
| eval base2 = pow(2, power)
| where x >= base2
| eval mydiv = floor(x / base2)
| eval mybin = mydiv % 2
| eval binx = mybin

| appendcols [
   | stats count
   | fields - count
   | eval y = 3

   | eval power = mvrange(0,20)
   | mvexpand power
   | eval base2 = pow(2, power)
   | where y >= base2
   | eval mydiv = floor(y / base2)
   | eval mybin = mydiv % 2
   | eval biny = mybin
]

| appendcols [
   | stats count
   | fields - count
   | eval w = 1

   | eval power = mvrange(0,20)
   | mvexpand power
   | eval base2 = pow(2, power)
   | where w >= base2
   | eval mydiv = floor(w / base2)
   | eval mybin = mydiv % 2
   | eval binw = mybin
 ]

| fillnull
| eval bitwiseOr = round((binx + biny) / 2)
| eval bitwiseOr = round((bitwiseOr + binw) / 2)
| eval bitwiseAnd = (binx * biny * binw)
| eval zAnd = bitwiseAnd * base2
| eval zOr = bitwiseOr * base2
| stats max(x) as x, max(y) as y, max(w) as w, list(binx) as binx, list(biny) as biny, list(binw) as binw, list(bitwiseOr) as bitwiseOr, list(bitwiseAnd) as bitwiseAnd, sum(zAnd) as zAnd, sum(zOr) as zOr
0 Karma

javiergn
Super Champion

I have edited the above a couple of times:

EDIT1: to use your naming convention.
EDIT2: also calculating Z back at the bottom
EDIT3: quick performance improvements and now you don't have to specify X and Y twice

0 Karma

whisperstream
Explorer

Wow, thanks for this, amazing how much work it is for such a basic operation, hopefully someone from Splunk will take notice and add the features to the both eval and stats functions.

Get Updates on the Splunk Community!

What's New in Splunk Enterprise 9.4: Features to Power Your Digital Resilience

Hey Splunky People! We are excited to share the latest updates in Splunk Enterprise 9.4. In this release we ...

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...

SignalFlow: What? Why? How?

What is SignalFlow? Splunk Observability Cloud’s analytics engine, SignalFlow, opens up a world of in-depth ...