Dashboards & Visualizations

dynamic field value extraction

klee310
Communicator

I'm trying to extract a field-value for comparison - in a dynamic fashion. First let me illustrate the problem with some sample data:

  DataType=2, MaxPower=10, MinPower=3, IdlePower=5
  DataType=3, Open=10, Close=23, HappyHour=15

I have a lookup table with something similar to this:

  DataType,FieldName,ValueMax,ValueMin
  2,MaxPower,100,10
  2,MinPower,10,1
  2,IdlePower,50,1
  3,Open,10,5
  ... etc.

Now my search looks like this:

  index=xxx | lookup LOOKUPFILEX DataType | mvexpand FieldName | ...

I want to map the results into a macro which will perform the actual evaluation, this part is ok. Now I'm trying to write this macro and i'm sort of hitting a wall. For example, the macro might be invoked like this:

  `checkRange("$FieldName$",$ValueMax$,$ValueMin$)`

So from within the macro, given the field name "MaxPower", how do I extract the value (10 in this case) so I can perform comparison with its associated max/min range?

I am currently on the path associated with another question "dynamic field substition"

By the way, the reason I don't want to hard-coding the evaluation (MaxPower, MinPower, IdlePower, Open, etc.) is because - what you see here is just a sample; the actual use-case I am confronted with contains upwards of 100 fields that are subject to change. Therefore modifications must be easily implemented (like via a lookup table)

Tags (3)
0 Karma
1 Solution

somesoni2
Revered Legend

Your search

index=xxx | lookup LOOKUPFILEX DataType | mvexpand FieldName 

Will give all fields present in index=xxx (already extracted by splunk) and fields ValueMin and ValueMax.

Create a macro which will take 4 arguments.

Macro Name: checkRange(4)

Definition:

eval validationResult=if($FieldValue$>=$ValueMin$ AND $FieldValue$<=$ValueMax$,$FieldName$." within range",$FieldName$." out of range")

Arguments : FieldValue,FieldName,ValueMin,ValueMax

Modified search (say validate MinPower field value):

index=xxx | lookup LOOKUPFILEX DataType | mvexpand FieldName |`checkRange(MinPower,"MinPower", ValueMin, ValueMax)`

This should return the field validationResult created in Macro. Obviously, you have to customize this per your requirement, but this should give you fair idea.

View solution in original post

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

Nope I don't. Actually I think you can't do this with macros (which is fundamentally one of the differences you get between macros and true functions). I guess my suggestion might be to use a scripted lookup or a custom search command. If you use a scripted lookup, you could combine it with the LOOKUPFILEX lookup. Or you could keep it separate.

0 Karma

somesoni2
Revered Legend

Your search

index=xxx | lookup LOOKUPFILEX DataType | mvexpand FieldName 

Will give all fields present in index=xxx (already extracted by splunk) and fields ValueMin and ValueMax.

Create a macro which will take 4 arguments.

Macro Name: checkRange(4)

Definition:

eval validationResult=if($FieldValue$>=$ValueMin$ AND $FieldValue$<=$ValueMax$,$FieldName$." within range",$FieldName$." out of range")

Arguments : FieldValue,FieldName,ValueMin,ValueMax

Modified search (say validate MinPower field value):

index=xxx | lookup LOOKUPFILEX DataType | mvexpand FieldName |`checkRange(MinPower,"MinPower", ValueMin, ValueMax)`

This should return the field validationResult created in Macro. Obviously, you have to customize this per your requirement, but this should give you fair idea.

View solution in original post

0 Karma

klee310
Communicator

actually as it turns out, i did write all 100+ macros - but each is just a simple wrapper which calls a main-macro with the appropriate arguments

0 Karma

klee310
Communicator

thanks somesoni for the reply, but I'm afraid this solution doesn't solve the problem I am having. If I had 100+ FieldName(s), how would I go about invoking this "Macro"? Wouldn't I need 100+ savedsearches, each unique to a FieldName?

0 Karma

klee310
Communicator

thanks gkanapathy for the quick response; if I have something like | eval foo=if(MaxPower<=ValueMax AND MaxPower>=ValueMin,1,null())... then MaxPower and MinPower will have to be hard-coded into my macro. Which is not want I want to do. On the other hand, if you are question why I don't use ... eval foo=if($arg1$ <= ValueMax...) well $arg1$ is a string, in this example, it is the string "MaxPower" - so that doesn't work either

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

In other words, why can't you just use if(MaxPower<=ValueMax AND MaxPower >= ValueMin,1,null())?

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

Why can't you just use ValueMin and ValueMax? Isn't the point of the lookup that the max and min are going to be stored in those variables, so you can just them?

0 Karma

klee310
Communicator

also, I want to avoid rex if possible since the field DataType, MaxPower, etc. are already automatically extracted by the search-engine; I just need to find a way to reference those values for comparison based on the name of the field dynamically

0 Karma
.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!