How to compare predefined fields against Splunk search query? And then for each mismatch in result?

[chandysir]

Hello All,

I am new to Splunk.

My Splunk index is already getting data from a Kafka source

index=k_index sourcetype=k_message

The query result is something like 
{Field1=abc,Field2=sdfs,Field3=wertw,Field4=123,Field6=87089R....}

I have got a use case where I have a list of fields and associated datatypes, 

I want to compare these predefined fields (fields only - no values) against the Splunk search query results and then for each mismatch in the result, needs to keep count of it and produce it as a percentage of the total. In short, give a score if the incoming events in the last 15 mins are good (like 100% or 90% ….etc)

Thanks,

Alwyn

[ITWhisperer]

You could try something like this

| rex "(?<good>\{Field1=.+,Field2=.+,Field3=.+,Field4=.+,Field6=.+\}"
| stats count count(good) as good
| eval percentage=round(100*good/count, 2)

[chandysir]

Thankyou @ITWhisperer , This is a good one.

The fields are already parsed out by splunk. 

So I am thinking of something without using regular expressions.

In simple terms it could be like

 > if (Field1, Field2... Fieldn) in predefined list (Field1,Field2,Field3.......Fieldm), then its a pass else the failure counter is increased.

Any suggestions ?

[ITWhisperer · Accepted solution]

SPL is not a procedural language - having said that, you could evaluate a field based on the presence of other fields in each event

| eval good=if{isnotnull(Field1) AND isnotnull(Field2) AND isnotnull(Field3) AND isnotnull(Field4) AND isnotnull(Field6), "good", null())
| stats count count(good) as good
| eval percentage=round(100*good/count, 2)

[chandysir]

Thankyou @ITWhisperer ,

Maybe its an overkill but I am thinking of printing the failed event or event identifiers (if percentage is 98, and eventCount is 100, then 2 events)  alongside the result.

Is that possible ? I think that would make a more complex spl search, right.

[PickleRick]

The trick is that they do not have to be extracted. If you search in fast mode, since Splunk doesn't need the field values (they weren't specified either in search itself or calculations in the pipeline), they would not get extracted. Only a single regex match would be performed on each event, which is relatively efficient compared to several separate regexes needed for single fields.