topic Re: How to filter multivalue null values? in Splunk Search

How to filter multivalue null values?

karthi2809 — Fri, 31 May 2024 17:15:10 GMT

Hi All,

I want to filter out null values.In my field the ImpCon having null values.Now i want to filter the values which i dont want to show in the table.I am trying below query .which is showing the null values.

| eval ImpCon=mvmap(ImpConReqID,if(match(ImpConReqID,".+"),"ImpConReqID: ".ImpConReqID,null())) | eval orcaleid=mvfilter(isnotnull(oracle)) | eval OracleResponse=mvjoin(orcaleid," ")

Re: How to filter multivalue null values?

tej57 — Thu, 30 May 2024 05:46:01 GMT

Hello @karthi2809 ,

I do not understand the use of mvmap command here. Generally, mvmap command is used to perform some iterative operations on the multivalue field. Your SPL currently interpretes as you're trying to map ImpConReqId field with following string: "ImpConReqId: <<value of ImpConReqId>>". And if the "if condition" fails, the value gets updated to null() and then ImpConReqId gets mapped with null() value.

I would suggest you to first filter out the null values using isnull() or isnotnull() functions and then perform multi value operations. Also, if you can share the full SPL query, it would be helpful to assist you better.

Thanks,
Tejas.

Re: How to filter multivalue null values?

bowesmana — Thu, 30 May 2024 06:44:57 GMT

What do you mean it's showing null values - your mvmap statement looks like it's doing what you want it to do, i.e. making sure that it only takes data with at least 1 character.

Can you demonstrate the issue as the mvmap statement works, i.e. this example shows that it will remove the empty middle element

What is the relevance of the 2nd two lines of your example to your question?

Re: How to filter multivalue null values?

karthi2809 — Thu, 30 May 2024 07:46:51 GMT

index=mulesoft applicationName=test | stats values(content.payload.requestID) as Request1 values(content.payload.impConReqId) as ImpConReqId values(content.payload.batchId) as batch1 values(content.payload{}.batchId) as batch2 values(content.payload{}.impConReqId) as impConReqId1 values(content.payload.OutputParameters.X_REQUEST_ID ) as Request2 BY applicationName,correlationId | eval ImpConReqID= coalesce(ImpConReqId,impConReqId1) | eval RequestId= coalesce(Request1,Request2) | eval batchId= coalesce(batch1,batch2) | eval ImpCon=mvmap(ImpConReqID,if(match(ImpConReqID,".+"),"ImpConReqID: ".ImpConReqID,null())) | eval batch=mvmap(batchId,if(match(batchId,".+"),"batchId: ".batchId,null())) | eval ReqId=mvmap(RequestId,if(match(RequestId,".+"),"RequestId: ".RequestId,null())) | eval oracle=mvappend(ImpCon,batch,ReqId) | eval orcaleid=mvfilter(isnotnull(oracle)) | eval OracleResponse=mvjoin(orcaleid," ") | rename applicationName as ApplicationName correlationId as CorrelationId | table ApplicationName OracleResponse CorrelationId

This the query which i am trying to get batchID, requestID, ImpconID.If the field value contains then i need to show in the table based on correlationID. Right now I am getting values properly. But in some scenario for the particular correlationID we have two or three ImpconIDwith values and with null values. So i want filter that null value ImpconId in the table .

Re: How to filter multivalue null values?

karthi2809 — Thu, 30 May 2024 07:53:35 GMT

Thanks,

Provided query which i am trying to do.

Re: How to filter multivalue null values?

bowesmana — Fri, 31 May 2024 02:21:49 GMT

Are you clear on the difference between NULL and EMPTY - your mvmap is checking for non-EMPTY values of one of the values of the MV field, it is not checking for NULL

This pair of lines is looping each of the MV values of impConReqID and removing only EMPTY values

| eval ImpConReqID= coalesce(ImpConReqId,impConReqId1) ... | eval ImpCon=mvmap(ImpConReqID,if(match(ImpConReqID,".+"),"ImpConReqID: ".ImpConReqID,null()))

so if you have real null values in your MV, then you need to check for null, not empty, i.e.

| eval ImpCon=mvmap(ImpConReqID,if(isnotnull(ImpConReqID),"ImpConReqID: ".ImpConReqID,null()))

Re: How to filter multivalue null values?

karthi2809 — Fri, 31 May 2024 10:37:33 GMT

Hi @bowesmana

Still the null field values is appearing.

Re: How to filter multivalue null values?

bowesmana — Fri, 31 May 2024 11:23:52 GMT

Did you understand my comment about the difference between null and empty?

Please confirm that these are null values you are taking about rather than empty values and provide some evidence that you actually have null values. Without that it's impossible to know what is going on

Re: How to filter multivalue null values?

karthi2809 — Fri, 31 May 2024 11:36:46 GMT

Hi @bowesmana

My actual requirement is that if the field with empty values then I dont want to show in the table.IF some of the correlationID we dont have ImpconID so i used above query to filter the empty values. Now i want to filter the null values from the field. PFA

Re: How to filter multivalue null values?

marysan — Fri, 31 May 2024 17:16:26 GMT

Assume that ImpCon is a multivalue field from which you want to remove null values this works for you :

| eval ImpCon=mvfilter(isnotnull(ImpCon) AND ImpCon!="")

Re: How to filter multivalue null values?

bowesmana — Sun, 02 Jun 2024 23:50:20 GMT

OK, so it seems you have a misunderstanding of the concept of null in Splunk.

Null in Splunk means no value, invisible, not a field value
Empty is a value that has no length

What you have is NOT a null field, it is a field with the text string "null" so to remove values of fields you don't want you can simply do either of these

| eval ImpCon=mvmap(ImpConReqID,if(isnotnull(ImpConReqID) AND ImpConReqID!="null", ImpConReqID, null())) | eval ImpCon2=mvfilter(ImpConReqID!="null")

Re: How to filter multivalue null values?

karthi2809 — Mon, 03 Jun 2024 15:08:08 GMT

Thanks for the brief explanation.