How to filter multivalue null values?

[karthi2809]

Hi All,

I want to filter out null values.In my field the ImpCon having null values.Now i want to filter the values which i dont want to show in the table.I am trying below query .which is showing the null values.

| eval ImpCon=mvmap(ImpConReqID,if(match(ImpConReqID,".+"),"ImpConReqID: ".ImpConReqID,null()))
| eval orcaleid=mvfilter(isnotnull(oracle))
| eval OracleResponse=mvjoin(orcaleid," ")

[marysan]

Assume that ImpCon is a multivalue field from which you want to remove null values this works for you :

| eval ImpCon=mvfilter(isnotnull(ImpCon) AND ImpCon!="") 

[bowesmana]

What do you mean it's showing null values - your mvmap statement looks like it's doing what you want it to do, i.e. making sure that it only takes data with at least 1 character.

Can you demonstrate the issue as the mvmap statement works, i.e. this example shows that it will remove the empty middle element

| makeresults
| fields - _time
| eval ImpConReqID=mvappend("a","","b")
| eval ImpCon=mvmap(ImpConReqID,if(match(ImpConReqID,".+"),"ImpConReqID: ".ImpConReqID, null()))
| eval base_elements=mvcount(ImpConReqID)
| eval reduced_elements=mvcount(ImpCon)

What is the relevance of the 2nd two lines of your example to your question?

[karthi2809]

Thanks,

Provided query which i am trying to do.

[tej57]

Hello @karthi2809 ,

I do not understand the use of mvmap command here. Generally, mvmap command is used to perform some iterative operations on the multivalue field. Your SPL currently interpretes as you're trying to map ImpConReqId field with  following string: "ImpConReqId: <<value of ImpConReqId>>". And if the "if condition" fails, the value gets updated to null() and then ImpConReqId gets mapped with null() value.

I would suggest you to first filter out the null values using isnull() or isnotnull() functions and then perform multi value operations. Also, if you can share the full SPL query, it would be helpful to assist you better.

Thanks,
Tejas.

[karthi2809]

index=mulesoft applicationName=test
| stats values(content.payload.requestID) as Request1 values(content.payload.impConReqId) as ImpConReqId  values(content.payload.batchId) as batch1  values(content.payload{}.batchId) as batch2 values(content.payload{}.impConReqId) as impConReqId1 values(content.payload.OutputParameters.X_REQUEST_ID
) as Request2 BY applicationName,correlationId 
| eval ImpConReqID= coalesce(ImpConReqId,impConReqId1) 
| eval RequestId= coalesce(Request1,Request2) 
| eval batchId= coalesce(batch1,batch2) 
| eval ImpCon=mvmap(ImpConReqID,if(match(ImpConReqID,".+"),"ImpConReqID: ".ImpConReqID,null())) 
| eval batch=mvmap(batchId,if(match(batchId,".+"),"batchId: ".batchId,null()))  
| eval ReqId=mvmap(RequestId,if(match(RequestId,".+"),"RequestId: ".RequestId,null()))  
| eval oracle=mvappend(ImpCon,batch,ReqId) 
| eval orcaleid=mvfilter(isnotnull(oracle)) 
| eval OracleResponse=mvjoin(orcaleid," ") 
| rename  applicationName as ApplicationName correlationId as CorrelationId 
| table ApplicationName OracleResponse CorrelationId

This the query which i am trying to get batchID, requestID, ImpconID.If the field value contains then i need to show in the table based on correlationID. Right now I am getting values properly. But in some scenario for the particular correlationID we have two or three ImpconIDwith values and with null values. So i want filter that null value ImpconId in the table .

[bowesmana]

Are you clear on the difference between NULL and EMPTY - your mvmap is checking for non-EMPTY values of one of the values of the MV field, it is not checking for NULL

This pair of lines is looping each of the MV values of impConReqID and removing only EMPTY values 

| eval ImpConReqID= coalesce(ImpConReqId,impConReqId1) 
...
| eval ImpCon=mvmap(ImpConReqID,if(match(ImpConReqID,".+"),"ImpConReqID: ".ImpConReqID,null())) 

so if you have real null values in your MV, then you need to check for null, not empty, i.e.

| eval ImpCon=mvmap(ImpConReqID,if(isnotnull(ImpConReqID),"ImpConReqID: ".ImpConReqID,null())) 

[karthi2809]

Hi @bowesmana 

Still the null field values is appearing.

[bowesmana]

Did you understand my comment about the difference between null and empty?

Please confirm that these are null values you are taking about rather than empty values and provide some evidence that you actually have null values. Without that it's impossible to know what is going on

[karthi2809]

Hi @bowesmana 

My actual  requirement is that if the field with empty values then I dont want to show in the table.IF some of the correlationID we dont have ImpconID so i used above query to filter the empty values. Now i want to filter the null values from the field. PFA

[bowesmana · Accepted solution]

OK, so it seems you have a misunderstanding of the concept of null in Splunk.

• Null in Splunk means no value, invisible, not a field value
• Empty is a value that has no length

What you have is NOT a null field, it is a field with the text string "null" so to remove values of fields you don't want you can simply do either of these

| eval ImpCon=mvmap(ImpConReqID,if(isnotnull(ImpConReqID) AND ImpConReqID!="null", ImpConReqID, null()))
| eval ImpCon2=mvfilter(ImpConReqID!="null")

[karthi2809]

Thanks for the brief  explanation.