Splunk Search

How to filter row if some fields are empty?

LearningGuy
Motivator

Hello,
How to filter all row if some fields are empty, but do not filter if one of the field has value?   
I appreciate your help. Thank you

I want to filter out row, if vuln, score and company fields are empty/NULL   
(All 3 fields are empty: Row 2 and 6 in the table below)

If vuln OR company fields have values(NOT EMPTY), do not filter 
Row 4: vuln=Empty                            company=company D(NOT empty)
Row 9: vuln=vuln9(NOT empty)    company=empty

If I use the search below, it will filter out row with vuln OR company that are empty (Row 4 and Row 9)
index=testindex  vuln=* AND score=* AND company=*

Current data

noipvulnscorecompany
11.1.1.1vuln19company A
21.1.1.2   
31.1.1.3vuln39company C
41.1.1.4  company D
51.1.1.5vuln57company E
61.1.1.6   
71.1.1.7vuln75company G
81.1.1.8vuln85company H
91.1.1.9vuln9  
101.1.1.10vuln104company J

 

Expected Result: ***NEED CORRECTION***

noipvulnscorecompany
11.1.1.1vuln19company A
2FILTEREDFILTEREDFILTEREDFILTERED
31.1.1.3vuln39company C
41.1.1.4  company D
51.1.1.5vuln57company E
6FILTEREDFILTEREDFILTEREDFILTERED
71.1.1.7vuln75company G
81.1.1.8vuln85company H
91.1.1.9vuln9  
101.1.1.10vuln104company J


Sorry, This is what I mean by FILTERED

noipvulnscorecompany
11.1.1.1vuln19company A
31.1.1.3vuln39company C
41.1.1.4  company D
51.1.1.5vuln57company E
71.1.1.7vuln75company G
81.1.1.8vuln85company H
91.1.1.9vuln9  
Labels (1)
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @LearningGuy ,

the solution to your "Expected Result" is the one hinted by @ITWhisperer .

Instead you can have to the last table simply adding 

 

(vuln=* OR company=*)

 

to you main search.

Ciao.

Giuseppe

 

LearningGuy
Motivator

I found solution for this:

index=testindex  (vuln=* AND score=* AND company=*) OR (vuln=*) OR NOT (company="")

(vuln=* AND score=* AND company=*)   ==>   condition for vuln, score, company exists
(vuln=*)  ==> condition for only vuln exists
NOT (company="") ==> condition for only company exists         

company=*   "is equivalent with"  NOT (company="")     "is equivalent with"    isnull(company)

any idea why company=*  or isnull(company) does not work?
Thank you

0 Karma

bowesmana
SplunkTrust
SplunkTrust

Empty and null are different things. If the field is "" then it is not null, so I use

| where len(company)>0

LearningGuy
Motivator


Hello,
Thank you for your help.

When I use one condition, it worked

 

| where len(company)>0

 


1) but when I combined "len", it didn't work - "The search job has failed due to an error. "

 

| where isnotnull(vuln) AND isnotnull(score) AND len(company>0) 

 


2) Why can't I use  len function without "where"?     
3) Can I use company=* to include "exist/non empty"?          It looks like * also didn't work

Please suggest. Thanks


0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

You have a typo

| where isnotnull(vuln) AND isnotnull(score) AND len(company) > 0

LearningGuy
Motivator

Hello,
Thanks for your correction and your help.
So this is what I am looking for:   

| where (isnotnull(vuln) AND isnotnull(score) AND len(company)>0)) OR (isnotnull(vuln)) OR len(company>0)

 Any idea why * didn't work?

It seems like "where" is faster than "search
Thank you

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

* doesn't work (as a wildcard) for where only search

LearningGuy
Motivator

Hello,
Thank you for your help.
Your answer is correct, the output literally put "FILTERED".    
Sorry if my original post is not clear. I corrected my post.

What I meant by "filtered" , completely removed like shown below:

noipvulnscorecompany
11.1.1.1vuln19company A
31.1.1.3vuln39company C
41.1.1.4  company D
51.1.1.5vuln57company E
71.1.1.7vuln75company G
81.1.1.8vuln85company H
91.1.1.9vuln9  

 

I think I figured it out
index=testindex  (vuln=* AND score=* AND company=*) OR (vuln=*) OR NOT (company="")
It's just weird that company=* does not work and I had to use NOT (company="") to filter out empty 
NOT isnull(company) also doesn't work

Please suggest.
Thanks



0 Karma

ITWhisperer
SplunkTrust
SplunkTrust
| where isnotnull(vuln) OR isnotnull(score) OR isnotnull(company)

ITWhisperer
SplunkTrust
SplunkTrust
| eval ip=if(isnull(vuln) AND isnull(score) AND isnull(company),"FILTERED",ip)
| eval vuln=if(ip="FILTERED",ip,vuln)
| eval score=if(ip="FILTERED",ip,score)
| eval company=if(ip="FILTERED",ip,company)
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Persistent Queue at TcpOut — One of Splunk's Most Practical Features

Splunk introduced persistent queueing at the tcpout layer as one of the most practical resilience features in ...

Skip the Awkward Silence: Have a .conf-ersation at .conf26

Picture this. You arrive at .conf26 already having your socializing and networking plans mapped out. No ...

Rethinking Zero Trust: From Product Purchases to Logical Control Evidence

Implementing Zero Trust (ZT) across complex environments often falters at the very beginning due to a ...