Splunk Search

After indexing data in Splunk, the field is present in the event, but I can't filter on it

Alaza
Explorer

Hello,

After indexing data, I can find the events in Splunk. All the events have the field_1 equals to 0 or 1.
But unfortunately although the field is visible in the event I can't filter on it.

index="test" | where isnull(field_1)

It returns events, but the events have the field_1 not null. But Splunk doesn't recognize it.
If I filter on field_1=1, it returns the events with field_1=1 but not all, some events have the field_1 not recognized.

I tried to extract a new field on these events, but it didn't work.
Thanks for your help.

0 Karma
1 Solution

DalJeanis
SplunkTrust
SplunkTrust

Okay, first, make sure you are doing this test with verbose mode one. Second, do this

index="test" 
| eval field_1=coalesce(field_1,"None") 
| eval matched=case(field_1!="None",1) 
| stats count sum(matched) as matched by field_1 
| eventstats sum(count) as totalCount sum(matched) as matched 
| eval pctMatched=case(field_1!="None",round(100*count/matched,2)) 
| eval pctTotal=round(100*count/totalcount,2)
| table field_1 count pctTotal pctMatched

The result should tell you how many you have of each type. If there are any with the value "None", then it means your extraction is not acting correctly. (Which we pretty much know.)

Next, do this

index="test"  

Look at the interesting fields to the left, and click on field_1. Look at what percentage of events have this field on them. This should match exactly to the results of the prior test.

If the above results are consistent, then you need to check your extraction and see what is different about the events where the extraction did not occur.

View solution in original post

0 Karma

DalJeanis
SplunkTrust
SplunkTrust

Okay, first, make sure you are doing this test with verbose mode one. Second, do this

index="test" 
| eval field_1=coalesce(field_1,"None") 
| eval matched=case(field_1!="None",1) 
| stats count sum(matched) as matched by field_1 
| eventstats sum(count) as totalCount sum(matched) as matched 
| eval pctMatched=case(field_1!="None",round(100*count/matched,2)) 
| eval pctTotal=round(100*count/totalcount,2)
| table field_1 count pctTotal pctMatched

The result should tell you how many you have of each type. If there are any with the value "None", then it means your extraction is not acting correctly. (Which we pretty much know.)

Next, do this

index="test"  

Look at the interesting fields to the left, and click on field_1. Look at what percentage of events have this field on them. This should match exactly to the results of the prior test.

If the above results are consistent, then you need to check your extraction and see what is different about the events where the extraction did not occur.

0 Karma

Alaza
Explorer

Thanks for the return, the first result show me 17 "None". I checked my extraction.
I found the issue, the extraction of a field before field_1 contain a description with at the end ''.

That's why the field_1 is not extracted.

Thanks for your help.

DalJeanis
SplunkTrust
SplunkTrust

Glad to be of assistance.

0 Karma

niketn
Legend

@Alaza, on the left hand side in the interesting fields if field_1 is showing a value which you feel is incorrect, you can click on the same to add to the search. Please check what search filter gets added. See whether there are any leading/trailing space or something else. If you can please add sample data/screenshot of what actual issues you are seeing.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...