How to filter a JSON data log when one of the fields in that JSON is empty?

[mayurk90]

Hi,
I am trying to filter the log event based on a json field which is empty. I have 3 million records and out of which 2 are having those field empty which I am trying to extract log for.

The json looks like this:

 "third": [
    {
      "ad": {
        "dd": "aaa",
        "value": "",                           <-----------this is the field which I want to search on when its empty
        "version": 1,
        "do": "bbb"
      },

So in this case how to search that kind of log?

[FrankVl · Accepted solution]

Assuming you already have json extraction working in general. Something like this should work (I believe the field will be missing when there is no value for it in the json):

index="foo" sourcetype="bar" NOT third.ad.value=*

So basically just search for NOT <fieldname>!=*.

Unless the automatic json extraction actually does extract that event with a value of empty string, then you could search for it like this:

index="foo" sourcetype="bar" third.ad.value=""

[mayurk90]

Also, I am using splunk web so don't know how to turn json extraction on.

[mayurk90]

I was going through some more details since I am new on splunk and my datasource shows as logstash.

[FrankVl]

Automatic JSON extractions should be enabled by default, but perhaps the specific sourcetype you assigned (or splunk chose to assign) has it disabled for some reason.

Can you provide some more details on how you got this data into splunk and perhaps some relevant screenshots showing the data, sourcetype value, which fields get extracted and such?

[mayurk90]

Basically my data is in the format like below:

@timestamp: 2019-06-19T12:32:23.591Z

@version: 1

app_id: 90333

host: fgfjfjfj00053141.server.net
message: 2019-06-19 08:32:23,373 INFO [AMPS Java Client Background Reader Thread ***] "third": [
{
"ad": {
"dd": "aaa",
"value": "", <-----------this is the field which I want to search on when its empty
"version": 1,
"do": "bbb"
}]
app_id = 90333 host = fgfjfjfj00053141.server.net host =hdhdhdhd.net

source = /apps/uat01/logs/abc-logger.log

sourcetype = logstash

So the json data is part of value field of message key field and from there I want to check if the field is empty then show me those events having empty field.

[FrankVl]

Ok, so the event is not fully json. Which means automatic json extractions won't work.

Did you do any field extraction for that field yet? Or do you at the moment only have a raw event in splunk?

[mayurk90]

I just have raw event in splunk for now

[FrankVl]

Then just add "\"value\": \"\"" to your search query.

So for example:

index="foo" sourcetype="bar" "\"value\": \"\""

[mayurk90]

My logs are In Json only but I don't understand how to activate json extraction?