Splunk Search
Highlighted

How to filter a JSON data log when one of the fields in that JSON is empty?

Engager

Hi,
I am trying to filter the log event based on a json field which is empty. I have 3 million records and out of which 2 are having those field empty which I am trying to extract log for.

The json looks like this:

 "third": [
    {
      "ad": {
        "dd": "aaa",
        "value": "",                           <-----------this is the field which I want to search on when its empty
        "version": 1,
        "do": "bbb"
      },

So in this case how to search that kind of log?

0 Karma
Highlighted

Re: How to filter a JSON data log when one of the fields in that JSON is empty?

Ultra Champion

Assuming you already have json extraction working in general. Something like this should work (I believe the field will be missing when there is no value for it in the json):

index="foo" sourcetype="bar" NOT third.ad.value=*

So basically just search for NOT <fieldname>!=*.

Unless the automatic json extraction actually does extract that event with a value of empty string, then you could search for it like this:

index="foo" sourcetype="bar" third.ad.value=""

View solution in original post

Highlighted

Re: How to filter a JSON data log when one of the fields in that JSON is empty?

Engager

My logs are In Json only but I don't understand how to activate json extraction?

0 Karma
Highlighted

Re: How to filter a JSON data log when one of the fields in that JSON is empty?

Engager

Also, I am using splunk web so don't know how to turn json extraction on.

0 Karma
Highlighted

Re: How to filter a JSON data log when one of the fields in that JSON is empty?

Engager

I was going through some more details since I am new on splunk and my datasource shows as logstash.

0 Karma
Highlighted

Re: How to filter a JSON data log when one of the fields in that JSON is empty?

Ultra Champion

Automatic JSON extractions should be enabled by default, but perhaps the specific sourcetype you assigned (or splunk chose to assign) has it disabled for some reason.

Can you provide some more details on how you got this data into splunk and perhaps some relevant screenshots showing the data, sourcetype value, which fields get extracted and such?

0 Karma
Highlighted

Re: How to filter a JSON data log when one of the fields in that JSON is empty?

Engager

Basically my data is in the format like below:

@timestamp: 2019-06-19T12:32:23.591Z

@version: 1

appid: 90333

host: fgfjfjfj00053141.server.net
message: 2019-06-19 08:32:23,373 INFO [AMPS Java Client Background Reader Thread ***] "third": [
{
"ad": {
"dd": "aaa",
"value": "", <-----------this is the field which I want to search on when its empty
"version": 1,
"do": "bbb"
}]
app
id = 90333 host = fgfjfjfj00053141.server.net host =hdhdhdhd.net

source = /apps/uat01/logs/abc-logger.log

sourcetype = logstash

So the json data is part of value field of message key field and from there I want to check if the field is empty then show me those events having empty field.

0 Karma
Highlighted

Re: How to filter a JSON data log when one of the fields in that JSON is empty?

Ultra Champion

Ok, so the event is not fully json. Which means automatic json extractions won't work.

Did you do any field extraction for that field yet? Or do you at the moment only have a raw event in splunk?

0 Karma
Highlighted

Re: How to filter a JSON data log when one of the fields in that JSON is empty?

Engager

I just have raw event in splunk for now

0 Karma
Highlighted

Re: How to filter a JSON data log when one of the fields in that JSON is empty?

Ultra Champion

Then just add "\"value\": \"\"" to your search query.

So for example:

index="foo" sourcetype="bar" "\"value\": \"\""
0 Karma