topic Re: How to filter a JSON data log when one of the fields in that JSON is empty? in Splunk Search

How to filter a JSON data log when one of the fields in that JSON is empty?

mayurk90 — Mon, 17 Jun 2019 14:05:00 GMT

Hi,
I am trying to filter the log event based on a json field which is empty. I have 3 million records and out of which 2 are having those field empty which I am trying to extract log for.

The json looks like this:

 "third": [
    {
      "ad": {
        "dd": "aaa",
        "value": "",                           <-----------this is the field which I want to search on when its empty
        "version": 1,
        "do": "bbb"
      },

So in this case how to search that kind of log?

Re: How to filter a JSON data log when one of the fields in that JSON is empty?

FrankVl — Mon, 17 Jun 2019 17:12:20 GMT

Assuming you already have json extraction working in general. Something like this should work (I believe the field will be missing when there is no value for it in the json):

index="foo" sourcetype="bar" NOT third.ad.value=*

So basically just search for NOT <fieldname>!=*.

Unless the automatic json extraction actually does extract that event with a value of empty string, then you could search for it like this:

index="foo" sourcetype="bar" third.ad.value=""

Re: How to filter a JSON data log when one of the fields in that JSON is empty?

mayurk90 — Mon, 17 Jun 2019 18:24:22 GMT

My logs are In Json only but I don't understand how to activate json extraction?

Re: How to filter a JSON data log when one of the fields in that JSON is empty?

mayurk90 — Mon, 17 Jun 2019 18:29:34 GMT

Also, I am using splunk web so don't know how to turn json extraction on.

Re: How to filter a JSON data log when one of the fields in that JSON is empty?

mayurk90 — Mon, 17 Jun 2019 19:35:38 GMT

I was going through some more details since I am new on splunk and my datasource shows as logstash.

Re: How to filter a JSON data log when one of the fields in that JSON is empty?

FrankVl — Tue, 18 Jun 2019 06:56:25 GMT

Automatic JSON extractions should be enabled by default, but perhaps the specific sourcetype you assigned (or splunk chose to assign) has it disabled for some reason.

Can you provide some more details on how you got this data into splunk and perhaps some relevant screenshots showing the data, sourcetype value, which fields get extracted and such?

Re: How to filter a JSON data log when one of the fields in that JSON is empty?

mayurk90 — Wed, 30 Sep 2020 01:00:37 GMT

Basically my data is in the format like below:

@timestamp: 2019-06-19T12:32:23.591Z

@version: 1

app_id: 90333

host: fgfjfjfj00053141.server.net
message: 2019-06-19 08:32:23,373 INFO [AMPS Java Client Background Reader Thread ***] "third": [
{
"ad": {
"dd": "aaa",
"value": "", <-----------this is the field which I want to search on when its empty
"version": 1,
"do": "bbb"
}]
app_id = 90333 host = fgfjfjfj00053141.server.net host =hdhdhdhd.net

source = /apps/uat01/logs/abc-logger.log

sourcetype = logstash

So the json data is part of value field of message key field and from there I want to check if the field is empty then show me those events having empty field.

Re: How to filter a JSON data log when one of the fields in that JSON is empty?

FrankVl — Wed, 19 Jun 2019 13:32:50 GMT

Ok, so the event is not fully json. Which means automatic json extractions won't work.

Did you do any field extraction for that field yet? Or do you at the moment only have a raw event in splunk?

Re: How to filter a JSON data log when one of the fields in that JSON is empty?

mayurk90 — Wed, 19 Jun 2019 17:27:16 GMT

I just have raw event in splunk for now

Re: How to filter a JSON data log when one of the fields in that JSON is empty?

FrankVl — Wed, 19 Jun 2019 17:32:51 GMT

Then just add "\"value\": \"\"" to your search query.

So for example:

index="foo" sourcetype="bar" "\"value\": \"\""