topic Re: Extracting fields from JSON file format in Getting Data In

Extracting fields from JSON file format

abhayneilam — Tue, 25 Feb 2014 19:19:12 GMT

Hi,
I want to extract the fields from JSON File format and after extrating the fileds. I want to remove one of the value of a particular key, such as :

My input data :

{"TID":"111222","EID":"44567","XXX":"00000x<?xml?>aasewl85765"}

Now, I want to remove the "XXX":"00000x<?xml?>aasewl85765" .. This should not be indexed other parts should be indexed like {"TID":"111222","EID":"44567" }

In my Splunk only two fields should come "TID" and "EID".

Please suggest the best possible way !!

Re: Extracting fields from JSON file format

dshpritz — Tue, 25 Feb 2014 21:28:15 GMT

Your order is messed up. First you want to filter out the data (prior to indexing) and then (on search) you extract the fields.

To filter, in your props.conf:

[mysourcetype]
SEDCMD-removefields = s/{"TID":"\d+","EID":"\d+"(.*)/}/g

To extract the fields, you might be able to use Splunk's automagical field extraction. In the same props.conf:

[mysourcetype]
KV_MODE = json

So, all together:

[mysourcetype]
KV_MODE = json
SEDCMD-removefields = s/{"TID":"\d+","EID":"\d+"(.*)/}/g

This will need to be done on your search heads and indexers.

Re: Extracting fields from JSON file format

abhayneilam — Mon, 28 Sep 2020 15:58:47 GMT

I want to have a separate events for each line of JSON file. I am getting all the lines merged with a single events. Please let me know how can I separate the events:

My Props.conf is :

[json]
KV_MODE = json
LINE_BREAKER = "}$"
NO_BINARY_CHECK = 1
TRUNCATE = 0
SHOULD_LINEMERGE = false

Please help

Re: Extracting fields from JSON file format

laserval — Wed, 26 Feb 2014 11:42:02 GMT

Have you tried without setting a LINE_BREAKER? When I've indexed JSON, it has worked well out of the box as long as the file has the format

{...}
{...}

i.e. each line is a separate JSON object (which it looks liek you have)

Re: Extracting fields from JSON file format

abhayneilam — Wed, 26 Feb 2014 13:57:29 GMT

I want to remove one of the field and its value from the file to be indexed. Please help me

Regards,
Abhay

Re: Extracting fields from JSON file format

dshpritz — Wed, 26 Feb 2014 15:04:15 GMT

Your linebreaker doesn't have a capture, which may be the problem. I would set the linebreaker to:

([\r\n]){

Note that that is the start of the line, not the end of it.

Re: Extracting fields from JSON file format

abhayneilam — Thu, 27 Feb 2014 09:01:06 GMT

"Note that that is the start of the line, not the end of it." This clue helped me to get a solution for my problem.

Thanks a lot !!