Extracting fields from JSON file format

[abhayneilam]

Hi,
I want to extract the fields from JSON File format and after extrating the fileds. I want to remove one of the value of a particular key, such as :

My input data :

{"TID":"111222","EID":"44567","XXX":"00000x<?xml?>aasewl85765"}

Now, I want to remove the "XXX":"00000x<?xml?>aasewl85765" .. This should not be indexed other parts should be indexed like {"TID":"111222","EID":"44567" }

In my Splunk only two fields should come "TID" and "EID".

Please suggest the best possible way !!

[dshpritz · Accepted solution]

Your order is messed up. First you want to filter out the data (prior to indexing) and then (on search) you extract the fields.

To filter, in your props.conf:

[mysourcetype]
SEDCMD-removefields = s/{"TID":"\d+","EID":"\d+"(.*)/}/g

To extract the fields, you might be able to use Splunk's automagical field extraction. In the same props.conf:

[mysourcetype]
KV_MODE = json

So, all together:

[mysourcetype]
KV_MODE = json
SEDCMD-removefields = s/{"TID":"\d+","EID":"\d+"(.*)/}/g

This will need to be done on your search heads and indexers.

[abhayneilam]

"Note that that is the start of the line, not the end of it." This clue helped me to get a solution for my problem.

Thanks a lot !!

[dshpritz]

Your linebreaker doesn't have a capture, which may be the problem. I would set the linebreaker to:

([\r\n]){

Note that that is the start of the line, not the end of it.

[abhayneilam]

I want to remove one of the field and its value from the file to be indexed. Please help me

Regards,
Abhay

[laserval]

Have you tried without setting a LINE_BREAKER? When I've indexed JSON, it has worked well out of the box as long as the file has the format

{...}
{...}

i.e. each line is a separate JSON object (which it looks liek you have)

[abhayneilam]

I want to have a separate events for each line of JSON file. I am getting all the lines merged with a single events. Please let me know how can I separate the events:

My Props.conf is :

[json]
KV_MODE = json
LINE_BREAKER = "}$"
NO_BINARY_CHECK = 1
TRUNCATE = 0
SHOULD_LINEMERGE = false

Please help