Getting Data In
Provide Splunk Cloud feedback in this confidential UX survey by June 17
for a chance to win a $200 Amazon gift card!

Extracting fields from JSON file format

abhayneilam
Contributor

Hi,
I want to extract the fields from JSON File format and after extrating the fileds. I want to remove one of the value of a particular key, such as :

My input data :

{"TID":"111222","EID":"44567","XXX":"00000x<?xml?>aasewl85765"}

Now, I want to remove the "XXX":"00000x<?xml?>aasewl85765" .. This should not be indexed other parts should be indexed like {"TID":"111222","EID":"44567" }

In my Splunk only two fields should come "TID" and "EID".

Please suggest the best possible way !!

Tags (2)
0 Karma
1 Solution

dshpritz
SplunkTrust
SplunkTrust

Your order is messed up. First you want to filter out the data (prior to indexing) and then (on search) you extract the fields.

To filter, in your props.conf:

[mysourcetype]
SEDCMD-removefields = s/{"TID":"\d+","EID":"\d+"(.*)/}/g

To extract the fields, you might be able to use Splunk's automagical field extraction. In the same props.conf:

[mysourcetype]
KV_MODE = json

So, all together:

[mysourcetype]
KV_MODE = json
SEDCMD-removefields = s/{"TID":"\d+","EID":"\d+"(.*)/}/g

This will need to be done on your search heads and indexers.

View solution in original post

dshpritz
SplunkTrust
SplunkTrust

Your order is messed up. First you want to filter out the data (prior to indexing) and then (on search) you extract the fields.

To filter, in your props.conf:

[mysourcetype]
SEDCMD-removefields = s/{"TID":"\d+","EID":"\d+"(.*)/}/g

To extract the fields, you might be able to use Splunk's automagical field extraction. In the same props.conf:

[mysourcetype]
KV_MODE = json

So, all together:

[mysourcetype]
KV_MODE = json
SEDCMD-removefields = s/{"TID":"\d+","EID":"\d+"(.*)/}/g

This will need to be done on your search heads and indexers.

View solution in original post

abhayneilam
Contributor

"Note that that is the start of the line, not the end of it." This clue helped me to get a solution for my problem.

Thanks a lot !!

0 Karma

dshpritz
SplunkTrust
SplunkTrust

Your linebreaker doesn't have a capture, which may be the problem. I would set the linebreaker to:

([\r\n]){

Note that that is the start of the line, not the end of it.

0 Karma

abhayneilam
Contributor

I want to remove one of the field and its value from the file to be indexed. Please help me

Regards,
Abhay

0 Karma

laserval
Communicator

Have you tried without setting a LINE_BREAKER? When I've indexed JSON, it has worked well out of the box as long as the file has the format

{...}
{...}

i.e. each line is a separate JSON object (which it looks liek you have)

0 Karma

abhayneilam
Contributor

I want to have a separate events for each line of JSON file. I am getting all the lines merged with a single events. Please let me know how can I separate the events:

My Props.conf is :

[json]
KV_MODE = json
LINE_BREAKER = "}$"
NO_BINARY_CHECK = 1
TRUNCATE = 0
SHOULD_LINEMERGE = false

Please help

0 Karma
Take the 2021 Splunk Career Survey

Help us learn about how Splunk has
impacted your career by taking the 2021 Splunk Career Survey.

Earn $50 in Amazon cash!