How to remove json key value pairs from events log data

[ayushram]

Splunk search events returns json format log data. I want to remove a particular key:value pair since the value of this key is huge (in terms of length) and unnecessary. How can I do so.

sample log data:

{
"abcd1": "asd",
"abcd2": [],
"abcd3": true,
"toBeRemoved": [{
"abcd8": 234,
"abcd9": [{
"abcd10": "asd234"
}],
"abcd11": "asdasd"
}],
"abcd4": 324.234,
"abcd5": "dfsad dfsdf",
"abcd6": 0,
"abcd7": "asfsdf"
}

The key:value pair to be removed has been marked in bold.

! NOTE THIS IS FORMATTED DATA, FIELDS CAN HAVE STRINGS, NUMBERS, BOTH, LISTS, ETC !

[ITWhisperer]

Try something like this - this assumes "toBeRemove" is not the first element i.e. is is preceded by a comma (which needs to be removed).

| rex mode=sed "s/(?ms),\s*\"toBeRemoved\":\s*\[([^\[\]]+|\[[^\]]*\])*\]//g"

[gcusello]

Hi @ayushram,

if you want to remove the highlighted data from the logs before indexing you have to add to your props.conf:

[your_sourcetype]
SEDCMD = s/(?ms)\"toBeRemoved\":.*\}\],//g

remember that this props.conf must be added on your Indexers or (if present) on your Heavy Forwarders.

Ciao.

Giuseppe

[ayushram]

I do not have access to pros.conf

Is there any way to do this from search itself?
I want my final data in " | table ", but it's not loading wherever this highlighted field appears (since it has too many characters)

[gcusello]

Hi @ayushram,

you can avoid to display a part of your logs in your searches, but accessing the raw log it's all visible:

| rex mode=sed "s/(?ms)\"toBeRemoved\":.*\}\],//g"

Ciao.

Giuseppe