Filter/modify data prior to ingestion?

[BogeyMan]

Not sure this is even possible, but I'll ask anyway...

I have application(s) that are sending JSON data into Splunk, for example:

{
 "key1": "value1",
 "key2": "value2",
 "key3": "value3",
 "key4": "value1"
}

As you can see, the value for "key4" is the same as "key1".

So, in my example, I don't want to ingest the complete JSON payload, but only:

{
 "key1": "value1",
 "key2": "value2",
 "key3": "value3"
}

Can this be done?

[PickleRick]

Manipulating structured data on ingest is best done by external means (particular method would depend on the use case). In some limited cases you probably could do some regex magic to remove some parts of incoming events but this solution would be highly sensitive to data format.

Oh, and I'm assuming you want to just remove key4, not dynamically decide which one to drop.