topic Filter/modify data prior to ingestion? in Getting Data In

Filter/modify data prior to ingestion?

BogeyMan — Tue, 22 Apr 2025 18:52:29 GMT

Not sure this is even possible, but I'll ask anyway...

I have application(s) that are sending JSON data into Splunk, for example:

{ "key1": "value1", "key2": "value2", "key3": "value3", "key4": "value1" }

As you can see, the value for "key4" is the same as "key1".

So, in my example, I don't want to ingest the complete JSON payload, but only:

{ "key1": "value1", "key2": "value2", "key3": "value3" }

Can this be done?

Re: Filter/modify data prior to ingestion?

PickleRick — Tue, 22 Apr 2025 20:30:21 GMT

Manipulating structured data on ingest is best done by external means (particular method would depend on the use case). In some limited cases you probably could do some regex magic to remove some parts of incoming events but this solution would be highly sensitive to data format.

Oh, and I'm assuming you want to just remove key4, not dynamically decide which one to drop.